# robots.txt automatically generated by PrestaShop e-commerce open-source solution # http://www.prestashop.com - http://www.prestashop.com/forums # This file is to prevent the crawling and indexing of certain parts # of your site by web crawlers and spiders run by sites like Yahoo! # and Google. By telling these "robots" where not to go on your site, # you save bandwidth and server resources. # For more information about the robots.txt standard, see: # http://www.robotstxt.org/robotstxt.html # # =========================================================================== # NOTA: las secciones marcadas con >>> ANADIDO MANUAL <<< no son de # PrestaShop. Mantenlas cuando PrestaShop regenere este fichero. # RECORDATORIO: robots.txt es ADVISORIO. Solo lo respetan los crawlers # buenos (Google, Bing, Ahrefs...). Los bots maliciosos lo ignoran: # a esos los para Cloudflare, no este fichero. # =========================================================================== # >>> ANADIDO MANUAL: crawlers agresivos que SI respetan robots.txt <<< # Se les prohibe TODO el sitio. (Si usas Ahrefs/Semrush para analizar TU # propia web, borra esas dos lineas o no podran rastrearte para tus informes.) User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: MegaIndex Disallow: / User-agent: ZoominfoBot Disallow: / # Bytespider y PetalBot a menudo IGNORAN robots.txt (refuerzo en Cloudflare), # pero se lo pedimos igualmente para la fraccion que lo respeta. User-agent: Bytespider Disallow: / User-agent: PetalBot Disallow: / # =========================================================================== # Reglas para el resto de crawlers (incluye Googlebot y Bingbot) # =========================================================================== User-agent: * # >>> ANADIDO MANUAL: ralentiza a los crawlers que respetan Crawl-delay # (Bing/Yandex lo respetan; Google lo ignora y no pasa nada). <<< Crawl-delay: 10 # Allow Directives Allow: */modules/*.css Allow: */modules/*.js Allow: */modules/*.png Allow: */modules/*.jpg Allow: /js/jquery/* # Private pages (parametros) -- ORIGINAL de PrestaShop Disallow: /*?order= Disallow: /*?tag= Disallow: /*?id_currency= Disallow: /*?search_query= Disallow: /*?back= Disallow: /*?n= Disallow: /*&order= Disallow: /*&tag= Disallow: /*&id_currency= Disallow: /*&search_query= Disallow: /*&back= Disallow: /*&n= # >>> ANADIDO MANUAL: parametros de la trampa de rastreo que FALTABAN <<< # Son los que generan la explosion de URLs vista en el access.log: # /pt/promocoes?order=...&productListView=grid&resultsPerPage=... # Funcionan para TODOS los idiomas (el patron /* cubre /pt/, /es/, etc.) Disallow: /*?productListView= Disallow: /*&productListView= Disallow: /*?resultsPerPage= Disallow: /*&resultsPerPage= Disallow: /*?page= Disallow: /*&page= Disallow: /*?q= Disallow: /*&q= Disallow: /*?selected_filters= Disallow: /*&selected_filters= Disallow: /*?id_attribute_group= Disallow: /*&id_attribute_group= Disallow: /*?id_feature= Disallow: /*&id_feature= Disallow: /*?layout= Disallow: /*&layout= Disallow: /*?from-xml= Disallow: /*&from-xml= # (opcion agresiva: descomenta para prohibir CUALQUIER URL con parametros. # Solo si tus URLs canonicas son limpias -en PrestaShop lo son-.) # Disallow: /*? # Private pages (controladores) -- ORIGINAL. Cubre carrito/cuenta/etc. # en CUALQUIER idioma cuando se accede por ?controller= Disallow: /*controller=addresses Disallow: /*controller=address Disallow: /*controller=authentication Disallow: /*controller=cart Disallow: /*controller=discount Disallow: /*controller=footer Disallow: /*controller=get-file Disallow: /*controller=header Disallow: /*controller=history Disallow: /*controller=identity Disallow: /*controller=images.inc Disallow: /*controller=init Disallow: /*controller=my-account Disallow: /*controller=order Disallow: /*controller=order-slip Disallow: /*controller=order-detail Disallow: /*controller=order-follow Disallow: /*controller=order-return Disallow: /*controller=order-confirmation Disallow: /*controller=pagination Disallow: /*controller=password Disallow: /*controller=pdf-invoice Disallow: /*controller=pdf-order-return Disallow: /*controller=pdf-order-slip Disallow: /*controller=product-sort Disallow: /*controller=search Disallow: /*controller=statistics Disallow: /*controller=attachment Disallow: /*controller=guest-tracking # Directories -- ORIGINAL (raiz) Disallow: /app/ Disallow: /cache/ Disallow: /classes/ Disallow: /config/ Disallow: /controllers/ Disallow: /download/ Disallow: /js/ Disallow: /localization/ Disallow: /log/ Disallow: /mails/ Disallow: /modules/ Disallow: /override/ Disallow: /pdf/ Disallow: /src/ Disallow: /tools/ Disallow: /translations/ Disallow: /upload/ Disallow: /var/ Disallow: /vendor/ Disallow: /webservice/ # Directories -- ORIGINAL (/es/) Disallow: /es/app/ Disallow: /es/cache/ Disallow: /es/classes/ Disallow: /es/config/ Disallow: /es/controllers/ Disallow: /es/download/ Disallow: /es/js/ Disallow: /es/localization/ Disallow: /es/log/ Disallow: /es/mails/ Disallow: /es/modules/ Disallow: /es/override/ Disallow: /es/pdf/ Disallow: /es/src/ Disallow: /es/tools/ Disallow: /es/translations/ Disallow: /es/upload/ Disallow: /es/var/ Disallow: /es/vendor/ Disallow: /es/webservice/ # >>> ANADIDO MANUAL: mismos directorios para /pt/ (idioma legitimo que # faltaba por completo en el original) <<< Disallow: /pt/app/ Disallow: /pt/cache/ Disallow: /pt/classes/ Disallow: /pt/config/ Disallow: /pt/controllers/ Disallow: /pt/download/ Disallow: /pt/js/ Disallow: /pt/localization/ Disallow: /pt/log/ Disallow: /pt/mails/ Disallow: /pt/modules/ Disallow: /pt/override/ Disallow: /pt/pdf/ Disallow: /pt/src/ Disallow: /pt/tools/ Disallow: /pt/translations/ Disallow: /pt/upload/ Disallow: /pt/var/ Disallow: /pt/vendor/ Disallow: /pt/webservice/ # Files -- ORIGINAL (/es/ paginas privadas por slug) Disallow: /*es/recuperar Disallow: /*es/direccion Disallow: /*es/direcciones Disallow: /*es/acceso Disallow: /*es/carrito Disallow: /*es/descuento Disallow: /*es/historial Disallow: /*es/datos Disallow: /*es/cuenta Disallow: /*es/seguimiento Disallow: /*es/albaran Disallow: /*es/pedido Disallow: /*es/buscar Disallow: /*es/seguimiento-invitado Disallow: /*es/confirmacion # >>> ANADIDO MANUAL: paginas privadas en PT. # IMPORTANTE: estos "slugs" dependen de como tengas traducidas las URLs en # PrestaShop (Parametros de la tienda > Trafico y SEO). Ajusta cada uno al # texto REAL de tu tienda en portugues. Las lineas controller= de arriba ya # cubren estas paginas por funcion, asi que esto es refuerzo, no critico. Disallow: /*pt/recuperar Disallow: /*pt/morada Disallow: /*pt/moradas Disallow: /*pt/acesso Disallow: /*pt/carrinho Disallow: /*pt/desconto Disallow: /*pt/historico Disallow: /*pt/dados Disallow: /*pt/conta Disallow: /*pt/seguimento Disallow: /*pt/talao Disallow: /*pt/encomenda Disallow: /*pt/pesquisar Disallow: /*pt/seguimento-convidado Disallow: /*pt/confirmacao # >>> ANADIDO MANUAL: sitemap. VERIFICA la ruta real (el modulo de sitemap # de PrestaShop a veces genera /1_index_sitemap.xml u otra). Ajustala. <<< Sitemap: https://www.ruronails.com/sitemap.xml