# ============================================================================== # File Location: /robots.txt # Environment: Production # Description: Standard exclusion protocol directives for web crawlers. # ============================================================================== # ------------------------------------------------------------------------------ # Global Directives # Rules applied to all compliant search engine crawlers and bots. # ------------------------------------------------------------------------------ User-agent: * # Explicitly allow crawling of all root content (Default behavior) Allow: / # Prevent crawling of experimental or development directories Disallow: /lab/ # ------------------------------------------------------------------------------ # Sitemap Declaration # Directs search engines to the primary XML sitemap for optimal indexing. # ------------------------------------------------------------------------------ Sitemap: https://monograph.org/sitemap.xml