Back

Crawler Documentation

StoreRouterBot

Last updated: October 21, 2026

StoreRouterBot is the public crawler identity used by Octogen Systems, Inc. Our customers are brands, retailers and agentic-shopping applications. StoreRouter's crawler reads ecommerce products. The platform then restructures those products into a common enriched schema to make it easier for LLMs to understand the crawled catalog.

User Agent

Requests signed with StoreRouter's Web Bot Auth identity use this user-agent string:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; StoreRouterBot/1.0; +https://storerouter.ai/bots

Unsigned crawler traffic may use ordinary browser user agents. The StoreRouterBot identity is reserved for configured Web Bot Auth traffic.

Bot Details

Bot name
StoreRouterBot
Operator
Octogen Systems, Inc.
Purpose
Product discovery, catalog interpretation, and AI shopping readiness analysis
Verification
Cloudflare Web Bot Auth using HTTP Message Signatures
Key directory
https://bots.storerouter.ai/.well-known/http-message-signatures-directory
Contact
crawler@storerouter.ai

Verification

StoreRouter uses Cloudflare Web Bot Auth for eligible crawls. Signed requests include HTTP Message Signature headers and a Signature-Agent header that points to StoreRouter's public key directory.

https://bots.storerouter.ai/.well-known/http-message-signatures-directory

Site operators should prefer Web Bot Auth verification over static IP allowlists, since crawl egress can change as infrastructure changes.

Compatibility

StoreRouterBot replaces the former OctogenBot name. Older deployments and existing site rules may still refer to OctogenBot/1.0. During the transition, site operators should include both robots tokens and update any allowlist that matches the complete user-agent string. The operator remains Octogen Systems, Inc.

The legacy directory remains directly available with the same public key:

https://bots.octogen.ai/.well-known/http-message-signatures-directory

Key IDs are JWK thumbprints and do not change with the hostname. Verify signatures against the directory named in each request; do not substitute a different Signature-Agent value. Existing old documentation links remain available through the website redirect. The crawler contact is crawler@storerouter.ai.

Crawler Purpose

StoreRouterBot reads publicly available ecommerce pages so StoreRouter can understand how AI shopping agents interpret product catalogs, identify missing product data, and help merchants make their catalogs easier for agents to understand.

  • Public product page URLs and page metadata
  • Product names, descriptions, images, prices, availability, variants, and attributes
  • Public structured data such as Schema.org Product markup
  • Public sitemap and robots.txt signals used to scope crawl behavior

Boundaries

  • StoreRouterBot does not try to access authenticated, paywalled, or private content.
  • StoreRouterBot does not collect personal shopper account data from retailer sites.
  • StoreRouterBot does not use signed bot identity unless the crawl is configured for Web Bot Auth.
  • StoreRouterBot is intended for commercial product-data workflows, not search-engine indexing.

Robots.txt

To block StoreRouterBot and the legacy OctogenBot identity, add this group to your robots.txt file:

User-agent: StoreRouterBot User-agent: OctogenBot Disallow: /

To limit crawling to public product pages, use path-specific rules:

User-agent: StoreRouterBot User-agent: OctogenBot Disallow: /account/ Disallow: /checkout/ Crawl-delay: 10

Contact

For crawl questions, allowlist requests, robots.txt issues, or urgent operational concerns, email crawler@storerouter.ai.