FlyweightBot

FlyweightBot is the web crawler operated by Flyweight GmbH. It reads the public pages of online stores whose owners have installed our AI shopping assistant, so the assistant can answer their shoppers’ questions from the store’s own products, policies and help content.

If you found this page from an entry in your access logs, everything you need to identify, allow or stop the crawler is below.

Why it is visiting your site

We crawl a domain only after someone installs Flyweight on that store and connects the domain as a content source. We do not crawl the open web, and we do not follow links off the connected domain.

The crawl runs when a store is first connected and again when its content is refreshed — it is not a continuous background crawl. If you administer a store and did not expect us, someone on your team most likely installed the app from the Shopify App Store.

How to identify it

Every request carries this token at the end of the User-Agent header:

FlyweightBot/1.0 (+https://flyweight.io/bot)

The crawler drives a real Chrome browser so that it renders pages the way a shopper’s browser does, which means the full header also carries Chrome’s own identification. A complete example:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/143.0.0.0 Safari/537.36 FlyweightBot/1.0 (+https://flyweight.io/bot)

The Chrome portion changes as the browser is updated, so match on the substring FlyweightBot rather than on the full header or a pinned version number.

Where the requests come from

Crawl traffic currently leaves from a single static IP address:

52.212.232.38

That address is ours alone and does not change, so it is the most precise thing to write a rule against today. It is the narrower of the two signals, though, not the more durable one: the address describes where we happen to crawl from, while the token describes who we are. If we ever add a second crawl route, the token comes with it and the address does not — so where a tool lets you write only one rule and you want it to keep working, write it against the token.

How to allow it

If your store is behind Cloudflare, add a WAF custom rule with the Skip action. Matching on the IP address is the tighter rule:

(ip.src eq 52.212.232.38)

To match the token as well, pair it with the address rather than using it on its own:

(http.user_agent contains "FlyweightBot" and ip.src eq 52.212.232.38)

Anyone can put FlyweightBot in a User-Agent header, so a rule that matches the token alone will also match anyone who copies it off this page. Use the user-agent-only form below where a tool offers nothing else, and prefer a narrow action over a blanket Skip:

(http.user_agent contains "FlyweightBot")

On other WAFs and CDNs, the equivalent is an allow or bypass rule on the same address or substring. In nginx, a user-agent block has to read the classification back for the exception to do anything:

# http block: classify the agent, then let FlyweightBot back out
map $http_user_agent $blocked_bot {
    default             0;
    "~*(bot|crawler)"   1;
    "~*FlyweightBot"    0;
}

# server block: the map above only takes effect once something reads it
server {
    if ($blocked_bot) {
        return 403;
    }
}

Shopify apps that block traffic — fraud filters, country blockers, bot blockers — generally offer an allowlist field. Most of them take an IP address; add 52.212.232.38 there. Where the field takes a user agent instead, use FlyweightBot.

If the assistant on your store is answering with missing or outdated content, a block of this kind is the most common cause.

How to stop it

The crawler does not currently read robots.txt, so a rule there will not stop it. To end the crawl, remove the domain as a content source in the Flyweight admin, or uninstall the app — either one stops any further crawl being scheduled. A crawl already under way is not cancelled and will finish the batch of pages it started with, usually within a few minutes.

You can also block it outright at your WAF or CDN using the same expressions above with a block action. We would rather you write to us first, though, because a block means the store’s own assistant stops being able to answer questions about its products.

Contact

Questions about the crawler, or a request to stop it crawling a domain you administer: hello@flyweight.io. We answer these directly.

Flyweight GmbH, Jungbuschstraße 28, 68159 Mannheim, Germany — legal notice.