> For the complete documentation index, see [llms.txt](https://adagio-io.gitbook.io/adagio-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://adagio-io.gitbook.io/adagio-documentation/operations-and-support/adagio-bot-crawler.md).

# Adagio bot - Crawler

If you have issue with Supply Chain Validation

## Requirements

{% hint style="warning" %}
Adagio needs to be able to crawl your `(app-)ads.txt` and `sellers.json` to validate:

* whether Adagio is allowed to sell the inventory, and which of its demand partners also are allowed
* whether you have access to this inventory

Therefore, **ensuring that Adagio's bot has access conditions the monetisation performance of your account.** 
{% endhint %}

Adagio platform complies with the [Supply Chain Validation](https://iabtechlab.com/software/supply-chain-validation/) from IAB.

In some situations, your web server might not allow bots to connect and get the content from these files, you can whitelist our bot by:

* **Allowing requests with** a `User-Agent` header containing `Adagiobot` (our bot's complete `User-Agent` is: `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Adagiobot/1.0) Chrome/112.0.5615.137 Safari/537.36`
* **Allowing the CIDR range** (IP adresses) listed in [this file](https://assets.adagio.io/adagiobot-ipv4.txt)

{% hint style="info" %}
The `(app-)ads.txt` and `sellers.json` file size is limited to 15MiB. If the crawled file exceeds this limit, it will be considered invalid.
{% endhint %}

Here's a documentation section you can drop straight in, written for non-technical readers:

***

## 🔍 Test if Adagio's bot can access your file

If Adagio reports that your `ads.txt` or `app-ads.txt` cannot be crawled, you can test it yourself in less than a minute using your computer's Terminal (Mac) or Command Prompt (Windows).

**Step 1 — Open your terminal** On Mac: press `Cmd + Space`, type `Terminal`, press Enter.

**Step 2 — Copy and paste this command**, replacing `yoursite.com` with your domain:

{% hint style="info" %}
Replace `yoursite.com` with the domain you're trying to test
{% endhint %}

```bash
curl -v \
  -H 'User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Adagiobot/1.0) Chrome/112.0.5615.137 Safari/537.36' \
  https://yoursite.com/ads.txt
```

**Step 3 — Read the result**

| What you see                       | What it means                                              | What to do                                                                                                        |
| ---------------------------------- | ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| The content of your `ads.txt` file | ✅ Our UserAgent isn't blocked, other issues may be at play | Contact our team to get a more elaborate investigation (e.g. checking that our bot's IP might not be whitelisted) |
| `403 Forbidden`                    | 🚫 Your server is explicitly blocking bots                 | Whitelist Adagio's User-Agent or IP range (see options below)                                                     |
| `Empty reply from server`          | 🚫 Your server is silently rejecting the connection        | Same — whitelist Adagio's User-Agent or IP range                                                                  |
| `HTTP/2 stream ... PROTOCOL_ERROR` | ⚠️ Your server has a configuration issue with its protocol | Contact your hosting provider or IT team — this is a server-side bug                                              |
| `Could not resolve host`           | ⚠️ The domain or file path is wrong                        | Check that `ads.txt` is published at the root of your domain                                                      |


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://adagio-io.gitbook.io/adagio-documentation/operations-and-support/adagio-bot-crawler.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.