# Crawl Site

> `POST /crawl` · Map and crawl

Canonical page: https://clawifi.dev/docs/api/post-crawl

`POST https://api.clawifi.dev/crawl` · 1 credit(s) per page

- Send your key as `Authorization: Bearer $CLAWIFI_API_KEY`; see [Authentication](https://clawifi.dev/docs/authentication).
- Add `"dry_run": true` (or `?dry_run=true`) to see what the call would cost; nothing runs and nothing is charged.
- Credits: 1 per page (+2 per page that needed JS rendering; at least 1). Every price is on [Pricing and credits](https://clawifi.dev/docs/pricing-and-credits).

## Request

**cURL**

```sh
curl -X POST https://api.clawifi.dev/crawl \
  -H "Authorization: Bearer $CLAWIFI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"seed_url":"https://docs.python.org/3/tutorial/"}'
```

**Python**

```python
from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.crawl("https://docs.python.org/3/tutorial/")
```

### Body

| Field | Type | Default | Required |
| --- | --- | --- | --- |
| `allowed_domains` | array of string |  | no |
| `browser_profile` | string (optional) |  | no |
| `concurrency` | integer | `3` | no |
| `exclude_patterns` | array of string |  | no |
| `force_browser` | boolean | `false` | no |
| `include_patterns` | array of string |  | no |
| `max_depth` | integer | `1` | no |
| `max_links_per_page` | integer | `100` | no |
| `max_pages` | integer | `10` | no |
| `robots_policy` | "respect" \| "ignore" | `"respect"` | no |
| `same_domain_only` | boolean | `true` | no |
| `scope` | "same-origin" \| "same-domain" \| "subdomains" \| "any" (optional) |  | no |
| `seed_url` | string |  | yes |
| `timeout_seconds` | integer (optional) |  | no |
| `wait_until` | "commit" \| "domcontentloaded" \| "load" \| "networkidle" | `"domcontentloaded"` | no |

## Response

200 with a `CrawlResponse` object: `discovered_count`, `errors`, `max_depth_reached`, `pages`, `seed_url`, `success`, `visited_count`.

Errors use the shared shape; see [Errors](https://clawifi.dev/docs/errors). Try public operations without a key in the [playground](https://clawifi.dev/playground).