# Map and crawl

> Map a site from its robots.txt, sitemaps, feeds and links, then crawl it: one bounded call, or a crawl you start, poll, stop and resume.

Canonical page: https://clawifi.dev/docs/map-and-crawl

## POST /map

Map Site. Full parameters on the [endpoint page](https://clawifi.dev/docs/api/post-map).

**cURL**

```sh
curl -X POST https://api.clawifi.dev/map \
  -H "Authorization: Bearer $CLAWIFI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com"}'
```

**Python**

```python
from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.map("https://example.com")
```

**MCP**

```text
map_site(url="https://example.com")
```

## POST /crawl

Crawl Site. Full parameters on the [endpoint page](https://clawifi.dev/docs/api/post-crawl).

**cURL**

```sh
curl -X POST https://api.clawifi.dev/crawl \
  -H "Authorization: Bearer $CLAWIFI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"seed_url":"https://docs.python.org/3/tutorial/"}'
```

**Python**

```python
from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.crawl("https://docs.python.org/3/tutorial/")
```

## POST /crawls

Start Crawl. Full parameters on the [endpoint page](https://clawifi.dev/docs/api/post-crawls).

**cURL**

```sh
curl -X POST https://api.clawifi.dev/crawls \
  -H "Authorization: Bearer $CLAWIFI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"seed_url":"https://docs.python.org/3/tutorial/"}'
```

**Python**

```python
from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.post("/crawls", json={"seed_url":"https://docs.python.org/3/tutorial/"})
```

**MCP**

```text
start_crawl(seed_url="https://docs.python.org/3/tutorial/")
```

## GET /crawls/{crawl_id}

Get Crawl. Full parameters on the [endpoint page](https://clawifi.dev/docs/api/get-crawls-crawl-id).

**cURL**

```sh
curl https://api.clawifi.dev/crawls/<crawl_id> \
  -H "Authorization: Bearer $CLAWIFI_API_KEY"
```

**Python**

```python
from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.get_crawl("<crawl_id>")
```

**MCP**

```text
get_crawl()
```

## POST /crawls/{crawl_id}/stop

Stop Crawl. Full parameters on the [endpoint page](https://clawifi.dev/docs/api/post-crawls-crawl-id-stop).

**cURL**

```sh
curl -X POST https://api.clawifi.dev/crawls/<crawl_id>/stop \
  -H "Authorization: Bearer $CLAWIFI_API_KEY"
```

**Python**

```python
from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.post("/crawls/<crawl_id>/stop")
```

## POST /crawls/{crawl_id}/resume

Resume Crawl. Full parameters on the [endpoint page](https://clawifi.dev/docs/api/post-crawls-crawl-id-resume).

**cURL**

```sh
curl -X POST https://api.clawifi.dev/crawls/<crawl_id>/resume \
  -H "Authorization: Bearer $CLAWIFI_API_KEY"
```

**Python**

```python
from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.post("/crawls/<crawl_id>/resume")
```

## DELETE /crawls/{crawl_id}

Delete Crawl. Full parameters on the [endpoint page](https://clawifi.dev/docs/api/delete-crawls-crawl-id).

**cURL**

```sh
curl -X DELETE https://api.clawifi.dev/crawls/<crawl_id> \
  -H "Authorization: Bearer $CLAWIFI_API_KEY"
```

**Python**

```python
from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.request("DELETE", "/crawls/<crawl_id>")
```