# Scrape Url

> `POST /scrape` · Read

Canonical page: https://clawifi.dev/docs/api/post-scrape

`POST https://api.clawifi.dev/scrape` · 1 credit(s) per call

- Send your key as `Authorization: Bearer $CLAWIFI_API_KEY`; see [Authentication](https://clawifi.dev/docs/authentication).
- Add `"dry_run": true` (or `?dry_run=true`) to see what the call would cost; nothing runs and nothing is charged.
- Hosted MCP tool: `scrape_url` on `https://api.clawifi.dev/mcp`.
- Credits: 1 per call (search, plain fetch/scrape, article extract, package/dataset/code/archive/academic lookup, single OSINT lookup, social reads). Every price is on [Pricing and credits](https://clawifi.dev/docs/pricing-and-credits).

## Request

**cURL**

```sh
curl -X POST https://api.clawifi.dev/scrape \
  -H "Authorization: Bearer $CLAWIFI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com"}'
```

**Python**

```python
from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.scrape("https://example.com")
```

**MCP**

```text
scrape_url(url="https://example.com")
```

### Body

| Field | Type | Default | Required |
| --- | --- | --- | --- |
| `artifact_policy` | string | `"metadata_only"` | no |
| `browser_profile` | string (optional) |  | no |
| `exclude_tags` | array of string |  | no |
| `formats` | array of string (one of 10 values; see [Sources](https://clawifi.dev/sources)) or ScrapeFormatOption |  | no |
| `mode` | "auto" \| "http" \| "browser" | `"auto"` | no |
| `proxy_config` | string or object (optional) |  | no |
| `query` | string (optional) |  | no |
| `requires_auth` | boolean | `false` | no |
| `skip_archive` | boolean | `false` | no |
| `source` | SourceId (optional) |  | no |
| `target_selector` | string (optional) |  | no |
| `timeout_seconds` | integer (optional) |  | no |
| `top_k` | integer | `5` | no |
| `url` | string |  | yes |
| `wait_for_selector` | string (optional) |  | no |
| `wait_until` | "commit" \| "domcontentloaded" \| "load" \| "networkidle" | `"domcontentloaded"` | no |

## Response

200 with a `ScrapeResponse` object: `artifacts`, `cache_hit`, `date_source`, `diagnostics`, `errors`, `fetched_at`, `final_url`, `format_results`, `formats`, `metadata`, `mode_used`, `modified_at`, `published_at`, `query_selection`, ….

Errors use the shared shape; see [Errors](https://clawifi.dev/docs/errors). Try public operations without a key in the [playground](https://clawifi.dev/playground).