Monitor pages and searches for changes

A monitor checks a public page, or runs a search, on a schedule and keeps every check and every change. It can email you and call your webhooks when something changes.

Two kinds#

Page (kind: "page", the default): Checks url on a schedule from every 5 minutes to once a month (schedule.interval_seconds, 300 to 2,592,000) and compares it with the last check (diff_modes: text, hash, structure, json, visual). include_selectors limits the comparison to the parts of the page you name, such as ["main"]; exclude_selectors drops parts such as a cookie banner.
Search (kind: "search"): Runs query (with optional search.include_domains, exclude_domains, time_range and max_results up to 20) at most once an hour and reports only results it has never seen before. The first check is the baseline. The change (diff_mode: "search") counts them in summary.new_count and lists the first 5 in summary.new_results; the stored diff has them all.

Get told when it changes#

notify_email: true mails the monitor's creator when a check finds a change: what changed, and a link to the monitor. At most 24 mails a day per monitor.
A webhook endpoint (POST /webhooks/endpoints with url, secret and event_types such as monitor.change.detected and monitor.tick.failed) gets a signed POST for each event. A monitor that lists webhook_endpoint_ids sends only to those endpoints; with none listed, every endpoint that subscribes to the event gets it.

Each POST carries X-Clawifi-Event-Id (use it to drop repeats), X-Clawifi-Timestamp (Unix seconds) and X-Clawifi-Signature: an HMAC-SHA256 of the timestamp, a dot and the raw body, keyed with your secret. Refuse a timestamp more than five minutes old, so a captured request cannot be replayed.

Verify a webhook (Python)

import hashlib, hmac, time

def verify(raw_body: bytes, headers, secret: str, max_age: int = 300) -> bool:
    """headers: any mapping of the request's headers; names are matched without regard to case."""
    found = {name.lower(): value for name, value in headers.items()}
    timestamp = found.get("x-clawifi-timestamp", "")
    if not timestamp.isdigit() or abs(time.time() - int(timestamp)) > max_age:
        return False
    signed = timestamp.encode() + b"." + raw_body
    expected = "sha256=" + hmac.new(secret.encode(), signed, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, found.get("x-clawifi-signature", ""))

What it costs#

A check costs 1 credit, 3 when it needs the browser; a search monitor's check is 1. A check with nothing to read, or where every search engine failed, costs 0. POST /monitors/estimate takes the same body as POST /monitors and returns the cost per check, per day and per 30 days, for free. Free accounts have 3 monitors, Starter 10, Pro 25, Scale 100. Ready-made recipes are on Templates; manage monitors in the console.

GET /monitors#

List Monitors. Full parameters on the endpoint page.

curl https://api.clawifi.dev/monitors \
  -H "Authorization: Bearer $CLAWIFI_API_KEY"

from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.list_monitors()

list_monitors()

POST /monitors#

Create Monitor. Full parameters on the endpoint page.

curl -X POST https://api.clawifi.dev/monitors \
  -H "Authorization: Bearer $CLAWIFI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"Competitor pricing","url":"https://example.com/pricing","schedule":{"interval_seconds":86400}}'

from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.post("/monitors", json={"name":"Competitor pricing","url":"https://example.com/pricing","schedule":{"interval_seconds":86400}})

GET /monitors/{monitor_id}#

Get Monitor. Full parameters on the endpoint page.

curl https://api.clawifi.dev/monitors/<monitor_id> \
  -H "Authorization: Bearer $CLAWIFI_API_KEY"

from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.get("/monitors/<monitor_id>")

POST /monitors/{monitor_id}/run#

Run Monitor Now. Full parameters on the endpoint page.

curl -X POST https://api.clawifi.dev/monitors/<monitor_id>/run \
  -H "Authorization: Bearer $CLAWIFI_API_KEY"

from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.post("/monitors/<monitor_id>/run")

GET /monitors/{monitor_id}/diffs#

List Monitor Diffs. Full parameters on the endpoint page.

curl https://api.clawifi.dev/monitors/<monitor_id>/diffs \
  -H "Authorization: Bearer $CLAWIFI_API_KEY"

from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.get("/monitors/<monitor_id>/diffs")

POST /monitors/estimate#

Estimate Monitor. Full parameters on the endpoint page.

curl -X POST https://api.clawifi.dev/monitors/estimate \
  -H "Authorization: Bearer $CLAWIFI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"Competitor pricing","url":"https://example.com/pricing","schedule":{"interval_seconds":86400}}'

from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.post("/monitors/estimate", json={"name":"Competitor pricing","url":"https://example.com/pricing","schedule":{"interval_seconds":86400}})

POST /webhooks/endpoints#

Create Webhook Endpoint. Full parameters on the endpoint page.

curl -X POST https://api.clawifi.dev/webhooks/endpoints \
  -H "Authorization: Bearer $CLAWIFI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://your-app.example/hooks/clawifi","secret":"a-long-random-secret","event_types":["monitor.change.detected"]}'

from clawifi import Clawifi

client = Clawifi()  # reads CLAWIFI_API_KEY
result = client.post("/webhooks/endpoints", json={"url":"https://your-app.example/hooks/clawifi","secret":"a-long-random-secret","event_types":["monitor.change.detected"]})