[ free tool ]
Sitemap to JSON
Enter a domain or a sitemap address and get its URLs and lastmod dates as JSON you can download, ready for a crawl list, a diff or an agent’s to-do list.
What it does#
- Reads the sitemap through Clawifi and returns
{ loc, lastmod }for each URL, without duplicates. - Follows a sitemap index into its first 3 child sitemaps (each one is a run) and notes which ones it read.
- Shows the first 25 rows and the full JSON, which you can copy or download.
Limits#
- The free preview returns the sitemap as text without its XML tags, and the tool rebuilds
locandlastmodfrom that text.changefreq,priority, image, video and hreflang entries are not kept, and a URL that itself ends in a date, with no lastmod, can be split wrongly. - Each sitemap file is cut at 20,000 characters, the list stops at 2,000 URLs, and compressed
.gzsitemaps are skipped. - Up to 4 runs per conversion (the index and 3 children), of the 12 a minute per address the free tools share.
Do this from your agent#
For an agent, map_site lists a site’s URLs from its robots.txt, sitemaps and links in one call (URLs, not lastmod). It works on the hosted MCP server without a key as a trial, capped at 10 URLs; with a key, up to 1,000.
map_site(url="https://example.com", max_urls=100)curl https://api.clawifi.dev/map \
-H "Authorization: Bearer $CLAWIFI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","sources":["sitemap"],"max_urls":500}'from clawifi import Clawifi
client = Clawifi() # reads CLAWIFI_API_KEY
site = client.map("https://example.com", sources=["sitemap"], max_urls=500)
print([u["url"] for u in site["urls"]])Read more: Map and crawl · POST /map · Remote MCP · Docs · Pricing
Questions#
How do I convert a sitemap.xml to JSON?
Enter the domain or the sitemap’s address above and press Convert. The tool lists each URL with its lastmod date and lets you download the list as a .json file.
Where do I find a site’s sitemap?
Most sites list it in robots.txt as a Sitemap: line, and many serve one at /sitemap.xml. The robots.txt and llms.txt checker shows those lines for any domain.
What is a sitemap index?
A sitemap that lists other sitemaps instead of pages, used by large sites. This tool follows the first 3 child sitemaps of an index.
What does lastmod mean?
The date the site says a URL last changed meaningfully, in W3C date format. Search engines and agents can use it to decide what to re-read; it is only as accurate as the site keeps it.
How many URLs can it convert?
Up to 2,000 per conversion, and each sitemap file is cut at 20,000 characters in the free preview. For full sites, the map API returns up to 1,000 URLs a call with a key.