Full crawler
Paste a URL. We detect whether the site publishes a sitemap or exposes the WordPress
REST API, then use whichever reaches the most pages — and hand it back as Markdown,
llms.txt, or JSON.
Walks internal links breadth-first from the start page. Works on any site, but only reaches pages something links to.
Reads robots.txt and every common sitemap location, follows sitemap indexes and gzipped sitemaps. Finds orphan pages nothing links to.
Enumerates published content directly. The most complete option on WordPress — and several times faster, since there is no HTML to parse.