Sitemap Crawler

Sitemap Crawler crawls links from a website base URL, remembers and traverses links to subpages of that base url and generates a sitemap of the found URLs.

A sitemap (a list of URLs) can be checked for validity; that they exist and do not return an error.

Source code repository at GitLab sitemap-crawler