owor

We're currently working on scaling up to 25,000+ pages, hopefully sometime in the next 12 months!

Noted, we'll look into returning more descriptive messaging for common errors.

Hey Tepix - head to https://rarchy.com/sitemaps/visual-sitemap-generator and enter the website you wish to crawl in the left-hand side input.

This will crawl the site & generate a visual sitemap without requiring you to signup for an account.

Thanks for your feedback dmje.

We've had similar discussions with customers in the past - we've found the vast majority (90%) are happy to pay monthly, and the remainder are very vocally against.

We are looking to accomodate both sides, and the current plan is to offer a subscription 'pause', where all of your existing data is stored for free, and you can unpause to edit/create new projects.

It's still being worked out, but hopefully this will be a happy medium ground in the future.

We use an auto-throttling algorithm for the crawler to ensure we are a 'polite' crawler!

We also send a unique user-agent with every request, so any host can easily block us if required - luckily, this has never happened so far.

Yep - Screaming Frog (desktop tool) has a limit of 500 pages in the free version, which is probably enough for most smaller-sized websites.

In terms of web-based tools, VisualSitemaps.com also has a freemium option with a 50 page limit.

Watch out, Google!

Thanks for the feedback.

We have looked into developing the crawler for SPAs, but haven't managed to create a robust enough crawler just yet.. we'll get there one day!

We do offer an XML sitemap importer if you're in a hurry, but the free generator is always from a fresh crawl :)

Thanks for letting us know about the pricing page on mobile!