PageTrawler - Automated Content Auditing and Site-Mapping | Hacker News Reader

PageTrawler - Automated Content Auditing and Site-Mapping

pagetrawler.com

9 pointsvmind7 comments

With regards to what content auditing is, there's a freely downloadable guide to answer that: http://www.pagetrawler.com/content/working-with-content-inve... Valid point though, more explanation would be handy.

There have also been a few requests for an example of the results before we ask people to get stuck in, so I imagine that'll get done soon. This is an Alpha, and there is lots of functionality and deeper / more valuable results we want to provide - any feedback is good feedback.

I can understand that you're aiming at "content auditing" as a task - taking an inventory of a website and gathering a bunch of statistics, as well as insights. If the minimal alpha product only produces an inventory and nothing else, well, ok, it's a minimum viable product and will get you closer to people who actually do what you describe.

The problem I see is that most of the information you mention comes from sources that are unavailable to you - besides the URLs and the page content, visitor statistics have to come from elsewhere (i.e., google analytics or piwek or whatever), and with navigation info (as well as "who owns it" or last-modified info) it's probably better to retrieve directly from the CMS than to take guesses.

Then again, if you can do without (or can provide a "simple" php page that people can drop somewhere and that allows you to siphon off the information that you need from WordPress/Magento/other standard software) in a secure-enough manner, this will probably be popular with those people who don't want to talk to the grumpy IT folks right away.

This is not so much a minimum viable product as a proof-of-content (we knocked it together in 5 days to see if there'd be interest in what we'd like to build).

Agreed that taking guesses is not ideal - we'd like to give users the ability to give authenticated access to services like Google analytics / Clicktales / native CMS analytics & siphon off the raw data they need, and then do our own additional assessments with it.

I'm not sure I understand what this does. What is "content auditing"? It says "sign up and we'll tell you what content auditing is" - you might lose people there.

Also, maybe show an example of the results - I put in my domain and it showed the first 10 links with a status code - but I'm assuming this is more than just wget!

I sure wish I'd had this tool 10 years ago when I was actually working on large websites.

How is PageTrawler or "content auditing" different from regular crawling of a website? I ran a website through and the output was basically the same as any off-the-shelf web crawler.

At the moment it IS pretty basic. However, this is just an Alpha which we built in 5 days as a proof-of-concept. Ultimately we aim to add much richer information that gives insights into how users are interacting with the content on the site, and whether individual pages (or even clusters of pages) are fulfilling their intended purpose. In fact, you can see some of our intended functionality over at our UserVoice page: http://redgate.uservoice.com/forums/129997-pagetrawler

Reply on news.ycombinator.com