Show HN: Monitorbook – Easily track anything on the web | Hacker News Reader

Show HN: Monitorbook – Easily track anything on the web

monitorbook.com

134 pointsevenflow54 comments

Copywriter here. Good writing is concrete. Instead of telling me that I can monitor things, give me examples (You can monitor prices online, etc.)

Here's a business case for how this could be useful and why I'm going to try it.

I operate a string of ~60 local businesses that have physical storefronts.

On average each store has 5 local competitors.

Our customers are VERY price sensitive - they will almost always call the "lowest price" guy in the area first.

As a result we currently have a duct-tape system to monitor competitor pricing and tweak ours as fast as possible.

MonitorBook would be a potentially much simpler mechanism to orchestrate this.

If the MB team is reading this drop me a line and we'd be a potentially great case study / early adopter.

Things that would have to work to make this usable long-term:

a) API so we could pull this data into our pricing algo.

b) Clear error-checking confidence - if a site we're monitoring changes their code / display and we miss it, we'd have to go back to duct-tape which is sloppy and labor intensive, but reliable.

What happens if your competitors realize you are using the service and devise some way to trick MontorBook into reporting the wrong prices?

[Edit: related link, which was the inspiration for my question: http://mailinator.blogspot.com/2011/05/how-to-get-gmailcom-b...]

You change your strategy :-) It's like Cryptonomicon "The most advanced technology, usually wins the war".

If they can do that, you'll have to figure out quickly and adapt... But to my personal experience, there are way more what if(s) flying around than what actually happens.

What happens if your competitors also use the same service, would the algos recursively spiral downward?

Each business has different profit margins so I'm sure each would have to set a bare minimum price thus the aforementioned "pricing algo".

Result of a quick try:

Tracking YouTube views doesn't work (shows "null")

Tracking HN points doesn't work (shows "null")

Tracking Reddit comment count works.

Thanks for letting us know (I'm one of the members of the Monitorbook team). HN was working till a few days until a few days ago. I will look into it. We have done our best to support as many websites as possible but it is not a surprise that our scrapper is not perfect yet. We will keep working on it :)

Why do I have to create a new account for this site? I have Google, Facebook, Twitter, etc; consider using one of those site and OAuth to provide a more frictionless experience for first-time users, especially considering your site requires me to come back X days later to really see it work. My $0.02...

I much prefer an email and password style login especially for new sites. I typically try a service with a burner email account, then I can easily walk away from it all without continuing to get email. Plus if you end up selling my address for spam I know who you are...

If your target audience is HN users, consider this comment. However, I've found in my analytics that most users don't care about the type of login.

You can't please everyone. If a website has only username and password, there's a complaint. If a website only has OAuth, there's a complaint.

"most users don't care about the type of login"

Majority of the population get paid, minority of the population go pay.

Same theory works here.

why not both?

that puzzled me too o.o

I don't mind creating an account, but I'd let users try the product without one. I don't want to go through signing up – no matter if via OAuth or creating a dedicated account – just to find out the product doesn't suit me due to some details I could not find out by the page or video. Unless absolutely unreasonable, I think you should always allow users to test your product anonymously.

How do you remember which one you created an account with?

Wow, seems like everyone has built a version of this. I've been working on something similar as well, and really went for nailing down the element selection. Seems to be a basic duplicate of what you have done. Best of luck.

Really? I couldn't seem to find anything other than webnumbr (which doesn't seem to work anymore).

Sorry, my work pc doesn't have sound so I can't hear the actual video. Does this track sites other than Amazon, or is that just the demo? Also, is there an estimate on the pricing for the premium service?

Overall, I think it looks pretty clean. But I would probably add just a bit more information on the main page. I see it's mostly in the video, but it would be nice to get pricing details, description of exactly what things you can track, etc..

Good job though :)

There is nothing to hear anyway. It is just background music.

Very cool.. but what's the implication on the legal side? Many of the sites forbid "automated" scraping of their sites.(sure they can be all circumvented). On that note, what about sites like kimonolabs (http://www.kimonolabs.com), they also let you create an API on top of an existing webiste. I imagine it's done by scraping. Would love to hear some thoughts.

Scraping Amazon might be illegal, or at least not specifically allowed, otherwise there would have been an API. However reading 'robots.txt' I guess that their policy is 'open to scrapping' by search engines. Because what google spiders do to virtually everyone, can be considered web scrapping.

[1] http://www.amazon.com/robots.txt

On the topic of legality/forbidding scraping, Kimono offers webmasters analytics and control of the scraping users do through its service. I'm not sure if this service is actually live, or forthcoming.

https://www.kimonolabs.com/webmasters

Has there been work after RSS to improve pub/sub protocols for websites to notify customers of changes/deltas, without the need for wholesale scraping?

I'm not aware of anything, but if someone else has some input I'd love to hear about it. FWIW, I believe Amazon does have an affiliate api that lets you search through the catalog. Same with Ebay, Alibaba, and most of the major companies. The only one I can think of that doesn't is Craigslist.

For what it's worth, I'm pretty sure that it's intentional on Craigslist's part. Due to the first come first serve nature of a lot of the free listings, it means that 'honest'/non-tool assisted users would never get a look in.

Nice website. Do you have plans to release an extension or an app for this? I have been using Distill Chrome extension[1] to monitor pages with dynamic content with pages that require authentication.

[1] https://chrome.google.com/webstore/detail/distill-web-monito...

Yes, the plan is to release a browser extension pretty soon. With the extension you will be able to track content in pages which require authentication. We also have an iOS app in the works so that you can check your trackings on the go, and receive push notifications.

PS: I am a member of the monitorbook team

See also Huginn (https://github.com/cantino/huginn)

I had something like this on my back burner for years called Sitebeagle. It's command-line only. Glad these guys took the initiative to build something way more robust. It'll be great for tracking 1 day deals, seeing if somebody's logged in somewhere, and getting alerts for when band tickets go on sale. The possibilities are endless.

Very cool, I like it. Only comment is that you should try and get the video zooming in on where the mouse is or do some close-ups of the screen. It was hard to see exactly what was going on and it felt closer to a techie screencast than to a customer product demo.

Is it possible to track the price of an ipad mini on craigslist? ;p

We will try to put up another video with better quality within the next few days. Thanks for the feedback!

I use websec as a cron command to do that for pages I care for and implemented at some point an app in python I called NoticeThat that fell into oblivion as I had no idea how to market it.

It's nice to see someone catering to that use case who can actually execute on it.

Neat - I have a similar side-project in the works and I bet one of the challenges will be how to handle pages that are difficult to scrape.

You might also want to check out https://visualping.io/

Very cool. I've created something similar in the past to monitor various pages but I did not get past the basic mock-up stage.

There are many use cases for this type of tool. Congrats on shipping something.

How does this handle HTML changes over time ? I believe this will store a URL + Selector , then periodically scrap the page over time. just curious what happens if the page changes later.

You probably some sort of notification and you have to go the web page and match the item again.

Awesome. Along with the other people posting here, I also created something similar a while back (https://keepupdated.co)

Conversation going on on Product Hunt: http://www.producthunt.com/posts/monitorbook

So... what does it do? I don't wanna sign up to figure out, in more specific language, what exactly it tracks. Users? News? API calls?

I found the 57s video on the front page to be quite instructive:

it's a bookmarklet that identifies some portion of the screen to scrape (a numerical value in the example), and tracks it over time, allowing you to set alerts at thresholds (presumably so you can buy at a favourable price)

The front page has an explanatory video. It looks like you can select a part of a webpage and it will produce alerts whenever that section changes by repeatedly polling the website (e.g., the price of a product on Amazon, like the video shows).

I'm in class and can't watch a video... would be nice to have it explained in text

Found a typo on the signup page for premium accounts: "Enter your email below to get early access to out premium account!"

https://www.changedetection.com/

Nice. There's http://feedity.com too.

Nice project. I'm working on something similar but command line based and open source.

"Receive real time nofitications"

Gentle suggestion to spell check.

copy of https://visualping.io :-) :-)

Smells like Shopify!

What does this have to do with Shopify?

https://monitorbook.com/user/sign_in

https://example.myshopify.com/admin/auth/login

Very similar styling

Ah, that makes a lot more sense now.

Excellent observation, China!

Reply on news.ycombinator.com