Show HN: Diff.io | Hacker News Reader

Show HN: Diff.io

diff.io

194 pointswickedlogic99 comments

I see some green blobs, some pricing info, and hear some marketing speak about letting my CMS be a "self-actuated change awareness system".

I want to know how I integrate this API into my tests, what exactly it tells me, and how to use it. How about a free plan to test it out? How about developer docs? How about something more than a few screenshots and a 30 second video that is incomprehensible marketroid speak?

Sorry for the harshness, but I really think visual diff tools are needed for integration testing. I want to figure out which ones are flexible and how they can be used. This site helps me do none of that.

Harshness is fine :)

There is still more work to be done on making the service more flexible, mostly just opening up the rest of the api on my end.

How would you describe a CMS that self documents externally?

Perhaps this helps a little: http://blog.diff.io/show-hn-diff-io/ (?)

This could be used as a way to make sure a web 2.0 app hasn't got some error and lost some content... e.g. some pane is empty

Would it find errors here? :)

https://i.imgur.com/xg3BSNs.png

I've been getting the same kind of display errors on lots of different sites in chrome. Inline elements aren't given enough width, and then when I go to inspect them it is fixed.

Hah, sounds like I need start tweaking that theme's css. That's bad. Email matt@diff.io the device/browser/resolution?

Weirdly, I can't reproduce it now, so you either fixed it, or it was a bug on my side (browser, extension, ..?). Anyway, it was a screenshot from Opera 29 (Chromium 42) on Windows 7 with uBlock as one possibly related extension.

I got the same the first time but can't repo either

Have a look at Applitools for a similar tool with advanced features

You might like this:

https://github.com/bfirsh/needle

Would be cool to get a Diff.io driver for it, or something like that...

or these:

https://hub.com/cburgmer/csscritic - JavaScript, runs in browser

https://hub.com/Huddle/PhantomCSS

https://hub.com/bslatkin/dpxdt - Python, Deploys to App Engine / CloudSQL / Compute Engine

https://hub.com/BBC-News/wraith - Ruby

https://hub.com/sindresorhus/pageres - JavaScript, nodejs, PhantomJS

I think you meant:

https://github.com/cburgmer/csscritic - JavaScript, runs in browser

https://github.com/Huddle/PhantomCSS

https://github.com/bslatkin/dpxdt - Python, Deploys to App Engine / CloudSQL / Compute Engine

https://github.com/BBC-News/wraith - Ruby

https://github.com/sindresorhus/pageres - JavaScript, nodejs, PhantomJS

whoops, yes. My extension to highlight words I'm interested in on HN must have removed them from the input box when I edited it.

If you want a DIY version of this, try ImageMagick's compare command:

compare bag_frame1.gif bag_frame2.gif compare.gif

Documentation: http://www.imagemagick.org/Usage/compare/

Interesting. And to render the html you can use cutycapt. Which is in the Debian repos too. I just tried it, and this rendered a nice diff of two pages for me:

  cutycapt --url=http://www.xyz.com/1 --out=1.png
  cutycapt --url=http://www.xyz.com/2 --out=2.png
  compare 1.png 2.png diff.png

This is why I love HN. Someone launches a web service with a flashy site, lots of marketing blurb and monthly plans from $10 to $200. And in the HN comments thread, two random people reimplement the functionality in three lines.

Now make it a webservice and charge $200/month!

> Now make it a webservice and charge $200/month!

I don't actually see the problem here. They provide a real service, so they can demand real money. There are multi billion dollar companies that do less for their users.

First part already done:

http://archive.is/

apt-get install the services and use them on my server ;-)

Very interesting approach.

Tip: Remove the "plans" page. This thing is too early stage to start talking about $ plans. Nobody even knows what the hell this is or what it is good for.

Your business model should be as follows:

1. Make something new, useful and free. 2. Get people hooked on it, like a drug. 3. Grow a modest user base. 4. Introduce "value added" features, marked as preview or beta, for enterprise customers and integration into other web services. 5. Once features added in step 4 have matured, remove the beta clauses and slap a price tag on them.

Absolutely, 100% disagree. Paying customers are not a foregone conclusion. You need proof, and product guidance, and you need it early and often.

I agree with you about only focusing on the essential at first. I too believe in launching a very MVP and I launched my saas service without even a way to rebill a customer, knowing I had 30 days to finish that. But the plans and subscription are essential for launch unless your product is not stable, in which case you should do an invite-only beta with your friends.

I sell products that don't even have a website. Even sell products that are not even finished. That doesn't stop people from paying for them or finding then useful. In fact, I close a client once a month in b2b sales and I don't even know how to sell that well. If I followed your plan, I'd be out if money and my clients would have way more problems.

Would love to hear some actionable tips or insight, or resources you found useful.

I mean, its not that complicated. I just talk to people and ask if I can talk to them about their business problems. I only do it with people I've sort of met before because I'm afraid of cold sales (based on a pure fear of rejection). For example:

Last Saturday was my kids birthday party. We invited her friends from school. The parents stayed for the party, which turned out to be great. I knew of the parents was a lawyer. I say hi to her and outright ask her if I could drop by her office to talk about making her life easier through the use of software. That is literally what I said. She smiled and said 'Yes! I need to organize myself better. Please, visit me next Wednesday (tomorrow). I will be there and you can help me.'I was afraid of asking here. Lawyers are scary! But they are also the kind of business owner who relies on technology to get their job done. Document management, appointments, client tracking, billing, etc. So I'm going there tomorrow, with absolutely no plan to sell anything. But an interest in understanding what she needs and wants.

Notice that I don't sell one product. I'm looking to build products for them and them charge them a monthly fee. It takes me about a week or so to build something that (barely) works. Which then gives them the chance to use it and give me feedback. Who knows if one of those products will turn into something big? The aim is not to have an idea for a product, but to find actual needs/wants from businesses and building a product to solve it. The exact opposite of what people normally do. Dont focus on an idea, focus on other peoples problems.

> if I could drop by her office to talk about making her life easier through the use of software

That is exactly what I always intend to say, but my brain has this awesome tendency of not knowing what words to put together in what order. Not dyslexic, just very happy all the time.

Also, I will say that I LOVE the idea of building products and charging them a monthly fee. Do you do any other one-time fees? What kind of fees are you imagining for the project you discussed Wednesday (today)? Again, curious as a cat.

Random guy from HN says: "Your business model should be as follows", geez..

I have found that in most cases asking for money earlier than later is the way to go. There is a difference between finding users and finding users willing to pay.

I don't apologise for cutting to the chase. I took issue with the pricing model and suggested a way that, I think, would be better suited.

There is a different between those types of users, yes, but frankly that is a nicer problem to have than just finding any users at all. What I'm suggesting is not without precedent. Pretty much every SV startup looks to build user base first and monetise later.

There are some successful business models that get massive funding, and go for explosive growth. These often work well for services that either have massive economies of scale (Amazon), or strong network effects (Facebook).

But another business model is to grow slowly, focusing on profitability along the way, and this works well for businesses in a niche (Pingdom), with low network effects (Basecamp) or high marginal cost per user (Digital Ocean). If a business with high marginal costs per user and low network effects goes after explosive growth by subsidising users, they run the risk of running out of cash before the point they have enough profit to cover the subsidy.

There are lots of exceptions - Spotify and Dropbox come to mind. But there are lots more forgotten startups that incorrectly thought "If we get eyeballs and engagement, the revenue will follow".

I should have known throwing in the SV comparison here would cause this tangent. Oh well.

The point is really that this was a Show HN post about a little hack someone put together in probably a single weekend. And with a totally unproven product idea, unexplained use cases let alone what ROI might look like, he has already put in place not just a 1 price plan, but a full swathe from startup to corporate customer. It just seems massively over the top at this very very early stage.

A simple disclaimer indicating that he is monitoring API usage, for now, along with a donation button and a "let's talk if you want to use this in an enterprise environment" etc, would have been sufficient.

I wish the guy well, honestly. But there is something to be said of taking things a bit more slowly and progressively. Verify the product idea is at all viable first. Before showing your full hand with badly conceived product plans. Those plans could very well be doing more harm than good. Until you've gathered feedback from customers you don't know if those $ values are a million miles of the expected mark. And the fact there is no free plan for even limited testing is ridiculous really.

Pretty much every SV startup raises investment, also. Without that, giving a product away for free and "figuring it out later" doesn't really work.

OT: thanks for bringing this up! It is this kind of clash of informed opinions that I read the HN comments for.

And pretty much every money making company is the opposite?

I agree that the landing page needs some work to make it clear what the product does, and why I would want to pay for it.

However I disagree that you must start by offering the service for free. By charging from day one, you can limit your user base only to those people who are prepared to pay for it. You're less likely to get explosive growth this way, but it's easier to keep the service running as you're not subsidising freeloaders.

One of the assumptions in an MVP is that people are prepared to pay for the service, and you can only establish that by requiring people to part with cash as early as possible.

i disagree with this assessment. What harm is there in having a pricing page? I'd support offering a free option, but why on earth would you ever disallow willing customers from paying you money?

On the contrary, i am now tempted to build a "Buy" button on a basic marketing page before i ever write a line of code on future product ideas.

"Plans" was the first thing I clicked on because I wanted to try it out, and then there's no free option so I clicked that little "X" by my tab.

I actually have a use case for this and so do a few people I know, and I'd love to try it out.. But you've got to be kidding if you think I'm going to give out my payment data to try a website version of FollowThatPage.

This is horrible advice. Charge as early as you can.

> 1. Make something new, useful and free.

FREE IS NOT A FEATURE

This seems like it could be great—can I drop this into my CI workflow to find visual regressions?

That's the question you should be answering. Not trying to sell some "self-actuated change awareness" mumbo jumbo. The video literally sounds like an infomercial for a cult.

Sorry for being harsh, but it sounds like you have some cool technology which is unfortunately overwhelmed by terrible marketing.

>mumbo jumbo...cult...terrible marketing

Meh, I don't think it's that big a deal, nor do I think the marketing is bad. You clearly understood that there was some "cool technology" there, right? And, CI is just one use case, right? Still, while the creator doesn't appear to have designed it so narrowly, you are nevertheless seeing the potential application and inquiring. I'd say that's a marketing win.

Of course, with more interest and feedback, the product and message can be honed.

So, maybe your feedback will help the creator, but there are better ways to communicate your opinion. Don't want to be "harsh"? Then, don't be. No need for the side commentary about cults, mumbo-jumbo, and terrible marketing. Those phrases contributed nothing to your otherwise potentially useful feedback. But, they may be discouraging early words for someone who's just poured a lot into a project.

Perhaps you're looking for Percy [0]?

Similar functionality unless I'm missing something, but more focused on the CI use case than CMS.

[0] - https://news.ycombinator.com/item?id=9624673

Good feedback, thanks.

I do see your pointed about CI workflow, but I think visual regressions are the current obsession and a limited use case. A very valuable and useful use case, granted.

But if something like diff.io was in place in your browser, in your content publishing system, in your social networks, having it just in your CI would seem limited. Would you agree?

> But if something like diff.io was in place in your browser, in your content publishing system, in your social networks, having it just in your CI would seem limited. Would you agree?

No, I don't. I have about 0 interest in seeing diffs of social networks or most web pages. It might make sense in a CMS but even there I think most people are better served by traditional textual diffs.

It sounds like you've found a good solution for a medium problem (preventing visual regressions) but want to expand it into a "movement."

One solution would be to use an existing GitHub integration to trigger a screenshot on diff.io.

Shameless plug: I'm the founder of https://commando.io, and we have a GitHub service that allows users to run scripts (we call them recipes) on push of a repo on servers. You could write a simple recipe in Commando.io in bash to trigger the screenshot on diff.io:

http://public.recipes/Xr1eCr

Your "bootstrap" level should be free. I don't see any bootstrapping startup paying for a service they don't need / can build themselves for free (if they're really desperate for it.)

You want to capture that potential future business, and hope that you can convert them to paying customers as they become profitable; you don't want them to go "hey, great idea, but I'm not paying for that" and then implement their own solution.

Isn't it better to just sell to people who can pay? I mean, why being so against asking for money? Its strange how you say that if they really needed it they could just build it. That's a bit of a stretch. Why build something you can pay for and focus on making money instead? Dunno. I'm just confused.

You barely have to build it, there is a great open source project already available. Not sure if it does everything diff.io does but it's a great resource to do a lot of it.

https://github.com/webdriverio/webdrivercss

https://github.com/webdriverio/webdrivercss-adminpanel

Maybe, but many people (myself included) may be reluctant to use it if they can't even try it out without paying for it.

Especially if the core product can be replicated with 3 lines of code using open source libraries.

Three lines of code? Is this something you could do in three lines? I mean, only deploying this live requires way more than that. One thing people seem to forget is that building your own tools means that you also have to deploy, maintain, and fix them. If programmer time is > $100, how much will it cost you to deploy something like this? I'd rather pay the money and move onto building products that make me money.

True, but this is just a service, you're going to have to write code to integrate with it anyway, why not just call ImageMagick (or whatever) directly?

You are severely limiting them if they don't pay. So I guess hooking them (or at least letting them try it) and then upselling sounds better for both parties.

I have the hardest time upselling from free. But its pretty easy to go from $300 to $500.

I agree that a very limited, free tier should help the product gain traction and let people try it out. But:

> can build themselves for free

Bootstrapping startups should not be focused on trying to save $10/month here and there by recreating microservices. A startup may decide to build their own solution for valid reasons, but it most certainly is not "free".

I don't know... I see your point, but people really need to learn that stuff isn't free.

If you do a free tier, you run the risk of having to support infrastructure for people that will never need to exceed whatever you free limit is, just because they couldn't be bother to do the work themself, even if the tools where free.

I just want to say that I strongly disagree with the people here who are complaining about your pricing page.

You need to validate that people will pay money for this product, and the quickest way to do so is to ask up front. I really don't buy this "Give it away for free, get them hooked and then staple on paid features" approach.

PG said it best: "Better to make a few users love you than a lot ambivalent."[1]

One of the best ways to find out if people love what you're building is to ask them to pay for it.

[1]http://www.paulgraham.com/13sentences.html

I've half built a tool to do this using PhantomJS, and a really half-baked API. The image grab part is easy. The diffing is what's hard... well sort of.

What I tried and what this site appears to do is a straight pixel change detection, which fails to account for how important that change is. Minor things change on a site all the time it's catching major breaking changes that's hard, say a CSS rule change that looks fine on desktop but ruins the site on mobile.

I've used CasperJS, PhantomCSS and PhantomJS for visual regression testing on dozens of Drupal sites and, while not a perfect solution by any means, it's worked pretty well. PhantomCSS seems to be pretty configurable and makes it rather easy to detect breaking changes.

O nifty I hadn't seen PhantomCSS before.

When I built something like this, I used http://wkhtmltopdf.org and http://pdiff.sourceforge.net.

Nice work! We have a similar feature built into Ghost Inspector: https://ghostinspector.com/#visual-regressions -- It includes a tolerance setting so you can basically say "I only want to hear about it if the screenshot changes by xx% or more." Effective visual regression testing at scale is tough though and is really more of a secondary feature for us.

If anyone is interested in other solutions, ImageMagick has a nice "compare" tool built-in. There are also plenty of open source projects like Huxley, PhantomCSS, etc. I recently saw a demo of Applitools Eyes (https://applitools.com/) and it's quite powerful (though it's a paid service).

Lastly, I did a presentation that involved visual regression testing here: https://www.youtube.com/watch?v=mK0l__jmpTA (starts around 12:25)

Applitools Eyes has a free account option as well, which grants access to all automated visual testing capabilities as in the paid packages - for a single user.

Monitoring is a good idea. However, pixel-based comparison services are quite pointless for web pages. Banners, dynamic content etc. simply drives spam.

Try Browserbite with its feature-based comparison. There are other regression-oriented tools out there as well that use pixel-based methods as well.

Hey nice! I've actually done something similar myself as a weekend project - https://page-watch.com

Also checkout the following projects, https://visualping.io/ and https://dpxdt-test.appspot.com/

If you are using something like phantomjs to generate website screenshots I wonder how you are dealing with dynamic content. A lot of pages have continuous animations that can screw with simple image diff comparisons. For this reason you may want the option to limit your compare to a subset of the page.

Very nice! Your CMS integration looks like a fantastic way to mitigate the onboarding pain of most integration testing systems.

Definitely a useful type of test to have and certainly comes from a place of pain. Last year I worked on a bootleg similar project during the YC Hacks event—it was hard to get it to work right.

Recently Applitools Eyes[0] started gaining popularity for CI-based visual testing, I hooked it up to some Selenium integration tests at work earlier this year and the things it catches have consistently amazed me. Catches nearly all of the bugs that manage to slip past the typical unit / end-to-end tests.

One hard part they navigated well has been the interface for being able to review changes, set new baselines, and set a certain area as "ignored". IIRC they even use some fancy computer vision algorithms to handle slight variations in screenshots (e.g. font alignment false positives).

[0]: https://applitools.com/

You are probably using Blink-Diff for the image comparison. Isn't it? https://github.com/yahoo/blink-diff

For example: https://raw.githubusercontent.com/yahoo/blink-diff/master/ex...

You can change the highlight-color to green (it is a configuration option), just like it is on the page there: https://diffio.global.ssl.fastly.net/v1/diffs/2015-05-28/eee...

And the "montage" is called a "composition" in Blink-Diff.

It isn't Blink-Diff, but I had looked at that project at one point. It has some interesting options. :)

I want to know what this does, the youtube video does not explain clearly in layman's terms.

Looks like a useful service. Just a word of warning, your SSL certificate is not trusted by Chrome and possibly some other browsers. You have to install Intermediate certificates to make it work for everyone.

Works for me on Chrome 43.0.2357.81 (OS X)

Green lock + "The identity of this website has been verified by COMODO RSA Domain Validation Secure Server CA but does not have public audit records."

No warnings.

Damn, I had the exact same problem in another thread with an unrelated domain name : https://news.ycombinator.com/item?id=9645297

Also a comodo certificate.

Now it works for me to, I guess they fixed it in a meantime.

Yes, thank you for pointing that out.

I had recently installed new certs, but failed to point it to the ssl-bundled cert and instead it was just the main cert without the intermediates.

Things are happier now: https://www.ssllabs.com/ssltest/analyze.html?d=diff.io&hideR... "A+"

Your API documentation isn't public. That's an instant turn-off.

The dashboard api is still a work in progress, but the actual api is documented here: http://blog.diff.io/talking-to-api-diff-io-via-json-web-toke...

Great project!

This is really a need for monitoring consistency of UX of any website.

Although I just think you should simplify your pricing model. I don't understand why you are talking about "requests". Who cares about bandwidth today anyway? Just make people pay for the number of pages the frequency of the checks.

Also, you should allow people to set up e-mail alerts when some parts of their website pages change (e.g. payment forms).

I would definitely pay for such product. : )

https://github.com/bslatkin/dpxdt

Here's a Video that explains how the Google Survey team uses perceptual visual diffs (dpxdt) in their deployments: https://www.youtube.com/watch?v=UMnZiTL0tUc

It is an option, in fact there are a lot of open source options. The subscription model we offer is one way to gain the ability without having to worry about the complications of hosting it.

the license plates image is misleading as hell!

for a moment you think it is smart enough to realize subject and background and only show you changes on the normalized objects.... but it is actually just an edited image on the other side to add a clean element for the diff.

This is strictly over images, not video. I guess I see your point, didn't consider it that way at all, since it was a picture of my plate and a modified img change to generate populate the diff.

I could swap it with the real plate and the real plate with a sticker, or something. As a regression I thought it was straightforward. :|

Was the confusion that you thought it was doing something like wordlens: http://1.bp.blogspot.com/-X95Ga7iMP74/T_vjrp2PF2I/AAAAAAAAFM...

yep. i assumed it did some image transformation to find the minimum diff between two images as to compensate for two differently angled pictures or something.

This reminded me of Wraith, a screenshot comparison tool which The BBC developed for integration testing: https://github.com/bbc-news/wraith

Your landing page might benefit from using wordpress as an example. I saw the service and immediately thought about how to integrate this into wp. Maybe even turn it into a wp plugin? Dunno. This does seem useful.

This looks fantastic. I was just looking into doing this "manually" with an automated screenshot process for a site I'm refactoring part by part.

https://github.com/facebookarchive/huxley

Hmm, is this built on top of url2png by any chance? :)

Yup, at least that is the primary source of screenshots :)

The site shows an SSL warning message. The certificate details says it was issued by "Avast untrusted CA".

It's more likely a problem with Avast security software, not the website itself. Are you using Avast Anti-Virus or Web/Mail Shield? Avast installs a root certificate to intercept and scan HTTPS traffic[0].

[0] https://itnerd.wordpress.com/2015/05/21/avast-responds-to-my...

Thanks. I use Avast Anti-Virus. I didn't know they are doing man-in-the-middle. But as I learned, this is quite common for modern antivirus softwares, so I turned off this option in Avast.

Why do antivirus seem to be more malicious than the virus they fight?

Well that sounds like spyware if I've ever heard it.

Could use this to monitor competitor's landing pages and see what they split test.

your ssl is signed by unknown vendor so i wouldn't trust it..

This error occurs while you are using a self-signed certificate, it is advisable to go through the trusted third party certificates.

You can use SSL installation checker to diagnosis troubleshoots.

https://www.ssl2buy.com/wiki/ssl-installation-checker/

https://www.sslshopper.com/ssl-checker.html

https://ssltools.websecurity.symantec.com/checker/views/cert...

Nice theme. http://startbootstrap.com/template-overviews/grayscale/

Nice idea. But it would have been great if there could have been a trial/free version. I would like to test the service so that I understand what exactly it is about. Watching a video and reading text does not help much until you try it out by yourself.

Reply on news.ycombinator.com