API-diff: a tool for side-by-side evaluations of JSON APIs

radar.io

107 points·blackmad··7 comments

Nice work by @blackmad!

For those wanting to start in the "shallow end of the pool" tuning Pelias geocoder indexes and config, there's also a web-based UI that can hit multiple instances and visualize the various results: https://pelias.github.io/compare/ Made by the good people at https://geocode.earth/

Shhhh don't tell anyone I also completely rebuilt that tool from the ground up earlier this year ;-) (it looks roughly the same as it did before, but has a million tweaks to make the development lifecycle more pleasant)

If anybody is thinking about testing APIs with this tool you might be interested in a concept called "Contract Testing" [0] or an implementation "Pact Contract Testing" [1].

[0]: https://martinfowler.com/bliki/ContractTest.html

[1]: https://docs.pact.io

It will be very cool to show diff in the API contract with every new version or release. Visually implementing - “what is changed”

This is great. I work with a number of API providers who are sometimes less than consistent with their output.

Maybe a naive question, but how is this any different than diff tools found in Git / SVN / etc.?

I’ve not used this tool before (am excited to in the future though because..) having done a lot of API diffing with text diff tools in the past, they break down very quickly on things like: JSON key order can be random, causing false positives. Some headers will always be different (any sort of timestamp based header for example) while others must not be different. URL query params can be in any order, etc.

I think if you do api diffing at any reasonably large scale you’ll find yourself immediately building tools to help cut the signal from the noise, and this is an exciting step in that direction.

Reply on news.ycombinator.com