UK Government Selects OpenAPI v3 | Hacker News Reader

UK Government Selects OpenAPI v3

technology.blog.gov.uk

158 pointsedent31 comments

This also covers CSV as well as APIs.

Links:

https://tools.ietf.org/html/rfc4180

While using the open API spec is good, I'd say the standardisation of CSV data will have more impact. More of the open data available is in the form of historic reports rather than real-time: https://data.gov.uk/ (e.g. https://data.gov.uk/dataset/27dd8d7f-c58b-4de3-bf0d-81058465...)

I can't find any mention of CSV in the OpenAPI documentation or the first link you provided, can you elaborate a bit how this works?

I think he means that the UK govt has mandated a particular CSV standard for tabular data, at the same time. I think you're right that OpenAPI has nothing to do with CSV.

Yes. CSV with the obvious extra conditions "UTF-8" and "one header row" is fairly interoperable, but note: "when you’re using Excel for Windows to load a file that is UTF-8 without a BOM, you can resolve the display of non-ASCII characters by using the ‘import text’ option". If a Windows user just clicks on the file then sometimes UTF-8 is interpreted as some 8-bit encoding and you get mojibake. Nevertheless, you shouldn't add a BOM because the standard doesn't permit that and it might cause all sorts of problems on other systems.

Mojibake

https://en.wikipedia.org/wiki/Mojibake

love this word!

Is that true? Iirc Openapi lets you define schemata for csv payloads in POST body sections. Whether they are validated by the backends is another question...

The page title is actually "Improve CSVs and API descriptions with these Open Standards Board recommendations"

I spent a while looking through the registered birth names data for about 5 years while we were naming our kids. Each year was in Excel sheets, which changed format repeatedly. It was really annoying.

Moving over to providing everything as well formatted CSVs is going to be so much nicer.

This is great news.

As someone working on GOV.UK projects and regularly working with large, unwieldy CSVs I can't recommend CSVKit[0] highly enough. Being able to go from CSV to PostgreSQL in one or two simple steps is _priceless_ for me.

[0] https://csvkit.readthedocs.io/en/latest/

I've had good success using Postgres' built-in CSV import/export command (COPY) in the past — depends on one's workflow, obviously!

https://www.postgresql.org/docs/12/sql-copy.html

Anyone know/recommend any API schemas/standards for REST APIs that allow machines to discover their resources?

For example I see OpenApi v3 mentioned here (which uses a yaml file to give context) and I am also familiar with NGSIv2[1] which will get standardized by ETSI supposedly (NGSI-LD[2]). NGSI seems to embed info in the json so that your application can know if this float is Celsius or Fahrenheit.

Any other open options?

[1]: https://fiware.github.io/specifications/ngsiv2/stable/ [2]: https://fiware-datamodels.readthedocs.io/en/latest/ngsi-ld_f...

Take a look at JSON-LD and Hydra

https://json-ld.org/

https://www.hydra-cg.com/

you can also look at Hypermedia as the Engine of Application State (HATEOAS)

Coauthor of openapi.tools here. If y'all have suggestions about things we can do to make the project better, we are all ears!

This is really cool! The one feature I would have asked for is already in place: authorization - so that's super cool.

That said, hmm, what do you feel like OpenAPI learned (so as not to repeat) from the pitfalls of SOAP WSDLs?

That is a fascinating question. Mostly because I wasn't around for SOAP. I started my development career kind of with the rise of REST. I've only ever used REST personally, so I couldn't really tell you. Looking up WSDL, I can tell you one thing I've noticed with OpenAPI is that it is both incredibly descriptive, and yet not in the way. I can look at an OpenAPI file, figure out what is going really quickly and get working. With WSDL/XML you have to parse the whole XML schema to really understand what is going on.

I wish tooling was available for the company I work for to use v3. So much of the ecosystem is stuck on v2 or worse.

What's the thing you miss most from v3 working in v2?

nullable fields for example

The oneOf keyword is really handy. I actually resorted to implementing the "sub-schema" validation using another tool when this wasn't available.

I think it's in JsonSchema validation and Open Api v3 but not v2.

Stoplight Studio is prett good. Also Apicurio, but lags slightly in features.

We use Connexion at work, which supports v3 now (though we haven't gotten around to updating it from 2 yet). If you use python/flask, I _highly_ recommend checking out Connexion: https://github.com/zalando/connexion

If youre in node.js land, I wrote a framework for handling the server side of OpenAPI v3: https://github.com/exegesis-js/exegesis

You might be interested in Pyotr [https://pyotr.readthedocs.io/en/latest/] -- still under development.

One of the criticisms of OpenAPI is the verbosity of the schema definition.

I've seen a couple of meta generator projects that generate OpenAPI specs from a meta description.

Is anything being done 'officially' to provide an abbreviated form for simpler use cases?

Why does it matter ? It's not supposed to be read by humans.

But it's to supposed to be written by humans

I think it is supposed to be generated automatically.

It is supposed to be read by humans: "The format is easy to learn and readable to both humans and machines."

Source https://swagger.io/docs/specification/about

Unfortunately, actual usability seems to be getting ignored in the rush for a "standard". In the meantime, you might want to consider using RAML. It compiles to OpenAPI and is far more optimized for use by humans instead of machines.

The CSV standard looks very sensible and reasonable.

Reply on news.ycombinator.com