Write OpenAPI with TypeSpec | Hacker News Reader

Write OpenAPI with TypeSpec

blog.trl.sn

118 pointsbterlson72 comments

As someone who has used JAX-RS (Java) and ASP.NET, APIs are basically created with these kinds of annotations right in the language.

    @GET
    public Character getCharacter(@PathParam("id") int id) {
        return db.getCharacter(id);
    }

That's very similar to TypeSpec's

    op getCharacter(@path id: safeint): Character;

Java and C# classes already have the type information that you'd be getting from a TypeSpec:

    // TypeSpec
    model Character {
      name: string;
      id: safeint;
      status: "Alive" | "Dead";
      class: Class;
    }

    enum Class { warrior; wizard; }

    // C#
    public class Character {
      public string Name { get; set; }
      public int Id { get; set; }
      public Status Status { get; set; }
      public CharacterClass CharacterClass { get; set; }
    }

    public enum Status { Alive, Dead }
    public enum CharacterClass { Warrior, Wizard }

But with C#/Java you don't need to write the spec and make sure that it stays in sync with what your endpoints are actually doing. TypeSpec looks a lot better than writing YAML by hand, but I'd still rather something that was tied to the actual code that will be executing so that things are never out of sync or wrong.

This is the code-first vs schema-first debate. Both have their pros and cons. Personally a die hard advocate of schema-first.

Agreed. If the schema can also generate the service routes, models, serialization, etc, and you just have to maintain the business logic in the service, you get the spec matches the service benefits that way as well. Best of both worlds? People using gRpc seem to be 100% fine with schema first and generating the service stubs.

One of the key things with having the spec is that you can actually describe a lot more than you can with the various attributes and comments in the code. Especially things that are not as much service concerns but potentially client concerns or documentation concerns. You can also encapsulate reusable API patterns so you know different operations are following the same pattern.

The author didn't go into all the details but there are lots of ways in TypeSpec to separate the concerns of different consumers of the spec. There is a lot of opportunity and creative thinking that can be done here.

Yeah I'm also on the schema first side of the debate.

I think for me it comes down to a few key points:

- APIs are forever, the choice of language/framework is an implementation detail

- Constraining yourself to what can be represented in the specification is better than generating a specification from implementation that may not be capable of expressing the full details

- When working with diverse languages it provides a common ground/language for discussing API changes. Eg: if you have java backend, kotlin android, swift iOS, react/whatever web you can bring everyone together with the spec

- Subjective, but a good spec will include a bunch of documentation and examples that tend to create a lot of noise in the code. I personally prefer to keep this in the spec and the implementation smaller

I think the main counterpoint to this is that you can generate the spec and then take that and change your mind if you later change language/framework etc - it's not a one-way door.

My biggest bug bear is that regardless of spec first or implementation first, you should have something you write once and generate the rest of the glue from (eg: docs, client sdks). Writing each piece manually/independently always leads to drift and bugs.

(I'm working on my own little openapi -> typescript code generator over here https://github.com/mnahkies/openapi-code-generator - eventually plan to support more than typescript, and adding typespec support is something I'm currently considering)

Hey, nice work on openapi-code-generator, its output is very nice. I think you could probably package it up as a TypeSpec emitter without too much difficulty, by emitting OpenAPI and feeding it to your generator. I am doing a similar thing for a Kiota emitter I'm working on. We recently added an API to get the OpenAPI as a JS object which may be of help in this quest:

https://typespec.io/docs/libraries/openapi3/reference/js-api...

Thanks for the reference! This is essentially what I had in mind - allow typespec as input and transparently convert to openapi then feed to the existing process.

I suspect there's potential for some loss of information with this approach (I'm relatively new to typespec so not sure) but I plan to do this as a first pass and then consider a more direct approach later if I see gaps worth fixing.

Code first for prototyping and early stage.

Then switch as soon as you have customers.

Wondering if you can generate your service stubs and your clients to use for testing from TypeSpec, could it be faster even in the prototyping/early stages to define your spec first? A la gRpc?

Sounds reasonable.

In an org where the customers are internal teams it makes more sense to sit down together and define a schema first. Then the consumer and provider teams just go away and write the code concurrently without having to waste time talking to each other.

In this case, write mock data first

Yeah also schema-first here. I think this is the “easy vs simple” debate all over again.

Writing code first schema is very easy, but when you start _using_ that api as part of a greater system is where the approach starts falling short.

Schema first allows for great communication between teams - one team requires changes to an api service, they can hash it out with the api, and then go back and implement both the client _and_ server simultaneously.

The great benefit here is the inevitable back-and forth can be done together, as each side might need to adjust the api while they are implementing the client/server as often happens with engineering efforts. And thats a lot easier to do while each side is working on it rather than the usual one side is “done” and moves to another task and needs to go back and modify.

In fact at the time I built quite a nice system of generating typescript types for both client and server - https://github.com/ivank/laminar

I guess because the project tried to introduce strong typing and fp style to node http servers plus a few other ideas stolen from here and there, it tried to do too much and never really got traction.

I like the idea, especially the TS-like syntax around enums and union types. I've always preferred the SDL for GraphQL vs writing OpenAPI for similar reasons. Most APIs I've run into in my career would benefit from modeling their API responses as ADTs versus the usual approach of overloading 4 different union members into a giant spare object.

I echo the sentiment others have brought up, which is the trade-offs of a code-driven schema vs schema-driven code.

At work we use Pydantic and FastAPI to generate the OpenAPI contract, but there's some cruft and care needed around exposing those underlying Pydantic models through the API documentation. It's been easy to create schemas that have compatibility problems when run through other code generators. I know there are projects such as connexction[1] which attempt to inverse this, but I don't have much experience with it. In the GraphQL space it seems that code-first approaches are becoming more favored, though there's a different level of complexity needed to create a "typesafe" GraphQL server (eg. model mismatches between root query resolvers and field resolvers).

[1] https://github.com/spec-first/connexion

We're also spec-first at my co (pretty mature, large API surface area), but maintaining the OpenAPI spec in YAML is extremely annoying and tedious. This checks a lot of boxes in terms of being interesting, but what does the path from an existing YAML spec to TypeSpec look like? In an ideal world we could just ingest our OpenAPI spec, get the TypeSpec version out, then be able to clean up/iterate from there.

I didn't see anything in the docs, but I would love for someone to tell me this exists.

Re: the post itself, this was much more compelling than TypeSpec's website imo. At the very least it feels like some of these examples, such as the enum bit, should be ported there. Also interesting that there isn't a single mention of languages like Cue[1].

[1] https://cuelang.org/

It does not exist, but it will be worked on fairly soon. You can track progress on GitHub [1]. It will be as you suggest - a one time conversion, after which you can iterate. We have this workflow for inside Azure, but the converter is Azure-specific (e.g. converts to TypeSpec that uses our extensive Azure-specific component library). It will take a bit to generalize.

[1]: https://github.com/microsoft/typespec/issues/3038

Targeting OpenAPI 3.0 is a bad idea. It wasn't until 3.1 that it became truly useful, if only because it fixed one glaring blunder: Pre-3.1, you inexplicably couldn't provide a description along with a $ref. This defeats one of the major purposes of OpenAPI, which is... documenting your API.

Example: If you define a structure called Rectangle and use it all over the place, you can't say what this rectangle means or that rectangle means when the structure is used in your API. How was that ever deemed acceptable? Widespread use of common structures (which one would certainly expect in a well-designed API) resulted in an API that couldn't be documented in OAS.

The OpenAPI code-generation landscape is truly abysmal. Not only do almost none of the tools support version 3.1 (which has been current for years), but the generators themselves are of widely varying (but mostly shitty) quality and scattered across a couple of apparent major repos. The documentation is likewise a confusing morass of conflicting and incomplete information from several sources. I flailed away trying to fix one generator for weeks, or write a new one. But I couldn't even find a concise list of the data structures offered to the template engine after an OpenAPI doc is ingested by OpenAPIGenerator. Isn't that a fundamental part of creating a new generator?

So it raises the question of whether OpenAPI is salvageable, or whether this new DSL is better... from design to code generation. Given the trashy state of OpenAPI code generators, why bother generating OAS at all? Just move on and write some competent generators using this new language.

For what it's worth, we support 3.0 because as you note the ecosystem doesn't support 3.1 broadly yet. I'm personally interested to see if 3.1 becomes prevalent before 4.0 is released. Maybe the ecosystem will just skip 3.1?

We generate OAS because it's useful for many folks, including for us in Azure. But like you we didn't have very good luck getting high quality codegen from OpenAPI. Our latest client codegen tech doesn't use OpenAPI, we generate code directly from the TypeSpec, which offers a number of advantages that result in higher quality. Probably at topic for a blog of its own! Anyway, you might be happy to know that we are working on bringing our emitters into the TypeSpec project so anyone can use it with their TypeSpecs.

Cool, thanks for the reply!

Is this client generation available anywhere, or internal tooling?

We're working on bringing it into the TypeSpec project as we speak. You can see an initial demo of it working in this repo: https://github.com/bterlson/typespec-todo.

Awesome – thank you so much for sharing!

By the way, you can use the "allOf hack" to put documentation alongside a ref. TypeSpec emits this for common cases like this one:

https://typespec.io/playground?c=bW9kZWwgRm9vIHsNCiAgLyoqIHR...

Hadn't heard of that, thanks.

Does TypeSpec have description fields? I don't think I've seen any in the examples.

The playground I posted has a description. They're pulled from JSDoc style comments. You can also use the `@doc` decorator, though that's usually reserved for more advanced cases like when you need string interpolation or something.

Thanks. I don't see any in that playground though. Which document are they in? I think I looked in all the ones in the drop-down.

The link should load up with a sample I wrote, I think? It does for me anyway! Let me know if you don't see it. But I'll also paste it here:

    model Foo {
      /** this is a description */
      x: Bar;
    }

    /** this is also a description */
    model Bar {}

Thanks. Never having seen a description in this language, I wouldn't know that the above syntax designated one; it looks like a comment.

I also can't find any similar strings in any of the samples in the drop-down list.

Wow, I was _just_ thinking that it would be excellent to have something like this the other day.

I really prefer spec-first development, since it gives other developers an opportunity to review the API _before_ the changes start being implemented, and trying to do this from a codegen-based OpenAPI implementation frequently leads to the PR being "broken", since you've only changed the signature of all your methods and not the implementation yet.

But changing a big OpenAPI yaml file is really hard to easily review, and breaking it up into included files only helps a little, honestly. We can compile the new spec and host the HTML in a temporary location to make it easier to view what the new spec will look like, but once you've done that, you're no longer looking at the diff of what's changed.

TypeSpec looks terse enough to be easy to review (and to write!) which really looks like it'll help with that. I'll have to mess around with it in some personal projects of mine with reasonably-complicated specs and see if there's no obvious speedbumps first, but I'm hoping not, because I'd love to start using this in all the projects I'm contributing to!

I wonder why this isn't written in Typescript directly so as to handle documentation, validation and type definitions in one go.

Especially if you want to reuse type definitions of objects elsewhere in your code.

An API request or response usually do not contain the exact model attributes. Only the public facing ones which are then often filtered down some more, depending on the user making the request (and their roles, groups, permissions).

> I wonder why this isn't written in Typescript directly so as to handle documentation, validation and type definitions in one go.

I suspect they want TypeSpec to be more runtime agnostic (as also tends to be the preference with TypeScript). Specifically to support arbitrary validation/serde.

Having written a fully integrated generalized solution (runtime-defined schema -> parser, types, documentation, serializer), by far my biggest regret the first time around is coupling it so closely to a particular schema runtime. Partly this is because I have bigger dreams for designing a schema system purpose built for the task (I had previously piggybacked on io-ts, which worked but it took considerable non-essential effort and complexity); partly it’s because there can be significant tradeoffs between different runtime solutions in terms of performance and ergonomics.

Our first attempt at a TypeSpec-like thing was in fact a TypeScript dialect! It works great for simple cases, but at the limit it falls over for a number of reasons. HTTP needs a lot of metadata that isn't found in TypeScript, things like HTTP verbs, query/path params, headers, routes, etc. API patterns also need their own metadata, like for pagination you need to know various bits of metadata that describe how to paginate an endpoint. Also many things in the TypeScript type system and stdlib don't apply in the context of API descriptions, so we needed a subset.

So after going down this road for a while, we found we were in a place where the syntax was actually very complex, with metadata spread between deeply nested generic type instantiations and JSDoc comments, and didn't feel at all like TypeScript. This didn't align with our goals of having something simple and easy to learn for all devs.

So I wrote the first implementation of a custom language in a day and folks really liked the direction. 4 years later, here we are!

Question for the author, why not fully adopt TypeScript? Ie have TypeSpec be a well-defined, limited subset of TypeScript? At first glance, it seems to me that a model maps to an interface, an op is a function, and an enum is a string literal union.

I understand that you can’t express everything that way, eg annotations, but you could get pretty far with putting those in JSDoc comments, no? I could see a big practical benefit to TS code being able to read an API spec directly without another conversion step, so I’m curious why you chose not to go with that.

I answered in more detail in another comment, but we actually tried this first, and couldn't make it work for the kinds of complex APIs we deal with in Azure. But, it worked great for simple APIs, and I hope someone releases such a tool at some point.

What are people's thoughts on this vs AWS's Smithy?

We've been looking to buy into Smithy heavily as we no longer have the appetite to deal with OpenAPI and it's horrendous ecosystem.

I would pick Smithy if I had to go spec-first.

I currently tolerate OpenAPI because I'm using a robust code-first approach (Tapir in Scala) on the server side, but I otherwise agree, the ecosystem seems brittle.

TypeSpec and Smithy live in a similar space here. You can generate OpenAPI specs from them, or you can generate assets directly from them. OpenAPI is one of the many possible outputs from TypeSpec.

This is an example the author created of TypeSpec being used to generate clients in many languages as well as the OpenAPI spec.

https://github.com/bterlson/typespec-todo

Ooh, magick.css ( https://css.winterveil.net/ ) spotted in the wild!

Discussion on magick.css from 5 days ago: https://news.ycombinator.com/item?id=39793513

It's a beauty! I had to get on the train early before it's everywhere.

I find it kind of discouraging that we now need a language to define an API that compiles into OpenAPI but honestly I hate writing OpenAPI specs (especially compared to gql).

So despite thinking it’s kinda dumb, Im all ears. Halfway through the article and it seems compelling so far.

Great idea - spend far too long reading & writing OpenAPI!

Particularly anyOf, allOf and oneOf (especially when nested) lead to really confusing nested specifications in OpenAPI. Really like how TypeSpec handles unions & intersections.

Playground is great for getting a feel for it fast too

This project seems really interesting, I'm surprised this is the first time I'm hearing about it. Like another commenter here I'm also surprised to see no mention of Cue (or now Apple's Pkl). Is this meant to be something similar to those?

Also given it's a MS project, would love for there to be some drop in NuGet package for C# use. All these data description languages are really interesting but the fact none of them have a C ABI or only work in the language of choice seems to limit how useful they can be outside of their original context.

I've been toying with the design of something similar here and there over the past couple of months. It never made it past the pen-and-paper stage though. Pleasantly surprised to find this today and will have to try it out for a work project.

My original motivation was the lack of OpenAPI definitions across projects, despite everyone agreeing their existence is a "best practice". As you suggest, developers just truly hate writing them. Unfortunately, even the "generate it from code" approach tends to be complicated and often feels duct-taped together.

Looks interesting. Can it handle asyncapi specs too?

We use boats (npm) to define our openapi and asyncapi specs which uses file based structure to separate definitions for models, paths, params etc into separate files which makes it much more maintainable. Native refs make it super easy to reuse definitions, and you can write custom helpers in js to abstract things like the Page definition in that example.

Having a shared templating language for openapi / asyncapi specs which is ergonomic and can be used by more than just JS devs is a great idea. I'll keep an eye on this project.

Hey, author here. Right now we don't have any streaming or eventing support, but I am working on it as we speak. My first goal is to support describing SSE and JSONL HTTP streaming endpoints, but I want to work toward an AsyncAPI emitter as a peer for our OpenAPI emitter.

My biggest frustration when working with AsyncAPI is how different it's bundling logic is versus OpenAPI tooling, like Redocly.

They're both based on JSON Schema, but AsyncAPI's bundling logic seems to fall down a lot when facing `$ref` in strange places (even though the actual spec allows), such as ref'ing an entire channel or operation.

Looking at boats, it looks a lot like regular open/async api spec documents, with a few features sprinkled in. How great is the transpiled output? Does it pass standard validators provided by OpenApi and AsyncApi? Does it support AsyncApi 3.x?

When I first saw TypeSpec, I was interested in it as a way to have a single-source-of-truth for both OpenAPI and GraphQL schemas... but unfortunately though the readme mentions GraphQL, the team at Microsoft didn't actually get around to adding support to output GraphQL schemas.

How does TypeSpec compare with Bru[0], the DSL used by Bruno? Why would I choose one over the other?

[0] https://docs.usebruno.com/bru-lang-overview.html

Bru is a DSL for actual requests (plus other things like js). This actually made me think of Bru too but I don’t think there’s really any overlap.

Compare Bru to OpenAPI or json schema. Its just kinda non sensical.

TypeSpec is great, but if you're working with Rust and you're about to write a new project that will require an OpenApi spec sooner or later, I'd like to recommend a web framework that has spec generation baked in:

https://github.com/poem-web/poem (see poem_openapi)

All you need to do is derive a trait on your response structs and in return you get an almost perfectly generated spec. Unions, objects, enums are first class citizens.

Also, if you're from coming from PHP, the controllers feel very much like symfony controllers.

P.s. Please do recommend an ORM that would feel closer to doctrine. I miss doctrine.

Writing OpenAPI by hand sucks, this looks fantastic. The only reason I prefer generators from code comments is because the structure and the code itself live in the same place, so it's harder to forget.

Cool, it certainly feels like there's some unrealized better way to describe APIs -- but the part that's more interesting for me is the codegen story? Or some way to validate the service I implemented is actually conformant?

I guess for now it translates to OpenAPI, but the footnote implies that TypeSpec-driven codegen could be better. Maybe more is coming soon?

The author has an example repo demonstrating some of the early support for codegen. https://github.com/bterlson/typespec-todo

Server side codegen is also one the horizon and is a key way to keep the spec and service in sync. So many possibilities here!

As someone who wrote a custom dsl to Open API spec script (in Ruby) I'm really interested to see if type spec would be a better output. Our spec is > 12k lines and we haven't even included everything that needs to be defined.

Does anyone have experience layering this on to an existing API (as opposed to defining the API and then building the code)?

Extensive experience inside Azure. In general, supporting existing APIs is harder, predominantly because it might not sufficient to produce a semantically identical OpenAPI when downstream tools are sensitive to e.g. whether something is a ref or not, the order of properties in the document, whether something uses `const` or an `enum` with a single member, etc.

If you have more specific questions I'm happy to consult!

Haha, thanks for the reply! We're spending some time this quarter working on our open API spec and have engaged some experts, but if we need more help, I know who to reach out to!

This is cool! I made a similar project to this one when we switched from GraphQl to regular REST at $work but decided to drop it since I didn't have the bandwidth to work on it.

I see editor support for VSCode, but is it backed by an LSP or is it VSCode only?

It's LSP, and we also have a VS extension in the marketplace.

I specified an existing api in this, and I think it turned out pretty good. It exports openapi and json-schema from the same spec. Looking forward to having more linters for specs and stuff like that.

We have been using this for a couple of months now and it works really well. It's so much better than writing OpenAPI by hand, and I never ever will do that again after working with TypeSpec.

Thank you!

As someone who prefers schema-first design this looks awesome. Having to hand cut OpenAPI specs is gross and quickly becomes unwieldy.

I found the blog's soft font hard to read. Fortunately my browser's reader mode fixed it.

This looks great, is there a converter to convert an OpenAPI yml file to TypeSpec?

Not yet, but we're working on it. Should be out in a few weeks.

I don’t see a way to specify the security scheme. Am I missing something?

This docs page will hopefully help:

https://typespec.io/docs/libraries/http/authentication

You can, see here: https://typespec.io/docs/libraries/http/authentication#avail...

This is typical Microsoft - overly complicated solution in search of a problem. Yaml and Json are beloved for a reason - they're simple and effective.

I think "beloved" is a bit of a stretch... tolerated perhaps, for their simplicity and versatility, in that you can make them work for most kinds of data transfer and configuration if you don't mind shoehorning your needs into that spare box. But beloved? hardly...

Just because you don’t have the problem it’s solving doesn’t mean the problem doesn’t exist.

Having written quite a bit of open API specs, I don't agree with you. Json is hard to read, yaml has own quirks, especially when you try to spilt it into parts. Amazon also tries to invent own language for describing apis, so I guess they are not happy with open API too. Anyway, without ability to generate code from spec, there is not much use from it. Code gen/nswagger/open API generator and others produce terrible code, at least for java/c#/typescript(there's 4.1k open issues for open API generator), using custom generators for codegen make problem less painful, but that is additional burden, I'm looking for better alternative, would be very interesting to see what they will do with code generation.

Codegen is coming online as we speak. We do codegen from TypeSpec in Azure across multiple languages, and the results are pretty great. We're moving that over to the TypeSpec project so everyone can generate code. Obviously my opinion is biased, but I think the results are significantly better than what you find elsewhere in the ecosystem.

I would be absolutely stunned if anyone, ever, has described OpenAPI as "beloved". It does its job, but that's about the best you can say about it.

Yaml is beloved when it's maybe two pages of little indented config.

It becomes a nightmare once people try to emulate a DSL with it.

Reply on news.ycombinator.com