Hacker News Reader

Comment by eandre | Hacker News Reader

I've been working on Encore for the past 3 years.

I was at Spotify for the past 8 years as a Staff Software Engineer. We grew frustrated with the disproportionally large effort needed to build even simple backend applications, and created Encore to solve this problem.

Encore uses static analysis and code generation to reduce the boilerplate you have to write, resulting in an extremely productive developer experience. It's a combination of a framework that combines with a cloud platform in order to simplify all aspects of building distributed systems, from how you write code to shipping it to production.

I shared some more background on Encore here [1]. Feedback very much appreciated :)

[1] https://twitter.com/_eandre/status/1381668476662734849

View on HN

I've built my fair share of Go microservices and have never felt the need to develop an entire framework like this.

We have everything like CI/CD, distributed tracing and one line project generation.

It's just based on some foundation code and some simple bash scripts to find and replace some service specific things.

And then it's just the basic routing and HTTP handlers to unmarshal and marshal responses.

When all your services are built off the same foundation then creating a new one is pretty much and copy and paste job in my experience.

It's a cool project but it seems like unnecessary complexity, something I strive to avoid.

If that works for you, all the better! There are quite a few things that Encore does beyond what you mentioned:

- Automatically sets up dedicated Preview Environments for your Pull Requests

- Provisions all the infrastructure you need, both for local development as well as in production (and other environments like staging or testing)

- Provides auto-completion and compile-time type safety for making cross-service API calls, with a single line of code

- Integrates secrets management, API documentation, frontend client generation, etc :)

What is your setup for distributed tracing?

Personally, I am using Opentelemetry to create and export my distributed traces. At first sight it looks like they have reinvented the wheel for this project.

Elastic APM. As far as I am aware it's an implementation of the OpenTracing specification.

We just use the Elastic client libraries. Tell it where our APM server is, do the little bit of application set up and then view all our application traces in Kibana.

All applications follow the same basic setup. Probably less than 20 lines of code and then wrapping some HTTP handlers and HTTP clients. Then we have some application specific spans.

Any plans to switch to Opentelemetry?

Not that I know of

isnt code generation just smarter copypasta? since there will be no difference in complexity during runtime i feel like it's equivalent, just maybe more powerful

what i mean is that you have complexity in your existing setup too... its just that you understand the ins and outs. and so the same applies to encore.

I don't think it is. It's fundamentally taking the code out of your control. With my services all the code is there, we can tweak it if necessary, we're not tied to a framework, it's just a barebones foundation of a Go application.

Nothing is hidden away from you with code generation. There's no in-between steps from what you write to what you execute other than Go compilation.

I think it's the wrong mentality to require a whole new set of third-party tools to maintain a Go application. It then starts feeling like you're writing Javascript, and does anyone really want that?

No it isn't.

If you did down enough, you'll find some form of code generation in your environment of choice. If nothing else, compilers, but it's not limited to them.

The largest difference between that and copypasta is - generated code can be easily updated, by regenerating it.

It's also an implementation detail. I've read elsewhere in this thread that code generation is bad and language supported features are good. I'm not sure it's so clear cut. It shouldn't matter if the generated code is mostly invisible (when it's baked inside your binary by the compiler, due to language support), versus code that is generated in the source language and then compiled.

One thing that's always bad is mixing generated and non-generated code. You have all the disadvantages and little benefit.

Curious - how do you end up handling transactions that span services?

I'm getting into go, but there doesn't seem to be a great pattern for this. Used to @Transactional on the service, so feels annoying in having to build that.

How does @Transactional span services? With a JtaTransaction manager?

If you are saying about micro services here which talk grpc/http then @Transactional does not help.

If you are talking about inside one service then you have write all that boilerplates, and it will not look as simple as Java

I think the answers requires more context, but for example one well known pattern is called “sagas” for implementing long lived distributed transactions.

The gist is you timeout but provide a way for transactions to resume.

Also obviously there’s 2PC but again it depends on what unite trying to solve.

> it seems like unnecessary complexity, something I strive to avoid.

This. Adding complexity to reduce boilerplate seems like the wrong direction to go in (pun intended).

I like boilerplate. I can overwrite it with special cases whenever I need to, and it takes up very little mental space (as opposed to the rules needed to generate it).

I hated working in Rails because of all the magic. I love working in Go because of the lack of magic.

But y'know. I'm sure it'll suit someone. Good luck with it OP.

Hi André. Encore looks really cool! It must have taken a huge amount of work to put this together. I think streamlining from IDE to production is the future of tooling and frameworks, and it looks like you are going in that direction. I feel like that is the entire hype behind "serverless", but no one has really been able to pull it off yet imo.

I wonder what you think about K8s integration? Spotify is all K8s right? I'm guessing that is tougher to monetize. I tried at a prior startup to work on something like this, offering a PaaS like experience on K8s with development libraries (it's actually still in progress - check out https://cloudstate.io/). It is a large effort to build such a thing in a nice way.

Anyways, I'm looking forward to seeing how Encore develops. Good luck!

Hey thanks a lot for sharing this, Could you recommend a couple of resources for someone to get into distributed systems? I read Data Intensive Applications, and have been finding such sources since.

A lot of the seminal research papers are surprisingly readable! I really recommend reading them to get a good feel for the sort of problems you face at scale and good ways of handling them.

Off the top of my head: a lot of stuff from Google (Dapper, Stubby, Zanzibar). Dynamo from Amazon. Leslie Lamport's guide on TLA+. The two generals problem, distributed consensus, and so on.

Most distributed systems don't need all that fancy stuff, but it's useful to know regardless. Even the simplest systems will get weird and interesting bottlenecks with scale.

And then a lot of just doing it in practice! Find a job at a startup with some scale, and learn on the job.

> We grew frustrated with the disproportionally large effort needed to build even simple backend applications, and created Encore to solve this problem.

I'm going to sound naive but where did running Docker daemon on a VPS and orchestrating deployment with `docker-compose.yml` or `Terraform` fall short?

This seems very similar to https://goa.design/? Did you know it exists?

Did you look at whether Go was part of the problem or were you using something else and you found that Go was part of the solution?

Reply on news.ycombinator.com