soamv

There is no clear definition with universal agreement. It's a hype-y term applied with... varying levels of rigor.

However, roughly speaking, "serverless" rolls together 3 features:

1. Fine grained pay-per-use (e.g. pay for a query by the number of rows scanned)

2. The pricing dial goes down to zero when usage is small enough.

3. You generally don't control VM/instance-level scaling but something closer to the abstraction level of the product being claimed as "serverless". For example in planetscale you get no control over how many mysql instances actually run your queries. This is great for reducing operational complexity but not so great for controlling performance. Performance tends to be quite opaque -- for example there's nothing I can find in Planetscale's docs about latency and throughput. The operational benefits are real, though. It's a tradeoff.

Hi there HN, This is a prototype of something that lets you build AWS Step Functions in regular Python code.

I think step functions are very neat, but have a suboptimal programming interface (essentially an assembly language in JSON). However, the idea of having a fully managed workflow system with serverless-style billing is really exciting, IMO. So I've tried to bridge this by building a tool that transforms a Python function to an AWS Step Function plus a set of Lambdas.

This is a prototype, it's far from complete. If there's enough interest, I'll open source this after cleaning it up a bit.

It's true that they're conservative, but there have been non-trivial forks in its history. If a fork proves itself by doing something significant that the original can't or won't, things can get interesting! I wouldn't write this one off so quickly.

I'm so perplexed that someone would feel the need to attack an open source developer for a gentle suggestion of an optional donation.

You're free to pretend there's a constrained license, and pay the project for their work. Are you saying you'd rather be forced to do so by a license? That makes no sense -- you can do it without being forced, if you want to.

Maybe VMware? There was a paper about how, in some cases, emulation was faster than hardware virtualization support. This was way back when intel's hardware virtualization support was new and VMware had already spent years optimizing software virtualization.

Because people use it. It's been around a while and has an ecosystem around it. (This is the interesting thing about AWS, they keep the focus on what people do use, rather than some opinionated idea of what they should use.)

I wonder how it compares to astronomer.io, and Google's managed airflow thing.

Not _just_ placebo. If you tend to favor very closed positions, or end up placing pawns in ways that make your bishops useless, then the knight might be better sometimes, even though the general idea of bishop > knight is correct.

And yet, snowflake

Wait how is github actions a BYO-agent system? I thought you can only run actions on github's infra?

They have a conference going on (see the banner at the top of nextjs.org), so they're launching a bunch of new stuff.

This is a fantastic comment, thank you for it.

> Kafka publishers habit of lying to their clients

I have no Kafka expertise but it feels like there ought to be a Kafka configuration option somewhere that tells it not to do that? (And in fact changing the batch size to 1 won't help if it continues ack'ing messages without syncing to disk)

Yes, the throughput results are pretty great, as is this work in general. It's worth noting the memory usage, though -- "Go’s garbage collector needs a factor of 2 to 3 of heap headroom to run efficiently (see section 8.6)".

The comment you're replying to does not describe a process list.

Users do understand the concept of apps, obviously. A log of "App X used the camera", "App Y accessed your location" is well within what non-tech-savvy smartphone users can understand.

There's a really interesting hour-long interview (more of a monologue really, heh) with Kramnik about no-castling chess: https://www.youtube.com/watch?v=RPe6xQjO98Q

Thanks for your answers.

> 2

I don't mean literally generating hspec from yaml -- I mean more like operational "source map" support. For example if I look at a pod taking up more CPU than I expect in a dashboard somewhere, I should be able to trace it back to the hspec file.

Congrats on this launch, it's great to see serious efforts at Kubernetes simplification and I hope you succeed! K8s bills itself as a "platform for platforms", and such projects are a real test of that idea.

Questions:

1. How do you deal with the mutability of K8s resources? Do you assume people won't change the underlying resources that you generate, or do you keep k8s controllers running to ensure there's no deviation?

2. How understandable is your generated output to humans? Do users have a way to go "backwards" through your abstraction? (Since your tool lives in an ecosystem with many other kubernetes tools, your users will sometimes end up having to deal with the generated output, since other tools operate at the K8s level such as prometheus, log aggregators etc)

3. Do you interoperate with K8s-level resources well? Do _all_ my services need to be in this abstraction for this to work well? e.g. Can my hyscale yaml reference a non-hyscale service for any reason? Or are they essentially two separate worlds?

I don't think people understand the vastness of resources that FB and Instagram have at their disposal. This is not a particularly hard problem. Yes it's not a simple SQL query. But they do solve it -- in fact this was a bug that they have already fixed.

This article dives much deeper than I expected. It sounds like a fantastic resource for a manager, especially in a startup, but a lot of it also relevant for tech leads.

direct link to paper (pdf): https://www.usenix.org/system/files/atc20-maruf.pdf

Right, you package up your dependencies. That doesn't mean you don't have any!

Anyway, what I'm getting at is, the platform has to load a bunch of code, sometimes that will take longer than the latency that you can hide behind an ssl handshake; plus the code might run something on init.

But yes, workers seem to be oriented towards relatively small bits of code, so maybe the vast majority of load times are a few ms.

Preloading functions is a good strategy, but claims of "eliminating cold starts" and "zero overhead" are a bit hyperbolic.

I don't know enough about cloudflare workers, but function cold start latencies are not fully in control of the platform. You can hide the latency of scheduling and loading, but the function might load a ton of dependencies; and it might have its own custom init code that you generally have little control over. So you can hide an SSL handshake's worth of latency, and that's a nice little win, but whether that makes the user-visible cold start latency zero depends on what it was in the first place.

But I'm guessing cloudflare workers are used for relatively simple stuff, so they don't often have dependencies or complex init code.

(And sorry, I don't mean to sound too negative, this stuff is pretty cool!)

AWS is not a great place for small projects that you want to set up quickly. There are a lot of options with a much more lightweight setup experience: zeit/vercel, netlify, digitalocean, firebase.

> Most of the people writing OSS do it for fun/side project/to solve a problem, not for the money.

Not to pick on you personally, but this sort of assertion is often thrown around with no data.

It's not even clear to me that it's one or the other. For me personally it was always a mixture of both -- there's a lot of fun but I wouldn't do it without the money. But that's kinda true for my non-OSS engineering work, too!

> pure engineers

I see what you're saying, but I think there is an element of "making a product" that goes into almost any new OSS project; there is an attraction of exploring the product space from a fresh start that is part of the motivation of many many new open source projects.

(Also I think the notion that a "pure engineer" doesn't care about all those things is just wrong, but that's a whole other conversation.)

Turns out to be a somewhat annoying problem at the VM level. Not impossible but complex enough that maybe higher-level solutions like functions are better.

Consider memory usage -- but operating systems (and some applications) are designed to grab all the memory they can, and use it for caching etc. So it's hard for the VM host to known when it can grab memory and stop billing the user for it.

But there is this idea called memory ballooning -- you have a little process running on the VM guest OS that grabs lots of memory, but is actually in cahoots with the host, and just tells the host -- "hey I got all this memory, you can take it back and use it somewhere else".

Okay, so doesn't ballooning solve the problem? There are a few problems with it -- you can't balloon when you need the memory, because it's not fast enough. So you have to balloon pro-actively. And you don't know how much to balloon, so you have to guess: do it wrong and the guest OS will start swapping, or it might activate its OOM killer and start killing processes.

So making memory usage follow the application is kinda sorta possible but comes with hairy problems. What about CPU? CPUs usage already follows the application so you could just measure and bill accordingly -- except that nothing is gained if memory doesn't also follow usage.

All in all it's way simpler to get towards this goal with clearly-defined higher level services like Lambda.

I'm so glad this realization is the top comment on HN.

I think Fizzbuzz can be actually a reverse test: if an experienced programmer can fail fizzbuzz in an interview, then maybe it's the interviewer who screwed up.

Direct link to the paper: http://chrisparnin.me/pdf/stress_FSE_20.pdf

Why doesn't the same idea of adding a type keyword work there too?

a, b := w<type x,y>(z)

No, sea levels have risen as predicted; see for example https://blogs.egu.eu/divisions/gd/2017/09/13/modern-day-sea-...

> consistently low latency

That's (a) just one objective and (b) too vaguely stated. What's an acceptable p95 latency for a FaaS? For a database write? For a message queue? The answer is a very big "it depends". And why are we only talking about latency?

There are tradeoffs literally everywhere you look in this space, and not knowing enough about the application's performance, reliability, and efficiency goals can sometimes be a real hindrance to being good at running that application.