matt_oriordan

Author here, and I'll declare my bias so I'm upfront. I am co-founder of Ably, a realtime infrastructure company, so I wanted Jev to be a low-latency story.

Two weeks ago I built Jev Pong and thought it was.

Then I tried to find a use our customers would ship and couldn't, so I sorted the whole first week of Jev posts instead of guessing. The data is on a separate page linked from the piece (see https://mattheworiordan.github.io/jev-landscape/ and https://github.com/mattheworiordan/jev-landscape/, which scanned ~6k posts and looked at the data from two public gateways and twenty rapidly emerging rivals).

The short version. Adoption is very real (a quarter of Vercel's AI Gateway requests, 2% of its tokens), but the 100x claim is against frontier models. In practie, Jev is more like 7x and 5x against small models you'd have likely used.

So my conclusion is Jev is a great model, unblocks things you could not do before when you're in builder mode, but when you move into production and definitely do things at scale, you're unlikely to be on Jev. We have solutions already for the things Jev does, just with some friction.

Hi HN, Matt here, co-founder of Ably. I built this after Jev landed on Vercel's AI Gateway: https://github.com/ably-labs/jev-pong

It's pong where the ball moves one step per model decision.

Four lanes, one model each, same serve, same rules.

Every step the model gets the same tiny JSON state (ball, paddle, predicted intercept) and the same question: up, down or stay. One answer moves the ball one segment; eight segments cross the court. No speed up / adjustments for real play, so a lane's speed is that model's decision latency.

Numbers, recorded on Vercel (iad1) next to the gateway, 45 seconds per lane: Jev averaged 227 ms (p95 400 ms). Gemini 3.2 s, Haiku 2.5 s, GPT-5.6 Sol 3.5 s.

In the first 12 seconds: 47 decisions vs 3, 2 and 2.

Bit of fun! Enjoy

Nice. I have spoken to literally 30+ developers who have hit these exact problems. I would have though, I am the co-founder of Ably and we were doing product validation

Oh, I'll take a look :/

An illustrated WebSocket history by the author of WebSocket.org

Ably’s LiveSync (https://ably.com/livesync) is conceptually designed to solve exactly this problem by providing a database connector that subscribes to changes committed using the outbox pattern, and a models SDK to subscribe to those changes in your app. It’s early days as it’s just been released, but at the core the team’s goal in designing that was to solve this exact problem. Interested if this is meets your needs? Disclaimer. I am the co-founder of Ably.

Wow, that didn't last long! This can only be an acquihire given Sunil worked at Cloudflare before.

Quite frankly whilst I have no love for Meta, if they are willing to spend stupendous amounts of money to open source their AI models, yet prevent use of this by companies who reach 700 million MAUs from benefiting from this investment, then they’re doing a service to the community and ultimately finding a way to commercialise their investment. Tldr; they’ve invested a lot of money and time to help the community benefit from AI. If you then go onto build a billion dollar business servicing near one a billion users, then they expect a thank you in currency. How is that not reasonable?

I am one of the co-founders of https://ably.com. We have built a Serverless websockets platform that is near infinitely scalable and designed to handle the complexity you’re describing such as cluster resizing, connection failures, automatic sharding and reallocations using hash rings across our global edge service etc. I’d be really interested to hear why you wouldn’t consider offloading this to a service like ours which powers the likes of HubSpot, Expedia, Spotify etc. Feel free to contact me directly @mattheworiordan if you’d prefer not to comment here. I’m really keen to hear different perspectives on what we’re doing right and where we can improve!

Been using this site for years, it's a far better way of consuming HN IMHO

Hello everyone, this is a new post from the Ably Engineering series. This time, we talk about a very important topic in cloud computing: autoscaling.

Hope you like it and let me know if you have any questions!

Awesome article. The author has absolutely nailed the issues one faces with distributed systems under stress.

I understand the SOC2 requirement, which is why we're SOC2 compliant. A long and painful process, but one that has opened up opportunities like the one you said would otherwise require an on-prem installation. We have in fact also got some VPN-like links (AWS PrivateLink) with customers who are particularly sensitive to security and data leaving their network, and that coupled with SOC2 has made it possible.

> If there was a less scalable, but still mostly functioning open-source substitute for Ably, then I'd be way more comfortable using Ably in the main public cloud deployment.

Thanks for the feedback, I will pass on to the product team!

Thanks for the explanation, useful to know.

Feel free to sign up for an Ably account and we'll help support your project with a community package, we love what you're doing!

Our customers use Ably across such a diverse set of use cases, but here's a few:

1. Sports events streaming live updates (see https://ausopen.com/live-scores, we are streaming live scores for the Tennis Australian Open right now). Companies like Toyota even use us to facilitate engineers tweaking the performance characteristics of their cars in realtime remotely.

2. Edtech - we have numerous customers using us to drive live classroom environments, think shared white boards, collaborative tests, presence, teacher engagement. You may have used Codewars in the past, that uses Ably under the hood for example, https://www.codewars.com/.

3. Live screen sharing and collaborative applications. You may have used the amazing Tuple.app, that uses Ably under the hood https://tuple.app/.

4. Collaborative and live web and mobile SaaS applications, where changes need to occur concurrently, notifications need to be presented, and other realtime updates are needed in the interface. You've probably heard of Hubspot, they use Ably under the hood to power their collaborative and live features, https://hubspot.com

5. Developer infrastructure and platforms that need realtime capabilities at scale. You have probably come across Split, the leading feature flag company backed by Atlassian and Microsoft. They use Ably under the hood to power billions of feature flag updates in realtime each month. https://www.split.io/

6. Financial market data - typically streaming updates to thousands or millions of subscribers with very low latency, and sometimes using features like Deltas (https://ably.com/documentation/realtime/channels/channel-par...) to keep bandwidth & latency as low as possible.

I could keep going, but I hope that gives you the idea, realtime is not just for Christmas or for Games :)

Matt, co-founder of ably.com

That's surprising to hear, we definitely don't see anywhere near the order of 5-10%, at best on order of magnitude less.

Out of interest, what geography and industries are you operating in where you see such a high rate of incompatibiltiy?

Matt, Ably co-founder

Yup, true, although tokens should be ephemeral so less of a risk. Authenticating inline over the Websocket connection is valid too, but it does expose the socket connections to slightly more surface area of attack i.e. if you pass in a token as a param, the Websocket request can be rejected immediately. If however you authenticate after establishing a Websocket connection, then there is an attack vector where you simply open Websocket connections and never authenticate. Of course timeouts can be used to disconnect rogue actors, but it is a consideration.

Matt, Ably co-founder

Yup, and it supports Ably too, see https://js.m-ld.org/#ably-remotes!

Matt, Ably co-founder

We are monitoring it closely and super excited about what WebTransport will provide, which is both a more reliable and in many cases more performant transport. However, much like WebSockets, it's still quite low level and only provides a basic communication protocol. As such, like we have done with SSE, HTTP, Websockets and MQTT, our service focusses on what developers can enable on top of these lower level transports, such as presence, deltas, history of streams, limitless scale and fan-out of data, and the list goes on https://ably.com/platform.

When WebTransport reaches prime time, I'm confident we'll be supporting it.

I know, we're very excited about WebTransport and what it can offer. As you say, on the surface it seems to provide both a more performant and reliable transport for realtime communications. Once it hits mainstream, we'll be certainly adding it as another transport we support in our stack (currently websockets, HTTP, SSE, MQTT etc)

Matt, co-founder of Ably

Thanks ddoolin. I am not sure which company you are, but I appreciate your comments and even more happy to hear you feel we're doing a fantastic job! You've sort of encapsulated why we exist: we know software engineers can build and run all sorts of things on their own i.e. we're not the only option from open source solutions to custom builds. However, we built Ably to be the easiest, most scalable, and reliable realtime option which for developers that removes complexity and helps them manage costs as they grow (pay for what you use).

Matt, co-founder at Ably

I am surprised to hear about on-prem deploys when the direction of travel is towards serverless/cloud solutions that don't require orchestration. Out of interest, why is on-prem so important for your use case? Separately, I'd be interested in how you'd view that decision if there was a less scalable but mostly fully functioning open source version of the third-party service you could deploy if you needed to?

Matt, co-founder of Ably

Founder of Ably here That’s odd and clearly a rip off of our post :( Thanks for flagging

I spent too much time reading HN as opposed to running my business :)

Tbh what’s worked best is when I’ve been focussed. Doing many things at once has mostly been the root cause of businesses failing.

AWS are reporting that they have an issue with an external provider outside of their network, which may be impacting Internet connectivity between some customer networks and the AP-SOUTHEAST-2 Region.

We can confirm that this is indeed having significant impact globally with intermittent connections from regions around the world.

AWS state that "Connectivity to instances and services within the Region is not impacted by the event" which is somewhat amusing in that they're saying so long as you don't need Internet access, everything is fine.

We've had to route all traffic away from the Australian region for edge messaging service, see https://status.ably.io/incidents/718

Amusingly as I read this my tinnitus is hugely amplified and unescapedly present. I have also feared this onset last year may have been caused by my AirPods. Sadly I am unwilling to draw that conclusion based on hundreds of people reporting this issue. If 1,000 people reported getting heartburn after buying AirPods, we’d laugh at the stupidity of this ridiculous correlation. I want my tinnitus to go and I want to blame Apple. I won’t thought ugh unless a clinical study is carried out.

We have had this same issue. We are migrating from our old ably.io domain to our new domain ably.com. We've been contacting Facebook through public channels, and private channels (our network) for around 7 months, and we're hitting a brick wall. Facebook is touting how they are doing what they can to validate what people are publishing on their platform, yet they incorrectly classify our domain ably.com as "the content doesn't meet our Community Standards" yet won't reclassify, respond, or engage in any way to change that (see https://developers.facebook.com/tools/debug/?q=ably.com). Like Alex from JitBit, Facebook truly is a tiny channel for us, but we have found it has a wider impact for us it stops people sharing links with the domain in it in other Facebook owned media too.

If anyone knows how on earth we can actually get Facebook to respond, I'd be hugely in your debt. This issue has blocked us for 6 months in migrating over our primary traffic from www.ably.io to ably.com.

Contact me at matt at ably.com

Ah, OK, glad I could help :)

Out of interest then, what is the reasoning behind the warning about additional latency. Is Google Pub/Sub doing some kind of reconstruction on the receiving end to piece together an unordered stream, or has Google Pub/Sub changed how the underlying streaming primitives work to natively support ordering.

This looks like a naive implementation of adding ordering keys on one side, and waiting until all messages arrive on the other in order, without improving the underlying delivery transport to support ordering.

From what I can tell, the publisher is responsible for now maintaining arbitrary ordering IDs and passing this in the publish method, see https://github.com/googleapis/nodejs-pubsub/blob/master/samp....

Given the documentation now clearly states "Publishing messages with ordering keys might increase latency", I think that what Google is doing is effectively doing is putting lipstick on their service that does not support ordering and instead offering a message reconstruction capability on the subscriber end.

This approach, whilst it may work a lot of the time, is pretty flawed given one failed published message could cause the entire stream to stall indefinitely. And given Google Pub/Sub does not support idempotency, and does not support exactly-once delivery, they recommend "In general, accommodating more-than-once delivery requires your subscriber to be idempotent when processing messages."

Google, I am afraid whilst it's nice to welcome you to the exactly-once semantic party, I think you may need to go back to the drawing board and bake ordering and idempotency into the transport layer, in the same way Kafka and Ably do so that true ordering is supported.

Disclaimed: I am the co-founder of ably.com, a far edge enterprise messaging solution, with exactly-once semantics and real ordering :)