So, you want to deploy on the edge?
zknill.io
zknill.io
Edge implies putting your stuff in a CDN-sized datacenter that has only a subset of the full regional services, may not be able to scale up significantly on demand, may be more expensive, have less local storage and failover redundancy, etc. The multi-region considerations come in here too, but there's a whole extra set of things to worry about.
Basically you rarely want to deploy your whole app to edge; have a central or multi-region thing under the hood, and let the edge handle some very specific functionality. And even then, only if a few ms of latency makes a big difference to your clients (and they can deal with the lack of failover redundancy).
But the term seems to get used for everything and anything apparently.
Your home or office with a bunch of computers can be classified as a data center. It might not be very redundant or efficient, but still.
If you complain that the definition of “data center” is inexact, you’re missing the point—the definition of “edge” is inexact, too. That’s okay. People have a good enough definition of “data center” that it doesn’t need to be explained here. And people also understand that given definitions are not perfectly precise—there are exceptions and unusual cases.
Only in the way that a plucked chicken is a man. (http://www.artandpopularculture.com/Featherless_biped)
But some people would say if it's in any kind of data center, it's not true edge.
Depends a bit on wether traffic from Singapore would be allowed to go to NY if the Singapore host goes down.
> The delivery of computing capabilities to the logical extremes of a network in order to improve the performance, operating cost and reliability of applications and services. By shortening the distance between devices and the cloud resources that serve them, and also reducing network hops, edge computing mitigates the latency and bandwidth constraints of today’s Internet, ushering in new classes of applications. In practical terms, this means distributing new resources and software stacks along the path between today’s centralized data centers and the increasingly large number of devices in the field, concentrated, in particular, but not exclusively, in close proximity to the last mile network, on both the infrastructure and device sides.
Ps. I work at https://edgenode.com
But to people selling "Edge AI" products, an edge device is something like a smart speaker or industrial camera.
At least that is my current IIoT definition. You could have bunch of pretty dump sensors and then somewhat beefy these days gateway. And the gateway here could do some computation.
Cloudflare and Akamai both have the concept of edge workers - light weight application APIs that can serve modest capacity. Cloudflare also has a cache on the edge.
https://blog.cloudflare.com/introducing-cloudflare-workers/ https://developers.cloudflare.com/workers/learning/how-the-c...
Exactly. Example: We wanted / needed to deploy on the edge. Use case: A very very small part of a live streaming service, basically cache preloading functionality, as well as authentication / session management / DRM. All the rest would be handled by some upstream service.
Imagine this: within a CDN’s compute you try to serve distributed db transactions optimistically then fall back to your big cloud region when it fails. The CDN is kind of a write-through cache. You still have to address the CAP theorem distributed state challenges of synchronizing state and making a consistency/availability/partition tolerance tradeoff, and it seems like a reasonable use of a CDN.
Hardware becomes the limiting factor when you’re talking edge versus multi region where hardware is one of many limiting factors, but often network is the more important factor as indicated in this link
Edge also has the distinction of being hardest to instrument for observability and often is required to work in multiple environments.
This is precisely the point. For most CRUD applications, writes are usually so sporadic and user-intentioned that you can show a spinning wheel for it, and users will forgive you for that slowness because it's so sporadic.
Edge computing is basically reinventing insecure HTTP caches and mirrors, but allowing you to include authn/authz so that user data stays private to them. If your edge data model is that much more complicated than that, you're probably doing it wrong.
> How are authn and authz different? To put it simply, authn has to do with identity, or who someone is, while authz has to do with permissions, or what someone is allowed to do.
I guess it's awkward that those two related but different concepts share the same first 4 letters. But I think authc would have been clearer for disambiguation than authn.
And we kind of do, in other (but closely related) contexts; e.g., the common cloud systems for managing both are “identity and access management systems" not “authentication and authorization management systems”.
(Though, yes, “access" sometimes means something else; nothing is perfect.)
I can say "auth service" and it's well understood, even to the user, that service identifies them and determines their permissions.
I now agree with you that "autho" would have been better for the same reason. Although "authz" sounds cooler :)
<https://www.etymonline.com/word/authority>
<https://www.etymonline.com/word/author>
So an author is one who creates, but also a "source of authoritative information or opinion".
Confusingly, authentic seems to have a different etymology, *autos "self" (see auto-) + hentes* "doer, being".
<https://www.etymonline.com/word/authentic>
So to authenticate an author as an authority derives three meanings from two separate roots.
Huh?
There are a huge number of applications that require writes at the edge.
This access pattern in no way indicates that the application is "doing it wrong".
• Remote Region Datacenter — It's certainly not this
• Local Region Datacenter — It's not this
• Local Municipality/County Datacenter/Co-lo/Central Office — Could it be this?
• Local mini-datacenters attached to mobile towers or fiber digital loop carriers? — Could it also be this?
• On-prem Datacenter — It's not this either
A lot of people argue that an 'edge' has to be in the field — like a local mobile tower or a digital loop carrier. I'd argue that it could also be at a nearby co-lo facility or CO. Basically anything ≤300 km should be ≤1 ms latency. Within 75-100 km you're talking 0.25-0.33 ms latencies. And for most applications these days that seems "real-time" enough. YMMV.
This is longstanding use and to pollute a useful technical term just leads to unnecessary confusion.
By “a lot” I mean they have a lot of devices dining so frequently; I imagine the workload is very carefully managed for battery/heat reasons.
I have recently spent a fair bit of time experimenting with this on Fly for my application (https://www.ssrfproxy.com). It's hard to beat the straightforwardness of deploying in a single region, with the database in close proximity. This approach probably to meets the needs of what 99% of developers require. Aka Heroku.
I think it's also useful to consider that tools like LiteFS aren't just for your main database. We have several internal tools at Fly.io that fan out their data to other internal apps to consume so they can share data without having to build an API or worry about rate limiting.
The point is, by the time you have solved all the problems with consultancy and sync to an edge node, you are most of the way to local first.
Isn't that the "client"?
The article favours dealing with latency during writes while making some assumptions, - most internet apps are read-heavy (reasonable, but what if it's not the case for you?) - you do not need sharding, all regions have all the data (what if you have a ton of data that is growing quickly? what if you have durations where some data is heavily contended for?) - the primary replica used for writes is highly available (how do you automate handling of the primary replica failure or primary replica getting partitioned?)
Another approach to consider would be Amazon Dynamo style databases (Cassandra, Riak etc.) which can help when the above assumptions are not met, - Dynamo was designed to be always-writable and suitable for heavy writes and reads as well (e.g. the paper mentions teams using it for catalog data with R=1 and W=N) - data is partitioned/sharded using a variant of consistent hashing, which helps with large data volumes and limiting the impact of hot spots as well - failure detection and leader election help automate handling the partitions for which a failed node was holding a primary replica
By doughnut rendering, I mean you can displace certain static placeholders with user-level content... such as replacing the "login" section of the header with the markup for the User's name and profile image.
These are both common use cases for Cloudflare Workers. Not just for complex compute but for near-line cache and simple transforms.
[1] https://fly.io/docs/litefs/proxy/ [2] https://docs.turso.tech/reference/data-consistency#on-the-re...
EDIT: https://fly.io/blog/globally-distributed-postgres/ talks about how this can work. The approach it suggests for implementing this is to try running all database queries locally; if there's an error from the local database being a read-only replica, add the fly-replay header pointing to the primary region and return an HTTP 409, and Fly's proxy will rerun the request in the specified region. There's also some commentary on handling consistency issues to at least get read-your-own-writes.
What we especially like about multi-reader single-writer clusters is that you can mostly keep all the pieces in your head at the same time. If you understand Postgres or SQLite, and can get your head around the `fly-replay` header (which is just "replay this request over there please"), you've got the whole system.
"rqlite supports adding read-only nodes. You can use this feature to add read scalability to the cluster if you need a high volume of reads, or want to distribute copies of the data nearer to clients – but don’t want those nodes counted towards the quorum. These types of nodes are also known as non-voting nodes."
You can put one of the secondaries on the other side of the world.
By default, all reads are sent to the primary. But you can easily specify "nearest" as your "read preference" on a query-by-query basis which would allow the app to read from the nreplica with the lowest latency (whether it be the primary or one of the secondaries).
The ability to specify read preference at the query level lets you have 2 categories of queries, 1) queries that must always be served with most recent writes, sent to primary at the expense , and 2) queries that are allowed to occasionally send stale data but benefit from latency improvements by reading from the nearest node.
This is something I want to play around with using Cloudflare. I wonder if there’s enough of a guarantee for a user to “stick” to a node that you could use Workers KV as an eventually consistent write through cache for user unique data and a Durable Object as a strongly consistent write through cache.
I feel like that would give a pretty solid balance, wouldn’t it?
These numbers seem high. This site shows transatlantic RTT of around 75ms: https://wondernetwork.com/pings
If a CDN or edge computing platform reduces that to 20ms then the difference is 55ms. And it matters only for read requests that cannot be cached locally.
Whether or not its worth it depends largely on the number of requests. Perhaps reducing that number should be prioritised.
The notion that you need to replicate your infra in 10 regions in order to provide the best user experience sounds like propaganda pushed by cloud companies to make you use more product than you need.
The one thing to keep in mind here is that the software architecture considerations of going to 1 to 2 regions cover about 95% of what it takes to go from 2 to 10. To a first approx., the hard part is being plural, not how plural you are.
That's why I think the article is incorrect when it claims that running an app in US East "adds an extra 100-150ms of latency". Not all of this is "extra". Only about 55ms are.
As a hobbyist doing recreational programming, I don't want to pay for a website when it gets no traffic. Everything needs to start up on demand and shut down when idle.
This means that cold start time is what really matters. Since Deno Deploy has no persistent storage yet, both the frontend and the database need to start up on first request. Having a multi-region database doesn't help if needs to be running all the time and doesn't start up very fast.
As for latency when warmed up, I plan to deal with that by avoiding round trips with stored procedures and caching.
Here's my "hello world" website. Any advice on technologies to check out would be welcome.
laughs in Rural New Zealand
But seriously really enjoyed this article. I was always a bit confused how the recent influx of edge services handled writes/consistency, and yeah they all seem to have a single leader that routes writes to them - the edge is all about fast reading.
At the time I advocated for an “always connected” model but we were concerned that cellular connections wouldn’t be that reliable so we thought disconnected operation would be necessary.
A few year back I was thinking about the future of “low code” and thought an open-source project similar to Lotus Notes but JSON-centric would be a good thing
https://en.wikipedia.org/wiki/HCL_Domino
In particular, Lotus notes has a model for disconnected operation that works quite well. My take is that “progressive web apps” haven’t gotten that far because they don’t have enough batteries included, a really good bidirectional sync model would make a difference.
For years you could say ideas from Lotus Notes didn’t make it into the open source world because they were patented but the patents have long since expired and the main reason they are obscure is ignorance.
In the end, it gets complicated to support offline web-apps and really depends on your usage. I had to work on a water dispensing system over a decade ago that needed to be able to work "offline" at a given site... I used firebird embedded locally, and a remotely available server for syncing against. All accounts and balances were synced to each site, so that they could operate in offline for dispensing water to carrier trucks. There were rules on offline term (up to a week) and balance carry to reduce the risk of a negative balance. In the end it worked well, but it wasn't at all simple.
Active/Inactive and syncing transactions was at least interesting, and that was for a relatively simple data model. More complex models would be, of course, more complicated... and almost exponentially so.
Added benefit is that it clarifies how to do local compliance. With a global model, the complexity of local laws can really overwhelming. For example a global site based in the US needs to be GDPR-compliant, while a global site based in the EU has to figure out how to file multiple forms of taxes (federal, state, sales, city, district) for the 16000+ US tax jurisdictions. As a European I am more afraid of US taxes than GDPR.
Really? I've never really been to the US, other than a couple of brief work trips, and that is astonishing.
Also there are Sales Tax holidays where the rate of sales tax is reduced for a set period: https://www.salestaxinstitute.com/resources/sales-tax-holida...
Also certain customers are exempt from Sales Tax...
There's no federal sales tax, but if you're selling internationally, there can be border fees of various kinds. State sales taxes are nearly ubiquitous. Large cities or counties might have a sales-tax addition for local funding.
Special-purpose sales taxes exist, infrequently: hotels are probably most common.
Basically: implementing a distributed global db without structuring the topology to minimize hops/making engineering trade offs can’t be fast. It’s all about deciding which part of the CAP theorem you want to relax.
If you do though, you should take a look at https://edgenode.com, which I helped build (we host server-less containers on edge)
I think either the Cloudflare Workers team or the Fly folks have talked about how they envision edge apps as sharding data by region or by tenant, then sticking each shard in the optimal location. That sounds potentially useful for some cases, but then there's the complexity of sharding to deal with.
We're a hosting provider, not really a database provider. Systems that do sharding, Raft consistency, and stuff like that are built on top of systems like ours, not so much inside of them. :)
Actual data sharding, without persistence in different regions is more difficult. Different database systems lend themselves differently for different loads and sharding characteristics... I've seen MS-SQL in master-master across the globe, postgresql with sharding that is replicated to base host, and various ScyllaDB/Cassandra setups. It's always a mixed bag.
The database is using the read in the log as the point-in-time where that read happened. The writes before that point will be viewed by the read, and the writes after won't be.
By sending the read to the leader, when we receive that read back, we know for sure that all the writes that should have been replicated to our local copy have been replicated (because our read has made the full round-trip that all our writes would have, so we're guaranteed to have received all the writes before that read's "point-in-time").
(disclaimer: I was the tech lead on this feature)
EDIT: I think I'm realizing people will disagree with me because I have a different perspective. For my use cases, my data comes from sensors on the edge, and so for me, I want my edge computing to be as close to those sensors as possible.
It is more helpful to think of it in terms of a traditional client-server architecture, where you want to move the server as close to the client as possible. This covers 95% of what people mean when they say edge compute.
I remember these distributed db scenarios being describe when reading "Designing Data Intensive Applications" and this was a really helpful supplementary piece to jog that memory.
Cheers