dmw_ng

I'd imagine the priority for the service provider is new product lines built around wifi sensing rather than the marketing about bandwidth. It is /really/ hard to come up with new applications that require so much bandwidth constrained to a one room, especially given we aren't even remotely tapped out on compression fanciness for visual media in mass deployment (see e.g. the recent obscenely low bit rate videoconferencing codec work from Nvidia)

My API key serves at least 5 people including myself, pretty sure that's a common setup

17070 (option 1), I think it still exists on BT at least

I doubt any amount of staffing could address lack of specialism in dealing with power or air conditioning issues, both likely involve infrequent maintenance by external vendors. 20 people blowing up the phone to a vendor doesn't fix a problem any faster than 1 etc.

I'd imagine somewhere well under 50, probably with plenty of CPUs

Assuming 256 bytes per query (very high estimate) that's about 3851 MB/s or about 3.5 10 Gbit ports

Packet count obviously matters, but a KV cache lookup followed by a short reply in the usual case is one of the easiest loads to engineer for

I was recently shocked to discover media attachments sent on Signal are uploaded to either Google Cloud Storage or some other service sitting behind CloudFlare. The recipient device(s) fetch the uploaded keys to access the images. The net effect is that there is almost certainly a log file somewhere that correlates the IP addresses/user agents of conversation participants for a very large subset of all Signal users

The point is mostly there are plenty of security issues with existing systems that probably aren't easily fixed with another layer of crypto woowoo, and it makes me uncomfortable that crypto is used to justify marketing these systems as secure. How do you explain to a user that the JPEG compression implementation on their particular phone with their particular photograph has a unique on-the-wire transfer size that may already be enough to correlate them with their recipient? etc

Have never understood the desire to hoard commit rights, half the point of RCS is rollback, and as the article says, once intent is established through effort ("proof of work") there is little risk in assuming good faith until demonstrated otherwise.

Commit rights are an asymmetrical reward for someone showing interest in a piece of code - cheap to offer and incredibly valuable to receive. They also immediately reduce the long term burden and "ownership" of the code, and for the trusted committer, massively reduce friction and any sense of gatekeeping in contributing to the project. I've done this on at least 4 projects, 3 of them were eventually transferred to the new long term maintainer and 1 became an org of its own, with two maintainers having full admin

You can also pass special commands in the second argument to SQLite:

sqlite3 db .dump | ...

Is there any documentation showing what kind of performance is given up by shoving a userspace server into the SQLite read path? LiteFS looks cool but I'd be worried about all block IO basically becoming a cross-process RPC negating large chunks of SQLite's efficiency. Instinct is screaming this should be a VFS extension for SQLite, but definitely appreciate the idea of doing it as a virtual filesystem.

In terms of pure perversion, I'm wondering if fanotify or kernel tracepoints could be used to gather the information needed asynchronously rather than sticking a userspace server in the way

Amazing work :) Do you know if glibc gmtime_r() might be impacted by the active timezone? My "massively expensive" experience above came from gmtime_r() on Amazon Linux running in Lambda, but I can't remember the exact scenario involved (or e.g. if TZ were set), it certainly seems unlikely worth putting the effort into it that I did if the costs were similar to those observed in your post.

Neatly omits the massively expensive calendar conversion step for turning the timestamp into a useful format for fast arithmetic. Parsing integers from strings is not why parsing timestamps is slow, any more than counting the length of strings is why XML parsing might be slow.

For time-sequential data, one easy speedup hack is caching the UNIX epoch for the current day start and adding the HH:MM:SS manually. No measurements to offer, but would suggest approaches like this would cleanly beat simdification of a trivial part of a complex task

never underestimate the ingenuity of a good QA person. "app freezes while triple-clicking About button while changing wifi network when storage is 89% full and screen reader is enabled"

it's the same with good security folk. sure you can pretend you'll catch 100% of issues, but it's a delusion, good security or quality testing is a totally different mode of thought

Video capture is something that's "just worked" for me for as long as I can remember (unlike audio). What's the value gained from the added latency of an intermediary? Is it only to ease containerization?

Cost to build the system conspicuously absent

This project https://www.gas.scigrid.de/ used a similar approach to map gas pipelines

It doesn't seem like an overly difficult problem: try to find a text fragment (e.g. DOM mutation observer) requesting consent in some container that has some other element with an onclick handler (maybe optionally trying to figure out if image filenames / labels of that element look sensible). Maybe an approach like this will only work 70% of the time, but that seems likely to be a better strategy than embedding a giant list of every relevant site on the Internet

Random aside: this extension had absolutely the worst internals of any I've ever looked at. Love the functionality, but really wish I'd never seen the spaghetti behind the illusion (source files below). It feels like approaching it as a text classification problem might produce a clean general solution

https://github.com/OhMyGuus/I-Still-Dont-Care-About-Cookies/...

https://github.com/OhMyGuus/I-Still-Dont-Care-About-Cookies/...

What value is added by swapping Docker out for an alternative on ECS? It's (at least as I understand it) basically an implementation detail, and well beneath the "covered by AWS support, don't touch" surface

What if the assets are a list of URLs computed incrementally? In my app's case, the specific JS bundle variant to load is itself a result of a database query. It's easy to see many scenarios where the majority of assets (e.g. say photos) aren't known prior to building the response body

Early hints solves a different problem, there you have a list of asset URLs you know your rendered page will need. This is about producing pieces of rendered page as computation completes on the backend.

Implementing this is a pain in the ass, any new intermediary (haproxy, nginx, ...) introduces a potential new source of buffering or IO loop that attempts to helpfully batch up your writes. But when the technique is implemented, it is legitimately an amazing way to reduce latency at the client.

I use this technique in an app where an opening chunk is used to ensure the browser has begun fetching/executing the app JS bundle before a list response is rendered. After the list is rendered (incrementally as each chunk becomes available) it is again used to execute a slow summary query without having to break out a separate request. All that fits in a single HTTP response body, no additional latency is introduced by making separate API requests for the list or for the counts. Because of the initial chunk causing the JS to execute, it is also possible for the app to display certain dialogs (depending on query string parameters) that are immediately ready for user input even before the subsequent responses have finished rendering or downloading. In the case of this specific app, the technique means some dialogs are usable 300ms earlier than otherwise

Another issue not touched on in the post is how all this newly unbuffered incrementally-arriving data interacts with visual rendering on the client, there is potential for a lot of flicker and burned CPU continually re-rendering UI elements.

It sounds like Airbnb are disabling compression to make their implementation work. So long as CPU isn't a problem, you can also implement compression in the backend, and flush the compressor each time some useful unit of work is produced (some task completes, or some amount of data e.g. aligned to the size of an Ethernet MTU is available for writing). Assuming all the stars align, this should produce compressed output in a form the browser can act on every time it manages to read any amount of data from the backend.

All this effort seems like nonsense until finding you have a perfectly functional app on a crappy airport wifi network where it was otherwise impossible to even get Google to load

viewBox can be comma-separated, so you can loose the quotes but only for inline SVG

Record tracking pings independently of any solution and figure things out later, a 1x1 GIF can absorb an unlimited number of query strings, and S3 storage for log files is extremely economical. It doesn't matter which analytics solution you end up with, you'll always be throwing away data. IIRC even GA enterprise doesn't allowing pulling the original click log back out of it, only some form of sampling or another. Put a 1x1 GIF in an S3 bucket, create a CloudFront distribution and enable logging for it into another bucket. Tada, unlimited scale achieved for pennies.

Matomo and GA's trackers both allow sending the query to a URL of your choice. Treat the analytics system as an index over the logs, rather than treating the logs as an auxiliary artefact of the analytics system

The smoking gun is probably the box that was previously labelled "Media Conversion Service" (Elemental MediaConvert - easily 5-6 figures/mo. for a small amount of snappy on-demand capacity, or crippled slow-as-molasses reserved queues) now labelled "Media Converter" running on ECS. For example, vt1 instances are <$200/mo. spot and each instance packs enough transcode to power a small galaxy, for fine-grained tuning an equivalent CPU-only transcode solution isn't that much more expensive either.

At some point the industry will wake up to the fact the AWS pricing pages are the real API docs, meanwhile dumb shit like this will keep happening over and over again, and AWS absolutely are not to blame for it, any more than e.g. a vendor of cabling is guilty of burning down the house of someone who plugged 10 electric heaters into a chain of double-gang power extension cords

A while back I tried to reproduce a Vercel enterprise invoice using logs generated by an external service. After multiple round-trips with their support team I concluded it is actually impossible to verify what was billed even remotely reflected actual usage. It's essentially an AWS reseller in the form of a random number generator attached to some horrific contemporary JS, that the brand is so popular speaks more to the state of the industry than it does the suitability of their product

The same tech works in both directions. Spammer creates 1000 email variants using a LLM, spam filter collapses those 1000 variants back into easily classifiable embeddings

Thankfully none of these efforts ever really go places, because they'd create a massive amount of churn for minimal net gain. Spam is primarily a social problem not a technical one

The effort to standardize chargers in the EU began at least as early as 2009, 2 years after the original iPhone release and not long after peak Nokia.

Currently sitting on 250gb of RAWs since October, at some point that will hopefully become 40gb or less. The last time this happened I batch converted everything to Adobe DNG, but support for that isn't great outside Lightroom. Is there some format with good compatibility somewhere between DNG/RAW and JPG that preserves dynamic range and white balance information? I think exposure and WB correction is pretty much the only functionality worth fighting to preserve. Keeping RAWs for long term personal archival seems silly

Big fan of ClickHouse, but in the case of S3 there doesn't seem to be much point fighting city hall. I was the author of the original clickhouse-local-in-Lambda ticket FWIW. Even if that approach worked well, it'd still only be <50% cheaper than Athena's fractions-of-a-cent for queries on efficiently partitioned data, and still there is setup work involved. I think CH-local-in-Lambda might still easily compete on cost with Athena for raw CSV queries though

If someone were following the setup steps to execute a query in a serious setting (as opposed to reviewing them just for the sake of writing a sarcastic forum comment), they might find CREATE TABLE .. MSCK REPAIR TABLE quickly becomes finger memory, and the only real usability complaint about Athena is that its interactive query editor only allows 10 tabs to be open at a time