8organicbits

This is a really good time to do a head-to-head comparison of different search engines. The last time I checked Google does not provide search results above the fold, instead showing AI overview, ads, and YouTube search results. Try a different search engine to see which one is giving you the highest signal results and what other junk they put on the page. Personally I use duckduckgo with AI turned off in settings, but it's best to try the experiment for yourself.

One concern about accuracy of the diagrams. In the example, there is a transition back to the session service labelled "wait for release" after the "no" decision. I'm not seeing that in the shown diff.

Looking at the code the "no" seems to relate to the context expiring, so you wouldn't want to wait more if the context already expired, you'd want to stop. Is there a reason that label exists?

I'm pretty wary of LLM development tools hallucinating and wasting my time, is that whats happening in the lease broker example?

The Kanopy model is fascinating. Library members get free views but there are limits as the library has to pay for those views. Our county library system has a Kanopy subscription at each library and county residents are eligible to become members at every location. Kanopy supports adding multiple library cards, so you can boost your limit.

The platform is quite open, so you can build whichever tests you want. Personally I run a dockerized ooni via scheduled GitHub actions, which shows blocks impacting GitHub runners. You should be able to see logs from that here, which shows three blocked websites https://github.com/robalexdev/ooni-unattended-action/actions...

You can build your own lists and share them using ooni run: https://run.ooni.org/

Many of the partners are country focused but the Tor Project, Citizen Lab, and Internet Society are well known to me.

For some of that, you don't need to crawl. Wikipedia offers database dumps which you can download in one go. Lots of programming docs are managed in repos, so you can clone the repo instead. Even stackoverflow seems to have a snapshot dump (https://archive.org/details/stackexchange).

How did I miss zstd?

Here are my benchmarks for 2.3 GB of jsonl, on a laptop. Compressed size, compress time, decompress time; using defaults.

    gzip  7.3%  21s  9s
    bzip2 4.6% 251s 50s
    bzip3 3.3%  82s 69s
    zstd  6.9%   2s  3s
    lzma  4.7%  51s  3s

I was processing compressed .jsonl files recently (JSON lines format). I found that lzma gave a much better compression than gzip or bzip2, which helps for archival costs, but it's challenging to work with as software support is lacking. I do duckdb processing which supports gzip transparently. There's an extension for bzip2, but not for lzma or bzip3.

I ended up using gzip because it's best supported by the software I use and most likely to have support in software I adopt. But it gave the worst compression results of the options I tried. These bzip3 numbers certainly give me FOMO...

That's a bummer.

It looks like they are open to adding the feature and open to outside contributions: https://github.com/desec-io/desec-stack/issues/579

> simply permanently end all service to the concept of subdomains at all

That doesn't sound simple at all.

I think DMARC works well because email tends to blindly trust DNS (opportunistic encryption). On the web we expect authenticated TLS, often strictly enforced (organization policy, HSTS). So it would feel weird if a website changes how it handles HTTPS cookies based on an insecure DNS record, perhaps delivered by the resolver on an untrustworthy WiFi router.

Specifically, if I register subdomain attack.co.uk and set up a malicious WiFi router, I trick some *.co.uk cookies to get set on co.uk and then steal them from attack.co.uk by tampering with the (proposed) SVCB record.

I think the signal needs to be secure, which means DNSSEC. Adding a hard requirement for DNSSEC validation in all web browsers is a huge change from where we are now.

There are limits to how accurate you can make this. Zones like .xyz renew at 18%, so you'd expect 0.2% of those domains to expire without renewal each day. The tranco list is based on a 30 day look-back and I've seen a small percent of those domains lack name servers, even for the latest list.

Similarly, domains can be registered since the last time you downloaded your lists. So you never have a complete or accurate list.

I think it's perfectly reasonable to suggest names that have recently expired or domains that could have been recently registered. The alternative requires an NS lookup.

Official docs cover those:

https://docs.docker.com/engine/network/firewall-iptables/

https://docs.docker.com/engine/install/linux-postinstall/

> Here's a stream of standard.site blog posts coming in over a firehose hosted in Chennai. Every single one.

This stream lacks any blog posts that haven't been specifically published to atproto. Conversely, I see RSS feeds for the content in the stream, like https://bsky.app/profile/did:plc:byf7jvh3yvhffackiumpddtf/rs... (if RSS feeds exists for every profiles then I'd argue that atproto is a strict subset of RSS, but I'm not certain how that feature works). The stream may give you every blog post that was published to atproto, but that's already a small subset of long-form content on the web.

> Notably this is not possible with RSS,

The web supports blog discovery so well that people forget it exists. A simple Google search can find a very large amount of content, much available over RSS/Atom. Many web-based feed readers index the subscriptions across all their users to help with discovery. Here's the Feedland firehose, for example: https://feedland.com/?everything=true

I don't think its helpful to argue how complete the Google index is vs a bluesky firehose though. Most users are drowning in content and discovery is about search and filtering. There's lots of interest right now in vouching for the content of others: https://susam.net/wander/wander.js, https://codeberg.org/robida/human.json, https://www.manton.org/2024/03/11/recommendations-and-blogro..., etc.

I wouldn't say the others lack that feature, they do it differently.

https://docs.joinmastodon.org/user/moving/#move

> more ease of exploration

Absolutely. One good thing going for this approach is that anyone can grab the OPML export (the URL is stable) and build their own frontend, I'd love to see more.

Could be a fun weekend project for frontend folks.

For blog posts, I'd look to RSS instead. That's where that content is traditionally published. Instead of SELECTing from bluesky's index, you can use OPML subscription lists. There are a bunch of places that curate feed lists, so it's significantly less likely to face API death like twitter did.

There are multiple sites that support follower semantics over RSS. Feedland tracks subscriptions publicly, so you can see the blogs I read (https://feedland.com/?username=robalexdev), and who reads my blog (https://feedland.com/?feedurl=https%3A%2F%2Falexsci.com%2Fbl...).

I run another variant which collects OPML blogrolls via crawling, so you can find out who else likes your favorite blog and what else they recommend. Here's the page for Simon Willison's blog (https://blogroll-network.alexsci.com/discover/feed-a34ee2a88...). Thinking of RSS and blogrolls as a network feels much more resilient than blueskys Jetstream api endpoint.

I'll recommend dnscontrol as well, although the post is an unseasonal April fools joke, so helpful suggestions may be out of place.

The is-a.dev project uses dnscontrol to manage a subdomain registration service in GitHub, which is really clever. See https://github.com/is-a-dev/register

Idk, Google seems worse there too. I tried "my left hip is hurting".

Google shows an AI overview and "people also ask" with zero search results above the fold. If I page down I see a single search result for clevelandclinic.org, followed by youtube videos and image search results. The next page has a single search result from rush.edu and then "discussions and forums" which has Mayo Clinic and Quora.

DDG also starts with the AI overview (although I disable that) and has two results from webmd.com with deep links to multiple pages on the site, all above the fold. Then the same clevelandclinic.org result as Google but again, adding deep links to other related pages.

I can't comment on the quality of webmd, clevelandclinic, or rush, but Google pushing the user to youtube and quora for medical advise seems worrying.

Interesting, I've seen much better results on DDG. Most recently was the search: `site:feeds.bbci.co.uk inurl:rss.xml` which works on DDG but gives zero results on Google. As far as I can tell, Google just decided not to index these.

The link shows the first six months of 2026 for "all items" as: 2.4%, 2.4%, 3.3%, 3.8%, 4.2%, 3.5%. Gasoline ranged from -7.5% to 40.5% over the same range, which I've definitely noticed.

I use RSS specifically because the people who publish via RSS are not trying to monetize their content.

I don't think this is helpful. The last statistic I saw put RSS at 77% of the web feed market share, higher than Atom. In the web feed market it isn't a protocol on the way out, its the dominant protocol.

While RSS is tricky to parse in practice, many of the challenges are due to invalid feeds, which Atom is not invulnerable to. Its very likely that there is a well-tested feed parser for your favorite programming language, so most developers using feed programmatically don't need to handle those challenges themselves.

For concerns like title encoding or summary vs full text description, you can handle those using namespaces. If you don't like any of the existing namespaces that do that you can create your own.

What do you like about JSON Feed?

One of the downsides I saw when evaluating JSON Feed was that it doesn't use GUIDs, which can cause problems when the same post is present in multiple feeds (I.E. syndication).

When I last looked I found that every feed reader supporting JSON feeds also supported RSS/Atom and every website with a JSON feed also had an RSS or Atom feed, so it seems easiest to ignore the JSON Feed format.

It's great to see someone else using FeedLand. While it has some quirks, is has some clever ideas, like that blogroll feature.

My favorite part of FeedLand is that you can see who subscribes to a blog and where else they subscribe. This makes RSS feel social, without the algorithmic tricks of traditional social media. Here's the listing for my blog, for example: https://feedland.com/?feedurl=https%3A%2F%2Falexsci.com%2Fbl...

> It’s Hovers problem what the price behind the door is.

This feels key to me. If .sexy is changing fees with very short notice, and Hover doesn't have enough time to inform their customers before renewal, then Hover should stop offering .sexy domain names. None of that is a problem for the customer.

The linked article "What I learned from one year of Substack" [1] perfectly sums up my experience. I can't read the article because I only see a coercive demand to subscribe first. Perhaps that's the joke.

[1] https://elizabethtai.com/2023/10/22/what-i-learned-from-one-...

When I checked in 2024 (https://alexsci.com/blog/is-email-confidential-in-transit-ye...), Cloudflare had more domains using DANE than Microsoft. But you're right, DANE is not widely adopted. Lack of support by Google is most notable due to the large number of domains using their service.

Typically you don't want to require authenticated encryption for all outgoing email as many mail servers use certificates that are self-signed or otherwise appear invalid. It isn't as simple as it seems and isn't generally recommended.

What concerns do you have with MTA-STS?

Email currently uses an HTTP request to https://mta-sts.<domain>/.well-known/mta-sts.txt, per RFC 8461. Depending on HTTPS/TLS instead of DNSSEC is one major reason you see this approach gaining popularity.