tedmiston

> Running it over my bookmarks

> Once I’d written the initial version of this script and put all the pieces together, I used it to create webarchives for 6000 or so bookmarks in my Pinboard account. It worked pretty well, and captured 85% of my bookmarks – the remaining 15% are broken due to link rot. I did a spot check of a few dozen archives that did get saved, and they all look good.

I was a tad confused by this part.

Did you (or how did you) verify that the headlessly saved web archives for thousands of bookmarks visually match the pages shown in the browser?

This is the biggest problem I've had with command-line archival tools: they save some version of the page, but it often differs substantially from what I actually see in my browser — things like pop-up artifacts covering the page or news articles are full of ads that are otherwise blocked in my headed browser.

The SingleFile extension for Chrome works more completely and accurately than anything else I've come across so far, but it does still break weirdly sometimes too.

I would love to find a programmatic way to automate the visual verification, e.g., archiving a page with multiple different tools and visually diffing the rendered pages across tools with small margins of error. Maybe someone else has worked on this already.

I tested this just now on the blog post that this HN page points to and SingleFile handled the background image fine.

It was also curious that BRK.A was affected, yet BRK.B was not.

The brokers will presumably reverse them as erroneous trades. ("No such thing as a free lunch")

https://www.investopedia.com/terms/e/erroneous-trade.asp

Update:

> In addition to the volatility halts, trades in Class A shares of Berkshire Hathaway Inc. appeared to go off at mistaken prices. About a dozen trades showed shares changed hands at $185.10 around 9:50 a.m., a discount of 99.97% to Friday’s closing price of $627,400. NuScale Power Corp. had a similar glitch, with trades that printed at about 99% below the prior price.

> “It’s very confusing that it’s happening in just a few shares,” said Jonathan Corpina, senior managing partner at Meridian Equity Partners, who typically works on the floor of the NYSE. “I would assume that those bad trades will be broken.”

https://finance.yahoo.com/news/volatility-halts-hit-numerous...

If you try to use serverless (the framework) for Python, you very quickly learn that its support is a second-class citizen vs building in TS/JS.

Adjacent concern but: It also makes for unholy large Docker images to combine an entire JS toolchain with an entire Python toolchain.

Zappa is a better (more native) choice.

Terraform modules provide a decent Python Lambda experience.

AWS also has multiple semi-overlapping projects in the space. Chalice, SAM, and Powertools specifically to name a few. CDK more generally as well.

Not a huge surprise. It filled a weird niche, which was very easy to use for basic use cases, but Chalice only supported a small subset of Lambda features and didn't get much attention from AWS either.

As soon as you tried to do slightly more than was possible with it, it become unviable to manage all of the Lambdas and you end up having to use Terraform or similar anyway. If I were doing lots of new Lambdas today, I'd probably just use a third-party Terraform module for all of it.

> At my friends coffee shop they make cold brew with an elaborate laboratory glassware setup that drips ice-water through a filter. Looks pretty neat and sciencey.

Likely the Yama cold brew tower [1].

[1]: https://prima-coffee.com/equipment/yama/yamcdm25sbk-yama-pp

Similar experience here, like: What about when your editor is set to auto save when unfocusing a tab or window and saves while you were mid-edit on your dotfiles and now every new shell is crashing? etc etc

I'm sorry, but running `rm -rf ...` in a script against $HOME is a bit too reckless for me.

Maybe you haven't lost data from this yet, but this is where Chezmoi has nice guardrails and protects against, e.g., modifications or additions accidentally being made to $HOME instead of the dotfiles dir, which look like they would be silently blown out by your current process.

Just my 2¢ from someone who used to do it this way and lost data because of it.

Fourthing Chezmoi. I tried so many dotfiles tools and it's the only one that does just enough, gets out of my way, and "just works".

Being able to run `chezmoi diff` and `chezmoi apply` are game changers.

    Location: Cincinnati, OH (Remote U.S.)
    Remote: Yes
    Willing to relocate: No
    Technologies: Generalist SWE but lately Backend, Data Engineering & DevOps.  Python, web apps & APIs (FastAPI / Starlette, Flask, Django / DRF, etc), workflows (Airflow) & automation, Cloud Engineering, AWS, GCP, Docker, Bash, Terraform, containers, infrastructure, CI/CD (especially GitHub Actions), architecture (software architecture, cloud architecture, data architecture).
    Résumé/CV: https://www.linkedin.com/in/tedmiston/
    Email: <in profile>

Summary:

- I'm a generalist software engineer (lately focused on backend and DevOps) with ~10 YoE in software engineering roles professionally (~20 years since I started programming) from frontend to backend to shells, sysadmin, cloud, DevOps, CI/CD, security, etc

- I'm coming back into the tech world off of a gap year focused on personal projects, and I'm excited to find something new that's technically interesting that's a great fit

- I'm in the top 3% all-time on Stack Overflow having helped over 8 million software developers [1]

Feel free to add me on LinkedIn — just include "HN" in the note or send me a message.

[1]: https://stackoverflow.com/users/149428/taylor-d-edmiston

I'm referring to the endless amounts of S3 buckets that ended up inadvertently exposing sensitive data publicly due to how complicated and confusing the various access control methods were / are. And also, Amazon very eventually doing something about, probably a decade after it was needed. [1][2][3]

[1]: https://cloudsecurityalliance.org/blog/2023/04/06/the-data-o...

[2]: https://news.ycombinator.com/item?id=18468753

[3]: https://news.ycombinator.com/item?id=34585670

> I am waiting for the moment when this assumption breaks because credit cards do not make users trustworthy.

IMO it is already broken, or beginning to crack at least, with the advent of throwaway virtual cards.

> Try to reduce the damage by activating Cost Anomaly Detection.

Yeah, this sounds like it could be a better solution than just using CloudWatch Billing alarms and billing alerts that I mentioned elsewhere. It's hard to tell at-a-glance if Cost Anomaly Detection is built on top of CloudWatch Billing alarms, or if they're just totally separate products. I don't seed the latter mentioned in the docs for the former.

[1]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori....

> I don't know how AWS meters, but, if your assumption is right, I can abuse this and generate a huge bill to anyone just by starting streams and not consuming then.

I suppose that would be easy enough to test.

I was curious how video streaming services handle the case where a stream is started or partially consumed. Mux meters it in terms of [seconds of] video segments delivered:

> Cost is per minute of video delivered. To calculate video delivered, we measure the number of seconds of video delivered to a video player. Note that if a segment of video is delivered, it is charged, even if the viewer doesn’t actually watch the video.

[1]: https://docs.mux.com/pricing/video#delivery

I think CloudWatch billing alarms [1] are the only viable "solution" for an average joe here. Of course, they do not actually stop the usage, but at least they bring automated awareness.

Even with that, there's no easy way to rename an S3 bucket [2], plus doing so would commonly require updating config + re-deploying any apps using the bucket too...

All of this to say, the customer level really feels like the wrong layer of abstraction to solve this problem.

[1]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...

[2]: https://stackoverflow.com/questions/41511034/how-to-rename-a...

To preface this comment, I'm not super familiar with range GETs or the library you're using. Are you saying that the library was performing range GET requests but instead of sending a byte range, it was only sending a starting byte number?

> Today I was hit by a surprise 500$ bill by using byte range request within zip files without an upper bound, but not consuming the whole stream. I tested for about 45 min at about 1gbps. My ISP meter says I downloaded 300gb which lines up with physics, but AWS says it was 6000gb.

Re the bill itself - You may be able to get the surprise bill waived by AWS with a support request, especially if you haven't had to do that before.

Re the metered billing - This isn't super clear anywhere I could find in the AWS docs, but I think from AWS's perspective your network data transfer out from S3 is the amount of data you requested + initiated, not what your app ultimately happened to consume from the stream. They really do not make this clear though, which might help your case.

FWIW, it sounds like AWS's implementation is behaving consistently about "range specifiers" with the RFC they link to in the S3 docs.

https://www.rfc-editor.org/rfc/rfc9110.html#section-14.1.1-4

Sidebar, but: If it works for your setup, I think putting a CloudFront distribution in front of the bucket and accessing that way would mitigate the damage by caching the whole object after the first request. I've used a setup like this for streaming very large files to many viewers before.

https://docs.aws.amazon.com/AmazonCloudFront/latest/Develope...

What's crazier is that turning on Requester Pays does not actually mean requester pays when the request is a 403 [1][2].

Essentially every S3 bucket, public or private, whose name is discovered can be DDoSed creating an insane bill...

That's a platform level security issue for AWS to resolve. We do not need another Bucketgate.

[1]: https://twitter.com/Lauramaywendel/status/178507498639629151...

[2]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/Reques...

> At this point, internet is like electricity. Except that I have more choices for power providers than I do for internet. In my building in New York City, I have one option (cable via Spectrum), and the cheapest plan they offer is $80 per month.

Why limit yourself to cable internet in 2024? Cellular home internet is widely available and fast.

https://www.cnet.com/home/internet/cellular-home-internet-ov...

https://www.usnews.com/360-reviews/services/internet-provide...

https://broadbandnow.com/5G/home-internet

Thanks for sharing this!

Direct link to the repo for those curious - https://github.com/simonw/llm

    Location: Cincinnati, OH (Remote U.S.)
    Remote: Yes
    Willing to relocate: No
    Technologies: Generalist SWE but lately Backend, Data Engineering & DevOps.  Python, web apps & APIs (FastAPI / Starlette, Flask, Django / DRF, etc), workflows (Airflow) & automation, Cloud Engineering, AWS, GCP, Docker, Bash, Terraform, containers, infrastructure, CI/CD (especially GitHub Actions), architecture (software architecture, cloud architecture, data architecture).
    Résumé/CV: https://www.linkedin.com/in/tedmiston/
    Email: tedmiston+hn@gmail.com

Summary:

- I'm a generalist software engineer (lately focused on backend and DevOps) with ~10 YoE in software engineering roles professionally (~20 years since I started programming) from frontend to backend to shells, sysadmin, cloud, DevOps, CI/CD, security, etc

- I'm coming back into the tech world off of a gap year focused on personal projects, and I'm excited to find something new that's technically interesting that's a great fit

- I'm in the top 3% all-time on Stack Overflow having helped over 8 million software developers [1]

Feel free to add me on LinkedIn — just include "HN" in the note or send me a message.

[1]: https://stackoverflow.com/users/149428/taylor-d-edmiston

Any recs for brands or specific products on 2-bay drive enclosures?

There are two main catches with Backblaze:

1. They will delete all backups of your external hard drive if you do not plug them in and fully sync them for at least ~12 hours every 30 days. You can pay an extra fee each month to have your external drives stored longer to mitigate this problem [1]. The fee is per GB so it does materially change the Backblaze pricing if you have a lot of external data.

2. Their client is extremely extremely slow, even with performance settings set to their recommendations it just obliterates CPU (irrespective of the size of the actual changeset being backed up). I've always been able to feel my computers slow way down when the Backblaze client spins up. [This issue may be resolved on Apple Silicon, but it's been a nuisance on every Intel Mac I've ever had.]

There are some other weird quirks in their macOS client, like you can set your backup schedule to "Once Per Day" overnight and it will still, multiple times per day, just start running unexpectedly and then you have to deal with the slowness or hit "Pause Backup". Somehow this happens 2–3 times every single day. I've tweaked the settings, reinstalled the client, and all of the things, but this issue has persisted for 5+ years on every machine I've used their software on. Super annoying.

Sidebar: They have a native mobile app but it is really really bad. Slow, clunky, somehow always getting logged out. The mobile client sucks so bad that I usually just wait until I'm at my desktop to do a restore.

[1]: https://www.backblaze.com/blog/backblaze-7-0-version-history....

The old Shipt membership via Target didn't include 2-day delivery without an order minimum. I also don't think having that membership made your shipping orders any faster (only same-day delivery orders), so it seems like an upgrade / expansion as well.

> If Target and Walmart want these subscriptions to be useful, they need to drop the minimum amounts for deliveries.

That appears to be exactly what they are doing here.

The free 2-day shipping under Circle 360 doesn't mention any order minimum in the fine print. They definitely could have made that more clear in the marketing material though.

That's a bummer that Walmart+ upcharges vs in-store.

If this new Circle 360 works like purchasing a Shipt membership via Target did previously, then for same-day delivery orders it's the same price as in-store or shipping.

3 things (IANAL):

I suspect this question is too nuanced to get much in terms of useful answers beyond speculation on HN. You will probably have better luck on r/legaladvice or r/AskALawyer.

That said, it looks like CO is an at-will state, but it may not be that cut and dry. Consulting with a lawyer could help you understand if any relevant protections like a Covenant of Good Faith might be applicable here. Unfortunately, it looks like he may not have FMLA eligibility since the position was so new.

https://www.paycor.com/resource-center/articles/employment-a...

He should be careful to do a legal review and not sign severance paperwork, if applicable, too quickly as it may entail clauses that forgo additional rights or compensation.

I know someone who went through a medical termination under a different scenario in CA, but similarly documented and well-known by the company etc, and they sued and were able to collect a settlement, but the whole process is still a PITA, especially given the present health scenario.

Yes, this is exactly why many baristas will choose a darker or more full-bodied coffee for an espresso type drink vs a lighter roast for something like a pour over.

Darker roast coffees are also much more forgiving in general, since they already have some bitter taste to them in the first place.

It's still a better end product to brew the coffee or espresso well in the first place vs try to mask faults (bitterness, sourness, etc) after the fact.

Preheating the water in a kettle mitigates the temperature concerns and variability mentioned in your first paragraph.

For the Moka pot itself, medium heat on a stove is a good place to start, but you can easily fine tune that based on what the flow looks like after one brew.

I've tapped to eject the spent puck for many years and it is not a problem. Maybe if you were aggressively whacking it against something much harder than the metal funnel it might become an issue, but against a hand or some soft plastic it's not an issue at all; there's been no change to the seal or shape.

The difference between quality grinders is super easy to replicate. Buy some freshly roasted beans from a specialty coffee roaster, then have the shop ground half and grind the other half at home on your home grinder of choice.

High-end (specialty coffee standard) commercial grinders produce a tighter variance amongst the ground coffee particles which leads to a way more even extraction, i.e. a more pleasant and balanced taste in the cup. Perhaps the most common model is the Mahlkonig EK43S, which is a ~$3,000 grinder.

Home grinders are far cheaper, in the range of ~$75–300 or so for popular ones, and tend to produce a broader variance of particle sizes which leads to big particles ("boulders") getting overextracted, thus imparting bitter flavors; and small particles ("fines") getting underextracted, thus imparting sour flavors. The higher quality and lighter roast your coffee is, the more noticeable the impact of the grinder is.

You don't have to take James Hoffmann's word for it at all, this is standard issue coffee grinder theory 101, which any specialty coffee professional would agree with. It's also easily testable at home, even if you haven't experiencend it in an experiment context. I took a short class on the theory of coffee once and the major experiments were comparing the same beans ground across different grinders (and then with or without use of a coffee sieve for further refinement) and the same beans across different brewing methods and techniques. The particle size distribution from the grinder is one of the most important variables in the whole process.