8,438 karma · joined January 1, 2016
But I wanted to add on the future benefits front - as more vehicles on the road become autonomous, they could communicate with each other and reduce the buffer distances between them, allowing for faster travel and improved saturation while still being safe.
It sounds like an autonomous taxi, but not necessarily. A taxi usually has only one rider, whereas a bus would have more seats, and potentially pick up and drop off other riders that are more-or-less on the way. This is Via's Microtransit[1] model, but without the cost of a human driver, it becomes much more cost-effective to deploy at large scale.
Are you arguing that a lesser evil (compared to the evil of offshoring) is not evil? Is offshoring supposed to be relevant here? Why bring this up?
Original sin is a sunk cost. If you want to make the world better, you focus on what's right for tomorrow, not what was right for a century ago.
Copyright needs to be reformed, absolutely. Not abolished.
The difference with accounting is that, relatively speaking and certainly within this context, few businesses are cash businesses. Your bank is keeping at least a basic audit log of money coming in and out of the corporate bank account. Your payment processor is keeping at least a basic audit log of who paid you and how much. You won't make your auditors happy if they're the only documents you have, but they're at least something to be handed over in an audit that pretty much every software business will have. Cybersecurity? By default, nothing is collected.
sighs heavily in 90's sysadmin
Testing backups is not just a question of whether or not the restore command works. Go back and read the Tao of Backup: http://www.taobackup.com/history.html . The application itself (in its current version, with its current features) needs to work with the backed-up data, and the only way to verify this is to attempt to actually work with the data.
If you don't trust your agent to ship to production without manually reviewing the output (in some way), you have no business trusting your agent managing your backups. The agent writing some tests doesn't mean that the tests adequately handle all of your actual scenarios, let alone that your system will adequately handle data that is missing since the last backup.
Why not? Because even pre-AI, very few companies spend time practicing restoring their backups, or disaster recovery, or picking infrequently-used runbooks to practice, or seeing whether they can easily rotate secrets without downtime, or trying to redploy the system onto another vendor's cloud/platform, or, or, or... It is the least-sexy operations work that exists. No executive cares about this. Ops organizations push for flashy work, same as everybody else: new infrastructure for new projects, cool chatbots, new flashy dashboards, make charts go up and to the right, etc.
Airline pilots go through disaster simulation training because the government mandates that training. If it wasn't a condition of holding a pilot's license, no company would pay for it.
Want SREs to spend time training for disasters? Take a step back. Support professional licensure. Make it a condition of holding a license. You won't get industry-wide professional behavior until you professionalize the work. It won't happen without licensing because every corner cut that is not immediately visible to consumers translates to additional profit, and increasing competition eventually requires these corners to be cut in order to keep up with competition and stay in business. Forcing all players to submit to licensing requires all players to pay these costs and thus forbids them from cutting them to become more competitive.
The problem with the stock market is when there aren't enough new companies going public to soak up the spigot of all the capital that people want to invest, leading average P/E ratios to increase over time. There is too much money chasing too few earnings. When the underlying fundamentals don't match up, then all that investors are doing is speculating that someone else will be the bigger fool and buy from them at an even more unreasonable price - and that is a gamble.
A lot of this feels like a requiem for the days when the only people on the Internet were "high-computer-skill" type folks. Most people will gravitate to "user-friendly" solutions: Gmail and other managed email providers were popular because they didn't stop working when you shut down your computer to save electricity, when your server's hard drive crashed, when you upgraded your computer to something with a faster processor, more RAM, and a newer operating system. It was hard enough to educate laypeople about URLs and email addresses (AOL keywords, anyone?), let alone a combination of random numbers in an IP address, or convincing people to register domain names.
Yes, NAT shoved fences into a network that was all about connecting everybody. But we'd still end up with server-client cloud architectures, even if we had started with IPv6 in the beginning. ISPs would have just sold highly restrictive firewalls as part of their home-install basic boxes, and we'd still have ended up with those fences.
Most software engineers are relatively well-paid compared to the wider labor market, even the engineers who are "underpaid". Kahneman's research showed that once you have enough money to pay your bills, making more than that has rapidly diminishing returns for your happiness. It's much more important to fit in - like your boss, like your coworkers, like your work, be recognized, have good work-life balance. None of that really requires above-market compensation.
When a firm pays below-market rates, it's a sign of one of two things: either they're abusing people who can't find work elsewhere, or people are happy to stay in spite of it. The latter are often really good places to work.
HN loves to complain about how money ruined the software industry. Let the people who will chase a raise from $180k to $210k go elsewhere. Let the people who want their moonshot at life-changing wealth go talk to a VC. There are other opportunities for people who realize that money isn't everything in life.
SQLite is great for small-production scale. Don't let anyone tell you different. Just keep in mind that it's a pain to move if you ever do need to scale up. If it's a startup, you might. If it's a weekend hobby project you're building for your personal use only, you probably won't.
Paying above-market rates is walking a tightrope. On the one hand, experienced employees who are well-versed in your systems and your organization are indeed worth more than market-rate (i.e. someone new), and compensating them as such will retain them. On the other hand, it also retains poor performers, who you want to steer to finding roles elsewhere. Being ruthless about firing fast is one option, but it's a deal with the devil - it erodes psychological safety among people who stay unless the firing is unanimously desired and there is a consensus among everyone who remains that it was necessary. So if you handle the firing wrong, you affect performance and social cohesion everywhere. If you pay market-rate, it's easier to just make someone miserable until they self-select out and find work elsewhere.
Unfortunately (or fortunately?), all successful startups pay above-market because equity compensation in the right company can be a life-changing amount of money. So most successful startups seem to successfully walk that tightrope.
How many paid auth providers let you export user password hashes so that you can seamlessly migrate to another vendor, if you want to?
The whole problem with auth is that both (a) login screens are shown to unauthenticated users, which is a superset that includes attackers, who will do everything from DDoS to crafted malicious input to try to grab user secrets, so you really want to pick something that is already running at large production scale and with all the production battle-scars, and (b) that need to go with a managed vendor is very much in tension against local development, vendor independence, data portability, and other Good Engineering Practices (TM).
Sure, AWS Cognito sucks. In many ways, the product feels stuck. Making compromises to get stuff shipped, working, and stable sucks. But honestly, unless you're going to prefer (b) over (a) (and there are times to do so, in particular with intranet applications behind a firewall that aren't really susceptble to those kinds of attacks) and pick something like Keycloak, you could do a lot worse than Cognito (shudder, Okta, shudder).
1. You're selling a system to a customer to use within their own AWS account, that you will have no access to, and it needs a transactional datastore (not just an object bucket) of some kind. The fact that it costs nothing by default (particularly valuable when the customer is trying to deploy a proof-of-concept), scales more-or-less perfectly without anybody touching it, requires zero day-to-day maintenance by you or the customer, and all it will ever ask is that you throw money at it, is very, very much a feature. One example I'm familiar with in the wild is Teleport: https://goteleport.com/docs/reference/deployment/backends/#d...
2. You have a huge OLTP workload that fits Dynamo's KV patterns (e.g. Amazon.com shopping carts, which is what it was originally built for). You don't care how much DynamoDB costs (in either dollars or engineering limitations) because any alternative would melt your face off if you even tried.
Most of the pain that comes from Dynamo is people who try to use it as a primary datastore in place of a relational database just to get the serverless pricing model. It's not worth giving up the flexibility on greenfield systems. It does become worth it to give up the flexibility when your system is mature and you don't have genuine flexibility anymore anyway.
OP's entire argument rests on this presumption, and while it certainly sounds like the industry is headed in this direction, it's definitely way too early to equate the success of building proofs of concept with success at maintaining mission-critical production systems across industry verticals, as OP attempts to:
> The prototype is working software. And the improvement and testing of that prototype is further enabled by more improvement loops with the AI. It gets better with more testing and verification, not through human code review, but through usage and testing.
Who drives usage and testing today? Who takes user feedback from the "usage and testing" and translate it into something that The Machine can use for improvements? Humans do. There is no agentic harness for managing at the level of the product itself, and I'm not convinced that there ever will be, because it's a fundamentally political concern. And not the low-stakes intra-team kind like tabs vs. spaces - the high-stakes, do-we-close-the-deal-or-not kind. Even if agents hypothetically could handle that level of stakes - they simply lack the context to do so, and will continue to lack the context to do so, at least until we get AGI in a humanoid robotic form factor.
Software engineering isn't dead. As a separate field with a dedicated job title, it's arguably dying in a world where it becomes a table-stakes skillset for Product roles. It is simply cheaper to employ 2x Product Engineers at $300k/year each, armed with $200k/year each in tokens, than it is to staff out a team of eight Software Engineers at $150k/year each. And this is before a hypothetical crash in API token pricing, or agility benefits from aligning fewer humans.
It's happening slowly in smaller companies, and hasn't happened yet at scale because, while you can teach Product skills to most Software Engineers and can't teach Software skills to most Product folk, most big-cap executives haven't gotten this memo yet. But the economic pressures are there.
Please take the following as expressed with genuine curiosity: Do you not use an editor with syntax highlighting and collapsible comments?
At least on JetBrains you can configure the editor to collapse all comments on open and to have the comments displayed in a low-contrast color. This way, LLMs add a bunch of comments, but it doesn't affect your actual experience in trying to read the code. If you encounter code that seems inexplicable, then and only then would you expand the comment to see if that helps you understand.
In Zig, you create a buffered writer for the stdout file handle, write hello world to the buffer, and then flush it. It exposes what's actually happening underneath.
This has not been my experience; Claude does a great job of writing unit tests for the Zig code, and combined with Zig's fuzzing features and a Python-based integration testing suite, I have avoided hitting runtime UB so far. In exchange, I get much faster compile times, which are important for the agentic development loop.
I actually agree, but this is a luxury that most large companies cannot politically prioritize (it is not Product/Sales-driven, see earlier comment). Especially when the company is large, and there are dozens if not hundreds of developer teams in a polyglot microservice environment, pushing application-level handling of these concerns is virtually impossible without executive support, and because it doesn't move the bottom line in an easily measurable way, you won't get executive support.
Companies much prefer infrastructure-based solutions to these problems, even if they're coarser, because the relatively small number of people who need to be involved makes it politically feasible. Easy example off the top of my head - mutual TLS encrypting east-west traffic has been implementable at the application layer for decades, but it was a pipe dream until service meshes made it easy to deploy (it's still a pipe dream for many orgs that refuse to schedule any infra work not Product/Sales-driven though).
> Application backends?
I don't think I'm taking crazy pills to suggest that it's preferable for services like rendering PR diffs, MR merge trains, even accepting new Git commit pushes, to be temporarily unavailable, so that the entire web application doesn't fall over, and cache-friendly read-only workloads continue to succeed.
> basically just one for system and majority of serving workload ran on another priority and the rest was for batch
RCA blames in-house load-balancing services (HAProxy) that reached capacity limits. Even if autoscaling is not working correctly because it didn't take Istio into account - why does it take more than seven hours to just raise the minimum on the Autoscaler for HAProxy and let the workload scheduler evict workloads that are less important than, say, their auth gateway?
Why not? Because sadly, fundamentally, most workplaces are not run by people who care about day-2 operations or long-term health. Product or Sales pushes customer-visible work into the pipeline, and you dare not say no. "Day-2" work is not considered to be something that moves the needle. Even now, with GitHub facing these severe outages, it's not like they're facing some massive exodus; their load seems to be getting worse over time, not better.
I'd be very surprised if there weren't any employees at GitHub who had read the SRE book. I'd expect that they're just not listened to.
> Reading isn't an activity where you can use more than a small part of the screen at once.
So reading through War and Peace on your smartwatch screen before you fall asleep is a good idea? /sarcastic
Sure, your eyes are focused on a small part of the screen, but what ends up in your peripheral vision still helps you keep your place. You feel more "grounded", somehow.
Large clusters with dozens of services and traces that go several services deep, with each service owned by a different team, are a whole 'nother ballgame, especially when overall production uptime is owned by an SRE team and not by the developer teams who wrote each of those services. And even in this scenario, you're not necessarily wrong; the risk attached to the cascading failure is domain-specific and may be acceptable.
This is the wrong prism through which to confront the problem. Technology is neither good nor evil nor neutral - it is a tool lacking agency, to be manipulated by people.
You might as well ask if it's possible to design a gun that can only be used in self-defense, or a knife that can only be used for cutting vegetables and cannot hurt people, or a car that will solve all traffic fatalities. That's impossible. There are certainly designs which improve outcomes, but not designs that definitively solve those problems (i.e. addressing author's use of the word "cannot").
So take your pick: the easier problem of settling for improving outcomes, or the harder problem of building an inclusive and equitable society, where people have a position they can take pride in, a community that they support and which supports them, and where violence (of both physical and other kinds, like aggressive alerting) is a last resort. The solutions for the hard problem aren't found in technology, but in the humanities.