sjansen

They (re)wrote most of the client SDKs on a Rust core, but the Temporal server is still written in Go.

The site loads fine for me.

See also: https://downforeveryoneorjustme.com/temporal.io

Putting work in a queue is only the start. Most organizations start there and gradually write ad hoc logic as they discover problems like dependencies, retries, & scheduling.

Dependencies: what can be done in parallel and what must be done in sequence? For example, three tasks get pushed in the queue and only after all three finish a fourth task must be run.

Retries: The concept is simple. The details are killer. For example, ifa task fails, how long should the delay between retries be? Too short and you create a retry storm. Forget to add some jitter and you get thundering hoards all retrying at the same time.

Scheduling: Because cron is good enough, until it isn't.

A good workflow solution provides battle tested versions of all of the above. Better yet, a great workflow solution makes it easier to keep business logic separate from plumbing so that it's easier to reason about and test.

Would "we screwed up by blindly trusting ChatGPT" annoy you less? Because that's how I read it.

Or more specially, given the context: "We were in a rush to translate a bunch of code and ChatGPT was doing such an impressive job helping that we became complacent and forgot that it just parrots back text it has seen before with something that looks like intelligence but without actual comprehension. So when it copied a common bug, we weren't paying enough attention to catch it."

Line 56 is executed as the file is loaded. Simplified, the line is essentially:

  id = Column(default=str(uuid.uuid4()))

As written, a UUID is generated once and used to set a class-level attribute. Each Python process would generate a unique value, so it wouldn't be immediately obvious. Most of the time Python's ability to run code as a file is loaded is helpful, but this is one the well known gotchas.

Although I'm not a SQL Alchemy user, I assume the fix is essentially the same as it would be for Django. So the correct code would have been essentially:

  id = Column(default=uuid.uuid4)

Instead of executing `uuid4()` and caching a single UUID value, it would have executed `uuid4()` each time a new object was created.

The author defined fair in a pretty industry standard way:

> One user should not be able to block another's work.

A multi-tenant architecture implies over committing resources to achieve better economics. Once upon a time, decades ago, when computers were so expense that companies payed for batch processing instead of owning their own, FIFO was acceptable. (And for a tiny part of the market it still is.) But ever since the rise of time-sharing operating systems, users have come to expect short delays for small tasks. If you let one customer monopolize your capacity because they submitted a mountain of work before a second customer submits a handful, you probably won't have the second customer much longer.

It actually uses Flex/Bison, and does take advantage of the extended features they provide.

Was just going to say, they're merely making more obvious a situation that has long existed. But most people don't bother checking if packages they depend on are part of `main` or `universe` so I can see how this comes as a shock to some.

> Why would you decouple the change from its context?

The issue is the context.

At work, we require a link to the issue/story because it makes SOC2 audits much easier. The commit message is a great place to explain the chosen implementation, but that's not all there is to software development.

It rarely makes sense to repeat in commits how the issue was reported, or why it was given a specific priority. And it can't track anything after merge, like how the change was tested, or whether the change was communicated to users.

Maybe you don't have to deal with auditors. Maybe your work has different processes to keep them happy. But as far as I'm concerned, is it important to link the issue for context? Yes! Just yes.

Experience has shown that often “worse is better”. Go does an amazing job of balancing complexity and power. I haven’t seen a ”better” language that isn’t either slower, harder to become productive, or both.

https://en.wikipedia.org/wiki/Worse_is_better

In Go, `rune` is an alias for `int32` and is used to indicate the value is a Unicode "code point".

For characters in the ASCII range, that means it's just a character encoded using more bits. If you need to worry about the full Unicode range then it's important to understand Unicode Normalization Forms.

https://go.dev/blog/strings

https://en.wikipedia.org/wiki/Unicode_equivalence

> Fake crypto is not, as far as I know, a problem.

As it happens, crypto-less cryptocurrency is very much possible as demonstrated by the OneCoin scam: https://en.wikipedia.org/wiki/OneCoin

https://www.fool.com/the-ascent/cryptocurrency/articles/what...

Goramit! You've reminded me Firefly was killed too ruttin quickly.

Unfortunately that's not necessarily true. There are plenty of examples of prosecutorial over reach and unfair sentencing because our laws were written by people who fundamentally don't understand technology. It will probably take a few generations to fix that.

Common sense won out in the case, but it certainly helped that the target was a reporter instead of a security researcher or ordinary citizen.

As someone who watched this bit if history unfold, and someone who diagnosed similar issues "back in the day", I see it as an example of sitting around the virtual water cooler trading war stories.

Sometimes it's nice to look back at the fun bugs we squashed, and be grateful they're not a thing anymore because our systems have improved.

> Figuring out problems like this when nobody else can and saving the day makes you feel like a wizard.

You're right. It was an awesome intellectual high each time.

But only the first time. Eventually seeing the same stupid problem come back got tedious. Thank goodness our systems continue to evolve. It's so long since I've seen a leaky environment at service startup cause problems that I'd forgotten it was a thing.

I don't understand anyone that seems to think UNIX was perfected in 1983 with the release of System V. We've learned so much since then by standing on the shoulders of giants.

They’re not required to stay at the old employer, they’re just not allowed to start at the new employer. Cold comfort for anyone with bills to pay, but they aren’t being targeted by the decision, just the new employer. They’re free to go find a third employer. (Which is probably how the judge sleeps at night after putting so many people in an impossible situation.)

My perspective on the piece is completely different, and I think you're being unfair. Clearly your experience was painful, and I can understand why you hear echos in the author's essay, but I suspect that's more likely a reflection of how raw the experience still is for you than it is proof of the author's narcissism.

From my own experience, there is significant overlap between burnout and depression. The root causes are different, but the symptoms are the same. The echos I hear are based on having experienced both.

If you have never experienced either, it's hard to understand how completely they alter personality and decision making. Seemingly mundane problems become insurmountable. Trivial items become weirdly important, and critical items become... not quite ignorable, but hard to prioritize.

It's very hard to behave reasonably when you feel like you're lost and drowning. Instead of making simple and obvious decisions, I found myself making completely irrational decisions. Looking back a couple decades later, some of my decisions were bafflingly silly. Others were tragic. In all cases, my decisions were based on overwhelming emotion not logic, experience, or my personal priorities.

Perhaps you've gone two or three days without sleeping, and noticed yourself behaving like you were drunk but tired. Now take away the loss of coordination, increase the loss of executive control, and let it sneak up on you over months or years. That's the best I can describe the experience to anyone that hasn't had it themselves.

I found the author's essay beautiful, cathartic, and helpful. It helped me to empathize with what he'd experienced, and reminded me how complex we each are as human beings. It gave me new things to think about as I consider my own career path and life. And it was interesting seeing the last two years of COVID from a European's perspective compared to my midwestern American perspective.

In other words, from my perspective the essay was an example of humility and generosity, and I'm grateful for it. If we were all so willing to examine and share our inner experiences, perhaps the world would be a slightly easier place to live.

> Stockholm syndrom much?

Yes, but not how you'd imagine. I spent years as a hard core Linux fanboy. Advocating for Linux. Declaring Linux the future when many thought Windows would obviously eliminate all competition. Volunteering for install fests and wrestling various distros onto unwilling hardware.

I also spent 9 years teaching and writing training materials for RHEL, SUSE, & Ubuntu. As a result I dug deeper into core system details than most. (How many remember SUSE's attempt to achieve faster boot times by introducing an optional alternative to normal init that dynamically generated a makefile to execute init scripts in parallel? That was one of many strange rabbit holes I got to explore.)

Thankfully, I've gotten over it. I still prefer to keep Linux near the center of my career, but I've been able to adapt as "the cloud" has rendered my old school sysadmin skills less relevant. Occasionally I call on them to rescue a pet, but today I find "herding cattle" and going "serverless" much more enjoyable.

And you?

There is so much truth to what you say. I used Linux exclusively from January 2000 until April 2012. A lot has changed in that time, but out of all the changes the three I'm most thankful for are:

(1) Xorg replacing XFree86 and making video "just work"

(2) Pulse Audio replacing OSS and making audio "just work"

(3) systemd replacing the dog's breakfast of scripting madness each distro built from scratch instead of collaborating

It amazes me that the same person was responsible for leading 2 of the 3 changes.

I for one welcome an increasingly unified operating system core, instead of the random competing and overlapping components we used to have. Sure they worked 97% of the time, but the other 3% when they didn't could be a royal PITA. Especially when two waring components wasted time pointing blame at each other instead of focusing on solving end user issues.

You've made the mistake of expressing an unpopular opinion, so you're getting down voted. I strongly dislike proprietary software, but you raise valid concerns that should be considered instead of being downvoted into silence. (edit: at the time I wrote this comment, the parent appeared to be headed for oblivion, happily that appears to have improved)

I don't know that open sourcing Java killed Sun. More likely it was Intel coming close enough to SPARC performance at a lower price, and Linux coming close enough to Enterprise reliability also at a lower price. But it's certainly true that open sourcing Java and Solaris didn't save Sun.

Likewise, open sourcing Mozilla didn't save Netscape. In fact, I would bet the vast majority of companies that start out proprietary before deciding to open source ultimately die because they're already unhealthy and doing it as an act of desperation.

If a company isn't designed from the start to support an open source model, it's very hard to make the transition and survive. And we have no right to demand they try. Just like everyone should respect the license F/OSS authors choose, everyone should respect Wolfram's right to stay proprietary.

Of course it's also important to remember that proprietary software vendors are more likely to yank the carpet from under their users without warning. Instead of demanding the vendor open source, users should just find and support open alternatives.

Although the author says his advice is for people "building new infra at large companies", there's certainly a number of points that apply to a larger audience.

For example:

> Measurement is a means, not an end.

Reminds me of a lesson I learned early in my career. Bad things happen when you lose sight of your purpose and focus just on measurement.

https://en.wikipedia.org/wiki/Goodhart%27s_law

One of our lead developers used it at the height of cloud hype. You could always tell which pages of our internal wiki he had edited recently. Amusing at first, but I got tired of it much faster than he did.

As already mentioned, Docker images are also available and that's my preferred solution.

If your concern is more about having access to ARM hardware, CodeBuild is a reasonable option. CodePipeline is more expensive than it should be given the feature set, but CodeBuild is cheap and easy to use directly.

Alternatively, I've been intending to move some of my build to Lambda itself. Easy to bootstrap if you use an architecture independent language like Python to script your build. And if your code takes more than 15 minutes to build, should it really be a Lambda function?

No, it means the lawyers responsible for handling SCO’s bankruptcy have stopped hoping for a bigger number and settled for being able to repay creditors something instead of nothing, while IBM’s lawyers have decided the number is small enough it’s not worth defending itself any longer.

The article mentioned why very quickly while discussing building multiple solar farms so that at least some aren't in shadow at the same time: sending enough cable up on rockets to connect all of them would be more expensive than the "expensive, complicated and inefficient solutions".

The two terms imply fundamentally different motivations.

Free Software came first, with the motivation being maximizing user freedom.

The term Open Source was created as a marketing hack in response to the observation that open collaboration lead to higher quality software but Free Software was being misunderstood as "cheap software".

While many Open Source advocates appreciate the freedom it provides, in the early days the focus was almost entirely on quality as they tried to increase awareness. Richard Stallman's reaction at the time made it very clear he felt that Open Source was trying to hijack his mission to provide user freedom and replace it with corporate value as the new goal.

(I suspect rms still does, but I stopped watching him tilt at windmills about a decade ago. We owe him a lot for laying the foundations that made F/OSS possible, but we owe just as much to many other early contributors, including Bruce Perens and Eric S. Raymond for the Open Source marketing hack that built on that foundation. Not to mention Netscape, Mozilla, & IBM for demonstrating that F/OSS was for more than just "long haired hippies".)

Tragic fact:

Although in modern usage decimated means "eliminate a large number", historically it meant "eliminate 10%". So if the predicted loss of 20% to 30% of the populations is correct, Myanmar will be much worse than decimated.

I can't wrap my head around the idea. I hope it's not true.

And most blockbusters movies never turn a profit. Color me skeptical. There’s probably similar accounting tricks at play.

An ethical argument could be made that by watching we are encouraging the behavior and therefore share any blame if something goes wrong.