Hacker News Reader

Comment by jakub_g | Hacker News Reader

For those short on time: "It doesn't go as planned" is explained in the middle of the article:

> When everything compiled, I shipped it. And to everyone's surprise, apart from a few bugs I had created while rewriting thousands of lines of code... it worked. It just did. No explosions, no obscure bugs, no memory leak, no performance degradation. That was rather unexpected.

...followed by

> Then we saw the JVM CPU usage rise to alarming heights, with unusual patterns. And no obvious culprit in the thread dumps...

> it was just the JVM that needed some tuning.

Details: https://lichess.org/@/thibault/blog/lichess-on-scala3-help-n...

View on HN

> "It doesn't go as planned"

bit of an editorialized clickbait title really.

I don't think there's anything wrong with an evocative title. In my mind, clickbait is when you never actually get what's promised, or it's vastly oversold. This title tells me, "we had a deploy that caused problems for a surprising reason," and I feel like needing to retune the JVM delivers on that. It's a weird, surprising issue. I'm sure there are old-hands who have deployed JVM apps at scale might be unsurprised and unimpressed, but I was surprised.

I think showmanship is okay if it's tasteful. The problem with clickbait is that it's manipulative and tacky and dishonest. After you read the article you feel like you've been had.

>In my mind, clickbait is when you never actually get what's promised, or it's vastly oversold.

Clickbait is trying to bait you into clicking. This definitely falls under that. Is it the worst case of clickbait I've seen? No. But it is definitely clickbait.

By this definition any catchy or interesting title is clickbait, regardless of how descriptive or accurate. "Nixon Resigns" will certainly bait me into clicking the article for more details, even though it's basically leading with the punchline. For the term to not be meaningless I think some level of deception or withholding important info has to be involved, "Nixon Made a Shocking Decision....You'll Never Guess What It Is!!!" is clickbait.

Based on your first example, I know what happened and can make an informed decision about whether I need or want to know the details of how it happened. I'd argue that that is not click bait, as it leads with the most important piece of information first. The title could further be improved by writing "Nixon resigned because of XYZ", but that would be nitpicking.

The most beneficial title from the reader's perspective is a title which allows the reader to quickly decide whether they need to know the information and informs the reader with the necessary information as efficiently as possible. Hence the now long dead practice of writing in the form of an inverted pyramid (https://en.wikipedia.org/wiki/Inverted_pyramid_(journalism)).

The inverted pyramid isn't dead. Blog posts are more like feature articles. Feature articles never followed the inverted pyramid.

Do you feel this article did something wrong? Because from what I'm hearing it seems like you're saying it wasn't to your taste, which is entirely valid but a different matter.

I'm making an argument that the article is click bait, which it is. I'm not sure where you see a value judgement about whether click bait is wrong in my comments, or why you are defensive about it on behalf of the author?

I don't like click bait. I voiced that opinion. That's all.

> I feel like needing to retune the JVM delivers on that. It's a weird, surprising issue.

Having spent several years in the running-JVMs-in-production trenches I would claim the opposite.

I did address this objection upfront:

> I'm sure there are old-hands who have deployed JVM apps at scale might be unsurprised and unimpressed, but I was surprised.

I've never done anything nontrivial with the JVM, so I'm totally willing to believe I was surprised because of my JVM ignorance, but I've never had to fiddle with my language VMs.

Regardless, I argue there's room for tasteful showmanship without it being clickbait.

Yes it's clickbait, so I didn't upvote it; an accurate title would have been "Lessons learned upgrading Lichess from Scala 2 -> 3 and the need to tune the JVM". A drama-free title like that would have been fine, and I would have upvoted.

Personally I'd never click on your proposed title as I have 0 interest in Scala, but I clicked on this one and was pleasantly surprised by interesting story. Making everything sound robotic is not the way.

Do you agree that title is clickbait? You are ok with titles that misrepresent or exaggerate to boost clicks (even if most of those viewers bounce upon discovering that)?

But the upgrade....didnt go as planned? In what way was it a misrepresentation? It was certainly generic but definitely not exaggerated. Felt fine for an editorialized title

sdwr's comment had already explained why this is clickbait: https://news.ycombinator.com/item?id=34008009

They planned to have a much bigger hassle with the upgrade. That "plan" didn't materialize, as the upgrade was a lot smoother than they expected.

Sure the title is jazzed up a lot, but calling it clickbait seems to be a bit harsh.

You just described the exact reason why this is clickbait. You wouldn't have clicked but due to editorialized title, you did click.

The actual title seems to be Lichess & Scala 3 and the subhead is "Lichess gets a big upgrade. It doesn't go as planned."

Or maybe the actual title “ Lichess & Scala 3”

It is not the articles fault the first sentence was used as the title

I agree. Clickbait titles are not universalizable. Kant would not be pleased.

INNOCENT dev migrates to Scala 3, what happened next will SHOCK you.

Spend a week of running the JVM in production without tuning it challenge [IMPOSSIBLE]

This should be a Leetcode hard test. Deploy a JVM for a large production upgrade without issues

Trick question, it's not possible

that's still conveying too much information

INNOCENT dev did THIS, and what happened next will SHOCK you

INNOCENT dev did THIS, and what happened next was <truncated text, click to follow article, followed by being bombarded by 20 ads which magically bypass ublock origin>

WHy iS NoOne Talking AbOut ThiS ? No7 SurpriZed Me ! :D

The actual title on the article is "Lichess & Scala 3".

Whoever posted it to HN decided to go with the title of the first section of the article.

That title is just as poor, in the opposite way. It says nothing about what the article is about.

What's wrong with a title that is a one sentence summary of the article?

I guess submitter thought it wasn't a good title for HN purposes too, and so he went with the subtitle rather than inventing his own.

So nobody intended clickbait.

How about the title and subtitle be taken together, as intended.

It sounds like clickbait when read in the Hacker News context. But if I'd come across this in the Lichess blog directly, I would've immediately known it was kinda tongue-in-cheek or at least nothing serious. Context matters.

Yeah despite being an interesting post about Scala, I came for the harrowing tale of a big deployment gone wrong. Downtime, angry customers, seat of the pants debugging and straight to prod gambles to save a sinking ship.

Instead, a smug tale of a huge upgrade that went unexpectedly smoothly - with a performance problem swiftly solved. Still a good story, and kudos to the team. Just not the horror show I came for.

Is it clickbait when the top contributor/creator posts on his website? The same website that makes a strong promotion against ad tracking: https://lichess.org/ads

I clearly don't have the same definition of clickbait.

> The same website that makes a strong promotion against ad tracking: https://lichess.org/ads

What does this have to do with clickbait?

that's a great question. I had reviewed the Wikipedia page prior to my comment: https://en.wikipedia.org/wiki/Clickbait

it seems there is no universal definition of clickbait. So I based my comment on this:

1. the article was posted on Thibault's own website that doesn't generate revenue per click. Isn't clickbait associated with the possibility of additional ad revenue?

2. Thibault recently called for help on the issue. This post recaps what he went through. He clearly expressed that the code upgrade process didn't go as planned for him. the post and sub-title feels genuine.

3. Clickbait (to me at least) indicates that there's either intent on generating revenue or self-promotion. I fail to see those conditions here

So that's my logic. So why would this be considered clickbait?

Clickbait is a sensationalized or misleading title meant to draw extra attention (in the form of clicks).

Both "big upgrade" and "didn't go as planned" are technically true, but overpromise. A big upgrade could be new features, improved UI, something that affects the user experience.

"Updated Lichess language version, had to tweak cache settings" is a title that isn't trying to sell itself.

It's not anyone else's blame if you don't share their life passions.

Thibault isn't posting low-effort spam for profit, he's pouring years of his life energy into a beloved project that is a gift to the world.

'if it builds it ships' in action, love to see it

edit: Since I got sniped pretty hard for not divulging further, allow me to 'kill the joke' by explaining. This will read more harsh than I intended

Shipping it isn't the problem. In fact, that's the expectation. Just... don't ship it to a place that matters first.

I feel baited by an otherwise cavalier operator, footguns be footguns.

as planned -- I'm not convinced there was a plan

I noticed this as well. There was no integration testing? No manual testing?

I don’t pay for Lichess, love the UX, and I know it’s a labor of love, but come on. Surely Lichess refactoring aren’t so urgent and user critical that they should just go straight to prod?

Yeah ... but nah.

Yeah, there should be some kind of shakedown test.

But Nah, not every issue can be tested for within the manpower/$ budget. Issues of load being one of them.

A valid strategy is: send it!, work with the beast a bit, if too unruly roll it back. In this case, they tamed the new beast. Not like Lichess is going to lose billions...

Load testing isn't quite the expensive proposition it seems!

Think about it - clients or requests are really just ephemeral ports...

There are something like 30 thousand of them available by default with Linux

If you use sockets you can have as many connections as your system allows for open file descriptors. The upward limit is...

    $ sysctl fs.file-max
    fs.file-max = 9223372036854775807

Spawn a ton of concurrent requests to hit the service/API!

apachebench is a decent example of the idea; imagine some proprietary sauce with it

It gets cheaper (and in some ways, more effective) if you actually remove the network -- latency goes down

You can also rate limit it in software, but it's really not analogous in some ways (eg: net.core.somaxconn limitations/realities)

Development time is totally understandable though. It's important I leave my green field of comfort

Load testing would have to be quite advanced to find this, doing more than just simulating users for the most common features. It took more than 24 hours of real-world usage to visit sufficiently many niche code paths to fill the code cache.

Advanced in that it's POST instead of GET, and some parameters to boot

I'm not saying it doesn't take effort, I'm just saying it can and is regularly done

Regression testing with speed. If you test your features as they're added, niche-ness doesn't matter - you know about it and test it

A regression test quickly turns into a load test if you do them all at once

To be clear, I'm not trying to twist the knife. I understand why it wouldn't be done. I just won't pretend it can't

I've been trying to avoid this 'hindsight is 20/20' thing...

I'm not trying to lay blame - just offer thoughts towards improvement, and encouragement against some defeatist thinking patterns

A lot can be done with very little equipment/time investment

Lichess being a project that is 85% written by one guy, and all other changes approved and read by him, he may have felt confident (until now) that he could maintain all the complexity of the software in his own head, and recall what to do and where to do it whenever he made changes. Seems bonkers to most of us writing software in teams, but compare this choice of workflow to ones made by others in similar situations: programmers in academia, who often work entirely alone, or Linus Torvalds in earlier versions of the kernel. Skipping testing is a common choice, especially when working alone and with a typed language. I could see Thibault bringing tests in now however. They would not have caught much in this case, because it seems like the problem came to light only when the software started running with real users.

While I do agree, these folks deserve a break... see my other comment in this thread RE: load testing

It's not particularly expensive or difficult, but it does need investment

Thought the same

JVM tuning is frustrating. The need for that is generally not an issue on Microsoft .NET.

If I understand correctly, not being able to tune .NET is a shortcoming. It is like having a JVM which does not have tuning options, it works optimally only for some workloads, and fails severely for others.

I feel like “it didn’t go as anticipated” would have created the same suspense without being a lie.

Kindly asking: did you consider that English is possibly not the author's mothertongue?

This whole subthread reads like an HN parody.

Reply on news.ycombinator.com