Hacker News Reader

Comment by ACCount37 | Hacker News Reader

We "wanted data driven decision making" because it beats vibe driven decision making. Even if data is meh.

There are more divisions than just “data” vs “vibes.” After all, even in the natural sciences, the best data is useless without an explanation/hypothesis that can never just be reduced to the data. Precisely what is thrown out in the decline of reading is familiarity with the centuries of hard-won Enlightenment knowledge, especially concerning the stakes of education, which isn’t just vibes and that ought precisely drive our further data-driven insight into these questions.

Second best, however, I’d take the “vibes” of a random teacher over the religion-based decision making that seems to be on the rise in the US. “Data-driven” religiously motivated educational policy is the worst of all possible worlds.

The primary gist of what you wrote is important for people to grasp. Allow me to expand on it a bit, because thoughtless attitudes about "data" are pervasive, perhaps especially among the SV crackpots.

"Data" comes from datum [0], that is, what is given. What are the data or givens of measurement?

Whenever we measure something, we do so from the standpoint of some prior conceptualization. It makes no sense to speak of measurement apart from some conceptual context, as the measurement is of something as it is understood. It is through this conceptual background that we can situate some thing as a measurement, as data, and understand the meaning of this measurement, infer implications, and so on. Some call this the theory-ladenness of observation.

So you cannot say "Data! QED.", first, the meaning of the given is inaccessible without knowledge of its nature and the prior knowledge that allows us to locate the data in the appropriate context, and second, because data are not arguments. Data are used in arguments.

So if your conceptual context is flawed, your measurements are vulnerable, both in their motivating rationale and in their interpretation. A little error in the beginning leads to a great one in the end. And there's a lot of crap people carry around in their conceptual baggage.

So, we have at least three attack surfaces: the conceptual presuppositions of a theory, the theory, and the data sought to corroborate the theory.

Of course, theory-ladenness does not necessarily entail relativism [1]. So, the point isn't that we can't know anything, so anything goes, or that we don't know anything, so burn it all down. The argument is that we should be more cognizant of the bases of our justifications.

[0] https://www.etymonline.com/word/data

[1] https://edwardfeser.blogspot.com/2025/08/hanson-on-observati...

We don't use things like SAT because it's an ideal direct metric that captures how educated a student is perfectly, and allows for impeccable measurement of how successful the educational system is.

We use it because it beats the alternative - which is either going off vibes or using even more indirect metrics to measure how successful the educational system is.

If there's one school that claims it successfully teaches children to love reading, and another school that makes no such claim, but has +50 on SAT over the first school across the board? The second one is probably a better school.

> If there's one school that claims it successfully teaches children to love reading, and another school that makes no such claim, but has +50 on SAT over the first school across the board? The second one is probably a better school.

Or it's better at SAT prep? That's the entire point of OPs comment. Metrics become targets and then anything (that may still be incredibly important) but doe not contribute to that target gets lost.

"+50 on SAT across the board" requires at least being fairly good at SAT prep.

"Claims it successfully teaches children to love reading" requires nothing but a willingness to make unsubstantiated claims.

Both are imperfect performance indicators, but one is considerably less imperfect than the other.

Let me make a comparison.

If your manager says your performance is based on lines of code, you will be incentivized to write lots and lots of code. Does lots and lots of code mean you are being productive and making good software? Sometimes yes! Sometimes heaps of code means you are being ultraproductive and making amazing software. It could also mean you are writing much more code than you need to, introducing new bugs, not thinking about generalizing patterns, creating technical debt, making a worse UX, all of which I'm guessing you would agree are important to software engineering. But none of those things are going to matter in the lines of code metric.

So yes, sometimes having metrics for performance are worse than imperfect. Sometimes they are antithetical to the supposed goals. Student time is a zero sum game, and having a large portion of a crucial time in their development spent cramming for one metric is not going to have good outcomes for a society, only good outcomes for a metric.

The optimal amount of "teaching the students the actual subjects" you need to do to have them get good SAT scores is significantly higher than zero.

Sure, you can cram for SAT, and you can get gains on the metric from that. But you can't just cram all the answers into the students and have them get a perfect score via rote memorization. Students still have to learn things to be able to do well. Which is why SAT beats the "performance is based on lines of code" tier of shitty hilariously gameable metrics.

Out of curiosity, when was the last time you were involved in the school system? Things have changed a lot over decades what time previously spent "teaching the students the actual subjects" or time spent with extracurriculars continues to be eaten away by SAT prep. The downside of a bad score is now a disastrous outcome, and other parents have continued to optimize for the best score.

Oh, I don't doubt it that parents and students are optimizing for SAT outcomes themselves. You can't expect them not to. But no amount of SAT prepmaxxing can save you if you just genuinely know nothing - and having SAT is a huge upgrade over not having SAT. For one, it's a huge equalizer.

I've seen a few countries that went from "no standardized testing" to "full send standardized testing", and the benefits are too large to ignore. You can improve upon the tests, but removing them is a road to nowhere.

I'm sure the introduction of standardized tests are an improvement upon a current system, but that's not really the point I am making. My point is the ongoing use and optimization of them leads to worse outcomes overall in terms of actual understanding of the material.

> For one, it's a huge equalizer

Could you explain this for me? It's nice that everyone is taking the same test but every piece of data I've seen points to a clear correlation between household income and SAT score, hardly an equalizer.

SAT prep per se is an unbelievably shitty thing for people to waste their time on. It's largely mindless and uninteresting, stifles rather than encourages curiosity, emphasizes judging people by substantially arbitrary numerical scores, gives the false impression that some people are inherently better than others, and, in the medium to long term, is a grossly inefficient way to improve performance on the SAT.

If you want your own kids to get a high SAT language score when they are high school students, the top things you can do to help them are: (1) read aloud to them when they are very young, as much as you have time for, ideally choosing excellent books of wide variety, (2) keep reading aloud to them when they are older, (3) encourage them to read for pleasure, (4) converse about the world with them, without condescending.

If you want your own kids to get a high math score, (1) surround them with technical materials (construction toys, logic puzzles, board games, circuit parts, programmable robots, or whatever) and play with them together – or if on a tight budget, improvise materials from whatever you have at hand, and (2) spend time working non-trivial word problems one-on-one. Start from https://archive.org/details/creativeproblems0000lenc

If you have the personal time to do these steps, you won't have to give a shit about what their SAT score is, because it will be good enough for whatever they need it for. (Sadly as a society we don't have the resources or motivation to get every child enough listening-to-books-read-aloud time or enough playing-with-technical-materials-with-adult-help time, so we try to replace it with cheaper and more scalable vacuous alternatives like multiplication drills, spelling quizzes, and SAT prep.)

In Poland, many schools pride themselves for how good they are at preparing for end of high school exams. Nobody fails, nobody gets bad score.

Because if they as much as suspect you will fail, they will not let you graduate.

But statistics are kept clean :)

you might find this interesting : https://www.morethanascore.org.uk Says a lot of what you're saying, with some statistical evidence to back it up

(Note that the the British "SAT" is different than the American "SAT".)

Vibes can be data. Take for instance the economy. All of these things like the GDP, employment figures, and so are supposed to be objective measurements. But precisely because of this they've been gamed to the point of absolute meaninglessness. Recent news regarding jobs numbers over the past 4 years emphasize this to the point of absurdity. All good numbers go up, all bad numbers go down. How's the economy doing? *shrug*

By contrast poll people on their 'vibes' of the economy and you'd suddenly get some real and meaningful data that can't really be gamed beyond outright lying about the results. You'd of course have things like people wearing rose colored glasses with regards to the economy when 'their side' is in power, but that doesn't really change the validity of their opinion. And those opinions, as an aggregate, can really provide a lot of really valuable information.

I would push back on that. I think often people's "vibes" are a lot closer to reality than extremely gamed metrics

There are people whose "vibes" are closer to reality than hard cold data. And then there are people who think their "vibes" are closer to reality than hard cold data.

The second group is much, much larger.

I don't trust vibes.

Do you trust your own intuition and judgment on at least some matters or topics where data is scarce, ambiguous, or contradictory?

In the absence of data yes, but intuition and judgement are just heuristics; they suffer strongly from personal biases and are not necessarily representative of reality.

It also requires judgment and critical thinking to decide things like:

a) is this data accurate

b) is this data complete

c) is this data relevant

etc.

So even the act of selecting data is subject to bias, good judgment.

You make it sound like you’re just shifting the problem elsewhere, when this isn’t the case.

It’s not wholly subjective. Some of the processes you can use to understand your data are mathematically proven. Many others are well-tested.

In any case the idea is to try to minimise your biases and check whether your assumptions are valid so that you can make better, more reliable, more informed decisions. It doesn’t have to be a perfect system to be better.

You might not want to, of course.

Sometimes vibes includes things we're trying to get away from like stereotyping visible groups of people.

You're right. Of course i'm not rejecting "data" or "analysis" entirely, what im pushing back against is the (very common) thing that happens in companies where you get into a situation of "who you going to trust? the data or your own lying eyes" kind of thing

Yeah I agree with you. I think it's a good approach to check vibes against data and vice-versa, and not blindly trust one over the other - use one to look critically at the other. They both have defects.

Interesting your mind went there.

Does it? How do you know?

It depends. Not all good things are legible and easy to put into a spreadsheet.

Blind data-driven decisions destroy all illegible good in this world that can't be boiled down to some number going up. And there's a lot of it.

Reply on news.ycombinator.com