I always enjoy the advocates who claim that students have mastered their subjects, but "don't test well".
Would you want a pilot on your flight who flunked flying school exams, but somehow "really knew how to fly!"?
Would you want a pilot on your flight who flunked flying school exams, but somehow "really knew how to fly!"?
The reality is sometimes tests in academia are just not very well made and don't really test what they are supposed to be testing, and that's usually due to multiple reasons like misaligned incentives, staffing shortages and maybe lack of resources / funding.
I don't think the comparison to flight school is relevant enough in this context because it's a too different of a world to traditional academia.
In my long career I've noticed a strong correlation between SAT scores and academic performance as well as job performance.
> I don't think the comparison to flight school is relevant enough in this context because it's a too different of a world to traditional academia.
My dad kept his flight school tests for flying all sorts of airplanes. They bear a lot of similarities with the SATs. There's a lot of math in there for things like fuel consumption, wind, maximum landing weight, glide distance, and so on.
For example, one day he was cruising along in his F-86 when the engine failed. he radioed the tower, and they told him to bail out. But he calculated his speed, altitude, distance, wind, sink rate, air templeratur, etc., and figured he could make the field after configuring the airplane for maximum glide. He made a perfect landing, but still got reprimanded for risking his life bringing the airplane back. But he had worked the math and disagreed that it was more risky to bring it in than bail out.
SAT tests intelligence (aptitude), not skills. Which is why it correlates with job performance, where intelligence can (over some time) matter as much or more than a starting point of relevant skills.
Look at this list:
Quadratic equations and functions (vertex form, roots, discriminant)
Polynomial operations and factoring
Exponential functions and growth/decay
Radical and rational expressions
Function notation, composite and inverse functions
Nonlinear graphs and their transformations
Standardized tests can be a great resource for assessing students, but they're not just testing for intelligence. Test-prep courses average increasing SAT scores by about 200 points. That's not because they're increasing the intelligence of the people taking them.
Two kids who both went to the same school, were told the same things about the importance of the SAT and dedicated equal time to study and preparation can probably say that the difference in their scores is indicative of differences in intelligence.
But any two random students? There are so many factors that could cause students to underperform their hypothetical max score if they had perfect conditions in the months leading up to the test and day of.
I don't understand the effort to try to eliminate the reality that innate ability exists. I think we should accept this and work to cultivate it. Just because somebody of average aptitude doesn't mean he might not excel in other ways. The world needs all sorts of people, and I think the ideal system would work to figure out and cultivate these talents in everybody.
I never studied for tests in school, not for the SATs, either. I tried that in college, that did not work. The stress of the tests was pretty bad. I realized I had to study hard, and I did, and it was effective at reducing test anxiety.
This is pedantic. Take it to the logical next step. Words and letters are symbols too. How would someone who can’t read do on the SAT?
> A mediocre student who had studied SAT-style questions for weeks leading up to the test would likely outperform a high IQ student who last solved those types of problems over a year prior.
Would make an interesting experiment, if it hasn’t already been studied. I would put my money on the lapsed high IQ student though.
But he was one in a million. I always felt like a moron around him. But Hal was also charismatic and everyone liked him, including myself.
I suspect Hal was a "skin job" alien.
That's why hard questions exist.
Perhaps because they finally bothered to learn the material. I am skeptical that "strategy" makes much of any difference.
> In my long career I've noticed a strong correlation between SAT scores and academic performance as well as job performance.
A test doesn't need to test the relevant skills for that, it just needs to test _something_ that correlates with academic performance and job success.
In that same way, any test, including the SAT and GRE have flaws. They can be gamed in ways similar to LLM leadeboards: test prep makes you better at them. That's one of the main reasons universities moved away from SAT; they were afraid that it disenfranchised lower socioeconomic status students (and it does to some degree). The issue is that the test is positively correlated with success in an undergraduate program, so they threw out the baby with the bathwster. The real issue is that the SAT is not able to distinguish the capabilities among students to the degree it purports to.
And if you want an anecdote to match all yours, the first time I took a GRE practice test, I got a 3 on the writing. Not because I'm poor at writing, but because I didn't really know what they were looking for. After reading a test prep book, I got a 4.5 on my next practice test and a 5 on my final practice test. When I finally took the actual GRE, I got 6 on the analytical writing. Trust me, nothing changed in my writing ability over that time. In fact, I didn't even practice the skill except through those three practice tests. Clearly the test was not capable of determining my real ability to make an argument; it merely tested my ability to adapt my writing to what was supposedly being tested.
Interestingly, the vast majority of universities that got rid of the GRE requirements for PhD programs are not going back on that. Turns out that the students with the highest GRE scores are the ones most likely to drop out of their STEM PhD. [1]
[1]: https://journals.plos.org/plosone/article?id=10.1371/journal...
Anyhow, the questions were all about freshman engineering knowledge.
Responding off the cuff without any reflection on the comment you're responding to doesn't move the conversation forward in any meaningful way. It just comes across as disrespectful.
The real issue is that standardized tests disenfranchise lower SES students less than any other metric.
Everyone who takes the SAT has to sit in the same room for the same amount of time answering the same questions. You can’t just pay someone else to take it for you (like essays) or select which difficulty level you take (like going to a prep school with grade inflation), or luck out in who your parents know (like recommendation letters).
Some may have better opportunities to learn the material, but, at the end of the day, you have to actually learn the material. There’s no getting around that.
As your own GRE anecdote shows: A little studying with some inexpensive books makes all the difference. Unless things have radically changed, a couple SAT or GRE test prep books are significantly less expensive than just one college textbook.
Bluntly, the reason SATs are better correlated to college performance than other measures are because of the reasons I mentioned. They strip away most of the privilege of coming from a high-SES family.
The issue is that the test is positively correlated with success in an undergraduate program, so they threw out the baby with the bathwster. The real issue is that the SAT is not able to distinguish the capabilities among students to the degree it purports to.The guy who got into my uni class as #2 in that exam dropped out after a few semesters because he couldn't beat calculus. The #4 took several extra semesters to graduate despite not working/not interning. Several others in the top third struggled through. We had _maybe_ 2 or 3 guys who straight-A'd the entire major.
I myself got in as #17 and still failed a few courses. Thankfully no one cared throughout my professional career.
My wife has anxiety, and the time-limited, high-stakes, pass-or-fail nature of a test makes it much, much harder for her to perform well. Outside that context, she's blazingly intelligent, performs very well under real-world high-pressure situations, is extremely diligent at getting any other assigned work done well on time, and has repeatedly written the guidelines for processes and procedures in her jobs.
I'm the opposite in many ways. I generally do somewhat poorly in a classic classroom environment, primarily due to ADHD hampering my ability to get homework done regularly and on time. However, I test amazingly well. I consistently finish in 50-70% of the time of other students, with scores in the high 90% range. (In my jobs, I think I do pretty well.)
Naturally the plural of "anecdote" is not "data", but I've known enough other people who fit both of our molds that I think it's fair to say that basing even a plurality of your assessment of someone purely on tests, especially standardised tests, is going to mislead you at least as often as not, in both directions.
On average. However, I've also had the experience that some of the most competent people I've known had rather difficult teens and twenties.
Hiring someone who flunked out of highschool, worked odd jobs for 10 years, then got a diploma and a degree is higher risk, higher reward. They are often times harder workers, unusual thinkers and more grateful for what they have.
Your single example identifies a situation where we would want someone resilient to stress to pass the testing process.
However if you are evaluating someone's ability to identify characters in Shakespeare who are most closely representative of the Bard himself, a proctored exam may not be the only environment where that could be demonstrated.
Anyone in software knows as well that for a test to be effective it has to be written properly. In the history of academic and standardized testing however there has been little rigor in the construction of tests, and those who pass are ones who give the "right" answers regardless of whether those answers are true proofs of knowledge.
As for people who cannot perform under the stress of a test, how are they going to perform otherwise? Anyhow, the solution to test anxiety is to keep taking tests - the anxiety will recede.
For example, the first time I tried public speaking I was paralyzed. But I kept trying it, and the anxiety went away, and now I do a fair amount of public speaking and enjoy it very much.
What is the point of “knowledge” that can’t be demonstrated. How will that person demonstrate their knowledge of Shakespeare?
Sure, just not in the cockpit
In addition to that, this claim is like the dragon in my garage thought experiment. It an unfalsifiable claim that they have “knowledge” but can’t demonstrate it.
The same can't be said for many other tests. If the test involves the practical application of the very skill being tested, then that test has direct relevance to he competency of said skill.
But many other tests are not like that. A teacher can be brilliant in the classroom yet stumble on a standardized certification exam full of pedagogical jargon. A chef can cook a variety of excellent dishes but fail a written culinary theory exam testing the French names of techniques they perform by instinct. And perhaps more relevant to this audience, a coding interview that relies on whiteboarding algorithms from memory can easily fail an excellent engineer who builds great software every day but doesn't recall the optimal solution to some puzzle on the spot.
"Academic tests involve academics"
"I'm not talking about academics!"
That about sum it up?
Of course you'd have refrained from making such a vacuous reply if you bothered to read the article.
Because both have been shown to have predictive power for success in college.
Maybe two.
> A teacher can be brilliant in the classroom yet stumble on a standardized certification exam full of pedagogical jargon.
A teacher that cannot explain how calculus works cannot teach it to anybody.
> a coding interview that relies on whiteboarding algorithms from memory can easily fail an excellent engineer who builds great software every day but doesn't recall the optimal solution to some puzzle on the spot.
I've seen too many coders using bubble sort because they don't know enough to look for a better algorithm.
In any case, the purpose of leet coding tests is to quickly filter out the utter frauds. I have a programmer friend who wanted a job at a major software corp. He knew he'd have to pass the leetcode in an early stage of the interviewing. He figured it would take 6 weeks or so to study that material. I suggested that, since he was applying for a $250K job, that would be the most productive studying he'd ever done. He agreed, did the 6 weeks of studying, aced the leetcode test, and got the $250K.
So ya, there is a point to those tests, in filtering out the frauds and the ones who aren't willing to do what it takes to get those jobs.
The question is whether a given test measures anything relevant - did your friend become a better programmer for doing 6 weeks of leetcodes? E.g. what kind of experience did he gain about large code bases and how to handle those? Continuing your analogy, would you fly with a pilot who drilled on taking off a bunch of time, but never practised flying in a storm?
I'm not saying leetcodes or exams are useless, but Goodhart's Law apply.
It's still worth discussing if the particular test is ideal, but the answer IMO is not to say let's remove tests in general.
He's the kind of person I'd want to hire.
Ground school most certainly does not involve a lot of math, it's not like there's any calculus or algebra involved... it's basic arithmetic. Furthermore it's categorically false that you need to pass ground school before you're allowed to fly.
Are you just making things up?
>A teacher that cannot explain how calculus works cannot teach it to anybody.
This is a strawman argument, I never made anything that could even remotely be interpreted as this.
>I've seen too many coders using bubble sort because they don't know enough to look for a better algorithm.
This is committing a very basic logical fallacy. The fact that someone who is incompetent likely can't pass a test is not the same claim as someone who can't pass a test is likely incompetent.
Hopefully you are able to identify this logical mistake that you're committing and revise your position accordingly.
Google sez: "The U.S. Air Force strictly requires you to complete and pass formal academic ground training before you ever touch the controls of an aircraft"
They're not going to risk an aircraft on an incompetent student.
> A teacher can be brilliant in the classroom yet stumble on a standardized certification exam full of pedagogical jargon
I stand by my statement.
> logical fallacy
A implies B meaning B implies A is indeed a logical fallacy. But that does not rule out B implies A. A and B can be strongly related to each other.
Furthermore, even if the Air Force did not require IFT before UPT (the Air Force's own training program), you've completely changed the nature of your argument. I have no dispute about whether the Air Force may or may not have stricter requirements for their pilots, but that wasn't your argument.
>I stand by my statement.
You've proudly planted your flag on a point nobody was contesting, which is a strange hill to celebrate on but you do you.
>But that does not rule out B implies A. A and B can be strongly related to each other.
Discussing a topic with someone who not only uses logically fallacies as justification for their argument but brazenly doubles down on said fallacy is a good sign that this is probably not a good discussion to continue spending time on. Like am I supposed to simply accept your logical fallacy and take on the burden of disproving every claim you can dream up simply because you've asserted it isn't logically impossible? The person making the claim carries the burden of supporting it, and "they're strongly related" is something you have to actually show, not something I'm obligated to refute on your behalf.
Here's something to consider: just because someone is good at writing compilers or designing a language, that doesn't entail anything about the quality of their arguments.
Note that a tree implies it is made of wood. If you find a stick of wood, odds are it came from a tree.
There is no singular "IFT"... you happened to find one IFT among hundreds across the U.S. that has such a syllabus, great... but it does not come first as requirement mandated either by law/regulation or convention. Here is the syllabus for a different FAA Part 61 and 141 approved IFT program that uses an integrated approach with the following quote:
"Each Module contains both a flight and ground lesson. This presents an integrated flight training process and will promote easier learning and a more efficient flight training program"
https://static1.squarespace.com/static/5ad1e29b372b96bedc6b1...
>Note that a tree implies it is made of wood. If you find a stick of wood, odds are it came from a tree.
This is false, not all trees are made of wood (palm trees) and there are natural sources of wood that don't come from trees.
https://en.wikipedia.org/wiki/Woody_plant
But of course... instead of just admitting you were wrong to make that logical fallacy... free to continue doubling down and making things up.
Regardless, as a civilian, you do have to pass a written multiple-choice test on flying theory before you can solo. Any time you spend in the cockpit before passing that test will be under the supervision of a Certified Flight Instructor (CFI), in a plane with dual controls, and you are not acting a pilot-in-command.
So, although the parent may be slightly over-focused on the USAF way, I think it is fair to say that for any type of pilot training in the USA, you do have to complete academic ground training (including passing a formal written test) before you are truly “given the controls”.
I do not consider that flight training.
You goofed. A palm tree is not pedantically considered a tree. You made that up.
https://texastreesurgeons.com/blog/2025/01/03/palm-trees-are...
> odds are it came from a tree >> there are natural sources of wood that don't come from trees
Confirms I was correct.
> Each Module contains both a flight and ground lesson.
And the ground lesson comes first. I read your link.
How would you calculate a crosswind component from the runway heading and reported wind speed and direction without trig? How would you think pilots measured their distance to non-directional beacons before GPS and DMEs existed?
How would you solve for fuel remaining without algebra? How would you estimate the best speed to fly with a given headwind to maximize fuel onboard at the intended destination without calculus?
A very basic principle of glider flying involves finding the tangent to a curve. Is calculus not applicable there?
Fuel consumption is often estimated by numerical integration of fuel flow rate. That doesn’t require an analytic solution of the integral, but I think most pilots have at least a passing familiarity with the concept.
> Furthermore it's categorically false that you need to pass ground school before you're allowed to fly.
I interpreted the parent’s statement to mean “before you fly solo.”
There certainly is when you're navigating.
Some of the more advanced math is boiled down to specialized slide rules, though these days they'd use a computers.
For example, the fuel consumption rate vs range is not a linear relationship, because burning fuel lightens the airplane and so it can go faster/further.
but to get to the city we had to take it was a 2 hour long drive through twisted roads that made me carsick. I lived in a small town far away from the city. By the time I got there my breakfast I had quickly eaten gave me a stomach ache, I had woken up far earlier than usual and not gotten my 7 hours too. I certainly would have done a lot better if I had lived in the big city.
Another factor: if you wanted to pay the fee you could just take the test over and over again until you got a great score. So kids with poor parents obviously had a huge disadvantage. Also kids who had the time and money could study for it with prep books - I did, while some of my friends were flipping burgers while still in highschool. Its not surprising I got a higher score than them, but it said nothing about my intellegence or understanding compaired to theirs.
Also, in a decent number of states, it's mandatory for all HS students to take (and I believe free to students as a result).
One who cannot calculate how much fuel he needs to cross the lake will kill me.
Remember JFKjr? He killed himself, his fiance and her friend because he did not pay attention to the instruments.
An acquaintance of mine died trying to fly through a thunderstorm. Another one didn't pay attention to the weather and nearly died from wing icing.
Flying is no joke.