Hacker News Reader

Comment by hospitalJail | Hacker News Reader

I wonder if this is going to level the playing field a bit.

People who cheated in school had a bit of an advantage, but with these advances, it forces teachers to find ways to test people in different ways.

I once had a teacher who gave us open book, open note, etc... for chemistry. The trick was, you had ~60 minutes to do it. If you didn't practice, you'd never finish the test in time.

I also had a different teacher in thermo 2 give us two different insane problems, and we had to solve them. GPT would easily fail at this. It involved drawing a few diagrams of layers and calculations. (although, I wonder if you could train LLMs on this problem) Still, you had to do this in class, no calculator, just set up the equations.

View on HN

> I once had a teacher who gave us open book, open note, etc... for chemistry. The trick was, you had ~60 minutes to do it. If you didn't practice, you'd never finish the test in time.

I hated tests like this... because I was a lazy kid who coasted through school and didn't understand why "simply understanding the material" was insufficient. In college (Cornell) the chaos continued, I got poor grades in calc because despite getting all the bonus questions right (which tested your understanding of the material), I couldn't finish the actual test in time because I hadn't done the problem-set homework (these problem sets took 6 hours to do btw... coming from a high school that only started challenging me towards the end, I was completely ill-equipped to handle that level of self-discipline).

Of course, in the working world, it turns out that persistence/commitment/self-discipline is the real payoff. I had to learn that the hard way later on.

Yeah, I failed calculus three times because I was working two jobs to support a failing business, and I just had no time to do the assignments. Finally I found a class that graded only on the tests and got my A.

I failed it twice: first because I got too cocky when the previous semester ended without incident. Second was due to the fact that I already had a part time job.

On the third attempt I had a different lecturer - an old guy who favoured attendance over anything else and gave the same problems each year. I knew most of the material already, so I got 80% out of that.

I learned nothing from that experience though and went on to fail 22 courses over the seven years I spent in college.

Holy cow, that's a lot of dedication and endurance! What kept you going despite the unusually high amount of failures?

It was actually youth, arrogance and stupidity. The same things also kept me going.

After my junior year I found a job through a school friend whom I helped preparing for his exams and from that point on I started weighing whether on a given day I should go to lectures or make money. My family was in a precarious situation financially and I only needed two full days to finance repeating a course, so the choice was obvious at the time.

I almost dropped out after two semesters because I failed too many courses (four - two was the limit after the first year), but I talked the dean out of deciding to kick me out - we had that option back in the day.

I spent the remainder of my time on what you could consider a Performance Improvement Plan - took 14 courses for the third semester, failed three, but thanks to that I was just three short starting the fourth, which was acceptable.

Four years of winging it like that and after a total of five years(instead of the planned four) I had my Bachelor of Engineering degree.

In hindsight I should have not went to do a Master's Degree because I never finished my thesis due to not being satisfied with the scope(or the lack of it).

I hesitate to ask this and have zero judgment implied when I ask it- Did you realize later that you may have had ADHD?

I never went for an assessment but it's possible, considering I was fired multiple times throughout my career - also a common sign.

I'd suggest doing so. Also a sleep study to see if you have sleep apnea, because it also results in frequent job changes.

I unfortunately have both (although both are now being managed and my stability has hugely improved). There was a lot of drama before the diagnoses, and I'm still working through having taken it personally...

My high school physics professor gave us a choice: You can bring hand-written notes to a tougher exam, or no notes to an easier exam. I forget what we chose, but it opened my eyes to the difference between memorizing formulae and understanding how to use ‘em. I now have (I think) a solid grasp of basic physics.

Our chemistry professor had the opposite approach – memorize all the things and test that. I still don’t understand chemistry.

In college every professor encouraged bringing notes, textbooks, anything you need. The exam is there to test your understanding, not whether you can memorize a bunch of trivia.

The toughest exam I ever had was for the compilers class – 4 hours open internet do whatever you want as long as your compiler passes my unit tests. 2 questions: Add feature A and feature B to the compiler you built at tutorials this semester.

Absolutely no way you could cheat on that.

That's awful. Same grade for memorizing trivia like a bad imitation of a reference book, as for learning and applying the concepts?

Oh no the whole class would get the same exam. But he gave us the option, collectively

Yeah, same for my prokaryotic molecular biology class. The questions were 100 percent theoretical, and you were even allowed to take the test at home. After the first one, it made sense why — the questions are totally ungooglable, and working with anyone else would’ve immediately polluted my mind and been obvious.

> the questions are totally ungooglable

If you remember one of the questions it would be interesting to know what the current crop of LLM chat bots return in answer. If you know the correct answer to the question it would be especially interesting to know whether the prose the things return has any semblance of truth to it.

It's not that interesting yet. AltmanAI isn't claiming yet that GPT can solve unique problems.

Well, the issue with take home tests isn't the Googling per-se, it's the hiring out of the work.

My preference for university exams was a simple, single page cheatsheet that you had to make yourself. You could write the finicky stuff that is hard to remember on it, but you couldn't fast-learn the material like you could from a book. Though I do remember writing out little programs for the weirder stuff like required coverage over concrete reinforcement bars. Which felt a bit like cheating at the time.

Next step is to go wheel of fortune style, and provide a standard cheat sheet with the finicky formulas, and then let the students at their own supplementary cheat sheet

There is a deeper problem.

Yes, we can find creative in-class test methods immune to cheaters and GPTs. But if the skill you are testing has been commodified, the deeper problem is to justify learning it at all. Once you have the calculator, learning long division hardly matters. Most adults have forgotten the algorithm.

We're making the skill of writing, and much else, into a calculator.

But we don't know yet what the implications will be... the skill is writing is bound up with thinking and culture in ways long division is not.

> Once you have the calculator, learning long division hardly matters. Most adults have forgotten the algorithm.

Long division does have applications in some areas of maths, like polynomial division and factoring.

If you didnt know long division, you would surely not be able to learn about polynomial factoring. Knowledge is knowledge, and it does not expire.

Cool, my calculator with built in CAS can do polynomial division and factoring for me.

In learning and practicing division, you gain an intuition about how numbers are composed. This is an intuition that is useful when you’re trying to figure out if you should pay for that 50 dollar item, or if 6 payments of 10 dollars is a reasonable deal. While you could always bring out your calculator every time you make a decision, most people don’t and rely primarily on quick intuition instead.

This is more important than ever in the world of chat GPT. When you’re dealing with something that can quickly generate something that sounds convincing regardless of truth value, you need that quick intuition to keep up. So the question is, if you’re a chemistry major, how do you gain that intuition? It’s possible that generating lots of data from GPT and just checking it for truth value more painstakingly before handing it in is sufficient to eventually learn. But it’s also possible that you do need to personally learn the fundamentals by working through problems if you want to eventually gain the intuitions a chemistry major needs to have. While a deeper problem exists in the commodification…it’s hard to say how much that deeper problem affects the ideal way to teach people to be skilled in a subject.

I'll add my story to the list here:

One Intro to Physics (for majors and engineering) professor was the toughest and reveled in it. The class, in general, was a weeder class, so we all knew going in that it was meant to be overly hard.

The final though. My lord. It was hard, yes, but he put a little extra bit of psychological torture into it: It was a 'choose your grade' final.

In that, he had written four separate final exams. One was a D, one was a C, one was a B, and one was an A. You had to choose which final to take, and that would be the grade you got for the exam, providing that you passed that test. If you didn't pass you got an F. You could only choose one exam to take, no trying to finish the D quickly and then move up to a B.

I think only one person had the chutzpah to take the A level test. Everyone else struggled along with the Bs and Cs. Which, I unfortunately have to remember, were very difficult themselves.

Do you realize you're describing a problem that shouldn't exist? People cheat on tests because they are there for reasons other than to learn.

If AI can "level the playing field" then we need to change the game?

What do you propose? I read your comment as we shouldn't verify knowledge and just let doctors go yolo on their patients, which I assume isn't what you intended.

Not to be coy, my point is that education is one thing, and contests to see who gets the cherry positions in society are another, and when you combine them you get cheating. Which, of course, undermines the "verify knowledge" function of the tests in the first place, eh?

In my domain (computer programming) I know enough and have enough experience that I can gauge the skill and knowledge of my peers (and they mine.) That is how I get jobs. I go in and talk to the other programmers, and I can demonstrate that I'm "the real deal".

In re: "just [letting] doctors go yolo on their patients" I don't want to get into a long thing here, but I will mention that a strong argument could be made that the current system is not very good at getting positive outcomes. What I mean is, doctors and hospitals kill a lot of people. In other words, the existing systems to train and employ doctors are far from perfect.

(Not to put too fine a point on it, the way to find good doctors is to ask other doctors whom they would let doctor them, ya feel me?)

- - - -

edit: I realized I didn't answer your question.

What I propose is that we throw out the whole obviously-broken academic system.

I propose that people go to colleges and schools to learn, at any age, at any time, and they don't get grades or degrees or anything.

You get to be a doctor or architect or whatever by demonstrating to other people who know that you too know, rather than "earning" a proxy (degree or whatever) that can be faked.

If human life were simple enough that it could be covered by a system of tests and grades we wouldn't have these huge brains on the ends of our spines, eh?

> You get to be a doctor or architect or whatever by demonstrating to other people who know that you too know

That's just describing a test? A (verbal, written, practical) test is just a demonstration that you know X.

I didn't explain my meaning well enough, I apologize.

I mean a scenario where the candidate for doctor (or whatever) talks to half a dozen or a dozen doctors who then say "yay" or "nay" and if all of them say "yay" they are a doctor but if any of them say "nay" then they are not (although this doesn't mean that they can never become doctors later, they just have to address the failings first that led to the reviewing doctors to decline to admit them to their ranks.)

It's about humans communicating with other humans in a way that is extremely difficult to fake, as opposed to whatever testing system we're using now (that people cheat at) that produces (unreliable) proxies for actual domain knowledge.

If you just want to learn tests are unnecessary.

If you want to become a doctor the only distinction should be that other doctors must be willing to submit themselves to your care.

(This might sound crazy, but the person who is the greatest healer I've ever met IRL is not a doctor and would be laughed at by most of them, yet he can effect miraculous cures of major diseases, among other things. His art is completely off the radar of modern medicine.)

Education really serves as a dual purpose in our society of both providing knowledge and demonstrating in a actionable way (i.e. someone will hire you based on it) that you know something.

But most of what you need to prove is not something you actually use in your job. A A in compilers will help you get a job as it makes you seem smart, but very few developers will ever write one. So in that case, the A in itself is the goal.

oral exams seem like the only way

Reply on news.ycombinator.com