We see something that works, and then we understand it
lemire.me
lemire.me
Very flawed comparison. At work I get to go off and do research, experiments, can collaborate with peers and people who might have more expertise in a given sub problem, and generally have much more time. An exam trying to test you on material you haven’t studied is supposed to test for what? Your ability to synthesize knowledge out of thin air.
The rest of the article is well written and correct, but this particular aside felt weird.
A time limited exam is probably the wrong place for that, though, due to the stress interfering with that kind of thinking. It would be better for a homework assignment.
If ChatGPT didn't exist.
Okay, maybe in class, on paper is the right place for that.
Academia is going to have to focus on the latter anyway because we have AI, which is exactly the same: it’s only useful when you can evaluate the output for accuracy independently. But how do you do this without throwing out objectivity entirely? Maybe we need to move to apprenticeships?
Here's an example, consulting at a large Danish company, every Friday morning all departments in this big building would share breakfast and the bosses would say some things.
So this one morning they explained that in the coming months people should register time in a particular way because of accounting and how it related to a particular government grant and money that needed to be used up by a particular time in order to get to the next step of blah blah blah.
I realized as my eyes glazed over, damn this is just the same reaction people who don't understand browser rendering engines get when I start telling them about different events.
I also noticed other clueless people gamely trying to question these finance nerds on how things worked, and the patient finance nerds explaining some detailed bit and the clueless person clearly out of their depth with that "uuuuhhhh, hope they don't ask me if I understand" look on their face.
Now, if it hadn't been for them explaining this stuff I would have gone around thinking the boss is a complete moron. I once saw him mistake a nail gun for a drill! He doesn't understand how search engines work and why stemming and decompounding might be important, I know because I tried to explain to the idiot one time!! But since he actually talked about his work for a bit I realized he just happens to know stuff I don't.
I'm betting most of the morons you know are maybe not quite so stupid, although probably not as forthcoming as why things need to be done in a certain way to those who work under them.
someone, somewhere, at some point, will think i’m a clueless idiot.
we’re all clueless idiots at the end of the day.
Stress testing the student's academic prowess, if you will.
As someone who graduated high school, I'd hope my more accomplished peers would know the difference between hypothesis, theory and proof. It is entirely possible, and useful, to test someone's ability to form a cogent hypothesis. If you were faced with a question beyond the scope of the ideas you were taught, and could not rely on any assistance, the only useful thing to know about you is how well you would handle it yourself.
If you would synthesize knowledge out of thin air, that would be a failing grade.
Who knows the scope of the ideas I was taught? I mean, I could use derivatives and sometimes to integrate in 8th grade, should I get a failing grade if I used them in a test?
I see the point of your post, it is important to distinguish between what you know, and what you don't know, but maybe suspect. School tasks are built in a way, that the very form of the task can be used to make good guesses that may help you to solve the tasks. It doesn't work in reality, but it works in school. But the solution is to add questions without answers, not questions with answers beyond someone's knowledge. Like "what was Mozart's opinion on pineapple pizza?" or "what size was Julius Caesar?". Well, tests needs less obviously unanswerable questions than these, but you got the drift.
Edit: Thought it will be tricky with math. For example: "Alice has 3 apples, and Bob has 2. How many oranges they have?". School math problems have a convention, which means that if oranges where not mentioned, then they have 0 oranges. Compare with "Alice has 3 apples and Bob has 2 oranges". If the problem doesn't mention Alice's oranges, then Alice has 0 oranges. It is implied and these implicit rules are the first difficulty kids face when starting with math.
It's a false dichotomy between the "thinkism" bogeyman (actually reading books and papers and putting work into theoretical design is just bad now? Have they tried building anything in the physical world? Checked in with nuclear physics, ever?) and hands-on experience. Both are important. It should be about balance, not trashing an incredibly valuable set of tools because others exist...
Not everything I did I learned at school, such as navigating codebases with more than a million lines of code. But most things? Yea.
With that said, I am curious how people say that they learned much more through experience, what did you specifically learn?
Some I ended up using more during my career than others, but knowing more definitely reduced my tendency to think "ah, that should be easy".
I will grant you that the current form of the education system, especially in light of how easy it is to engage with the primary resources nowadays, is massively inefficient and needs an overhaul to provide more individualized learning. Something like Alpha School is promising in this regard. But don't throw the baby out with the bathwater. Even if a kid today learns entirely "on their own" with an AI teacher...neither them nor the AI accumulated that knowledge, and we still need that infrastructure of people contributing to advancing and refining the shared knowledge base.
That's a fair question and I'll do my best to answer this. It'll come in an edit. I think it's fair to say: not all courses were created equal in this regard but I'll do it course by course. I studied a bachelor information science but I tweaked my program so close to computer science that I almost daresay it's computer science (if I had 3 courses different, it was). I studied a bachelor in psychology. A two year master in computer science and a one year master called game studies (officially a specialization of information studies, but in practice it wasn't and it really was game studies as a whole field that we studied).
I'll try to do it in order per study program too.
AI-kaleidoscope: general overview of AI algo's. It's a shame we didn't know how to program or that we knew the usefulness of BFS or DFS but we learned it here. Not a useful course due to scaffolding issues (teach programming first).
Business mathematics: partial derivatives, etc.
Problem solving: useless course (teach programming first).
Privacy and security: we didn't learn much about security. We learned a thing or two about privacy. Should've been a TED talk, not a course.
Graph theory: graph visualization (and when not to do it, helped me out as a data analyst later), mathematical proofs, social networks (helped me to actually network a bit), graph algo's (made leetcode easier), not being scared of math notation. This was a really math heavy course as it was taught by someone that studied math in undergrad and grad and then switched to CS as his PhD (and by the time he taught it he was a full professor). He did not skip on the math which was wild since I was under the impression that I did a "business informatics" bachelor so I didn't need to have advanced math as a high school prerequisite. But I definitely needed that here so this course was hard.
Web technology: this was a bit too early but it was a good overview of web programming at the time. I remember being explained what the DOM was and I remember thinking "wtf is the point?" That was because they explained it way too theoretically and simply should've opened firebug or something to really show it.
Language of Logic and Methods of Reasoning: propositional logic and predicate logic. Practically speaking: after this course if statements are not a problem. This was true for me at least.
Pervasive computing: fun course but could've been a TED talk about how tech is used in interesting ways.
Introduction to programming: basic programming stuff in Java. I learned that Java is a terrible language to start programming in. I recommend JavaScript for app/web-oriented people and Python for "just pick a language" people. Nevertheless, while it was a terrible start, it did teach what it needed to teach which was a basic understanding, and skill, in programming.
Empirical methods: the better name is statistics 1. We learned about statistics and we had to program in R. Since our actual programming ability was quite weak it was a double course. It was the second language ever that I had to take seriously and it taught me a lot of programming stuff and statistics stuff at the same time. It helped that I had friends studying psychology at the time as that degree has a lot of stats in it, so I knew the lingo. Almost everyone else was hopelessly lost.
Programming project: no lectures, just one big programming assignment. After this it was expected that the student could write readable code. I failed this course the first time by a hair. And when I came back to it the second year I realized that I failed because my code was unreadable. So I refactored the whole thing and learned how to write much more readable code. We created the game engine for an Othello game. The graphics library was provided by the TA's. So we also learned to program with a library that was way beyond our heads at the time. And that also implicitly teaches you to trust certain abstractions.
Interactive multimedia project: we learned XIMPEL a hypermedia framework, kind of useless. But through XIMPEL we also learned about storyboarding, creating scenes and general video editing. This teacher was super hands off and allowed students to be creative. So I also learned to create simple PHP websites and learned my way around bash a bit. Then I decided to create an upload script where I used PHP to call the ftp command on bash. I thought it was impossible but it wasn't and had to rethink about what web applications could really do. None of these things were formal course requirements but this teacher encouraged this type of explorations so I do credit it to him that "I learned it in school". It's part of these efforts that he also gave me an amazingly high grade as my XIMPEL story graph was a bit meh but my creativity and extra explorations where a 10 out of 10 effort. So he gave me a 9 out of 10 in total. Also deepened my HTML/CSS knowledge.
-----
I'll write the rest later in another comment this is about the first year and I studied 9 of them. As you can see, I was just learning the basics here. But there are a few patterns:
* Some teachers allowed us to work on real stuff if the student chose to (e.g. Interactive Multimedia) and that got way more serious later on as I created an iPhone app for a client when the teacher taught Multimedia Authoring in the master.
* Some courses were quite useless or could've been condensed to a TED talk.
* Some courses taught something useful or semi-useful but it's not up to industry standard. But as you'll see, this will lead up to a level where - while not quite industry standard - makes the gap between industry standard and whatever I did small enough to just make the jump easily by simply applying what I thought was common sense.
In France at "prepa" (2 year intensive courses to prepare for exams for big eng uni) I learned the theory behind computer science (for example how to modelize a regex machine with graph / automata / matrices). That was useful theory to me, but that's just a drop in the ocean of uselessness
Academia is about pure learning about the world in a very deep and philosophical sense. It's about underpinnings and history and giving you the abstract tools to reason about things. Academics are often aggressively against learning practical things to solve a specific business problem. Which is what industry is all about. I think it's easy to take a degree program for granted, but it's difficult to understand how your brain would be different had you not done it (assuming one did apply themself and attempt to get its value. If you just cursed it the whole time yeah it probably was a waste.)
People like having a job and a paycheck and accomplishing units of utility. But kludging together some libraries and maintaining another CRUD app isn't all that important or interesting in the grand scheme of things. The Von Neumann architecture and asymptotic analysis are, though.
You will learn some things through trial and error entirely on your own of course, but you will also take a lot of time independently re-learning things that are well known. Which isn't necessarily a bad thing. I'm a huge DIYer. Sometimes you have to try and see for yourself. But I do acknowledge that I could have learned a lot faster about, e.g., carpentry, as an apprentice, than I did mucking about in my backyard with youtube videos. I think the concept of "balance" is more important than taking sides on this matter. They're both incredibly useful tools to be used in conjunction. Read the manual. Then go muck around. Then read the manual again and go, "oh now I get what they were talking about."
I think what I'm learning from this is that a lot of the "entirely self-taught" folks don't realize how much they've benefited from "thinkism" and standing on the shoulders of giants.
But taste about the best ways to structure a codebase? How to balance speed vs quality and when to do that? What does good vs bad observability look like? How to handle services going down in production and run post mortems? How to handle ambiguity both at a leadership and technical level to chart a path forward for a team? Even reading about these things from blog posts and things is nothing like actually experiencing these things first hand and certainly not taught. It requires accurate and constant self reflection and retrospective to guide an exploration process and if that self reflection tool is askew you can end up learning weird lessons and habits that are difficult to correct (and we all have them somewhere).
I’m team learning and was team academia early on, but the real value in academia is in teaching you how to think and to give you some minimal useful knowledge to about that capability. But it often struggles to even that because it’s still following an assembly line model of teaching whereas learning is relational.
Source: nearly 20 years as a SWE from a well regarded university.
Ambiguity: there is a lot of ambiguity at uni as well. So you should have had the practice, especially working in groups of students.
> It requires accurate and constant self reflection
Skills I sharpened at university by studying psychology. Do you know how big the mind fuck was to grok the reproducibility crisis while being a 3rd year student in that program (5th year student overall)? I had to re-examine my whole degree.
I also had courses where self-reflection was a weekly part of the course. And I took that seriously.
> whereas learning is relational
I consider it a mode.
Try mathacademy.com it will teach you the “shut up and calculate” version of math really well, and there is nothing relational about (since it’s an app).
I also had courses that were about computer and network security in C and assembly. In it, I self-taught vim, assembly, C and git. The self-teaching was partially the point.
One thing I have learned from university is that what many people see as good practices is more akin to a culture rather than an actual good practice, because the actual empirical work in figuring out what works hasn’t been done. And I find what you say to strongly fit a particular culture of software engineering. But there are many forms. For example, in the Smalltalk/Pharo community it is recommended to extend the base classes to your liking. Whereas in other languages they look at you with horror. Back in the day (2012) Airdrop was what people used and didn’t want to use git, and these people have programmed for decades (I disagree with that one personally, just showing the culture).
Design is useful as a tool to sketch out a plan but it can definitely become thinkism if you always hand off your design to someone else to action.
Also, the author is talking specifically about research type problems where there’s a lot of unknowns and the problem space is not well trodden and understood.
Also, I don’t know when the last time you built anything is, but talking with mechanical engineers and construction engineers, it’s all the same. You have your designs and then you open up a building to do repairs or start construction and some surprise presents you that you weren’t expecting. That’s what the research phase is for where they learn all they can about the problem space before they start designing (in this discussion ignoring all the years of trial and error that built a base of experience already) and even still in practice problems arise.
And yet, the field of architecture is still highly valuable. Without those designs, there would be a lot more problems. Try building something complex without a design. You're going to waste a lot of time and materials. My point there was that it's not like the digital world where mistakes and trial and error and rethinking things can sometimes cost almost nothing.
Theoretical design is pretty worthless until you try it out. Until then it’s just an idea or hunch you have....ignoring all the years of trial and error that built a base of experience already
Lots of (smart) people have tried lots of things, there is a ton to learn from that, you're calling it worthless and taking it as an assumption at the same time.
The valuable thing about hands-on experience is that the real world doesn't let you ignore constraints. You get that feedback quickly if you are paying attention. This in turn allows you to build a more accurate mental model of the true nature of the problem you are solving and where the hidden limitations and leverage points are. A lot of academic literature tacitly works from a set of assumptions that don't map to any real-world environment.
Once you have that hands-on mental model, the flaws and limitations of much of what is in the academic literature becomes obvious from first principles. Most of the insights might be academically interesting in a theoretical sense but they often aren't reducible to useful practice in real systems. Non-academic careers require implementations that actually work well.
[0] The computer science literature from the 1970s and earlier is much better in this regard than what came later. Many early papers were written by people that clearly had concrete experience in the trenches with the problems they were writing about. Those papers are both more readable and more applicable. This awareness of constraints is lacking from a lot of modern computer science papers.
I mentioned nuclear physics because it's a wonderful union of theory and practice. The experimenters need theories to test, and the theorists need their ideas tested.
Contemporary AI is massively driven by research. There are a handful of influential papers from the past few years that have gone right into practice. Industry players have famously invested in their own academic divisions.
We have a literature full of spatial indexing algorithms that can't work in any real system because they assume cache replacement algorithms. This problem isn't even mentioned in modern academic papers. That is extremely low-value research. That's like doing physics research under the assumption that the fundamental laws of physics don't apply. It might be an intellectually interesting exercise but it isn't useful.
It isn't all like this. The spatial indexing literature is actively bad to an unusual extent. If you look at e.g. academic graph analytic algorithm research, where I also worked, it is mostly just decades behind the non-academic state-of-the-art. The literature won't mislead you but it also won't tell you where the frontier is.
This article[0] says "Data changes are usually much less frequent than queries, so incurring an initial cost of processing data into an index is a fair price to pay for instant searches afterwards."
Is that what you're referring to with cache replacement? There's no way to quickly update the index?
Wikipedia says "The R-tree was proposed by Antonin Guttman in 1984 and has found significant use in both theoretical and applied contexts."
I'm gathering that there are applications which are ok with that limitation? Generally there is communication and awareness between industry and academia, if everyone really needs cache replacement for it to be useful, why have they not attempted to account for it? What is the cause of the total disconnect you describe?
[0] https://blog.mapbox.com/a-dive-into-spatial-search-algorithm...
New stuff gets explained in class by the teacher - the details are for self study. This is how it was communicated to me in school, so I would call it part of the "contract".
But giving a homework on a fundamental new subject without explicitely telling that this is what it is about, will rightfully confuse, as the normal mode is different.
And when it happened to me, it always happened because the teacher messed up and got confused themself what they had and have not teached before.
Teaching to self study is a valuable skill in itself that should not be taught by accident, but with a clear pedagogic concept.
Just being upfront with people can break low performance patterns of behavior.
I got through college, a very well-regarded engineering program, putting in the bare minimum effort to graduate.
I make a lot of money now, 20 years later. I attribute none of my success to public education or college, I didn’t learn anything besides our education system (in the US) is not what they tell kids it is growing up.
We beat the drum of education because it seems like a nice simple solution to fix a lot of perceived problems. “Oh you mean if we just throw money at this problem it’ll fix it?!”
Baltimore city gets some of the highest per-student federal funding in the country. My daughter is in a public middle school there, living with her mom. I’m in the burbs with my wife and kids.
She told me just yesterday she wants to go to high school in my district because “well it just seems more normal than city schools”
She also thought California was a country up until yesterday. Straight A student.
At some point, probably too late, we’ll realize we did it a wrong. Floating kids through high school, dropping an anvil of debt around their neck for a college degree they won’t use. Met a fellow the other day, he has his masters in social work. Making very, very good money, as a regional manager of a car sales group, I forget the specifics. Said he never used his degree and he is very grateful he has a clear path to pay the debt off.
I don’t know what the fix is, but I’ve seen enough to know what we do now, ain’t it.
Its strange to say, but its actually thanks to black racism is why we're here.
It started with https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co.
The result was that the courts determined that "aptitude/IQ tests" unfairly discriminated against black people. However, education requirements are not seen as such.
So, we get "decent paying job" requiring 'a degree'. What degree? Doesnt matter. The degree is the legal form of 'we dont hire blacks'. It also has the side effect of 'we dont hire the poors' as well.
Obviously there are positions that require advanced schooling. Im not talking about those.
Griggs says that you can't employ IQ tests to discriminate for jobs where the IQ threshold you established is immaterial to the job. For modern knowledge work, that's a trivial bar to clear. It is essentially the case that Griggs is a nonfactor in white collar employment.
My motivation for calling this out isn't that I'm OK with discriminating against Black people; quite the opposite, in fact. IQ tests aren't used more widely because they aren't effective at qualifying employees. IQ is an idee fixee among A Certain Part Of The Internet, and part of their pitch is that IQ tests are a secret cheat code that would enable hiring purely on merit, if only the woke wouldn't prevent it. No, false.
Wow. I was never able to escalate situation to the direct accusation of cheating. I just got my marks lowered a bit because I used methods that were "above my abilities", as my math teacher believed. Though really I thought school math was boring and tried hard to make it more interesting, less routine, less relying on memorizing things, and more clever shortcuts from general principles.
But she never accused me directly. I know her explanation for lowered marks just because my mother visited her once with a specific test results and asked why marks are so low. I wanted a direct accusation, I was trying to provoke her to doubt my methods, but either I tried too hard and she knew deep inside of her the real situation, or I needed to do something truly exceptional and failed.
Yeah, thinking about it now, I should probably try some existing and very known methods but far above the school level, something she felt herself familiar with, so she could hope to prove that I've used methods without understanding them. And then it would come to my ability to prove, that I understand my methods. Something like raising e^(ix) into n-th power, to derive a formula sin(nx) = f(sin(x), cos(x)). Though, it is might be too weird for her.
> They do not reward creativity or ingenuity.
You can stimulate yourself if you look at it as a rebellion against the system. Teenagers are good at rebelling. Though, rebellion can make matters worse. I rebelled against history trying to get the lowest marks possible. It didn't end well, and I not just didn't learn history, I've got a life-long trauma, I start to regress to my teenager mind level every time I face history.
I'm so lucky such a thing never happened to me. The closest thing was a math teacher in middle school asking me about a problem solution of mine in a test, that he had marked "mysterious" (I had to come up with my own solution path during the test because I hadn't memorized the canonical one we were taught). He asked me to explain my reasoning, and when he was satisfied that it was sound he gave me full marks for the question.
I failed some, did well on others, felt a bit of unfairness emotions completely understandable for that young mind which just wanted to do the test and move on. But when looking back, they were good questions, testing my actual skills in topic I was studying.
> As a teacher, I can tell you that students get really angry if you put a question on an exam that requires a concept not explicitly covered in class. Of course, if you work as an engineer and you’re stuck on a problem and you tell your boss it cannot be solved with the ideas you learned in college… you’re going to look like a fool.
I really hope this kind of thinking dies away because the longer you wait the more money you waste, not just your own salary but the opportunity cost of time. And this person is a teacher?
Of course you would not test a completely unrelated concept but where I grew up "Transfer of Learning" is supposed to be part of every test, with a proportion that increases as students progress through school.
To test pupils ability of transfer of taught concepts means to let them apply information, strategies, and skills they have learned to a new situation or context.
Yes, actually. Synthesis is the penultimate level of Bloom's taxonomy [1]. Perhaps not appropriate for a first quiz on new material, but absolutely in scope for a final exam.
I had many exams in university that challenged me to synthesize a proof to a hypothesis I'd never seen before. I had to bring together my knowledge of the material I'd learned in class with the bit of new material presented in the problem statement, and then devise a proof with one or more steps I'd never done before.
Many of my fellow students in first year struggled mightily with questions like this. There was much wailing and gnashing of teeth after the exams. In later years we'd all gotten used to it, and it was to be expected. That students in high school or younger years have been sheltered from synthesis-type problems on exams is a great disservice to them.
Edit: I should also note that outside of upper-year pure mathematics courses, synthesis-type problems rarely accounted for more than a third of the grade on an exam.
I'd be furious to get some lofty theoretical synthesis problem on an engineering exam where the other problems are grindy analysis of "looks like concept you know but expand into a form you haven't seen and then will take 1/n time on your n question exam". I suppose these are already 1 extension in, so 1 more logical extension isn't that far off
Expecting a final exam consisting only of calculation and memorization type questions is wholly inappropriate in a mathematics course full of proofs.
Off-topic, I know, but I always disliked the time pressure of exams. All the bad scores I got were down to not feeling quite right at the time of the exam: maybe I didn't sleep well, the food I ate did not quite sit with me, had a fever, etc. None of these things actually impair too badly but if the time is short, there is an impact.
I suppose it's similar to sports where you have to, at least somewhat, adapt on the go. Many teams botch a game here and there. But... your final exam? Botch it and you're screwed. It just does not feel right.
maybe this is good for weeding out students who would not be a good fit for high stakes speed math.
It's really no different from a software dev interviewee who can't solve basic programming questions without being handheld and coached through the entire process.
I didn't get those exams in math courses. Engineering courses I did get those where some of the point of the test is the speed of your pattern matching ability (did you use design skills to reason about it or did you brute force a solution).
Math you're already extrapolating theory and you need ample time for that.
That’s what the exam is testing for. If you need unlimited time to write the proof, then you haven’t studied the material yet, and should be expected to perform poorly on the exam.
It’s like the difference between someone who has never touched React before the interview, and someone who has been programming with React for several months. You expect the person with several months of experience to know how to solve problems with React several times faster than the person who doesn’t know it at all. If you’re hiring and looking for someone with a minimum of 1 year of React experience then you shouldn’t expect the person to take a full day to do basic tasks with React (especially if you the interviewer or anyone else on your team can do those tasks in just a few minutes).
Maybe it has been a while since you have worked on something difficult, then? Difficult for you, I mean. Maybe the difficult things are now easy.
> If you need unlimited time to write the proof, then you haven’t studied the material yet, and should be expected to perform poorly on the exam.
I mean, maybe. I remember some classes and exams where I had simply never seen the topic before, and the problem relied on having picked up on metapatterns that weren't explicitly taught during the class and applying them in novel ways on the spot. Though my guess was that at least some of those problems were markers for "if you can one-shot this, we want to know".
> If you’re hiring and looking for someone with a minimum of 1 year of React experience then you shouldn’t expect the person to take a full day to do basic tasks with React.
If you're talking about the basics, then yeah I would agree. I guess I don't really know what counts as a synthesis problem in this context, and if we were looking at specific examples it would be more obvious why one might be fair and another might not be. I've heard that line about TAs being touted as being able to solve the tests in just a few minutes, yet somehow unsolvable problems fell through the cracks to the students in the same class more than once in mine.
That's the crux of it, really. For a lot of students, the kind of proofs we do in undergrad pure math courses are very difficult. For a math professor? They're extremely basic.
There are levels to this. If a typical high school math student is at level 2 or level 3 (of math ability), a 3rd year pure math course may be at level 10, but the professor teaching the course is at level 100. Okay, so these are made up numbers, but I hope they help to illustrate.
For someone who has never tried to write a computer program in their life, writing Fizzbuzz may be quite difficult. For a working lead developer with 10 years of experience? Quite basic.
I quite fondly remember a situation with a two question three hour open everything exam where I stared at the 1st problem and relevant equation for 20 minutes because I had no idea what I was looking at and was wondering if I should just leave and drop out.
Then I realized that a bunch of terms could cancel out and the problem became 3 lines trivial. The professor later admitted he wanted to give a "easy" question that you couldn't just copy out of your notes and had to think and to spend the rest of the exam on the actually difficult one.
Being able to do deep thought quickly isn't a valuable skill, and you don't pick it up on the way ti mastery with lots of practice like you do the speed for rote problems
If you really want to grade the students its fair to put very hard problems on the exam to separate out mastery from merely very good understanding or recital.
What skill are you grading? Can you expect some students to be able to churn out the problems? Are they doing design work? Is it more inventive or more creative? Is there more than one solution? How rigorous is the solution?
My point is that you can design an exam to test for speed and ability to do the day to day well. You prepare for this exam with lots of practice. There are a few problem types but you might not have seen every one. There's a basic level of extrapolation you're expecting.
You can design an exam for abstracting theory, and you prep for this exam by doing the hardest problems you can find, reading more theory, etc. You might not have seen anything like this but the test is to use the tools you're given in a class well. The test is the extrapolation.
I had theory courses where I expected the latter and practical courses where I expected the former.
There's more than one type of mastery; I bet your math professors would get destroyed by gifted 14 year olds in algebra competitions that involved arithmetic and timing. Do those 14 year olds not have more mastery?
Undergraduate exams tend to be short. Which means that a perfect grade should be interpreted as "meets the expectations".
None of the mathematics exams I wrote during undergrad at Waterloo (as a math major) would fit that description. Nearly every single one of them had midterm grades with unimodal distributions centred below 70%, tending toward 60%. Typically, only 1-5 people in the class (of 100-200) would score a perfect grade. In upper year pure mathematics courses it was common to not have any perfect grades (in a class of about 20-30).
In my experience, studying mathematics is a bit weird. If you are ready to learn a topic, it's probably the field where you can get top grades with the least effort. But if you can't learn something with reasonable effort, hard work is unlikely to help. Doing something else and trying again after a few months might help.
Depends on your school. At my school, someone who was not super-talented in math but who works hard and is actually smart about studying is a 70s-80s student. The students who got 100s were basically IMO-level elite mathematics kids who were heavily recruited by the school and given full ride scholarships.
The course exams were essentially designed so that a "meets expectations" was a final grade of 65%. A grade of 100% is someone they were looking to recruit into research internships, grad school, and potentially a tenure-track position.
The skill the class is teaching is the ad hoc synthesis of knowledge from other building blocks. That kind of reasoning is most of the purpose of the class; the concepts they teach you are in some sense just a foil for teaching this. The derivation from the concepts you were explicitly taught is pretty trivial and you've had many opportunities to practice.
This is common for any kind of systems-y domain. These types of questions were standard on my (physical) engineering exams.
And learning what stripped threads are....
Part of learning to repair things is also learning how to not make them more broken.
Screws and bolts have to be stretched to work. torque wrenches are used to enforce "just enough" tension without too much and that's why we use them. Over tensioning can break the bolt by snapping or strip the threads.
What's frustrating to me is that I learned to repair things, but there is SO MUCH practical knowledge to fix your iphone font size, but it is much harder for you to run into the real practical knowledge of life, like how to turn a nut/bolt/screw.
Wish there was a university of this stuff. how to use a wrench and when power tools. how to ask someone on a date. how to cook easily. how to choose cool clothes to wear.
1. "Thinkism": As described, over-engineering before writing code for a complex system and seeing where it takes you. Maybe decision by committee, or just overthinking. But its like one form of replacing on-the-ground adaptable, creative thinking, with a dumber process.
2. Which should be completely separate, it's saying that students are mad if they're forced to think for themselves. This is a complaint about underthinking and the tendency of inexperienced coders not to come up with a grand plan before writing a line of code.
So which one is the problem? I'd say the problem is not knowing when to over or under-think something.
Also buildin' stuff! (Which is the best type of doin'.)
It is guaranteed failure mode of large orgs. Curious to hear about more references on how to fight this at an organization level, besides the one given in the OT.
Not everything need to be made so easy to refer, like using three or four of words instead of one..
Portal (a puzzle game by valve) had levels built in such a way that it introduced the player to new mechanic, and only then building on top of that
Cuz in real life also its more about "doing" you're physically fixing a clock, or writing code, or designing a building
Doism shouldn't be 100% but it certainly should be more
We definitely should try to improve and experiment with any system, including education, but I really doubt it is that easy to improve education and it will depend on objective, culture and political environment more than doing A or B.
AI improves our problem solving in another way though! E.g. I can now quickly spin up frameworks to test/eval my hypotheses. This speeds up our ability to trial-and-error towards progress.
Well then I think you omitted a rather important topic in your teaching: that the purpose of teaching is to provide a toolkit with which the student can extend their abilities.
Thank you for this term. In my view, the belief that AGI singularly will rapidly destroy us because it will think 10,000 times faster than us is a form of thinkism.
> Thinkism sets aside practice and experience
thinking succeeds experience & precedes practise, its not apart from it