The Illustrated Guide to a PhD
matt.might.net
matt.might.net
To those in the comments who mentioned you are just starting your own PhD: Good luck to you! And, I hope you, like I once did, find a problem that you can fall in love with for a few years.
To those just finished: Congratulations! Don’t forget to keep pushing!
To those many years out: You have to keep pushing too, but there can be tremendous value in starting all over again by pushing in a different direction. You have no idea what you may find between the tips of two fields.
I'm sure over the years you've known students who have started a PhD and not finished. What (if anything) have you said to them? Do you feel their efforts had any value?
Disclaimer: I have no idea what I'm talking about. I've never participated in a graduate program.
This is just to say I found it incredibly compelling and moving; I hope mentioning it doesn't make you feel bad.
My son’s life changed my own in profound ways, and even though he died four years ago, he is still changing my life in profound ways. I am always grateful for the reminder and to reconnect with the purpose that his life gave to mine.
That post also reminds me that while he was alive, I did the best I could for him under my abilities, and that’s all any parent can do in the end.
If you want to know more about his life, I wrote on it here: https://bertrand.might.net/
This rings true for me at this time. Done about 10 years now, never went into academia but direct into industry. Things seem a bit stale, maybe its time to pick and research something new. I've been hesitating on the "going back to school" thing. But quantum does show promise, for curiosity and potential rather than immediate impact.
Thanks for the articles!
I’ve read through them and they are timeless.
Also the finding cures pivot from CS was inspiring how I could start one problem space and pivot. Definitely a top computer scientist story.
- people torture data until it yields unreproducible results;
- people choose venues that maximise their chances of getting published (and pay for publication sometimes, I'm looking at you, APC);
- little concern given to excellence, rigour, and impact;
- the chase for a 'diploma' from a renowned institute without putting the effort;
I could go on and on, but I'll stop now.
Perhaps something changes, I am waiting for this to happen for some time now (10y and counting).
It's a bad system but that's what we have (at the moment).
It's because the kpis of assessment are built like this. Goodhard's law. I know lots of good researchers who get frustrated with the system and end up giving up and faltering to those 2 points. If within a uni 2 research groups are putting out research at different rate at different quality, the one with higher quality, lower frequency, and higher standard and ambition gets heavily penalized. Seen it in action.
There is, "You get what you pay for." So, want papers, you will get papers, and you can count them. It goes, did Haydn write 101 symphonies or 1 symphony 101 times?
Early on, had a good career going in computing but where occasionally some math made a lot of difference. So, to help that career got a Ph.D. in applied math. Never had any intention of being a professor but for a while did to try to help my seriously, fatally, stressed out wife (Valedictorian, PBK, Woodrow Wilson, NSF, Summa Cum Laude, Ph.D.) -- took a professor job near her family home and farm.
In my little Ph.D department, saw the Chair and four professors get fired and one more leave, fired or not. The career I had before grad school was a lot better than the one those professors had.
Had to conclude that, tenure or not, being a professor is, on average, a poor way to even reasonable financial security. Generally there is low pay, e.g., too little to buy a house, keep cars running, support a wife and family. There's a LOT of dirty politics, infighting, higher-ups who don't want you to be successful.
Bluntly, a research university takes in money that a lot of people care about and puts out papers that only a few people care about: Net, there is no very strong reason to pay professors enough for even reasonable financial security. Key sources of the money are short term grants from the usual suspects, NSF, DOE, DARPA, NIH ("too many for them all to be turned off at once" -- JB Conant?), but that is essentially just contract work and not steady employment, with little promise that when a Professor's baby is ready for college there will be money enough for them to go. It's a house built on paper that can be blown away by any thunder storm.
Now, for a career, e.g., financial security, to leave something for the kids in the family tree, regard business, e.g., now involving the Internet, as the best approach, and there regard computing and math as important tools but only tools. Research? Did some, and it is a key to the business. Academic research? Did some, published, on my own dime, still waiting for the checks.
History, how'd we get here? Used to be that some guy built a valuable business and had several sons. One of the sons inherited the business, and the rest went to the military as an officer, academics as a professor, or to politics. Then WWII showed that the STEM fields can be crucial for national security, and some related funding started, e.g., summer math programs for selected high school students, research grants.
"If you are so smart, why aren't you rich?"
Ah, "The business of America is business."
That's not true, the university pays you salary. Depending on details you may be able to increase your salary using grants, but you won't lose your salary if you don't. And when you hit tenure it is very hard to fire you
When I, an expert in the field, tell them they need to produce something novel at their research panels I’m told I’m wrong. When I list all the work they are ripping off I’m told it’s somehow different without explanation. When I question the obvious sloppiness in their work (the simulation data showing major artefacts) they blow up at me screaming and shouting.
I’ve never experienced arrogance like this before. It’s shocking. Their supervisors tell me that they are close to firing them but then also celebrate all the publications they are getting.
The mind boggles.
Maybe it's time we unshackled ourselves from these 'prestigious institutions'?
They’re using an industry tool to do well trodden industry problems that were solved by academics decades ago.
I’m not tolerating his behaviour and I’ve made my views clear to my colleagues. He’s going to burn every bridge possible with this behaviour.
I think Socrates was a hoot, and he taught in a cave or something like that.
Priests teaching rural peasants to read in their monasteries, and collegial colleges for the public benefit are definitely meritorious.
But,I mean, there is enormous corruption going on.
How did Ren Youzai get into MIT? He was a body guard. Just because you've married into a billionaire's family MIT says "hey, send anyone you want in"?
And I'm sure MIT isn't alone in mysteriously average students who not only get in but graduate when linked to massively rich and powerful families. A recent US president comes to mind. Is that anti-intellectual?
And the "large numbers of students" covers up the possible cronyism and/or corruption of the institution.
I provide an example of a totally unqualified individual being allowed into a prestigious institution solely on the basis of his marriage family. Your response is that they mostly do a good job for most people?
I've suggested that the research they do is not obviously beneficial to anyone except perhaps the person doing the research, possibly simply to advance their own careers (in or out of academia). Others have suggested the same.
You haven't disagreed.
And I have no hobby horses, just a high horse, and you better hold your horses or else you'll be just be whipping a dead horse.
Your unwillingness to defend and advance your position is duly noted. Have a nice day.
I'm not sure whether you're joking or serious, but in any case, Socrates didn't teach in a cave, and you're probably referring to Plato's allegory of the cave.
The interlocuters and followers of Socrates were mostly the wealthy elite of Athens.
I think I was mixing him up with Aristotle, e.g. https://www.ancient-origins.net/history-famous-people/caves-...
who had some cave school or something.
but there was some jokingness, yah. But I'm not anti-intellectual, which wasn't a joke.
I wasn't making any point about his students or wealth. Education then, as now, is the plaything of the wealthy and wealthy nations.
(first quack) "For example, in 2022–23, the average total cost of attendance for first-time, full-time undergraduate students living on campus at 4-year degree-granting institutions was higher at private nonprofit institutions ($58,600) than at private for-profit institutions ($33,600) and public institutions ($27,100).4"
https://nces.ed.gov/fastfacts/display.asp?id=76
Or, if the requirement of a college degree for high school level work suggests, an expensive barrier for employment?
(random quack on the topic I could find) https://www.vox.com/policy/23628627/degree-inflation-college...
You can say luxury sports cars are "accessible" if you want to finance a $150,000 car. And effectively that's what many (most? all?) college degrees are: luxury sports cars.
Do you think criticism of religious organizations is “against god”?
At risk of relying totally on assumptions, that wouldn't be a surprising reaction for someone facing first serious criticism after an entire life of probably being unconditionally lauded for their smarts (or the projection of it). When parents push children towards something relentlessly without providing any constructive feedback on account of living their dreams through their children and/or the fear of discouraging the child, any criticism can feel like someone is trying to destroy your life goals.
> Their supervisors tell me that they are close to firing them but then also celebrate all the publications they are getting.
Probably trying to protect themselves from being in the crosshairs of one of many things that can blow your career apart.
Some from reputable universities. I have no idea how they defended them.
I think there is value revisiting some of this work with our modern toolsets and publishing the code in some public repository.
But of course with a clear citation chain, and no pompous lies that a new discovery was made.
When I made the point that there is no scientific novelty here they insisted that their PhD was a ‘generic’ one and that means they can continue to run basic simulations according the to the recipe.
What will change is that PhD will become an inherited title. If your parents were/are PhDs, you will ceremoniously be granted the title when coming of age. That title can then be rented out to people or organizations (such as companies) who are required to have a PhD by government regulation for the activity that they are in. You can of course also mortgage this title to a bank or other company that will take care of the process.
"There is no journal of negative results." he would say at our weekly meetings. In order to secure his future, he set ablaze the dreams of 5 PhDs in my lab (all of which took their masters and went into industry; One developed severe OCD). Data was massaged, lies were told to his bosses.
Guess what? He's still a professor there, his lab still publishes dubious, unreproducible research. No recourse was to be had at the university (all of the PhDs went to the head of the department and were told to f*ck off).
Academia is on a death spiral at many schools, and I worry that it's up to the industry to carry the torch of research in the future.
In the kindest possible way: screw all of you!
If anything, take it all with a "grain of salt" and reflect on whether or not anyone you meet might resemble these comments. Hopefully, not your future self.
Good luck in your career.
If you have a good advisor, your passionate about your project, and you got some good funding, you'll have a wonderful time of exploring interesting ideas and becoming a competent researcher. Good luck!
> You still have to be realistic
I'm expecting it to be very challenging. But that's the point — isn't it?
Good luck to you too.
It's hard to believe this 'sucking up to those with money' thing applies everywhere, though it's easier to imagine it applies in certain domains.
Physics (since it's supposed to be rigorous) seems like a less likely area than some to be driven by politics and trends, but I suppose I can imagine that competing research programmes and ideas benefit from a certain amount of marketing and smooth-talking of people with funding rather than relying purely on empirical evidence for their claims.
Physics may be in some sense more falsifiable, but it is absolutely subject to politics and social norms, both in how it lies about itself for money, and literally in which theories are chosen (since we can rarely empirically distinguish between them)
Academia is rare for having the engineer also be the salesman
My wife is the lead mechanical engineer at a small company and she definitely doesn’t have to go around convincing customers they need her products
Everyone else basically had to reformulate their research to pretend it was applicable to the government's funding subject de jure. This led to some quite large stretches in definition to achieve "<Main area of research>, and some applications in <funding stream>". This very much felt like it was sucking up to money.
I got out of the academia in the end because it felt like the more senior I got the more time I spent applying for funding and managing the spending and the less time I spent doing research/development. (Also given I was in a UK public sector institute, the pay was shit due to 40 years of below inflation pay rises crippling the institution).
I left because the only path forward here in Germany is to become a professor, aka a life full of admin and sales
As the commenter above observes, physics is (supposed to be) falsifiable, so it should be clear when you have a result and when you don't. In the some of the more 'wooly' disciplines, this is not the case. You can write BS and as long as you're able to argue sufficiently eloquently that your particular strain of BS is valid, you win — in some cases, you needn't even supply data or perform experiments. It is in those fields that I assume the forces of politics/fashion/social pressure are strongest.
There is a technical floor to participate, but to potential funders all the physics project proposals from Physics professors sound equally probable. They are going to choose projects based on internal initiatives (fad), name recognition, track record, etc.
I feel like people are needlessly bitter about all of this stuff. Life isn’t fair; no one ever said it was. And why does anyone expect that you’d be able to get research funding without having to form relationships and make some effort to sway influential people in your direction? Yes, it’s not ‘pure research’, but it’s still part of life. I don’t see how it could be any other way… unless we get AGI as promised and then we’re free to sit on our backsides all day and become philosophers with infinite funding.
As I mentioned, my wife is a mechanical engineer. "All" she needs to do is do her work, and her manager will be happy. Going out and selling to customers and convincing them they want your product is not her job.
In academia, you can do the work, you can know it's excellent and groundbreaking, as I did for my own work, but unless you go out and sell it, no one cares. You can't just do science, you also have to do sales.
And, to pre-empt a response, yes, it is true that you still have to "sell" the idea that you've done the work properly to your boss in the industry, but it's totally different to in academia, where you will very often be in the situation where no one even knows that you've done any work at all, let alone is expecting something from you. Academia is just a very different working dynamic. Much more independent, much less collaborative, much more responsibility, much less praise
I have a PhD, got an academic position and then worked in various companies (startups, big tech company). These paths aren't exclusive.
I'm glad I did the PhD.
- it gave me time to work on a variety of interesting topics. In my company, I always feel rushed and don't have time to learn as much as I'd like to.
- I had more than one career. Working only in industry after graduation would have been pretty sad I think. Not that it's bad but it's great to see something different
- I developed some skills (for instance talking in front of audience, write scientific papers) and got to meet a lot of interesting people, and worked in different countries.
I also learned that research wasn't for me but it was worth doing the PhD anyway. If I had to do it again, I would pick my topic more carefully, and go straight to industry rather than pursuing an academic position (which I actually didn't like). Also money wise, even though I'm not materialistic, the pay was too low. Certainly enough to live, but not enough to secure my future and retirement.
Best of luck for your PhD! You might want to check out this ted talk: https://www.ted.com/talks/uri_alon_why_science_demands_a_lea...
I'm only very junior, though, so I don't have total confidence that I'm right. But I'm pretty certain I am.
A math PhD might take 6-7 years to complete and I hope that, at the end of it all, you won’t have to come to London to look for C++ or Ocaml jobs at hedge funds or banks.
...this is the discouraging negativity I'm talking about. I do, respectfully, wonder what your agenda is.
I am not trying to discourage you, just a different perspective.
I hope you achieve good things, and have fun while at it!
Every country is different.
Then you're going to have a great time during your PhD, good luck and have fun!
> screw all of you!
"Disregard!" https://stepsandleaps.wordpress.com/2017/10/17/feynmans-brea...
I wish you the best of luck for your PhD, a caring and supportive advisor, and great results!
Yes, a highly motivated college dropout with a computer, a strong financial safety net, and the right social connections can be in the right place at the right time to seize big opportunities. Most people are not in that position. Many high-impact technologies need more than what just a computer can do.
The main thing is to be self aware enough to know the path you’re on, what paths are available to you, and how to make the most of the connections and resources you have available to you. The second you start to get pigeonholed, wrap things up and move on.
That seems like good advice.
I hope you have a similarly rewarding experience. You will encounter unfair systems and unscrupulous people, and it will be frustrating. The data will be confusing as hell. My only advice is stay true to yourself. Maybe look into some of the new trends that could fix academia -- pre-registration, open access with public comment periods, reproducible code, etc. For inspiration, I cheer for crusaders like Data Colada who are trying to save the academic system.
Also, this is HN, which revolves around an occupation -- computer programming -- that is unique in terms of having high demand while remaining flexible about how and where people learn their skills. Not all fields are that way.
I got a PhD in physics, in 1993, and have worked in industry since then. There are a couple of "negatives" that I still think are wroth pointing out:
1. PhD programs have very high attrition, and you bear most of the risk on your own shoulders. It's worth going in with eyes open, and knowing the risks. Getting out with your PhD may require some compromises along the way. I won't necessarily call them ethical compromises, but perhaps compromises to the (typically) idealistic views that many students start out with.
2. The little nub of specialized knowledge shown in TFA is your research, not your brain. You can do specialized research without becoming a specialized person if you want. This is a personal choice (academic freedom and all that). My dad, who also had a PhD and a good industry career, always told me to avoid hyper-specialization.
Don't forget to learn how to code, just in case. ;-)
How many people do you know who “failed to meet the standard”? Zero. If you do the time and work for your professors you will get the reward. There is no risk.
> PhD Envy" is a real part of office politics
The most vocal critics are not bachelor degree holders, but those who did it and had a bad experience.
It could be because you realize you don't really like research - that involves reading and writing a lot of papers, going to conferences not just tinkering. It could be because you had the wrong professor who failed to lead you and left you by yourself. It could be because you gave up at a low point, where most PhD student go through. It could be because after 4 or 5 years your professor keep saying "you're not ready yet" (I've seen that in humanities).
So it's not really a problem of "not being good enough", but it definitely happens.
1. Your experiment fails to produce a result after a few years of effort (my project, we don't know to this day what went wrong, and I was lucky to find a new project).
2. Loss of funding or institutional support. (A large program at my state's university pulled its support for a process that required regulatory approval, and an entire group of faculty and students all had to leave.)
3. Your advisor quits, changes jobs, gets fired, goes to prison, dies. (Many cases).
4. Your advisor holds your thesis hostage until you publish a certain number of articles (a friend of mine, she sued and won).
5. Mental health issues (high incidence of clinical depression).
6. Personal animosity between members of your committee (another friend).
How these risks instantiate themselves is that you have to start from scratch, often with a completely new research project, and finding one isn't guaranteed by your department. You are almost completely at the mercy of one person -- your advisor. There is virtually no oversight.
Most of these are factors in any employment, and I would argue things like chance of losing funding at your job is worse than academic funding threats.
In comparison, most PhD students work for a very low salary on the expectation of a payoff after something like 3 to 6 years. Framed that way, being forced to either start over or depart is incredibly costly.
Unfortunately, that is not always a positive as many real life situations require you to make decisions under extreme paucity of information and reverse or change course at short notice. For such professions and roles it is a liability.
I'm pretty sure that without the research done by people with PhDs and people who don't give up at the first hurdle, we wouldn't be able to be sitting at our keyboards now having this conversation. Of course, it's not for everyone. Maybe it's not for most. But I don't think you should write all of it off as 'signalling'. Some research simply cannot be done without several years of focus, outside of industry or 'the real world'.
All the evidence shows that fields are completely ignorant of each other and reinvent the basic solutions. This coincides with the theory that cohorts of experts develop expertise which is not transferrable.
Watch as ML rediscovers harmonic analysis while awarding plenty of Phds to those involved.
Rediscovery is a great thing. You bring new meaning and context. I’s just not “expanding circle of knowledge”
More likely is you will dig further down the track of the fads your advisor is into. The trend will be forgotten in a few decades, with a small change of unforeseen utility later. And its contribution will be to your personal life.
The model proposed is also lacking in ambition because historically PhDs were significant.
1) I know my professor and he's a solid guy
2) Pays decently well, money isn't too much of a concern
3) I get paid to do research, university provides generous grants if turned into a startup
Cons
1) Hear a lot of bad things about the academic rat race, pressure to public even at masters/PhD level
2) I could probably hack out some paper into journals but whether I could have any real impact "on demand" (versus say spontaneously coming up with something) is a big question mark, especially within the deadlines given in the program
Any thoughts on this? Especially heuristics, methods or ways to increase impact?
If you’re looking to be impactful, you are much better off joining a job and working in your free time, than doing a PhD. A PhD is a program to compete for academic prestige. Grad students want to publish papers that get them noticed at conferences, invited to talks at prestigious universities etc, those are the incentives, always has been in academia. The brightest minds join academia because they care more about prestige than money (as they should, anyone can earn money, few can win a Nobel prize). In a healthy academic system, prestige is linked to real world societal impact. That is still somewhat true in fields like Machine Learning, in some fields it seems to be completely dis-aligned from any real world impact whatsoever (which seems to be PL research). Our academic system unfortunately is a rotten carcass.
You could still, advisor willing, do research that interests you and not care at all whether you get noticed by conferences/ journals, your peers etc. But that takes a certain level of anti-social behavior that very few humans possess and so I say join a job. Plenty of companies are still building programming languages, like Google, Apple etc which are being used by engineers worldwide and if you finagle your way into a job at those teams, you will have a meaningful, impactful job, which is also well paying as a side bonus.
The goal of PL research is not, usually, to produce languages that see commercial adoption but to produce ideas that industry adopts. You cannot say a language like Rust is not influenced by PL research.
PL research today is actually the study of something called “type theory,” whose relation to the act of building programming languages is the same relation a math PhD has to a carpenter. You will be a great mathematician if you do PL research but I would prefer if you do it in the maths department and not con us into believing it has something to do with programming languages. This is apparently what undergrads are taught in a compilers course: https://x.com/deedydas/status/1846366712616403366 I rest my case. (imagine the grad course syllabus)
On the fringes, you might find research groups who are doing interesting useful stuff in programming languages, but that is an exception to the rule. Which is probably why, you never hear any of the new language developers ever cite programming language research.
Also Rust has been influenced by type theory. Rust first compiler was written in OCaml and the influence of OCaml/Haskell (and many other languages [2]) is pretty clear.
Goal of PL research isn't to design programming languages but academic research has a lot of influence on programming languages.
[1] https://popl24.sigplan.org/program/program-POPL-2024/ [2] https://news.ycombinator.com/item?id=34704772)
Edit: regarding https://x.com/deedydas/status/1846366712616403366?mx=2 these are just the formal specs of a type checker. Nothing magic or very complicated there, it's just a notation. Anyone who can understand and implement a type checker should be able to understand this notation as well.
“ Introducing: Rust Rust is a language that mostly cribs from past languages. Nothing new. Unapologetic interest in the static, structured, concurrent, large-systems language niche, Not for scripting, prototyping, or casual hacking, Not for research or exploring a new type system, Concentrates on known ways of achieving: More safety, More concurrency, Less mess, Nothing new? Hardly anything. Maybe a keyword or two, Many older languages better than newer ones: e.g., Mesa (1977), BETA (1975), CLU (1974) … We keep forgetting already-learned lessons., Rust picks from 80s/early 90s languages: Nil (1981), Hermes (1990), Erlang (1987), Sather (1990), Newsqueak (1988), Alef (1995), Limbo (1996), Napier (1985, 1988).”
If modern PL research is trying to take credit for the latest hot programming language (which I doubt they are, it’s only internet commentators who have nothing to do with PL research who argue with me. Actual PL researchers don’t care about Rust), they should be embarrassed.
Thank you for linking latest PL research, it has been a while since I’ve gone through it, glad to see nothing has changed. Ask yourself, how many of those talks in day 1, have accompanying code? is it even 25%?
For giggles I decided to peruse through “Normal bisimulations by Value”. A 54 page dense paper with theorems, equations and lemmas. Lol, what are we even doing here? You can also notice that they don’t bother justifying their research in the intro or the abstract, claiming relevance to any actual programming language. They themselves realize it’s just math, and PL researchers has become a euphemism for math. Frankly, even one such paper being accepted to a PL conference tells me something is going awry in the field, but if a majority of papers are like this, then the field is a wasteland, that only serves to grind young talented minds into spending their lives chasing academic prestige with no value to society.
57 out of 93 papers (61%) published at POPL 24 have an artifact available. Note that this may also be automated proofs etc, it's not necessarily "running code".
But I also think focusing on POPL as a representation of the PL community isn't entirely fair. POPL is the primary conference focused on type systems within the PL community. It's a niche within a niche. Conferences like OOPSLA, ECOOP, or ICFP are much broader and much less likely to be so focused on mathematical proofs.
“Based on my analysis, I estimate: - ~35-40 papers (roughly 35%) likely have significant accompanying code - ~55-60 papers (roughly 65%) are primarily theoretical/mathematical proofs “
I suspect even the remaining 35% doesn’t have much to do with programming languages, and I don’t think these stats change much for other conferences.
I'd severely doubt that: there is a large difference in focus on theory vs practice between conferences. POPL really is one of the more theoretical conferences. At a conference like ECOOP, you're unlikely to see many proofs (I'd guess at most 20% of papers, based on personal experience).
Claude estimates 10 papers related programming languages and its features and 27 related to theory, verification etc.
I assume "Normal bisimulations by Value" talks about equivalence relations between concurrent programs. If you want to prove correctness properties of concurrent programs or cryptographic protocols, this is one of the tools. It's not because there's no code and only maths that it's not relevant.
> Actual PL researchers don’t care about Rust
Not true, I just watched this video a few days ago about Rust semantics [2]. How would you prove that a Rust program making use of unsafe construct is actually safe? what does safe even mean? how to describe rigorously the behavior of the rust type checker? AFIAU there's not even an informal spec, let alone a formal one. How are you supposed to write correct program or compiler if the language isn't specified?
> Rust is a language that mostly cribs from past languages. Nothing new.
Doesn't mean that Rust isn't influenced by academic languages and ideas. Anybody who knows Haskell or OCaml see the direct influence.
Research isn't industry. A lot of what is produced may have no direct applications but may in the future. This is the point, it's research. Also it's not because you don't see the connections between research and application that they don't exist. Lots of people working on these industrial tools have an academic background and bring their knowledge into the equation.
> If modern PL research is trying to take credit for the latest hot programming language (which I doubt they are, it’s only internet commentators who have nothing to do with PL research who argue with me. Actual PL researchers don’t care about Rust), they should be embarrassed.
You're the one explaining that Rust didn't benefit from academic research which is obviously not the case.
[1] https://compcert.org [2] https://www.youtube.com/watch?v=9y1dLDnS4uE
2. Similarly, mypy was created by Jukka Lehtosalo as part of his PhD [1] and part of a wave of research in applying gradual typing to dynamically typed programming languages.
3. Rust's ownership types and borrowing are based on PL research, such as linear logic / linear types. Same for traits. Early Rust even had typestates.
4. Several of the core developers of Rust, Go, TypeScript, C#, Dart, Scala, have a PhD in PL or a background in research.
5. Generics are another feature that was heavily researched in academia (admittedly a longer time ago) before becoming part of mainstream programming languages.
So I completely disagree with you: most modern languages have been heavily influenced by programming language research. In fact, I'd be hard-pressed to find a modern PL that hasn't been in some way influenced by PL research.
(One thing I agree with in your comment, is that current PL research focuses too heavily on type systems and should look more at other interesting PL features. My recommendation to InkCanon would therefore be to look broader than type systems. The problem with research on type systems is that, because it looks math-y, it feels more like "science" and hence "cures impostor syndrome". But cool stuff can be real science too!)
That's true for some people but others have different motivations, such as learning useful skills so they can gain the ability to work on interesting problems in a given field.
Doing a PhD in PL can also help you get the kind of jobs you mentioned, and achieve more once you're there. For me, the most valuable I thing I got out of the process was extensive exposure to the literature, which has been useful in a range of contexts.
Scala
1) Use is sufficient, but not necessary for impact. Theory of relativity, a lot of QM, etc has had only uses in real world edge cases, but have enormous value. The value function for impact, so to speak, includes more than just use.
2) There is the structure of academia and it's incentives, the average behavior of people in it, and it's outcomes. I don't necessarily have to bow to it's incentives, nor behave like the average person in there. Academia is also sufficiently large and fractal that you can find people less interested in the incentives and more in some thing they obsesses about.
PL has had some interesting, although sometimes unheard of, real world uses. CUDA for example. A significant chunk of PL now focuses on ML. Awhile back a company called Monoidics got acquired my Facebook for work on static bug finding with formal methods. Rust has been pretty influenced by PL concepts. New languages like WASM are formally verified from the ground up, and there are exciting opportunities for that.
I have considered slinging my resume to more research oriented companies, but hearsay from people is that the golden age is over. Under FOMO and stock market pressure, these companies are eradicating the kind of freewheeling research they used to and dumping money into ML and ML hardware. Not to mention it's a bit of a dice roll and a circus to get a job at such companies nowadays as a fresh graduate.
My 2 cents, you will be more likely to encounter creative coders who are passionate about a field in the right industry team than Academia. Unfortunately getting into the right industry team is also a grind, and you likely won’t get there right out of undergrad, but within 10 years, if you put effort and grind, you can get there. I think it’s better odds and more fun than going to academia, but your mileage may vary.
It's very strange to me that you think other people would pay you millions or tens of millions of dollars over an average 30- or 40-year career, without you generating at least that amount of value back to the external world as a whole, and probably generating some huge multiple more, and yet all that counts as "no impact" to you. Especially when your comparison point isn't oncology or something, but doing research in PL theory of all things.
But I thank you for giving me the opportunity to get a little riled up on a lazy Sunday morning, it's one of my favorite hobbies. My recommendation to you for "increasing [overall] impact" is to read https://80000hours.org/ and follow their advice, and for "increasing impact [in this niche I really care about]" is simply to be more bounded with your claims.
Some of it is empirical observation. I've seen many friends at big/elite tech firms get paid to do very little. Many claims online to that effect, although I weigh it lesser. And I think it's completely plausible. I think because of the exponential advancement of technology, huge accrual of capital and inability of human incentive structures to keep up, value does not universally equal money. IMO many examples. Many people are tech firms do things that are very loosely related to revenue generation - so you can almost double your headcount during COVID, fire tons of them and still function the same (a substantial amount of hiring and firing was tech companies FOMOing about each othe). Meta's VR division has burned through $50 billion, but it's people got paid incredible salaries. One in three Nvidia employees are now worth over 20 million. Many of them were working decent jobs making GPUs for video games and suddenly because of AI, their net worth went up 100x. Oncology is another possible example. By far the wealthy people today are all in computers, instead of curing cancer.
I'm not saying these people are bad or anything like that. The other part of the equation, wealth as a signal, has become incredibly noisy. In some areas it is still a strong signal, typically smaller companies and startups where providing value is a lot more closely related to what you make. And conversely, I don't agree with money generated being a signal of impact in itself.
What matters is the outcome, not the amount of effort one puts in. If you're working at e.g. Google for $200,000 a year, your changes can affect millions to billions of people. At that scale even a small improvement like making Google Sheets load 1% faster can equate to millions of dollars of additional revenue downstream -- and likely tens of millions of dollars of actual value, since the largest tech companies actually capture only a low percentage of the value they create for their consumers.
You've just justified that $200k several times over for what might amount to two or three day's worth of effort for you, that's true. That's not a bug - that's a feature of working in a successful, scalable business. If you're inclined to do more than this "bare minimum" which you observe so many doing, just imagine how much value you could create for others if you actually worked close to your capacity in such a place.
>[B]ecause of the exponential advancement of technology, huge accrual of capital and inability of human incentive structures to keep up, value does not universally equal money.
I don't understand the thread of logic here. Claiming that human incentive structures are "unable to keep up" with value creation suggests to me that money is, if anything, a heavily lagging indicator of the real value one is generating, which is in line with the point above. But I don't think that is the point you are trying to make.
>Meta's VR division has burned through $50 billion, but it's people got paid incredible salaries.
Most company actions are bets that the company's leadership think are net positive. Sometimes those bets don't pan out the way we expect them to - that's normal. Your own research might take longer than you expect it to, but that in itself isn't a reason to look back and say you made a bad bet.
As for the people, yes, you generally have to pay a lot to get top talent, and even that doesn't assure you of success. That's probably 2-4 years, out of a 30- or 40-year career, where their contributions may have been net negative to the bottom line. Maybe. If we include caveats like "Meta VR never becomes profitable in the future, either" and "none of the innovations from Meta VR turn out to be profit-generating via a different, unexpected mechanism". This probably equalizes out over the course of a career for the vast majority of these engineers. Not exactly a ship sinker.
>One in three Nvidia employees are now worth over 20 million. Many of them were working decent jobs making GPUs for video games and suddenly because of AI, their net worth went up 100x.
AI is hugely, hugely useful for all kinds of people. I use it every day both professionally and personally. Almost everyone I know does the same. If you truly derive no value at all from it, you are decidedly in the minority.
Is the claim here that they shouldn't have made money off of helping to manufacture the hardware that enables this invention which so many have found so enormously useful? Or maybe it's that since they never intended for their hardware to be useful for such a thing, their involvement should be worth less. That sounds way more like trying to invent a human incentive structure that can't keep up with the exponential advancement of technology than what we actually have. The current incentive structure, however, is wonderfully open to serendipity like this.
>The other part of the equation, wealth as a signal, has become incredibly noisy.
You've just given two examples where one company's wealth fell up to $50b because they made a bet on something that (for now) nobody wants, and another company's wealth went so high that a plurality of their employees are now millionaires because they made something everyone wants. That doesn't sound like a low signal-to-noise ratio to me.
>At certain companies the scale could be enormous
The latter is true and I think the most legitimate reason for working at big companies. I should specify in the first they also accomplish little and affect very little. Things like internal tools that went nowhere, running basic ETL scripts, things like updating financial trade systems to comply with some new regulation. And this at a pretty slow pace.
My meaning about Nvidia and Meta VR is how people who didn't create value got enormously wealthy anyway. In Nvidia's case, traditional GPU teams (which I suspect received most of the benefit because they've vested the longest and made up most of Nvidia's pre AI boom) got hugely rewarded by data center GPUs, which they played little role in. Conversely Meta's VR team still got paid really well (their stock is even up because of AI hype, despite VR losses) despite their failure. So you have these systems where even if you fail or don't play any role in success, you're still paid enormously well. This is because companies capture the value, then distribute in their very imperfect ways.
You're right that the valid reason for this is that tech companies act as risk absorbing entities by paying people to take high risk bets. But the necessary condition for these are
1) Hiring really good people (not just intelligent, but really motivated, curious, visionary etc)
2) A culture which promotes that
The on the ground reality of 1) is that it's a huge mess. The system has turned into a giant game. There are entire paid courses advertised to get a job in MAANG. The average entrant to MAANG spends six to eight months somersaulting through leetcode questions, making LinkedIn/Twitter/YouTube clones, doing interview prep, etc etc. Many causes for this, including the bureaucratization of tech companies, massive supply of fresh grads, global economic disparities, etc. It's no longer the days when nerds, hackers and other thinkers drifted together.
2) Because of FOMO, AI hype and frankly general lack of vision from many tech CEOs, it's just a mess. Anything AI is thrown piles of money at (hence the slew of ridiculous AI products). Everything else is under heavy pressure to AI-ify. I've heard from people inside Google has really ended that kind of research culture that produced all the great inventions. There are still great people and teams but increasingly isolated.
"Impact" is an ambiguous term, so it's quite vague what you mean. I assume "positive impact on the world and knowledge".
While this mantra is indeed motivational, it can set you up for disappointment, both in corporate as well as research/PhD settings, at the moment you realize how many hurdles there can be (toxic colleagues, bureaucracy, ignorance, etc.).
Also, for this interpretation of "impact", a corporate job can be very impactful as well.
This is the core of the issue (most replies usually involve some slightly different definitions). I take many definitions of impact, including societal use, contributing to knowledge, etc. But it's much clearer there are many things people do that are low impact, especially in places with a lot of bureaucracy, politics etc.
A corporate job can, but it seems to me as a result of various incentives corporate jobs tend to be compartmentalized, low impact and repetitive. We're also at a down cycle where tech, the historical haven for impact in a job, is scaling back a ton of things to focus on stock prices. If you know of any corporate jobs that do have impact by some definition of it, I'd love to hear it. In my experience these have been mostly startups.
Your relationship with your advisor is very important. It seems like you already have that sorted out.
Most successful PhDs (in CS) involve tackling a relatlively small and easy project, usually suggested by your advisor, early on, and then expanding and iterating on this. Once you make some progress on a topic you'll easily find more directions to take it.
Working with other people is one of the easiest ways to increase your productivity. All the great groups I saw had a lot of collaboration. Don't fall into the "lone scholar locked in the library" stereotype.
Avoid bad people. Avoid getting stuck in your own head. Realize a PhD is a project like many others. It doesn't define you. You start it, you work consistently on it, you finish it.
Doing a research Masters is usually a waste of time. Doing a taught Masters is a lot of fun, but something quite different to a PhD.
>A PhD is an apprenticeship to become a researcher That's a good way to look at it. I suspect one of the biggest possible benefits of a PhD is that you're put in an environment structured to and pressuring you to develop something new, which is the opposite of most other human work.
>Start a relatively small and easy project and collaborate Sound advice, it's the general approach I've taken for my undergrad thesis.
>A research masters is a waste of time, a coursework masters isn't Really? It looks the opposite to me. A research masters let's you collaborate with different people and work on new things. A coursework masters is taking advanced classes.
Strongly depends on the advisor and your goals. If you want to stay in academia, some amount of publications is required. Your advisor, especially if he pays your salary, may also push you to publish. If both are not an issue, I guess you can even finish without publications.
> I could probably hack out some paper into journals but whether I could have any real impact "on demand" (versus say spontaneously coming up with something) is a big question mark, especially within the deadlines given in the program
Nobody comes up with good ideas on demand. As you progress in your academic career the rate of ideas (theoretically) grows. That's why you need the advisor: he can come up with ideas at rate sufficient for his students
That's fair. I'm just cautiously eyeing the likelihood of coming up with something publishable that's not a going through the motions kind of thing.
1) There's a kind of "hard" learning you're learning a fixed, structured way from a textbook.
2) There's a kind of "soft" learning which is transmission of knowledge, which happens a lot more face to face when you're working together.
3) Then there's a kind of research learning, where you're doing something new, usually with collaborators.
The second and third are really best done in certain environments like research or good companies
Furthermore, do not underestimate the importance of sheer luck. Exaggerating a bit, deep learning was just another subfield of ML, until GPU-powered DL really took off and made the researches behind the most fundamental ideas superstars. This is not a given, and it might take years or decades until it's really clear whether you're making an impact or not.
I wish you the best of luck, InkCanon, and stay excited!
chances of getting professor-ship, tenure, or even a post-doc is close to nil, due to extreme competition and limited seats. academia is the most slowly moving enterprise, some folks in their 80s still around, when young grads kicked out.
getting a job after PhD may also be very hard. you would be very over-qualified, likely huge ego, and very narrow skillset in your domain, that is likely lagging behind industry. managerial (or even just "work at corporate") skills will be lacking. unemployment of PhDs is wildly high, even higher than if you did not do PhD.
turning research into startup may be much harder than you would expect. milking government funding for years (and surviving jungle of academic politics to get its cut) is very different from market outside of it (i.e. venture capital, startups, tech, etc.), at the time you would want to make startup you would have to learn all from scratch, or even un-learn, as many would be detrimental.
then there is toxic academic culture (funding, publishing, power dynamics). and in recent years academia become pit of wild woke left agenda, even more oil on fire.
tbh, if you want to do something special, academia as we have it today is not the best place.
if you still want to do it, guess best strategy is to "do it quick and get out". some smart people I know doing exactly that. doing accelerated PhD asap and getting the hell out of academia. (but then, it depends all on your professor power dynamics. in some places they would not let you graduate unless they wish so.)
What an extreme exaggeration. Yes academia is competitive, yes tenure is hard to get (obviously). But the chance is not "close to nil" for that at all, and it's certainly not "close to nil" for a postdoc lol.
> getting a job after PhD may also be very hard. you would be very over-qualified, likely huge ego, and very narrow skillset in your domain, that is likely lagging behind industry. managerial (or even just "work at corporate") skills will be lacking. unemployment of PhDs is wildly high, even higher than if you did not do PhD.
This is just not true x) There are no numbers where PhDs have worse unemployment than grads.
Conversely, it does open a lot of doors for industry jobs (think ML, quant finance, to name a few)
In my side of the world its a little bit better, the CS department has plans to double headcount in the next few years. They've got whole new faculty apartment buildings set up and everything, and the funding situation is quite generous (I'm told). Although I have also read in the USA the bar for even stipend paying masters/PhDs has gotten incredibly high.
>Milking government funding for years
There are special programs for startup oriented funds, so it's more like VC pitching to academics with equity free grants (although naturally there's the whole university research IP issue). But I'm quite willing to put up with it to do something meaningful (at a decent number of jobs you put up with it just to keep your job). I do keep an ear on startup-y things, I don't think I'll have it any easier than an undergrad but I think I won't be too disadvantaged.
>Do it and get out
I don't place too much emphasis on the PhD per se but the real value of it.
>If you want to do something special academia is not the place
Ten years ago tech would've been a good place. But now especially for a new graduate its a bloodbath, not to mention there's been huge layoffs. Academia seems like the better option nowadays.
The biggest obstacles to getting an academic job are personal. The jobs are wherever they are, and your (or your partner's) preferences cannot change that. If you are willing to relocate, your chances of getting a good academic job are much higher than if you restrict your search to a single city / region / country / continent.
- The student becomes hyper focused and pigeonholed into some esoteric and unemployable domain, destined to run on the postdoctoral treadmill for decades.
- The PI is a control freak who only cares about publications, and considers students who leave for industry jobs after graduation to be failures.
These stereotypes can have an element of truth, but there are more enlightened PhD programs and PIs that understand the value of cross-cutting and commercializable research than you’d expect from the discourse. Not everyone is stuck working on a pinprick of knowledge, and if you choose your program and PI wisely, you can go much further and do many more things than you would never have access to with just an undergraduate background.
Someone who only published in 2 top conferences is obviously not worth anyone's time. But 3, now we're talking.
For some reason that seems slightly non-optimal.
Top people come from non-top schools, and lots of non-top people from from "top" schools. And some top people come from no school at all.
And intelligence agencies are government mandated, not marketing made. Or at least I haven't heard any marketing from the NSA saying how selective they are in admissions (as if that means anything).
Elon Musk skipped his PhD program and did many more things than spending time in school would have allowed him to do. Of course, most people aren't Elon (probably a good thing).
Other than preparing you for a career in academia or some highly regulated environment where education is erected as a barrier to entry, it's hard for me to think of "many more things" that are open to a phd holder than to someone who is not.
In some fields all you need is a computer and an idea to be impactful, but in plenty of other fields you’d be hard pressed to make any credible, let alone meaningful impact without significant intellectual preparation and tacit knowledge. These things only come through experience, and for many people, the PhD program is that experience.
Carlos Ghosn started out as a factory manager (although well educated), and in his Stanford interview the presenter noted that Stanford produced no factory managers, although it produces lots of would be global CEOs.
Perhaps it should produce more factory managers.
Musk has shown an ability to make an impact in multiple fields for which he seems quite under qualified for, for which he did not have "significant intellectual preparation and tacit knowledge". He read alot.
I think there are more non-celebrity exceptions that are simply not well known.
And there are lots of people in PhD programs who, despite their education, do not make credible or meaningful impacts, quite possibly not at all due to their competence or training quality, but due to wholly accidental or uncontrollable factors: industry shifts, business culture, changes in government research funding, or their entire paradigm being based on faulty assumptions that were simply not known and discovered later, or superseded by some innovation, etc.
Academics are rarely comfortable discussing the shortcomings of academia.
- There aren't enough post-doc and tenure positions for the glut of PhDs.
- Plagiarism scandals have reduced the public's perception of a PhD to become almost something unprestigous.
So there's that.
So, I prefer a narrative guide to PhD - "The Lord of the Rings: an allegory of the PhD?" by Dave Pritchard, http://danny.oz.au/danny/humour/phd_lotr.html
than writing papers nobody will ever read? yes I am.
A PhD can be a great contribution to your life, and the opportunities it can bring you and your family.
The phony marketing that appeals to young people is that you are advancing the progressive human narrative.
* Choose your advisor with care. This is not very easy as an applicant looking at professors' websites, but if you are admitted, any good school will probably have an in-person or virtual admitted students day where you can talk to current students out of faculty earshot. Take advantage of these times to ask about your potential advisor. A truly bad advisor will probably produce at least one person who will warn you about them. If you can't do this in person, try to get a quick phone/video chat -- something off-record where they can be honest. I was always happy to do these for my advisor, because I liked him and wanted him to get more good students. Conversely, I know people who were warned off specific advisors during these events, for good reasons. A bit of subjectivity: a good advisor at a decent school is usually better than a bad advisor at a good school.
* The financial niceness of doing a PhD in field X seems to correlate pretty well with the current job market value of a masters in field X, at least partially for reasons of leverage -- if you can leave and transition into a cushier job, advisors have to provide a bit more value. Computer science scores highly on this metric.
* There is a ton of negativity about PhDs in places like HN. This isn't unjustified: doing a PhD with a bad advisor can be a very bad experience. At the same time, I think "person who had a bad PhD experience" is also "person who writes comments on the internet" with higher probability than "person who had a good PhD experience".
It's the archetypal hero's journey where you dive into the deepest darkness, discover yourself, and emerge triumphant.
At the end, you get a hat.
Highly relevant for the aspiring researcher, and it describes really well the nonlinearities of (academic) life.
An MD (Medical Doctorate) is like a master's degree. It's not like a bachelor's because many MD programs start out with or require a BSc, biology is a popular choice but a lot of STEM majors are possible.
But MD+PhD programs exist and those are definitely PhDs.
You are right that an MD is not a PhD, though. Notice how they don't call it a PhD.
In many countries, the degree you must obtain to qualify as an MD is indeed a bachelor’s degree, the “Bachelor of Medicine, Bachelor of Surgery” (often abbreviated MB ChB or MBBS).
I agree it's not a research degree though... But some master programs don't include writing a research thesis either.
"Professional degree" is juxtaposed with "research degree". So if you say it's a professional degree, you're basically agreeing it's not a research degree...
In my country at least MDs are not required to be researchers and the degree has no novelty requirements. They're required to be competent medical professionals.
There are countries where the base medical degree is the MBBS and MD is a graduate research doctorate. That's not what I'm talking about here.
I'm not talking about "difficulty", by the way. Just the differences between the degrees.
It has a lot of truth to it and it's been making the rounds for well over a decade. Unfortunately, sometimes it can set folks up to feel bad about themselves if what they do doesn't line up well with whatever is in vogue as the boundary of knowledge that's currently being pushed.
There's a _ton_ of value in more pragmatic parts of fields that focus on applications or combining relatively well-know parts of different sub-fields. Those parts of science often don't feel like you're pushing some "boundary". It's more layering on top of, filling in holes, and building up than building "out". Sometimes what you do is to use multiple things that lots of folks already knew, but the people who knew X and the people who knew Y didn't talk to each other, so no one thought about how to combine them. It can also be hard to get papers published because reviewers will consider one part obvious/well-known and the other part irrelevant because they come from one sub-field and not both.
It will often feel like you don't belong because your work doesn't look like these figures. Applied and interdisciplinary fields "feel" different. However, this type of integrative work can be among the most valuable parts of modern research.
Don't feel like what you do has to fit into the "penis pimple model" of science. There's also nothing wrong with pushing some known boundary of a field, either! Both are valid.
Sure there's plenty of ways to contribute to society without publishing academic research, but that's not what a PhD is.
It's novel work that makes a PhD. Novel work is distinct from pushing the boundary of knowledge. Often what you do doesn't change what "humanity knows" in any way.
Both my wife and I have PhDs. Neither of us did things that look like the figures there. It's not a good mental model for what all PhDs mean, though it is a good model for some.
I combined fields and reinterpreted a ton of things that had already been done to draw very different fundamental conclusions about what was going on in a particular location. I put out alternative hypotheses for observations that had already been collected. It's not new knowledge at all and I didn't add to what we "know". I just added an additional hypothesis to the set of multiple working hypotheses that will hopefully be tested decades from now.
My wife worked on how to actually apply well-known methods in other fields to our field. Her work was half engineering, half field experiments. Lots of folks had been working with fiber optic strain gauges for decades. However, no one was using them to measure in-situ strain in rock masses yet (which has since become common). The application was broadly "known", but actually doing it and demonstrating that is novel.
At my university, my favorite professor’s title was “senior lecturer” because he only had a bachelors. This was despite being a Times bestselling author. (He taught literature and writing.)
Your job title probably won't be 'professor', but you'll be doing basically the same work as one.
As a former graduate student myself, I'm actually not aware of any non-PhDs who are adjuct faculty or community college instructors. I'm not claiming that they don't exist anywhere, but given the number of PhDs and the number of available academic jobs, the competition is fierce, and non-PhD candidates are likely to lose out to PhD candidates.
- The university sets a schedule and you are assigned to classes that are otherwise short staffed - there's little consideration for your interests. Basically you get bottom of the barrel courses and inconvenient hours.
- The students can barely program and do not care. I know it's a cliche, but it can't be understated. These "masters" students could not handle the equivalent of leetcode easy problems. Get ready for a lot of late submissions, half-assed homework, and begging for extra credit. Oh, and the final is open-book and you're not allowed to fail anyone.
- The student body is largely H1B visa holders. Anyone that's been paying attention to the H1B story knows that part of the visa scheme is funneling students into masters programs to improve their chances in the lottery. Nothing against visa holders, but this is obviously a cash cow for universities.
- Academic personalities and elitism. You are an outsider and will be looked down on. In my friends case, the Dean started getting very bossy and started dumping responsibilities on him that he really had no business being apart of. Ex. Being a judge for someone's thesis defense. My friend got a lot of satisfaction out of submitting his resignation after just 2 semesters.
I personally have a fondness towards teaching as well and tend to romanticize it, but my friends story really turned me off to any interest in that line of work. Of course this is just my friends anecdata, YMMV.
If not then can someone (original author or someone else) try to review the steps with roughly realistic average scales assuming surface represent knowledge volume and color complexity spectrum from advanced (indigo) to advanced (red)?
Any candidate to make another pimple on the outer circle?
At many of our leading institutions in the hard sciences, rehashed work, data stolen from other teams, photoshopped images and a bit of plagiarism is enough to get you by.
Once you get to the social "sciences", it is much worse.
Karma cannot be denied and the result is that work outcomes with a PhD are derisible, since employers have worked out that something does not compute about the quality of freshly minted PhDs.
https://news.ycombinator.com/item?id=29138570 (158 points | Nov 7, 2021 | 121 comments)
``` For safety and well being of you and your family, EVACUATE NOWWW. ```
I really treasure my (non-US) PhD time. I had a great adviser, was decently paid, and had a lot of time/opportunity to explore things, which I think lead to interesting research. And when I retire, it provides part of my pension. Also when I look back I often realize how relaxed my PhD time was compared to the much more stressful life after doing a PhD.
The field really changed after doing my PhD (I'm in NLP/CL and when I did my PhD, HPSG-like grammars and maxent models were still reigning), but I think I still benefit a lot from the methodologies I learned while doing my PhD.
(to clarify I'm not disagreeing with you, US academia does have lots of issues that lead to many having a bad time. But still, getting paid to learn is a dream)
Can't find it now