I have a bachelors in physics. I went for a phd in biology but had to bail to work to support family, plus I can't seem to come up with original ideas.
I now have a job with physicist as part of the title, but what I really do is try to read bad handwriting from records from nuclear weapons plants. It's a dull job a monkey could do but it pays the bills. For years I worked as a computer guy for an academic department and that was fun and I'm trying to get back into it but nothing yet. I teach math at a community college for fun too.
I have great kids, so all and all I'm happy about things, but I am sad the physics thing never panned out. As an undergrad I was excited about chaos and nonlinear dynamics. Still read math texts for fun and play with Haskell.
As a further aside, I do get to do some fun graphics in R and I'm pretty impressed how R has become the statistical standard and graphics too, mainly I think from one super dynamic statistician.
Now that R is entrenched I can work on moving everyone into Haskell!
So pure and innocent. I've found that programmers are not too fond of having even a simple IO operation lead to discussions of category theory. Good luck in your noble lambda crusade :-). The future is functional! Hack the type systems!
+
"""For years I worked as a computer guy for an academic department and that was fun and I'm trying to get back into it"""
Sounds like you could enjoy automating your work. Learn a bit of Python and some Machine Learning/Deep Learning and digitize the handwriting and build a little program that reads the stuff for you. The default example for reading handwritten digits is called MNIST if you want to read further. I'd suggest fast.ai and watching the first couple of lessons. That should get you started to play around with this. You don't have to tell anyone that you do this (I probably wouldn't) but hey at least it might be a nice way to do a little less boring stuff and ease into a bit of programmin/data science?
First of all, you definitely don't want any chance of incorrect recognition of handwriting coming from a nuclear weapons plant.
Secondly, who knows what policies he might violate by using some unauthorized, untested software (whether OP is the author or not) with potentially sensitive information.
First of all, you definitely don't want any chance of incorrect recognition of handwriting coming from a nuclear weapons plant.
On paper it saved a lot of man hours but it was never fully rolled out. The project didn't have a long lifespan and we had enough people sitting around doing nothing to just manually do all the work. But I got a nice award for it LOL.
Then deciding on his own on the advice of HN that he should try a car for that task instead, the car breaking down, and the guy getting fired because his employer never approved of this whole "car" thing and he shouldn't have introduced that into his job workflow without having talked to his employer about it.
I am looking into auto-scrolling pdfs and batch loading, sequencing of pdfs to make things easier. But I will definitely check out fast.ai - thanks!
Pick up a copy of Automate the Boring Stuff with Python.
Still, I feel I could take any of my students and in one month have them up to speed on what to look for and what things mean.
And I don't mean to suggest that finding challenging work is any sort of failing of my employer. I have been able to diversify the position to include doing graphics in R and some stats things and, along with some other people, I think we can sell some sexy Bayesian things.
They are very receptive of new ideas, but... it is not a growing industry (thankfully), so any sort of advance is going to be a jump to a different field. Which is possible, and I'm working on it.
My point primarily is when you read these things about people doing all sorts of wonderful exciting things, you get the impression that what you have done or are doing is crap. Perhaps it's nice to hear that a lot of people do not set the world on fire. Takes a little pressure off.
Is there anything even tangentially related to your dayjob that you could mine for ideas? Or maybe take some time to come up with some research direction?
A places like Oak Ridge could have something interesting available.
I, too, read the Strogatz book :^).
We used this textbook by Devaney: https://www.amazon.com/First-Course-Chaotic-Dynamical-System...
Was absolutely fascinated by the class. One of my favorites from undergrad.
I wonder if there's a good sort of "postmortem" on what happened to NLD and Chaos. It seems like everyone said, "Ok, that's a thing and it explains a lot but we can't do anything with it."
Although the case for building new and more powerful hadron accelerators doesn't look good, accelerator physics is flourishing with other types of machines.
Particularly interesting to me is ultrafast electron diffraction (UED)[1,2]. UED is cool because you can create atomic resolution movies with speeds that can (in the near future) resolve chemical reactions as they occur. (eg. imagine being able to see a protein change conformations in a biological reaction)
This application is limited by the number of electrons we can stick in a given volume and get traveling in the same direction. The only way to improve this is by increasing the electric field in your electron gun or by choosing good materials for your photocathode. [3]
My research is on the second route and I'm currently building a measurement system that will allow us to test several theories related to how we choose these materials. Improvement in this domain is important and could open up a huge amount of research, but unfortunately doesn't get the kind of publicity that the big projects do.
[1] https://lcls.slac.stanford.edu/instruments/mev-ued
[2] Dwyer, J. R., Hebeisen, C. T., Ernstorfer, R., Harb, M., Deyirmenjian, V. B., Jordan, R. E., & Dwayne Miller, R. J. (2006). Femtosecond electron diffraction:‘making the molecular movie’. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 364(1840), 741-778.
[3] Rao, T., & Dowell, D. H. (2014). An engineering guide to photoinjectors. arXiv preprint arXiv:1403.7539.
Relevant "I saw this on YouTube" video:
- "Should we build a bigger particle collider? - Sixty Symbols"
- https://youtu.be/-cD66O01E4E
- TL;DR: the LHC has only found one new particle, and it was looking for it. Before spending 30 years and 10-20 billion pounds on a 4x bigger collider, maybe we should wait until we have an idea of what we'd be looking for, as it's not clear what the larger collider would be looking for. The downside of not building a new collider soon is that the people who know how to build a collider now won't sit around waiting until we decide to build one, so starting from scratch in the future will presumably take longer & be more expensive.
However, I think there is a lot of interesting physics for accelerators beyond colliders. For instance, the linac coherent light source (LCLS) is a huge x-ray laser out in California [2]. They are in the middle of a big upgrade and employ a large number of accelerator physicists. There is also still a huge amount we need to learn about how free electron lasers work and how we can improve them.
Another big topic of research is energy recovery linacs (ERLs). We are just on the cusp of being able to implement a technology that could save something like 90% of the wall plug power of current accelerators. They draw a huge amount of power, so the benefit of this saving is pretty clear. The first machine to demonstrate this new technology will come online this summer (hopefully). [3]
[1] https://physicsworld.com/a/disappointment-as-japan-fails-to-...
I have begun looking about at jobs outside of academia in order to decide whether to stay or go. Data science is one of the easiest transitions a physicist with data-analysis can make, which I think explains the prevalence of physicists in that role. We have the training in both the techniques and a sober assessment of uncertainties, which makes us desirable.
So far, in my search for outside employment, I haven't found anything that draws me as much as my present work, but if you're in the Seattle area and looking for an experimental physicist with a broad range of experience, please get in touch.
Imagine taking Newton down in the Replication Crisis Wars!
"The Replication Crisis" has basically involved rather different fields - psychology and human biology/medicine most often. The crisis could be called "human and animal testing crisis" imo.
What we do is search for deviations from Newton's inverse square law. Depending upon the distance over which an experiment is performed, the ISL can be correct to a precision as high as ~10^-10 [1]. In particular, our group works to test whether the inverse-square law (or indeed gravity at all) happens as predicted at distances shorter than 100 microns [2]. It still amazes me that at distances shorter than half the diameter of a human hair, nobody has any idea whether gravity even happens.
Newton's early observations (and Kepler's) have been replicated many times -- there is no risk of a major replication-crisis upheaval there.
[1] https://arxiv.org/abs/hep-ph/0307284 (Figure 4)
I graduated and did a few strtuos, now working on autonomous self-delivering ebikes in SODO.
Any room for scientific software developer at LIGO? Gravity is a beast... I will leave it at that. But I can signal process and I am not afraid to ask stupid questions. Looking for growth industries, I've had my eye on quantum computing and am just putting resumes together now. I'm really into differential geometry and topology but my understanding is amateurish and what I've learned of QFT, geometric physics, topology, etc., is uneven at best.
If you really are interested in software dev / data science I can ask around my company. They have a Seattle office* and a broad range of needs for different developer backgrounds --- at least when all offices are considered. Typically there is wiggle room to work from one office with a group based elsewhere. I can't guarantee anything though -- it's been hard to get anyone hired lately!
Back to gravity and such: I doubt you find anything, without physics in it, that draws you in as much as your present work, but it can't hurt to look around! How about a switch to focus on quantum computing if data science doesn't draw you? Bar that, what about numerical simulation?
*I'm not in Seattle.
That said, if one wanted to move toward LIGO from a software background, the chink in the armor is that the signal-processing is hard and computationally bound. Helping signal-analysis groups to find efficiencies there would have an impact.
I continue to keep an eye on quantum-computing infrastructure hires in the Seattle area -- so far, I've proven to be too physics-focused to want to work solely on provisioning the (awesome) cryostats that house the quantum-computing devices under test. That may prove to be a sub-optimal life-decision -- time will tell :).
Embedded graphics drivers for real-time systems.
I keep the physics part of my brain alive by developing physics based Unity assets (nbodyphysics.com) and supporting a package for GR on github (grtensor).
I still buy WAY too many physics books. Current aspiration is to work through "Modern Classical Physics" Thorne/Blandford.
If you're thinking of getting it but want to check it out, there's a 2012 draft version that the authors have previously taught from here: http://www.pmaweb.caltech.edu/Courses/ph136/yr2012/. It's not the same as the book, of course, but from a skim it seems quite similar.
A good choice for less math might be "Gravity from the ground up" by Schutz.
My grad work had no connection to data/stats and I have not bumped into any low hanging fruit where Riemannian geometry might be the answer!
The central issue is understanding how changes in control parameters (for instance concentrations of catalysts in a chemical system, or local fields in a spin system) affect the evolution of the probability distribution over states. Some work has been done in close to steady state (for instance [1,2,3]) but it's far from resolved.
This has some nice applications - designing efficient protocols for microscale devices, for instance.
[1] https://arxiv.org/abs/1603.07758 [2] https://arxiv.org/abs/1507.06269v1 [3] https://arxiv.org/abs/1201.4166
None of this pays the bills, right now I help build clouds, and I used to build supercomputers, and high performance storage systems.
Yeah, I'm keen to give that a go. Interestingly it is not a problem that is very easy to parallelize, since every mass will affect every other mass. There may still be some cool ways to use ECS though.
Somehow I doubt it! In any case, you are like me - doing engineering and software, while doing my physics as a hobby and loving it. I'm working on Physics from Symmetry. Definitely a 21st century book - not at all like the traditional development of the subject.
PhD 2001 in physics, working on quantum computation. Postdocs at Caltech and Santa Fe Institute, then landed a research faculty position at the University of Washington. Yeah, raise your own funds! Jumped ship in 2011 (burnout, quality of life, university not caring about quantum computing) and went off to become a "real" software engineer at Google. Worked on ads (as one does), then helped build Google Domains, then worked on distributed privacy preserving machine learning. About two years ago, my background in quantum computing caught back up to me, and now I run the team that builds software for Google's quantum computers.
People ask how to get into quantum computing if you are a software engineer. I will say that you really need to spend some deep time in quantum computing, either a masters or a PhD or some very very serious self study. There are certainly parts of writing software for quantum computing that don't require that, but if you really want an expansive career working in quantum computers you'll want to have a deep background.
> my background in quantum computing caught back up to me,
I would be curious on that part. Could you describe the process what happened there? Internal skill screening program? Did you jump projects?
Thanks!
Is there any significant difference to payscale and hierarchical autonomy in these non AI research teams? I am assuming distributed privacy preserving ML team (is this team close to federated learning one?) also falls under more non AI research teams right?
I enjoyed grad school for the most part, and was really into teaching. By the time I got a professor job (a visiting one, not tenure-track), I was getting anxious about the long-term job prospects and it was getting harder and harder to justify the workload (teaching, grants, advising students, etc) given the relatively poor job security and pay. I felt if I was going to switch careers, I should do it soon since it’s not going to get any easier.
By this point (~3 yrs ago) I had several physics/astro friends who had become data scientists or similar jobs in the tech industry. Some had done programs like Insight and some got jobs on their own. Everyone I talked to seemed happy with their decision to switch careers. I ended up doing Insight and getting a job quickly after and am glad I did. The variety of the work, amount of collaboration (more), and new things to learn is still keeping me interested. I was also surprised at how many opportunities there are to give talks and seminars in the industry, which helps scratch the teaching itch.
I've been very happy doing data science ever since. It's great to work more collaboratively, ship more quickly, and learn from great engineers.
People in my field are fairly fortunate as there is a career track as a clinical medical physicist that is highly paid and pretty low stress, so most people end up going there. The work consists of maintaining and calibrating the radiation therapy machines, along with implementing new technologies in the clinic, and fixing problems that don't fall within the job description of the radiation therapists. Like what to do when a radioactive seed falls on the floor instead of going inside the patient where it's supposed to go. There's also a separate track as an imaging physicist where you maintain and QA the diagnostic imaging machines.
I'm personally doing a postdoc at the junction between optimisation, machine learning and radiation therapy. Just starting out though. Basically just extending my PhD work to automate the treatment planning process and remove the variability in treatment plan quality due to the level of experience of the people making the plans.
Motion during treatment can be tracked with cameras or IR sensors or subcutaneous probes but that doesn't tell you about internal organs moving. The topic of deformable registration, where you find a non-rigid mapping between initial imaging conditions and the current ones, is still a topic of active research. Adaptive planning, where you actively change the treatment plan every N sessions based on the most up to date information, is also actively researched / implemented in some good research centers.
For treatment planning you just use a standard Cartesian grid, or a "beam's eye view" coordinate system that's aligned with the radiation beam axis as it rotates around the patient.
Not to be glib but on behalf of the thousands of people going into a radiotherapy clinic today for treatment, thanks for working to improve these techniques.
Sounds like engineering and SE, not physics
I saw a postdoc who is now rather well known struggling with anxiety over his career even though he had written half a book and done a lot of great work. When we were both at Cornell I'd come to the conclusion that many papers involving "power law" distributions were bogus because nobody knew how to test for them with any rigor. It was years later, after he had tenure, that he published something about it in a statistics journal.
Seeing that made me run for the exit after my first year as a postdoc.
> after he had tenure, that he published something about it in a statistics journal
If I'm reading you right, you're saying he struggled for awhile doing bogus things only to question those things outside the relevant field?
(Also, can you say more about power law papers..?)
So there is a theory based on the renormalization group that can explain many (but not all) "fractal" phenomena that are observed.
I was part of the false paradigm of "bin up a probability distribution", "plot it up on log-log paper", and "draw a straight line."
Sometimes when you do that you will get an answer that has something to do with reality but it is not a valid answer to the problems of: "is a power law distribution a good model of this phenomenon", "what is a good estimate of the exponent", "are these two power law charts drawn from the same distribution" not to mention how to handle the problems that turn up at the highly frequent and highly rare ends of the distribution.
The root cause is this attitude
http://thinkexist.com/quotation/if_your_experiment_needs_sta...
and an academic system where power is too concentrated, where people who write review articles do the most good but get the least career advancement, etc.
I've been out for a few years, so I don't know what the current state of affairs is.
Is there any published criticism against his work? His papers get cited everywhere, I know citations are more metric of popularity than scientific quality but I find it puzzling that his work continues to get attention if the results don't hold up.
Salary is decent, everything is moving a lot faster than at uni. Wouldn't say I miss academia as such, but I definitely miss working on actual fundamental physics problems.
I might go back some day to apply my new knowledge of operational data science. There's definitely a need for updated methods, especially in data heavy fields such as astrophysics.
Edit: To answer your question about what's better outside academia - I'd say for me it's the tighter collaboration with colleagues, better project management, clearer goals, more diverse team (in terms of educational background / role in company), and last but not least job security - I can live where I want, not where the next postdoc happens to be.
Now I work optimizing fantasy sports teams and building websites that display betting lines. Don't even like sports and I have no idea what anyone in the office is talking about, but they are paying a lot for me to put buttons on their website I guess. Leaves me a bunch of room for my hobbies. Offered to do some actual math, maybe personalization algos or some AI stuff. Backtesting automation to determine if our data is at all valid that we're selling?
Really just need these buttons on the site is all.
Focused on modelling/simulating materials during my thesis and realized that I loved the software aspect of things and did not love working on the same problem for many months at a time.
As others mentioned, transitioning to data science is not that hard if you have a physics background and there are many interesting problems to solve in the area.Most of my graduate student peers are also in data science/ML and related areas (software/finance).
Although money was a contributing factor, the main reasons to leave academia were being able to live in a city that I liked and where my partner could also find a position.
Did not look back on physics at all for the first couple of years post-PhD, but missing it quite a bit nowadays. End up buying a lot of Physics books every year, although don't get through many of them. Latest purchase was Exercises for the Feynman lectures.
This is not really true, I know a few people who managed.
In my case, my partner is not in academia which makes it a bit easier.
Those are wonderful! I dug them up late in my undergrad and spend some fun times with them.
I now work on mostly computer vision, machine learning, and robotics. Robotics was a hobby of mine since high school, and is now my work.
I'm still passionate about physics and physics research, but I'm not happy with the academic system of today, which encourages pursuing a lot of low-hanging fruit just to publish and get tenure, instead of going after high-risk, potentially-groundbreaking, but likely to fail topics.
While doing my PhD one of the biggest questions I kept having is why the primary goal the system set for me is "to graduate" and not "to advance science". On several occasions faculty told me to not try something "because I would never graduate" if I went down those paths.
I jumped into industry hoping to improve how the world makes things on a very large scale. Current project has potential to change how a ubiquitous product is made; it would be the first major manufacturing change in 80 years!
"For now, however, in hard-core physical science at least, there is little evidence of any major BD-driven breakthroughs, at least not in fields where insight and understanding rather than zerosales resistance is the prime target: physics and chemistry do not succumb readily to the seduction of BD/ML/AI. It is extremely rare for specialists in these domains to simply go out and collect vast quantities of data, bereft of any guiding theory as to why it should be done. There are some exceptions, perhaps the most intriguing of which is astronomy, where sky scanning telescopes scrape up vast quantities of data for which machine learning has proved to be a powerful way of both processing it and suggesting interpretations of recorded measurements. In subjects where the level of theoretical understanding is deep, it is deemed aberrant to ignore it all and resort to collecting data in a blind manner. Yet, this is precisely what is advocated in the less theoretically grounded disciplines of biology and medicine, let alone social sciences and economics. The oft-repeated mantra of the life sciences, as the pursuit of ‘hypothesis driven research’, has been cast aside in favour of large data collection activities [7]. And, if the best minds are employed in large corporations to work out how to persuade people to click on online advertisements instead of cracking hard-core science problems, not much can be expected to change in the years to come. An even more delicate story goes for social sciences and certainly for business, where the burgeoning growth of BD, more often than not fuelled by bombastic claims, is a compelling fact, with job offers towering over the job market to anastonishing extent. But, as we hope we have made clear in this essay, BD is by no means the panacea its extreme aficionados want to portray to us and, most importantly, to funding agencies. It is neither Archimedes’ fulcrum, nor the end of insight."
https://royalsocietypublishing.org/doi/full/10.1098/rsta.201...
> Yet, this is precisely what is advocated in the less theoretically grounded disciplines of biology and medicine, let alone social sciences and economics. The oft-repeated mantra of the life sciences, as the pursuit of ‘hypothesis driven research’, has been cast aside in favour of large data collection activities
The thing is, I've just spent two years working for molecular neurobiologists in the field of Single Cell RNA Sequencing, and large data collection has definitely lead through tons of breakthroughs there.
We can now classify cell types based on gene activation, on top of the previously existing morphology and location the cells are found. That can then be used to discover new subtypes, the origins of cells during embryonic development, and even predict which cells will evolve into others[0][1][2][3]. All of this requires vast amounts of data to ensure there is enough statistical power. In fact, the insistence on using unbiased samples before applying clustering algorithms is a big part of overcoming biases based on pre-existing expectations.
(Also, may I request that you edit your comment and break up that block of text into sub-paragraphs, for the sake of readability?)
[1] https://linnarssonlab.org/osmFISH/
So if we take some of the thought processes behind Stat Mech (or S&M as we used to call it in grad school), and you kinda squint your eyes hard to blur the less robust discussions you read about, you get the sense that ML is more about the "thermodynamics of information" than anything else.
I find this intriguing and definitely want to spend more time on this stuff.
This is so sad
You can merge ML and theory in at least one way. I attended a talk by Prof. Karen Willcox of the University of Texas at Austin (I'm a PhD student in mechanical engineering there) where she argued that in fluid dynamics and combustion at least, it's better to use "model order reduction" instead of machine learning. The problem with many models (e.g., Navier-Stokes equations) in these fields is that they are computationally expensive. Model order reduction looks for ways to reduce the computational cost of the model while maintaining accuracy, and it uses many of the same techniques as machine learning. Based on the examples she gave it seemed to be the closest thing I've seen to merge the two.
This makes me so incredibly depressed.
The field of ultrafast science brought forward by the advances in ultrafast laser technology (Nobel 2018!) is exploding. On the largest scales, the Linac Coherent Light Source and similar machines are generating a lot of buzz. It hasn't yet reached the public recognition of the LHC, for example, but it's only a matter of years.
A colleague of mine was just hired as a data scientist at a major tech company. Based on the interview questions I heard from him, it seems that the data science field is a natural extension of the kind of data analysis we perform daily.
I worked various SWE jobs initially, mostly C++.
Then I started a Phd in Physics, but didn't finish, bc I did a startup [1]. The startup failed.
Since then I've been working in various data/science roles at companies (Prezi, Facebook, now Fetchr).
Data Science is a perfect fit for me (CS+Physics), there's nothing around it that I can't do, from setting up stuff on AWS, building dashboards, A/B testing, getting data out with SQL, doing ML with SKL or Pytorch, defining metrics and setting high-level company goals, explaining stuff to the CxOs.
I'm really good at it, I have very high impact, get paid good money, make my own rules (mastery, autonomy, etc).
Overall I got lucky because data/science exploded, and it just so happened that my interests/background/experience were a perfect fit.
I got doubly lucky bc now Deep Learning is exploding [2], which is a perfect playground for people like me (play around with models, metrics, training, loss functions, etc.)
I still buy lots of Physics books and sometimes read papers, but I'm trying to quit that, bc it's mostly pointless.
I just sent my first paper for publish, a new design for a gravitational wave detector with reduced quantum noise
The reason was I sucked in physics, did not find my PhD position motivating and loved computer graphics.
13 years later I'm a valued technical contributor in a team in an ISV creating a valuable global software packages in the CAD field.
Middle class income, could probably make lot more in US with my skillset but family situation really is not awesome for expatriation.
I'm doing OK.
With physics skillset you can pretty much make your career what you want it to be. You just need a proactive attitude to read on other fields. You need to understand the other guys mindset so you understand the overall game going on. It's usually not nefarious (although it can be), but most of the time the rules that are used are not the vocalized ones. In this empirical physics is a wonderfull philosophical background. Organizations have certain dynamic rules which half of the people are not aware of. You don't need to "play the game", but you need to understand the rules so that when the wind blows into the direction you want to go into you can grab the opportunity.
I had a pretty good idea where I wanted to be 13 years ago (R&D in a position that values quality over quantity and speed with a great team) and that's pretty much where I am now.
You need to know where you want to go, and go there. No one will guide your path. Physicists are an outlier but that math and mindset is really an asset. Just don't get stuck in fixing bugs in some legacy monstrosity, that is absolutely soul crushing. But such a position can function as a stepping stone if you are operating in the industry you want to work in.
What's an ISV?
ISV - independent software vendor. An entity having an ownership of a software product, usually developing the software as well.
The other entity this is often compared against is the ESP - external service provider. I.e. consultants.
I spent years with legacy monstrosity and it does develop important skills. If the organization is otherwise ok it might not be that bad - but generally production software is absolutely horrible. The thing is legacy maintenance is important, and there's quite nothing like it that will teach you about the lifecycle requirements of software development. But I find it much more fullfilling to maintain and develop software that is alive and well and actively kept out of the 'legacy' label.
Not all old software is 'legacy'. One definition is that does it have tests. I've found an equally good definition of 'you are not afraid to modify it' and 'you can understand what changes in one place do elsewhere'.
So to be clear, I was not saying "anyhing but greenfield development sucks".
Physics undergrad (2015) with thesis in nonlinear dynamics, but have done research in astro. Always wanted to do GR-related research.
Landed an awesome job in quantitative finance where I write computational code (distributed computation) for various purposes (researching markets, analyzing risk, etc.). Job is interesting - but not "capital I" Interesting in the same way that physics was. I guess that is part of the trade-off.
I'm considering going back to academia - especially in light of the G-wave phenomena finally having sizeable datasets to analyse in recent years - its getting increasingly difficult to keep my studies of physics/math at "hobby" level.
Interested to hear if anyone else out there has "gone back".
I did a PhD in condensed matter, finishing in late 2014. I'm now working as a SWE at a quantum computing startup, focusing on internal manufacturing and test data. My graduate experience is very useful for this position, even though I'm not doing much physics directly.
I absolutely don't regret getting a PhD -- I went to a well-run program, had a great advisor, and met tons of awesome people. I also don't regret leaving academia. I enjoy the "working on hard problems" bit a lot more than the "unlocking secrets of the universe" part.
The number of basic research jobs in physics is a pretty fixed supply, since it's determined largely by public funding, and there are always many more applicants than jobs. Therefore, any trends you've noticed about physicists leaving the field for other jobs is much more reflective of the number of students entering the field (always rising) who eventually must leave, and the extent to which they publicly discuss their experiences. If anything, hearing about more people leaving physics would be a positive indicator for the desirability of the field, since it just means more people tried and failed to obtain a slot. (As it happens, I think fundamental physicists research is a pretty diseased field, but a shortfall in researchers due to PhDs being drawn away to greener pastures is not a symptom.)
Note that applied physics is different, since there is an actual market in researchers for industrial R&D; a shortfall in jobs really could reflect a failing field. My vague impression is rather that applied physics is booming, but know very little about that.
And just to be clear on the numbers: If, over a career, every research professor mentors N grad students, and the number of research professor jobs grows slowly (as it has since the explosive growth in prof jobs that peaked in the '60s has waned), then the chance of any PhD becoming a professor must be about 1/N. If you look at the numbers, that chance is a few percent, which agrees with the typical professor advising ~20 students over a career. That means ~95% of PhD are going to be leaving to do something else.
Anyways, for me personally: I have a PhD in quantum information and am currently searching for a mathematical definition of branches in a many-body wavefunction. This would potentially lead to large computational speed-ups in numerical simulations of out-of-equilibrium systems. I'm in my 7th year of postdoc-ing which...is not ideal.
Happy to expand on my experience if that would be helpful.
I love physics.
Back in 2007, I was writing my diploma thesis (think master's just twice as long, German system prior to bachelor/master) at Auger. The collaboration has a rule that for the first year in, you don't go into the author list, but stay on it for a year after you leave. That optimizes for having the actual contributors on papers. Towards the end of my not-on-the-papers year, the collaboration published a really big paper on correlations between the arrival directions of the highest energy cosmic rays, and certain objects, active galactic nuclei (giant black holes at the center of other galaxies). This didn't just get published in Science, it made it to the cover. It was a step towards working out where these mysterious particles are coming from, and what mechanism might possibly exist to accelerate them to such energies! I was kind of bummed not to have my name on that paper (though even at the time thought the one year rule made sense).
The publication had been hotly debated internally. It had received all conceivable internal scrutiny (the author list contained over four hundred names). There had been a stark debate about what the right statistical significance was for claiming a discovery. Astronomers were used to less rigorous requirements than the particle physicists that together made up the collaboration. Ultimately, it was decided to claim a correlation. (A nil result would have been published as well.)
Almost from the day of publication, the statistical significance of the result, but in data continuously collected since, started to diminish. I was involved in running the nightly data reconstruction, and often times we would huddle around one of the workstations on the morning, to check whether a new high energy event increased or decreased the significance. It kept going the "wrong" way.
The collaboration went into a frenzy to double check everything. They redid the calculations, checked all hardware (a field of water tanks and electronics 3000km^2). Had lots of internal conferences with heated debates.
Ultimately a note was published describing the staggering decline of significance. If I remember correctly, my name was on the author list by then and I got to joke that I knew it all along.
I'm no longer in the field. Since then, the collaboration has published a di-polar asymmetry of the arrival directions of the highest energy cosmic rays. That proves something we already pretty much knew: these are not particles from our galaxy. That was a meaningful discovery. But I'm not aware of a publication with as exuberant a discovery as the source correlation.
So I started studying Python and since them I got various jobs as a developer, including 7 years working in Finance, where I did a lot of Postgres and also some web programming. In the last 6 years I have been doing numerical simulations for Earthquakes, helping the geologist on the IT side of things (like parallelization and performance). The funny thing is that all work in Physics I did had nothing to do with computers, except for writing papers in Latex. I was doing analytical calculations with paper and pencil, since Mathematical was not good enough (found a lot of bugs in it when computing integrals).
One thing I noticed is that because physics has multiple decades more experience with dealing with big data compared to just about every other scientific field, a lot of physicists who jump ship tend to end up in a position where they can apply that expertise.
I worked as a programmer for a molecular neurobiology research group for two years. Biology is going through a kind of Cambrian explosion of new data (especially when it comes to anything that involves genetics). So it's probably not surprising that a number of people at work told me that it is extremely common to see physicists switch to biology because that's where all the exciting new research is happening, with new theories and discoveries, and lots of people who are very happy to steal whatever the physicists have already figured out about how to process and interpret mountains of data.
I've been learning it for the first time recently, and there are data science problems that are somehow tractable in Mathematica that were very hard for me to do in Python. Some of this stuff, like FindDistribution, seems only to have been added in the last few years. The random process library is really amazing as well.
Any blog posts on mathematica helping make data science problems more tractable?
My thoughts are mathematica is good at intuition building, but not fast enough to deploy without converting into subsequent languages.
The physics background means that, even if I don’t understand the exact science of why my coworkers are working on, they don’t have to worry about explaining everything to me.
Even my boss forgets I don't have a PhD. There's a huge need for people that are really good at software development (relatively speaking) that don't get tripped up by the physics, whatever it is.
Basically labs/institutes and large universities that get government funding looking for software developers.
National Labs (though it's getting rougher), flagship universities with large labs or targeted programs like MIT (Lincoln Lab), Johns Hopkins (JHAPL, STSCI), UIUC or UChicago (NCSA, UChicago pioneered much of grid computing but some of that was Argonne if I remember too), USC, and especially smaller labs/institutes or even specific projects at large universities. You can also look outside the US, France (INRIA, IN2P3) and Germany (Max Planck institutes) can be good places, Italy is okay, the Netherlands is good but not as large, UK is good, Australia is okay. In Europe you can't expect to do anything with CERN unless you are working at a National Lab in the US generally. Japan does tons of stuff but most people I know who have worked on Japanese projects in Japan don't enjoy it as much.
Secondary jobs are also a possibility near big national labs (Bay Area, New Mexico, Illinois), but usually that's less software and more big-E engineering.
Materials science and bio have a different layout but much of the cutting edge of that tends to be located near light sources these sorts of facilities too, and the need for software devs is growing there.
Edited: Re-reading batbomb's reply, I also remembered a friend who is currently a data scientist for a venture-incubator, but was previously at a national lab. Many of the projects he described were very cool, and challenging.
Though theory groups in general tend to use computational simulations as a tool to complete calculations, groups that develop novel computational methods and techniques tend to be headed by younger, more junior professors. These groups are typically well-funded and do very exciting (trendy? cutting edge?) work with distributed computation, machine learning, neural networks, etc, so they tend to pull quite a few students.
While these computational groups tend to bring in funding and are well-staffed by excited grad students, the junior professors leading them tend to be marginalized by the more traditional, seniority-focused establishment. Which is to say, a new PhD might have a lot of trouble landing a prestigious postdoc because a) their adviser might have been too young to have high name recognition outside their field and b) departments might place limits the amount of staff for these more junior professors/young groups doing exciting computational work. This is, of course, on top of the overall scarcity of jobs in academia.
But there's no such job scarcity in industry-- especially not for stats-smart programmers with years of experience a) wrangling data in python, b) writing fortran that runs on distributed clusters, or c) designing algorithms to solve /approximate hilariously expensive problems. Advisers know this and point some of their students who might thrive more in industry than academia towards that route.
(Anecdote: And of course, as a physicist who builds models/simulations in industry, I can speak a personally a little re: thriving. If you're someone in love with solving disparate problems, you're unlikely to find that in academia. Some of us learn in graduate school that we can't spend our whole lives-- or in my case, more than a few months-- solving one problem. Academia just... didn't seem like something that would be worth fighting for.)
I assume that this will gradually change as there's turnover within physics departments and we get more computational-first professors with seniority (or even in leadership). There are a few departments with better-known professors you can see it happening now. Universities are spinning up incubators and institutes for computational research. Physics departments are just slower to adapt to new developments, and the hierarchy of theorists can have more to do with seniority and internal politics than it does with technology.
The fundamental issue is the field is not growing (very much). Each professor will graduate 20-50 students over their career, but only one will get their job when they retire (on average).
> Some of us learn in graduate school that we can't spend our whole lives solving one problem. Would you please expand on this. I am not sure if you meant that problems are hard enough or what.
It was 2003 when the company I was working for acquired InterBiz from Computer Associates. By the time we were integrating teams from both companies I met this guy who were the guru, the #1 product manager from one of the most profitable lines of business that the company had. I remember the shock in people's faces when he, a Portuguese guy, said he's got a PhD in Physics, in Germany, in a second language to his own, but then ended up developing Warehouse Management Systems (WMS).
Yeah. I keep kicking myself for not taking the Wall Street guys who called me more seriously. That would be one important message I'd like to send backward in time.
I remember some of the Physics departments I knew well commenting that they were getting 250+ applications for each open tenure track position (early/mid 90s). It skyrocketed to 1000+ at one point for a few places.
Based on some back of the envelope math, and a realization that a fresh Ph.D. with ~6 publications (2 PRL, 4 others) would not be a good competitor to a senior FSU physicist with 50+ publications to their name, a solid reputation, and an interest in moving out of the FSU quickly.
I determined that market forces had flooded the supply side of physics with, well, extremely good candidates for the same positions I was applying for. And my ability to compete with them was low based upon the main components that hiring committees cared about.
So I looked elsewhere. 20+ years later, I've made a good career working in computing, but I really do miss physics and research in general.
This was certainly a lot fun, but you won't be getting any thing particularly revolutionary out of it. I remember my advisor at the time running a few dozen CPUs of our poor excuse for a cluster (built mostly by yours truly with off the shelf components) for the entire summer to add a couple of digits to the ground level energy estimate of a completely artificial magnetic model :)
On the bright side, it taught me a lot about programming, numerical simulation and optimization
> So why don't we have a good theory of brains? People have been working on it for 100 years. Let's first take a look at what normal science looks like. This is normal science. Normal science is a nice balance between theory and experimentalists. The theorist guy says, "I think this is what's going on," the experimentalist says, "You're wrong." It goes back and forth, this works in physics, this in geology.
> But if this is normal science, what does neuroscience look like? This is what neuroscience looks like. We have this mountain of data, which is anatomy, physiology and behavior. You can't imagine how much detail we know about brains. There were 28,000 people who went to the neuroscience conference this year, and every one of them is doing research in brains. A lot of data, but no theory. There's a little wimpy box on top there.
Not that physics has no theories, but I dropped out of studying physics myself over a decade ago, and at that time it felt a lot like the balance in physics has shifted towards having to measure and process disproportionate amounts of data with so much precision that it has to be automated, or like you said do a ton of really complicated modelling. It feels a bit "stuck" that way.
[0] https://www.ted.com/talks/jeff_hawkins_on_how_brain_science_...
Most parts of the Standard Model are verified to 5σ at all energy levels that are accessible to us in everyday life and in contemporary particle accelerators, so it's increasingly unlikely that we've missed any obvious mechanisms.
If we're going to find some completely new physics (in the same way that quantum theory and relativity theory were completely new at the turn of the 20th century), it's probably going to be at energy levels that contemporary particle accelerators cannot yet explore.
EDIT: That's not to say that we know everything that is to know about the accessible energy levels. There's so much in our theory that's yet unexplained, e.g. for all I know, we still don't have any idea why time works the way it does.
The reason for is just how damn good all the Physics Theory is. People keep having to look closer and closer to try to find a place, any place, where the theory "fails" (in a sufficiently spectacular fashion) hoping that that might point the way to new phenomena and new theory. There's a giant multibillion dollar hole on the ground in Switzerland who's main purpose was to find a chink in the armor of HEP or help point the way to new theories. It failed spectacularly, but they're already talking about building a bigger one :)
> (majewski's comment below) I think you're seeing this in physics because, in terms of experimental accessibility, the low-hanging fruit are mostly picked.
I mean, yes and no. The Standard Model is amazing. But I think we're also kind of focused too much on the Big Questions, and in the arena of the not-so-big questions there are still a ton of gaps and missing insights.
Perhaps in the not-so-big questions, most of the low-hanging fruit may appear to be picked as well, but I suspect that is more a kind of myopia that can be solved by collaborating outside of our field.
Whenever I see an article these days about physics that really excites me, it tends to be a story where one or more physicists got excited about a seemingly small question, or ate a bit of humble pie and joined forces with chemists, biologists or some other field, and or both, and that working it out turned out to bring far-reaching consequences and novel insights. Same with maths, really.
I'm aware this is completely personal of course, and that there is a likely bias in that this kind of research tends to be more accessible and thus more fun to read for me, but still.
Awww it isn't that bad. They found the Higgs and at least verified we need to look at higher energy levels to find anything new.
He convinced me (and a few other phd physicists) to help him tackle the problem of industrial controls for heavy industry (think large refrigerated warehouses, steel refineries, food processing, etc.). We we're all graduating so we decided to give the startup route a try. Since then we've been designing/deploying cloud-based control software to regulate the energy of these huge power consumers. https://www.crossnokaye.com/
In the day-to-day its more data science/computer science than physics but the core models we design are physics based so our white boards always have some derivations on them.
^This. LOL. IMHO a lot of opportunities arise at the borders between fields. This helped me put my finger on it; physics and its related math are a great basis for interdisciplinary work. Selling that idea to SW-company HR and hiring managers can be hard, in my experience, but startups more often benefit from versatile employees early on. Cool business idea BTW!
Now I do data "science", and create data products.
The main downside is that now that I don't have a lab, but only a laptop, my eyesight has changed dramatically :) Upside is pay and work/life balance.
Like how much are we talking here? Do you feel like you've found strategies to mitigate that?
Such lenses will "push out" the focal distance of your screen, so it is optically closer to infinity. If you already wear negative-diopter glasses, for nearsightedness, you can get a milder negative prescription for close work or check out the new "computer" lenses that are multi-focal and give you different zones for nearer, farther, and in-between.
Your eyes "work" to focus more closely-in, via contraction of the relevant muscles to thicken the lens. So prolonged close-work, such as with a computer, will cause fatigue and eye strain. There is also decent evidence, though it is debated, that prolonged exposure results in a trophic response and acquisition of myopia (nearsightedness). I've seen this empirically multiple times.
Wearing glasses for nearsightedness while you're doing close work is a double-whammy. They make things look even closer. At the minimum, don't wear these glasses if you don't need them at the computer.
Hope this helps!
BTW if you don't entirely trust ergonomic advice from a random dude on the internet, your local optometrist can help. There is a type called (in the US) a "behavioral optometrist"; these folks generally are more focused on optimizing for occupational needs like this, vision therapy etc.
PS. Hey, just realized this is a physics audience! You guys can figure this all out yourselves with the thin lens equation... :)
It's nice to read all the other survivor stories in this thread of those of us that spent years studying to be physicists only to be crushed by reality. Physics is tough, and I'm happy to read that many folks here have found success in the field.
https://medium.com/the-mission/elon-musks-3-step-first-princ...
I then left aerospace in 2017 for a quantum computing startup [1]. I'm currently focused on simulation software, where my physics background is certainly useful.
I like to think I'd still pursue a physics PhD if I became sufficiently obsessed with a specific topic.
Also, I'm pretty happily obsessed with physics. It's really heartening to read so many Physics folk ended up in software. I kick myself for not going for it sometimes. Then I buy another book.
BS Physics 2008. Somehow landed a 'dream' job with a major DoD contractor, despite the recession (total miracle). They closed up our plant, due to the recession, in 2010 and I moved to a smaller contractor. Got through the whole clearance process only to find that my (now) spouse was a lot better choice than the rest of my team. Good work, but the heart wants what it wants. Jumped with no safety net, and got into neuroscience where my spouse got into grad school. Worked there for free for a few years and got into grad school. Boy, was that a mistake! Horrible grad experience in neuroscience and quit with an MS. Was unemployed for about a year with a nasty depression and health issues in the family. Finally working in DBA stuff and data science. Still love bio/medtech and neurosci, but there just aren't the jobs here (need to stay local due to family health issues).
Overall, not that bad considering the recession, doing about average against my other graduates of 2008. Still, the corporate DBA stuff is ungodly boring and the family health issues aren't a snack.
In the end, we all try really hard, but kids, health is everything. Everything else falls to the floor in the face of health issues.
Looking back perhaps one major regret I have is how far and fast I am moving away from fundamental sciences. I don’t think I can leave the bay area or coding anytime soon, but nonetheless, I have started to look out for ways to stay involved with the world of physics in as many ways possible
(OP here) Quite amazed to see the number of physicists here and it is very heartening to see so many of you doing so well in such a wide variety of fields. Especially since during my college days physics was considered to be a slightly dangerous choice from a job/career growth perspective
I cannot find a 'real' (meaning permanent) job in my field. I applied to about 50 tenure-track university and college positions and staff scientist positions. I applied for early-career fellowships from the Royal Society, CNRS (France), the Helmholtz Gemeinschaft (Germany), among others. Almost everybody I know thinks it's crazy that I don't have a job yet, but nobody has the money to create one. So I'm moving to UMD for a 2 year non-tenure track Research Assistant Professor job to give me 2 more cracks at the job market.
Lattice field theory is a computational technique by which we can extract approximation-free, fully non-perturbative from quantum-mechanical theories (I've described it on HN in a variety of comments, see eg. https://news.ycombinator.com/item?id=15782932). We use an enormous amount of leadership-class computational power.
Physics is now very computational (even theory), and often works with data sets that make industrial 'big data' problems look like toys. I mean, one of our lattice QCD calculations produced hundreds and hundreds of terabytes of intermediate results. Data analysis, correlated analyses, and all sorts of things that were old hat for physicists suddenly became lucrative. And presumably it's a lot less soul-sucking than further "improving" high-speed trading.
A lot of successful physicists have stories about unorthodox career paths and lucky breaks. This should be a red flag for anybody considering study of physics. But maybe it suggests that exploiting opportunities and lucky breaks is part of what physics education is about. You have to decide if you want to live your life that way.
Why didn't I go into engineering? That was kind of an accident. Note that when I was in high school thinking about what I wanted to do, the digital revolution had barely begun, and maybe engineering still seemed a bit stodgy to someone living in a sleepy suburb with little exposure to the world at large. I had intended to major in math at a small college with no engineering school, and ended up adding a physics major and heading to grad school. I loved experimental science, and thrived in the lab. My parents are both scientists, and had pretty good careers, so there was that whole role model thing.
At my present job, we have a full engineering staff, including programmers. Why do we need scientists? There are actually a lot of scientists working in "engineering" organizations. I've noticed that the scientists tend to be more multidisciplinary and quantitative. Whatever the difference, I think it helps to have both perspectives. I get handed weird, unsolvable problems, that can't be categorized. I develop a "system" view of how things work. I work on manufacturing problems, customer applications, and so forth. I'm one of the "math people," and I handle weird things like understanding measurement noise. I actually like theory.
When I think about whether I should have been an engineer, I remind myself that I might have failed at it.
I've been pretty lucky. My job isn't glamorous, but I've had a good career, and my job has never been super intense in terms of stress or hours. I enjoy my evenings and weekends.
Plus, the parent never even "shamed" anyone, they just said it makes them sad that a person they admired is doing SEO of all things.
But at least this guy is honest about his goals.
Physics turned out a little more dull than I expected (I wanted something with more creativity), and an app I’d started working on as a hobby turned into a full time income, so I started pursuing this instead (https://classtimetable.app).
I took a few full time iOS jobs, continued to improve my engineering skills, and I’m currently working for a top five tech company on a popular iOS app.
Physics certainly taught me a few skills that I use on a daily basis - math and problem solving are good examples. I didn’t move to software specifically for the money, but comparatively physics seemed a little more dull, and software seemed to have new and exciting opportunities (like the app that I started building as a hobby).
On the hardware side, I enumerate and evaluate qubit topologies, and solve combinatorial puzzles of packing of qubits, couplers, couplers and their control structures, for my team to implement said topologies. Our processors are a fun mix digital and analog, and in development, that's "digital until things get too analog"
On the software side, I research, write and maintain embedding algorithms which are used to fit problems onto the chip, and I also work in hybrid quantum / classical optimization and sampling algorithms.
May I ask in what company do you work? Also, what kind of study did you go through? I'm almost at the end of my bachelor in mathematics, and I want to get close to physics and quantum computing. Your experience seems relevant!
At the moment I'm analyzing data from W7-X in Germany. It's a really cool device off the beaten path of tokamaks but seriously catching up in performance [1].
[1] See fig. 15 of https://iopscience.iop.org/article/10.1088/1361-6587/aaec25
The field was really interesting, but building a carrier in it is a pure lottery - hard work and talent alone won't cut it, you need connections, politics, and salesmanship skills to get a permanent job.
On top of that, there were probably only three job openings a year (in the whole world!) that I was a good fit for. Money factor did not come into play at all - junior dev salary is often lower than the postdoc one.
I'll probably retire early one day and then work on other interesting stuff I like as well, e.g. rocket science which I did briefly.
>are there any other reasons for this as well?
I love physics, but I also love a lot of other things. I was a musician well before I was a physicist. My exit from "doing physics" was driven by a desire to start my own company. I found I really enjoyed developing software for the web, it was a nice blend of technical and aesthetic.
I think the draw of physicists towards data science is because of the familiar mathematics. My experience in recruiting data scientists is that candidates that have formal degrees in data science generally have little experience outside of school. My assumption is that these degree programs are relatively new.
>Of late, except for few headline-friendly fields (colliders, quantum computing, gravitational waves and astrophysics in general), I don't get to see/relate with a lot of activities in Physics
Do you think this used to be different?
A friend (also a physicist, though harder-core) who is a now a data scientist opined to me that the current wave of cross-trained data scientists will be replaced by "kids with their new data science degrees from new data science programs". He didn't mean this ill, but simply thought there was a window of time to make a shift.
Your comment implies you are actually seeing relevant benefit from people with more diverse experience. Can you elaborate? Do you mean lots of hands-on data science per se, or simply a broader practical experience?
Edited: for formatting
My experience with hiring is limited, I've only hired 3 data scientists. One had a bachelors in physics and masters in data science. One had a masters in data science (i forget her bachelors degree) and one was a PhD nuclear engineer.
I think the main difference was "broader practical experience" as you said. That' was mainly my point. Although my sample size is small, I was just making the point that the degree programs are so new that graduates can't have much real work experience. Obviously if someone returns to school after years in industry and gets a MS in data science then they could have work experience as well.
A surprising amount of people whom I studied with ended up as programmers themselves, after finishing the physics degree.
Then again, I guess I value stability and predictability in my day job a bit more than most Silicon Valley startup employees.
But I enjoyed my time as a physics student. Physics is an interesting field to study even if you don't end up working in it, in my opinion. :)
I also have always liked computers, so the switch is a good fit for a lot of reasons. But I'll also say that it feels like a completely different world (same with when I worked as an engineer).
Now I'm doing aerospace vehicle modeling and simulation in MATLAB and C++ and primarily work with other physicists. This is by far the most enjoyable work I've done and the pay is excellent - my salary is 3-4x the average household income for the city I live in.
I've spent a lot of time over the past year thinking about what my next move should be. In this time I've also become hooked on coding, mostly python. My current plan is to pivot into (you guessed it) data science. I expect that this will lead to better pay, vastly more potential employers, and allow me to get back to working on interesting projects.
In hindsight, if I could do everything over I'm not sure I would go into physics. What I've seen is that physics is mostly a field of niches that are filled by specialists. To go far in physics you need to become a specialist, however this can really limit your options later. I think this is why many physicists eventually find themselves going into other fields like data-science.
Went right into industry after PhD working for a life sciences company in systems engineering (the product design type, not the computer networking type). Now managing a small team designing robotic systems to automate chemistry/biology research.
Best job-hunting advice I've found is "Guerrilla Marketing For Job Hunters"
My new job in industry is consulting about HPC systems in the context of computer aided engineering.
I had two kids during my postdoc and quickly became disenchanted with the prospect of hunting for postdocs in a random part of the world. I was as interested in statistics and machine learning techniques, so moving into industry was not terrible. I still love the formalism of supergravity, but it looks to be becoming less and less relevant in hep-ph.
You're absolutely right that data science is a common destination for exiles. It makes the most sense because we get to still read interesting mathy papers, develop computational tools. The mechanics are very similar for physicists (in certain fields). Literally every single former physicist friend I have on LinkedIn is working as a Data Scientist, except one who is teaching physics at a private high school in NYC.
When I left my postdoc, I worked at a data science startup for a year, and now I do general software and applied ML at google.
I am greatly excited about ideas at the interface of probabilistic inference and quantum/statistical physics. On the one hand, these should help create better ML models/algorithms; on the other hand, I believe that tools from probabilistic inference will help better understand emergent phenomena in complex systems. The former is what I'm focusing on right now, the latter, I think might take a couple of decades.
When graduating with a PhD and thinking about what I'd like to do next, I didn't think I was a good fit for life on the academic track (post-doc, tenure-track, etc), given the kind of questions I wanted to think about and the manner in which I wanted to pursue them. I also wanted to gain some experience writing software and applying ML to real world problems (as a "regularizing" effect on my theorizing), so I took the path I did.
I've come to realize that I'm a researcher at heart, and it's difficult for me to not spend time exploring new ideas. I just need to find the time and space to do that, and I'm trying to structure my life so that I can.
> Also I have noticed a growing trend of physicists becoming data scientists post phD. Although I understand the money factor, are there any other reasons for this as well?
I think this has always been the case, at least as far back as software in the '90s and then finance and now "data science" added to the mix. The typical physics education/training makes one a generalist with a broad background in problem solving and mathematical tools, and a flexible mindset, so that one can adapt to be effective on the problem du jour, while there is a dearth of specialists with the specific skills necessary. I imagine this overarching trend continue to be true going forward as well.
My first post-doc was in laser-matter interactions. We had an experimental team that were focusing PWs of power on to ultra-thin films and separating the substance into its constituent parts (electrons and protons I mean). The plan was to increase the proton acceleration yields & make a consistent, tight bunch (in the energy spectra) so that we could use it for next generation cancer treatments. That's a long way off and no-one really has a good idea what else we can do with this system.
That annoyed me. There's not enough application or relevance there. So I've moved to Earth System science. I do a lot of global climate-economy coupled models & attempt to implement climate models with human decision making as a part of the system rather than some form of external forcing.
The transition was an easy one for me as I found a particle physics collaboration shares many characteristics with a start-up.
One thing to note is that responses here will be biased towards physics people that are more interested in the tech side than the physics side.
Working on data science/ML now.
Started undergrad in 2008 and had to decide between CS and engineering physics. Went with the latter because the school had a particularly strong program.
Applied for PhD programs and national lab positions in 2012. The PhD route seemed more interesting, so I went with that. Ended up spending 6 years on materials simulation.
My peer group ended up going to academia, software engineering, and data science. Technology companies seemed exciting so I interviewed for software engineering and data science roles. Ended up with a company that predicts accident risk from driving data.
I think that physicists are particularly well suited to the machine learning space. I was trained to work systematically work through problems with no analytic solution through clever approximations. This tenacity can help with tackling problems in the machine learning space.
My background in physics and math really helped in CS classes I've taken since graduating, and the general technical background and comfort with math has been very helpful in general. I do wish I had done more research and programming in college, as I ignored them in favor of experience that would line up better with work in DC.
Also depending on what level of magnification you use to observe the markets: they are just random. Maybe there is some consistent process that works today on some subset of the market given a certain forward-looking horizon.
Then, that opportunity disappears at some other time in the future. Its ephemeral and fleeting... and this is probably where the daily challenge comes in.
Working on my thesis analysis (collider based HEP), squeezing in time for OSS development and prepping for a jump to software dev.
I love physics, but academia is in a rough place right now. Almost everyone I know pursuing the academic career path has nightmare stories.
The physics job market has been, and will remain in a "rough place". As with all things, it is who you know.
If you want to stay in the field, network like mad. Get people to know your name. Make sure you have work known to them. Get a great Postdoc with someone who will make calls for you when you are done with your project, to help your search.
Otherwise, computing is nice :D
Navigating the politics and culture of the academic world never really made sense to me, and at the time I felt that I was not smart enough to ever really make a meaningful contribution to Physics.
I do not regret studying physics; I think the mental stimulation and growth I gained from those challenging 4 years of study have served me very well and actually made me a smarter human. However, I am happy to apply the logical rigor and analytical skills I learned to more simple and immediate problems in business.
my research in undergrad and applications for grad school were for computational neuroscience (sort of the whole what would Feynman do if he was still alive route)
1. Didn't get into as many programs as I wanted.
2. East Coast schools even explicitly told me that wetlabs were more important (I semi-agree).
3. West Coast schools were all being gutted for ML/Data science work.
4. all the post docs and grad students I had worked with had switched to doing ML
I deferred for a year and worked at a biotech startup doing neural network simulations to prove the product worked and scrappy hardware startup things.
I've since been at a startup doing NLP for the last two years.
Don't regret the degree which to me is like a STEM liberal arts degree.
If I could do it over again, I'd have dropped out of grad school in Year 1.
As an aside - there are way more physicists on HN than I expected.
Haha. I can see how you are trying to take on the current education system itself. To some extent, the education system has failed us to some extent.
I left academia after my postdoc and went to work at one of the think tanks in the DC area for a few years. When I got tired of being a government contractor and left to work in natural language processing and machine learning.
Now, I build machine learning software to detect financial misconduct like insider trading. I also built software that helps law enforcement find minors who are being trafficked.
Some of the mathematical and computational methods used in physics are used in machine learning, so it was a relatively painless transition.
I'm also developing some mobile apps in areas where there isn't already saturation.
It's staggering to see your Stanford and your Harvard grad students become scientific experts only to then work on improving ad delivery and how to move someone else's money across the world to make the rich richer. It feels wasteful. But there's no room for everybody, there's very little room for a few and it's stressful, competitive and the path is riddled with anxiety and mental illness, which I've had enough of already in grad school.
People move to data science because it's a transition into the software industry that is eating the world. It's a safer bet, pays well, and has the name "science" in it :) It helps people transition.
I've stalked many on linkedin over the years and it does seem that science is just a phase, very few are still doing science. I think all these people liked or loved research, but the combination of the nasty, petty environment surrounding research, and the lack of permanent positions makes it very difficult for someone to have a fruitful, worthwhile career in science in my opinion.
I think we have a problem. We have a lot of smart people but we don't know what to do with them. And I think no one up high is really questioning what is the purpose of a college education and of a post graduate education in order to modify these programs to better address a student's path. Academia changes one funeral at a time, and that might be too slow to tackle some of the problems we are facing.
I've really enjoyed the research aspect, but I have come to hate my interactions with my PI who is sometimes too busy doing administrator stuff and in my opinion sometimes refuses to see reality for what it is: Sometimes to do modern research you have to invest in modern equipment. Sometimes I don't understand why my data looks a certain way, why is anger the default reaction to this?
I haven't had enough time and space to gauge whether or not this phd is worth it, all I know is that I'm really excited to leave an environment that has broken me a few times, and that continues to inflict pain on others I love.
My perspective is that there is just not many academic positions available. I realized I could do a couple of post docs for a few years and hope that something opens up, or not delay the most probable outcome and start an industry career sooner rather than later.
I left physics for data privacy in 2017 (PhD programme).
The main reason was that I didn't really feel like I was tackling real issues (real and immediate to society) if I continued work in physics.
The swap has been difficult and I am at a disadvantage wrt to colleagues that come from more relevant backgrounds (e.g. Computer science or Applied maths) but at the end of the day I feel a lot better about the contribution of the work (to society) and also about job security in the future.
got a job at Intel as a back end process engineer, which was pretty cool, but I was then laid off in 2015.
These days I'm all about machine learning. Teaching myself by creating content, and hopefully educating others. It's been really cool seeing the overlap in some concepts (systems seeking minimum energy vs. gradient descent) between ML and physics, and I'm hoping my background will pay dividends as I get deeper into the field.
I get my information from scientific articles and approaches from clinicians in the field even if you can't prove everything 1-to-1.
This approach gives me loads of credibility in a space that's filled with very unscientific approaches.
I'm deepest in probiotics and digestive health. But the basics for improving health are quite simple - good sleep discipline, clean water, understanding signs of nutrient deficiencies, blood sugar management (even for non-diabetics), stress management, movement.
Now my job title is "data scientist".
Now I develop software for a medium-sized ISP and IT outsourcing company.
These days I'm back in the UK (Brighton) and enjoying the arrival of my first child. I'm still working in Node for API and unusual proxy servers, with some Rust on the side. Enthusiastic about the indieweb movement.
Now I work optimizing fantasy sports teams and building websites that display betting lines. Don't even like sports and I have no idea what anyone in the office is talking about, but they are paying a lot for me to put buttons on their website I guess. Leaves me a bunch of room for my hobbies. Offered to do some actual math, maybe personalization algos or some AI stuff. Really just need these buttons on the site is all.
I don't regret getting the PhD. Always was very interested in physics and learned a lot. Did a lot of computational work which helped with the transition to software. I still use the problem solving and some of the math I learned for physics.
Today I teach math and physics in high-school and feel like never having had a better job in my life.
Where do you guys search for jobs, in Europe? I was mainly looking for a Data-Science position, though I don't know ML yet (other than regressions ofc)
I'll also be joining the data science world.
There's some debate about whether it's the best or worse time ever to be in particle physics. Either way I see a field that is overstaffed. Add to that the fact that CERN accelerators are shut down for two years and we're in the middle of the european strategy review.
Seems like a good moment to take some time out
While my PhD did start out as supposedly being on Spin Glasses it quickly diverged to Complex Networks and what people would now call Data Science. Since then, I've worked on Social Science, Epidemiology, Human Behavior, etc... For the last year or so, I've been doing Data Science and Finance in one of the Big Banks.
You can look at the results here: https://intelactica.com
I'm happy with the change, my life is more relaxed now. I've time for my family, for me. I've also better and stable earnings.
Now I work on educational software trying to extend my reach.
There's so much which I don't know about everything outside my field of work (and inside too, I guess).
I don't see myself staying in academia though I love research.
I've been watching compressed sensing (CS) from the sidelines for the past decade, while wearing experimental-cosmology, synthetic-aperture RADAR, and instrumentation/FPGA hats. (Not electrodynamics or optics, per your question, but nearby.) Here's my perspective.
If you're doing compressed sensing correctly, it transforms the entire instrument from electrically complex and algorithmically simple, to algorithmically complex and electrically simple. Unfortunately this isn't a strategy you can bolt on to a traditional architecture and turn off as a de-risking strategy if it doesn't work. From the perspective of a team building a tool to accomplish a task, it's an all-or-nothing gamble. Consequently, the challenges to aggressive adoption of CS are both technical (can it work?) and programmatic (is it feasible given human, political, and financial constraints?)
In the fields I've been exposed to, state-of-the-art instrumentation is complex enough that domain experts spend their entire careers understanding the quirks of a few established instrument topologies. Where CS is applicable, it would take a big leap of faith from an entire team to build an instrument around CS. Before that leap of faith is accessible, the team would have to be conversant in CS research and the implementation of CS algorithms in practice. And, after the leap of faith, a successful CS project requires secondary leaps of faith from funding agencies to get these instruments built. These barriers are highest for exactly the kind of complex, expensive projects where CS is supposed to shine.
As an aside, the latest IEEE Signal Processing magazine (https://ieeexplore.ieee.org/document/8653526) has an interesting article on hardware architectures for compressed sensing. As CS progresses, and as CS researchers transition from pure-CS research to applied-CS techniques, the use of CS in physics will probably grow.
I am though ever in awe of the hype that ML/NNs are getting for the past 2-3 years, where as Compressed-Sensing is likely to be a more fundamental shift in many fields.
Vanilla CS theory assumes a very simple system model for the measurement process (ie, linear measurement) and shows that with a few assumptions you need far fewer measurements than you might expect. This by itself is a valuable observation.
However, I'm less impressed with CS as a practical tool because in many realistic systems the measurement process is nonlinear and difficult to model. That is, the desired system state you're trying to infer is some nonlinear function of the experimentally measured quantities.
I think the best instruments of the future will use CS ideas to motivate the use of an appropriate sensor domain to reduce the number of measurements required. But that actual computational reconstructions of system state (ie, inference) will be done with a nonlinear differentiable model which can be optimized using a data driven approach and with ML-like tools.
In addition to the benefit of allowing nonlinearities, this approach gives you a lot of freedom to optimize the computational complexity of the reconstruction (ie, complexity of the system model) vs accuracy of the results.
Physics is a great general purpose education.
Finished my PhD in quantum optics in 2014, but immediately moved to data science.
Why not physics? Full version: http://p.migdal.pl/2015/12/14/sci-to-data-sci.html
tl:dr: I wanted a fast-pasted field with more freedom. Physics is now stale (no fundamental changes in the last decades, compared to each year in deep learning; cf. physics in 1900-1920), and academia offers a rigid framework of grants and feudal dependence. In data science, as a freelancer/consultant, I get much more freedom. Even in companies, one is able to migrate in a matter of weeks, not years.
Money was a nice perk, not anywhere near to the main motivation.
For myself at least I would add job security as another benefit. I struggled in vain for 6-7 months during my lectureship applying to any and all postdocs I could find, so the prospect of cycling through 2-3 year postdocs in various locations eventually overwhelmed me. I loved the research but realised I didn't love it enough to potentially live apart from my partner and regularly go through months-long application processes.
On the flip side, for me freedom is actually something I miss from academia. Admittedly that's partly because I'm a regular employee rather than a freelancer. Outside of teaching, my time was basically my own and I was free to work from home or anywhere else. I'm somewhat resentful of having to justify what I'm spending my time on day by day. Not that my manager would particularly mind if I spent a day researching something slightly blue-sky, but the fact that I might have to respond to questions about it creates this background tinge of anxiety that I find prevents deep creative thought.
Went commercial rather than academic, as the job market was insanely crowded, and I didn't have "enough" differentiation in my opinion, to land a tenure track, at a lower tier school given the huge influx of high quality talent from the former soviet union (FSU). I was around during the whole Young Scientists Network days (early 90s) where we collectively did deep soul searching on whether or not Physics as an academic career actually made sense.
I joined a supercomputer maker in 1995 ABD, and finished writing up (my third rewrite, first was in 1994, final accepted one was in 1997) and defending . I stayed with them for 6 years, until I saw that they had no real hope of long term survival.
During my time in school, I'd been a consumer of Supercomputing systems across the US, and in my department. I decided that was the career direction I'd go, with the idea that I'd become an entrepreneur after watching successful companies develop and grow. Learn from them, not just their successes, but their failures.
Needless to say, my first job I learned a great deal. My thesis advisor was still trying to push me to postdocs with her former classmates, and I was tempted, but my wife and I decided to start a family, and that kind of nuked that direction.
I left the first place, and was recruited to help another company bootstrap an HPC division. That was fun, I got experience in all the non-technical side of businesses as well as the tech.
They had a financial crisis, and I took a small package and started my own HPC company. I ran that for 14.5 years. It was a wild ride, and I learned a tremendous amount (e.g. I failed in many non-fatal ways). Unfortunately, the last learning experience was in fact, fatal. I joined a cloud company and have been helping to build a "next generation" cloud.
My thesis advisor just retired, and she's been sending me things about Astronomy and Physics openings. They are mostly adjunct teaching things. I love teaching, I get a real blast out of it. But the adjunct life is a massive pay cut, at a time I cannot afford one. I don't have enough spare time to do a good job either.
Recently, my alma mater has an opening in the CS department for HPC-like people, which is definitely up my ally. I've got a real interest in ML and its connections to statistical mechanics, quantum computing, numerical simulations, etc. But I am undecided as to whether or not I should look into this more.
* Linear algebra: stuff like the QR decomposition for solving the "normal equation" of least squares regression, eigenvectors for PCA and the singular value decomposition for T-SVD, and so on. Linear algebra shows up everywhere in applied mathematics, statistical modeling, and data science. It's actually relatively simple and can be understood completely in about two semesters, but physicists do get a lot of practical experience and intuition. For example, eigenvalues turn out to be very important in quantum mechanics, so I suspect physics students spent a lot more time thinking about them compared to almost any other major.
* Vector calculus, matrix calculus, and optimization: physicists see this stuff in classical mechanics, E&M, etc. We can easily visualize vector fields and know a ton of relevant theorems and notation. We learn specific techniques such as the method of Lagrange multipliers for solving constrained systems. All of this helps because a huge chunk of statistical modeling and machine learning is formulated as an optimization problem, often a constrained optimization problem. For a statistician studying, say, SVMs for the first time, things like Lagrange multipliers and KKT conditions seem to come out of nowhere, while a physicist would have seen them in several other contexts first.
* Scientific programming: Most physics undergrads will have at least some experience with numerical optimization or simulation. This is a little different than application development or implementing an algorithm. The main trick is to be able to translate equations into performant, vectorized implementations. You also need to understand lots of practical things like rate of convergence, condition number, singular gradients, etc., just to be able to debug when things aren't working correctly.
* Experimentation: machine learning is basically empirical. Cross validation is essentially designing, conducting, and analyzing experiments. We improve our models not by proving theorems, but by forming and testing hypotheses. Basic stuff but extremely helpful in practice.
You have to remember that 10 years ago you couldn't really get a degree in machine learning or data science. (I believe degree programs and specializations with those names now exist and becoming fairly common, although I've had mixed results when interviewing graduates of such programs.) I would say the physics joins statistics and applied mathematics on the short list of degrees that, purely by chance, covered most of the relevant material. Of these, physics is probably the worst choice if you know you're going in that direction, because a physics degree only provides a thorough grounding in the prerequisites and little to none of the details. I had to learn much of that on my own through self-study.