Be in a field where tech is the limit
mathiaskirkbonde.substack.com
mathiaskirkbonde.substack.com
Worst pay, top heavy salaries.
When a phd makes 80k a year and a “ML/AI” data scientist is lucky to make 100k you won’t find any progress like software
They need to cut the top heavy executive bloat, respect the mid tier with better pay
It just sounds like completely different systems / way to run the numbers to me, not at all apples to apples.
At those income levels pretty much the only thing you’re really priced out of are nice single family homes, but I suspect that’s the same in Zurich.
I have a Senior Principal position and make products you've probably read about on Ars/Verge type sites recently, at an established tech company in Berlin - for about half of those 300k. And it's not a bad deal for the region.
You'll never pay it completely but if you can afford a mortgage you'll build equity into it and probably be able to resell it at a profit.
I earn a lot here in London, but moving to SF would be a significant bump. However, I very much value my time off.
There are also all sorts of special purpose leaves (jury duty, infant care, bereavement, etc...) that can become quite substantial.
All that being said I’ve almost never had a boss who cared about tracking my vacation too closely, and I’ve never felt pressured to not take time off when I want to, other than a half dozen critical weeks a year.
Officially, though, I (just found out that I) have 20 discretionary days per year (plus American holidays).
The "unlimited" thing became popular in California in part as an attempt on behalf of companies to avoid having to issue payouts for accrued time off to employees leaving their organizations.
I'd suggest simply being very up front and specific in interviewing and negotiations about how much time you plan to take and see what kind of reactions you get.
Generally the US pays better and at the top of any field you care to mention except perhaps finance it pays far, far better.
[1] https://en.wikipedia.org/wiki/List_of_countries_by_household...
I hear this a lot, but I've never seen it. I know plenty of people in tech making the better part of $1 MM a year at FAANG, and a few who even breach that. Most people I know in finance never break $500k.
So how much do people make in finance?
We will see salaries in the UK increase over Germany.
Lower salaries make it easier for businesses to compete.
https://www.bls.gov/oes/current/oes151256.htm#st
https://www.bls.gov/ooh/computer-and-information-technology/...
HR asked what sort of salary range we were looking at, we suggested that we won't get any decent candidates for less than 70k EUR and were laughed out of the room and they decided on a 50k limit. I've since left, but I'm pretty sure they've still not manage to hire a software engineer.
Porting it to java will speed it up a few x,
and if it uses some silly library like pandas or numpy or spark then consider it a great time to rewrite it from scratch properly with no dependencies :)
DIY doesn’t sound like a saving here.
Also, if you wanted to rewrite some scientific code relatively painlessly, perhaps D would be a vastly better choice than Java.
Just relocate your lab to be in the US and associated with a US University. Then you can hire a Masters/PhD student from Europe's top schools at that salary.
On the negaive side, yeah, all the truly high talented people will leave because the salaries in Europe can be so laughably low.
Trying to figure out how to move back to my home country now, but short of director level positions, there’s just nothing with a comparable salary range.
I live in Southern France, which is a nice place (with good food). I have enough money to do almost whatever I want and have savings. I have 7 to 8 weeks/year of paid leave, which I can use pretty much when I want to. Health/unemployment insurance is included. Granted, I have no kids (by choice), but what more could I want? More money for what? Luxuries I won't even enjoy because I would be overworked?
I like the US and its people, and I understand that my way of life is not everyone's, but that's to give you some perspective on why some very talented European may want to stay in Europe despite the "low" pay.
A friend of mine, a mother of one, was almost begged by Google to work for them, with I suppose, the kind of pay a PhD in natural language processing can expect. She refused, instead preferring a much lower pay but with an incredible work/life balance.
But... he is self employed, which means a significant part of his work includes sales and management. That's something I don't enjoy doing and I am very happy having a boss do it for me. He also has more tax and less employee protection and other advantages, which he compensates by having insurance, but that extra expense is to be considered when comparing income (it is still more than me in the end).
As I said, I am content with my life, even though I know I could make is "better" if I really wanted to. It is just an insight on a reason why many engineers prefer to stay despite the "low" pay.
Also worth noting that many French people dislike the very idea of working in the US. Sometimes because they don't like the culture or the sometimes misguided idea they have of it. Sometimes it is the language, or just moving and leaving their relatives behind. Canada, especially Quebec can be a "milder" alternative, I have several friends who went there, some stayed, others came back, I think the pay was higher there every time.
No excessive self-promoting, networking, or book keeping necessary.
I am sure you are making the best trade off and respect that, but use the real values. Double is for a fresh grad in top US markets.
The Greater NYC area will net an entry level dev 60-75k max in most cases. 108k is much closer to the average, taking into account highly experienced outliers.
National average of entry level dev salaries for reference: https://www.ziprecruiter.com/Salaries/Entry-Level-Software-D...
If you’re not then I would suggest learning a bit more about negotiating. Ask for what you want. The worst they can do is say no... but you’re very rarely going to lose an offer for asking for too much. They already spent months finding you. If it’s in budget or the pay band you’ll get it.
I'm sure there are also office-based US companies with good pay and W/L balance.
They are increasingly difficult to find.
I think someone on HN commented on life in Denmark from a US perspective; that people are only happy because they settle for less. My personal experience is the exact opposit, better life can only come at the expense of work productivity.
Let’s just start by saying that back home, every worker has 5 week of pays vacation by law.
Most qualify worker get 6 , and it’s not uncommon to go in the 7 or 8 with seniority.
Just with that simple fact the « work life balance » a US company has to offer is kinda cute compared to what I would get automatically.
Then come childcare, education and healthcare. Being childless in the US is fine, but when you see the cost of a child you understand the large salaries.
Something I still don’t get after 10 years is how poor people afford kids in this country.
The laws and baselines could be better for sure. But in the context of Software Engineering for a talented person it's possible to work for non-FAANG companies that pay quite well and offer things like "unlimited" vacation. Unlimited enough that up to 8 weeks a year spread out a bit is not going to cause any issues.
> how poor people afford kids in this country
They live a different life-style. Much more multi-generational and community support for child raising.
That varies by a huge amount depending on company and team and manager. I have unlimited PTO and if I took 8 weeks in a year, I’d be an outlier. I’d also have to be very very careful about optics.
Like I said in another comment the money is stupid high in this industry and country.
I take it of course. And I had unlimited PTO 3 times in 3 different places at this point.
It depend greatly of the implementation. But overall I feel that the only real time off I get is between jobs. Where I usually take one full month. ( if the next job refuse; I take it as a red flag )
It’s not the same, unlimited PTO have to be placed carefully and you have to be mindful about optics.
PTO that you earned and are enumerated on your pay stub are more enforceable.
I would rather have 4 week of normal PTO that a unlimited vacation policies that I’m not sure will be well inplemented.
Large companies tends to do it correctly. In that context I was usually taking 5 weeks. But with some stress and guilt and kinda check my work inbox.
Not the same. But then …you guys have those massive paychecks… I will take them while I can.
Salary are insane here in the US. Like: not following sane guidelines.
I’m currently at times 5 what I would earn back home. I don’t think it’s gonna stop. Every job is a solid 20% increase.
As soon as I can, I’m still gonna go back to civilisation. Auvergne, here I come.
They are just pushing me out of the workforce earlier by giving me more money. Also: what is your fucking problem with vacations?
I would rather move to other European country to ever bother with US, exactly for the same reasons you mention.
It also sort of implicitly includes the assumption that you still won’t work more than your 7.5-8 hours a day. Working 50% more for 50% more salary obviously is no gain at all.
That said, while I know that the word ‘stress’ didn’t really apply to my jobs in Europe, I find it hard to turn back the clock on my expectations (for both pay and work/life balance).
If I don’t go back I’m hoping I can build a little enclave of sanity for my team here at least.
I am French and live in the western suburbs of Paris. I have a job that does not require me to get into Paris and despite having had several opportunities, I passed.
I like the quiet environment, biking to the office, even if it means a lesser pay.
In most of Europe you can find a good young engineer/developer with several years of experience for €50k.
Different story in Switzerland where you can easy get 140k+ CHF.
Besides the top heavy compensation problem, the other problem I had with biotech is that everything is swamped in patents, NDAs, secret patents, secret NDAs and in the US there's also a lot of spooky secret government bullshit. I worked on information security so I'm not that scared of three letter agencies, but in my short time dealing with biotech they were actually not letting me do my job, which had never happened before.
Shameless plug: I'm CTO at Streamline Genomics, a Canadian biotech startup, and tech is our limit. We're remote-first and we're hiring for a bunch of positions: https://www.streamlinegenomics.com/careers
Our domain is precision oncology. In brief, this is about matching cancer patients with available targeted therapies by examining the genome of their cancer. This is different from how the majority of cancer patients are treated today: surgery and {chemo,radio}therapy.
Here is a problem in our domain where tech is one of the limiting factors.
If you look at the DNA of any cell and compare it against the reference genome, you'll find a lot of differences, aka variants. Typically even more so if you're looking at a sequenced tumour (~1e6 variants). This is your hay stack. And a variant that can be medically targeted to treat the cancer is the needle. The definition of what variant is "clinically relevant" is layered, context-dependent, and (partially) regulated. Software is responsible for automating away the majority of variants, say down to 10-100, in a justifiable, traceable way. It's also responsible for giving tools to the clinician to deal with the remaining ones. This manual step typically involves an informed line of questioning about each variant backed by 100s of supporting data points about it.
Without these two tech pieces, interpreting a single molecular pathology report can and does take many hours of (expensive) expert time, instead of minutes. For a rough sense of scale: human genome has ~3e9 nucleotides (ACTG), has ~3e4 known genes, ~1e6 known gene interactions, and ~1e8 known variants. Typical whole genome sequencing produces > 30GB of raw data (compressed).
This is probably the first problem everyone runs into. There are plenty of other ones, some more challenging and interesting than others. Feel free to send me an e-mail if you'd like to discuss this more! amir[at]streamlinegenomics.com
Facebook, as an extreme example of profitability, can pay its directors and EMs $1m+ per year in total comp, but it's not top-heavy - it pays its engineers ~$200k out of undergrad and >$350k after a few years.
By contrast, I know of many cool hyped-up hardware unicorns and biotech companies, and none can compare to FAANG in pay because software scales in a way that other businesses can't. (One hardware unicorn pays its new grad SWEs $60-70k).
Unless the biotech company is actually a biotech-focused SaaS company, it's inherently going to have a higher unit cost that prevents software-level comp. It's a disappointing effect of the current system we live in.
A parallel way is to start a company that only does one part, say a software based part. Pharma will pay more for tools that solve their problems than they would pay for a couple of FTEs to solve their problem directly. The problem is your customer may not know how to put your product to use, or even understand its value. So sales is no slam dunk.
I loved my time in the pharma business but was happy to go back to what’s called “tech.” Culturally the life sciences CES are full of some dreadful pathological practices, and I prefer the pathological practices I’m familiar with :-(.
Speaking as someone who has spent the last six years of their career working on advanced physics in various technology sectors (including biotech) and then trying to make various 2D-xene materials work for semiconductors, I’ll tell you one thing:
They pay you shit and if you think you’re all treated badly in FAANG, hoooboy, at least nobody has nearly caused deaths in the lab through negligence!
No one thinks that. Workload at some places maybe a little on the higher side but still on average monetary and toll on life wise FAANG is probably one of the best jobs.
Citation needed. I'm pretty sure innovation is happening in all fields.
> Software is no longer such a field, our brightest minds should be going elsewhere.
Citation needed. Pretty sure innovation is happening in software too.
This post is... utterly fact-free, and really just reality-free. It literally says nothing besides "biotech excites me". The author had no need to add a bunch of false generalizations on top of that.
I think a fact-based article exploring the question of whether software really does have a slowing (or just slow) pace of ideas and innovation would be much more interesting than this article, but not every article has to be factual reporting and synthesis, some can be of the form "I believe this is how the world looks, and I draw these conclusions starting from that belief".
Personally, I relate to his premise, though I'm unsure it's true. It does seem to me that there aren't many new ideas in the industry. We seem to "just" be doing things we've been able to do for a very long time, but making them more accessible to more people, more efficient (with respect to human time while not necessarily computer time), and more scalable. We seem to me to be doing a lot of streamlining but not a huge amount of innovating.
I have a lot of non-programmer friends who sometimes say that programming must be very dry and boring. My shiny go-to example to convince them otherwise is a nice desktop planetarium app, which you can't develop without first learning how the solar system works. Once you do you can write that down in code - an executable, computing form of knowledge, a living document that allows you to tinker, refine, share, reproduce. Software truly is pretty neat as a human societal tool with a wide range of applications.
It should be for everyone. The other thing I tell them: If you've ever been in bed in the morning and planned out your steps for how to get that cup of coffee you need, designing an efficient bed-to-coffee algorithm, you've already been a a programmer.
Also tangential: I find Balaji completely insufferable. His Twitter feed almost looks like that of a cult leader than of a CTO.
Crazy that they haven't fixed it yet, but that's Twitter I suppose.
As a relative dummy, I guess I'll remain in software.
It takes a long time to develop biotech, test it, approve it, market it, and make money. There is also a limited market (ie the people sick with that condition, specifically in rich countries). The reason tech companies make money, grow/iterate, pay more, is because they are in a field that does not require the same oversight and moral safety obligations (maybe they should to an extent) as well as being marketable to basically everyone in rich countries.
Ever written Javascript? Good luck doing real maths with only floats.
Ever used Electron? Hope you didn't need that 8 GB of RAM.
I could go on, but most mainstream software is one step above absolute garbage. There are isolated islands of extremely high quality tools, but they tend to be esoteric FLOSS packages that are isolated from market pressures. OpenBSD is a work of art.
It seems to be equal parts users stuck in a local maxima of computing skill and how that enables lax software engineering standards.
When’s the last time you’ve sat down with a user who isn’t remotely interested in tech and watch them work/use a computer? Most of the population’s mental model of a computer is starkly different to the average hacker news reader. You can still hear the same complaints about how computers “don’t do what i want it to do” that i remember my parents generation saying, and they were experiencing the first waves of computerisation in their offices.
The story became that the older generation just couldn’t understand the new generation, but kids are amazing with computers because they’re growing up with them. Well, some of those kids are just as hopeless. It’s partly an education problem (hard to learn computing from a teacher who doesn’t understand it themselves), and partly because UI design trended to simplifying everything as much as possible so that users who don’t understand computing can still enjoy and use their devices. Now there’s not a great incentive to learn more than you need to just use the UI you’re given, and computing skill tends to get stuck in this local maxima.
I won’t go on about my other point in detail as it’s a perennial favourite for hacker news discussion. But hardware gets faster so quickly, but our software is so hastily thrown together that it eats up all the gains. Users don’t notice that software they’re using is crap because their mental model of computing isn’t developed enough to know what’s happening. Instead we get this casting of devices as somewhat malevolent entities (“ugh, my stupid computer keeps losing my stuff. I need to buy a new one that isn’t so dumb”)
We use to think this would be resolved with time and generational change, but it seems like there’s just a more-or-less static percentage of the population that just doesn’t get computers. (Which is completely understandable, people have different interests, it’s hard to inculcate an appreciation of something in your entire population, look at peoples relationships with mathematics)
The biotech industry (which is made up of at least three rather different verticals: tools, diagnostics, and therapeutics) is changing quite a bit today. There are certainly companies that have less of a technology or data emphasis or who are still trying to figure out the value those could bring, and those companies are far less likely to pay well in SW/DS roles. There are others that either from their inception or more recently realize the value these approaches can deliver and compensate accordingly. I personally find the new wave of biotech startups that are focused on being hybrids of experimental and computational capabilities extremely exciting (which is why I'm at one) and these are the firms where software and mathematical skill sets are most likely to be valued.
You'll probably still make more on Wall Street than you would in biotech. But you don't have to be _badly_ paid in order to work on a meaningful mission. OP is, IMO, correct that biology is entering a phase in which computational skills are a rate-limiting factor in our ability to make advances (note: not _the_ limit -- experiment is still absolutely critical), and it's a super exciting and impactful field to be in.
Shameless plug: Recursion is hiring a TON of positions in data science and machine learning, engineering, and elsewhere. Check us out: https://www.recursion.com/careers. (Contact info is in my bio.)
We've been doing a lot of remote work during covid and I have definitely noticed the decline of water-cooler solidarity. We still do hire remote engineers and data scientists, but only in cases where they are the literal top of their field.
[0] Standard disclaimers: I work at Recursion, have drunk the Kool-Aid, etc etc
I work in "Wall Street".
My firm sells things to those who want to buy them in a highly regulated marketplace with considerable governmental and self-regulatory oversight.
We do this on behalf of investors who entrust us to use their capital as fiduciaries for their, and their clients, best interests.
We solve challenging problems with cutting edge approaches involving non-trivial technical, statistical, and business considerations.
And we are not paid _poorly_ for our efforts.
Please stop pushing a strawman financial industry narrative that we've no meaningful mission to serve your industry's recruiting needs. I don't poo on biotech to source my hires.
(Of course none of this constitutes financial, or personal, advice.)
I understand the point of places like Fidelity or Charles Schwab, that offer the general public effective, low-cost investment vehicles. It's hard to deny that's a meaningful, socially beneficial thing to do. But when people punch at "Wall Street," they usually have in mind places like G-Research, whose only real purpose seems to be making an already rich owner richer.
Suppose that such a firm accepts money from a state (or other) pension plan. Then part of the mission of that firm is to provide a stable retirement for every participant of that particular pension.
Know anyone on a state (or other) pension?
Again, I take no issue with the firms that responsibly manage people's pensions, help the public access equity and bond markets, and so on. But those are not the firms that the people making demeaning comments about "Wall Street" are talking about. So I don't think that comment is responsive to the question I asked. Again, that question is, what is the "meaningful mission" of those other firms?
Many people disparage used car dealers. The used car salesperson is a trope punching bag. They offer bad prices versus what someone can get by using Craigslist or AutoTrader or something and putting in modest effort.
But those businesses serve a purpose because they allow folks to buy or sell a vehicle real darn fast relative to Craigslist or AutoTrader or something. Like, hours instead of days.
Market makers are used car dealers. They collect a spread every time they buy then sell the same car. Or when their car inventory goes up in value. They don't have perfect information, like a used car dealer, when someone is selling them a lemon so they build that risk into their quoted spreads.
The market maker's mission is be the most profitable used car dealer they can be. Unsavory? Perhaps. But they are kept in check by competition from every other used car dealer in town. Consequently, you get a reasonable price for reasonable amounts of liquidity. Just like you do when buying a first, used car one afternoon for some lucky teenager.
The other types of market participants you mentioned also accomplish useful things. I won't go into other examples nor will I field questions about any specific firm.
And, sure, everyone wants to be as profitable as they can be. That's business. Providing valuable service to customers for the highest possible profit in the face of competition isn't unique to the financial industry.
First, it's not so clear this is meaningful work. If I'm a smart college graduate and go to work in biotech, I get to save lives and cure disease (ideally). If I go to work at a market making firm, I get to ... lower the spread? I think it's quite reasonable for someone to forgo doing this because they feel it doesn't make a meaningful contribution to the world.
Second, I've watched many incredibly talented people disproportionately go work at HFT firms, and it's hard to to avoid the impression this is socially wasteful. Efficient markets are nice, but do the markets really need to be efficient on a microsecond scale? It seems like the talent sink here is unfortunate and society might be better off if some of them worked instead on, say, biotech or alternative energy sources.
I agree the the cause is obvious: finance pays more money. But it's hard to avoid thinking about solutions that reduce the profitability of HFT and don't harm market efficiency significantly, for example eliminating the subpenny rule [0]. It seems clear to me that if the current incentives result in one industry essentially monopolizing the nation's most quantitatively able workers, then those incentives might need to change.
I don't think this point was made explicit in the original post you responded to, but it lurks in the background.
Finally, I really wish you would treat other examples of market participants. Because right now you've done the easy cases, the ones that I, a total outsider, can mount a defense for. Again, it's the other ones that are interesting. And it's clear there are a ton of participants who are a) not aimed at increasing access for retail investors and b) not market makers.
[0] https://www.chrisstucchio.com/blog/2012/hft_whats_broken.htm...
Not sold? There are 168 hours in a week! Someone's raison d'etre need not fall into the 40 hours of the work week. For example, my job permits my spouse to pursue what she absolutely loves.
"Socially wasteful" is in the eye of the beholder. We probably have different values or philosophies. Let's just disagree.
You seem fixated on HFT. And you want more examples. Say there's some low-latency statistical arbitrage shop full of biotech luminaries that were sucked into finance only for the pay. And say they trade only to line their own pockets. And they hate all puppies and every single kitten. Real morally objectionable dirt bags. They're still (still!) trading with willing counterparties who wanted to trade that specific instrument at that specific price or per those specific terms. Every single trade requires two participants.
Hit me up by email if you want to talk more.
My point was that _if_ one's primary goal is to get the highest comp possible, it's probably better to go into finance than biotech. No bones about that; we don't pay as well. But that does not mean that biotech universally pays _badly_ for technical skills, as casual readers might assume based on reading this thread. My point was that if someone is both technically-minded and interested in the mission in biotech, it's worth taking a closer look, especially at the wave of companies from the last ten years bringing tech and biotech together.
* Not everything, I'm sure, but neither is everything in biotech!
If you're exploring the world of AI for drug discovery, IMO, it's important not to fall for the trap of companies that just do "AI on public available data". That's just the same "the fundamental problem of software is a lack of ideas" phenomenon the article was talking about. You want to meld software and data science skills with the fundamental work of whatever industry you want to be the vanguard of.
Seriously, if you're in the market, check out Recursion. My contact is in my bio also, and I'll answer any questions Imran can't (or won't :p).
Which I think is fair. Innovation does also occur where it doesn't necessarily need to.
the brightest don't spend their time & energy making others wealthy
Fields where you don't make other people wealthy aren't so rosy either. They bring their own drama to the table.
The benifits I see:
1. No commute. (Getting cars off the road is good for global warming?) And my sanity when I do need to go somewhere.
2. More sleep.
3. Less busy work to keep some middle manager happy.
4. Happier employee.
5. I have no clue over productivity.
negatives:
1. I guess schmoozing is important for some people. I did make most of my younger self's friends at work because we were both at the same lousy job?
2. Some people find office banter comradery important for their mental health? I used to be one of those people, but would happily give up a little socialization for more sleep, and less hrs driving.
If the government spent some tax money on a campaign to keep workers home, if the job was doable at home, I would be behind it.
I wouldn't even mind if they gave credits to employers who didn't drag their employees into a office.
Frame the promotion over Global Warming, and not employee satisfaction. We all know employees are way down on the list of what they care about, but a tax credit, and some kind of carbon rebate whatnot might keep many of us home? And less cars on the road. In my county, it seemed like everyone went out and bought a second car. Traffic is back to unbearable in the Bay Area.
If you think they would react negatively, then wouldn’t you concede that there is an element of dishonesty here?
I'm not GP, and I'm self-employed, but I think it's fair to compensate people based on the value they create, and not how much time they spend at the office. If a guy gets his work done in half the time, and his employer is happy with the cost, then everyone gets what they want.
They had written proof that their employer was happy with their work.I certainly don't run my business by paying all my staff the absolute bare minimum I need to so they can do work that is just-as-good-but-no-better than needed.
For a company reliant on innovation and driven by human capital, the dynamic you are describing seems like something that would normally come into play at a late stage of its life. Ignoring other factors, imagine how easy it would be to compete against a company that is entirely run this way and staffed by people who work like this. Literally no one there would even care that they're facing disruption.
The brightest absolutely do spend their time and energy making others wealthy (ex: I would consider most senior eng at FAANG to be bright and although they are definitely rich, they are not wealthy). I suspect that's because of the cycle of responsibilities and spending most of their incomes.
I've seen millions of dollars-worth of software development waste as FAANG and BigCo. Nobody bats at eye. Because it's a bizarro world with no consequences. All you gotta do is not get wrapped up in it and collect checks ;)
I think people on both sides forget this. You don’t need 300 engineers to solve very many problems, but if you use 300 it means 2 other companies don’t solve it too.
And just because you’re making a ton of money working for a big company, doesn’t mean that what you’re doing is that important. It might just be to humor you. Even at small places, sometimes the established players get thrown a bone, working on something frivolous because it makes them happy, so they stick around.
As such it isn't mutually exclusive to be a software engineer while working in a field where tech is the limit. (But even so in my opinion, software itself is still just getting its bearings)
"Ideas" were and still are largely worthless. They are absolutely not the bottleneck in software today. There are a billion implementation-level problems that are still unsolved, and there will always be new ones.
Like, I've heard senior leadership at a 'computation-focused' biotech company outright call engineers 'bad people' only to quickly correct this to 'bad at being people', which is so much better!
We have capability based security as a model for having computers that don't get taken out by any flaw anywhere, but... like doctors who refused to believe that washing hands helped save lives, most programmers don't believe in it, or have never heard of it.
Computers used to be leading edge because anyone could just get a machine and start hacking away at it, with physical hardware being the only limit. Now our operating environments are about as secure as a forest during high fire season... only little spark, and poof... your house is gone.
We're about 10 years out, not because of a lack of ideas, but because of a lack of adoption of technology that works, instead of the old stuff extended way too far on a bad local maxima.
Innovation as a goal sounds noble initially, but in my experience it's like chasing the wind. Faithfully doing what is already known to be good seems better for everyone. It might even be the quicker road to innovation.
As to your second point, we did have people with exceptional sense of social responsibility in software, but multiple factors, including easy/fast money, unspoken agendas, both co-opted and corrupted some, and drew in 'fresh blood' that was motivated purely by the new social cachet & easy fortunes of software work. (These are the ones Alan Kay would say are the 'pop artists' of 'pop software'.)
As to innovation and "limits", there are legion and everything from the hardware, OS, libraries, to languages are on the table for innovating, to say nothing of theoretical breakthroughs in pure comp-sci.
Software is the closest thing to magic we have in the modern world. The imaginal sky is the limit.
One option is research software engineering, where SWEs team up with researchers to produce better code for models and simulations. Are there any research fields where synthesis of domain knowledge, programming skills, and computational thinking could bring great benefits?
I think the speed at which they were able to develop mrna vaccines just shows how far along we've come, but also how much more we have to go. Things like protein folding at deepmind definitely requires all these things you mention.
I imagine that technology in any kind of manufacturing or mining field is going to be similar or predominately dominated by one or two big players that haven’t faced an innovative competitor in decades.
Ditto for Wall Street. The “innovation” tends to sit on top of woefully outdated systems rather than replace them.
Most of the time when someone is saying something that amounts to "I can't imagine what else we could make", it's a failure of their imagination that's the problem.
Sure, it can be frustrating to be banging your head against the same wall as everybody else, but there are people that thrive in such a setting. The most extreme example might be pure mathematicians.
I'm not even sure what to think of it, honestly.
But I can think of all sorts of jobs that are more important to society than working on some random doomed-to-fail SaaS that won't pay remotely near that.
In the case of research positions, the funding situation has oversaturated the market in most entry level positions - turning negotiation and career advancement into a trial by fire.
One thing I've always wondered about biotech... I imagine there are many non-obvious correlations and interactions in medicine, which would be easily detected using nothing more advanced than Excel-spreadsheet level data analysis.
Making up an example: people with a certain DNA trait/allele who also have a diet with a high amount of XYZ tend to not develop disease ABC as frequently as most people. Even if we don't know the pharmacological reason why that is, it would still massively benefit lots of people, right?
So it always seems to me like tech from 2007 was ready to tackle this problem. Dump in a bunch of anonymized data, find correlations, repeat.
But I feel like I never hear anything about this type of work. Is it happening, but not publicized much? Is it actually not as simple as it sounds? Does nature simply not work in this way?
Even if 95% of diseases are just "bad luck", I assume that other 5% is made up of environmental factors we don't yet understand, but could easily learn using well-known data processing techniques?
So... anyway you're right that this is a natural way to approach the question of understanding the genetic basis of disease and physiology. But it's been beaten to death and found to drive fewer insights than were hoped
[1] https://en.wikipedia.org/wiki/Genome-wide_association_study
But finding a correlated gene is only the first step. One issue is that a single protein can participate in hundreds of seemingly unrelated chemical reactions throughout the body depending on the cell type and environment. So simply tweaking the genes expression will have many unintended consequences.
For instance, each cell is constantly maintaining a baffling complex balance between growing and dying. Any external perturbation has a good chance of either killing the cell or causing cancer.
The important part is “and there is promising tech on the horizon”
I think trying to get a startup based on space travel at relativistic fields would be pretty difficult.
Steve Jobs was a master of this. Seeing promising tech trends that were just about ready, and putting them together at just the right time to make innovations that were world changing.
I feel like the author has an overly one-dimensional definition of both innovation and what it means for someone to be one of "our brightest minds".
There are two different goods and two different talents at play here. The first is taking an idea that already has been had and making it possible. The second is inventing new ideas. Both are goods, but each requires very different talents.
Biotechnology desperately needs people who, given a great idea, can break technological barriers and enable it. If we accept that software is bottlenecked by ideas, then software desperately needs people who can radically change paradigms. "Our greatest minds" consist of both types of people.
Because better technology won't get the entrepreneur a satisfying reward.
Your enjoyment of it hinges on whether you can be happy collecting a paycheck doing the bare minimum and satisfying your tech itch outside of work (FOSS, side hustles). For some, that is perfectly acceptable or even ideal, especially if you can get away with working fully remote.
For someone who is relatively better at ideating, I would argue the opposite is true.
There are some important differences that are often overlooked by those coming from the world of computing...
I thought this in the 1980s
The caveat ofc, is that I'm still young with no ties.
Chomsky was talking about this some 15 years ago.
1) If you are able to bring a major drug to market 3 months earlier, it's worth billions. Hence the continued interest in computational approaches.
2) Salaries in the pharma/biotech biz are set nationally. Yeah, there are variations by geography, but less than one would expect. Thus, a PhD with x years can look up the salary range per region, etc.
3) The data is confusing and the error range(s) are unknown. So, many/most of the models are retrospective rather than prospective and if the initial guess at the biological target or model fails, everything else is a waste of time. Google for all of the failures re: Alzheimers.
3b) As we can't test on humans (at least not ethically), we're totally dependent on animal models being good predictors of human behaviour. But, while chimps are like 98% similar to humans, the difference has resulted in catastrophic failures in Phase 1 testing. Diseases by the score have been cured in mice...
4) Computational modelling occurs at the start of the process, which is the most efficient. I think they had a sequence for the mRNA vaccine a few days after the Chinese published the data. Getting it made, stable and deliverable is where the time was consumed. And then the various clinical trials are significant costs in time and money. Hard to trust a model for a new class of disease or mechanism.
5) Computational methodology has been (over)sold since the 60's. Yeah, there have been successes but they've been way fewer than hoped and people have grown rather jaded when presented with the latest breakthrough. ML/AI isn't really new as it was studied in the 90's, but there's way more data. See (3) above.
6) The crystal doesn't always form. The reaction yields brown oil rather than white powder, or doesn't scale. Chemistry is messy. And there's a lot of material design problems that have not been amenable to modelling. There are new ways of gathering information (CryoEM), but we still need more/better.
7) We need newer software and better parameterization. Both of these trace back to academic work on Vaxen, maybe SGI's. Visualization software is probably the most valuable tool right now, with broad acceptance in the research stage.
7b) Physics might bite us in the ass. MD software, for example, tries to model explicit protein, ligand and solvent atoms/molecules. Even given revised software and parameterization, entropy or chaos might prevent accurate numbers or what we can calculate might not be pertinent.
I could go on (and on), but I wanted to leave you with an upside... If anybody DOES deliver the goods, they'll be bloody heroes. Fame, fortune, the whole gig - like CRISPR and the other advances that have occurred. So, if you and your buddies are smart and dedicated, it'll beat the snot out of selling ads on handhelds in terms of making a difference.