Hacker News Reader

Comment by dumpster_fire | Hacker News Reader

I majored in semiconductors out of genuine interest. It was fascinating to me how miniature components can be turned into digital and analog devices. It turned out to be one of the most boring things I've ever touched. Cadence was an endless loop of crashing and losing progress. Anything you designed was called worthless because there's already a library of capacitors, amplifiers and DAC ADC components for you to drag and drop in. It was so incredibly dry.

During our final semester in school, or professor walked in and said, "All of you but X needs to know that you're going to be unemployed." He said that to a class of 4 people, the only ones left after most of the original class of 20+ had quit.

"The industry only hires the exceptional, you are all inept. Except for X, my favorite student."

X ended up being the only person to go on to work in a semicon company. The rest of us ended up in software, earning 5x the pay X was making. The barrier of entry to software engineering is so so much lower than semicon, which still makes no sense to me.

View on HN

Here's what I learned working at Samsung:

At the end of the day, the stakes are way too high to allow everyone a chance to play cowboy with the tools. You want your best possible champions to bring a design to bear. Then, the other 99% of the army is responsible for implementing the design in as repeatable & standardized a fashion as physically possible so that there is even the remotest chance of yield+profit. Software is the antithesis of this. The cost of playing around with your tools is not even worth accounting for. Due to certain cultural effects, you probably don't want to do software in semiconductor industry unless you really, really like the problem domain.

Even if you don't get to play around with the billion dollar tools, you still get to help troubleshoot some of the most intensely complicated problems on earth. Solving these riddles is very rewarding and the experience will stick with you forever. Hard to package those 2 sentences up into a PR campaign for the young generation, but I'm sure we can spin it if the DoD can still find ways to recruit.

>Solving these riddles is very rewarding and the experience will stick with you forever.

That was also my experience working in the semi field (in Europe). People working there weren't in it for obscene compensation, they were into it for the hands on puzzle solving of uniquely complex HW problems with cool and rare expensive machinery. Some were quite significantly underpaid and while they knew it they never felt the need to complain too badly about it. I guess it's kind of like arts, which is a shame, because here some publicly traded corporation is abusing you for your passion.

Also, the fact that most of Europe doesn't have many FAANGs and big SW companies paying orders of magnitude more to un-balance the jobs market, surely help to not discourage people from this industry.

I'm a hobbyist in this area, and think the field is somewhat early in its development. MCUs and cyber-physical systems was like this until the Arduino happened. Arduino may not have been the first to do exactly that but it was just good enough to cause an (re?)explosion in electronics as a hobby. During its hayday, I would say Arduino was a core element of the Maker movement.

So what needs to happen to make this a reality for semi-con? First off, we need cheap, cheap fabrication. I actually looked at public funding in Canada and how that was going to the big name Universities who had their own in-house fab labs (at older process nodes). The costs of someone not in the inside was nuts. The actual cost should be in the 100s of dollars to fabricate a design (considering the marginal costs).

There are people that do this at home but it doesn't work either due to chemicals being pretty dangerous and the need for a bunch of equipment. I bet the amount of money the EU spent on its first metaverse townhall (or whatever it was called .. the thing very few people attended) or a tiny fraction Canada wastes on silly things promoting youth culture or whatever, they could fund a lab that is actually open to the public, with the express mission of promoting hobbyists and education. This will NEVER happen because (a) it needs a professor who is on the inside with a kid-like passion in this tech and a commitment to bringing it to the masses (I see some profs like this at schools like MIT but it is so rare at large, competitive schools like the big ones in Canada), and (b) it does not have an instant payoff for the govt. They don't want dabblers and vague educational outcomes. They want workers with degrees.

I am convinced before I am dead, advances in robotics and fabrication will simply the process (or use home equipment such as future laser printers for printing stencils). I'd love to spend my retirement fabricating my own CPUs :D

Edit:

Let me add: I don't mean the cutting edge process node. I mean the kind of process node that was used to make the very first chips (but less toxic, repeatable, cheaper equipment). If it is possible for synthetic biology, it must be doable for semicon :D

https://developers.google.com/silicon might be of interest to you.

I don't believe Google is sponsoring any more free shuttles. Your best bet nowadays is ponying up $10k to get a tiny chip on sky130 (so small you really can't hold even mildly complex in-order RISC-V cores). Sky130 is also generally so old that you can probably get better performance (let alone area) on a modern FPGA for a fraction of the price. Efabless's sky130 MPW is nice for semiconductor research insofar as it makes it more realistic to actually fab designs but it's not particularly useful for hobbyists beyond just the novelty of holding a chip you designed in your hands.

I'm a 25 year professional in the industry so I've never really thought of the hobbyist side. I use software from commercial Cadence and Synopsys that has a list price of over $1 million for a single physical design tool license and we use about 200 of those licenses simultaneously to tape out a chip. Then we spend about $30 million in mask costs. If we make a mistake it is another $20-30 million for new masks and another 4 months in the fab for a new chip.

Google / SkyWater / eFabless have this program for 90/130nm chips. That is really old technology from the 2002-2006 time range but it is still useful for a lot of types of chips.

https://opensource.googleblog.com/2022/07/SkyWater-and-Googl...

https://www.skywatertechnology.com/technology-and-design-ena...

https://efabless.com/open_shuttle_program

I am curious what kind of hobbyist chips you want to make that can't be done in an FPGA? You can't do custom analog in an FPGA but these days you can find FPGAs with multiple PCIE / USB / HDMI serdes links, DAC, ADC, etc.

The problem with FPGA's isn't the hardware... it's quite complete and capable. It's the software stack you're forced to access most of them via that makes them a non-starter for many/most hobbyists. (cost is a major issue, but hardly the only one) My guess is that as low cost producers who have embraced the existing open source FPGA software being developed start producing higher end parts (more recent process nodes, large LUT count etc), that's when you'll really start to see them take off in the hobbyist world.

I think the OP isn't even talking about 2006-level capability but rather something akin to 3D printing for semis that approached even early (i.e. somewhere in the 1960's-1980's range) level fab capability for hobbyists. Obviously it doesn't exist currently, but that's the dream for some.

I get paid to do this professionally so I don't follow the hobbyist side but I've heard of people using this for FPGAs. I have no idea how it compares to commercial tools.

https://github.com/YosysHQ/oss-cad-suite-build

OSS CAD Suite is a binary software distribution for a number of open source software used in digital logic design. You will find tools for RTL synthesis, formal hardware verification, place & route, FPGA programming, and testing with support for HDLs like Verilog, Migen and Amaranth.

Cost of the FPGA design tools isn't really the biggest barrier. The big barrier is that the FPGA design tools from the FPGA companies are full of bugs that you spend an inordinate amount of time trying to work around. Some open source tools like Yosys have emerged. So far they're limited to the ICE-40 FPGAs from Lattice and a few older Xilinx parts. I've used them with the ICE-40 and they do work well. Hopefully FPGA companies will come to realize the value of opening up so that 3rd party tools can be developed for them.

> The big barrier is that the FPGA design tools from the FPGA companies are full of bugs that you spend an inordinate amount of time trying to work around.

Do they not have older, more stable, releases?

No.

you cannot fathom how bad eg Xilinx's FPGA tool suite is. like beyond the fact it's like some kind of frankenstein monster because they bought up a bunch of smaller tools and glued them together with fucking tcl, any individual piece is worse than your worst open source compiler/ide/whatever tool you use to build software. and yet people say that it's 10x better than the competitors' offerings. hell on earth is being an FPGA dev without a direct line to Xilinx employee for help debugging (which is how things really work professionally).

just lend some credence to myself here - vitis, their supposed hls offering that turns C++ into RL by magic and genius and oracles, today embeds LLVM 7.0 (released in 19 Sep 2018). gee i wonder how many perf and etc improvements have landed in LLVM since 7 since today LLVM is approach 17.

i could tell you more but it would just spike my blood pressure after 6 months of not having to deal with that mess.

I'll second this, but for Intel / Altera.

I started working with a Stratix 10 GX dev kit last year, and the Intel Quartus Prime Pro software. It's been a nightmare.

1. Unless you have a direct line to an experienced support engineer inside Intel, technical support amounts to pleading with new-hire support engineers who can't do more than try to look something up and regurgitate it back to you.

2. Sample designs don't work with the tools you have. A design might be for version 18 of Quartus and you have version 23. You can try to auto-upgrade the IP, and maybe 3/4 of the time it works. The other 1/4 of the time it's some obscure error you can't track down, so you're back on the "Community Site" begging for help.

3. Doing something like programming your design into flash on the S10 board involves lots and lots of research, including watching Intel produced YouTube videos.

I could go on, but the whole process is like running in wet cement.

> I could go on, but the whole process is like running in wet cement.

just quit. seriously it's not worth it. no job you have (not even low-latency work in HFT) will compensate you enough to make investing time, blood, sweat, hair, and sleep into this work worth it. there is a better, more rewarding, with an actually transferable set of skills, software job waiting for you somewhere out there.

Can confirm. I used Xilinx Vivado to compile for their FPGAs in Amazon F1 servers a few years ago. There came a point when each time I edited a single small Verilog file in my small project, the GUI took 40 minutes of 100% CPU just to update the file list in the GUI. That's before you could do anything useful like tell it to compile or simulate. The GUI wasn't useful until it finished this silly update.

I knew FPGA compilation could be slow, but this wasn't compilation, this was a ridiculously basic part of the GUI. I knew it had to read my file and analyse the few module dependencies, but seriously, 40 minutes? At that time I just wanted to run simple simulations of small circuits, which shouldn't be slow.

In (open source) Verilator, the same simulations ran from the same source files in just a few millliseconds with nothing precomputed or cached.

I looked into what Xilinx Vivado was really doing and found a log indicating that it was re-running a Verilog read on the changed file several thousand times, each time taking a second or so.

That was such a ridiculous bug for software Xilinx said they had spent over $500M developing. If there were good parts of the software I didn't get to see them due to this sort of nonsense. I think it was fixed in a later version, but due to (yay) enforced licensing constraints I couldn't use a fixed version targeting these FPGAs. That's when I abandoned Vivado and Xilinx (and Amazon) for the project as I didn't have that much spare time to waste on every edit.

I am under the impression the current crop of open source FPGA and especially open source ASIC tools are much better designed.

> I looked into what Xilinx Vivado was really doing and found a log indicating that it was re-running a Verilog read on the changed file several thousand times, each time taking a second or so.

Just reading that made me nauseous, to think this forced so many trillions of CPU cycles to be spent on nothing, and nobody at Xilinx corrected it until some time later.

> i could tell you more but it would just spike my blood pressure after 6 months of not having to deal with that mess.

There was a time when I could've transitioned to FPGA development and I was pretty keen to do so. But after using (fighting with) Xilinx's tools and watching other people who were deeper into the FPGA side wage similar wars I decided it just wasn't worth it. That it would be better just to stay on the software side of things where the tools are mostly open source and so much better. In FPGA development you just don't have any control over the backend tools - fortunately there are a lot of front-end simulation tools like Verilator that are open source and some limited backend tools like Yosys are open source. Hopefully open source will continue to make inroads, but it's slow going.

> I use software from commercial Cadence and Synopsys that has a list price of over $1 million for a single physical design tool license and we use about 200 of those licenses simultaneously to tape out a chip. Then we spend about $30 million in mask costs. If we make a mistake it is another $20-30 million for new masks and another 4 months in the fab for a new chip.

I've thought for a long time now that this is an area ripe for disruption. But it's very difficult to disrupt - it hasn't been yet. EDA software is probably the easier part to disrupt. Some open source EDA tools are out there, but not so much on the physical design side.

I've read there's actually some excellent ASIC open source design flows now, thanks to some open PDKs (notably Skywater) as well as recent funded research. I haven't tried the toolchains myself but they are readily available and you have courses like Zero to ASIC using them.

They don't target the advanced nodes where masks cost as much as the GPs prices (though $20-30M sounds higher than I expected even at the leading edge), but they are workibg their way forward and the space in general is being disrupted at last.

I've taped out 4 chips in 5nm and our managers and VP's have said about $30 million for a full set of masks (base layers plus metal layers)

The $20 million is for metal layers only for a respin.

We are moving to 3nm right now which will be even more expensive.

The open source tools can't handle the latest process nodes. Maybe in the future but this is a highly specialized area with tons of NDAs from the fabs for PDKs and DRC decks.

There is the libresilicon project: https://libresilicon.com/

But I don't know how usable they are.

> MCUs and cyber-physical systems was like this until the Arduino happened

I'd say it technically happened when cheap flash based microcontrollers become available.

Before you had to have *at the very least* EPROM programmer and eraser (or CRT TV I guess) to even put your code onto microcontroller. Flash based microcontrollers meant all you needed is parallel port and some wires to start programming microcontrollers, dropping the barrier to entry by orders of magnitude if you already had PC to program.

Then we had first wave with Basic STAMP and similar making it even easier.

Then it was Arduino that exploded it and there were other factors on that too.

>So what needs to happen to make this a reality for semi-con? First off, we need cheap, cheap fabrication. I actually looked at public funding in Canada and how that was going to the big name Universities who had their own in-house fab labs (at older process nodes). The costs of someone not in the inside was nuts. The actual cost should be in the 100s of dollars to fabricate a design (considering the marginal costs).

Like that. Before making PCB was either doing it at home that took a bunch of time and not everyone wanted to play with chemicals, or expensive to prototype PCB.

Now we have extremely cheap low volume PCBs and even well priced low volume manufacturing options. Same thing with 3D printing, went from massively expensive to affordable.

The problem really is that it's still a bit profitable to do so for companies doing it, while doing same for chip-making would be oh so much harder unless someone put some SERIOUS R&D into making "industrial chip printer" where each wafer could have different set of chips (packaging I guess could be handled by requiring each submitted chip to have connectors in same place). So no mask but some kind of DLP or similar projector to do lithography (dunno if that even possible for anything in hundreds of nm range, just guessing)

I'm in digital physical design and part of that 99% of the army. The schedule is king. If we make a mistake it is another $30 million for mask costs and 4 months in the fab to get a new chip to test. Contrast with software where you can change 1 line of code and recompile and test in seconds. We do so much to minimize risk by reusing existing blocks, licensing third party IP that has already been validated in silicon, and armies of verification engineers.

Yeah but that speed also means we can go from idea to broad release quickly which both means quicker to market to deliver value but can also mean that a screw up scales quickly too.

> recompile and test in seconds

i wish

And you don't need to compile in many cases..

Compare that to the lead time of electronics, though. I just started in PCB design which is leagues more time forgiving than silicon design and fab, and even then if a non-generic component I'm designing for goes out of stock, the lead time for a new batch is anywhere from a month to a few years, on average around 4-6 months.

Thinking about the design process, testing and revisions, all the way to fab and then market, we're probably talking years just to see a single design reach your test bench (I'm speculating). Oh and millions, because the major chip fabs only do things in large batches.

Compare this to software. It is, indeed, seconds in comparison.

I'd love for semiconductor fabrication to get fragmented out like it sounds like it will.

> I'd love for semiconductor fabrication to get fragmented out like it sounds like it will.

Why does it matter for lead times?

If 2 fabs with a lead time of X turn into 5 with a lead time of X... your still waiting.

Lead time isn't because of some magical bottleneck. It's because of demand and batch size. If university students can somehow decimate the batch size needed per design, and enough groups do this (or find new ways to reduce the process size in order for more firms to do this), and enough of them do it at once to help demand, then the lead time goes down, and so do costs.

Do you understand what lead times are?

For example, 9 women can't have a baby in 1 month, because there is a series of necessary processes that has to be undergone and can't be accelerated.

You seem to not get the GP's point at all. The bottleneck in the lead time of semiconductors is not physics at all.

Those are called supply chain shortages.

I'm fairly certain the actual lead times to procure semi-custom chips are indeed 1 month+. And a fully custom chip is 1 year+.

The months long lead time is not purely a scheduling/capacity issue though. Producing the masks for these modern processes is very complex and slow. Adding more fabs won't help because because its a latency issue, not a throughput issue.

The OP is talking about years-long lead times.

AFAIK (but never studied the latest processes), making a mask takes weeks. That means that the minimum lead-time on making masks is weeks long, because there is no interdependence between them. But I'm sure this is one of the steps that add up to years on practice, because of production limitations.

Actually making the chips has a higher floor, because every step is interdependent. With a hundred steps, each one taking half a day, we are talking about 2 months here. There are probably more than a hundred, and a few take more than half a day (but many take less than it), so yeah, I'd easily expect a 4 months minimum. What doesn't compare to years at all.

>> 99% of the army is responsible for implementing the design in as repeatable & standardized a fashion as physically possible so that there is even the remotest chance of yield+profit.

That sounds like a mature and competitive field. I am suspicious of companies and fields where people aren't working like this. Look at the car industry, or energy, or farming, or shipping, or even aerospace. Only one of every thousand engineers at Boeing will ever decide the shape of an aircraft wing. The other 99% are there to implement and optimize its construction. Any company not spending 99% of its energies on optimization does not operate in a competitive environment and is therefore very likely on borrowed time. Eventually a competitor will appear or an IP monopoly expire and the easy times will be over.

(Or it's a very young industry)

In the grand scheme of things... in a universe that's billions of years old. Humanity is 200k+ years old. Civilization is 6k years old.

Computers? 100 years old.

What part of any "modern" industry isn't still very very young? even industries like construction that have been around for thousands of years are still relatively young.

Computers and computing? infinitesimal.

Old by how close the industry's products are to physical limits.

For example, the newest gas turbine designs are already exceeding 50% of the maximum theoretically possible efficiency as allowed by thermodynamics. So it doesn't matter how many thousands or millions more years gas turbines are optimized, by us, aliens, future descendants, etc...

No future gas turbine industry in this universe can possibly double the thermodynamic efficiency of the finished product.

>Old by how close the industry's products are to physical limits.

I was just listening to an AI podcast and they were duscussing going from the 1 second, 1 minutes, 1 day [unit of response - I cant recall the name of the measurement] -- but I assume thats the "moores law" of AI right now?

And as we get closer to the physical limits of chip production scale/die/etc - I assume they will be scaled horizontally while the GPT-X capabilities will scale volumetric-ally.

Is this a sound assumption?

Not OP, but even GPT-X and ML reaches limits due to lack of compute and/or datasets.

For example, CNNs were largely known and discovered by the 1990s-2000s, but the compute simply didn't exist yet until GPU manufacturing became commoditized.

OpenAI's massive quantum leap is thanks to the massive corpora they were able to leverage, which until the past 5 years, simply didn't exist.

This is why we've seen massive jumps in Mandarin Machine Translation and Computer Vision from the PRC due to their massive corpora/dataset of English+Mandarin language news from CCTV/CGTN and local surveillance camera data respectively.

This compute limit is a big reason why the US Federal Govt has been working on the Exascale Computing Project for example.

>"reaches limits due to lack of compute and/or datasets.

Yes! thats what I mean by scaling VOLUMETRICALLY -- scaling horizontally and vertically are now replaced by volumetrically. (coining a term?)

its going to be an obloid-worping sphere-oid

Thats what I see down the pipe..?

Thoughts, anyone?

The technical terms are Scale Up vs Scale Out.

Even for training on a massive dataset, you are still limited by compute and processing time (good ole Computational Complexity), which is why HPC projects like the Exascale Compute Project were created in 2015 along with additional funding+research in efficient and alternative data structures.

I highly recommend going down the rabbit hole of High Performance/Accelerated Machine Learning.

Scale 'out' has been around forever.

The eminence of AI is more 'volumetric' than 'out'

Out scales "up and out like a hill or a lift"

- volumetric is spherical - it presses into the future AND the past (it already has been harvesting history, but created a fire-hosed spigot-interface for the future as well) -- and draws it into its center for eval...

They are different.

Always on the positive expressions of XYZ axis - but had ignored the negative of each, where AGI will go in all dimensions...

I don't think this aligns with the Chinchilla scaling law. There is indeed a point at which you can oversaturate a model with data, as well as such thing as not giving it enough. Compute is the constraining factor, and it scales more-or-less linearly in both directions.

Thank you - what is your reco for best info on that law?c

By "very young", I'm talking about industry that is still figuring out the right direction. Monoplanes made biplanes obsolete, no matter how perfectly the biplane's wing was designed. That kind of dramatic and rapid change tends to happen during the infancy of an industry.

See the early shifts from vacuum tubes to semiconductors, from expertly-crafted BJT to crude CMOS, or the rapid march of good-enough architectures in the latest process node clobbering beautiful architectures in older nodes. As the industry has matured and course has stabilized, perfecting the design has grown in importance.

> Any company not spending 99% of its energies on optimization does not operate in a competitive environment

That’s how a company stagnates and eventually gets replaced. Yes operational efficiency and optimization is important, but you only do that if you have a big moat and not much growth ahead of you. You need to spend at least 30% on growth and innovation so you can stay relevant. Higher if you are the one trying to replace the 99% ones.

A "mature and competitive field" is going to already have all low-hanging fruit taken and implemented. The moat is operational efficiency. The barrier to entry is the high costs of investment and general lack of access to know-how.

Not every industry is like software/tech. And the tech industry is already moving towards a "mature and competitive field" model. Otherwise why is Google and Meta cutting all their unnecessary spending and touting efficiency over growth? (besides AI tech, I suppose) Why all the layoffs?

Hah, having worked in automotive, things are dysfunctional as shit. Oh, they ship things, sure, but mostly as a consequence of sheer will of plowring man-hours in thins untill they kinda work.

Curious what was your experience operating in those fields.

And here I was wondering how manufacturers can take a thing that works and make it not work in new engine version.

Shit like "we did chain driven timing and it worked, but in new engine it prematurely wears", or "someone decided it is fine for belt-driven timing to take dip in oil, oh surprise, we now have parts of the belt landing in various parts of the engine".

Or my favourite, "metal, belt driven water pump is almost never replaced thru the lifetime of the engine and it's cheap, let's make propellers plastic and also drive it from separate electric engine"

Forget chains. Gear-driven cams are best. My motorcycle had them, but honda switched to chains in the new models because cam gears made a small high-pitched noise.

For RPM for sure, although I'm not really fan of the noise.

I think it's simpler than that. Semiconductor manufacturing is such an advanced field, and there are so few openings for engineers, that they can be very selective in who they hire. We don't need to conclude that semiconductor engineering is particularly difficult, it's sufficient to conclude that they are simply picking people who will work lots of unpaid overtime, simply for the thrill of working in such a cutting-edge field.

You missed the part where they do it for a fraction of the salary.

I suggest there's a kind of 'cultural imperative' in Korea that is so different from the US that it doesn't even begin to factor into our equations.

My Korean grad school colleagues went to intern at Samsung for serf pay, the only way that could work is if there was some kind of culturally implied merit.

The US also needs a 'different kind of ethos' to compete in semiconductor, Cali Surfer Vibes won't cut it, neither will NY banker vibes, neither will SF Social Media AI Allbirds vibes, nor Cambridge High Intellectual Healthcare Conference vibes, nor Texas Energy Industry or DC Government Consultant etc..

And that will be impossible without a fair number of migrants who will be mostly Asian - so where is this going to happen?

Maybe they need a 'New South Valley' about 50Km south of San Jose, a cross between Orange County, Silicon Valley, Cambridge, and Seoul aka Cali, but slightly more formally regimented, a bit like the actual old-school Silicon Valley which was at the time, away from the buzz, a bit of a sleepy burb where people focused on hard stuff and economics mattered.

> The US also needs a 'different kind of ethos' to compete in semiconductor, Cali Surfer Vibes won't cut it, neither will NY banker vibes, neither will SF Social Media AI Allbirds vibes, nor Cambridge High Intellectual Healthcare Conference vibes, nor Texas Energy Industry or DC Government Consultant etc..

If someone like Netflix/HBO did a proper documentary with Samsung about their LSI fab in Texas, HR would have more applicants than they would know what to do with. This 'ethos' can be created. I saw what it could be like when I worked there. I really believe seeing the whole thing in action is the primary selling point. The draconian security makes sense once you walk that catwalk and witness with your own 2 eyeballs what is at stake. Capturing this sensation for the masses is the challenge.

They don't pay enough is the point. Really smart and competent people don't work for 1/2 wages unless they are caught up in some kind of system like in Korea where it's a culturally backed norm and frankly they can't really escape.

Americans will not work for fractional pay unless it's a kind of a 'wartime' / 'emergency' situation.

> The US also needs a 'different kind of ethos' to compete in semiconductor, Cali Surfer Vibes won't cut it,

Wait till you hear about the history of the semiconductor industry!

> the stakes are way too high to allow everyone a chance to play cowboy with the tools

This is really what pushed me into just doing digital design as a hobby and making my money elsewhere. It's understandable that chips companies are so risk averse (a bad tapeout or, worse, a major escape can cost the company billions) but it is really a miserable experience to have so much passion and ambition but never being able to do anything other than extremely conservative incremental changes over a decades long career.

I worked on a campus with alot of semiconductor investment and education for a somewhat affiliated entity.

The thing I noticed was that the crunch work was brutal, parking lots were full till 11pm half the year. The parking lot told the story - the execs had Teslas and BMWs in their gated lot. The troops were in 7-10 year old Hyundais and couldn’t afford to buy lunch in the cafe.

If you want to be in the semiconductor industry, the real winners are the tradesmen. They were all driving $70k pickups. Pipe fitters, electricians, operating engineers, etc make bank. If you know how to build the supporting infrastructure around some ASML tool, your income is limited by your desire to work. I can think of a dozen serious contracting companies in those lines of work that spun out of building that place.

Unfortunately, this is how Soviet engineering was operating until it all tanked. Engineers with university degrees getting paid peanuts as an official policy.

It's sad seeing this practice repeated. And people falling for that.

Official policy sounds like the Soviet Union was the only employer and they were deciding to pay peanuts unilaterally.

Is this the case with modern hardware engineers? No one is forcing them to accept low pay, or learn how to write software and earn more.

Sounds like they are getting paid market price, and it has been decades since it is known that pay for hardware engineers is low, indicating that people should stay away unless they want relatively low pay.

> Official policy sounds like the Soviet Union was the only employer and they were deciding to pay peanuts unilaterally.

The official doctrine was that it was a Laborer's Republic, and so physical labor very often paid more than mental one. It was all about making the common man feel as if the state is really organized for the benefit of people like him, and not some narrow elites. Of course, it was BS.

No the tradesmen were traditionally making bank (well the skilled ones) because “power of the proletariat”.

There’s no compulsion, but people tend to nerd out in the specialty and lose their way. It’s hard to be the respected engineer in <pick a topic> and start over as a newb. Many of the folks in these fields get caught in a visa trap as well that makes the friction that much stronger.

One of my classmates in high school was a well regarded engineer that specialized in some sort of critical manufacturing process. Published papers, had patents, etc. Unfortunately, a new tech replaced that process, and he found himself laid off and kinda screwed.

He had a good network, crammed for a PMP and got into the project management racket and went back to school. Now he does cyber analysis stuff.

This comment is why the industry died in the US.

There is no such thing as a 'free market' really. Once people have been pulled into a specific academic program without 'true knowledge of market forces', the switching costs are high. They take an entry level job because, well, it's a job. 4 years later they find out there are only 3 employers who want their ultra niche skill, and so the employers use their power to leverage to shit wages.

This shortsighted action causes the opportunities in high wage areas to fail as devs do 'other things', while the jobs move to places where there are even more rigid systems and the 'work hierarchy' is integrated right into culture. Like Korea.

It's also why even very talented software developers outside the US are paid so little - it's hard to justify high pay in most companies, as it's not clear how to discriminate between the 'true talent' and 'regular devs' - so talent gets anchored very low. Devs languish or move or find solace in working on something interesting and intellectual, which is common.

It's why there is no real 'hardware' industry in the USA, all of the jobs are niche even if they are very hard, industry can hold them to the wall for low salary, it's more effective for them to be carpenters, while the jobs go to Asia.

All of that plus the fact that software can move much quicker and can be more profitable, the 'windfall' economic part of software makes the VC math work out better to justify more investment.

If we were to reorganize that around net value creation instead of just local barter between trades and enterprises, it'd be a different situation. Very hard to do. Government is stepping in with a big starting point, will industry follow suit? Maybe, I doubt it.

Somewhat related but anecdotal—there was so much analog gatekeeping during my EE degree it really turned me off of analog electronics design. Some professors acted like nobody was worthy and delighted in ridiculous trick problems on tests to weed everyone out. Many also had the favorite student syndrome that you mentioned where everyone but X was an imposter. The industry also seemed to not want anyone in analog and semi manufacturing anyway or had very high entry requirements, although kept complaining about a lack of candidates publicly. I bet it’s somewhat different today with the resurgence of modular synths and hobby electronics, but I can only imagine that it might make some professors double down on the hazing mentality. All that is to say, in some areas STEM educators have serious cultural issues to work out before just blasting away with funding expecting results.

Is this in the US? I suspect (am hopefully) there may be a different culture elsewhere.

Yes, the US which is what this article is focusing on.

> The industry also seemed to not want anyone in analog and semi manufacturing anyway or had very high entry requirements, although kept complaining about a lack of candidates publicly.

Didn't take EE in the US, but your statement here seems to be universal. Especially true about Analog professors being gatekeeping jerks.

This thread is so cathartic for me.

Completely the opposite here (not in the US either), my analog professors were some of the most passionate and caring professors I had.

Didn't end going in to analog design because although I think it's interesting I don't think it's actually fun to do the design. I feel like digital design has much more variety.

The funny part is EE professors (analog or digital) don't even teach you any of the areas where analog is practical to digital electronics. They'll have you doing tons of abstract differential equations and studying transistor characteristics, and then kick you out into the world unprepared for the things you should know. Which is why Johnson & Graham should be the first book on the shelf of any new digital EE grad.

Well, to be fair, if there is not a place at all for the people, those professors do in fact help their students.

They got that mentality from some kind of trauma themselves. I'm not sure it's even the wrong adaptation, since it's not in their power to fix the systematic issue.

One of the systemic issues that needs fixing is the gatekeeping itself. Weeding out serves no purpose because there's work besides just analog ic design roles. Industry complains because undergraduate programs have become so detached from reality focusing on stupid stuff that everyone graduates way more underprepared than they should be.

Did you attend the University of Illinois Urbana-Champaign?

I had a similar experience in Germany. Studied Engineering Physics with a major in semiconductor technology. At university we learned everything from high level programming, VHDL, electronics down to semiconductor chemistry. I didn't find it boring at all.

Judging from our industry partnerships the semiconductor industry seemed to do well at the time. At least that was my impression. When I graduated in 2001 the signs of impeding death were already there however, so I worked in aerospace for a couple of years, went back to university for a computer science degree and ended up in software anyways. Not the worst thing to happen for me personally, but honestly I still don't really understand how a promising industry could die like that.

> in 2001 the signs of impeding death were already there

What were those signs?

For me specifically Siemens divesting or spinning-off its semiconductor assets. Before that my study program was almost a free ticket into Siemens. Working for Infineon, EPCOS or (a little later) Qimonda did not seem attractive to me, specifically because it was well known at the time that these divisions were spun-off because Siemens wanted to reduce the risk from the quote "highly volatile semiconductor business" for the parent company.

MBAs, investors, and stock market ruining stuff again...

Reminds me of talking to brewers and winemakers. The brewers said they could experiment as much as they want because the worst case was you get some different beer.

The winemakers had to follow things exceptionally carefully because you basically couldn’t improve anything, but you had innumerable ways to destroy the wine and end up with vinegar.

In other countries, with less technical jobs, this effect applies to all kind to technical fields.

You go to an electronics engineering class. There are a few superbrilliant people. The rest are quite clever though.

As there are a few jobs available for the technical skills that you learn at class, only the superbrilliant get those jobs.

It turns out that the companies say to these superbrilliant guys: "I am giving you a unique opportunity to work in this top field in this country. So your salary is not about money, but motivation".

End of the story: the rest of the guys, not the super brilliant, end up working in more generic stuff like management, consultancy, generic simple software.... And they make way more money that the superbrilliant guys.

These examples are taken from Spain. Althoug I have seen similar in other European countries with more technical jobs.

The way that toxic prof treated the class makes no sense, but the barrier to entry in industry makes a lot of sense, even if we don't like it. Semiconductor is just insanely capital-intensive. Now that it's become a major issue of strategic concern for the USG there is finally some capital available for it.

People have finally begun to realize that if all you do is build software, you're completely dependent on your hardware suppliers. It's like being a commuter society that doesn't make their own cars.

Then pay up. If the industry is so important and a ruined batch costs $30M, it's a disgrace if you can't pay the people six digits.

They do, for the 1 in a thousand engineer. For example the vast majority of TSMC engineers aren't making that much, but a select few easily earn 10x FAANG compensation.

In Taiwan working in semicon will earn you more than software engineering. It’s insane that majority of software jobs in Taiwan earn less than monthly rent costs.

Why don't they work remotely or outsource to the West?

How many years ago was your experience?

There’s definitely an increase in interest and demand for semis. Your experience might be very different from graduates looking today and in the future as demand increases.

Having said that, the vitriol and toxicity in colleges and towards one another needs to cease. I see that being more naturally achieved in an economy that is growing and seeing things getting cheaper generally. That would look like vast investments in small, big, and mega projects. Imagine how much different the US today would be had it never built a highway system and stayed to only where the trains/horses went. I think it’s 80-90%+ of US GDP transports via truck over a highway at some point.

What kind of job is X doing and at what company?

Are they working at a fab? Working in the manufacturing portion of the fab? Doing process design work on a computer? There are so many different types of jobs within the semiconductor industry and some are going to pay far higher than others.

As I said in my other comment in the thread I have worked at many fabless semiconductor companies. The different between software and hardware salaries at companies like Nvidia and Apple is around 10% or so, not 5X.

Last I heard it was Micron. Not too sure what X got into. He was definitely a genius though.

[Edit] Just checked. He has left the industry a few years back, and is now a software engineering manager. Even the brightest student has given up. Someone on this post said it best, they need to have salaries matching the amount of skill required.

If X works in something low level like device physics then there are a limited number of companies doing cutting edge fab stuff like that. Intel, TSMC, Samsung. The small number of companies limits your salary potential.

It's the same with being a maintenance technician for a piece of ASML 3nm semiconductor manufacturing equipment. You may be highly skilled but if there are only 3 companies in the world that will hire you then your salary potential is limited.

I know about 30 people developing hardware at Apple (writing Verilog, custom analog serdes design) and you can see that their software and hardware salaries are similar. In some cases hardware is higher although I suspect that is because of the limited sample size of salary data at the ICT6 level.

https://www.levels.fyi/companies/apple/salaries/software-eng...

https://www.levels.fyi/companies/apple/salaries/hardware-eng...

There is more to life than money but people who care about money need to look into the companies and salaries before they commit themselves to 4 to 10 years of college.

Yeah you're right. It probably really comes down to a lack of potential employers. Hopefully this push by the US can change that.

Other branches of engineering: mechanical engineering, material science, chemical engineering, certainly civil engineering tend to pay at a lower level overall as well. And the purer sciences often require advanced degrees to get "real" jobs at all.

When I worked for a computer systems company after school, my impression was that the various types of engineers at the company were on fairly similar pay scales. What I think you've seen the past couple of decades is that out-sized salaries at some mostly software-related West Coast companies have really driven up compensation for specific types of jobs. It seems more likely that will settle down to levels with comparable jobs over time than the other way around as a lot of people who are late to the game try to jump in.

That's an interesting angle I'd never considered. If the government simply creates a lot of potential employers via funding, the labor market becomes competitive, which improves working conditions, compensation, and ultimately attracts talent that would not have otherwise considered the industry. The most interesting part is that outcome is almost a certainty if the government focuses on creating that competitive labor market through their investments; it doesn't require moonshots, breakthroughs, or wild successes to work, yet it can produce them, and can become self-sustaining.

> If X works in something low level like device physics then there are a limited number of companies doing cutting edge fab stuff like that. Intel, TSMC, Samsung. The small number of companies limits your salary potential.

You should be very careful about comparing raw salary numbers: a much better metric to compare is "'salary' minus 'cost of living at the job location'".

If you're pulling in 400k USD per year, it actually doesn't matter what the cost of living is. You tie up 40% of your income in a 1-2 million dollar house and still save $50-$100k a year while buying everything you want on a daily basis, then retire after 20 years and live like a king

I think maybe in Taiwan and China semiconductor engineers are earning a good salary, especially China when government is dumping insane amount of $$ into it. But there is a lot of hype though.

Software salaries in both places are also a lot lower.

Yeah but for companies as Ali and Tencent you can have lucrative salaries.

How does it not make sense? As a software person you are completely relying on the hardware to do its job. You are putting your blind faith into the work of semiconductor engineers in order for you to get your paycheck

It makes no sense because if the semiconductor industry only hires the absolute best student of the cohort, that's a way higher barrier of entry than software where 90% of the students in my coding class is now working in MANGA. That should guarantee an insanely high pay for the semiconductor engineer. Instead he's currently making less than what fresh grad MANGA hires are paid.

But you said they hired 1 person out of a class of 20? Obviously the few who get in can't ask for a high salary with that many candidates per position.

>But you said they hired 1 person out of a class of 20? Obviously the few who get in can't ask for a high salary with that many candidates per position.

Obviosly they can, assuming that these companies hire "only the best". That means that in the demand/supply calculations, the demand is not for "graduates of semiconducting programs" but "only top top graduates of semiconducating programs, ignore the rest".

That's in much shorter supply, and means that 95% of the "class of 20" aren't even considered worthy as candidates, and aren't competition for those top semicon jobs. They'll need to find work in another field (like switching to software), or low level semicon jobs.

It does not make sense because software could be written using the same quality standards that CPUs use, which would involve a lot of formal reasoning and proofs.

This work style, however, is not valued at all and is actively suppressed by "flat", non-expert hierarchies that reward people for being popular and sloppy.

It works for web companies because errors have no consequences for them and the web has turned into a gigantic tabloid anyway.

Abstraction layers become simpler, more rigid and more robust as you peel them.

A painter uses rules of thumb all of the time, yet his work is entirely dependent on chemical engineers getting their science and formulas right. Artistic painting is just one of many uses of paint (and probably its most approximate), but both the painter and the industrial product engineer need the same quality assurances regarding their colors.

Hardware needs to be solid to allow all kinds of software the freedom to be built.

In my experience what trade-off is the correct one depends on your ability to iterate and where your risk comes from. Can you ship infrequently, but you know with high confidence what product/market-fit looks like (extreme example Mars mission)? In that case you want all the planning and verification you can get. Can you ship often, but have little confidence in what your users/customers want and what will grow your business (extreme example social media app)? In this case you probably want to prioritize speed of iteration and rework technical aspects once you know it's what's needed.

I've had a very angry product manager because a feature took almost a second to load. I told him that we could make it load much faster but it would be almost a week of work and he should validate the feature with customers before we invest in that. "Of course we need this feature! It's core to the product!". So we spent a week. It worked realty nicely and scaled pretty well. In the next 12 months the feature changed completely twice because customers didn't find it useful. Then it was removed entirely and then we had to bring it back because the sales team liked it for demos. All that iteration was required to find what the market needed, not fit technical reasons, except for the performance work which was entirely wasted because it all got redone after the first user feedback.

I think you also overestimate the formality in CPU design. Yes they do a lot of testing, but it's not lik an airtight mathematical document for the whole chip.

This reminds me of an article I read a while ago https://alastairreid.github.io/mrs-at-scale/

MRSs would enable much more of the formal verification parts to be generated.

More often in startup land, you are relying on the investors for your paycheck.

The place where the value is is not at all related to where the difficulties are or the bottlenecks are. The universe does not have that kind of sense of distributive justice.

You could write the same thing about the concrete slab the office or data centre is on. The barrier to being a concreter is very close to "applied for a job as a concreter".

Yes, and civil engineers have to follow a lot of regulations and have to plan things carefully

That’s an hilarious story because the asshole professor ends up being right, a naive optimist (the supposed reader) gets a dark surprise, and all students involved get reasonably happy endings.

Could be a Hollywood movie, if you fill in the gap between the penultimate and ultimate paragraphs with some interesting action

The professor told them after they already invested years, they will end up unemployed. They did not, because their skillset was valuable somewhere else.

Unemployed as a semicon guy. I doubt the professor meant they wouldn’t get a job as a fry cook

The way op told the story, the professor meant what he said to humiliate the students (despite he likely knew, they would be wanted in other professions). And a fry cook is not quite comparable to a software engineer.

I worked in process monitoring and supply chain departments at a couple semiconductor companies.

There's an impenetrable culture of "best practices" that keeps the tech from catching up with the times. I eventually left because my career stagnated being on teams who couldn't update any of the internal tools because of this or that exec who was too scared to explore beyond the tiny niche they carved for themselves the past 10-30 years.

The last team I was on was using JDK 8 and Postgres 8. Vendor lock-in on downstream dependents, I guess. We finally managed to convince leadership after 3 years that if they wanted any ML in their systems at all they'd need to update their stack or make room in their fabs for a Python server on the intranet at the very least. So they gave us a Windows box with 2 cores and 1GB of memory because "that's all that will fit" and "we aren't a security company".

Came here just to say your professor's attitude is infuriatingly toxic, sorry you had to go through that.

I'm curious if your experience of crashing/losing progress is still the norm. Do we have better CAD programs and PCB simulators now?

My partner is an EE but I just learned how electricity works (beyond a middle school level) a few years ago.

There are basically only two firms which produce chip CAD software, and they get to be a conservative duopoly of "enterprise grade" (i.e. user hostile) software.

A lot of the simulation software for PCB design is stuck in the 90s. The core of the simulation hasn't changed so it's not too big of a deal from a results perspective. However, it integrates terribly into the workflow so designers only simulate a small portion of a design and there's little automatic checks and things.

This is just true for a whole lot of the industry tooling. Xilinx Vivado is a bloated piece of crap that'll crash all the time unless you have half a terabyte of RAM. Same goes for lots of other EE-tooling in general. The L and B in MATLAB stand for Legacy and Bloat. People still write programs for PLCs in Ladder, where programs still cannot be portable between vendors, or even different products from the same vendor.

All the companies that produce anything invent their own language for the thing and write their own compiler for it. These compilers are clearly not written by compiler experts.

I don't blame EEs for building bad software. They weren't trained to do it and aren't paid to do it. I blame the "if it works, it works" culture that the industry seems to have. Never go back to refine anything, just keep pushing more plugins, more software; create a patchwork of programs until you get the job done.

Having lived on both sides of the fence, I feel comfortable blaming the EEs. Many are allergic to basic scripting, don't bother to learn how their tools work, and have an elitist attitude towards other design responsibilities (ex. layout, verification).

P.S Set up Vivado in scripted mode with either an in-memory project or non-project mode. It works like a champ.

> Many are allergic to basic scripting, don't bother to learn how their tools work, and have an elitist attitude towards other design responsibilities (ex. layout, verification).

In my very limited experience, most people don't learn new stuff until they're forced to; either by their employer, by their university, or by needing to learn it for something they want to accomplish. This is why you'll have self-taught developers go for years using strings as enums, linear-searching in huge sorted arrays, because it works and why would you seek out something else? I think the solution is to introduce more software development in EE education; forcibly expose them to it. My EE bachelor's degree contained a whopping ONE class that was focused entirely on Python programming. The rest just used cobbled-together C code for microcontrollers, or arcane languages with dumb IDEs for PLCs.

I'll take your hint on Vivado, thanks!

> Set up Vivado in scripted mode

The tried and true :) It's funny seeing people complain about Vivado bugs when I haven't run into any in years. Sure, the IDE may be absolute garbage, but I've thankfully never run into any bugs in the actual synthesis and routing parts of the package, which is all that really matters.

It isn't EEs building the likes of Vivado, they hire software engineers to do that stuff

And the industry is so brutally cyclical. They'll hire thousands, work them into the ground to get some machine or line done, then let them all go, then hire them all in a flurry next time they need to build something. It's also very toxic in my experience, lots of mean people.

> X ended up being the only person to go on to work in a semicon company. The rest of us ended up in software, earning 5x the pay X was making

Sounds like a great way of nurturing talent... not. Apparently they will only look at your CV if you have a PhD or a very specialized MsC

And all this for what again? "The privilege of working with hard problems?!" I've done enough C already and that makes my head hurt now. Thanks but I think I'll take it easier with "higher level software" while making more money and having less stress

Top engineering school hazing, creating underpaid and not quite mentally balanced people

> The barrier of entry to software engineering is so so much lower than semicon, which still makes no sense to me.

Why ? So much software is far less complex to design than semiconductor stuff

The confusing part is that working on the simple software will likely get you a much higher salary.

For me it's more of sad realization that what's very hard and requires either skill, talent or both just isn't that well correlated with what earns most money.

Gotta have enough demand so other competent people interested in same thing won't drive the value down.

And I do feel lucky that my set of skills correlates with stuff that pays decently.

Just demand/supply I guess. Semi-conductor companies need just a few (even if extremely talented) engineers compared to software. The industry is also very concentrated and not growing that so not that much competition for employees.

I wonder if it's because of the nature of the companies involved. The semiconductor one is actually producing something tangible with huge costs associated with that. Most of the "tech" companies with the crazy salaries are just in the business of seeking rents from an entire market by collecting royalties or subscriptions for automated things, so they have way more money per person to give away.

Almost exactly my experience, plus all the other bad experience mentioned in this thread.

I wish I could have seen this when I was in school.

How many people do you think exist on earth that could build a usable but most minimal form of computer/computation from scratch? Roughly same utility just lowered expectations on horsepower with also the assumption the software would be better optimized.

Is in the US?

> Anything you designed was called worthless because there's already a library

This attitude (also in software) indicates to me someone who isn't very interested in a subject academically. There is so much to learn from prior solved problems and reinventing the wheel yourself.

Are you my prof?

If we weren't interested, we would have quit during the first semester.

Our very first assignment was to design a modified Miller. After nights of struggling with the software and connecting transistors, we were then told psyche! You could've imported from the library that we've never told you about! The rest of the assignments were essentially just importing circuits and moving stuff around and hitting the simulator run. Over and over and over again. It takes a certain grit to stare at the Cadence UI for 10 hours running the simulator button to yield a random result. There's no Stack Overflow, the TA was of no help. Just learning the software alone was hell. The instructions were on printed paper that didn't work half the time. Then you have the crashes that wipe out hours of work. We didn't dare to save our work sometimes because it'd crash the software. Loading the software took forever. I genuinely hated Cadence with a passion.

I'll admit I don't have that grit. My best memory from the course was still being able to design and fabricate my own chip and PCB to go with it, and successfully converting an analog signal to digital and back. The ENOB took longer than needed to solve and it was rewarding. But I'm not doing that as a career.

Only 3 out of the original 20+ people graduated. Only one bothered to apply to semicon. The barrier of entry is high, but the educators were of abysmal quality.

I'm not endorsing bad professor games (which are real). But from my time in academia a lot of students have similar experiences about courses that just aren't for them.

For example, in a state school analysis course, almost half the class decides they just need to squeeze by to graduate and maybe 5 students will enjoy it and do graduate school.

Fair point.

Then again it's difficult to figure out what you are interested in in university at 18/19yo. Maybe just a vague curiosity, maybe the paycheck. But you have to actually take the course to know. I wish there's a better way.

Reply on news.ycombinator.com