62951413

Let's talk about you growing older. The closer you're to 50 the more difficult it gets to find a new job. You get laid off in a major downsizing round. Your COBRA healthcare plan is $2K+/month which incidentally is comparable to your 1-bedroom apartment rent. It could take you a few months to find the next job. Or longer, people discuss it on this very site every other week.

The FAANGs and the capital gains are real but for a minority of us. If you're single and under 30 the Bay Area is a top destination for sure. You won't be that person forever though.

CORBA, Avro RPC, Thrift RPC, gRPC, now this. In this industry each generation wants to re-invent IDL every decade. But anything is better then JSON over HTTP so why not this.

I remember using j.u.c in production back when it was still concurrent.jar. There were surprisingly many jobs in NYC/London circa 2006 if you had Core Java skills.

Core Java by Cay Horstmann (https://horstmann.com/corejava/) is a well-structured and concise review of major features. Historically, the most interesting chapters were scattered across both volumes. So completely skipping the "Fundamentals" volume was ill-advised while the "Advanced Features" volume had some boring and mostly unnecessary chapters.

Did you know that you could have 15 years of working in the Bay Area experience and still be refused a German blue card because you majored in Physics or EE instead of CS? But mere work visas were easily obtainable a couple years ago with a signed contract.

Last years I worked for a SJ-headquartered public company with spectacular compensation. While I was still there they stopped hiring in CA and the gossip was about TX. But on my team new roles were opened in Europe instead (surprisingly, not Eastern). The layoffs were concentrated in the Bay Area too. I heard that since then they started looking for even cheaper locations.

I have recently seen large American companies where managers/team leads are full-time employees while their teams are mostly contractors from LatAm. And even those managers are not on PST. They advertise "W2 contract-to-hire" roles here in the US though.

In the Bay Area I see only a few full time positions. They average literally 500-700 applicants each (1.5K for a Databricks-level company). Accepting a W2 remote contract (sometimes from a CA-headquartered corporation) seems to be the only alternative. They say it's a "contract-to-hire" but I'm rather skeptical. It feels completely unlike the fleeting setback of '08-'09.

Is it really $35K now? Because the way I remember it, a couple years ago the top trim Elantra was ~$25K and the same for Civic was ~$30K. So the difference wouldn't be as dramatic in purely monetary terms.

Even people with 20 years of Java experience stay clear of JEE. I imagine the closest to Django on the JVM would be the Spring framework anyway. As of now, ML/AI is synonymous with Python so it could be your hint.

It's a pity calcite.apache.org has never been known for extensive documentation.

It has the most interesting/conceptual/detailed discussion of the streaming system semantics (e.g. interplay of windows and stateful stream operations) I'm aware of to this day. At least as far as Manning/O'Reilly-level books go. So I'd put it on the same bookshelf as DDIA.

It's a little biased towards Beam and away from Spark/Flink though. Which makes it less practical and more conceptual. So as long as it's your cup of tea go for it.

For a Spark shop Delta is the default choice. If you deploy to AWS then Glue encourages you to go with Iceberg. What makes people use Hudi?

Consider https://www.amazon.com/Dynamic-Programming-Coding-Interviews... for the most pedagogical approach I have seen.

A casual search on LI shows 300+ applicants for a regular hybrid position and easily 1,600+ for a remote position in a prestigious company such as Databricks. I'm really curious how one can compete with candidates outside of Bay Area/CA if the CoL alone disqualifies you. Go East, young man?

If you subtract Spark/Flink-related ones not really. Plenty only in comparison with Kotlin backend roles (i.e. non-existent in the Bay Area). It's very disappointing to me personally but it is what it is.

SF to LA and SF to Crescent City are both over 350 miles and you never leave CA.

Keep in mind that Scala had been released before golang was. And became popular around the time golang was released. Without GOOG-scale resources behind it.

It's a real mystery. Some popular conspiracy theories:

* Scala projects the Haskell-on-the-JVM image too much to scare off the practically-minded. It has no ORCL or GOOG behind it (its cousin F# has had official MSFT support for a decade at least).

* Kotlin is strongly associated with the frontend. Just try to search for a backend Kotlin role in the Bay Area to see.

* Java 17-21 is catching up with Kotlin fast enough to make many people doubt the upskilling effort.

There was a _long_ hiatus in Java progress between 1.5 (release in September 2004) and 1.8 (released in March 2014). Before Java 8 the language was, basically, the previous generation's golang. in that timeframe there was no Kotlin, Scala had little toolchain support and some sharp edges. So many smart people left the JVM altogether.

If you are a Spark shop then choosing Delta over Iceberg is a no-brainer. It's simpler and perfectly integrated. Not to mention that the Spark's Delta connector can now generate Iceberg-compatible metadata too.

The choice between the two resembles the choice between Parquet and ORC circa 2016. Two formats of broadly the same power, initially biased by a particular query engine, eventually at feature parity and universally supported.

We have got a decade of experience with OSS from Databricks so doubting their "open ecosystem" status seems a little theoretical.

Every time you touch anything ML-related you need it. Data scientists not only love their Python but also don't usually use libraries that have JVM bindings. I have not noticed much improvement in this area since 2018 and for some reason djl.ai is not popular in the wild.

Other than that, I can think of only really niche use cases (e.g. calling ffmpeg, query engines using the C++ Arrow implementation, likely something trading-related).

You can glean more from VC blogs. For example, https://www.saastr.com/carta-startup-shutdowns-are-up-237/

Aren't we going to get copious amounts of documentation for free? To me chatGPT logs look like JIRA descriptions, commit messages, and basic design specs in disguise. And they are not optional anymore.

On a related note, I'd expect the "DSL+generated code" approach to grow in popularity for common domains. Think gRPC or JOOQ. A terse IDL snippet and you get a lot of functionality that you don't even need to test or even put under version control.

You can actually see a VC seriously discussing real-life experience with doing it. For example, in the latest "This Week in Startups" episode (https://youtu.be/RcRdwo0lI08?si=528IZSVd1kTjT5cZ&t=3335).

Just think about golang as the next generation's Java. The same strong points (concurrency, GC, "simplicity" for junior developers), the same egregious mistakes (generics alone prove it for me), the same BigCo backing, the same undeserved early popularity.

Re: math professor elites

"David Coleman is often described in the media as the architect of the Common Core ... earned a B.A. in philosophy .. a second B.A. in English literature .. a MPhil in classical philosophy .. he couldn't find a high school English teacher job .. he became a consultant at McKinsey"

I don't know about the US but in the Bay Area PTO hasn't been negotiable for about seven years. It's all "unlimited" now and so there are no official numbers to compare anymore.

The good news is that in large publicly traded companies people routinely fly back to India for a month so 4-week vacations are a thing if you are good at leetcode.

4. You'd like to work on advanced technology without a "PhD from CMU". Think CockroachDB, Druid/Imply, StarTree/Trino, ClickHouse (Hadoop/Cloudera a long time ago, Spark/Databricks when it was much smaller).

Workers on collective farms up until the early 70s were not allowed to have passports issued to them. Without a passport one couldn't move to a city.

Also, those millions of the most productive of them murdered in the Gulag "from the 1920s to mid-1960s" surely "made their own decisions" poorly.

Where your position touches upon reality is that the US economy has been getting increasingly centrally planned since 2008. It's a major problem.