throwawaybbq1

Does this mean a huge hiring uptick in the US/layoff reversal? I do think this law caused some of the bad market. Will undoing it get us back to where we were?

Can you clarify this? You saying non H1B's would not pay the crazy high rents of SV's housing?

Curious where you sourced the parts? In Canada, shipping kills it for me. When I priced out the robot + electronics + $100 in shipping, I am around $700 - far cry from the $100 on the "sticker".

My taste buds have become extremely muted after covid bouts and other sinus issues. Wonder if this would be helpful.

In the US? I don't think what you are saying is supported by real data. My understanding is that US works did see an improvement in incomes in the last decade but Canadian workers did not.

What makes life better for everyone is competition. Canada's stagnation can be be summed up in a single phrase - lack of competition. Generally, the US has been a free-for-all when it comes to competition and hence its populace enjoys some of the best living standards.

I'll also relate my experience traveling the subway in Asia vs. Manhattan. Asian transit seems like space-age compared to what we have in the West. I think UBI won't save us as the income must come from somewhere. Hiking taxes kills incentives. The better way is to have more freedom/efficiencies in my humble opinion.

In the limit, this is like arguing against the use of wheels. Automation improves labor productivity. Economies that have not invested in capital, have seen labor productivity and incomes stagnant. This is a current debate going on in Canada (especially as compared to the productivity and income gains in the last decade). Canada does have strong Unions, so I wonder if this is related.

Another thing that seems troubling is how a small group of people can hold a majority of the country by the bXlls. Given how this is an election year, I can see this turning into huge fiasco. The rest of the economy is collateral damage.

GGUF files seems to be proliferating. I think some folks (like myself) make an incorrect assumption that the format has more portability/generalizability than it appears to have. Hence, the horror!

I've also been trying to figure out GGUF and the other model formats going around. I'm horrified to see there is no model architecture details in the file! As you say, it seems they are hard-coding the above architectures as constants. If a new hot model comes out, one would need to update the reader code (which has the new model arch implemented). Am I understanding this right?

I'm also a bit confused by the quantization aspect. This is a pretty complex topic. GGML seems to use 16bit as per the article. If was pushing it to 8bit, I reckin I'd see no size improvement the GGML file? The article says they encode quantization versions in that file. Where are they defined?

This is neat to know. On Ollama, I see mistral and mixtral. Is the latter one the MoE model?

I assume you are referring to Llama 2? Is there a way to compare models? e.g. what is Llama-7b equivalent to in OpenAI land? Perplexity scores?

Also, does ChatGPT use GPT 4 under the hood or 3.5?

I am not good at investing (lost money every single time I've tried). Liquid cash gets spent. I am paying off my 5 year fixed mortgage as fast as I can, as I will be up for renewal at the end of next year.

Some people don't have the financial savvy or time to optimize. One size does not fit all.

Immigrants. 95% (made up stat) may be poor but the 5% who are not, can afford to. In Canada, there is a recent narrative which I think is correct that immigrants are driving our inflation. We expanded our population of 30 million by about a million in the last year. That's a million new household formations, and some people bought a buttload of cash.

This was actually a very nice article! Thanks!

I liked the last line the most .. I think we've all gotten accustomed to zero interests, and this has changed our behavior. My father to a large extend screwed up his (and our family's) life because he lived in an era of high rates. He did not understand the world had changed. We could have bought a house for cash in 1995 but he chose to rent. After that, it was always a bubble, with no buying opportunity like 1995 ever.

It makes me wonder if all of us have similarly not realized the world is a different place post rate hikes. There is this inevitable dogma that "rates will go back down". A lot of people are making that bet and I wonder if it is just history again.

What I don't understand is what is driving the US economy today. It seems to be firing on all if not most cylinders. People I know who got laid off are finding work (I hear negative experiences too and feel for those people). Hiring in tech seems like it is picking up.

US mortgages are unique I believe. Canada does 5 year fixed and you refinance at the current rate.

I believe the US mortgages work because of Fannie Mae/Mac and a market in mortgage backed securities ( MBOs? I thought they caused the 2008 crisis but I think they are still a thing with better risk management, I dunno?).

I work at an industry research lab. Key challenge for LLMs is the legality and massive resources needed to train. I have research colleagues that are convinced that even OpenAI may be on shaky legal ground. A lot of non-profit and academic liasoning helps to muddy the issue (academics have fair-use exceptions).

If you don't see the potential of the tech and the rapid advances, I can't help you. But the issue around deployment is more legal (and perhaps not enough GPUs to go around).

I'm a hobbyist in this area, and think the field is somewhat early in its development. MCUs and cyber-physical systems was like this until the Arduino happened. Arduino may not have been the first to do exactly that but it was just good enough to cause an (re?)explosion in electronics as a hobby. During its hayday, I would say Arduino was a core element of the Maker movement.

So what needs to happen to make this a reality for semi-con? First off, we need cheap, cheap fabrication. I actually looked at public funding in Canada and how that was going to the big name Universities who had their own in-house fab labs (at older process nodes). The costs of someone not in the inside was nuts. The actual cost should be in the 100s of dollars to fabricate a design (considering the marginal costs).

There are people that do this at home but it doesn't work either due to chemicals being pretty dangerous and the need for a bunch of equipment. I bet the amount of money the EU spent on its first metaverse townhall (or whatever it was called .. the thing very few people attended) or a tiny fraction Canada wastes on silly things promoting youth culture or whatever, they could fund a lab that is actually open to the public, with the express mission of promoting hobbyists and education. This will NEVER happen because (a) it needs a professor who is on the inside with a kid-like passion in this tech and a commitment to bringing it to the masses (I see some profs like this at schools like MIT but it is so rare at large, competitive schools like the big ones in Canada), and (b) it does not have an instant payoff for the govt. They don't want dabblers and vague educational outcomes. They want workers with degrees.

I am convinced before I am dead, advances in robotics and fabrication will simply the process (or use home equipment such as future laser printers for printing stencils). I'd love to spend my retirement fabricating my own CPUs :D

Edit:

Let me add: I don't mean the cutting edge process node. I mean the kind of process node that was used to make the very first chips (but less toxic, repeatable, cheaper equipment). If it is possible for synthetic biology, it must be doable for semicon :D

Very insightful!! A 175B parameter model with 2 bytes per weight, and say 2 bytes per gradient (not sure if single precision gradients makes sense?) comes in at 700GB, which is beyond a single 8x80GB beefy machine!! I recall reading with tech such as RDMA, you can communicate really fast between machines .. I assume if you add a switch in there, you are toast (from a latency perspective). Perhaps using 2 such beefy machines in a pair would do the trick .. after all .. model weights aren't the only thing that needs to be on the GPU.

I saw a reference that said GPT-3, with 96 decoder layers, was trained on a 400 GPU cluster, so that seems like the ballpark for a 175B parameter model. That's 50 of the hypothetical machines we talked about (well .. really 100 for GPT-3 since back in those days, max was 40 or 48 GB per GPU).

I also wonder why NVIDIA (or Cerebras) isn't beefing up GPU memory. If someone sold a 1TB GPU, they could charge a 100grand easy. As I understood it, NVIDIA's GPU memory is just HBM-6 .. so they'd make a profit?

If you want simple data, lets look at the price of bottled water or soda at an airport (I usually bring an empty with me but not the point). It is priced the freakin same across every eatery. No collusion?

I'm convinced some people have a special gift in terms of # of hours of sleep required. Some famous people like Churchill and Thatcher just needed 4 or 5 hours of sleep. I envy such people. I am in my mid 40s and also awake at 2am but I need 8 hours of sleep. Powered by coffee at the moment :D

I only see one solution to right the wrongs of the past (with respect to the housing bubble). Tax wealth annually. Tax home sales the way the US does it.

As a society, we need to acknowledge how zero interest rates were directly responsible for the housing bubble. The people that caused that should be shamed publicly.

I would have gladly had more kids if I could have afforded a place to put them. Do you know what a 3 bed condo costs in Toronto? It is financial suicide.

I'm an immigrant and got really burned during the Harper admin (Kenny's handling of immigration destroyed my life). That basically meant my options are Liberal or NDP. NDP policies seem to think if I make more than 200K, I am a rich fat cat (ignore that I have a freakin PhD at the cost of a delay in buying a house - effectively making me far poorer in wealth terms than someone with a mediocre undergrad who bought a place anytime before 2015). Last election, I recall thinking, how can I vote NDP if them winning means I'll leave the country. That means, my only choice is Liberal. I am so fed up with their mismanagement, that I think I need to vote conservative just to make my voice heard, and then if (when?) the xenophobes of that party come out, go liberal again. I dunno .. feels like Canada isn't for people like me.

Working with someone professionally is not the same as a beer buddy. In fact, I value having colleagues who are distinct enough from me that I would feel a bit uncomfortable having them over/hanging out (e.g. political views, humor, world view), as long as they are 100% professional and technically excellent. Btw .. a lot of technically excellent people do not drink (religion or just choice), and a bunch of people would likely be classified on the ASD scale. Your colleagues DO NOT need to be your buddies.

I recall when I was in a team that was super buddy-buddy and everyone would hang out and go drinking, everyone shared the same view, etc. When we had a major crisis in the office, things devolved into a shouting match in a manner that may be okay with buddies but not in a professional environment.

I'll tell you one thing I got taught in management training that rung true to me, and had I known this earlier, would have saved a ton of grief. Your technical team will be gauging the candidates technical competence. You need to focus on other things .. most important being grit, ability to be a team player/collegiality and communication.

Thank you for your post. Two questions for you:

1) Did your Masters cover non-deep-learning vision (classical vision?) in sufficient detail? There is a ton of math in there. Going from being a shallow user of OpenCV to a deep one seems a big jump. I'm not sure a Masters focused solely on classical vision would get someone there (let alone one covering other things like ML, DL, etc.).

2) Did you end up training large models from scratch or is it all just fine-tuning? I am trying to do the former and I realize getting things to scale for from-scratch training is a whole other topic. I suspect getting things ready for inference would be similar.

Thx!

I switched to ML .. well .. Deep Learning in a big way in 2018. Switched from Cloud backend engineering/research(which I had been doing since its early days). I gotta say ..not the smartest move in hindsight. Peers doing cloud are now in very high positions as corps grew rapidly in that area and needed technical leadership. DL is very math intensive and it took me a while to get my sea legs. I finally feel pretty comfortable in my (technical) neck of the woods, and realize it is all going to get blown away soon by newer advances (LLMs, stable diffusion, NERFs).

I don't think my career is anywhere close to a rocket ship at the moment .. there is a chance that ML/DL gets a lot bigger in the next 2-3 years (e.g. Satya Nadella's recent talk circuits, that is the timeline he predicts). What this means for me is that I need to get exceptionally lucky to get on to the next rocket ship. I have no clue who that is, despite having a CV for it.

Not discouraging you but just sharing my journey. If anyone has any suggestions for what the rocket ship, please enlighten me.

I thought it was me getting old (in my 40s now, with little kids, life feels like shit). My thoughts:

1) Globalization caused massive changes to this world that have not been properly acknowledged. Sort of like none of the politicians cared about redistribution. In aggregate, the average person is better off, but there are significant structural impacts. A lot of people in the west, got hit badly (e.g. manufacturing). Conversely, a lot of people in Asia (specifically, China and India) have had spectacular improvements in standard of living.

2) Mobile & Internet: info travels at the speed of light. Along with general computing advances, this makes everyone more efficient, but simultaneously more starved for human contact and with little to no downtime.

3) We are still paying the price for the 2008 fiasco. Interest rates were down too long and inflation measures not accurate. House prices (and other asset prices) went up dramatically. This simultaneously increased wealth inequality (those with assets gained, those without got left behind), and made the so-called American dream harder to attain.

I was very romantic about democracy growing up in countries without it. Now that I am older, I can see how messed up the system is (even in the West). I remain optimistic that the Internet and computing will somehow improve things, though I don't exactly see how.

This is brilliant!! Thank you for sharing.

I had no idea who this guy is, and the talk series. How much field-relevant brilliant content am I missing?

People saying it is deferred PIPs or house cleaning are deluded. At Meta and Amazon I know some senior people who were laid off. People got decent severance so not too much negative comments, but I think it is a blood bath. 10% is not a small number and these are not boot camp employees who will switch out to different careers. It is a bonanza for smaller tech firms, but I think this is going to feel like the dot com bubble :(

I went to grad school during those days and have to say, it really sets you back. It will also depress wages in tech for the next 2-3 years.

Lowered spending a bit assuming I am already laid off. X-mas, black-friday did not help my budget. One major thing is we are omitting a fancy vacation in our plans this year. Had to explain this to my non-tech spouse but she got it, as others in our friend-circle are in the same boat. We may go somewhere via car or something.

Taking online courses in my spare time now, so when I really need it, I have some momentum. Passing that first assignment was tricky .. getting easier now (been out of school for ages).

Amount of LinkedIn pings I get from recruiters has definitely gone down. Still getting pings but these are companies I don't wish to work for.

There is some good advice here .. leetcode and general interview practice is a good idea, reducing unnecessary expenses, and networking. I am not doing great on all 3 fronts.

I have a family with little kids. It is a bit tough to lower expenses. My biggest fear is if my decent tech salary goes away, it will be hard to adjust to a new normal. The hedonic treadmill only speeds up. Need to have significant discipline to go the other direction.

Good luck all!