Hacker News Reader

machdiamondson Tesla created secret team to suppress thousands of driving range complaints

It's very easy to see how favorably or unfavorably Tesla's claimed range compares to competitors based on independent tests of multiple EVs in the same conditions:

https://www.youtube.com/watch?v=6LWL90paufE

https://www.youtube.com/watch?v=ynCaTDR4rDQ

https://www.youtube.com/watch?v=eFB6hsYXDiA

https://www.youtube.com/watch?v=fvwOa7TCd1E

Spoiler alert: Tesla models fare about as well, if not better, than their EV cousins, hitting around 80% of the stated range in the wild.

No matter how much you guys say you hate Tesla, it doesn't change the fact what the best selling car in the world is.

machdiamondson Dalle-3 Sneak Peak [video]

The July version of the model is much more impressive than the May version and it shows up at around 7 minutes into the video.

machdiamondson Microsoft Owns the Trademark for “X”

I think I'm going to trust Karpathy on this one over random HN commenters, but that's just me:

"Elon also understands deep neural nets a lot more than I think people imagine. He starts with good intuitions and mental models, but also actively asks for technical deep dives, and has very good retention. E.g. I recall teaching him about our use of focal loss in contrast to binary cross-entropy for the object detection neural net (I said it had given us a 5% bump and he asked to know more) and he understood how it works about as quickly as you'd expect a PhD student to. The fact that he can do this across many technical disciplines is impressive and borderline superhuman. I don't think people understand or would believe how low-level and technical typical meetings with him are. Just saying because I get triggered reading way off innacurate takes on this topic (original comment)."

-https://news.ycombinator.com/item?id=33703617

machdiamondson Microsoft Owns the Trademark for “X”

You are in good company: https://www.reddit.com/r/SpaceXLounge/comments/k1e0ta/eviden...

machdiamondson GPT Bots in RuneScape Are Reconstructing Player Experiences

Video link: https://www.youtube.com/watch?v=RbTsHEPMQoo

machdiamondson xAI

- You've mentioned that Musk isn't good with cars, yet under his leadership, Tesla has emerged as the world's most valuable automaker. This seemingly contradicts the notion of Musk being inept in the automotive industry.

- Your claim about Tesla's autonomous vehicles lagging behind competition appears misinformed. From what I understand, Tesla's self-driving technology, which does not rely on high-definition maps, is far advanced compared to most companies in the same space. Moreover, the self-driving team, led by Karpathy and now Ashok, are highly competent individuals.

- To suggest that SpaceX hasn't been successful or valuable to NASA is just wrong. You label the Starship project as 'dumb,' but isn't it a more cost-effective way to launch larger payloads into space? I'm curious if you had similar thoughts when the Falcon 9 was announced? Today, it delivers over double the tons to orbit compared to all other organizations combined.

- Regarding the open letter signed by SpaceX employees, less than 3% of the workforce supported it. The letter primarily critiqued Musk's Twitter activity, which is unrelated to his employees' professional roles and contributions.

- On the subject of the promised 'Tesla bots,' they appear to be progressing at a considerable pace, as evidenced by the following link: https://www.youtube.com/watch?v=XiQkeWOFwmk.

- With Neuralink, the ultimate objective is to integrate human brains with AI. More immediate goals include improving the quality of life for individuals suffering from conditions such as blindness or paralysis.

- Assessments of Musk's technical and engineering aptitude appear more convincing when coming from sources like this: https://www.reddit.com/r/SpaceXLounge/comments/k1e0ta/eviden...

I sense a bit of humor in your comment, but I still wanted to clear up a few things. These inconsistencies make me think twice about the other points you're making.

machdiamondson xAI

You think SpaceX, Tesla, etc. All succeeded because people weren't allowed to disagree with him?

machdiamondson xAI

There's lots of negative things about him on Twitter right now

machdiamondson xAI

I would bet on this not happening.

machdiamondson Giving LLM’s a <Backspace> Token

It seems that this problem has been somewhat tackled already (Apologies for the sensationalized video title): https://youtu.be/SCfCuLy4RJA?t=298

machdiamondson GPT4 is 8 x 220B params = 1.7T params

From my perspective, there's a vast divide between open source models and GPT4 at present. The lmsys leaderboard rankings are derived from users independently engaging with the LLMs and opting for the answers they find most appealing. Consequently, the rankings are influenced not only by the type of questions users ask but also by their preference for succinctness in responses. When we venture into the realm of more complex tasks, such as code generation, GPT4 unequivocally outshines all open source models.

That said, the advancements in models like Orca and the "Textbooks are all you need" paper are noteworthy (https://arxiv.org/pdf/2306.02707.pdf, https://arxiv.org/abs/2306.11644). I'm optimistic about what future smaller models could achieve.

machdiamondson GPT4 is 8 x 220B params = 1.7T params

https://twitter.com/soumithchintala/status/16712671501017210...

It looks like at least one other person has also heard the same information.

machdiamondson The Secret Sauce behind 100K context window in LLMs: all tricks in one place

I've been wondering about this, as simply extending the context window in a straightforward manner would lead to a significant increase in computational resources. I've had the opportunity to experiment with Anthropics' 100k model, and it's evident that they're employing some clever techniques to make it work, albeit with some imperfections. One interesting observation is that their prompt guide recommends placing instructions after the reference text when inputting lengthy text bodies. I noticed that the model often disregarded the instructions if placed beforehand. It's clear that the model doesn't allocate the same level of "attention" to all parts of the input across the entire context window.

Moreover, the inability to cache transformers makes the use of large context windows quite costly, as all previous messages must be sent with each call. In this context, the RWKV-LM project on GitHub (https://github.com/BlinkDL/RWKV-LM) might offer a solution. They claim to achieve performance comparable to transformers using an RNN, which could potentially handle a 100-page document and cache it, thereby eliminating the need to process the entire document with each subsequent query. However, I suspect RWKV might fall short in handling complex tasks that require maintaining multiple variables in memory, such as mathematical computations, but it should suffice for many scenarios.

On a related note, I believe Anthropics' Claude is somewhat underappreciated. In some instances, it outperforms GPT4, and I'd rank it somewhere between GPT4 and Bard overall.

machdiamondson NHTSA tells automakers not to comply with Massachusetts right-to-repair law

"It's so difficult for legacy car companies to get software right. You'd be surprised. Let me explain it quickly. To save probably 500 per vehicle, or let's say 350 quid a vehicle, we farmed out all the modules that control the vehicles to our suppliers because we could bid them against each other. So, Bosch would do the body control module, someone else would do the seat control module, someone else would do the engine control module, right? And we have about 150 of these modules with semiconductors all through the car. The problem is, the software is all written by 150 different companies, and they don't talk to each other. So, even though it says Ford on the front, I actually have to go to Bosch to get permission to change their seat control software. So, even if I had a high-speed modem in the vehicle and had the ability to write their software, it's actually their IP. We have 150, what we call, a loose confederation of software providers. 150 completely different software programming languages. All the structure of the software is different. It's millions of lines of code, and we can't even understand it all. That's why, at Ford, we've decided in the second-generation product to completely insource electric architecture. To do that, you need to write all the software yourself. But just remember, car companies have never written software like this. They've never written software. So, we're literally writing how the vehicle operates, the software to operate the vehicle, for the first time ever."

- Jim Farley

https://youtu.be/8IhSWsQlaG8?t=1476

machdiamondson The SEO arms race has left Google and the web drowning in garbage text

Ironically, LLMs could be a game-changer in dealing with SEO noise, not at the search stage, but during the indexing of web pages. By understanding context and semantics, they could help create a more refined and relevant index, effectively cutting through the clutter of SEO-optimized content.

machdiamondson Tell HN: Google search sucks even more during Reddit blackout

machdiamondson Reddit CEO tells employees that subreddit blackout ‘will pass’

There was an idea floating around here about users downvoting en masse as a form of protest. While this could certainly disrupt the algorithm and content quality, it wouldn't require a massive participation. However, identifying and banning accounts engaged in systematic downvoting would be quite straightforward. A more subversive approach could be to invert the system - upvote low quality posts and downvote high quality ones.

machdiamondson A developer's view of Vision Pro

Hi, curious about the foveated rendering on this headset. Theoretically, it should offer a 10x+ performance boost, but headsets like Quest Pro and PSVR haven’t achieved much with it. How does this compare?

machdiamondson A developer's view of Vision Pro

Norm from Tested, a highly credible source for VR, actually mentions the FOV is akin to Valve Index's: https://youtu.be/f0HBzePUmZ0?t=825

This is actually a bit disappointing since I would rather not have to move my entire head to look at a virtual side monitor. It seems like the technology is there now and many companies are looking into it, hopefully other headsets will be released with higher FOVs: https://youtu.be/y054OEP3qck?t=283

Another noteworthy point from that video: Apple has bought Limbak, an optics specialist that was formerly tied with Lynx R1's optics. This means Lynx can no longer use Limbak's future optics that, admittedly, fall short in scaling to higher resolutions. Now, Lynx has shifted its gaze to hypervision optics, intent on preventing a similar acquisition by another tech behemoth like Apple.

machdiamondson Reddit Strike Has Started

As a user, I think Twitter is better now, notably with the Community Notes feature that even Zuckerberg commended on Lex's podcast - I hope he introduces a similar concept. The option to post extended threads is appreciated. I hardly come across political content as I avoid engagement with it, and I find it beneficial that creating an account is no longer necessary for viewing Tweets and responses.

machdiamondson Mark Zuckerberg on Apple’s Vision Pro headset

I agree, the prospects for cloud gaming do seem promising. However, we can't forget that in VR, latency is a much more important factor. One solution could be for companies to establish datacenters near high-population areas to guarantee low latency. But, I believe a more effective approach might involve advanced foveated rendering coupled with technologies like DLSS. At present, it seems like a viable strategy for both Apple and Meta would be to sell "boxes" equipped with console-grade hardware that can wirelessly connect to their headsets, providing additional compute.

machdiamondson GPT-4 “discovered” the same sorting algorithm as AlphaDev by removing “mov S P”

No worries, dang! Appreciate your efforts in upholding the standard of conversations here.

machdiamondson Desktop GPU Sales Lowest in Decades: Report

I think the used 3090s on Ebay, going for about $700, are a decent deal, especially since they've 24GB of VRAM.

machdiamondson Mark Zuckerberg on Apple’s Vision Pro headset

The best gaming headset on the market would have to be able to connect to a Windows PC. The M2 chip in the headset doesn't have the power to run an intensive game at 4K resolution for each eye. Even if the headset could be tethered to a Mac for additional power, the GPUs in Macs don't come close to the capabilities of the 4090, which itself struggles to support 2K per eye for demanding games. One potential solution could be foveated rendering, which has shown some performance improvements on the Quest Pro and PSVR2, albeit not significantly. However, it's possible that Apple's implementation of this might yield better results.

machdiamondson Mark Zuckerberg on Apple’s Vision Pro headset

The Quest Pro already has eye tracking and even the Quest 2 has good hand tracking now: https://www.youtube.com/watch?v=Io7_NyTmXv0

But yes, they really do need to improve the resolution, It seems that the resolution was kept similar to that of the Quest 2 due to using the same SOC, which couldn't power a display with a very high resolution.

machdiamondson Mark Zuckerberg on Apple’s Vision Pro headset

I'm guessing that the Vision Pro will actually help Quest sales in the short term, since many people will want the Vision Pro but will settle for the Quest 3, since it'll be 7 times cheaper.

machdiamondson GPT-4 “discovered” the same sorting algorithm as AlphaDev by removing “mov S P”

This thread is someone using GPT-4 to re-discover the optimization made by Deepmind in that thread.

machdiamondson Bard is getting better at logic and reasoning

I think Google still has a decent chance of catching up. It's just a bit surprising to see them fall behind in an area they were supposed to be leading, especially since they wrote the paper which started all of this. Also, Anthropic is already kind of close to OpenAI, so I don't think OpenAI has some magic that no one else can figure out. In the future, I predict that these LLMs will become a commodity, and most of the available models will work for most tasks, so people will just choose the cheapest ones.

machdiamondson Bard is getting better at logic and reasoning

If Google only used data that isn't copyrighted, they'd probably make a big deal about it, just like Adobe does with their Firefly model. Also, it's not really possible for OpenAI to just take out certain parts from the model without retraining the whole thing. The drop in quality might be due to attempts to make the model work faster through quantization and additional fine-tuning with RLHF to curb unwanted behavior.