pavelstoev

Very interesting. Well done authors!

Happy Juniper customer since 2010. Across SRX, EX, QFX lines. What to do now, what to do...

In the undergrad Control Systems course, I brute-forced Kalman Filter matrix to balance the inverted pendulum on a track experiment. Worked fine.

Say no to drugs.

There is still so much performance to gain by applying software-based optimizations for superior performance of transformer LLMs on GPUs. From ~ 30% GPU utilization to over 90% is possible. I wrote a white paper on this, page 5 and 6: http://tinyurl.com/uujr3z4f

(while nail biting)...we are back?

Looking forward to MLPerf results https://mlcommons.org/benchmarks/

Try Hidet - open source deep learning compiler written in Python, generating optimized CUDA https://pypi.org/project/hidet/

I agree. I am surprised that many folks, not you of course, think that is okay.

Not exactly idle but only at around 30% utilization on average (measured on a ~900 GPU cluster over ~25 days)

Agreed. I have 20+ Hyper-V Windows Server VMs running since 2017, seeing 400-600 Mbps live video traffic - no major crashes, only due to facility power loss or annual patching reboots.

Late in 2006, as a local engineering consultant working for MBTA, I designed and then physically installed OFDM receiver station on top of the Oak Grove (Orange Line) station. Google Street View still shows the antenna present.

Why do you single out Grammarly in the title of this post when other larger orgs are part of this. I have no relation to Grammarly and I appeal to your fairness. Perhaps “Large organizations making OAuth implementation mistakes” would be more appropriate.

You can finetune 40B falcon on 4 x A10 with compiler optimization technology from CentML. No changes to the model.

Imagine someone paid you 25c/hour for 4090 compute sharing.

The claim is likely not false just like a horse pulling a buggy is several orders of magnitude faster than a snail.

which is why we created Hidet, open-source deep learning compiler, written in Python and which can be easily extended to create all kinds of ML projects (wink, wink) with support for Nvidia GPUs. I am still looking for information on Mojo support of Nvidia GPUs, de-facto compute standard for modern AI/ML.

https://docs.hidet.org/stable/gallery/developer-guides/hidet...

I have a 12 year old and a 9 year old. They pretty much grew up during the smartphone age. We teach them to understand smartphone is not a toy but a gateway to information, knowledge and communication. When we use our personal smartphones around the kids - it is primarily for work purposes and that is the explanation. iOS implementation of screentime, when properly configured is great. It is like a real information firewall.

Humble self-promo here, may I also recommend the team at CentML who dedicated their academic life (PhD and above) to GPU optimizations for high-performance ML/AI to lower the costs.

I actually very much appreciate how adult it is.

To answer the question why there is not much competition, I would like to point out that the required expertise is very rare. Not specifically the chip design expertise but the expertise spanning both chip hardware and low-level software for such chip-specific hardware, that is all necessary to create the compilers, the frameworks and libraries and which can also plug-in well with the existing AI/ML ecosystem such as PyTorch.

Thanks for your comment ! Current focus of optimizations is for Nvidia GPUs but others are in the works. Hidet comes with Hidet.Script which abstracts some of the CUDA struggles and may make the ML optimizations efforts easier to implement. It is still evolving so documentation is limited but here are some examples: https://github.com/hidet-org/hidet/tree/main/python/hidet/gr...

Have you tired Hidet ? https://pypi.org/project/hidet/

They should talk to CentML.

When capacity or cost constrained (or both) consider running ML on “lesser” GPUs like A40, A10s and even consumer grade 3090/4090s. With compiler optimizations these uncover surprising price-performance ratios.

Would like to see updated benchmarks supporting the claimed performance gains in this project. Currently showing torch 1.12 which is rather weak compared to the latest 2.0 and torch.compile()

Removed at OpenAI's request – let's discuss what this means for transparency and open discourse in AI development.

Archived version of the post: https://web.archive.org/web/20230531203946/https://humanloop...

This content has been removed at the request of OpenAI.

Just when I went back to the post for some quote material...

Unfortunately I did not know him but I will surely follow now :)

I worked at RIM/BlackBerry in 2005-06. It was my first engineering job. I was in my early 20’s and at some point, way before iPhone, I went to my manager and suggested we should add a camera and some multimedia features to the device, because that’s what the younger people wanted and used. He said - but why, BlackBerry is all about serving the business people market. We argued. Turns out we were both right and wrong…