sabhiram

If frontier labs can distill the internet and all of our data, then we should be able to distill their models further too.

The fact that billions were spent on research to distill the internet should not preclude others from spending 10s of thousands to do the same to these frontier labs.

Time to create a bigger moat than "but we spent so much money doing this ...".

Sama and OpenAI, I am waiting on my data bundle to become available so I can delete my account. This has taken more than 48 hours - either you are getting hammered on deletion requests, or as usual you are playing games hoping I forget. I won't. People won't.

Yep, there is absolutely no problem with that at all.

Never imagined politics so obviously manipulating the talking heads with nary a care about perception.

Between vi(m) and VSCode - sublime has grown increasingly less useful in my day to day life. Used to use it exclusively, but it has lately been squeezed on both ends due to increasing server side development needs.

The grapes are sour because their moat is crumbling.

What was supposed to be a model, training, and data moat - is now reduced to operational cost, which they are not terribly efficient for.

OpenAI has been on a journey to burn as much $ as possible to get as far ahead on those three moats, to the point where decreasing TCO for them on inference was not even relevant - "who cares if you save me 20% of costs when I can raise on a 150b pre money value?".

Well, with their moats disappearing, they will have no choice but to compete on inference cost like everyone else.

Log space is nice, multiplication can be replaced by addition.

This part is easy and anyone can implement hardware to do this. The tricky bit is always the staying in log space while doing accumulations, especially ones across a large range.

We are talking about it aren't we?

Muni style. People crapping all over it from time to time.

The point above regarding convolution had to do specifically with accelerating 3x3 and above convolutional operations, as the product and the accumulation can be done in a few clock cycles if setup with care and love.

Fascinating paper.

We design an inference accelerator which more or less accomplishes this by quantizing input tensors into logarithmic space. This allows the multiplication (in convolution especially), to be optimized into very simple adders. This (and a few other tricks) has a very dramatic impact on how much compute density we achieve while keeping power very low. We keep the tensors in our quantized space throughout the layers of the network and convert the outputs as required on the way out of the ASIC.

We achieve impressive task level performance, but this requires some specialized training and model optimizations.

Very cool to see ideas like this propagate more into the mainstream.

No moat like a hardware moat.

Depending on the application, you could literally build software in the open and still maintain if not expand exposure.

Yikes, no! But hit me up here: shaba@recogni.com.

AXI primarily.

But, I should also mention, we only use the RISC-V cores as a pre/post-processor, scheduling engine, service processor. We have custom hardware that does the bulk of the inference math (we are a convolutional accelerator with a number of constraints traded off for speed and power). The fabric itself is driven by engines that are programmed by the scheduling engine (RISC-V).

Happy to answer more specific DMs.

We run a cluster of RISC-V CPUs based on open source designs to schedule and post process workloads for our edge accelerator ASIC.

10/10 would do it again, except this time we may pay SiFive or someone like that for something requiring less "customization".

Get ready for downvotes from metamate haters, but if the project is cool and compensation is not on your mind - it is a win!

need a revisit, that seems appropriate and deep

What this "kid" has done in his garage would still be impressive with more than modest support from the venture community.

Well, as opposed to hearing "oops our black box implementation of your information got hacked", I honestly don't mind the trend of "read the contract, it is code". Sure it can be misleading, sure it can be intended to trick someone. However, code is law, and even backdoors are "code". Instead we should fix the backdoors so that code can be reasonable "law".

It pays for engineers to be able to inflate and justify their worth, especially when moving ships.

It is not about everyone getting equal pay. It is about everyone having equal visibility. The former creates unmotivated employees, the latter creates one of two outcomes: 1. Satisfaction of relative comp for the amount of work done -or- 2. Understanding of what is valuable in this venture; how to spend your time

Often times in larger establishments (after playing enough politics to gain visibility), I found myself in camp (2). What you think is important is seldom actually important to the powers that be. If you find yourself disagreeing with the last statement, go hug your management team / coworkers!

Anyone who has boiled water understands the difference between the working enough to make steam vs working enough to keep the water hot.

Regardless of how much this made, the OP sold a business and crossed a chasm many find very uncomfortable to cross. Congrats on breaking past the inertia to ship, and then sell; it is truly commendable regardless of the naysayers.

One of those, "play a little and then use it to play other things".

Now I want to build a shed.

Super interesting. Been reading some Alan Watts lately, and he points out that while you go to "work", you "play" the piano, you don't "work" the piano. You don't "work" basketball, while still expending effort and incurring risk!

Saying "I want to build a shed", is like saying "I want to lose 10 pounds", it is a way to achieve a measurable goal. One where it is easy to see your progress, humans like the constant reward (or reminder).

We live in a world where even if the outcome is measurable, often times, it is not in our control. A shed, the gym, other such "hobbies" are tangible goals with indicators along the way where our influence of control intersects the measurable progress we see. These are essential to sanity, methinks.

I spend a lot of time designing the thing I am trying to build to be spread-out vs built-on-top. This way, I have a big list of fun and interesting problems to solve, after which comes the gluing them back together for the prestige.

In reality, it never quite works out that way - but I still maintain it is a good way to start. Like many have said, it is essential to identify the path by which you multiply engineering as demanded by the needs of the thing you are building.

Many projects can keep that factor to 1, but it has to be about the love of doing it, not the outcome. The minute the outcome is more important, hire, scale and delegate!

This looks like a very fun thing to get into, and I love the idea of being able to help in this new (for me) field. However, thumbing through the issues listed in the GH repo leaves me realizing that there is a bunch of terminology that I would need to catch up on (15% of the stuff feels familiar). Any pointers on what stuff I could read / study that would serve as a nice primer into the terms and ideas being coded and modeled?

My mom would read this with me back in my wee years. I still find myself stopping to think to its words from time to time, especially when I am about to lose my calm.

Happy new year to all the awesome moms out there, especially my own :)

Sounds like someone has been listening to Alan Watts :)

Well, the Rock did just announce that he is interested in politics.

Kidding aside, he would likely make a much better leader than the current self indulgent noise.

Seems useful, the name gave me a chuckle. If I only saw the URL, I would imagine this was a service that providing info on preserving your buttocks.