leecarraher

BLAS is just the library definition and not the implementation, so BLAS implementations could implement GEMM anyway they want. But in practice the triple loop method (n^3) is the most common, despite Strassen's and the more numerically stable, Winograd methods being well known and available for decades. But with most things involving real computing hardware, memory access patterns and locality tend to be more important for performance than operation counts

is there a difference, legally, regarding vehicular involuntary manslaughter, between hitting someone who is jaywalking and someone who is not. Example, a person walks into a 65 mph thoroughfare after a curve, since they are no longer in the commencement of a crime, does that make the act tantamount to hitting someone in a crosswalk?

steve gibson of grc apparently uses sound as a second input channel to handle various tasks. as i recall he made an app decades ago called wizmo that allows you to script sounds, and as listeners probably recall, yabba-dabba-doo is one such audio channel input to let him know someone has made a purchase order on his website.

This is exactly why i like this game. Upon first playing it, the attacks were obvious to beat anyone who was playing legitimately. Assuming both teams know this fact, means the choice is, play the game to win and it is boring or play the game to have fun and challenge yourself and partner. During every occasion, the latter has prevailed.

Was the article written by an llm difference engine? In that it failed at discussing the differences of the two and really just regurgitated the etymologies and ancient definitions of the two terms with no insight about modern usages. My quick difference as a matter of opinion only is that engine is related to carnot some heat cycle creates enables work that can be turned into locomotion. Motor is a superset, and is some system, process or machine that turns potential energy into kinetic energy. That is my opinion, of which i am open to, in fact very interested in reading others. Unfortunately this was not that article, instead you get a brief history lessons on words, with the thesis that some word's meanings change over time (allow me to pick my jaw off the floor) and don't worry about delineating the two terms because they pretty much mean the same thing. Not what i was expecting from mit engineering.

but how could you falsify all those records so quickly and convincingly, oh nm

it seems the primary benefit for sand over water, is a 1:10 operating temp vs. 5:1 specific heat. So it depends on whether the added complexity of working with a hotter, solid is worth not having to build a facility that is 2x bigger. Are there other benefits I'm missing, or is this concrete block gravity storage vs pumped water storage, all over again?

additionally trees have fewer, often computably optimal parameters, whereas DNN often require extensive hyperparameter tuning and even neural architecture search(how many rows, what activations, what optimization method...). Furthermore, trees are generally more interpretable. There have been some recent interesting papers relating random forest trees to adaptive smoothers, that tries to understand why they beat dnns on tabular data.

create bindings and externalize function libraries for other languages, hope to your prefered deity nothing breaks

no semilog/log plots, heatmaps, contour maps, violin charts, eye diagrams. No love for the engineers i guess

periodic table of finite simple groups is pretty awesome https://www.reddit.com/r/math/comments/99brzy/the_periodic_t...

they're groups so symmetries are kinda what they do. There are families etc. in columns, with the final column being the generator. Then the sporadic groups are the rare earth elements which don't define a family of groups. order is increasing on the way down, and the very obvious cyclic groups are the noble gases equivalent.

transforming the recurrence to a matrix then applying the matrix as a transformation of itself can be expressed as multiplication. but AAAAAA doesnt really save you much to get the 6th fibonnaci number. the power is in powers of the matrix. Now you can say AA = A^2 , and A^2A^2 = A^4 still not computationally helpful, but A^4A^4 = A^8 A^8*A^8 = A^16 ...32 ...64 and then you've got something useful.

I wish there were more explanation for why this work. There is a proof but a more opinionated description would also help. My summary of it is that parallel prefix sum is work efficient for computing general recurrences of the form xi = x_i-1 + yi. Furthermore, a product of elements can be expressed as a sum by taking the sum of the log of elements and exponentiated the result. These are two super useful properties, but merging the two operations to this general formulation is not clear. Is it simply sum the logs, and the addends via prefix sums, and then sum those two results? That seems pretty straightforward, so I suspect I am missing something that makes the solution less obvious and more complexity. Is there some fancy fixed point combinator, or solution to a differential equation of the linear system for the recurrence equation?

Unfortunately independent trucking companies are already stretched pretty thin, while we all benefit from these transportation infrastructures. perhaps a flat tax is just a way to distribute the burden. Ff course this is not the case for large logistic companies with massive distribution networks who regularly commit gas tax fraud.

good for them, they totally could really use that extra cash flow from folks who can't afford to pay their bills

> Right. However, this is a comparison versus Python and the GIL, which can’t do that at all.

Single process python does not take advantage of a multicore architecture but neither would single process mojo. Embarrassingly parallel operations like mandlebrot can trivially be written with multiprocessing (https://github.com/DipanshuSehjal/Mandelbrot-set/blob/master...), or joblib to run in parallel in otherwise vanilla python. It would be trivial to implement this in jax and run on a gpu or tpu, but i wouldn't say that jax is the reason for the speed up.

that is good to hear. I read a post on Mojo months ago, signed up to the waitlist and then crickets. It would seem insane to think a non-open source, non-free compiler/interpreter could be successful these days.

35Kx speedup is not scaled speedup. Throw this, naively parallelizable task at a bigger computer and get 70kx speedup, etc.

While i think there are tons of optimizations to be done for python (looking at you GIL) giving access to low level cpu primitives is not one I think that will be broadly adopted by the python community. That's one of the joys of python: system agnostic, looks pretty close to pseudocode, coding. If you want speed, glue together a bunch of compiled code calls, and hope the call overhead isn't too large. Or write cpu intensive operations in numba, or pyrex. At the end of the day, mojo's pay to play programming language harkens back to the early 90's Borland days.

The author says to add a unity vector to the context, i presume of each layer, to not mess with gradient calculations. But most modern DL frameworks compute the gradient for you, (i know this is true for JAX and Pytorch). Is it maybe that hand coded gradient for a well-known enough dl architecture like transformer is faster than letting the framework autodiff it?

Otherwise, i fear some of the 'magic' of transformer networks is that this amplification effect allows it to encode/memorize some results verbatim. And we often are seeing a heavily tuned internet regurgitator. So similar to the rise of RNNs with attention, which supposedly allowed them to focus on some things and ignore others but really often was just overfitting stuff, yielded more interesting results with the overfitting than without.

I like the analogy at the end regarding nuclear power vs piping in water from freshwater sources. The upfront costs are high, but over time it would be cheaper than desalination, but due to short term governments and borders, it's hard to justify the upfront costs. So instead we are somewhat stuck with more expensive short term solutions.

Agreed, great article and figures. I'm guessing from the comments no one actually read through it.

i feel we need a steve gibson, "What could possibly go wrong..."

The title was originally 100% but the editor felt that to be too unbelievable and ratcheted it back to just greater than 99% , that way it covers 100, but looks fancier, there's a math equation in the title, it's gotta be legit!

if i could order the food from the digital menu, and pay, then I'd be on board as a nice value add. Otherwise it's just a cheap cost cutting measure that restaurants ignore or are ignorant of, the tracking implications of having a 3rd party aggregator know what food i like and where I am.

Rattlegram is an app i've used with Ham for some time to send audio based messages over analog audio. It also works well in air-gapped noisy rooms too, due to the excellent efficiency of COFDM (coded orthogonal frequency division multiplexing). Of course not many situations are actually like that, so more of a fun experiment, but the Ham use is actually pretty useful.

loopers role on a movie set - https://www.npr.org/2015/02/19/385237280/never-seen-and-some...

interesting, seems like it could take the job of a movie looper with very little additional tuning.

2013, so computers >> faster than human hearing so the following probably doesn't matter but devils advocate : memcpy puts stuff in the heap instead of the stack, which requires a malloc, a system call, which could introduce some latency. Then layer on that our audiophiles are probably listening to ridiculously high bit rate audio which results in large files, such that without swap buffering there may be some periodic buffer-play-buffer artifacts introduced. This is why you buffer but [new] is also a heap allocation, so same syscall, shouldn't be any different. Is it calloc? calloc in theory should be worse.

An entire article about compression being similar to what a DNN does without a mention of Naftali Tishby's Information Bottleneck principle for neural networks. https://en.wikipedia.org/wiki/Information_bottleneck_method

is there more context for: "Well, with the addition of the sparse matrix multiplication feature for Tensor Cores, my algorithm, or other sparse training algorithms, now actually provide speedups of up to 2x during training" 2x what? CPU, then it's hardly worth it. 2x per core, so #cores x 2. I'd imagine the level of sparsity is also important, but there is no mention of it.