krackers

It seems plausible given that people have been using off the shelf 5.5 xhigh to decent success with some erdos problems. There is likely still some scaffolding around it though (like parallel sampling or separate verifier step) since it's not clear if you can just "one shot" problems like this.

Saving bytes on the wire?

>fine boo, but you need do something about it

Well they are doing something about it, just not the way the speakers had in mind.

It helped that he actually used and believed in the products he was pitching.

I don't think there is a difference today. Stocks used to be about long-term value but now the same gambling-esque mentality has applied to stocks as well, see things like GME. The modern stock market is basically a prediction market for attention/virality, in the same way that cryptocoins are.

>It's just a question because we don't have a good way of explaining how/why it occurs.

It's that you can't even measure it, since the way it's defined as a subjective experience, no external measure could ever capture it. This is what gives rise to the p-zombie argument.

To get rid of that you have to accept "functional qualia" as basically equivalent to qualia, which solves the p-zombie issue and resolves half of the hard problem. From there, explaining consciousness is no "harder" than explaining other scale-depedent phenomenon in complex systems like LLMs: still hard, but at least tractable with scientific measurements and experiments.

I don't think AI safety/psychosis/alignment issues are on most people's minds. All of the other ones are basically downstream of AI furthering wealth inequality. Job market for obvious reasons. Environmental concerns are mostly downstream of the fact that cities are suddenly accommodating data centers while they didn't care about promoting growth of infrastructure before, and citizens are asked to foot the electrical/water bills. (People would not care about environmental impacts as much if they weren't immediately impacted, say the DCs were built in Africa).

Also missed is the pushback against AI art: the further devaluation of talent, and an associated loss of meaning many people have. I think this is probably still downstream of it threatening jobs though, since people would not react as violently if they could truly treat art as a hobby instead of as a profession.

I personally feel "happiness" is more correlated with agency (or at least perceived agency), and in that measure civilization has been regressing since the industrial revolution. The amount of long-term planning required has increased and it's less possible to live "in the present", moment to moment.

It's also basically what was written about in that infamous manifesto.

How about "permanent underclass"?

There's a talk by Filippo that explains this nicely https://www.youtube.com/watch?v=0DV8WnqhH2Y

> which can be unwritable for a human

Unfortunate, those types of refactorings are my favorite, since they're tightly scoped, easy to verify correctness, and it's like a little puzzle. Bonus points if you write your own collector or use some more obscure parts of the stdlib.

> But how do you get 'Paris' into the value vector in that case?

Ok wait I think I see what you mean. Although maybe it's not getting paris _into_ the value vector that's hard, but isolating the residual stream to _only_ that instead of things like other capitals.

So as a naive example maybe at the very first layer consuming your tokens: Q{France} would have high inner product with K{capital} and so our residual would now mostly contain V{capital}, which maybe contains embeddings of all the capitals of all countries. You need some way to filter out all the other stuff, but can't do that without a FFN + activation.

Just throwing in a relu by itself won't help since that would still work on all the elements uniformly, you need some way to put weight on "paris" while suppressing the others, i.e. mixing within the residual stream itself.

Although maybe if you really stretch it, somewhere in a deeper layer you could have 1-hot encoded values with a "gain" coefficient so that when you do the residual addition it's something like {<paris>, <tokyo>, <dc>} + 10000*{<1>, <0>, <0>} and then if you softmax that you get something with most of its mass on "Paris". But it seems like this would not be practical, or it's just shifting the issue to how that the right 1-hot vector is chosen

I guess this had always been bugging me. I get while you need activation/non-linearities, but do you really need the FFN in Transformers? People say that without it you can't do "knowledge/fact" lookups, but you still have the Value part of the attention, and if your question is "what is the capital of france" the LLM could presumably extract out "paris" from the value vector during attention computation instead of needing the FFN for that. Deleting the FFN is probably way worse in terms of scaling laws or storing information, but is it an actual architectural dead-end (in the way that deleting activation layer clearly would be since it'd collapse everythig to a linear function).

If giving the customer more filter/searching power was something companies wanted, Amazon's search result page wouldn't be like visiting a flea market.

Funnily enough Fibonacci sequence is also matrix multiplication

Vector addition is just matrix multiplication in a homogeneous coordinate system, what's the problem?

Seems like the obvious confounding factor is just aging: Old people have problems with hearing, and old people are also less likely to walk briskly. The subset of people with undiagnosed hearing problems probably aren't taking care of their health in general.

> substantial amount of money from $99/year developer subscriptions

You actually do get some value, you can file two DTS tickets [1] a year which are (supposedly) looked at by a real apple engineer. Assuming they haven't outsourced it, that feels worth about $100 considering how badly documented their APIs are.

[1] https://developer.apple.com/support/technical/

I remember you used to be able to right-click and then press open instead of double-clicking which would bypass gatekeeper just for that run. Not sure if it still exists though, I don't have any unsigned apps handy to test.

Skimming the video there's also important unstated context that the person was non-white foreigner, had tattoos, and on visa. It's possible that the combination made an ambiguous grey-area situation much worse.

>In any technologically advanced society the individual’s fate must depend on decisions that he personally cannot influence to any great extent. A technological society cannot be broken down into small, autonomous communities, because production depends on the cooperation of very large numbers of people and machines. Such a society must be highly organized and decisions have to be made that affect very large numbers of people. When a decision affects, say, a million people, then each of the affected individuals has, on the average, only a one-millionth share in making the decision

I wonder if iCloud private relay might also work. Apple probably negotiated some special treatment

Half-serious thought: Would giving them an appropriately sized dose LSD (with proper setting/supervision) or similar thing be a better alternative? If the issue is lack of empathy for others isn't this a much better solution that actually fixes the root cause instead of papering things over. Maybe caning might fix the superficial symptom, but those people may well end up as sociopath CEOs or something or find other ways to gain satisfaction from asserting their power (just look at the state of the world, you can be a "bully" in many other ways than physical ones).

Note that as I understand the main claim of Marx is that the efficiency and productivity gains from automation don't actually go to the laborer, they're captured by the "capital owner". Example being how despite all the automation we're all still working 8 hr days, 5 days a week just to get by.

Now of course there's also jevon's "paradox" here, and the automation does allow us to support a larger population so in that sense not all the increased productivity is just "skimmed off the top" as profit. But on the flipside the crux of the other recent [1] HN post is that the wealth disparity is increasing. And if all the increased productivity directly translated to more "physical resources" in the world, that wouldn't be the case.

So something must be getting skimmed of the top, and intuitively you can feel the "rent seeking" layers in society have increased. Gains in efficiency are no longer resulting in surplus of physical products and decrease in prices.

[1] https://news.ycombinator.com/item?id=48038307

>The reason for this is that movies are graded with that EOTF in mind, so by linearizing with that EOTF

I guess this is the part I find anachronistic. Why do we work in the source scene light for photography, but do the opposite for videos? It makes sense if you assume the viewing device is "dumb" (like a television or CRT, especially in the analog days) but by now I assume the workflows are all fully digital, and even the most basic output device can apply LUTs. When digital video container formats were introduced, why didn't they align with what ICC did? It would have saved a lot of headache for everyone, compared to limited NCLC tags and the mess around EOTFs.

>despite similar ideas had been running in Google datacenters for idk how many years

I guess this is referring to https://www.youtube.com/watch?v=KXuZi9aeGTw ?

Sure but modern ICC color-managed workflows (e.g. digital photography, printing) basically don't distinguish between EOTF and OETF. Assuming your source is tagged correctly and all displays have a profile matching their true response, you necessarily need to linearize with the inverse of the encoding gamma. All edits are done directly in the source color space, if the viewer's profile differs from the source it's up the viewer to translate accordingly. You don't edit images "on the assumption" that they'll be decoded with a 2.2 gamma.

For some reason video workflows never adapted to the ICC system (probably because in CRT days you couldn't really adapt your decode gamma on the fly) which is basically where the whole debate in https://gitlab.freedesktop.org/wayland/wayland-protocols/-/m... comes from.

I'm not saying that the people using 2.2 EOTF are wrong, but all this just adds to the absurdity: in the modern day where LUTs are cheap and plentiful, instead of tagging content as an ambiguous sRGB it could simply be tagged as gamma 2.2 if it's actually intended to be decoded at that gamma.

For more hilarity, many people also believe the EOTF (decoding gamma) is supposed to intentionally differ from the OETF (encoding gamma). You can read an entire debate at https://gitlab.freedesktop.org/pq/color-and-hdr/-/work_items...

I'm reminded of this jane street puzzle https://news.ycombinator.com/item?id=47146487