tveita

Personalized advertising! Personalized high-pressure sales! Personalized phishing! Personalized scamming! Personalized extortion! Personalized coercion!

The possibilities are as exciting as they are endless.

Before AI, could you call up 4.2 million numbers in an instant with responsive robo-agents? Could you add convincing details to your pitch based on behavioural information collected from over 100 data partners? Could you deliver a heartbreaking plea in the voice of each and every person's child?

We're not afraid to take advantage of the very latest in personalized advertising, and that's what's going to set our toothbrush subscription service apart from the rest.

There's also a cute display which I assume is much of the appeal.

The sophistication of the scanner seems a bit oversold at the moment.

So SpinQuant learns a rotation for activations and weights that, to my understanding, "smear" the outlier weights out so you don't get extreme values in any one weight.

Random anecdote warning - In the old days, before vector search became AI and everyone and their dog offered a vector database, I had a task that required nearest neighbour search in a decent amount of high-dimensional vectors.

I tried quantizing them to bit vectors in an index and scanning through it to get an initial set of candidates. Performance was actually quite decent - reading through RAM linearly is fast! But the selectivity wasn't great.

Somewhere along the way I found this paper[1] that iteratively finds a rotation to apply before quantization to reduce the quantization error. Very similar goal to SpinQuant, but focused on bit quantization only.

As it turns out the 'random rotation' baseline they benchmark against worked great for my use case, so I never tried implementing the fancier algorithm. But it's a pretty rare day at work that "apply a random rotation matrix to a 128-dimensional vector" is the solution to my problem.

[1] https://ieeexplore.ieee.org/abstract/document/6296665 / https://slazebni.cs.illinois.edu/publications/ITQ.pdf

A web of trust or reputation based system can be built on top of the signature scheme - maybe an appearance of smaller invite-only forums that share reputation. Or maybe it will just become an integrated part of the moderation of existing platforms like Reddit, Mastodon and Bluesky

If you put your name on AI spam people will flag your post and no-one will bother to see your posts for at least a few years.

(2016), this is still technically true, but "utf8mb4" is the default encoding since MySQL 8.0, and "utf8" is a deprecated alias for "utf8mb3", so this isn't nearly as much of a footgun.

I think Rama [1] (by Nathan Marz behind Apache Storm) is interesting as a "NoSQL" solution for a similar problem space, as I understand it. Impressive if this can support similar scale using only SQL.

[1] https://redplanetlabs.com/

That still looks like a more targeted approach than "precise strikes" on schools and humanitarian zones like in https://www.bbc.com/news/articles/cwyx9znxl4eo

Who programs the tapes? https://en.wikipedia.org/wiki/Profession_(novella)

You can handle missing values if you plan for it when you design the API. As the article says you can use the well-known optional types, or now mark fields as optional.

Protocol buffers include a set of rules to follow for backwards and forwards compatibility between servers and clients, I think these should be understood as part of the philosophy of protocol buffers. https://protobuf.dev/programming-guides/proto3/#updating

> Imagine building a data format for temperature measuring stations around that. Is it 0 degrees, or did the thermometer just skip reporting?

What does it mean for a thermometer to "skip reporting"? If that is something you intend to support you will need to define what it means and how the data should be read. Should a reader always check if the field is present? Should it crash?

What would typically happen is that you start out with a required field, because of course you will always have a temperature:

  required sint32 temperature = 1; // temperature in Fahrenheit

But soon enough you realize you need a richer format.

  required sint32 legacy_temperature = 1 [deprecated=true]; // temperature in Fahrenheit - DO NOT USE
  message Measurement { 
    optional Unit unit = 1;
    optional float value = 2;
  }
  optional Measurement temperature = 2;

- or you decide to allow submitting a batch of temperatures in each message, etc.

You configure new measurement stations to set both the old and the new format so they'll be compatible with old servers - maybe your customers are running servers on-prem. You mark legacy_temperature as optional in your server code to prepare for eventual deprecation (luckily in this case the server never sends temperatures back to measuring stations)

Finally all the customers have upgrade to the new server, and you ship new stations that don't set the previously "required" field. New measurement stations don't show up in the UI. A batch job that doesn't even look at the temperature field was compiled with the old definitions and is now crashing.

General text embeddings aren't that great for keyword search. The best embedding match for a paragraph is going to be another paragraph with the same meaning. "Cats are natural carnivores." should be very similar to "Felines eat meat.", and somewhat similar to "Tigers eat meat."

"cat" is not a sentence - the embedding should be close to other contextless single words, and maybe just slightly closer to paragraphs about cats than paragraphs about non-cats.

A simple way to use embeddings for search is to generate text of the same kind you expect your users to input - e.g. if you expect users to ask questions in natural language, have a cheap LLM turn paragraphs like "Cats are natural carnivores." into questions like "What do cats eat?" "What is natural behaviour for cats?" and then index the embeddings for those.

Even a few years ago it would be perfectly possible to do audio based content ID on known content to uncover viewing preferences on streaming media - a proven business model, ref https://samsungads.events/acrguide-pr

These days you could easily run Whisper locally on each cable box and report back whatever information advertisers will pay for. It will run a bit hot, but what do they care if they waste your electricity.

An unlocked bootloader is the only way to take a local backup of app data now, afaik, since they've made the adb backup tool more and more useless - and phone transfers too.

So you can either have no banking app, lose all your data when your Pixel glitches out, or (recommended option) upload all your data to the cloud for the voyeurs at Google to perv at.

Apple is leaning into this for cloud AI: https://security.apple.com/blog/private-cloud-compute/

Probably enough to be genuinely hard for an APT to backdoor even with insider help. Still, ultimately the security relies on Apple hardware signed with Apple keys.

Signal users should be happy about this, because it exposes the SGX enclaves as the false security it is. It's not like this is the first SGX exploit.

You should assume that Signal engineers, if they chose to, could access the user data protected by SGX, just like they could log metadata about your message sending and receiving patterns. You only have their word that they don't. The NSA could certainly bypass the SGX given access to the server.

I suppose there may be some legal benefit in putting the data "out of reach" - it would be hard to prove in court that you were capable of leveraging an exploit towards SGX to provide the requested data. But NSA/others will happily take possession of the hardware and do it themselves.

The same authors also have a language model at https://github.com/Zyphra/Zamba2 but it's not clear to me if that model is connected to tree attention.

The announcement at https://www.zyphra.com/post/zamba2-small links to this paper, but the paper doesn't actually mention Zamba2 anywhere.

Same with the Lusophon Mathematical Olympiad 2023 example.

It makes a bunch of specious claims about parity. (Adding 4 to a number changes the parity? Therefore, the parity of each colour always changes for each turn? Therefore, the parity will always be the same as it was initially?) And then it concludes that since the parity is right, 2022 of each colour must be a reachable state.

Which, as you say, is quite possible the correct answer, but it's really weird to put it out there as an example with no comment on the reasoning.

The text of Arthur Conan Doyle's short story for the library is available online

https://www.arthur-conan-doyle.com/index.php/How_Watson_Lear...

The combat reminds me of https://en.wikipedia.org/wiki/The_Typing_of_the_Dead, a touch-typing game that plays the cheesy zombie genre oddly straight.

Maybe this can save you by accident but it feels too fragile to rely on for security. For in-house code you might know every compiler and CPU you will run on now and in the future. For open source, are you comfortable releasing code with silent assumptions that turns into a security hole if someone runs it on an AVR.

A good old fashioned AI approach, e.g. compare with Peter Norvig's https://github.com/norvig/paip-lisp/blob/main/lisp/simple.li...

> A third, quieter feeling - if the universe was to communicate some message, who’s to say it would not choose this as a conduit?

Look away before you feel compelled to write your own operating system - maybe "SacredOS"?

On the one hand, yes, of course, and on the other hand on articles about "double employment" you'll see comments basically arguing that the company deserves it if your manager isn't micromanaging you and tracking metrics enough to spot that you're phoning it in.

So your business metrics suddenly dropped, but what has changed?

This service is using 80% CPU, that seems a bit high... but is it always this high? Looks like it spiked within the last hour. But wait, it does that every Monday at 9 am, so probably a red herring.

This cache has a hit ratio of 60%... is that good? A bit low? Actually it's suspiciously high compared to last week - looks like a lot of people aren't getting a personalised feed.

Metrics are incredibly cheap to keep around for the value you get from a good operational dashboard, despite what Datadog/Amazon/Grafana Cloud tells you. It's just the most egregiously overpriced data you can buy since 20 cent text messages.

A good start is to set up VictoriaMetrics with some collectors and set retention to 14 days.

It mushes all the concepts together into very recognizable AI soup.

Let's see how it does with Native American

https://stockcake.com/i/historical-handshake-moment_626739_3...

https://stockcake.com/i/historical-handshake-moment_626741_3...

These could cause some upset... If you have to do this maybe at least avoid publishing anything tagged "History"?

https://stockcake.com/i/reindeer-caravan-march_309941_464989

Northern natives all have horns, right?

Notably that was a "no-one-but-us" backdoor, that requires a specific secret key to exploit. We'll see when someone analyzes the payload further, but presumably this backdoor also triggers on a specific private key. If not there are ways to do it that would look far more like an innocent mistake, like a logic bug or failed bounds check.

I can see some arguments that might persuade the NSA to run an attack like this

  - gathers real world data on detection of supply attacks
  - serves as a wake-up call for a software community that has grown complacent on the security impact of dependencies
  - in the worst case, if no one finds it then hey, free backdoor

For an idea of what's possible you might be interested in this story that was just on HN where they fine tune a quantized 70b model in 48GB VRAM: https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html

At one point the official App Store guidelines outright included a threat to not complain publicly about Apple's policies, because why not.

"If your app is rejected, we have a Review Board that you can appeal to. If you run to the press and trash us, it never helps."

https://www.sec.gov/Archives/edgar/data/1580732/000119312514...

They may have removed the text but there is still the silent understanding that they hold grudges.

It's incredibly petty and other companies should take lesson. Oil companies should refuse to sell you gas if you talk about global warming. Musk should disable your Tesla if you own him on Twitter.

Some "didn't make it in" things that looked promising were the perceptual vector quantization[1], and a butterfly transform that Monty was working on, IIRC as an occasional spectator to the process.

[1] https://jmvalin.ca/daala/pvq_demo/

Right, you read something nebulous about how "the LLM combines the retrieved words and its own response to the query into a final answer it presents to the user", and you think there is some magic going on, and then you click one link deeper and read at https://ai.meta.com/blog/retrieval-augmented-generation-stre... :

> Given the prompt “When did the first mammal appear on Earth?” for instance, RAG might surface documents for “Mammal,” “History of Earth,” and “Evolution of Mammals.” These supporting documents are then concatenated as context with the original input and fed to the [...] model

Finding the relevant context to put in the prompt is a search problem, nearest neighbour search on embeddings is one basic way to do it but the singular focus on "vector databases" is a bit of hype phenomenon IMO - a real world product should factor a lot more than just pure textual content into the relevancy score. Or is your personal AI assistant going to treat emails from yesterday as equally relevant as emails from a year ago?

RAG doesn't require much data or involve any training, it is a fancy name for "automatically paste some relevant context into the prompt"

Basically if you have a database of three emails and ask when Biff wanted to meet for lunch, a RAG system would select the most relevant email based on any kind of search - embeddings are most fashionable, and create a prompt like

"""Given this document: <your email>, answer the question "When does Biff want to meet for lunch?"""

Another dubious optimization you could do is finding an optimal transport solution to move the apples the least amount of distance, something similar to the Earth Mover's Distance.

https://jeremykun.com/2018/03/05/earthmover-distance/