snats

did a pretty similar thing last month for the text rendering library last month.

trained and made a viz for the model and then made it displace text.

should probably do a proper write-up:https://x.com/i/status/2038367016969724259

you can also have removed layers of models and keep the same score in benchmarks [1].

i feel that sometimes a lot of the layers might just be redundant and are not fully needed once a model is trained.

[1] https://snats.xyz/pages/articles/pruningg.html

personal website: snats.xyz weblog: weblog.snats.xyz

I also did a couple of experiments with pruning LLMs[1] using genetic algorithms and you can just keep removing a surprising amount of layers in big models before they start to have a stroke.

[1]https://snats.xyz/pages/articles/pruningg.html

i get it. i want one of those. the problem is that most cellphones are not actual cellphones, they are entertainment machines. they are a pocket tv / social media feed place. most usage for my normal friends is for that.

I am working on the https://moviemovie.club/about, it's a tiny website about film review.

It works like a run club, where you have to make a review first to see other people's reviews.

I am currently implementing watchlists, comments and a mural to make it feel a bit less lonely. Right now I like the UI but it feels to lonely.

Not an insider but imo the work on diffusion language models like LLaDA is really exciting. It's pretty obvious that LLMs are good but they are pretty slow. And in a world where people want agents you want a lot of the time something that might not be that smart but is capable of going really fast + searches fast. You only need to solve search in a specific domain for most agents. You don't need to solve the entire knowledge of human history in a single set of weights

It's pretty funny to test in-distribution for AI models. But they fail horribly once you push them a bit[1].

I recently made LLMs play Minesweeper and ALL LLMs that I tested had a pretty bad win to loose ratio. Like the only model that won more than 3 times was R1 (mind you there were 50 games).

[1] https://snats.xyz/pages/articles/minesweeper_bench.html

yup, if i went to do a PhD interpretability is the only interesting subject for academia IMO right now

It's more of a distilled model, not a fair 1:1 comparison

I use .XYZ because it was pretty cheap when I bought it

I post about ML/DL and SWE at https://bsky.app/profile/snats.xyz

This paper is not comparing against MOLMO or Qwen, so I would take it with a grain of salt

I made some image generation with CLIP and Evolutionary algorithms not so long ago[1] and the results had a bit more life than I was expecting. It is still a contested area of research and you have some cool stuff like CLIPDraw[2] where they use gradient decent to approximate a vector to the embeddings of CLIP.

[1] https://snats.xyz/pages/articles/optimizing_images.html [2] https://arxiv.org/pdf/2106.14843

let me vouch for 11ty.dev, i love it. it is simple, easy to use, and slim. you don't have to worry about fighting the million dogs around and if that seems like to much. just go and use raw html, i think that ever since i've been using raw html i've actually been publishing more blogposts

just changed! thanks for the heads up

hi! i used the average accuracy over the entire dataset made originally made by the llm

Hi! Author here, I wasn't expecting this to be at the top of HN, AMA

Is there any goodmodel for OCR but on handwritten information? I feel like most models are currently kind of trash

Deeptech is going crazy, all of the AI boom is pretty much an obvious subject if you are in media, and finally you still have your usual gossip. You still have a lot of eyes

You can do the second part that is C!

Remember that Llama400B is still in training and could end up crushing a lot of the use cases for GPT-4

I recently wrote a writeup on bigrams and the infinigram outputs[1]. I genuinely believe that ngrams are making a comeback. For query searching it's really good.

[1] https://snats.xyz/pages/articles/from_bigram_to_infinigram.h...

A bit off topic but every time I see DTrOCR I remember that marketing is a good idea because DTrOCR[1] and Fuyu[2] are basically the same architecture[3].

[1] https://arxiv.org/pdf/2308.15996v1.pdf [2] https://www.adept.ai/blog/fuyu-8b [3] If you don't want to search for the figures I made a tiny post about it on my weblog: https://weblog.snats.xyz/posts/2024/02/16/

It's been pretty interesting to see some research trying to solve this. For example, Stanford researchers recently published QuietSTaR[1]. This makes LLMs "think" before they speak. By generating a chain of thoughts and just choosing the best possible path from the generated thoughts.

This method is better than Chain of Thought and I think is a step in the right direction.

[1] https://arxiv.org/abs/2403.09629

to this day i am impressed that they have not figured out how to embed advertisments to bard outputs so that they can go free.

No! I only tried PCA, but I still have the embeddings.

I'll try later and post results.

I built a map of all the PDF urls on the internet recently.

I used a tiny embeddings model and PCA for dimensionality reduction.

https://weblog.snats.xyz/posts/2024/03/20/

Love the website! Just beware that by making it completely free without any credit card required you are going to battle a bit of spam. I liked the solution that bearblog[1] implemented. I would love to hear how you solved this problem!

[1] https://herman.bearblog.dev/the-chatgpt-vs-bear-blog-spam-wa...

A little bit of context. Basically, all of tabular deep learning has been stuck and SOTA has been tree based algorithms like Catboost and XGBoost. This seems like a big step forward towards getting a generalizable deep learning model besides this "tree models"