Hacker News Reader

skumar17on Show HN: r1_vlm – Open-Source Framework for Visual Reasoning with GRPO

That’s a good observation. For this project, I found that while the base model could “read” the image, it didn’t really understand how to use it. GRPO allowed it to effectively search the solution space.

skumar17on Show HN: r1_vlm – Open-Source Framework for Visual Reasoning with GRPO

The attention demo code is in the /attention_demo directory if you want to try it on your own messages too :)

skumar17on Show HN: r1_vlm – Open-Source Framework for Visual Reasoning with GRPO

Unfortunately it might be a bit challenging as there’s a nontrivial amount of extra computation we do for the viz, but it’s probably possible?

skumar17on Show HN: r1_vlm – Open-Source Framework for Visual Reasoning with GRPO

Thanks! I really love the visualization too. We have a hosted demo you can try as well!

https://huggingface.co/spaces/Groundlight/grpo-vlm-decoder

skumar17on Finding inputs where deep learning models fail

Hi there! Last month at NeurIPS (an ML conference), I read an interesting paper "Human Expertise in Algorithmic Prediction" that describes a framework for determining where ML models are outperformed by human experts. I found the authors' work to be very interesting. Below, I explore their framework further and extend it to multiclass classification. My results are pretty surprising, showing that a group of modern model architectures have trouble with dogs and cats in CIFAR-10.

GitHub Link: https://github.com/sunildkumar/model_indistinguishability

Paper Link: https://arxiv.org/abs/2402.00793

skumar17on Building your first computer vision model just got easier

I just setup this awesome way to easily deploy computer vision models using natural language, check it out: https://code.groundlight.ai/python-sdk/blog/getting-started

skumar17on Grime Guardian: Dirty dish detection with declarative computer vision

I built an application that automatically detects and publicly posts when my coworkers leave dirty dishes in the sink using computer vision.

Check it out: https://code.groundlight.ai/python-sdk/blog/grime-guardian

Source code: https://github.com/sunildkumar/GrimeGuardian

skumar17on LoRA from Scratch

Hi there! I was interested in learning more about LoRA but I was having a hard time finding a good simple example of implementing LoRA, as most sources are training large models and use a combination of huggingface transformers and the loralib package the original LoRA authors wrote. As a result, I ended up writing a simple LoRA implementation from scratch in pytorch lightning, and I figured other people might find it helpful as a learning resource or springboard.