whoami_nr

Quick calculator to visualize/simulate voice latency and compare API costs across voice model providers

Author here. Yeah totally agreed. The more rigorous way to do this would be to use a fixed seed and temp and in a local model setting and then sample the logprobs and then analyse that data.

I had an hour to kill and did this experiment.

Author here. I know it’s silly. I understand to some extent how they work. I was just doing this for fun. Took about 1hr for everything and it all started when a friend asked me whether we can use them for a coin toss.

Veritasium did a video on this. Most people guess 37 when asked to pick between 1-100

Author here. I know 0-10 is one extra even number. I also just did this for fun so don't take the statistical significance aspect of it very seriously. You also need to run this multiple times with multiple temperature and top_p values to do this more rigorously.

Small difference. Its called llms.txt

https://llmstxt.org/

Author here. I just messed up while posting.

Yes, I am not American and I had no clue about the connotations.

The Llada paper: https://ml-gsai.github.io/LLaDA-demo/ here implied strong bidirectional reasoning capabilities and improved performance on reversal tasks (where the model needs to reason backwards).

I made a logical leap from there.

yeah but you can backtrack your thinking. You also have a mind voice to plan out the next couple words/reflect/self correct before uttering them.

So, in practice there are some limitations here. Chat interfaces force you to feed the entire context to the model everytime you ping it. Even multi step tool calls have a similar thing going. So, yeah we may effectively turn all of this effectively into autoregressive models too.

Github pages + Jekyll for the main blog. All the subdomains(mostly side projects which I have put the effort to deploy) are hosted on Replit.

Blog: rnikhil.com

Send your prompt to multiple LLMs and get the best response. Based on this tweet from Karpathy: https://x.com/xundecidability/status/1870678482700923325

I was learning to code again, and I built this backroom simulator(https://simulator.rnikhil.com/) which you can use to simulate conversations between different LLMs(optionally give a character to each LLM too). I think its quite similar to what you have.

On a side note, I am quite interested to watch LLMs play games based on game theory. Would be a fun experiment and I will probably setup something for the donor game as well.

Made a simple llm backroom simulator. Give the AI agents a name and personality and then watch them get lost talking to each other.

Its a lot of fun. you can setup rap battles between random two people, make gandalf and terminator debate the meaning of life. etc etc. Be descriptive in your character details. Give some sample messages on how you want it to respond etc. Give very strict do and donts.Currently its bring your own key.

Built entirely using replit agents and hosted on replit.

PS: I know this is a clone of many such habit tracking tools. I am re-learning to code again

Wouldn't 64 in base64 be represented as = ? (and not 10). Love the comic!

Dynamo AI | ML Application security customer facing engineer, and other ML research engineer roles too | REMOTE/US/Europe

Dynamo AI builds evaluation suites for your LLMs to detect hallucination, security and compliance risks. We also build real time guardrailing products for enterprises. The ML application security engineer role is customer facing. You need to have familiarity with ML systems and architectures and be on top of all the security vulnerabilities (both at AI model level and surrounding supply chain)

Apply on the website: https://jobs.lever.co/dynamoai

Building LLM evaluation suites. Basically trying to test LLMs for privacy problems (data leakage/memorisation, PII extraction sort of thing), hallucination(RAG, summarisation etc) and security/compliance stuff(like bias/fiarness, toxicity, jailbreaks/prompt injection).

Involves a bunch of reading research papers, figuring out which ones are relevant to enterprise customers and getting our ML team to build it out. The most interesting part of this is how you present the insights of a given test to a customer in a consumable and usable format. (Ex: Just dumping a bunch of RAG hallucination metrics isn't enough but you want to figure out what are the key insights and interpretations of these metrics which could be useful to a data scientist or ML engineer)

He mentioned it on the Dwarkesh podcast: https://www.youtube.com/watch?v=bc6uFV9CJGg

Oh my god. This brings back a ton of memories. I was writing a port knocking implementation back in 2016 or so as a side project and I was using this exact flowchart to use iptables for opening/closing ports and routing packets.

Thats what the whole thing is about. He is complaining that they don't respect robots.txt

That’s just jailbreaking(like DAN prompts) and a simpler terminology solution is to stop classifying jailbreaks under prompt injection.

You can just email me at contact@rnikhil.com. For good measure, I added it to my HN profile too.

Not sure about the Cloudfare thing but I just got an alert that bot traffic has spiked by 95% so maybe they are resorting to captcha checks. One downside of having spiky/non consistent traffic patterns haha. Also, yes never been a fan of KDE. Love the minimalist vibe of xfce. Lxde was another one of my favourites.

Edit: Fixed the cloudfare email masking thing.

Thanks. Your blog has been my goto for the LLM work you have been doing and really liked the data exfilration stuff you did using their plugins. Took longer than expected for that to be patched.

Fair, I agree and shall correct it. I've always seen jailbreaking as a subset of prompt injection and sort of mixed up the explanation it up in my post. In my understanding, jailbreaking involves bypassing safety/moderation features. Anyway, I have actually linked your articles on my blog directly as well for further reading as part of the LLM related posts.

Author here. Thanks for your list.

Every paper I read on this topic has Carlini or has roots to his work. Looks like he has been doing this for a while. I shall check out your links though some of them have been linked in the post (at the bottom) as well. Regd. FGSM, it was one of the few attacks I could actually understand and the rest were beyond my math skills and hence I wrote about it on the post. I agree with you and have linked a longer list as well.

PS: I love and used to run xubuntu as well.

Author here. Some of them are black box attacks (like the one where they get the training data out of the model) and it was done on Amazon cloud classifier which big companies regularly use. So, I wouldn’t say that these attacks are entirely impractical and purely a research endeavour.

Author here. I get what you mean and I remember the incident happening when I was in college. However, I also remember that they were reproduced across multiple publications which means you are implying some sort of data poisoning attack which were super nascent back then. IIRC the spam filter data poisoning was the first class of these vulnerabilities and the image classifier stuff came later. Could be wrong on the timelines. Funnily, they fixed by just removed the gorilla label from their classifier.

Yes I have seen the thread. Catching a player using just the winrate is hard. 9k hands is peanuts and some of the assumptions in that thread (like a 53% VPIP player must have 0bb/100 winrate) are slightly far fetched.

I know for a fact that GG has a GTO detection algorithm. If you play too close to GTO/optimal strategy they investigate. Lot of RTA folks got caught this way.

Jason Koon and Fedor Holz are part of the team which manually reviews statistical anomalies. Its stupid to think they dont know about the data side of Poker. Moreover, a ton of Poker players end up becoming traders and data scientists. There is a lot of skill overlap.

I agree that some of the stuff is PR nonsense but to dismiss their entire anti fraud operation is just stupid. They are literally the best in the world at this.