dongecko

What a great read! You got me at the base64 oddity. I also stumbled over this, while trying to dodge some LLM limitation. (was trying to generate images in a time before multimodal was a thing. it only worked to a degree).

Naaa, you have to think of the poster in options. Like, "would you want to have some extra publicity with your poster for an extra 100 bucks?"

I never realized; thanks for pointing that out. Sneakers is one of my favorite movies.

A while back, someone made https://meet.hn. It's a map where people from hackernews could locate themselves.

Of course this is by far not comprehensive, but it gives you an idea of where the people are from.

Exactly my point of view. For the most part I do not root for my preferred technology, but rather try to inform my powers about the caveats I see. This way at least the right aspects to check have a chance to enter the debates above my payroll.

Yes, they can. Meta described a clever way in their paper on training Llama3 [1] (in section about factuality). The idea is to sample several answers to a question that you know the answer to. Let an LLM decide if the given answers are different from your known truth. If so, you found a question, that you can train your LLM in the next post-training round to answer with "I don't know". Do that a couple hundred of times and your LLM will identify neurons that indicate doubt and from here on have the ability to answer with "I don't know".

[Edit] The article here also mentions a paper [2] that comes up with the idea of an uncertainty token. So here the incorporation of uncertainty is already baked in at pre-training.[/Edit]

[1] https://arxiv.org/pdf/2407.21783 [2] https://openreview.net/pdf?id=Wc0vlQuoLb

We have a couple of systems at work that incorporate LLMs. There are a bunch of RAG chatbots for large documentation collections and a bunch of extract-info-from-email bots. I would none of these call an agent. The one thing that comes close to an agent is a very bot that can query a few different SQL and API data sources. Given a users text query, it decides on its own which tool(s) to use. It can also retry, or re-formulate its task. The agentic parts are mainly done in LangGrah.

Came here for exactly that. Thank you.

Bolzman machines were there in the very early days of deep learning. It was a clever hack to train deep nets layer wise and work with limited ressources.

Each layer was trained similar to the encoder part of an autoencoder. This way the layerwise transformations were not random, but roughly kept some of the original datas properties. Up to here training was done without the use of labelled data. After this training stage was done, you had a very nice initialization for your network and could train it end to end according to your task and target label.

If I recall correctly, the neural layers output was probabilistic. Because of that you couldn't simply use back propagation to learn the weights. Maybe this is the connection to John Hopkins work. But here my memory is a bit fuzzy.

This comment caught me off guard! I literally blew my coffee all over the table.

I second that thought. There is a pretty well cited paper from the late eighties called "Multilayer Feedforward Networks are Universal Approximators". It shows that a feedforward network with a single hidden layer containing a finite number of neurons can approximate any continuous function. For non continous function additional layers are needed.

Motorola used to have a one bit microprocessor, the MC14500B.

I simply love this book!

At the very beginning of my journey I did some fine tuning with Lora on a (I believe) Falcon model, but I haven't looked at it since. My impression was that injecting knowledge via fine tuning doesn't work, but tweaking behavior does. So your answer makes much sense to me. Thanks for bringing that up! I will definitively try that out.

I have not encountered this problem yet. When I was talking about the format of the answer I meant the following: No matter if you're using Langchain, Llamaindex, something self made, or Instructor (just to get a json back); under the hood there is somewhere the request to the LLM to reply in a structured way, like "answer in the following json format", or "just say 'a', 'b' or 'c'". ChatGPT tends to obey this rather well, most locally running LLMs don't. They answer like:

> Sure my friend, here is your requested json:

> ```

> {

> name: "Daniel",

> age: 47

> }

> ```

Unfortunately, the introductory sentence breaks directly parsing the answer, which means extra coding steps, or tweaking your prompt.

I also think of the retrieval part as a bottleneck and I am super excited of what the future holds.

In particular, I wonder if RAG systems will soon be a thing of the past, because end to end trained gigantic networks with longer attention spans, compression of knowledge, or hierarchical attention will at some point outperform retrieval. On the other hand, I can also see a completely different direction coming, where we develop architectures that, like operating systems, deal with memory management, scheduling and so on.

That is an interesting observation. I have not gotten to the point of too long cycles and I can think of two reasons for that.

Maybe my use case is narrow enough, so that in combination with a rather constraining and strict system message an answer is easy to find.

Second, I have lately played a lot with locally running LLMs. Their answers often break the formatting required for the agent to automatically proceed. So maybe I just don't see spiraling into oblivion, because I run into errors early ;)

The company I work for has tons of documentation and regulations for several areas. In some areas the documents are well over a thousand and for the ease of use of these documents we build RAG based chat bots. This is why I have been playing with RAG systems on the scale of "build completely from scratch" to "connect the services in Azure". The retrieval part of a RAG is vital for good/reliable answers and if you build it naive, the results are not overwhelming.

You can improve on the retrieved documents in many ways, like - by better chunking,

- better embedding,

- embedding several rephrased versions of the query,

- embedding a hypothetical answer to the prompt,

- hybrid retrieval (vector similarity + keyword/tfidf/bm25 related search),

- massively incorporating meta data,

- introducing additional (or hierarchical) summaries of the documents,

- returning not only the chunks but also adjacent text,

- re-ranking the candidate documents,

- fine tuning the LLM and much, much more.

However, at the end of the day a RAG system usually still has a hard time answering questions that require an overview of your data. Example questions are:

- "What are the key differences between the new and the old version of document X?"

- "Which documents can I ask you questions about?"

- "How do the regulations differ between case A and case B?"

In these cases it is really helpful to incorporate LLMs to decide how to process the prompt. This can be something simple like query-routing, or rephrasing/enhancing the original prompt until something useful comes up. But it can also be agents that come up with sub-queries and a plan on how to combine the partial answers. You can also build a network of agents with different roles (like coordinator/planner, reviewer, retriever, ...) to come up with an answer.

* edited the formatting

I have been to a couple of hackathons as participant and as organizer. Some sessions where really good and some were really terrible. The biggest takeaway for me in organizing a hackathon is to keep in mind that technical people who participate tend to be rather competitive folks who put a lot of soul into their work. So if you have no fair and clear evaluation system and PowerPoint designs compete with code, frustration is sure to happen.

I read it the same way as you did, or at least it's an approximation.

In general that's not really surprising. I remember discussions from some years ago about larger networks leading to smother loss surfaces.

There is also the evo64 project which rebuild the c64 with modern parts.

I love this thread. The question is so unspecific that it's like a version of "To get the right answer you need to post something wrong on the internet".

I think that the display is not used at all. If I understood correctly, the scart connector is hosting a hidden uart interface. This let's you connect a serial terminal via three cables. (You can e.g. buy a serial to usb adapter for 5 bucks on Amazon and use it via putt.)

I use monokai and dracula, depending on the environment. If i can alter something, I usually go for dracula and FiraScript as font.

I really love this book. Beautiful data and beautiful visualizations are also nice reads, but beautiful code has a special place in my heart.

The Frank-Wolfe algorithm for optimization. Christian Bauckhage has a couple of papers, where he uses it to connect optimization problems to the fix states of a certain type of recurrent neutral networks (echo state networks).

The Marchenko-Pastur distribution derives from random matrix theory a nice theoretical border to estimate if a principal component is more noise then data.

Also I am a huge fan of all sorts of embedding/projection/matrix factorization algorithms and I use them quite regularly.

I'd second that. Some years ago I had a look at the quake 3 bots code, which I think were also written by John Carmack. The code was amazingly intuitive.