albertzeyer

This is Windows only.

I wonder, why don't they use Lazarus (https://www.lazarus-ide.org/)? That would also make it cross-platform, and probably gain much more interest in the project.

Oh, I just checked, there is an active fork: https://github.com/ianBBB/boa-constructor

Ok, this post is mostly about text-based IDEs, but I think the point mostly stands as well for IDEs in general. I'm thinking about Visual Basic or Delphi.

I think such a IDE for Python would really be helpful for beginners. Not text-based, but more like Visual Basic. But everything integrated, everything easily discoverable (that's very important!). Maybe also with such a GUI builder as in VB. And a built-in debugger. I think for the code editor, as long as it has some syntax highlighting and tab auto-complete, that would already be enough. But some easy code navigation would also be good. E.g. when you place some button on your window, and double click that button in the GUI editor, you get to the call handler code of that button.

Some time ago, a small group of people (me included) (I think from some similar HN post?) got together and we brainstormed a bit on how to implement this. But we got lost in the discussion around what GUI framework to base this on. I already forgot the options. I think we discussed about PySide, Dear PyGui, or similar. Or maybe also web-based. We couldn't really decide. And then we got distracted, and the discussion died off.

Note, Free Pascal was mentioned here. But then you should also mention Lazarus (https://www.lazarus-ide.org/), which is the same as Free Pascal but cloning Delphi. Lazarus is really great. And it is actively developed. But Object Pascal is too little known nowadays, maybe also a bit outdated.

ICLR 2026 reviews are happening now (or soon). This paper here was accepted at ICLR 2025.

Why not store the data directly as Arrow files, to allow for mmaping? I see F3 also supports such zero-copy mmap, and skimming through the paper, it actually seems that it uses Arrow buffers, so I wonder what is the difference to directly using Arrow files? (Arrow files is what is being used by HuggingFace datasets.)

Long time ago, that was actually very easy on Mac, via SIMBL (https://github.com/albertz/simbl) and Afloat (https://github.com/rwu823/afloat) and you could hack around using FScriptAnywhereSIMBL (https://github.com/albertz/FScriptAnywhereSIMBL) or Pyjector (https://github.com/albertz/Pyjector).

But that doesn't work anymore since a while (I guess due to SIP).

This sounds interesting.

I would really like to read a full research paper made out of this, which describes the method in more detail, gives some more examples, does more analysis on it, etc.

Btw, this uses LLMs on pure text-level? Why not images? Most of these patterns are easy to detect on image-level, but I assume when presented as text, it's much harder.

> LLMs are PhD-level reasoners in math and science, yet they fail at children's puzzles. How is this possible?

I think this argument is a bit flawed. Yes, you can define AGI as being better than (average) humans in every possible task. But isn't this very arbitrary? Isn't it more reasonable to expect that different intelligent systems (including animals, humans) can have different strengths, and it is unreasonable to expect that one system is really better in everything? Maybe it's more reasonable to define ASI that way, but even for ASI, if a system is already better in a majority of tasks (but not necessarily in every task), I think this should already count as ASI. Maybe really being better in every possible task is just not possible. You could design a task that is very specifically tailored for human intelligence.

Why not use getaddrinfo_a / getaddrinfo_async_start / GetAddrInfoExW?

Or just use some standalone DNS resolve code or library (which basically replicates getaddrinfo but supports this in an async way)?

See also here the discussion: https://github.com/crystal-lang/crystal/issues/13619

I keep my /etc under Git. When the system does changes automatically (via an update or whatever), I make a Git commit with a special distinct message, and so I can easily filter out all my own changes.

It's funny: The article talks about GPT and diffusion in the context of technological innovations, and then getting to AI/LLMs. But "diffusion" and "GPT" have some very different meaning in the context of AI/LLMs, referring to specific kind of models.

That said, I don't really get the conclusion of the article. We should put more effort for productization of LLMs? But isn't that exactly what the big companies are trying to do already?

I don't really need the versioning aspect too much, but sometimes I modify the photos a bit (e.g. rotating or so). But all the other things are relevant for me, like having it distributed, syncing, only partially having the data on a particular node, etc.

So, what solution would be better for that? In the end it seems that other solutions provide a similar set of features. E.g. Syncthing.

But what's the downside with Git-annex over Syncthing or other solutions?

How much data do you have? I'm using git-annex on my photos, and that are around 100k-1M files, several TB of data, on a ZFS. In the beginning, everything was fine, but it starts to become increasingly slow, such that every operation takes several minutes (5-30 mins or so).

I wonder a bit whether that is ZFS, or git-annex, or maybe my disk, or sth else.

I have seen such variants as well, where you actually need to install some antennas (RTK antennas?).

But that was already too much effort for me, so that's why I chose the eufy. Also, from reports that I have read, it doesn't necessarily work better with antennas. Actually, from reports that I have read, robots with camera-only had the least amount of reports where navigation was not properly working. At least that was my impression.

Maybe 300 m^2 or so.

The eufy E15 is for up to 800 m^2. There is the eufy E18 for up to 1200 m^2.

I have seen other (more expensive, bigger) robots for much larger lawns.

My garden is relatively flat with a few bumps here and there.

You can also easily mark some areas of the lawn as always excluded so that it wont drive there.

In my case, my lawn isn't easily accessible and also not visible from the street (because the house and garage is between the street and the garden), and I trust my neighbors. So, I think (I hope) this isn't so much an issue for my case.

I think the eufy has also an optional GPS module just for this purpose, to better track it. I don't think this GPS module has any other purpose (e.g. it would not be used for navigation). But I didn't really checked the details on this.

I have seen this a few times for such robots, that you can buy an optional GPS module for tracking.

> The current generation of robotic lawn mowers sucks. Basically all of these bots drive in a random direction until they hit the border of the lawn, rotate for a randomized duration and repeat.

I recently (a few weeks ago) bought one. While researching on the available options (which seemed relevant to me), actually almost none of the robots work this way. Most of them systematically go through the lawn. I think from those that I checked, only the Worx Landroid does it randomly.

I was searching for some model which works without wires, because I was too lazy to set this up. Basically, in general, I wanted sth which required as little effort as possible.

I decided for eufy E15, which uses camera (no GPS, no wires, no lidar, nothing else really). And it just seems to work. It creates a map first, and then systematically goes over the lawn. I didn't really need to do anything.

(I'm not affiliated with eufy in any way. I'm just quite happy with it so far.)

That said, obviously, having an open source variant of such a robot would be even nicer (if it works)! So I'm quite happy to see such a project.

This robot here uses GPS, as far as I can see, as the sole technique for navigation and localization. From reports that I have read, GPS mostly works fine, except for some cases where it does not (where GPS coverage is not great). Camera on the other side always works (during daytime). Maybe this could be added to this project? Of course, using the camera is probably quite a bit more complicated, and more prone to errors, but overall might be more robust and reliable.

I thought that is what you mean when you said "a lot of interesting bits about the history of the development of neural networks (including backpropagation) can be found in the book Talking Nets", that there is some relevant reference to backprop which is missing here in the linked article.

I don't really understand your negativity here, and what you are reading into my comment. I never asked you to do a research project? I just thought you might know some other references which are not in the article. If you don't, fine.

Note that I don't expect that any relevant reference is missing here. Schmidhuber always try to be very careful to be very complete and exhaustive cite everything there is on some topic. That is why I was double curious about the possibility that sth is missing, and what it could be.

What do you mean? This popular paper is cited:

[RUM] DE Rumelhart, GE Hinton, RJ Williams (1985). Learning Internal Representations by Error Propagation.

How do you not have the citations in front of you? They are all in the article? I don't expect that any relevant (re)invention of backprop is missing there. Or, if you really know some reinvention of backprop that is not mentioned here, tell Jürgen Schmidhuber, he is actually very curious to learn about other such instances that he is not aware of yet.

It's not about the probability of individual tokens. It's about the probability of the whole sequence of tokens, the whole answer.

If the model is good (or the human comedian is good), a good funny joke would have a higher probability as the response to the question than a not-so-funny joke.

When you use the chain rule of probability to break down the sequence of tokens into probabilities of individual tokens, yes, some of them might have a low probability (and maybe in some frames, there would be other tokens with higher probability). But what counts is the overall probability of the sequence. That's why greedy search is not necessarily the best. A good search algorithm is supposed to find the most likely sequence, e.g. by beam search. (But then, people also do nucleus sampling, which is maybe again a bit counterintuitive...)

No, it was not difficult at all. I really wonder why they have such a bad example here for GPT1.

See for example this popular blog post: https://karpathy.github.io/2015/05/21/rnn-effectiveness/

That was in 2015, with RNN LMs, which are all much much weaker in that blog post compared GPT1.

And already looking at those examples in 2015, you could maybe see the future potential. But no-one was thinking that scaling up would work as effective as it does.

2015 is also by far not the first time where we had such LMs. Mikolov has done RNN LMs since 2010, or Sutskever in 2011. You might find even earlier examples of NN LMs.

(Before that, state-of-the-art was mostly N-grams.)

> Once someone hits AGI/SGI

I don't think there will be such a unique event. There is no clear boundary. This is a continuous process. Modells get slightly better than before.

Also, another dimension is the inference cost to run those models. It has to be cheap enough to really take advantage of it.

Also, I wonder, what would be a good target to make profit, to develop new things? There is Isomorphic Labs, which seems like a good target. This company already exists now, and people are working on it. What else?

This is exciting. So this is using unified memory of CUDA? I wonder how well that works. Is the behavior of the unified memory in CUDA actually the same as for Apple silicon? For Apple silicon, as I understand, the memory is anyway shared between GPU and CPU. But for CUDA, this is not the case. So when you have some tensor on CPU, how will it end up on GPU then? This needs a copy somehow. Or is this all hidden by CUDA?

It's a bit weird that such a paper (standard ML topic, would fit to NeurIPS, ICLR, ICML or so) is published in such a conference (IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON)), which seems not really the best fit. This indicates that the quality bar would not be reached for the top venues.

If you want to read some recent interesting ML papers (on Transformers, or whatever you care), check e.g. recent ICLR 2025 papers (https://openreview.net/group?id=ICLR.cc/2025/Conference#tab-...), or other recent venues.

The API can be used both via your normal Google account, or via API key?

Because it says in the README:

> Authenticate: When prompted, sign in with your personal Google account. This will grant you up to 60 model requests per minute and 1,000 model requests per day using Gemini 2.5 Pro.

> For advanced use or increased limits: If you need to use a specific model or require a higher request capacity, you can use an API key: ...

When I have the Google AI Pro subscription in my Google account, and I use the personal Google account for authentication here, will I also have more requests per day then?

I'm currently wondering what makes more sense for me (not for CLI in particular, but for Gemini in general): To use the Google AI Pro subscription, or to use an API key. But I would also want to use the API maybe at some point. I thought the API requires an API key, but here it seems also the normal Google account can be used?

"hundreds of thousands to potentially millions of tokens" - that's the same order as current commercial LLMs.

Also note, if the sequence length is not really much larger than the model dimension (at least two orders of magnitude more), the quadratic complexity of the self-attention is really not such a big issue - the matrix multiplication in the feed-forward layers will be usually 8x the model dimension squared, and thus that part will usually dominate.

Also note that there has been so much research on this already. While this particular approach might be novel, there has been attempts to avoid the O(n^2) complexity in self-attention basically almost since the original transformer paper came out in 2017. I wonder a bit that this paper does not cite xLSTM, or Block-Recurrent Transformers.

Also, this paper comes very short in experiments. There is basically only table 2. There is no study on length extrapolation (which is very relevant for the topic), or needle-in-haystack experiments, or scaling studies, any larger scale experiments, etc. Also, even in this main table 2, I see a couple of typos. And looking at the results in table 2, the improvements seems to be quite minor.

So I would conclude, this needs a lot more work.

I'm not sure what type of model Google uses nowadays for their webinterface. I know that they also actually provide LLM-based translation via their API.

Also the traditional cross-attention-based encoder-decoder translation models support document-level translation, and also with context. And Google definitely has all those models. But I think the Google webinterface has used much weaker models (for whatever reason; maybe inference costs?).

I think DeepL is quite good. For business applications, there is Lilt or AppTek and many others. They can easily set up a model for you that allows you to specify context, or be trained for some specific domain, e.g. medical texts.

I don't really have a good reference for a similar leaderboard for translation models. For translation, the metric to measure the quality is anyway much more problematic than for speech recognition. I think for the best models, only human evaluation is working well now.

Do you have any comparisons in terms of WER? I doubt that GPT-4o-transcribe is better than the best models from that leaderboard (https://huggingface.co/spaces/hf-audio/open_asr_leaderboard). A quick search on this got me here: https://www.reddit.com/r/OpenAI/comments/1jvdqty/gpt4otransc... https://scribewave.com/blog/openai-launches-gpt-4o-transcrib...

It is stated that GPT-4o-transcribe is better than Whisper-large. That might be true, but what version of Whisper-large actually exactly? Looking at the leaderboard, there are a lot of Whisper variants. But anyway, the best Whisper variant, CrisperWhisper, is currently only at rank 5. (I assume GPT-4o-transcribe was not compared to that but to some other Whisper model.)

It is stated that Scribe v1 from elevenlabs is better than GPT-4o-transcribe. In the leaderboard, Scribe v1 is also only at rank 6.

That's not quite true. State of the art both in speech recognition and translation is still a dedicated model only for this task alone. Although the gap is getting smaller and smaller, and it also heavily depends on who invests how much training budget.

For example, for automatic speech recognition (ASR), see: https://huggingface.co/spaces/hf-audio/open_asr_leaderboard

The current best ASR model has 600M params (tiny compared to LLMs, and way faster than any LLM: 3386.02 RTFx vs 62.12 RTFx, much cheaper) and was trained on 120,000h of speech. In comparison, the next best speech LLM (quite close in WER, but slightly worse) has 5.6B params and was trained on 5T tokens, 2.3M speech hours. It has been always like this: With a fraction of the cost, you will get a pure ASR model which still beats every speech LLM.

The same is true for translation models, at least when you have enough training data, so for popular translation pairs.

However, LLMs are obviously more powerful in what they can do despite just speech recognition or translation.

There is also optical neuromorphic computing, as an alternative to electronic neuromorphic computing like memristors. It's an fascinating field, where you use optical signals to perform analog computing. For example:

https://www.nature.com/articles/s41566-020-00754-y

https://www.nature.com/articles/s44172-022-00024-5

As far as I understood, you can only compute quite small neural networks until the noise signal gets too large, and also only a very limited set of computations works well in photonics.