HNHacker News
TopNewBestAskShowJobs

nestorD

2,038 karma · joined June 11, 2018

French, researcher, engineer, consulting mentalist.

Specialized in artificial intelligence, high-performance computing, and floating point arithmetic.

Nestor Demeure (https://nestordemeure.github.io/about/)

submissionscomments
nestorD··on The Value of Differences: Jennifer Lindsay on Noticing Translation
As a French speaker who has done quite a bit of paid translating between French and English (so an uneducated translator rather than an amateur one I guess...), I have found that a lot of translation (in non-fiction, but also sometimes in fiction) have a feel to them. You might not care or notice at a glance, but the text does feel unmistakably translated. That is a constant reminder of that fact that translation is a job that takes real skill, not just knowledge of both language.
nestorD··on Why Does Claude Speak Byzantine Music Notation?
It is a direct ancestor but I find that what most people picture when they hear Old English (and have no prior knowledge of it) is something closer to Middle English, which is somewhat redeable by modern English speakers, rather than something like `Oft Scyld Scefing sceaþena þreatum, monegum mægþum, meodosetla ofteah, egsode eorlas.` [0]

[0]: https://www.poetryfoundation.org/poems/43521/beowulf-old-eng...

nestorD··on Why Does Claude Speak Byzantine Music Notation?
I was also surprised to find out (roughly a year ago) that Claude is good at Old English (which, despite its misleading name, looks nothing like English and is more of a Germanic language) whereas ChatGPT would output pure hallucinations.
nestorD··on 4o Image Generation
I believe so! Since it is good at consistency and can be feed reference images, you can generate character references and deed those, along with the previous panels, to the model working one panel at a time.
nestorD··on 4o Image Generation
Short answer: the model is good at consistency. You can use it to generate a set a style reference images, then use those as reference for all your subsequent generations. Generating in the same chat might also help it have further consistency between images.
nestorD··on The danger of relying on OpenAI's Deep Research
Yes.
nestorD··on Building a Medieval Castle from Scratch
France is not big on billionaires pet project... Also, a lot of the hard work is done by volunteers!
nestorD··on Tesla Sales Are Tanking in Europe
Musk has been using his money to influence politics (including in the US, accross the board). Buying a Tesla is giving him money and thus supporting that behavior. It seems obvious to me (a European living in the US) that, if you disagree with his politics, you should not buy his cars. This also apply to people living in the US but, having seen a strongly pro choice north Californian friend buy a Tesla recently, I do not know if US citizens think in those terms.
nestorD··on Norms-Based Intellectual Property Systems: The Case of French Chefs(2008)
I am not talking about exposure to the lay public, which is a different problem, but tricks sold on the inside. A magician with low ethics can take a pre-existing idea or presentation, tweak it or change the method, and sell it as their own. It's bad, but it is legal. An example of that would be the work Teller had to do to protect his shadow rose effect. Meanwhile, someone wanting to play by the rule might not publish a significant improvement on a 1945 idea that is now out of print, because they could not acquire the rights from the, long deceased, original author.
nestorD··on Norms-Based Intellectual Property Systems: The Case of French Chefs(2008)
Intriguing! But, being familiar with a domain that work of a mix of copyright and norm based intellectual property (magic). It is a mess in that people who do not care will abuse the system with little retribution, while people who care often do not dare do thing because the rules are too fuzzy to know if something is fine.
nestorD··on NumPy QuadDType: Quadruple Precision for Everyone
Yes! And no!

You will indeed need higher precision if you need a very large number of digits (more than what double precision gives you).

However, even with high precision, the algorithms you use to compute your moments matter: you might not have as many digits as you hope for. Variance, for example, is famously numerically unstable (sometimes giving a negative variance) when computed naively. A stable summation algorithm plugged into a variance computation (even a carefully written one) can sometimes give you six additional digits of precision in the variance that you might not get when doubling the precision (it depends on how big of a cancellation you had in your operations; too big, and doubling the precision will not be enough to recover, while a compensated summation would do fine).

I have seen this play out once in a clustering algorithm that would only work on some data when using a compensated summation to compute the variance.

nestorD··on NumPy QuadDType: Quadruple Precision for Everyone
1. Parallelization: Yes, they do parallelize! Conceptually, it is fairly easy to use accurate algorithms locally, then reduce the local accumulators. A side effect of doing this is cutting (or reducing) the noise in the output introduced by the parallelization (note that this is not to be confused with deterministic summation/BLAS, which enforces the order of operations to achieve the same side effect but do not improve accuracy).

2. It varies wildly depending on the algorithm you are using, the implementation of said algorithm, and the operations you are performing outside of the summation (if you sum the results of a loop performing non-trivial operations, that can easily overlap with the summation overhead making it essentially free). The only rule of thumb I can give is that runtime cost increases with desired accuracy. My recommendation would be to switch from a naive sum to using an exact accumulator and observe the difference in output value (letting you know how impactful numerical errors in that sum were) and runtime (letting you know how expensive the switch is for you), then dial it down if needed.

nestorD··on NumPy QuadDType: Quadruple Precision for Everyone
Yes! I suspect that it boils down to 32 bits not being that much (you can enumerate them to test a function), while 64 bits is a lot. By that point, if you are having problems, then you are likely hitting a wall that will not be crossed by throwing additional bits at the problem.
nestorD··on NumPy QuadDType: Quadruple Precision for Everyone
I have done my PhD on measuring floating-point error in large numerical simulations. In my experience, most of the time, if 64-bit floating points are not enough, then trying to increase precision further is unlikely to help much (there are exceptions).

In general, using a well-placed compensated (or exact) sum/dot product algorithm will solve most of your problems at a low cost. Nowadays, most people are familiar with Kahan summation (and related tricks like sorting numbers), but they do not know that it does not help much (often less than 128-bit floating points) and that the state of the art for those algorithms has moved quite a bit since. Nowadays, you can ensure that your sum/BLAS operation produces a result within one ulp of the exact result!

I highly recommend perusing the Handbook of Floating-Point Arithmetic (the Accurate Algorithm online page also seems fine [0]) to get an idea of what can be done. Implementations of these ideas can be found in most languages used for numerical computation (a quick Google search gave me options for C++/Python [1], Rust [2], etc.).

[0]: https://accurate-algorithms.readthedocs.io/en/latest/index.h...

[1]: https://github.com/yafshar/xsum

[2]: https://github.com/bsteinb/accurate

nestorD··on NumPy QuadDType: Quadruple Precision for Everyone
In practice, Posits grabs very few additional bits and those are mostly felt in low bits (16 and less) regimes. That's a big reason it is mostly seeing adoption in memory constraint regimes.

A great thing in the Posit proposal is that they also consider a distinct accumulator type for things like sums and dot product. In my experience (I did a PhD on measuring floating point error in large numerical simulations) those operations are the ones that are most likely to sneak on you and need fixing. But most people don't realize that those accumulator type are actually easy to use with IEEE floats, they are entirely orthogonal to using Posits.

nestorD··on NotebookLM's automatically generated podcasts are surprisingly effective
My first instinct was to not see why one would want to consume such a podcast, a simile instead of either the original or an (AI?) summary of the original. Then I remembered a partially disabled friend who regularly asks for audio books, because he physically cannot read long form. This, condensed, output would make a lot of ideas accessible to him.
nestorD··on Are psi researchers more like lay believers or skeptics?
The full article is here: https://www.frontiersin.org/journals/psychology/articles/10....

Note that the editor's presentation linked in HN only shows the belief plot (showing that psi researchers tend to be believe in psi), but the actual article draw most conclusions from their figure 2 (which studies correlation between belief in psi and what they call open-minded thinking)

nestorD··on Show HN: Chili. Rust port of Spice, a low-overhead parallelization library
Rayon is known to have a higher overhead for the simple, and arguably most common case in numerical codes, static block size scheduling. Things get better, and you start to benefit from rayon, as work gets less balanced.
nestorD··on Zngur: A C++/Rust interop tool
Not the OP but I have done similar things. Claude has a workspace option that let's you preset a set of documents (here the documentation of the library used) and a system prompt to be included when starting a new chat in the given workspace. Those will be in its, fairly large, context window (overall, independently of its intelligence, I have found Claude current UI to have an edge when dealing with files).
nestorD··on Crafting Interpreters with Rust: On Garbage Collection
Note that there is quite a bit of thinking and available implementations of GC implementations available in Rust: https://manishearth.github.io/blog/2021/04/05/a-tour-of-safe...
nestorD··on LivePortrait: A fast, controllable portrait animation model
The "generalization to animals" part seems like it opens a lot of interesting avenues!
nestorD··on Tao Te Ching translated by Ursula Le Guin (1997)
I strongly disagree. Her essays (?!) and short story might be a good place to demonstrate the breath and width of her style.

The one exeption to this, for my taste, might be Earthsea, parts of it definitely fell short of its reputation as far as I am concerned.

nestorD··on Benjamin Thorpe translated almost all Old English texts (2016)
Most LLMs do well at Middle English (it is not that difficult of a task). Old English is significantly harder but I have found Claude Opus to give good results.
nestorD··on A long guide to giving a short academic talk (2022)
Here is my own list of recommendation (written down to give interns): https://nestordemeure.github.io/writing/guides/presentation/
nestorD··on Medieval game pieces emerge from the ruins of a German castle
You might also like the Lewis Chessmen 3D scans: https://sketchfab.com/britishmuseum/collections/lewis-chessm...
nestorD··on llama-fs: A self-organizing file system with llama 3
A `dry run` mode producing a symbolic link based reorganized copy of the folder would be nice.
nestorD··on Wikimedia Enterprise – APIs for LLMs, AI Training, and More
The approach described above is what is commonly referred to as RAG[0]. I am not aware of someone having used it on Wikipedia but, from experience and while it helps, it does not eliminate all hallucinations.

[0]: https://en.wikipedia.org/wiki/Large_language_model#:~:text=t...

nestorD··on A web version of Anthropic's prompt engineering interactive tutorial
It is already done by some platforms (for example ChatGPT generates a prompt based on your query when generating images).

The downside is that you might lose information (the improved prompt might not be an exact semantic match with the original prompt), plus it becomes less and less useful as model's ability to understand commands improve.

Basically, if you express yourself clearly, betting on model improvement is enough, there is no need to add a faillible extra-step.

nestorD··on The Death of a Mnemonist
I can confirm from experience that all of the things described above can be done by a normal person with visualization (it is technically auto-hypnosis but the term makes it sound more complicated than it is).

It is easier to get started with some instruments to give you feedback but it is not particularly hard.

nestorD··on GPT-4o
The press statement has consistent image generation and other image manipulation (depicting the same character in different poses, taking a photo and generating a caricature of the person, etc) that does not seem deployed to the chat interface.

Will they be deployed? They would make the OpenAI image model significantly more useful than the competition.

← PreviousPage 2 of 22Next →