w-m

Possibly also nvitop, which is a different tool from nvtop: https://github.com/XuehaiPan/nvitop

The documentation reads like it was written by a programmer who documented the different parameters to their implementation of a specific algorithm. Now when you as the user come along and want to use silenceremove, you'll have to carefully read through this, and build your own mental model of that algorithm, and then you'll be able to set these parameters accordingly. That takes a lot of time and energy, in this case multiple read-throughs and I'd say > 5 minutes.

Good documentation should do this work for you. It should explain somewhat atomic concepts to you, that you can immediately adapt, and compose. Where it already works is for the "detection" and "window" parameters, which are straightforward. But the actions of trimming in the start/middle/end, and how to configure how long the silence lasts before trimming, whether to ignore short bursts of noise, whether to skip every nth silence period, these are all ideas and concepts that get mushed together in 10 parameters which are called start/stop-duration/threshold/silence/mode/periods.

If you want to apply this filter, it takes a long time to build mental models for these 10 parameters. You do have some example calls, which is great, but which doesn't help if you need to adjust any of these - then you probably need to understand them all.

Some stuff I stumbled over when reading it:

"To remove silence from the middle of a file, specify a stop_periods that is negative. This value is then treated as a positive value [...]" - what? Why is this parameter so heavily overloaded?

"start_duration: Specify the amount of time that non-silence must be detected before it stops trimming audio" - parameter is named start_something, but it's about stopping? Why?

"start_periods: [...] Normally, [...] start_periods will be 1 [...]. Default value is 0."

"start_mode: Specify mode of detection of silence end at start": start_mode end at start?

It's very clunky. Every parameter has multiple modes of operation. Why is it start and stop for beginning and end, and why is "do stuff in the middle" part of the end? Why is there no global mode?

You could nitpick this stuff to death. In the end, naming things is famously one of the two hard problems in computer science (the others being cache invalidation and off-by-one errors). And writing good documentation is also very, very hard work. Just exposing the internals of the algorithm is often not great UX, because then every user has to learn how the thing works internally before they can start using it (hey, looking at you, git).

So while it's easy to point out where these docs fail, it would be a lot of work to rewrite this documentation from the top down, explaining the concepts first. Or even rewriting the interface to make this more approachable, and the parameters less overloaded. But since it's hard work, and not sexy to programmers, it won't get done, and many people will come after, having to spend time on reading and re-reading this current mess.

In the meantime I realized that the apad part is nonsensical - it pads the end of the stream, not at each silence-removed cut. I wanted to get angry at o3 for proposing this, but then I had a look at the silenceremove= documentation myself: https://ffmpeg.org/ffmpeg-filters.html#silenceremove

Good god. You couldn't make that any more convoluted and hard-to-grasp if you wanted to. You gotta love ffmpeg!

I now think this might be a good solution:

    ffmpeg -i video-audio.m4a \
           -af "silenceremove=start_periods=1:stop_periods=-1:stop_duration=0.15:stop_threshold=-40dB:detection=rms" \
           -c:a aac -b:a 128k output.m4a -y

With transcribing a talk by Andrej, you already picked the most challenging case possible, speed-wise. His natural talking speed is already >=1.5x that of a normal human. One of the people you absolutely have to set your YouTube speed back down to 1x when listening to follow what's going on.

In the idea of making more of an OpenAI minute, don't send it any silence.

E.g.

    ffmpeg -i video-audio.m4a \
      -af "silenceremove=start_periods=1:start_duration=0:start_threshold=-50dB:\
                         stop_periods=-1:stop_duration=0.02:stop_threshold=-50dB,\
                         apad=pad_dur=0.02" \
      -c:a aac -b:a 128k output_minpause.m4a -y

will cut the talk down from 39m31s to 31m34s, by replacing any silence (with a -50dB threshold) longer than 20ms by a 20ms pause. And to keep with the spirit of your post, I measured only that the input file got shorter, I didn't look at all at the quality of the transcription by feeding it the shorter version.

What specifically about this paper caught your eye that you wanted to implement that, what does it do better than other methods? Can you give a quick primer on what it does, and what the optional kmeans refinement does?

That’s not quite right for Fraunhofer: The financing model is also heavily dependent on research grants. The base funding is indeed around 30%. But another 40% are from publicly funded research projects. The last 30% are from research contracted by industry. In 2023, license fee revenue was €157M / €2991M, so roughly 5% of total contract research volume.

https://www.fraunhofer.de/en/about-fraunhofer/profile-struct...

It matches my anecdotal experience here (from Germany). When talking about travelling to the US, previously the discussions of the drawbacks would circle around climate impact, expense or flight time. Recently, this has shifted to feeling unwelcome or unsafe. I talked to two different people last week who had gone to the US regularly in the past years, who have decided to not go this year (attending a wedding and the other for touristic reasons), due to the current political climate.

Certainly possible, read 4.6. Finetuning for Downstream Tasks in the paper, the first subsection is "Feed-forward Novel View Synthesis". They chose to report their experiments on LVSM, which is not an explicit representation like 3D Gaussian Splatting, but they're citing two feed-forward 3DGS approaches in their state of the art listing.

Should be quite exciting going forward, as fine-tuning might be possible on consumer hardware / single Desktop machines (like it is with LLMs). So I would expect a lot of experiments coming out in this space, soon-ish. If the results hold true, it'll be pretty exciting to drop slow and cumbersome COLMAP processing and scene optimization for a single forward pass that lasts a few seconds.

I read the paper yesterday, would recommend it. Kudos to the authors for getting to these results, and also for presenting them in a polished way. It's nice to follow the arguments about the alternating attention (global across all tokens vs only the tokens per camera), the normalization (normalize the scene scale - done in the data vs DUST3R, which normalizes in the network), and the tokens (image tokens from DINOv2 + camera tokens + additional register tokens, handling the first camera differently as it becomes the frame of reference). The results are amazing, and fine-tuning this model will be fun, e.g. for forward 3DGS reconstruction, looking forward to this.

I'm sure getting to this point was quite difficult, and on the project page you can read how it involved discussions with lots and lots of smart and capable people. But there's no big "aha" moment in the paper, so it feels like another hit for The Bitter Lesson in the end: They used a giant bunch of [data], a year and a half of GPU time to [train] the final model, and created a model with a billion parameters that outperforms all specialized previous models.

Or in the words of the authors, from the paper:

> We also show that it is unnecessary to design a special network for 3D reconstruction. Instead, VGGT is based on a fairly standard large transformer [119], with no particular 3D or other inductive biases (except for alternating between frame-wise and global attention), but trained on a large number of publicly available datasets with 3D annotations.

Fantastic to have this. But it feels.. yes, somewhat bitter.

[The Bitter Lesson]: http://www.incompleteideas.net/IncIdeas/BitterLesson.html (often discussed on HN)

[data]: "Co3Dv2 [88], BlendMVS [146], DL3DV [69], MegaDepth [64], Kubric [41], WildRGB [135], ScanNet [18], HyperSim [89], Mapillary [71], Habitat [107], Replica [104], MVS-Synth [50], PointOdyssey [159], Virtual KITTI [7], Aria Synthetic Environments [82], Aria Digital Twin [82], and a synthetic dataset of artist-created assets similar to Objaverse [20]."

[train]: "The training runs on 64 A100 GPUs over nine days", that would be around $18k on lambda labs in case you're wondering

Sure they can. o3-mini can do web searches, which puts it far ahead of o1 if you require current information. You can also tell it to go read a particular paper from just the rough name.

If you have things organized neatly together, you can also use pre-existing compression algorithms, like JPEG, to compress your data. That's what we're doing in Self-Organizing Gaussians [0]. There we take an unorganised (noisy) set of primitives that have 59 attributes and sort them into 59 2D grids which are locally smooth. Then we use off-the-shelf image formats to store the attributes. It's an incredibly effective compression scheme, and quite simple.

[0]: https://fraunhoferhhi.github.io/Self-Organizing-Gaussians/

So they want to build a hollow structure, 1 km high, with all its weight concentrated at the very top (when charged)? How is that supposed to not immediately collapse?

These are the slides for the talk "The Aging Programmer" by Kate Gregory at CppNorth. Here's the recording of the talk: https://www.youtube.com/watch?v=LArOT95LTJU&list=PLsAtvvJ8KX...

This criticism isn't compelling to me.

The smiling emoji at the end of the first paragraph indicates that these statements were made somewhat in jest, or perhaps exaggerated. Of course some uses can be found for absolute pitch. I saw one a couple weeks back, when Jacob Collier was tuning the audience choir to lead into "Somebody to Love" played on the piano. But, hadn't he had absolute pitch, he might just have picked up a reference note from the piano or his in-ear monitors, like a filthy commoner. Usually when making music, having good relative pitch is required, and a reference instrument is mostly handy, making perfect pitch somewhat redundant. But do tell what you're doing with perfect pitch, I'm curious.

And on curiosity, I went to your website and randomly listened to "The Fugue Song" [0]. Really loved it! Very nice moment when the singing comes in, repeating the phrase from the fugy guitar intro. Good song! (I'm a total sucker for Nina Simone's "Love Me Or Leave me", do you know that? A song where she's inserting some counterpoint improvisations in the middle). I'm listening to a bit of "Hiss" now.

> 8.3%

Rounded to the next semitone of course, I left that detail out, it's in the paper.

[0]: https://josephweidinger.com/project/the_fugue_song/

Absolute pitch: a completely useless skill, which having can in some cases even be detrimental. While being very hard to impossible to acquire. So naturally I will stop at nothing trying to develop it :)

A couple of months ago, this paper made the rounds: Absolute pitch in involuntary musical imagery [0]. In a small sample group, nearly half the time (44.7%) when someone was asked to sing their current earworm, were they perfectly in pitch. Random chance would be 8.3%.

It’s a fun thing to try for yourself. Just hum your current earworm into a voice memo, and check the correct pitch against the recording of the original song. You may discover a skill you never knew you had, implicit perfect pitch on involuntary music!

Trying to make this more interesting, reproducing a particular song on demand (there’s references to that too in the paper - it also works better than random chance, but less so than the involuntary kind), I find it works best for songs that start off with a single note, preferably sung. Or then at least you can immediately check whether you were right, e.g. “Tom’s Diner”. I’ve been having a lot of fun humming the first tone to Laufey’s cover of Sunny side of the street [1] whenever I open YouTube. I’m more often right than wrong, and if I was wrong, I can just listen to the whole thing to brighten my day anyways.

[0]: https://link.springer.com/article/10.3758/s13414-024-02936-0

[1]: https://youtu.be/wK6gbKC90Ps

If you are a regular HN reader who is (or was until this post) unfamiliar with Back to the Future, I'd love to know three more random facts about your life. In my world view, you are part of a fascinatingly small group of people.

Interesting tool, congrats on the launch!

I was wondering: have you thought about automation bias or automation complacency [0]? Sticking with the drop-tables example: if you have an agent that works quite well, the human in the loop will nearly always approve the task. The human will then learn over time that the agent "can be trusted", and will stop reviewing the pings carefully. Hitting the "approve" button will become somewhat automated by the human, and the risky tasks won't be caught by the human anymore.

[0]: https://en.wikipedia.org/wiki/Automation_bias

I like the app and don’t need any customization options. Downloaded it after an earlier HN discussion I think.

The workflow of typing, then selecting a file name to save at the very end makes me somewhat uncomfortable. What happens if the app or the computer crashes? Will everything be gone?

This stops me from writing for really long sessions in there. Don’t want to type for an hour only to find everything is gone.

Wait, why do you have the same living room as Bojack Horseman?

Bat echolocation is a topic that's very personal to me. My grandpa worked as a researcher in telecommunications and fibreoptics in the 1970s, in East Germany (GDR). This being at the height of the cold war, it was completely prohibited to publish anything relevant to their actual work, so that the "class enemy" wouldn't be the wiser of what they were doing in the lab. When publishing anything, they had to water down the content so much that it became useless to anybody reading it. Also, publishing on data transmission with fibreoptics, while nobody in the country is able to get a telephone line in their homes, wasn't seen as helpful.

It was sometimes possible to travel to western/international conferences - of course only after the Stasi gathered from interviews with all your colleagues and neighbors that you loved your children, and were likely to come back. And even then they were not allowed to make any contact with western scientists at the conferences, making any kind of useful exchange near-impossible.

After finding and reading a book on Bionics, which was a bit of a hype topic at the time, he noticed that the biologists were talking about the inexplicable nature of bats' echolocation. As he had experience with code multiplexing, he wrote a paper on the signal theory behind that, in 1974 [0]. As this topic was considered completely harmless, it was no problem to get this published in the GDR. After a request, it was even published in a journal in West Germany. But then being invited to give a talk on it in Frankfurt/Main, the authorization was given three times by the institute, and then revoked again each time (by higher-ups/Stasi). He was then asked to cease any communication with West-German researchers (who had sent him a book manuscript to review). It was a bitter-sweet thing for him. In later years, he was often asked about bat echolocation, as it made for a good pop science topic. But of course he would have loved to publish, and exchange, on his actual research topics at the time much more.

My grandpa died a couple of years ago. When I travel to international conferences now - being able to present whatever I like, talking to whoever I choose - this history often comes back to mind. Sometimes these stories feel like a very distant past. But then I read about how Linux maintainers are removed just for having a Russian email address, as happened recently. So maybe we haven't come as far as we think. I believe it's important to be conscious of the great freedoms most of us currently enjoy, how precious they are, and how brittle they can be.

[0]: https://link.springer.com/article/10.1007/BF00611870 - paywalled, unfortunately. And German!

Don't make fun of renowned author Dan Brown!

> Renowned author Dan Brown woke up in his luxurious four-poster bed in his expensive $10 million house – and immediately he felt angry. Most people would have thought that the 48-year-old man had no reason to be angry. After all, the famous writer had a new book coming out. But that was the problem. A new book meant an inevitable attack on the rich novelist by the wealthy wordsmith’s fiercest foes. The critics.

> Renowned author Dan Brown hated the critics. Ever since he had become one of the world’s top renowned authors they had made fun of him. They had mocked bestselling book The Da Vinci Code, successful novel Digital Fortress, popular tome Deception Point, money-spinning volume Angels & Demons and chart-topping work of narrative fiction The Lost Symbol.

> The critics said his writing was clumsy, ungrammatical, repetitive and repetitive. They said it was full of unnecessary tautology. They said his prose was swamped in a sea of mixed metaphors...

https://onehundredpages.wordpress.com/2013/06/12/dont-make-f...

Or mix together your own plot, by combining any of the tropes in the "Periodic Table of Storytelling": https://jamesharris.design/periodic/

Oh wow, really? Even if you feed it whole research papers? The ones I tried until now were more in the 8-10 minute range. I haven’t looked in to how to control the output yet. Hopefully that’ll get a little more transparent and controllable soon.

If you're not bored with it yet, here's a Deep Dive (NotebookLM, generated podcast). I fed it the project page, the arXiv paper, the GitHub page, and the two twitter threads by the authors.

https://notebooklm.google.com/notebook/a240cb12-8ca1-41b4-ab... (7m59s)

As always, it's not actually much of a technical deep dive, but gives a quite decent overview of the pieces involved, and its applications.

Rollout otherwise completed: https://x.com/sama/status/1838864011321872407

As a paying customer from EU, I’m not happy about it. What’s the reasoning for this being held back (while normal voice mode is available)?

I don’t think many people would have thrown the OZO up to take cute panoramas though, as it went for something like $60,000.. At the time (~2017), the OZO was the benchmark quality-wise, and it hit the community pretty hard, when Nokia cancelled the project over night and disbanded/fired the team behind it.

At the time I was working for a startup called VideoStitch, later Orah, which had launched the Orah 4i, a much smaller, 4-lens, 360 degree 4K live streaming camera. Expectations were huge, that 360 degree video streaming would go through the roof, with a camera at every wedding. Didn’t really work out that way, I wonder whether stereo 180 degree “Spatial Videos” in the Apple ecosystem will go the same way.

Throwable 360 cameras owe their heritage to Jonas Pfeil, who built one for his thesis at TU Berlin [0], and later commercialized the idea in a camera called Panono [1]. They never really went mainstream, but it was a really cool idea with lots of buzz back then (10+ years ago).

[0] https://www.jonaspfeil.de/ballcamera/ [1] https://www.theverge.com/2013/11/12/5092670/panono-camera-ba...

The data on which authors are part of which arxiv papers is already in the arXiv database, and in Google Scholar, and in other libraries. I appreciate that it's not an easy task to get that as a third party. But the burden should be on the operators of alphaxiv to figure out a solution for this platform to take off, not for me as a user?

OP here, what I'm actually using is "{first}.{last}@hhi.fraunhofer.de" (https://arxiv.org/pdf/2312.13299). I see how my earlier comment was confusing.

In our case it's for saving space in the paper, and also for reducing spam. This small change may now seem silly in the age of LLMs, but the papers that have full email addresses in them get a considerable amount of fake conference and journal participation emails, which is annoying.

Hey alphaxiv, you won’t let me claim some of my preprints, because there’s no match with the email address. Which there can’t be, as we’re only listing a generic first.last@org addresses in the papers. Tried the claiming process twice, nothing happened. Not all papers are on Orcid, so that doesn’t help.

I think it’ll be hard growing a discussion platform, if there’s barriers of entry like that to even populate your profile.