Hacker News Reader

beckhamcon Why don't machine learning research agents overfit?

And sadly, in academia, complexity (opposite of Occam's razor) is what gets you published.

beckhamcon I-have-ADHD: A skill to stop coding agents from burying the answer

Pretty much this. I actually find this more annoying than the way it writes, but that is also annoying as well.

beckhamcon Our decision on Cursor following its acquisition by SpaceX

I am also wondering the same thing as OP. Somewhat tangential thought, but I recently realised that what I would really love in an IDE is a "first class" Markdown editing experience, just because most of my day-to-day is managing and writing docs rather than editing code. Cursor has Markdown rendering support of course but it seems to have two implementations, each sort of clunky in their own way. And I want something more WYSIWYG but with Latex / flowchart / etc. support built-in.

OpenKnowledge seems to maybe fill that niche somewhat (it also supports annotating Markdown with comments to feed into an agent), and has agent integration within the GUI, but one killer feature that seems to be missing is Remote SSH support (short of hosting its web server on the remote and having the IDE live in the browser).

beckhamcon Our decision on Cursor following its acquisition by SpaceX

This may be of use to you, I wrote a similar reply earlier in the thread: https://news.ycombinator.com/item?id=49486172#49491426

beckhamcon Our decision on Cursor following its acquisition by SpaceX

I was looking at CLI-only solutions a while ago and the best thing I found was to use something like revdiff: https://github.com/umputun/revdiff

For instance, in Claude Code you can have it bring up this TUI overlay after every edit it makes (it literally is an overlay, rather than a separate tab in your terminal) and you can annotate line-by-line anything you want changed or rejected. Then once you exit the overlay those suggestions are automatically piped back into the agent.

I tried a few of these TUI-based code annotators, and this one is by far the best. It also works for files not under version control.

I'm currently not using it as I also prefer something more akin to "text editor with agent built in", but I wanted to mention here in case you were curious.

beckhamcon Our decision on Cursor following its acquisition by SpaceX

Interesting to see someone else is in the same boat as I am. Regarding editor review I agree, surely the solution would be to `git diff` at the end, but if you only want to git diff what was literally just changed by the agent it means you'd have to commit as you go so that your git diffs are more localised. And then at the end of your work just squash the commits. But it feels annoying. (EDIT: ok I forgot about git staging, that seems to be better)

I mainly use Cursor for another reason though: all of my work is remote over SSH and I don't like keystroke latency, so I prefer code editing / agent conversations to feel local. (Everything else is done in the terminal.) But I would also prefer to use something less propriety. I may go back to VSCode with plugins to achieve a similar experience.

beckhamcon Questionable Practices in Machine Learning

I'm not sure if this is directly mentioned in the paper, but I didn't see any mention specifically about the conflation between a validation set and test set. When people actually make a distinction between the two (which is seemingly not all that common nowadays), you're meant to perform model selection on the validation set, i.e. find the best HPs such that you minimise `loss(model,valid_set)`. Once you've found your most performant model according to that, you then evaluate it on the test set once, and that's your unbiased measure of generalisation error. Since the ML community (and reviewers) are obsessed with "SOTA", "novelty", and bold numbers, a table of results purely composed of test set numbers is not easily controllable (when you're trying to be ethical) from the point of view of actually "passing" the peer review process. Conversely, what's easily controllable is a table full of validation set numbers: just perform extremely aggressive model validation on your model until your model gets higher numbers than everything else. Even simpler solution, why not just ditch the distinction between the valid and test set to begin with? (I'm joking, btw.) Now you see the problem.

beckhamcon Were RNNs all we needed?

Your description of tanh isn't even correct, it squashes a real number to `(-1, 1)`, not "less than one".

You're curious about whether there is gain in parameterising activation functions and learning them instead, or rather, why it's not used much in practice. That's an interesting and curious academic question, and it seems like you're already experimenting with trying out your own kinds of activation functions. However, people in this thread (including myself) wanted to clarify some perceived misunderstandings you had about nonlinearities and "why" they are used in DNNs. Or how "squashing functions" is a misnomer because `g(x) = x/1000` doesn't introduce any nonlinearities. Yet you continue to fixate and double down on your knowledge of "what" a tanh is, and even that is incorrect.

beckhamcon Were RNNs all we needed?

Learnable parameters on activations do exist, look up parametric activation functions.

beckhamcon Were RNNs all we needed?

How was that person derailing the convo? Nothing says an activation function has to "squash" a number to be in some range. Leaky ReLUs for instance do `f(x) = x if x > 0 else ax` (for some coefficient `a != 0`), that doesn't squash `x` to be in any range (unless you want to be peculiar about your precise definition of what it means to squash a number). The function takes a real in `[-inf, inf]` and produces a number in `[-inf, inf]`.

> Sure there's a squashing function on the output to keep it in a range from 0 to 1 but that's done BECAUSE we're just adding up stuff.

It's not because you're "adding up stuff", there is specific mathematical or statistical reason why it is used. For neural networks it's there to stop your multi layer network collapsing to a single layer one (i.e. a linear algebra reason). You can choose whatever function you want, for hidden layers tanh generally isn't used anymore, it's usually some variant of a ReLU. In fact Leaky ReLUs are very commonly used so OP isn't changing the subject.

If you define a "perceptron" (`g(Wx+b)` and `W` is a `Px1` matrix) and train it as a logistic regression model then you want `g` to be sigmoid. Its purpose is to ensure that the output can be interpreted as a probability (given that use the correct statistical loss), which means squashing the number. The inverse isn't true, if I take random numbers from the internet and squash them to `[0,1]` I don't go call them probabilities.

> and not only is it's PRIMARY function to squash a number, that's it's ONLY function.

Squashing the number isn't the reason, it's the side effect. And even then, I just said that not all activation functions squash numbers.

> All the training does is adjust linear weights tho, like I said.

Not sure what your point is. What is a "linear weight"?

We call layers of the form `g(Wx+b)` "linear" layers but that's an abused term, if g() is non-linear then the output is not linear. Who cares if the inner term `Wx + b` is linear? With enough of these layers you can approximate fairly complicated functions. If you're arguing as to whether there is a better fundamental building block then that is another discussion.

beckhamcon Ask HN: Former gifted children with hard lives, how did you turn out?

ACE 7, recently got my PhD in ML and am doing reasonably well, at least from an 'objective' lens (I have a well paying job and bought my first condo recently). My mind is still chaotic as ever (I have a mix of OCD / Tourettes / PTS) and it's still often hard for me to concentrate without overthinking some portion of my life or something completely unrelated to the task at hand.

If I can plug a Youtuber who really 'gets it' (mental health and depression), it's Dr Scott Eilers. Everything else is just clichéd garbage.

(Disclaimer: I am not 'gifted' but did very academically well during high school and uni)

beckhamcon Ask HN: Machine learning engineers, what do you do at work?

no we just replaced feature engineering with architectural engineering

beckhamcon Why is machine learning 'hard'? (2016)

The issue is the obsession with benchmark datasets and their flaky evaluation

beckhamcon John Carmack and John Romero reunited to talk DOOM on its 30th Anniversary

sophisticated AI != aimbot accuracy

beckhamcon Yabai – A tiling window manager for macOS

I wish I had the ability to toggle PiP for any open window, while I am in full screen mode. For instance, I have both Chrome and Emacs side by side full screen and I can use a hotkey to drop down my iTerm window over both of them. (Basically like a Quake terminal but that feature is specific to iTerm).

beckhamcon Ask HN: Could you share your personal blog here?

http://beckham.nz/blog

Mostly theoretical-ish deep learning stuff as of late (I'm a PhD candidate in that field). But I want to expand it into really anything: psychology, dating, video game reviews, etc.

beckhamcon P < 0.05 Considered Harmful

A relevant quote here seems to be: "if a measure becomes a target, it ceases to become a good measure". Also true of academia in general because of the obsession with citations.

beckhamcon Weka Violates MinIO's Open Source Licenses

Not to be confused with open source ML toolkit WEKA: https://en.m.wikipedia.org/wiki/Weka_(machine_learning)

Huh, that's funny, I guess which one came first?

beckhamcon TabFS – a browser extension that mounts the browser tabs as a filesystem

Nice work! Is it possible to somehow switch to a tab somehow using this? (Use case: switching browser tabs within Emacs). Thanks