Weber–Fechner Law
en.wikipedia.org
en.wikipedia.org
So, I tend not to release performance improvements one-at-a-time. That's a career-limiting move in the passive sense. Instead, I batch performance tuning changes and release several at the same time, which provides a much bigger "bang" for the users. The next morning, they'll notice, write emails to managers with words like "Wow!" in it, and then you get promoted.
The other side of this is that batched changes are delayed, can cause conflicts with other development, and are riskier.
I’ve always thought that it’s also a powerful argument against the concepts of Heaven and Hell, as described in many religions, I.e. it’s impossible to continuously torture people or keep them ecstatic.
I think that’s an easier explanation for why we’re biased against it that isn’t necessarily tied to why the brain uses a logarithmic response curve. The latter could be because it provides better dampening to random excitations (ie the brain doesn’t have to expend as much energy dealing with them at the cost of missing signal in lower energy levels). Of course it could be this is why we’re bad with exponential but that seems like a larger leap to make.
That's true, but not really saying very much. Any differentiable function is locally linear around a neighborhood of any point where the derivative exists.
> Also exponential growth does hit some kind of ceiling relatively quickly.
Well... that depends. Much like how markets can remain irrational longer than you can remain solvent, exponential growth can often remain exponential for much longer than it takes to create a problem. Conversely, sometimes it can't remain exponential long enough to prevent a problem. Exponential growth is a hard beast to tame.
What I’m saying is that it’s hard to know if you are dealing with an exponential scenario. You’d respond differently more quickly but doing so for a linear function may be the wrong response. There are certain failure modes when you do encounter an exponential but most things we encounter are more linear / dampened so the bias humans have against exponential is rational despite the failures we have dealing with exponential problems (climate change being a notable counter example). I’m saying it’s a rational trade off to evolve when dealing with the world.
Can’t remember the link. I might have found it through HN.
> They were first published in 1860
> Perceived loudness/brightness is proportional to logarithm of the actual intensity
The example: knowing whether there’s one or two lions in the bushes is a lot more important than distinguishing eight lions from nine!
But I've heard that between a half-marathon and a full marathon there's 'the wall'. A qualative change in difficulty which makes a marathon much harder than running a half-marathon twice.
Weber–Fechner law - https://news.ycombinator.com/item?id=17301360 - June 2018 (6 comments)
Weber–Fechner law - https://news.ycombinator.com/item?id=5240806 - Feb 2013 (1 comment)
Case in point: Early Google search box and subsequent changes to the Google home page.
When features get tacked on to an app, users may just give up due to cognitive overload. Case in point Microsoft Word features or the current Google apps tucked under a link.
I think the same goes for neural nets. Numerical features are often provided in some scaled manner anyway (e.g. not “how many cents has this stock price fallen in the last second” but “what is the ratio between the price now and the price a second ago?”, or even “what is the ratio of the stock’s price change in the last second, represented as a number of standard deviations from the mean in this background data?”.
And then there’s the fact that a (useful) neural network isn’t linear to begin with (if it was, it’d just be a simple matrix transformation). Each layer has an activation function. None of those I’m familiar of are even remotely logarithmic, but e.g. tanh and sigmoid functions are more sensitive to small changes at values near 0 than values far from zero. Perhaps over many layers it kind of resembles something kind of logarithmic? IDK