Hacker News Reader

Comment by jemfinch | Hacker News Reader

This is one of the dumbest things I've read in a long time. Algorithms are not where the real value is: that's why you can pull them off shelves at the bookstore. Algorithms are public knowledge, commodities to be written and rewritten at will by common, workaday developers.

Data is where the value is, because you're the only who has it. It's your competitive advantage. It's your secret stash that no one else can access. And if you're willing to ship it halfway around the world to a microservice provider based on flashy marketing, well, I've got a bridge to sell you before your VC money runs out.

View on HN

Disregarding the article, I sort of disagree with your comment too, maybe you are oversimplifying it too much. Or maybe because it's kind of a chicken and the egg kind of problem - which came first, the algorithm or the data.

Sure, generic sort algorithms come "off-the-shelf", but that is not true for all algorithms. Not all algorithms are public knowledge if they are patented, proprietary, or restricted in use/distribution.

You really need both - algorithms and data, right? And systems of execution for algorithms... and networks to interconnect the systems... and potentially humans to review the algorithm's output... and..... and... and..

I think the key is, algorithms can also represent embedded capabilities (information? knowledge? insight? not sure what to call it) generated from data. I mean, look at a RNN - the values/weights/etc. calculated inside the model are calculated from observations/training on the data. Once you disconnect the algorithm from the data itself, the algorithm alone now has embedded value without the data, potentially enormous.

Or, expert/rule-based systems which contain algorithms explicitly designed to embed human knowledge - those algorithms may not even be trained on actual data, they are simply a simplified expression of codified human knowledge.

If you go one step further and try to make some assumption that the way our brains work is by codifying inputs we receive into algorithmic models... and then using those algorithms to calculate based on new inputs from the environment... we can't keep all the data in our heads, but we can probably keep more algorithms than data.

Maybe I'm being too philosophical, but in my opinion, it's not just about the data.

Very interesting viewpoint. Any algorithm can be replaced by data and a lookup (e.g. Rainbow or multiplication tables.) On the opposite direction, the claim that any data can be replaced by an algorithm is indeed deeply philosophical (c.f. Materialism and free will)

If we want to go deeply philosophical, then fundamentally data and algorithms are the same thing anyway. Whether you interpret something as data or as code depends only on the angle you're approaching a problem at.

From the perspective of a Turing machine, they're the exact same thing. We just created abstractions to separate them. And from the perspective of physics, you've just got yourself yet another isomorphic transformation of the above mentioned Turing machine, so still the same.

Have you got an algorithm to produce my production database?

Philosophically you can reduce them to the same thing, but practically speaking you wouldn't really want to.

Yes, Data is a competitive advantage. However, there ARE plenty of companies who do not know how to interpret that data. Especially companies who are NOT "tech" companies (Home Depot, Ace Hardware).

Hence, companies like Palantir and Pivotal that provide this service. To say that the ability to interpret data is ubiquitous is simply wrong.

This is exactly where SAP/IBM/Oracle are making crazy money. It is not a new untouched market, plenty of big and small players.

    Algorithms are public knowledge, commodities to be written and rewritten at will by common, workaday developers.

I don't think the article is talking about sorting a linked list. Clever and non-trivial algorithms are being researched and discovered everyday both in academia and the industry (take ML as an example). Programs = Algorithms + Data (Structures). You can have a competitive advantage on either or both.

Look past the marketing. They aren't selling "we run algorithms on your data" they're selling "we run algorithms on all your data". There is still a large value proposition in managing large data and until something like GCE manages to completely remove the liability of bad developers using its platform, there will be room for these types of services IMO.

Totally agree.

Facebook - good luck having a news feed algorithm with no actual stories and status updates

Google - good luck having a search algorithm with no actual search index

Amazon - good luck recommending products to others if you zero data on what others are recommending.

With all 3, you can at least create a minimum viable algorithm if you have the data.

It's not that black and white. Sure, all the algorithms I use are available in published papers and often have open source implementations, but knowing exactly which combinations of algorithms to use on which datasets is a real value. As is knowing exactly how best to tweak those algorithms when the underlying dataset changes. Furthermore running a bunch of algorithms on some data is only the first and easiest step. Interpreting the results of that and then deciding what action, if any, to actually take is often the real challenge.

That being said I agree that having access to data that no one else has is a massive competitive advantage and with your skepticism about the concept being presented.

> Data is where the value is, because you're the only who has it.

For those of us who have no data, algorithms are the most valuable things.

Control (i.e. 'moats') is where the real value is, always has been and always will been for all businesses.

E.g. with Google Search it is was perhaps the algo at first but now I'd say it is partly that and partly the high barrier to entry in obtaining all the data (although the data itself is freely available)

Reply on news.ycombinator.com