An App Store for Algorithms | Hacker News Reader

An App Store for Algorithms

wired.com

101 pointsdoppenhe45 comments

Personal opinion, but I think this is probably good for professors / post-docs, but a horrible idea for the software community as a whole. Algorithm patenting and closed gardens are probably the worst hinderances to software development that exist right now. It feeds the big boys, who can easily pay, while imposing additional taxes on new folks. Marching Cubes is the classic example of this, but its easy to think of others. Lets say they pattent holographic image recognition / stabilization, or O(N) methods for N-body simulation, or pattern recognition that order of magnitude outstrips SVD or K-means methods. Then everything will have a choice: pay them their pound, use it without paying, or find a functionally similar version that is so different as to not terrify everyone of lawsuit hell. Marching cubes, you only managed to set graphics back by 17 years, meet marching tets.

Agreed. I was just just about to throw money at them to read the algorithm implementations - then I realised there is no code to read.

actually many algorithm developers choose to share their source code on our platform. They still have an opportunity of making money because when our users run their algorithms (even if they are open source) on our platform they get paid.

price discrimination is a potential solution: free for the little guy; or, royalty (per usage).

e.g. game engines being free til you hit significant volume.

I can't help but have a strongly negative opinion of anybody who would use this service.

IMO implementing algorithms is what computer programming and software development are all about. It's the fun part of the job. A person buying an algorithm as a web service makes me wonder why the person even became a software engineer at all.

Not to mention that if this idea takes off, it will ruin it for the rest of us by encouraging algorithm researchers to patent new algorithms and not publish them.

I do wonder, though, if a person won't look for a freely available, openly published algorithm, why would they look at this service for an algorithm they have to pay for?

The competent software engineer would understand that spending a week implementing a buggy, ill-optimized version of the algorithm would cost more than the purchase price could ever be (unless it's a ridiculously high price like $100K). The fact that it would be "fun" is probably a hard sell towards the client.

I guess I made my point badly.

Implementing algorithms is what software developers do. If, as a software developer, I'm just a middle man between the client and the people implementing everything on algorithmia.com, it's kind of stupid for me to even be involved - the client should hire the people posting to algorithmia directly.

No, you just had a bad point that you'll come to realize if you ever have to manage a team or company.

More often than not, you outsource items that are not the highest value projects your resources could be spent on. Or, you simply cannot afford to hire a specialist for a specific need. Try to find me experts for some of the pattern recognition algorithms offered who will work at your shitty startup, your shitty big-co with minimal tech wing, or your random mid-sized in the middle of nowhere. I just described 95% of companies on the planet.

And to second relix's point, try to justify implementing something that may have taken years to optimize, even if you think you can do it better. Also, do you want to maintain it? What happens if/when the company loses you? Would they rather have some poorly documented, poorly optimized in house version with nobody around who knows how it works or have a contract with another party they can count on for maintenance and support?

Is this always the case? No. Sometimes it's a core function you want to develop in house. But, it's usually considered.

As a developer, a big step in growth is realizing how to best spend your time and resources, and realizing that contracting implementation of X to further speed up building Y is a choice you'll often have to make.

The condescending tone is completely uncalled for.

Fair enough. Otherwise, does that help shed any light on this side? Thoughts?

There's a balance to be had though with every project between IP/licensing concerns, the implementation of the algorithm itself may not be appropriate for all hardware / software combinations, and cost vs. implementation time and complexity of algorithm.

Component reuse is generally preferable to re-implementation, but it's not a blind creed to follow.

You also have to account for the additional cost you'll be paying for data transfer and http requests to Algorithmia's API. I doubt a week of a developer's time is more expensive than a lifetime of those expenses and the fees Algorithmia charges.

But writing your own algorithms is so last week bro!

I have to agree. I can imagine the exact type of person to make use of such a service, and can see a considerable hype following becoming the market. The creators no doubt had the best of intentions in mind, but I think it does prey on the uniformed, even if unintentionally.

I'm also wondering who exactly would use this service. Any worthy developer should be able to read a description / paper on an algorithm he needs and then implement it in their language of choice.

Also, a bulk of the work would probably go to massaging your input to satisfy the input constraints of this web service. If you go that far, you might as well implement the rest of the algorithm yourself.

Maybe instead of an Amazon-style store, it could be more of craigslist/elance/topcoder type of store – people submit the exact classes/data/input they have, with the desired output. Developers write code (and the platform checks input/output validity), and the submitter can then haggle with the coder with the best score for the fee to pay. I guess it could be used as an outsourcing tool for companies that can't be bothered or don't have the brains to develop these kinds of things. Just an idea.

You bring an interesting point specially because we built this service for developers. Algorithmia was created to allow subject matter experts to provide their knowledge and share/sell it to the world. An expert in computer vision might not be an expert in path planning algorithms but both of these can be combined for things like autonomous vehicles. The goal is to allow deep individual specialization (as most researchers/algorithm devs want to do) and yet be able to leverage the deep specialization of others. Algorithms in Algorithmia can call other algorithms in the library directly which allows for the creation of new innovative meta solutions. I don't believe this takes away at all from the developer experience.

http://en.wikipedia.org/wiki/Not_invented_here is so 20th Century

Here's how I'd like this to work. Each algo is implemented server side on well specified hardware. I can upload sample data to benchmark the algorithm. After reviewing the results I could then pay a small amount for the source code in a language of my choosing (or language + bindings).

All code would be unrestricted and the value-add would be in the search, review, and benchmarking functionality of the service.

This is the same model that makes me pay Amazon for instant video rather than nothing to TPB.

This is great feedback thanks!

The project listed in the article, http://algorithmia.com/ looks like it's a web service.

Anyone that's used/written algorithms knows they need to be run natively on your machine/server to run the fastest. Calling a web service will slow this down greatly.

If this site was turned into a marketplace of code listed for each algorithm that can automatically translate between languages, I could see it going somewhere.

It's not just a web service -- it's any web service. The algorithm is up to you.

I'm not sure what value they're providing, exactly. Anyone can put algorithms on the web now: you write them in a standard language like PHP and upload them to any hosting service.

Just because it's using a web service doesn't mean the algorithm itself isn't running efficiently on hardware.

I imagine you ship off the data, let them run the algorithm, and they return the results. This has no effect on the Big-O efficiency of the algorithm; the only additional cost is a simple roundtrip of the data.

I'm failing to see how that's so different from ex. running an intense algorithm in a worker thread.

This would see a significant slowdown for any algorithm that requires large amounts of data, as a lot of them do. The core philosophy of Hadoop which made it successful was "move the computation to the data and not the other way around."

Although the algorithms may run efficiently, ferrying significant amounts of data to and from a web service could quickly negate any gains.

absolutely agreed which is way in almost 80% of cases we ship the computation to the data by running in the same data centers where the data lives.

I'm thinking back to some image processing processes I've implemented that had multiple algorithms. Passing images to a service like this back and forth would slow down those processes greatly. YMMV.

From the article: "The creators will also have the option of making their algorithms open source, so developers will be able to integrate them directly into their own applications"

The creators already have the option of making their algorithms open source.

True. But maybe if you create an O(n^4) algorithm, and the site has an O(n log n) algorithm, then it could take far less time sending it over. But that's all conjecture..

If it's running on AWS , isn't it possible to just develop your whole system in AWS and gain the efficiencies?

There are tons of algorithms from academia like DBSCAN, BFGS and so on that are fairly complex to implement by yourself. It would take days to translate papers on these subjects to get usable algorithms - if you have enough background and skills, of course. Even then you would miss out on all the improvements that subsequent papers have brought out. In nutshell, to get to the state of art for cutting edge algorithms is extremely time and skills incentive. This website can solve this huge problem.

I think, as a startup, you might want to cut down on "making money" part and especially muddy waters of patents. These are likely not the greatest incentives either for researchers or consumers given that lot of grants don't allow this and even if it did many researchers settled in academia have severe repulsion to it. Instead try to grow the site organically keeping it as frictionless as possible for both sides of users. Motivate researchers to post code and data, for example, to allow them compete against state of the art. Be the that reputed site for computational problems and algorithms that people go to like they go to Stackoverflow for programming problems. Once you establish yourself like that, researchers want to get benchmark from you to cite in their papers and that's great motivation for them to give you their code. Consumers can then find them in your repository. I would suggest to keep making money as TBD. Just grow the site making it largest possible index of computational problems and known state of the art solutions.

Thanks for the thoughtful insights. Feedback is greatly appreciated.

I'd like to know how a startup still looking for a founding engineer...

https://news.ycombinator.com/item?id=8120070

...gets written up in WIRED. Nice PR work.

It looks like both co-founders are engineers (or technical, at least), maybe they just want additional help and are offering a "founder" title in return for the risk of joining such a young startup? http://algorithmia.com/about

Link to actual post: https://news.ycombinator.com/item?id=8120514

We are looking to put together a strong core team to deliver our vision. Both myself and specially my cofounder are engineers and code constantly. We are looking for like minded folks with a true passion for state of the art algorithms to join this team.

Not sure this has the utility that its creators envision. You're effectively putting the "CPU" on a REST interface and shuffling data across a very long bus to have it operated upon.

What kinds of algorithms is this even good for? Cryptographic algorithms are right out. The image on the page says "Dijkstra's shortest path planning algorithm" - I don't think that's a candidate. Results for pure functions can just be cached ...

The only possible beneficial (and profitable?) use I can imagine is to gather statistics on algorithm performance with real-world data provided by developers from around the 'tubes.

Either this was created by someone who had no CS experience or it was created by someone who is a developer but is equally clueless.

Algorithms are worthless by itself.

> Algorithmia will host the algorithms on multiple cloud service providers, such as Amazon and Rackspace, which will speed things up for customers who use those particular hosts

I've long wondered if this business model of service components is how industry will evolve. It seems to make economic sense - but so did ordinary components, and they lost to (free, modifiable) open source libraries.

Haxe would be a perfect language to express shared algorithms in. Grab an algorithm, and transcompile it to your favorite language.

I suppose an actual existing 'app store for algorithms' is CRAN.

Why wouldn't someone just create their own API service?

Because it's less work to outsource? Same reason someone doesn't host their own mailserver.

I am predicting this thing will die a horrible death.

Care to explain why?

Well, there are several. I am not sure where to start. But I guess the most fundamental problem (to me) is this: at the core of the app are "implemented algorithms". Right? What is exactly "implemented algorithms"? They are just "software". The app is just ill-defined at its core. If you are selling software, then call it software.

Another thing is. Implemented algorithms are useless since they solve no real-world problems. For example, take an implementation of Dijkstra's shortest path. No real-world problem can directly use this implementation. It will have to be modified and adapted to a specific situation. This means customers will have to figure out somebody else's codes and modify it. Why don't they simply write it out themselves while consulting a classic algorithm textbook, instead of buying it?

FUCK OFF NAY-SAYERS. Just because you don't like it doesn't mean idea is not good.

Idea is good and certainly has potential.

Reply on news.ycombinator.com