Hacker News Reader

Comment by gfodor | Hacker News Reader

What's the monetization strategy here? It would be a major step backwards for the industry if it turns out the plan here is to try to make money off of what is currently a public resource. I'd hate to see researchers having to decide if they want to publish their results or just make some cash by creating a closed source API with your service. You guys should make it clear if source code will always be available or if this is basically going to be a land-rush for people to provide an API for all the standard CS algorithms with you monetizing access to them.

View on HN

We are algorithm developers ourselves and our goal is just to make these awesome algorithms that are being developed more widely available.

We are not trying to monopolize basic algorithms. In fact in a free market I don't see how that could happen, since anyone else could come along and implement the same basic CS algorithms and offer a lower price.

The thing that makes us different from say Wikipedia or Github is that we don't provide just source code, but live infrastructure delivering the algorithms as a service. So rather than uploading code to github, and putting the burden on a user cloning the algorithm, setting up virtual servers, deployment systems, load balancers, etc. We handle all that. That way algorithm developers are free to focus on algorithm code, and the infrastructure is taken care of. This infrastructure needs to be paid for however, so our general plan is to charge for using algorithms through the API. It will be up to the developers whether to share source code or not, since they own the IP.

The status quo now is not broken: developers who want to share IP are encouraged to incorporate their work into open source libraries so they can be used by everyone and be made robust and evolve over time. In various domains these libraries are more or less agreed upon, and the patches go through extensive review to get incorporated.

This type of system will encourage these folks to release their source code in a form compatible with your API, and 3rd parties who want to leverage their work will need to either cut-and-paste it to incorporate it into their own system (a nightmare), or access it through a slow network API, likely paying for it along the way. Libraries have solved the problem of code re-use of algorithms for the last several decades. You're basically proposing that LAPACK and BLAS and Mahout and all these great algorithmic libraries would have been best served up without any centralized coordination or binary distribution, but with each individual algorithm behind a separate API with each author publishing each one individually. I don't see how if the goal is really to enable wider distribution of good algorithmic work that you can't see that an heterogenous API-centric approach to this problem (while one that would certainly make you a pile of cash if it catches on) wouldn't really be a great step forward for the industry compared to the centralized open source library model we have now. Good algorithmic work is hard, and the model we have now results in battle hardened, robust, agreed-upon implementations of well understood algorithms.

If you really want to make me a believer, all algorithms that your API supports that have public source code should also be integrated and built into a binary library that a user can download instead. At the very least, the path for a contributor to take what they have written and additionally publish it as a appropriate binary package themselves should be minimal. For example, if I implement something in Ruby that uses your service, I should be able to push it as a un-coupled, reusable gem in a straightforward manner.

I agree that the existing libraries are great, and if anything that success is what we to encourage even more of. We want to build a community around algorithm development, and have a central place for developers to share their work, and collaborate with other developers to make Huge library of algorithms so that everyone can benefit.

The first language we implemented was Java/Scala, largely because of the excellent libraries like Mahout and Weka easily available in Maven Central. But even though the code is there in maven, it is still not trivial to deploy those algorithms as a service. We are trying to make that easier, while at the same time supporting the developers of those libraries in the process.

Being able to run the code locally is in the plans. For the most part, the algorithms in our system are already simply maven packages, or ruby gems, npm packages, etc. Being able to distribute them in binary form is definitely possible. We're still figuring things out right now, but our focus is always on doing right by the algorithm development community.

cool, good luck!

This doesn't make sense to me. You display Dijkstra as an example on your homepage. Imagine I want to use Dijkstra in an app that needs critical performance and calls the function 1000000 times per minute, making this an API rather a library just breaks the whole point of centralizing the algorithms.

Plus, the choice of Dijkstra algorithm implementation depends on the graph size, type, and even CPU. See http://www.cs.sunysb.edu/~rezaul/papers/TR-07-54.pdf , with 10 different implementation variations and several different benchmarks.

The 9th DIMACS Implementation Challenge from 8 years ago (see http://www.dis.uniroma1.it/challenge9/ ) served as a way to gather the best algorithms for the shortest path problem. I don't see how this project can be significantly better, or even close to, a traditional effort like that.

How many of those shortest paths algorithms can you run right now?

It doesn't have to be better at producing good algorithms. It just has to be better at making those algorithms implementations easy to use.

But I still don't think the business model is viable.

How many do I need to run?

The Boost Graph Library includes three different shortest path implementations, and I have that already downloaded. That library likely fits my needs.

In the DIMACS challenge papers at http://www.dis.uniroma1.it/challenge9/papers.shtml you'll see "Single-Source Shortest Paths with the Parallel Boost Graph Library".

Indeed, the Parallel Boost Graph Library library is easily available, and contains two parallel shortest-path implementations.

The flip question is, what if it doesn't fit my needs? Well, then the question is "what do I need"?

The other DIMACS challenges include 1) pre-computation, which is useful for road-based graphs, 2) supercomputer parallelism to search billions of nodes, 3) out-of-core graph searches, for when you don't have enough RAM, and 4) k-shortest path search.

It's very unlikely that this proposed algorithm service provider will implement these alternate algorithms.

Yes. It might be confusing the lack of apparent centralization for a need where a diffuse format already accomplishes the desired goal. Either there has to be an unmet Tragedy of the Commons that open standards/academia isn't addressing or that people would be willing to pay for. I can't see either because it would probably already exist.

Maybe we need more crowdfunding of open source and independent researchers that are independent of academia. I really think open source hardware is crying out for a project and a community to revolutionize how a community can participated with the mostly closed EDA / silicon stack, because frankly, I don't trust anything I can't get the source code to.

> The thing that makes us different from say Wikipedia or Github is that we don't provide just source code, but live infrastructure delivering the algorithms as a service. So rather than uploading code to github, and putting the burden on a user cloning the algorithm, setting up virtual servers, deployment systems, load balancers, etc. We handle all that. That way algorithm developers are free to focus on algorithm code, and the infrastructure is taken care of.

Virtual servers? deployment systems? load balancers?!? Huh? It all looks like gibberish to me. What does any of this have to do with developing implementations of new algorithmic techniques?

Because that's what is necessary to actually use algorithms in practice if the demand is non-trivial.

I've read your blog post twice now. I've looked at your website. I've seen your comments here. Still I have no idea what you're proposing or why I would be motivated to use... whatever it is you're trying to convince me to use.

Your message is vague as vague can be. It also doesn't help that the word "algorithm" is sufficiently general to cover any piece of code.

In this post we were trying to focus on the problems we've personally experienced as algorithm developers. The plan is to explain the details of Algorithmia more in future posts, but here's the basic idea:

We want smarter machines, and that means making algorithms more widely available. To achieve that, we're creating a community around algorithm development, where people can contribute code and collaborate with others on that code. Github has done a great job of this, but it is not always trivial to get from a github repo to live code in production. We think that creating a built-in market for algorithms-as-a-service could be a way to reward algorithm developers for their work, and make algorithms easily accessible via REST API.

The types of algorithms we are talking about is not limited by us. My opinion is that the most useful algorithms in the near future will be related to: machine learning, classification, analysis, and optimization. There are also numerous applications for things like image recognition, speech-to-text, and translation, to name a few.

The idea makes a lot more sense to me in this context. Even if you are aiming at other algorithms too, I'd suggest don't be shy about giving these examples.

Good luck... that seems like a really intimidating area to learn and a collection of ready to try algorithms could be super useful.

Yeah: What's the goal and where's the value to investors?

Goal: It's hard to see algorithms innovating except for large scale microoptimizations using higher levels of abstractions, because many everyday purposes have already been solved using whatever the developer chose to ship a product.

Value: For-profit research is a really hard model to scale. Well-respected universities already do this and are more of a factory for talent (via publishing) rather than IP, so it becomes a very indirect horse bet for a business to invest in. In a sense, this would be building a double-ended marketplace of investors and researchers. Also really hard to scale.

Reply on news.ycombinator.com