How my open source hobby project became my job: 10 years of OptaPlanner | Hacker News Reader

How my open source hobby project became my job: 10 years of OptaPlanner

optaplanner.org

246 pointsge0ffrey40 comments

I can't help thinking this isn't the optimal result for the author. After 10 years of working on his thing, it has essentially been taken over by this company who sells it in their name for a profit, while keeping the author working on it for a salary. They seem to be getting a really good deal.

Imagine if, instead of building the "Leading Open Source Constraint Satisfaction Solver", this guy had instead spent the same time building the "Leading Constraint Satisfaction Solver", differing in that it had a price tag associated.

For as smart a guy as he seems to be, I can only imagine he'd be doing a lot better today.

I read this as a sad commentary on how the Open Source philosophy, that seems to be so drummed in to kids at University, leads them to think that the best course in life is to give away their best work for free.

Not everyone wants to run a business. He does show a lot of business-like initiative but mostly in service of a technical/creative goal. Ultimately, though, a lot of people might prefer to be paid a steady, reasonable wage and have their head space clear for the work.

On the other hand I do agree with you, that the Open Source philosophy is used in an exploitative way by business. I guess the lesson is: understand the true nature of the commercial relationship and what everybody is getting from it.

I concur, that is the dream. To work on my own project, without worrying about finances. To have other people handle hr, sales, most of clients support, taxes and regulations, all of the crap that distracts for the interesting problem that made the project worthwhile long before there was money to be made.

That's exactly what I had thought at first. I think that there are too many brilliant engineers with great (and technically working) ideas out there who are just too afraid (or lack business acumen) to enter the market a bit more boldly than through Open Source.

Specifically in this case where the software is so niche but adding such tremendous value to end users, a very premium price tag wouldn't be too hard to sell. But it would have to be sold. A skill that the author didn't have and didn't want to acquire / hire.

I'm the author of www.jooq.org, which has gone an independent dual-licensing route and I really wish more people did this rather than hoping for an OSS sponsor... (hello Groovy/Pivotal). And I can totally say that all fears are wrong. I am doing even more what I want in my company, both from a technical perspective as well as from a personal branding perspective. Comparing this with being driven by a corporation like Red Hat...

> I read this as a sad commentary on how the Open Source philosophy, that seems to be so drummed in to kids at University, leads them to think that the best course in life is to give away their best work for free.

Of course, this naturally contrasts with Fucked Company/TechCrunch's Deadpool, which is a sad commentary on the "turn a feature into a company"/startup philosophy, also drummed into kids at University, which leads them to think the best course in life is to give away their best years for free (a.k.a. worthless equity).

We can speculate about how things might have turned out, but we will never know. There are lot of dead open source projects that never amounted to anything. Life isn't in black and white

While I agree with you in principle, after reading the blog post I can only say it did work out quite well for Oliver. He works on what he likes full time and that's already more than what most of us can say. Would his own enterprise tool have made him richer? Possibly. But he himself says that it didn't have many enterprisey features at the time it was still his hobby project. Pretty sure he didn't reach that conclusion on his own and had someone at RedHat to guide him on the way.

In any case, nothing is stopping him now from starting his own company and consulting practice. It's still his open srouce project. Maybe he just doesn't want to do that and there's nothing wrong with that. You know who else was a genius that didn't want to sell things? Steve Wozniak. He even gave up most of his Apple stock to colleagues who he felt deserved getting it. Then there's Linus Torvalds. For some people, there are stronger motivations for building a product than running a company.

Are you sure? I would assume that Red Hat requires all employees to sign copyright assignment and other such legal agreements. He would no longer be able to change the license at all, simply because he no longer owns the copyright to many of the earlier contributions.

> leads them to think that the best course in life is to give away their best work for free.

As much as I like and use Open-source software, as someone who is trying to bootstrap is own "thing", I think I can now understand the two sides of the coin.

For one, if we didn't have open-source languages, or open-source frameworks, we couldn't have built what we have built in the last years. Lots of startups and small businesses out there rely on open source frameworks, open source libraries, etc., and some of them could have never reached profitability or growth if it wasn't for the reduce costs that open source allow for.

On the other hand, if you give away too much things for free, you won't have anything to sell, unless you are being sponsored somehow. For instance, most linux kernel developers are being sponsored to work on the kernel, the author is being sponsored to work on his constraint solver, etc. By the way, this is one of the cases where I think something like the Universal Basic Income would be making a huge difference..

Personally, as a single developer trying to make a living, I've given enough thought if I should open source my own software. The way I see things now is, if you are working on "infrastructure" software (software that others can build upon, such as protocols, servers, browsers, OSs, etc.), it may make sense to open source it, either because it is a massive task for a singe developer or because you need mindshare and free marketing for it. If it gets to be widely used, you can eventually build a consulting business around it, or get sponsored to work on it.

But if you are working on small projects that are hardly going to be used as infrastructure, the chances that you are ever going to be sponsored to work on it are quite slim. So, if you think that you will ever be able to sell something based on that kind of software, maybe it does not make much commercial sense to open source it and give it away for free. On the other hand, it may make sense to open source it if you think you'll benefit from the exposure (such as getting a job because recruiters can check your code).

Please note that I'm mainly talking about open source making commercial sense or not, and not if it is the right (as moral) thing to do. If I was being sponsored by someone, or didn't have the need to participate in the market economy, I would give all my work for free. Unfortunately, I live in _this_ world..

I don't think that would have worked. He needed exactly what he got: Help from established organizations. Bootstrapping that yourself is orders of magnitude harder. Notice that the people employing him get their money elsewhere, so it's doubtful that product itself could have bootstrapped itself into a full business. Also, as soon as there is money to pay adoption of your little tool drops dramatically, because now people using it want to be damn sure it's worth it well in advance, usually even before even trying it.

> leads them to think that the best course in life is to give away their best work for free.

What, exactly, would he have built his tools on if others had not released Java as open source, Drools as open source, Tomcat as open source, JBoss as open source?

Notice that your list is made up entirely of Open Source products built and maintained by for profit companies.

Those companies are not doing that out of charity. They all have well thought out strategies for what they are doing, so you can be certain that it makes business sense to them.

There are in fact useful, valuable Open Source products built by starving developers as a full time labor of love with no thought of personal reward. But in the context of the ones you mention, doesn't it seem a bit silly to do so, producing so much value for the software industry as a whole yet capturing none of it?

You don't see many people voluntarily spending their days flipping burgers at McDonalds for no pay, alongside other people doing the same work for a salary. But for some reason we're indoctrinated to think that doing the exact same thing for, to choose at random, Firefox, is somehow a noble trait in a software developer.

You are comparing apples to oranges.

Flipping burgers does not have same added value. As software devs you can leverage work on Firefox to get better salary. Flipping burgers can get you other flipping burgers job.

Software devs are usually in better position than those flipping burgers. So you can expect from people who are better off to start with, to contribute to society a bit more.

Compare them to apples for me then.

In what other industry do skilled professionals volunteer their time to work for other for profit companies in their free time?

It's certainly common enough to volunteer professional skill for charity, but I can't think off the top of my head of an analog in another industry to the frequently-seen case in software of a gainfully employed, good developer giving his free labor to a commercial project like Chrome, MySql, etc.

Portfolio work for photographers, illustrators. Loads of musicians working for peanuts or for promises of fame and also building portfolios.

I think lawyers and doctors are abused by working long hours and are not compensated for it as they should. There it is called duty or civil service.

But I do not have any hard example...

If only the author had software to find the optimal result for 10 years of working on his thing...

>> "Meanwhile, I learned more about other, similar projects. And they seemed to have more people working on them. And they had sales people."

This is exactly my situation right now with my open source project http://socketcluster.io/ - We actually already have a clear monetization plan (see http://baasil.io/) but we have no funding, few industry connections (aside from companies which are using SocketCluster in production), and our competitors are spawning up like mushrooms.

The one positive thing about being in such a position is that it forces you to be resourceful with how you spend your time.

I was extremely enthusiastic about doing something in optimization just after graduating, but realised that businesses were happy solving things manually and reaching sub-optimal solutions. Also most problems needed to be solved once and there were few use cases of continuous optimization.

Thanks for sharing your story of perseverance and spirit.

Now that you are at it, are there any easy-to-use solutions to solve time-tabling problems for school/colleges. I know of solutions which use GA, but since its a generic problem for every academic institution, someone(you) might want to provide an easy to configure and use solution.

The "problem" with optimization software is that the fun optimization algorithms part is just 1% of the work. The software also needs to be complete enough to handle most of the everyday workflow of most employees at the users' company, or provide integrations to any other software that they use daily (from Excel to a custom calendaring system built by the founder's son 15 years ago).

Some friends are working on a courier dispatching system, which includes very clever optimization for choosing trucks and mapping routes. But most of the effort goes into making sure that the mobile app works on 5-year-old smartphones, and the fonts on the PDF invoices look right, and so on.

If you want to work on just the optimization algorithms, then that can be done only in academia or at very large companies which can afford specialization.

After reading the article I have a strong feeling that without RedHat there wouldn't be such a happy ending. Great contribution on their part.

Thank you for this software. It is on my list of "need to try for real". What do you see as the main reasons holding such software back from mass use? Is it real world soft constraints, or prevalence of "small enough for humans to do well" datasets or something else?

In my opinion there are two key factors:

First, OptaPlanner does require a programmer to use. That means a random business user with a problem isn't able to just do it themselves, and instead needs a larger effort to try it out and see the results. It would be amazing if there was a way to give an "Excel" like experience to OptaPlanner, so that anyone could play around with models and optimization.

Second, Lots of companies have found ways to accomplish their business without optimization tools. Often their processes are heavy on manual planning, and there is a general sense that the problem is too complicated and has too many factors for an algorithm to be able to do. I've had clients with perfect use cases for OptaPlanner stick with a-priori business rules because that is how they did things and they felt more in control. Convincing people to try out a magical optimization program that finds solutions is difficult.

Disclaimer: I work for Red Hat. I've been a big fan of OptaPlanner even before it became one of our products!

Could you share why you want to try it? What particular problem are you facing that this project solves?

I am in general fascinated by software where you get non-obvious output from simple input. I would like to try this for employee rostering. My main reasons not to is an abundance of soft constraints (Mary prefers Fridays off, Lucy prefers her days off bundled together) and the fact that I live in a small country with small organisations where I am unsure of real benefits compared to manual work.

As an aside, vehicle routing is a lost case here. We have few main roads along the coast and across mountains, nothing like the grid roads of central US. Vehicle routing is obvious here, no need for software.

Which country ?

"So I must have been doing something right. But I did a lot of things wrong, too. The winner had also open sourced his implementation and when we started discussing and comparing our implementation, we learned a lot from each other."

i loved this bit. rather than hyper-competitive moat building, they chatted and helped eachother.

As someone with an open source C# constraint solver hobby project, used by my former employer Moody's Analytics, I can see many similarities.

However, I can see the differences... I haven't spoken about it at conferences, or entered it in any competitions, which explains why it remains a largely ignored repo.

This is a good write up that shows one route of what's required to take something from a bit of fun, to your full time vocation.

Sounds interesting, is a repo available?

Thanks for the interest.

http://github.com/lifebeyondfife/Decider

A very interesting topic (optimization) and the documentation does a good job at introducing it. I also like the facts that the author published under an open source license and that he is financed/employed by a company that does all the heavy lifting like accounting and marketing.

Is there a reason why the author choses the Apache Software License over something similar like the LGPL?

Some people here tend to think the author could have made a lot more money if he would have turned the software into a for profit business right from the start. This may be so but, aside from beeing a lot more work in non-programming areas, this would probably not have generated the other effects that occurred: the author learned a lot more by participating in the programming community and also created a valuable relationship with a company. Both these effects open up new ideas and possibilities, be it intellectually or in terms of business opportunities.

LGPL is not similar in one important way. Any reference to GPL in a license scares the shit out of corporate managers and lawyers when it's attached to something in their proprietary code. That discourages business use and adoption, sadly, which slows down adoption overall.

As someone who has (comparatively) recently started an open-source java project https://github.com/lwhite1/tablesaw, I loved this post. I know the both the pleasure and desire to just quit.

It's a bit frightening to think I could put 10 years into working on something without compensation when corporations benefit every day. And that's a good outcome: OptaPlanner is a success!

In some ways it just doesn't matter because the pleasures of creating something are so strong. I can see why the author pushed on for so long. Congrats on your success and on an awesome project.

Just an FYI, there's a typo here:

> During this [[though]] period, I regularly considered giving up my Open Source project a few times. Somehow, I couldn’t. I just soldiered on. And in the end, it all worked out well.

Fixed, thanks :) https://github.com/droolsjbpm/optaplanner-website/commit/b12...

Hi Geoffrey, thanks for sharing this, very interesting!

One-word summary: grit

Pretty cool read.

As someone tinkering away on their own optimization algorithm, how big is the market for these sorts of optimization solutions?

In the millions, once you find an application domain.

One project I abandoned part way through (because it was too big for a side-project and just wasn't that exciting to me at the time) was an automated employee scheduling system for organizations with complex constraints.

Its a big problem for many organizations, for example event organizers. Think SAAS apps like shiftplanning.com (which started as a 1 man project and was released about the same time as I just started working on mine.)

The Drools project which had an example rostering component already included was going to used for the guts of the system with a calendar like web front end, and lots of customizability.

I guess many people have something to gain from optimization and this means you could get a cut of the money they save. So I would say the potential is big.

Actually the market is also limited by the required effort or competence to use the system. OptaPlanner needs apparently a Java programmer. This is where you could make a difference. Make it really simple to use as a toy.

It may seam that the use of such tools is a one shot and done. But rules keep changing, and users may want to test and explore the impact of changing rules.

The market is huge, but mostly what we need are more useable systems, not algorithmically cleverer systems (unless your cleverness makes it more useable of course).

Reply on news.ycombinator.com