280 karma · joined April 21, 2016
I suspect the problem is that the people who're using the software are techies who would rather enjoy spending some time compiling it and setting it up themselves, maybe learning some things in the process, instead of paying for it.
This kind of software would have to be sold to CEO/CTO folks who then ask the developers to set it up afterwards.
Regarding the AMI/post-instal part of the question, this is already covered by AutoScaling. Remember, it works against AutoScaling groups, where all instances start with the same AMI and runan userdata script at boot. All this is preserved for the spot instances.
We can add a tl;dr with the root cause for each item in the list. Feel free to send me a PR with this.
My particular project is designed to run in AWS and help reduce my users' bills. I spent a lot of time and effort to make it as easy to install as possible and to minimize friction, runtime costs and security risks for my users (it's entirely serverless based on Lambda and can be installed in minutes).
The side effects of these decisions are that it's so stable,secure and easy to use that pretty much nobody really needs any help with it, and that it is quite unlikely to break in the future by itself(although I see it break from time to time because AWS often changes some things that I depend on but I usually address these quickly as well), so getting a support plan doesn't make so much sense.
I mainly get these support plans sold to a handful of very enthusiastic users who would otherwise donate much smaller amounts.
I tried to bundle these with a guarantee that the supported code is thoroughly tested while the trunk is best effort, but that doesn't seem to work yet.
Hopefully someday the side project launches and makes it to the first page of HackerNews or ProductHunt and then you need to cope with insane amounts of (mostly junk) traffic, which can be hard if you just happen to have a $5 VPS, so you're likely to fail your launch.
At sustained scale serverless solutions can be very expensive indeed, but that's a great problem to have for a side project.
Large organizations or established projects would likely be an order of magnitude cheaper to run their compute on EC2, maybe with RIs or spot or a combination of these (or their equivalents on other cloud/VPC providers), or even on bare metal if the usage patterns are sufficiently flat.
It's also more convenient if you are shipping your software to your users instead of running in SaaS mode, like I do with my https://autospotting.org project. I wouldn't want to ask my users to pay, run and maintain an EC2 instance for it.
Those who appreciate me doing this would hopefully donate those $5 to me on Patreon :-)
The other critical feature is how easy it is passed on to a new victim, the worst scenario is being airborne.
From my experience the alternative support and donations business models don't really work. I'm the initial author and maintainer of Autospotting, a piece of software that saved the users in the millions dollars aggregated over my entire user base, and as much as I tried so far I couldn't get much money out of it.
I'm currently getting about $100 monthly on Patreon for support plans and a few donations here and there but that's nothing compared to the effort I put in it and the value it creates.
I have an iPhone 6s and my wife had a 6 and needed a new phone. We bought a new Xr but it was a small fortune, even though we're both working in IT and having good income. The phone is a brick and I saw lots of slicker Android phones costing half the price and providing a very similar if not better experience.
My next phone is very unlikely to be an iPhone.
But a very small minority like it enough to donate to support the development efforts, even though it is saving them significant amounts of money and time.
As someone who runs a small open source project that clearly states that Patreon donations are accepted and even offers some convenience benefits to donors, I often see people jumping extra hoops to avoid it, including clearly profitable companies that saved many thousands by using my software, spending extra time that would amount to more than those donations. So far I get donations from less than one percent of my estimated users.
It's just often misconfigured because there are a plethora of plugins and ways to store and use secrets, and nobody audits it enough to look at the console output or build artifacts for leaked secrets.
The project is simply missing someone familiar enough to configure Jenkins properly.
As for the complexity, from my experience it's a mix. There are indeed many complex issues, but often times the fixes are relatively easy and take little time to fix.
For the hard ones it's often enough if you can contribute to the conversation to better understand the problem, providing valuable information on how to reproduce, so that someone familiar with the code base can solve it easier.
Also, larger companies internally operate much like an open source community, having multiple projects that accept contributions from anyone in the company.
Reporting and fixing public issues as part of your employment is a sign that you care about the ecosystem that you are part of, giving back rather than just benefiting from it.
It's not necessarily a showcase of your coding skills, because these contributions are usually small, but it shows that you will get your hands dirty if needed and fix the problem at the root, instead of hacking a workaround in your own code base. It also shows that you have no problem with learning a new code base and delivering code according to that project's quality standards.
If you don't have any public activity it may imply the opposite: that you are just freeloading your community, that you are prone to doing workarounds instead of fixing the problem upstream or that you are unable or reluctant to contribute to other projects.
I wouldn't be surprised if companies like Atlassian, Slack, Jetbrains, Docker or Hashicorp are next.
To answer your final question: I wrote a spot instance automation tool, you can check it out at autospotting.org, so I would give spot instances a try. The latest developments from AWS on the spot market are real game changers, I think most of the workloads can now safely run on spot, my AutoSpotting tool makes it a breeze to migrate from on-demand AutoScaling groups while keeping them a bit more reliable than the native AutoScaling integration for spot.
As of a few months ago the pricing is much more stable than before, I've rarely seen terminations even over the maximum three months of history for instances that used to go bust multiple times a day. You also now pay them on a per-second basis, and you can hibernate the last one to keep the state of the group while everything is down.
So my approach for this would be to have an AutoScaling group of the smallest spot instances that can run your app, scale them to N nodes right before your experiment, then when you're done scale down to a single one, which you use as data seed next time, which you detach and hibernate with API calls.
Next time you re-attach the seed to the empty group, and scale out to N once again and run your test. So you only pay for the length of your test on a per-second basis.
You can also keep the seed as an on demand node outside of the spot group and have it run from the free tier if you still have some time left, or just hibernate it as well.
The reality is that both software and battery show their age and need to be updated in order to keep the device in a good shape, within the specs people paid for when they bought the device.
Apple does a great job with the software part but it could also do something about the battery,without much impact on their bottom line, as someone else said,the battery costs them just about $5.
This practice has to be pioneered by someone and considering their pricing and margins, I think Apple is in the best position to start doing it.
In my spare time I've been maintaining my Autospotting pet project, which is maturing nicely, growing a lot and already generated savings in the six-seven digits for its users: https://github.com/cristim/autospotting
I also spent time learning to play guitar, made a habit of practicing and working out on a daily basis and towards the end of the year I became a father.
All in all it was likely my best year so far.
The problem is this is against one of the criteria for a license to be approved by the Open Source Initiative, which states that the users are all treated equal regardless of the purpose for which they use the software, so you can't claim to be Open Source if you discriminate against businesses like this.
I was actually contemplating to use LicenseZero for AutoSpotting, my largest and most successful project that can save companies a lot of money, but I'm afraid this will discourage users and would slow down contributions.
So far I am playing with a dual license model somewhat similar to the way Redhat handles RHEL and Fedora. The software is open source under the MIT license, and anyone can use it from source if they are willing to compile it themselves.
In addition I am now offering evaluation nightly builds that expire after two months, and I am selling for a relatively small fee stable and well tested builds that I verified to be working well. I am also only actively offering support and trying to fix issues for these stable builds, the other builds are supported by the community on a best effort basis.
So far very few people were interested in the official builds, but the first evaluation builds will start to expire within a few days so I expect the number to grow.
The sad reality is that costs are not distributed, most companies just grab whatever open source software they can and only contribute the least they can get away with, most of the times nothing at all.
Many projects survive just because of dedicated individual developers and maintainers spending their limited spare time, who end up being burned out.
In an ideal world open source software that brings significant value for companies should be considered as part of their stack, they should either hire or assign someone for supporting it, or pay the existing maintainers, and ideally this should be somewhat proportional to the value they get out of it.
Unfortunately donations as method of payment rarely work(usually end up being paid by individual employees from their own pocket), so the only option to get money from the company itself is to somehow get it charged.
They usually have no problem with being charged money, as long as the price is reasonable for the value it brings and less than would cost to write the same code in-house or to even have a meeting about acquiring it. It also should be significant, they wouldn't go through the bureaucratic procedures for a 10€ one off payment.
Developers don't expect to become rich out of this, they often would settle for less than their full time job, as long as they can work full time on the project they care about and still make a living. But because so few companies share the costs, they tend to be significant for those who do pay, so the prices will be higher at first but should decrease quickly as more companies share the burden.
But running standalone spot requests is indeed much worse.
Autoscaling groups come much more natural to people, it's just that the native spot implementation is unreliable.
I mentioned before in this thread that I implemented a tool called autospotting that allows your on-demand autoscaling groups to be automatically converted into a sort of spot fleet, by replacing their members with the best available spot instances.
You get the best of both worlds: easy configuration based on your group settings, so you don't have to worry about bid prices, instance types and weights: the bid price is automatically set to the on-demand hourly price of your original instances, the instance type is also automatically determined based on the original instance type, so you can get any other type that's at least as large, even from a different generation, the selection is currently based on the lowest price, so you will pretty much automatically get various types without explicit configuration. When no spot instance types are priced lower than the on demand price the group will happily run the initial on demand instances until eventually some become available for a better price.
Gradually these spot instances are launched and attached to the existing group, and on demand instances are terminated to keep the capacity constant.
Unlike the spot fleets, this supports out of the box anything that is backed by autoscaling groups, such as Elastic Beanstalk, Kubernetes clusters built using kops, environments managed by CodeDeploy, and so on.
And unlike spot fleets, the migration to spot and back is a matter of setting a tag on the group, so you can easily revert back to the original group configuration if you decide to.
It will continuously replace your group's on-demand instances with the best priced compatible(at least as big and identically configured) spot instance types, with minimal configuration changes performed by simply tagging the group. The group configuration is unchanged so it would keep launching on demand when scaling out or when replacing outbid spot instances.
The tool is open source and available on github and can be set up in a few minutes: https://github.com/cristim/autospotting
This allows monetizing those, and does it targeted to large companies who can easily afford it and would happily pay a few bucks to use the software (they would likely burn more money in a meeting discussing whether to adopt the said software or to switch to a free alternative) and I think it's a good trade-off if it allows indie open source developers to pay their bills.
On the long term those developers will generate much more open source output than when working as employees somewhere and creating only proprietary software all day and struggling to create open source only when they have some spare time a couple of hours a month.
In addition, all this licensezero software will be developed in the open, which brings many of the practical benefits of open source to a vast majority of the users, and the open source ecosystem will grow as a side effect that the developers will also create other software distributed under ordinary open source licenses. Even if just a fraction of their time would be spent on those other open source projects, that can make a huge difference to the ecosystem.
I think monetizing those, targeted to large companies who can easily afford it and would happily pay a few bucks to use the software (they would likely burn more money in a meeting discussing whether to adopt the said software or to switch to a free alternative) is a good trade-off if it allows indie open source developers to pay their bills.
On the long term those developers will generate much more open source output than when working as employees somewhere and creating only proprietary software all day and struggling to create open source only when they have some spare time a couple of hours a month.
In addition, all this licensezero software will be developed in the open, which brings many of the practical benefits of open source to a vast majority of the users, and the open source ecosystem will grow as a side effect that the developers will also create other software distributed under ordinary open source licenses.