What we've learned from building Ghost after 5 years and $3M
blog.ghost.org
blog.ghost.org
I was looking to port over our homegrown django blog app into a proper blogging engine and I immediately though of Medium because of HN conditioning, and had so many problems with it, all of them boiling down to lack of control: No way to embed stuff we wanted (we wanted to embed interactive graphs and tooltips), no way to disable some bullshit features (highlight popups), no way to get our own domain name, etc.
Ghost solved all this. And on top of that, it's markdown, which makes it just fantastic. And open source. And self-hostable. And you guys run a wonderful, exemplary foundation behind it. I have nothing but admiration for what you do, and I'm proud to be a paying client for it and recommend it to anyone for company blogs (it is unfortunately expensive for personal stuff however).
My #1 wishlist for it is better image embedding/resizing/thumbnailing support (native image gallery & captions, native thumbnailing, automatic thumbnailing of hero headers for the article list view).
My #2 wishlist is Postgres support. I'm very sad you dropped support for it :(
I use Postgres for other things, really don't like having to have a separate MySQL setup just for ghost...
> Developers regularly show up on Github, rage at us for something like not supporting Postgres - and then we say "ok so are you going to write and maintain Postgres support for Ghost?" and they say "of course not, I don't have time for that!" - and then occasionally they'll go on Twitter and tell all their followers to give us hell. As if organising a mob and shouting louder is the best way to get a bunch of people writing free code to do what you want.
Another question: are there any aspects of your product that are open core? Does the SaaS have any "special sauce?" Thanks.
For one, it's not an American company, is it?
Upgrades are painless, typically requiring around an hour at most. If there are bigger framework changes we can usually upgrade immediately then clean up deprecations later as necessary but even then the work is normally minimal. When there are more stylistic changes that touch many files it's usually very quick work thanks to the codemods that are released alongside. All of this despite huge and continuous under-the-hood improvements in Ember since we've been using it.
Compared to what I've seen in other frameworks I think we've benefitted a lot in being able to spend pretty much zero time bike shedding which libraries will make up our core framework. We also save a lot of time not needing to upgrade disparate parts of a custom framework built of 3rd party libraries that aren't necessarily in sync with each other. The Ember addon ecosystem has also been stellar, with common solutions rallied around by the community such that they are usually as stable as core and kept up-to-date with the latest idioms without breaking backwards compatibility.
In conclusion Ember has allowed us to focus almost entirely on product rather than framework. It stays out of the way and provides very clear happy paths for typical workflows whilst allowing us to break out of those where needed without sacrificing any tooling or upgradability.
As for the downsides I think the biggest has been the limited pool of contributors. A lot of react bandwagon jumping has happened since we started with Ember and it's interesting when talking to other developers outside of the Ember ecosystem because there's a lot of historical and outdated views about what Ember is and isn't. The Ember core team and community is working hard to improve Ember's marketing message to counter that but it's tough to break through when the framework's core selling point is consistency through hype cycles.
Beyond that point, we can spend money on anything which reasonably furthers the mission of the organisation - which has many aspects but the main ones in our case revolve around creation and education for journalism and open source software.
Some pipedream ideas we throw around from time to time:
- M&A (boring/obvious answer, but valuable)
- Create either grant-funding, or sensible investment funding (like indie.vc) for other open source projects and publishers.
- Create an independent media company, reporting on issues and stories we care about (running on Ghost)
- Buy some land and build a collaborative space for us and for others to work on open source, journalism and social good projects
1. Build more tools that solve people's problems created by others vendors. As in, move sideways a bit diversifying your operation. Even better if the tools/apps complement each other where you can increase benefit per customer while cross-selling. Although not endorsing them (not a customer), Zoho seems like a good example of this where they keep developing additional products that might benefit their customers on top of their original ones. Enough FOSS-oriented non-profits doing that in enough sectors can stop a lot of lock-in and abusive practices over time via the alternatives they provide.
2. Invest in maintenance or security improvements of dependencies, esp libraries or infrastructure. These are often treated as externalities leading to a lot of problems we see like in report below [1]. Ghost could deliver its good products/services on top of taking some responsibility for infrastructure it depends on. People buying Ghost get the immediate value plus knowledge they're supporting those things as well.
Just some ideas for you. Keep up the great work over there and good luck to you all! :)
[1] https://medium.com/@nayafia/how-i-stumbled-upon-the-internet...
So, a ghost town?
Some things which come to mind:
- lower prices
- hire more staff to provide new features
- make everything faster and more reliable
- provide more free services
- give everyone raises
- donate to charities which are reasonably related to the aims of the non-profit
- make investments to improve the long-term financial health of the non-profit.
Besides actually just reducing prices for services to not get profit, Ghost could create a foundation and give grants to any deserving work that serves the Ghost mission, such as supporting the FSF or EFF or funding other upstream or aligned stuff such as working to get more of their tools to be 100% free/libre/open…
In the early days there's no structure, you just get on with it. As the company grows, generally a board of directors (or non-profit: board of trustees) act as an independent governance level for the org above the exec team and (amongst other things) determine executive pay.
[1] https://twitter.com/JohnONolan/status/941572225890533376 [2] https://ghost.org/features/ [3] https://i.imgur.com/b5OO3Q6.png
And then, to back up your argument, you link to a page which is not running on Ghost - and actually does have optimised+responsive images.
All so you can spam your own image compression app.
Without a trace of irony.
Are you serious.
The HN Guidelines, which specifically prohibit this kind of content, can be found at https://news.ycombinator.com/newsguidelines.html
This whole sentence is false. I did not "bully" you anywhere (!), requested a feature from you, or pressured you to do anything. Quite the opposite, I showed an easy way to improve. Again, I could have been wrong on the effect. That's why the question. The "spotlight" reference was just a small joke. Sorry.
> And then, to back up your argument, you link to a page which is not running on Ghost - and actually does have optimised+responsive images.
I meant only the Ghost homepage in the link, and better optimization (proves [1]).
> All so you can spam your own image compression app.
I gave you the facts. Yet, you smeared me with false accusations. Is this even fair?
[1] https://github.com/vmdanilov/optimage/files/2011804/ghost.zi...
HN seems to cut the link text.
A question: Do you think your choice of technology stack (JavaScript, node.js) played a part in your success?
I've read many times from product creators that developers wildly overestimate the importance of choosing a language or technology stack when in fact it has little bearing on whether a product will succeed.
Do you have any thoughts on this topic?
When it came to selecting Node, the biggest benefit was that it was notable. If we had said "HEY we're making another blogging platform in, wait for it, PHP!" -- it would likely have been significantly more difficult that the project would've been seen as noteworthy / attracted any users.
Conversely, Node is not without its drawbacks. The ecosystem is young today. It was positively infantile when we started. I don't even think there was a fully formed RSS library. In fact, from a technical point of view, most of what we've built would have been easier to achieve with other tools and frameworks.
Nowadays when we select technology, we tend to go for the oldest most reliable stuff we can find. The sort of stuff that when you have a problem and you google it, there are 5,000 people who have done it before. Because there are only SO many innovation battles you can fight simultaneously.
At one point we tried to rewrite our infrastructure using Docker/Swarm - it was a 9 month project which ultimately got completely cancelled at the last minute. Why? Turns out Docker Swarm doesn't scale above 500 instances and we needed to run about 10,000 for Ghost(Pro). We asked Docker Inc and their response was pretty much "idk lol". So now, especially with our infrastructure, we almost always select the most tried and tested technology. Boring stuff. Stuff that is very, very well understood.
The problem is, it's not cool to talk about how our infrastructure is built on very straightforward LXC containers, managed by HAProxy, connected to a cluster of MySQL servers. The tech isn't bleeding edge enough to hear about on HN, so you end up not hearing about it. But most of our best tech decisions end up being when we choose older technology.
Sorry, this got really long. Did Node.js influence our success? Yes! It made Ghost more popular and also more difficult to build. Would I change it? Definitely not.
Which is fine. "can execute binaries in isolation" and "can schedule rollouts and efficient disk/networking structures across many thousands of instances" are vastly different problems. Even "can deploy to 100s" and "can deploy to 1000s" are quite different, without something targeting 1k+ I wouldn't expect it to handle beyond a couple hundred without compounding issues.
And those are exactly the ones I want to hear about.
Like you I find the old, established tech choices to be good. New tech is fun to play with, but when I need to get work done and keep it running (for years) I take old every time.
Please write more on this!
Also, I like the interplay between new/risky tech in one area of your story and established in others. Pushing a manageable number of boundaries.
So, I guess I agree that his intent could be more clearly stated.
My biggest complaint is that Ghost doesn't yet have a built in SSL/TLS option (e.g. Let's Encrypt). Their recommendation is to use Cloudflare. It's fine for hobbyist or personal blogs, but as a corporation it's basically prevented us from using SSL.
We're in a regulated industry and need to do appropriate vetting of vendors. Onboarding a blog platform is very, very easy because of the relatively low risk nature of a blog. Onboarding an infrastructure vendor, like Cloudflare, is very complicated. Making it more complicated is we'd have no interest in actually paying for a Cloudflare service so it's impossible to get resources to complete the vetting process. On top of all of that, last year's Cloudbleed incident cast some real doubt on Cloudflare's security practices and ability to properly protect potentially sensitive data.
We will likely eventually self-host so we can get proper SSL support. I'm really disappointed by this since it will be more expensive than paying our Ghost subscription for several years.
For self-hosters, Ghost-CLI has deep Let's Encrypt integration. The tool will automatically guide you through provisioning an SSL certificate during the install process.
On Ghost(Pro) we have full end to end SSL support (you're right we used to only support CF's UniverSSL but that is no longer the case) - so anyone who uses a custom domain with Ghost(Pro) will automatically have an SSL cert provisioned and deployed for them in the background. For business customers who require the use of their own, custom certificates, there a few extra steps but we support that too.
----
This is great. That was my only complaint with Ghost so far.
Initial onboarding can take 20 to 50 hours of effort from an expensive resource. Plus, routine reviews which would alone cost more than I'd be willing to pay for such a service.
> [Github] has become too transactional - more support tool than collaboration
I strongly agree with this. I wish Github "Issues" were relabeled "Comments" and need not be "Open" or "Closed". Issues need action, comments do not. I've been burned many times now by solving someone's issue and then finding out they use my code for their business and just wanted free support from me.
I wish GH users felt obligated to disclose they work for a company when creating issues that are for their company, but that's not something GH could (or should) enforce.
[1]: https://github.com/schollz/find3/issues/66#issuecomment-3829...
[2]: https://blog.github.com/2017-05-30-introducing-temporary-int...
I also find some of the best improvement suggestions come from people using the software in heavy real world use, which tends to be businesses. They have very real pragmatic needs.
I took the time to write to the Ghost support email with my thoughts on the switch and my main issues (namely no categories, lack of i18n, editor live-view performance with many images, no search -- this was middle 2016, maybe they have these features now but they didn't then)
They took the trouble of answering my email, I found it rather nice.
The pricing levels simply do not offer enough page views.
> Our plans are limited by how many readers you have, and how often they visit your publication. We only enforce these limits on a 3 month rolling average, so we'll never make you upgrade plans if you happen to have one big traffic spike or particularly successful month.
I was a backer in the kickstarter campaign, but various "issues" eventually led me to look elsewhere.
Ghost is (was?) rather good, but the thing that ultimately sold me on it back on kickstarter was the much hyped dashboard, which never made it.
Before Docker it was a pain to manage dependencies and keep everything up to date, but with the official Docker image, things got better - until a flurry of almost daily releases to the 1.x branch made it a chore again.
The final nail in the coffin was dropping PostgreSQL support. While it wouldn't be much of an issue if starting on a clean host, with a clean slate, i run all of my other stuff in PostgreSQL, and i much prefer to only administer/backup one DMBS.
I'm currently on Hugo, and i really miss the online editing/publishing from Ghost.
I for one would love to hear more from the Ghost team on how they approach solving these very essential problems - any insight is welcome!
That's my favorite takeaway from the article. Seems like a difficult thing to do in practice, but also something I've been considering lately.
Anyone else have experience with doing repeated launches that they'd be willing to share?
...
> We're currently looking at testing an affiliate program whereby we'll pay out a 30% commission every single month on the lifetime revenue of anyone referred to Ghost(Pro).
...
I believe this is called colocating your money and your mouth. I'm inspired.
Putting user docs as part of a marketing role is not lining things up for a good result. :(
How about putting it under engineering instead?
The skills overlap much better there: user research, usability testing, copywriting, information design... these are all things UX teams think about and are part of making good docs.
The only similarity between engineering projects and documentation projects is... there are engineers around.
Of all the places to put user docs, marketing is not the right one. :)
Thank you. This is exactly my point.
The only way making them a function of "marketing" makes sense, is if the leadership of Ghost view user docs as something primarily to bring in (new) sales.
As a person who has often done implementation roles in their career, having to rely on docs written by "marketing" people rather than engineers is generally a lousy experience. :(
The primary function of the role is to do writing, not engineering - and the success of the role will be measured in terms of marketing, not engineering - so to me the title we went with makes the most sense. I can definitely understand the discussion here though :)
Err... trying to grok that, and failing. Are you ok to flesh out what you mean? :)
It's fairly easy to see that a Marketing manager will view the desired end result quite differently than an Engineering manager would.
With a corresponding difference in the focus and priorities then set accordingly.
Check out her number of commits compared to other contributors: https://chart.ly/github-dashboard/TryGhost/Ghost
Keep up the good work guys!
I have a co-worker who commits 6x more changes than me, everyday, but only because she commits many errors in the morning and have to fix them during the day after multiple code reviews. She often forgets to run the test suite before pushing her changes up. Even after the creation of a Git hook to pre-test the changes after every commit, she still manages to break things when they fall outside the scope of the automated tests.
The number of commits mean nothing if the quality is low.
Really? I just read a comment from the CEO of HashiCorp who has a very different take:
https://news.ycombinator.com/item?id=17022563
Unless all remote employees of Ghost are just contractors...
Same with Automattic, Zapier, Buffer - and most of the other big remote companies. In fact, we all use the same set of contracts. They tend to get passed around founders.
All 3 of those other companies are USA based, so for anyone in the US they do standard employment contracts - and anyone outside = self employed contractor on a retainer. We're based in Singapore and don't have any staff in Singapore, so effectively everyone is a remote self-employed contractor.
Our retainer contract looks a lot like an employment contract, practically speaking, and builds in all the same sorts of provisions as you would expect. We also cover expenses, where applicable, for team members to get independent legal and accounting advice to make sure everything is set up correctly for them.
Mr HashiCorp has gone way too far down the rabbit hole, by the sounds of it. It makes sense to set up a subsidiary when you have a larger group of staff in a single location. For instance, Automattic have a subsid in Ireland because they acquired a company there and as a result have a lot of staff based there. But otherwise it doesn't make sense to set up a full blown subsidiary in every country when you're a tiny startup just trying to contract a couple of people.
> In fact, we all use the same set of contracts. They tend to get passed around founders.
Care to share? ;)
Or when you want to do things like give them stock or healthcare or pensions
My company is looking into our options as to what sort of structure would be best. We've considered being a worker-owned cooperative, a B Corporation, an a non-profit. We have the same goals of developing the best products we can an never selling out.
I wonder if you had any insight or suggestions on where to learn more about these options and the consequences. What do non-profits offer to funders? Everybody wants equity.
I've only used it for a local install when playing with my site theme, but it worked great (albeit quite picky about the versions of the dependencies).
Very cool and inspiring post. That said:
- Ghost's marketing materials tend to outright lie to users when making comparisons to WordPress or other platforms.
- I dislike OSS projects that hide their OSS nature. Why does a user need to pay $29/mo for an entry level blog? Why isn't there any obvious mention of the OSS nature of the project on the pricing page, home page, or features page?
- What's been done about the memory issues? Last time I tried ghost, it was a bit of a memory hog.
For reference the mobile homepage only says "The professional publishing platform"
However, I'm pretty sure a person that was really interested in self hosting would not have trouble figuring it out.
I don't necessarily think it's bad that they don't emphasize how to use the open source version. After all, it is a business.
"Being a non-profit means that the company has no shares. I don't own it. Hannah doesn't own it. Nobody owns it - it's an independent entity. The company makes money and pays expenses, salaries and taxes as normal - but there's no way for it to be bought or sold either in part (investment) or as a whole (aquisition). Any profit the company makes can only ever be reinvested, not distributed. We can't cash out. Ever. Also the entire product is open source and has no copyright. Anyone can do whatever they want with our code, for free."
I was under the impression being setup as a non profit grants various tax optimization strategies / returns. Which can be probably redistributed as salary.
But in this case it’s really an absurd implication. There is literally no personal benefit to founders making their company a non-profit. Yes they can pull salary from it, but they could do that in a for-profit company too. Not only that, but it’s impossible to sell a 501c3 non-profit because there are no shares. It can only dissolve, in which case the board must distribute any remaining assets to another non-profit.
Because one can cash out through a 500k a year salary
Seriously, looking for nefarious motivation here is really grasping at straws.
They are a Singapore non-profit. https://blog.ghost.org/moving-to-singapore/
I don't argue being a non-profit is a cause for suspicion in case of a saas, I just don't see why would it be noble.
Conversely, there is little to no reason for being setup as a non-profit in Ghost's case.
But really the important point is you cannot sell your company. How many years of 10% extra salary is worth sacrificing your ability to liquidate all the blood sweat and tears you put into your company?
As mentioned several times in this thread, our company is based in Singapore, not the US. The structures, laws and taxes are not equivalent.
I wrote a post about this a while back here: https://john.onolan.org/what-it-means-to-be-non-profit/
And how is it lying about their comparisons? It is indeed simpler to set up a Ghost site than a Wordpress site, and has more features out-of-the-box that you'd otherwise have to download sketchy plugins for with Wordpress/etc.
Ghost is 20 times faster. And once you add a single WP plugin, it's 100 times faster.
For security beyond running updated software, it includes a Wordpress-tuned web application firewall and the $24.95/month tier includes "Malware Scanning & Repair."
For 4 dollars a month you can.
https://www.siteground.com/wpbeginner-managed
50 dollars a month seems extremely costly. For that amount $600 a year it would be cheaper to hire someone to self host for you.
Here is an independent speed test which finds Ghost (more specifically: Node) to be up to 1900% faster than WP - https://blog.appdynamics.com/engineering/example-node-js-fas...
We don't make baseless claims and we're not in the business of lying to anyone.
Managed hosting - $29, you get your own server, app, automation, backups, and 24/7 support to make sure everything is working great
Centralised, multi-tenant application - $0-$9, you get no server. You get access to our app for your use. You cannot modify it or run any third party code.
We do managed hosting - and managed WP hosts... also do managed hosting. That's why it's a reasonable comparison point.
WordPress.com is not managed hosting. It's a heavily modified centralised multi-tenant application which loosely resembles WordPress. It does not run the open source codebase at all.
:vanishes in a puff of php:
It's still not equivalent to running the same code, or being in any way a comparable to a managed host. Which is the point that you skipped right over, as is your custom :)
I...am not sure about that. WordPress, for all its warts, is exceedingly easy for even non-technical users to get up and running; that was a huge contributing factor to its rise. And a large part of that ease of use came from its choice of PHP and MySQL.
I know that in 2018 it's a lot easier to get non-PHP web apps up and running than it was in, say, 2008, but with the exception of hosting providers that have gone out of their way to set up "one-click installs" for Ghost like Digital Ocean, Node, Python, et. al, are still harder. This is something that John Nolan's post pretty explicitly acknowledged:
"We spent several years trying to engineer our way out of this in increasingly complex ways, so that people could set up a publication on Ghost with the same level of ease as they do on Medium. We never even got close. It's just not how modern web technology works."
(And, no, "just use a Docker container" is not a viable answer to give to anyone who doesn't already know what container technology is. Sorry.)
> Decentralised platforms fundamentally cannot compete on ease of setup. Nothing beats the UX of signing up for a centralised application.
And self-hosting Ghost is just as simple as Wordpress. Instead of sudo apt-get install'ing apache and PHP, you're sudo apt-get install'ing nginx and nodejs. The ghost installer handles the rest.
I know people don't want to believe that and it's a difficult thing to accept, which is why most people just cling to the "non-profit" label, but reality is realty, whether you accept it or not.
I also agree it feels like static site generators are starting to take over (personally think its an awesome direction the web is headed towards for some sites) but not everyone is familiar how to use them so Ghost is another nice alternative.
I guess wordpress beat them in every possible aspect.
>How open source works is: If you want something, you can build it.
I read it as two parts. Interactions are flawed & Github doesn't really care about open source. Just one guys opinion, and my wife works for gitlab, but there is a deep commitment to open source and contributors, as well as an abundance of places for deeper dialogues to occur, between projects / contributors / the general public / gitlab.
+1 for also living in Minneapolis. We should get coffee.
How many HNers are around here?