The sad state of web app deployment
eev.ee
eev.ee
But having tried out Docker for some production deployments, I think that it (or some less goofy successor) is the way forward. You get a sealed box with all the necessary dependencies, and the box has just enough holes poked in it that you can connect up the bits you care about. It turns apps into appliances, not things you need expert craftspeople to lovingly install.
As much as I have enjoyed 25 years of doing everything on an OS whose conceptual model is a 70s university department minicomputer, this era is coming to a close. We already know it's a poor match for existing hardware and use cases because we now mostly run it in virtual servers. But "virtual server" is the new "horseless carriage". It's a phrase that tells us we are living in a future we don't yet fully understand.
You'll need to learn OCaml (though in my mind, that's actually a major plus).
Caveat emptor: AFAIK, MirageOS is still undergoing heavy development. That said, I believe it is essentially functional at this point (pun not intended).
I am still hoping that something as radical as MirageOS will eventually take off, since we really seem to need it--at the very least, from a security perspective, if not a complexity one. Just by dumping C and Unix, and using a memory-safe language (such as OCaml) instead, we eliminate an entire class of exploits.
I will readily admit, though, that trying to force the entire world to use OCaml will surely be the fastest way to fail at this goal. Perhaps aggressively sandboxing is an adequate alternative, which would allow people to write code in environments sitting on stacks containing code from unsafe languages, without single exploits leading to entire system compromise.
Kernel ABI stability has brought forth options like running docker containers on FreeBSD and Solaris/Illumos/SmartOS Standardised workflows around "containers" are breaking down the need to care what your container is on the inside. Imagine running statically compiled Go binaries that have been compiled against s CloudABI tool chain deployed to CloudABI native hosting with maximum flexibility.
I've got a notebook full of details about "Linux compatible" system designs. Throwing away the Linux kernel for fun and profit! It's closer than you think.
There's a vast tectonic shift occurring and it couldn't come sooner -- "tinkering" with an increasingly complex environment of microservices, message queues, RPC servers, databases, caches, and so forth is headache-inducing and everybody knows it, as it detracts from actual application development.
I like the nest of snakes analogy, it's quite accurate. The twelve-factor application architecture[1] is here and many are moving towards it in one way or another, be it simply starting out writing unit tests or refactoring large applications into something easier to grok.
Docker seems to be moving forward really fast and gets a lot of hype which tend to make my old-school sysadmin self a bit suspicious, but I'd love to hear critique that is based on the actual usage. (N.B. I'm just about to start making my first Docker deployment)
Many people aren't ready to further abstract away from bare metal by adding a container layer for process management on top of the operating system which may be running inside a hypervisor. And there are situations where that's problematic. So it's not a great fit there either.
When it gets interesting is when you invert the relationship, where you have multiple apps in Docker. Then the main server starts to look quite simple and bare, much easier to maintain. And then things like ECS start to make sense:
At that point, the OS becomes irrelevant except as a host for containers.
It's a better, more isolated mess - but for anyone trying to enforce configuration policy into all of the running services in their environment, untangling gortesquely basic shit like not granting superuser privs on a database to your webapps - which would never fly in a traditional distro package - becomes even more work than the old source tarballs with INSTALL.txt.
The other security issue mentioned, piping curl into sh isn't really that big a deal. As long as you're a) using HTTPS and b) you resonably trust the source you were probably never going to read the script anyways.
I mean, when I first evaluated this app, it didn't even have a sane launcher script. Instead, ~15 lines of ruby config and a ruby script that I had to follow just to discover it was a weird, idiosyncratic way of executing "docker run".
Edit: I am not saying that Docker is somehow inherently flawed in this respect. It's solved a lot of problems for me and you can build great, well-architected stuff with it. But there is a trend to use it to wallpaper over poor development and deployment practices.
Now the nice thing about docker is you could have spent about 5 minutes to trivially docker run the app, decide if it's actually worth the effort, then untangle the mess beneath.
That's a net win in my eyes, though by the time I actually run something in production, I generally try and untangle the mess before I do. I also find it's easier to get help developers deliver something legible as sadly many of them often don't even know which packages they installed to get something running.
Listen, I'm not saying it's perfect. I specifically really don't like that Docker is basically root without Selinux (which is often turned off). I'm happy to see docker be replaced. I am however, not really interested in returning to puppet modules to handle dependencies when a container is, in my opinion, so much better.
This is the entire value proposition of all successful technology. Dubious business practices notwithstanding, all successful yet imperfect technology is successful because more people found it let them be lazier than the competition. But in reality of course it's not laziness, it's efficiency. They can get on with other tasks sooner than before.
Indeed, and I use Docker myself to improve my own efficiency, and I've seen great stuff built with Docker that has been architected well.
However, it is painfully obvious that without the pressures that used to force most developers to keep their shit sane, there's now more workload for serious users that actually need to untangle this mess in order to support, secure and get stuff deployed.
"I've got a weird set up OS for a reason, but this software should work even though no one else would ever run this."
"Whatever- this database should work, even though I'm using a 4 year old version of the free software(that the docs specifically say needs to be a newer version)."
"Also, I'm extremely inexperienced installing Rails and Rails apps, and despite the fact that it's a language and server that we literally teach the newest of the new, it's just impossible to do anything with."
This isn't a story about the state of web app deployment, this is about the state of the server from hell. This sounds like the kind of thing you hear out of megacorps with 20 year old Mainframes.
Should open source be supporting such convoluted, needlessly minor cases of awful environments?
I actually have a 64-bit kernel, but a 32-bit userspace
And it wasn't my choice — I started out 32-bit to save RAM on what used to be a fairly dinky plan. Some years later, Linode said I oughta switch to a 64-bit kernel for some obscure reason I can't remember.
That said, it represents a significant departure from being able to
apt-get install foobar
There's giving up on traditional package management, and then there's throwing yourself off the cliff of decent release/config/dependency management that used to be a side-effect of traditional package management.
"Works for me on my computer" is now "Works for me with my Dockerfile [ and all the configuration assumptions I couldn't be bothered untangling from my app ]"
When it comes to deploying, supporting and securing complex web application software, it is very rare that an out-of-the-box reference image from the project or vendor will have much relevance to your own environment's security and configuration standards.
Unless you're paying for support, and the vendor insists you'll have to suck it up and accept whatever madness their support agreement entails.
I'm all for convenient docker images, it just frustrates me to see the total and utter lack of imagination demonstrated by some projects that can't imagine their users ever locking down their database access, or running with SELinux, or separately patching the software/service components they depend on, etc.
And again, I don't expect the reference docker image for a given project to allow all of that flexibility and configurability, it's more the fact that these possibilities seem to be actively sabotaged with reckless abandon.
> If two features exist, someday, someone will find a reason to use them together.
I strongly believe that if you find a valid reason to use one specific brand of software, this has to be both clearly documented and explained. From my own experience, Ruby apps tend to be written with rather hard dependencies and are actually hell to deploy if any of the components you have preinstalled (be that process supervisor, message/job queue, web server) differ from recommended ones. Last time I needed to deploy a Rails app, I've gone down the route "bring up a new vm, secure it and deploy on that" and that was a breeze in comparison with attempts to integrate with already installed software.
Now, if there are simply two features, that if used together, fail for whatever reason? That's clearly a user pitfall, and should be fixed. But to endlessly try to support the Nth scenario is unsustainable.
Edit: Also, taking a look at what the author was(here: https://github.com/discourse/discourse/blob/master/docs/INST...), no wonder they want to use a system like Docker- it's an incredibly complex, full-featured setup. This isn't one-purpose software, this is a monolithic solution, that if you want to modify, you've gotta dedicate real time and effort toward.
This means that if you don't want to use defaults, you'll be constantly getting further and further away from the master project, which makes updating and maintenance so much more difficult. In which case- aren't you already signing up for the long haul, here?
I never complained that it didn't work with an old database. Ubuntu's versioning was just a fun surprise. I don't see why it wouldn't work with Postgres 9.1; I just figured I'd do the upgrade while I already had my foot in the door.
Maybe if installing Rails apps is still a huge headache after the fourth or fifth time I've done it, something is wrong with Rails. Maybe.
I have a fairly mundane server running the latest Ubuntu LTS with all stock vendor packages. If your app is such brittle crap that this is the "server from hell", well, it's no wonder everyone is using Docker.
If it helps, I'd be glad to throw in a free Digital Ocean $20/month droplet -- I can set it up following our guide at http://blog.discourse.org/2014/04/install-discourse-in-under... and then hand it over to you for everything else. We also have a special Mandrill reseller code so we can give you a Mandrill account with 50k emails/month as part of the package. Just email us at team@discourse.org and I'll make it happen.
(Obviously I am a fan, because PHP fractals, man.)
Yes, this will require Docker, which means a 64-bit OS and a modern Linux kernel. We initially tried supporting arbitrary Discourse installs but it quickly became a support nightmare for our small 7 person team. We adopted Docker because we saw it as the only way forward to have sane support both internally (for hosting) and externally. We are all-in on Docker, for what it's worth.
I'd argue Rails has historically had very little incentive to support super easy server installs; how many large open source Rails projects can you even name? Certainly 37signals nee Basecamp isn't too concerned about how hard it is to install their webapps on a server...
Long term, the only real solution is VPS and Docker. I think that has a lot of other benefits for the whole hosting ecosystem, too -- it opens the door to not just Rails deployments but all kinds of alternatives.
I'm sorry about your frustration -- I've been there many times, but taking your hellish experience and parlaying that into "web deployments suck these days because my development environment didn't meet the system requirements and therefore their methods suck" is a bit nonsensical.
I would direct you to the system requirements for the installation in question:
Hardware Requirements
Dual core CPU recommended 1 GB RAM minimum (with swap) 64 bit Linux compatible with Docker Software Requirements
Postgres 9.3+ Redis 2.6+ Ruby 2.0+ (we recommend 2.0.0-p353 or higher)
I would say that 32bit != 64 bit. Blaming the system seems like blaming a toaster that won't make tea. We could certainly hack the toaster into making tea, but your inability to quickly make tea with a toaster is not the fault of the toaster, toast or people that make toasters. Dismissing toast as a silly breakfast food and lamenting why more tea isn't made is pretty much the point of your rant.
You are certainly welcome to install the forum or do whatever it is you want, it is Open Source. However, it's rather uncouth to blame a process for which the problem was your situation. Rather than ranting about how badly Docker sucks, perhaps you could contribute back to the forum software in question and create your own Awesome Deployment System. Nothing is stopping you.
Nothing about Ruby forum software necessitates a 64-bit kernel. (Which I have, incidentally.) According to Docker, nothing about Docker necessitates a 64-bit kernel either! It's an entirely arbitrary requirement, inherited by the forum software for support reasons rather than technical reasons, and for some reason you are blaming me for not meeting it.
I certainly believe that people manage the 30 minute installation. I would've been much happier if I managed it as well.
And Web deployments must suck, or we wouldn't need Docker. Right? I mean, the whole appeal is that we can take all this arduous ad-hoc crap and just shove it in a box. Unfortunately, it's still arduous ad-hoc crap; we just don't have to look at it as often now.
Hardware Requirements
[...]
- 64 bit Linux compatible with Docker [1]
RVM itself is kind of a hack. Of course, if you have a working Docker environment, no reason to worry about RVM, as each container can have a full Ruby stack with whatever gems you'd like.
In an alternate universe where the author had a 64-bit virtual machine (takes about 55 seconds to set one up on Digital Ocean, with Docker pre-installed) I can subtract at least 8 hours from this story, as a prebuilt Ruby image and a prebuilt PostgreSQL image are both available.
Docker, like most kool-aid, is best if you buy in completely.
* Docker on 32-bit: weird, non-standard deployment * RVM by itself: not the best to begin with * RVM on 32 bit linux: not tested
Although I agree with the lack of dependency management on modern stacks and a few other points from this essay -- it seems like the core idea which led to this whole situation:
"I have a non-standard environment and it was tough to roll out things to it"
That's why nearly every newbie tutorial recommends Heroku.
Rather than accept that there are skills and experience they don't have, the author instead chooses to blame the entire world for their ignorance. The frustration on display is understandable and real and I totally get it, but the attitude is less forgivable.
Faced with a problem, instead stopping for a hot second to read some documentation, the author instead concludes that Docker is garbage, does some insane shit and then claims that the software industry has failed them. What incredible arrogance.
This article says way more about the character of this person than it does about the state of web app deployment.
e.g. Set aside the oddball tool breakage and consider that if you follow the instructions to the letter, this web forum requires: cloning (not installing!) the software’s source code and modifying it in-place. Copy-pasting hundreds of lines of configuration into nginx, as root, and hoping it doesn’t change when you upgrade. Installing non-default Postgres extensions, as root. Running someone else’s arbitrary database commands as a superuser.
and
You’d think that a web app could just have some metadata saying “I need Postgres and, optionally, Redis”, but this doesn’t exist. And the other side, where the system can enumerate the services it has available for a user, similarly doesn’t exist. So there’s no chance of discovery. If you’re missing a service the app needs but failed to document, or you set it up wrong, you’ll just find out on the first request.
and
"What we should have by now" section.
and
And I’m really not picking on Ruby, or Rails, or this particular app. I hate deploying my own web software, because there are so many parts all over the system that only barely know about each other, but if any of them fail then the whole shebang stops working.
For everyone else, it's giant shit sandwich, and a constant reminder that the project couldn't arsed untangling their own weird idiosyncrasies from their distributed app, and that this crap wouldn't fly with a traditional distro package, or wouldn't have mattered with the more traditional tarball/REQUIREMENTS/INSTALL.txt that made way fewer assumptions about the end-user's environment.
I love docker, but it's letting people get away with murder.
The ad hominem fallacy is a fallacy of irrelevance, of the form "the arguer's argument is invalid because the arguer has this negative quality," when the negative quality is irrelevant to the validity of the argument.
Well, maybe there is, but a lot of this article seems like "the sad state of nobody has this web app set up to be installed on my frankenlinux."
And the point of that is, once you know Docker, now installing just about any set of dependencies becomes a skill you know. I've never worked with Discourse. I've never worked with Rails. But because I've done Docker-based deployments, I bet I could indeed get it up and working in 30 minutes.
Incidentally, my hosting machine is basically clean - I don't use it as a personal or dev box, it does not have a ton of random userspace stuff in 32bit, it just hosts things.
>> The 30-minute claim came because the software only officially supports being installed via Docker, the shiny new container gizmo that everyone loves because it’s shiny and new, which already set off some red flags. I managed to install an entire interoperating desktop environment without needing Docker for any of it, but a web forum is so complex that it needs its own quasivirtualized OS? Hmm.
I understand the reflexive dismissal of things that become popular topics. I'm guilty of it myself repeatedly, most recently with "microservices." But sometimes technologies become popular because they are good and useful.
I haven't run into Docker's 32-bit install problem. They should fix that. But to dismiss the whole technology as some sort of obviously useless quasi-virtualized OS mumbo jumbo is taking the rant too far. Dependency isolation is a good thing. Deploying server applications with one command is a good thing. Knowing that your runtime environment is always the same is a good thing. Having a simple source controlled script that completely describes that runtime environment is a good thing. Some people try to use containers for unreasonably complicated things, or to hide unreasonably complicated software, but that is not an indictment of the technology. It's popular for a reason.
I can't tell you the number of times I've hoped to use a public docker image or just the Dockerfile, only to spend hours futzing with it because I was unhappy with the grotesquely insecure configuration or because I needed to work around a bunch of assumptions that are invalid once I've tuned the Dockerfile for my environment.
It's also quite likely that a pre-built docker image isn't going to satisfy everyone's deployment requirements.
Starting out with a philosophy that you'll make all the configuration and security choices for every user of your project isn't a great start, and that's how I read all these hard-coded assumptions.
That being said, it is still a bit silly how hard a Rails app is to install.
Seriously, things like uwsgi should have an "auto-configure" feature where when an error is encountered with a dependency, it searches the hell out of your /usr filesystem and caches the resulting configuration. If a module is missing, automatically try to apt-get and pip install it. Nginx should portscan the hell out of localhost and detect uwsgi servers. Install uwsgi if a wsgi app is detected but uwsgi isn't installed. Write the .ini file automatically. This process is so automatable that there really should be a "magic deploy" feature where I can just drop a Flask app as /var/www/html/some_app/index.py and it should be instantly up and running at http://localhost/some_app/ with zero questions asked.
After installing the required dependencies (downgrade VirtualBox etc) all that was left was: vagrant up. My lord, what had I gotten myself into. Que problems with the configuration, ruby version during provisioning, paths, puphpet had gone through multiple breaking changes in the meantime, the documentation was unhelpful. The only inspiration were GitHub issues on tangentially related projects ... all I know is it was 3am having stared 6 hours earlier.
Then, in a moment of madness I deleted everything and created a completely new/fresh puphpet configuration using the puphpet site. Again during provisioning I was met with a problem, there are non-ascii characters in the php-fpm upstart configuration file (the authors name!). Luckily, this was an issue already discovered a couple of hours earlier, so a small change to that and the app was up and running. It was 4am.
Needless to say, I was very frustrated with this state of affairs of these supposed aids to setting up the dev environment. Granted, my mistake was going in the wrong direction trying to fix an outdated configuration and all the problems it generated rather than just generating the config again, but I had never used this tool before, and hadn't looked at the PHP ecosystem for the past few years, it seemed crazy at first that a config which worked in the past, presumably, would not work on my very mundane Ubuntu dev environment.
I ended up doing deployment with git pull.
The sweet spot for Rails is custom apps that is continuously maintained over a long period, or discardable prototypes. It is not a good choice for deploy-and-forget, or organizations without any in-house programmers.
Heroku, for example, has a nice one-click deploy button for Rails (and many more languages/frameworks). It works straight from the source code, such as with this open-source rails app, and it's really quite impressive:
https://github.com/heroku/starboard#deploy-the-app
The author also calls out error reporting as being terrible. And there's also great tools for managing that, such as newrelic and airbrake.
So surely this author was just unused to the tool ecosystem, I thought. What a perfect opportunity for a constructive yet snarky comment! But lo and behold, discourse has deprecated all non-docker installs:
https://github.com/discourse/discourse/blob/master/docs/inst...
I was, and am, completely baffled at that decision. And I learned a valuable lesson about trying to out-snark snarky blog posts.
Actually, he never got far enough to see the error reporting tool, Logster.
The problem is not the state of deployment (which is still admittedly quite bad), it's the state of your system.
Install a fully 64bit version of 14.04 and watch all your problems just disappear.
1. Tinfoil hat time! "Pisshorse" make money off of hosting their own software. So they are financially incentivized to make it as difficult as possible to install yourself, but at the same time they get to go "woooo, open source" as much as they like.
I'm convinced Oracle did the same sort of thing by making their DB product impossible for normal people to understand, in order to charge outrageously expensive consulting fees to companies. Or so I hear; I've never used it so I could be wrong.
2. Your users aren't always who you think they are. I learned this one switching from back-end/n-tier work over to front-end CMS based web dev. Yes, the users are the people browsing the website, but the users are also the people trying to use the damn CMS, so the website UX and design extends to those people as well.
Meaning, instead of forcing your hapless marketers to use some crazy admin panel with thousands of options and checkboxes, or even try and edit XML configuration (I've actually seen this), any time one extends the functionality of a CMS, creating some custom front-end UI to control it is a basic necessity.
In the same vein, any sort of software (and hardware! printers, tools, cars, whatever) has to consider the installation and maintenance of itself as a UX/design concern and the fact that it has multiple domains of users.
http://blog.discourse.org/2014/04/install-discourse-in-under...
This does require Docker, though, which means 64-bit and a modern Kernel.
It's absolutely true that Rails is a very tough stack to install (and let me put my tinfoil hat on: do you think 37signals nee basecamp was ever incentivized to make it easy for people to install Rails apps?), but it needs to be competitive with PHP in the long term if we want things to change.
Docker was the only sane way we saw to move forward and support the community with our small (currently 7 person) team. We're all-in on Docker, all our internal hosting uses Docker, and it's the only officially supported install method for Discourse.
So never let it be said that we don't eat our own dogfood. Because we do, and it's goddamn delicious.
This, IMO is where Docker shines. It shouldn't matter if it's setup with a microservice 12-factor architecture or if everything is setup in a monolithic VM-like container. I don't have the patience for ops -- I just want something that works. That's the point of having isolated, replicable containers.
In any case, I encourage you to try out the discourse container on Nitrous. I was actually surprised it happens to be the least popular container for us. I assumed because it's such a pain in the ass to get started, that it would be more popular =p
So I think in the end this was just a case of really unfortunate timing, because if it weren't for the Docker installation issue, the only real complaint left in this post would have been the fact that the official method of installing Docker was to curl a script and pipe into sh[1]. And the rest of the post would have been singing praises of how amazing Docker is to be able to take setting up a Rails app along with all its dependencies and turn it into such a simple, painless process.
[1] Which is a perfectly valid criticism, by the way. They should really document the much saner method of installing through their official repos:
The whole article is basically a "there has to be an easier way!" infomercial for Docker, only it doesn't realize it.
The article is pretty ranty but I can agree with the author that many things nowadays are way more complex than they have to be. As a plug, I'm going to say that this is one of the reasons why we started the CloudMonkey.io project. It lets you deploy a docker container with no fuss to production. It's up to you, however, to ensure that you don't over complicate your system unnecessarily.
I've done the mistake in the past where very early on in the project I started using a web server, Redis, ElasticSearch, MySQL, memcached, RabbitMQ, etc.
In most cases, more than three moving pieces only bring you headaches. Now I always try very hard to keep things simple to at most a web server and a database, plus maybe a memcached caching layer. If you need to have a queue or full text search functionality, I'd try to bring it in as an outside service.
Just learned the trick to become root when you belong to docker group. Awesome
I think he's talking about Discourse [1]. I tried Discourse few years back when it was released but it was too bare-bones at that time. Haven't tried it recently.
- Besides a web app, we must also create mobile apps, and they usually don't share much code. It would be much better if apps could just run HTML and Javascript instead, and still be integrated like native apps are now.
- Every web app that is deployed now is not guaranteed to work X days into the future. In fact, it often breaks. We need better guarantees for this.
- Browsers are incompatible in subtle and not-so-subtle ways.
- It is way too complicated to write a web-browser. The number of abstraction-levels combined is too large. Security problems are therefore unavoidable.
- On the front-end, React is a suboptimal solution because it requires O(N) time and energy for simple O(1) updates. We need smarter incremental computation techniques.
- The primitives for the web (HTML+Javascript) have been created for novices, instead of for developers. However, even novices can't read the W3c specs, so it doesn't really help anybody.
There is also a very usable "postinst" installer shell script that can do additional fixups. It can handle things like "if we are on openvz do this, otherwise do that" or "if this is debian 7 (wheezy) fix this; if it is debian 8 (jessie) fix that".
If that is still not enough -- which it sometimes isn't for really complex platforms -- I write an additional shell script that will do the remainder of the work.
But then again, that probably means that the image will have too many moving parts anyway. That means it is time to split the image and stuff a network protocol between the parts. You can use virtual machines for that. The front-end goes left, the back-end right, and other things go top and bottom. Now we are back to more simple images to install and to test.
Learning some of the innards of the package management system of your machines is surely a good idea in itself, if only because you are relying on them so much. Even if you just say "screw it" and use shell scripts to deploy anyway.
There is, but the influx of hacker types into what's an engineering discipline just reset what the field collectively knew to do (when was autoconf introduced again?).
Hell, even Docker can work well if used properly. The secret here is to keep your development machine clean and standardised - this is what Vagrant is for, right?
Done.
"Sandstorm is an open source platform for personal servers."
What is it?
So my only alternative would be to run the service myself, but then I'm storing other people's data, I have to worry about scaling if lots of people use it, and user accounts, and all this stuff.
The idea of sandstorm is folks run this platform on their personal servers, and then it lets you browse an app store like interface and one-click install these server side apps. So I'd bundle up my budgeting rails app as a sandstorm package, and if someone wants to track their expenses from a variety of devices, they install the app. Now they're running it so the data is theirs, there's no scaling issues, and user authentication is provided by sandstorm.
Sandstorm is still in its infancy so there's not a lot of apps available and the development APIs are being worked on, but I hope it's the future. It would lead to a more decentralized web with better privacy and users owning their own data.
I'm hopeful, if not optimistic, for a future where every family is expected to have their own little server running somewhere. And they access that server through the sandstorm web interface, and can easily add little apps to it. My budgeting app, a webmail app, some future federated profile app to replace facebook, etc.
With that said, it's true we've had a tough time coming up with a two-line summary that fully explains what Sandstorm is. If you could suggest what would have made more sense to you, that would be helpful. Otherwise our strategy has been to try to push people towards trying the demo, which I think illustrates what it is much more quickly than words can.
There's also my fave, Cloud Foundry -- try Pivotal Web Services or BlueMix.
These PaaSes have a buildpack model and basically, it Just Works.
Or OpenShift, though maybe its shift to a pure docker-y model will put you back where you started.
Disclaimer: I work for Pivotal Labs, part of the company which donates the most engineering to Cloud Foundry.
Aside: complex software is hard to deploy because it's complex. Software gets complex because devs are very comfortable managing large amounts of complexity. Docker lets you hide that complexity so someone other than the dev can do the build/test/deploy steps, but it does nothing to address the high level of complexity in the underlying software. Let's not mince words here: Discouse is hard to deploy because Rails wasn't designed to make its projects easy to deploy by someone other than the devs primarily working on it, and no amount of tooling around it will change that.
Meanwhile, the users of the forum that I've seen, mostly at TDWTF, seem to mostly complain how short of features and functionality it is/was compared to other BBS solutions.
So which is it? Is it too complex, or not complex enough? It seems to be doing all right at the current level of complexity. If you're suggesting a different one, what makes you think that would be better? Better for who, exactly?
Also note that Discourse is trying to take business from a pile of well-established competitors. To do that, you generally have to be not just as good, but much better, and with flashier features too, for both the users and the mods and admins.
Particularly the mobile experience, which is supposed to be one of those things Discourse was designed to do well.
Which is what the whole rant is about.
You can't deploy an asp.net app that has a database and other external dependencies on a clean install by just `xcopy`ing some files around.
Demanding your users manually install prerequisite software -- especially dependencies that are already included with their OS distribution, and when it's not clear that it's necessary -- is not good practice. And if newer versions of such software are needed, that's what "vendoring" is for: you package all the dependencies along with the software. Sure, it means you might have a duplicate copy of Postgres or whatever, but at least it works out of the box.
I bitch just as much as the next guy about the mistakes that have long since been baked into the Java environment, but that dedication to backwards compatibility does pay off. Similarly, the fact that it's running on a JVM rather than bare metal has its performance costs, but it does allow you to trust that the actual computation will Just Work wherever you run it (although of course the stuff that touches the surrounding environment will not).
In the comments the author states that he wants something "less invasive" than Docker. I'm not sure what that actually means - for me, having to actually install it and deal with the versioning on my real system is more invasive. Containerized applications that build their own runtime environments is a step in the right direction there, just like the JVM. Docker has some really dumb defaults right now, running as root has been known to be retarded since forever and you should not encourage users to do it. But running as root is problematic for lots of other reasons, and causes the potential for jailbreak on other jails too (eg chroot). FreeBSD jails seem to have some good rules on what you're allowed to do inside a jail.
If you really want "minimal invasiveness" and your software really really requires that you have superuser privileges then you should probably be running a full-fat VM with an OS inside, with the network sandboxing happening at the host. Even then, it's not like even Xen hasn't had the occasional jailbreak. The real problem is that the program itself should never require running as root in the first place - but that's a problem with Discourse, not a problem with jails.
cf push
Actually, I'm lying. Here's how I deploy:
git push
cf push
Ops is hard dude. Sorry about that.
Isn't it suppose to address most of those grieves the author was complaining about (and I have also experienced so I have up on Ruby years ago and did some nodejs but that's not really different either)?
With Docker apart from the app you have to deal with the additional complexity of a single app environment that is not an environment that most users are familiar with, for instance you need to launch all your apps in 'non daemon mode' (how many uses know how to do this, and why you would need it?) figure out how to deal with storage persistence, networking and 'linking containers' and things like logging, cron and ssh (there is no place for these in single app containers). Installing something as simple as Mysql with Docker is non trivial because of storage. So this adds a whole new layer of complexity on top of the app that's makes it completely unsuitable to package apps if simplicity and accessibility is the objective. You need to be a real expert to deal with this, much more than LXC containers or VMs.
We have a guide on building a simple Wordpress stack from scratch with Docker [2] that gives readers an understanding of the level of complexity involved with Docker and what it seems to be designed for. It's not for end users and certainty not to making things simple. It would take a tenth of the effort to install Wordpress in a VM or LXC container. But its pointless to compare VMs or LXC containers to Docker, as Docker is intended for a completely different use case and that's what its users and advocates should highlight and push.
We have packaged over 60 apps in LXC containers at Flockport and Ruby apps are by far the most complex to setup, usually requiring a complete build environment. PHP is the simplest both to install and troubleshoot, and usually just works. The Ruby apps like Redmine, Gitlab are more complex and Discourse is the toughest to get going. You need to be an expert to install it and an even bigger expert to run it so much so that I think unless you are a rails developer the only reliable way to use Discourse is to have the Discourse team manage it.
Then you install more, configure more if you want something more advanced.
"Hosting Rails applications is complicated. Even if you already have Postgres, Redis and Ruby installed on your server, you still need to worry about running and monitoring your Sidekiq and Rails processes. Additionally, our Docker install comes bundled with a web-based GUI that makes upgrading to new versions of Discourse as easy as clicking a button."
The basic install guide is a bit lengthy, but amounts to writing these commands:
ssh root@machine.address.of.choice
wget -qO- https://get.docker.com/ | sh
mkdir /var/discourse
git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse
cp samples/standalone.yml containers/app.yml
nano containers/app.yml
./launcher bootstrap app
./launcher start app
For small things it works well and I've never had a single installation as painful as any of the stuff described in this article. For big things, well there's Facebook. It may be ugly, but it's definitely doable.
It's not the new shiny, it's full of warts, and it's unfashionable. (If you tell a fellow programmer you code in PHP you immediately need to rebuild credibility somehow.) I personally prefer working with JavaScript and nodejs but I don't think the deployment situation for nodejs is much better/different from what's described here.
Yeah, it's really not. npm is designed by monkeys. It tries to be too general and do too much, so they thought a completely open format would be kosher. Turns out, it just allows everyone outside npm to define their own pet solutions for interoperability (a word that zero developers under the age of 30 have heard today). There is no standard way to do the simplest of things. Say, bundle an ES6 file or a CoffeeScript file or Sass, or whatever. "Preprocessor? What's that?" says npm. Oh lordy.
What's worse are the standards for documentation today. I'm finding the answer to about 99% of my problems in a fucking github issue thread. Our industry is completely doomed if this continues. Software we depend on is abandoned within months.
Is this one of the reasons people use Java? Is it way wrong to say Docker is like a JVM for non-Java?
If this thing is so simple a toothless toddler could use it, how come now we need experts in it, huh?
I'm not gonna rehash the full deal, everyone knows the "PHP is a fractal of bad design" article. Whether you think that's an exaggeration of the problem's impact or not, it's hard not to agree that PHP has a lot of features that are counterintuitive, encourage bad coding practices, and make modularity/expansibility of the codebase a real challenge. PHP is fine if you do it right, but it's really a challenge to ensure that a bunch of volunteers are doing it right, 100% of the time. It's not a coincidence that the people who are using mega-scale PHP codebases like Facebook aren't actually running PHP - they've gone through and cleaned up some of the awfulness and run a language based on PHP. Those are the design decisions that the actual PHP standards group isn't competent to make, because anyone who's sane has run for the hills.
Discourse is indeed extremely overengineered, but I don't think the urge to start over on a modern language that has better design features is wrong per se. If you had a web forum written on ANSI C or COBOL, wouldn't you say that at some point the costs of an inappropriate platform or a limited pool of engineering talent outweigh the cost of just rewriting it? You don't have Facebook-level resources to refactor the whole platform, you don't have billions of dollars of sales riding on your software. It's a comment forum. To throw your comment back - if it's just a simple app to display text on a page, why not fix the technical debt and put yourself on the right long-term track?
This reminds me of what Jeff Atwood had to say about people over simplifying the nature of stack overflow (and by extension, claiming it to be over valued).
He eventually responded about the fact that the little details mattered. From auto searching for existing similar questions to little prompts on framing the question.
Similarly, Discourse is being built with principles in mind that go beyond a threaded discussion. It's attempting to answer the question of "can we use software to encourage a civilized discourse?". It's easy to ignore this, and look at it as bloated. But in that vein, Stack overflow is a bloated Yahoo answers or any other. And we all know it isn't. Essentially, it's more than just "a web-forum software". That distinction really matters.
Relevant links:
1: http://blog.codinghorror.com/this-is-all-your-app-is-a-colle... 2: http://blog.codinghorror.com/code-its-trivial/ 3: https://news.ycombinator.com/item?id=678501
After carpet bombing this thread with my own criticism, this was a good reminder of why that was probably stupid. Thanks :).
The reality is that almost nobody uses it that way, it's not tested for that and due to the huge API surface area the chance of some kind of exploit is so large that it cannot be secured even if it people tried.
The only reason "root" exists currently is to make it less likely to accidentally change or destroy the operating system.
There is no way to provide a multi-user experience on a physically accessible system (cannot defend against physical attacks), and the way to provide it on a remotely accessible system is to assign a virtual machine to each user.