Self-Host Everything | Hacker News Reader

Self-Host Everything

slashgeek.net

40 pointspavs65 comments

Totally OT rant here: more and more I'm seeing certain grammatical mistakes crop up that disturb the hell out of me. I understand that the first two 'w's in www are for 'world wide,' and that not everybody is a native speaker of English.

But these things are creeping into native speakers' use also. They are:

1) Confusing the words 'lose' and 'loose.' Lose is the word you use when you mean the opposite of 'find.' It is also used to mean the opposite of 'win.' Loose is something that is the opposite of 'tight.' There is no 'loosing,' (EDIT: a1k0n corrects me here, see his comment below) there is 'loosening,' however, as in what you do to your belt after a large meal. OP makes this error.

2) Using apostrophes for plural. Apostrophes are used to show (a) possession and (b) contraction (leaving letters out of a word). Writing "I'm buying two car's" is incorrect. Replacing your car's winshield wiper blades is ok. So is leaving out the car entirely when drawing a 'toon.

To be even more pedantic: There is 'loosing', as in releasing, but it's probably not what you mean unless you can already spell 'lose'.

I had to look this up and you are correct.

My favorite mistake along these lines is "payed", which is a nautical term and not synonymous with "paid".

AFAIK both forms are acceptable [in en_GB] for both the past participles of caulking your boat and making a financial transaction, cf http://en.wiktionary.org/wiki/pay#English or http://oxforddictionaries.com/definition/english/pay--2.

This IMO highlights why one shouldn't get too anal about specific word forms as long as the intent is apparent. Of course one can generate inaccuracies this way but generally an honest attempt to receive the communicated information doesn't gan aft agley.

The English language has a lot of quirks where the pronunciation doesn't match the spelling or two words are pronounced the same but are spelled differently. This disparity has been enforced by the school system and media publishing gatekeepers for a long time but now that anybody can publish anything on the net a more loose way of spelling is proliferating. It will probably just get worse over the next few decades.

My favourite is guessing whether there is an s or c in some words, sometimes it feels completely arbitrary. I cannot count how many times I've misspelled 'choice' (actually misspelled it just now but the spell checker caught it).

"It will probably just get worse over the next few decades."

Languages evolve; some of the current "mistakes" will no doubt become new norms. The changes that stick will tend to be those that cause no ambiguity and confusion.

Have to admit, I smirked that SlashGeek is outsourcing its comments system to disqus.

Edit: even better, the Disqus comments are now failing to load with the message "Sorry, we're undergoing temporary maintenance."

I would say you should own your own domain and only use services which can hang off your domain. That way you get the convenience of third party services, but any time you want/need to, you can shift to self-host without having to change your email/blog/whatever addresses.

I totally agree. I started on the Internet circa 1998 and I've always regretted using popular free services back then, because most of them didn't survive to our days (the popular "X got acquired by Y, X was closed").

I got a domain in 2002 and since then I've used it as part of my identity. It costs some money, but not much and it's totally worth it.

Non technical people may freak out a little bit when they ask you for your mail address because you don't answer with the expected gmail.com (and hotmail.com before that), but that's all right.

EDIT: typo

Self hosting is fun and fulfilling. It's simple and cheap. The problem of course is replicating the services you already use. Web hosting is painless but email is absurdly involved if you want to make sure that gmail etc. accept your messages as non-spam. The rules are not made clear and there doesn't seem to be a universally accepted way of setting up email where you can say "this message I'm about to send will 100% be accepted by the recipient's mail server"

I've never had problems to get my email accepted by gmail, but well... I don't think Google gets a 100% either.

There must be a checklist somewhere, but off the top of my head:

- Your server's IP must be static and must have a reverse map (IP -> name); check your IP is "clean" and not in a blacklist, specially when you get it from your ISP range (may be someone used it before you and it has a poor reputation).

- Setup your DNS correctly using MX records.

- Use SPF or DKIM so other servers can detect forged emails not coming from your server(s); besides this is nice to filter backscatter spam.

- Obviously don't configure your mail server as an open relay, use SMTP with SSL or TLS, always with user authentication.

I've been using this setup for years and most of my problems are receiving mail and never sending it (ie. misconfigured servers not playing nice with graylisting, Google servers blacklisted by my DNSRBL, etc).

EDIT: formatting

Thanks for sharing the tips. Will give email another attempt soon, and maybe write something up

email is absurdly involved if you want to make sure that gmail etc. accept your messages as non-spam.

I've self-hosted DNS, mail, web, etc. for years, and several people have posted the same kind of comment here, so here's what people will need to know/do to self-host their own mail.

First, you'll almost surely need a non-end user IP address, as many/most of such address ranges are in the Spamhaus PBL list [0]. To see if your IP address is listed in PBL, for the example IP address 192.0.2.200, do

dig 200.2.0.192.pbl.spamhaus.org

Here, the octets are reversed and prepended to pbl. While you're doing this, you may as well also do

dig 200.2.0.192.zen.spamhaus.org

Again, replacing the above IP address with your own. This checks the Zen list [1].

For a non-end user IP address you'll most likely need a datacenter machine, a business line, a VPS, or a machine hosted at a large organization. A home Internet connection won't cut the mustard for mail.

Second, you'll need FCrDNS [2], or at least rDNS. rDNS is just short for reverse DNS, which means having a PTR record in DNS for your IP address. It could be anything, but something non-generic is best, such as mail.example.com, or puffy.example.com. wireless-cust-0-200.example.com is generic and will score "bad" points on some remote systems.

It's best if you can do FCrDNS, which is trivial. All that entails is matching PTR/A records.

This is all you need for Gmail or any other serious mail provider to accept your mail. Unfortunately it makes self-hosting from home more difficult, but the upside is a huge reduction in spam.

Last thing, you should check http://dnsbl.info or a similar site to see if your mail server's IP address is listed in any DNSBLs.

0. http://www.spamhaus.org/pbl/ 1. http://www.spamhaus.org/zen/ 2. https://en.wikipedia.org/wiki/Fcrdns

Thanks for this.

One problem with self-hosting is that getting security right and tight is really hard. You have to be aware of all the components used in your stack, and monitor them all for known security problems. And then know how to react to them.

And then you probably want backups, and monitoring, and what not.

Ideologically I'm fully a favor of self-hosting many bits, but having some experience with it, I'm also acutely aware of the downsides.

Somebody needs to come up with a universal self-hosting plan that automatically captures everything I create on to a box where I have total ownership. I'd pay for that. So if I reply to a friend on FB, it captures the thread including my reply. If I tweet, it captures that. My email would be a separate folder. And so forth. Heck even lifelogging would go there.

In this fashion the things I create in my life would be under my control, at least to a certain degree. Computers are an extension of our minds, and we deserve to own and control our output in one place, even if we choose to share lots of things in various forums.

interesting you want a box with total control but your 2 examples are using social networking with a history of increasingly locking out 3rd party tools. with that said you can download your facebook and twitter history. I have for facebook and the static html output isn't that bad. could be worse anyway.

Have you considered (for the time being) moving to e.g. Tent (https://tent.io/) and/or Diaspora (https://joindiaspora.com/)? Of course the whole trouble is migrating your current implementation-dependant social circle (e.g. facebook friends), but I've seen people tweet their last messages along the lines of "I'm not staying in twitter for reasons X,Y; moving to Z (join me if you'd like [to follow me])"; my (very limited) exposure to these cases would seem to show that it might just work (to an extent / YMMV) for (at least) folks on twitter with significant amounts of followers.

In any case, I like Tent as they are developing / have developed a generic distributed social networking protocol (with some sample implementations etc.) [/old news]

That's the goal of the Locker Project: http://lockerproject.org/

It seems it has stalled, though :(

Small plug related to the learning curve. Many of the most popular apps, like ownCloud, WordPress, etc. you can get free, easy to install packages at http://bitnami.org You can choose to self-host (downloadable installers) or run on Amazon/Azure

I've self-hosted as much as I can since 2000 and as with everything, one should weigh up the costs of self-hosting versus the benefits. First, honestly assess your own level of skill. Then pick a service and decide what it would cost you to host and the cost of failure. For example, let's look at hosting your own email service.

The cost of hosting for someone not comfortable with email is quite high. Setting it up correctly requires many steps and if you get it wrong, perhaps your emails don't get delivered and you lose business. Compare this to the cost of using Gmail - effectively zero. However, this does not take the risks into account. Google owns your email and email address. If they, or one of a number of governments decide that the service should no longer be available to you for whatever reason, your email is lost. You could mitigate this by mirroring all your Gmail through IMAP/POP3 and using your own domain name. In security terms, the risk of Google taking away your account is low, but the impact is very high.

As an aside, I'd love to use Owncloud but after a incidence where it deleted files, I decided that it's not yet ready for production use.

For me it's not so much about hosting it myself as it is having control of my data. It's a trust issue.

I used http://github.com/coto/gae-boilerplate to write a few sites which host data objects in GitHub gists. Here's my OpenStack blog which uses it: http://StackGeek.com/, and an example of a raw post stored in a gist: https://gist.github.com/kordless/93e491c4880a416907be.

I still rely on GitHub for keeping my data safe, but most of my code there also lives on my desktop in the form of cloned repos.

Coincidentally, I just gave a talk last tuesday at the Philadelphia LUG on DIY email hosting at home. http://jve.linuxwall.info/ressources/taf/DIY-eMail@Home-PLUG... [PDF, 4.5MB]

I wouldn't use something like owncloud as a dropbox replacement right now if you were looking to go ahead with this. It seems to be going through that phase where the security bugs are being identified and fixed.

I self-hosted for a long time, knowing it breaks my ISP's ToS. just recently, I received a letter regarding the web server and an extremely throttled connection until I shut it down.

If you use just a couple services, good. Now imagine self-hosting all of these:

    - version control
    - bug tracker
    - invoicing app
    - document sharing
    - project management
    - file server
    - mail server
    - static server
    - ruby/node server
    - php / wordpress blog

Just off the top of my head. That will cost you a lot of time and head-banging.

I do at least some of this with the server in my basement.

* version control - git and gitweb - I mirror all my repos to github for discoverability though * invoicing - I've done this with text files to track hours and PDFs for invoices (via LibreOffice) - low tech but simple enough * document sharing - my home dir on my server (weak, I know, but simple) * file server - see above * mail server - postfix, amavis, and spamassassin aren't too hard to set up * static server - apache or nginx are trivial to set up * ruby/node - well, I mostly serve Perl stuff, but same deal using Starman - this isn't too hard either * wordpress - yep, installed this too

Things I don't host myself:

* bug tracker - I mostly use rt.cpan.org and github * project management - I use Hiveminder

Now if I had to learn all this from scratch it would indeed be a lot of time and head-banging, but I've learned about this stuff over many years of using Linux and hosting my own stuff. And they're all useful skills to know as a developer.

I think any developer who's working on web apps should understand at least the basics of how web and mail servers work. Even if you outsource this stuff to a CDN and something like Sendgrid you should know the tradeoffs involved instead of just picking the outsourcing because you can't set it up yourself.

a home made home server with amahi / tonido / own cloud would take care of a few. Add pancakeapp for invoicing, bugify for bug tracking. The mail thing still a problem.

I can be fun if you have your own server, I have webhosting with 1and1 but no virtual/dedicated server. So hosting everything myself would cost me money to take the server.

Also I don't find it very secure, you would need to have multiple backups to make sure nothing gets lost if your server gets erased.

What is the backup guarantee of 1and1 ? Usually hosting providers don't give any guarantees for that.

And even if it feels safer because you assume they have people working on it that know what they're doing, in many cases this is a very bold assumption.

How would one go about self-hosting a website? I have shared hosting but would love the challenge of setting up a LAMP server that is physically in my possession and have full control over it.

I'd say get a cheap VPS from a service like linode or digital ocean (there are many others listed on lowendbox.com but I would recommend sticking with the well-known/bigger providers) then take it from there. Most major distributions have tutorials in their wikis for setting up commonly used software--linode even has a community resources section with a lot of material for server software.

Edit: My comment was slanted towards linux and I forgot to mention that maybe some familiarity with linux would be useful, but windows is also an alternative (usually more expensive).

If you have a spare computer laying around you could put Ubuntu server on it. The stack setup can be pretty easy for something like PHP or Django.

You'll probably be behind a dynamic ip, so go to dnsdynamic.org and set up your server to be "followed." (You set up a cron job to ping dnsdynamic.org periodically, and dnsdynamic.org updates its record of your ip. So you get to your server by going to myproject.dnsdynamic.com rather than myproject.com.) Set up your router so the only port(s) that is/are forwarded is/are to the port(s) you want to access (perhaps, ssh and http ports). Configure your ssh daemon to allow only private keys to log in (that is, turn off password 'logging in') and don't allow root logins.

Set up rkhunter and denyhosts.

Then, sit back and watch the logs as the attacks come in.

Some ISP will assign you a static IP if you ask. I've never been turned down.

Good tip. My biz isp sells them for $30/month. I'd get a linode first.

But with rsync, dvcs (I use hg), and even davcal (which I turned off for some reason I've now forgotten- I just use Google), life is pretty sweet.

Strange stuff. My current dirt-cheap residential connection assigned me a static address for no cost.

Static address or dynamic-but-never-changes address?

My IP almost never changes (it's been the same for at least two years), but I still can't run an SMTP server off of it, since it's listed in Spamhaus' PBL[1], so many servers refuse my mail.

[1]: http://www.spamhaus.org/pbl/

Certainly static. My hostname is .static..com.au.

Linode has some excellent step by step guides on setting up a server.[1] You could also start with a virtual machine using those same guides.

[1] https://library.linode.com/lamp-guides

Second this, even if you don't use Linode, their docs are great.

If you're in it for more than just the learning experience, hosting at home will probably give you bad/unreliable upload speeds (assuming you have a typical ISP). A VPS service like Linode or others would give you basically the same learning experience with the added benefit of a good connection.

Grab yourself a Raspberry Pi. You can set up a LAMP stack, and it'll even stand up to moderate abuse. Best of all, it's cheap.

Do you have any links to tutorials on getting LAMP up and running on a Pi?

I've none that I know of personally, but a quick google shows an absolute tonne of tutorials. There's really no shortage of instruction when it comes to setting up a basic webserver.

There's probably a boxen script for getting a lamp stack up and running on a lot of linux boxes:

https://github.com/boxen/our-boxen

It'll save you a lot of the configuration problems initially, until you're ready to configure.

while part of me knows that a Pi will work, I feel like minimum for a home server really should be an intel NUC mini box.

I've had a 450mHz G4 PowerMac running my local services for close to five years now. There's really not really a whole lot home server needs to be doing, CPU wise.

Pretty painless:

1. Get an old computer and plug into router

2. Install ubuntu server

3. Buy your domain from namecheap.com and look up their documentation on dynamic DNS (assuming you are on a home net connection

I did self hosting for years (age 16 - 24) with the same K6-II box. Just recently retired it and use a VPS now, but I highly recommend doing it to anyone

I have steps 1 and 3 already, now its time to fire up ubuntu server.

Good to hear. The Ubuntu install process will eventually ask which services you want installed so just check off SSH, apache, mysql, etc

4. Fight with your ISP who doesn't want you hosting services on a "home" internet connection.

Honestly this might work OK for a personal blog or something but is not a realistic approach to hosting anything with volume or for business purposes. Also you will still probably need to pay for SMTP service as there is almost no chance of anyone accepting mail from your self-hosted SMTP server when the reverse lookup resolves to an e.g. Comcast residential DHCP name.

Of course you aren't going to have access to outstanding bandwidth, but I've used it for business purposes for years. Whether or not you would want to use it for business depends on a number of factors, but I don't think you can flat-out say it isn't realistic for any business.

I've personally never had issues with my ISP blocking it, but I read accounts from time to time (in this thread for instance) where people had issues with ToS enforcement. You'll have to evaluate your own situation, or just give it a shot until they explicitly react.

But you are right about email- it's a pain. Most likely you will have a bad time trying to get something authentic-looking working with a home ISP. Even doing it on a legit VPS is fairly complex.

self hosting an e-mail server is a headache.

Generally you don't want to host email or DNS yourself but I think if done right and you keep them on their own servers it can be great in the long run.

Well, the headache is in making careful selection and configuration choices.

Done properly, it really requires very little ongoing maintenance. Maybe a few hours per year. DNS requires even less.

Spam filtering? Security updates?

I am a fan of self-hosting, right up until the point it crashes on you and you have to spend a weekend fixing it. You are right, email is one of the biggest headaches out there.

Depends on your experience level. If you have never setup postfix/exim/sendmail to do more than accept local mail then it will indeed be a giant headache. With a bit of work with dovecot and postfix you can have yourself a very nice mail-server. That said, there is a point at which the self-hosting pain point is not necessarily realistic, and if you have two accounts you need and they're $4/month elsewhere, why bother would be the discussion to have.

Add to the list of reasons it's a bad idea: Security.

Unless you're confident you can securely configure your network and services AND stay on top of patches, etc it's probably a bad idea to self host.

Couldn't we start a co-op where every member user/company/organization owned the servers? That way nobody is hosting on a third-party server where the federal government feels that they have eminent domain?

A hosted OpenStack controller node which controlled a bunch of servers sitting on people's refrigerators in their garage would do the trick. Would have to figure out routing.

It's an interesting concept, but I'm not sure the financials would add up to just renting you're own 1u collocation space with bandwidth.

It's an extremely cheap setup if you don't need to worry about downtime in the event of hardware failure

There's also Project Byzantium which was started by the hacker space in DC: http://project-byzantium.org/

They were asked to assist with Hurricane Sandy relief efforts as well, and the project seemed to be successful.

Whether the servers are owned by a co-op or a big company, if the U.S. federal government comes calling, you will have to answer if you or your servers are in the U.S. That is just the nature of a government.

Reply on news.ycombinator.com