The quick website launch checklist
newsletter.agiliq.com
newsletter.agiliq.com
> You may have a lot of things you do not want the world to know about. (Eg user uploaded media). Use robots.txt to hide them from search engines.
That's not what a robots.txt file is for. It doesn't hide files, if anything it makes files that you don't want the world to know about easier to find.
If you have a ton of user uploaded media (and such), you probably want to not want SE to index everything. (For example to save bandwidth).
I should probably re-word that line, but I dont want to get too technical.
Bandwidth is cheap. Ranking poorly in search engines is not.
If you've got a ton of user uploaded media, you've potentially got a ton of content for Google to pick up on and rank you for.
I'm curious, what is the standard practice for the storage of static media - where are the static files (text/images) stored on the server, and how are the appropriate config files that concern that media set up?
In a way, it gives you a bunch of excuses to put off your public launch - "I just need to fix some CSS bugs in Firefox 2" or "I just need to test the backup restore process one more time to make sure it works". Most of us aren't building bank software here, and if you're building anything consumer based (think Facebook), the advantage of having a site up and getting feedback on it over having all these things checked off is astronomical.
I few folks have emailed me with some great apps to do these things. I am tracking these here, and will publish these.
My abridged version of it: https://docs.google.com/document/d/1jJGUFN6CxQUXQX49mRsJYv1H...
It's also probably illegal in Europe now, though at least in the UK the authorities have declared a one year moratorium on taking action over it because the new privacy rules on cookies etc. are practically unworkable even if well-intentioned.
While I do not think it supports Do-Not-Track at this time, Piwik does have configuration options to respect emerging laws on data protection and user privacy.
[1] http://piwik.org/blog/2011/03/piwik-can-be-used-in-complianc...
Here is the spreadsheet I was using to think this post.
https://spreadsheets.google.com/spreadsheet/ccc?key=0Aqqb8SO...
If you have something, add this here, I will do a follow up post, with additions or update this post.
SQLInjections etc? You need to have that covered more than the rest in my opinion.
You could also let a program like Nessus run on your website. But that goes globally and will detect it.