Hacker News Reader

Comment by Scarblac | Hacker News Reader

In my first job (at a pretty small company) I needed to debug something. I made a dump of the production database, downloaded it to my dev machine, then deleted the live production database instead of the one on my dev machine by accident (typed the command in the wrong shell).

This happened while the CEO was talking to someone external to the company, right beside my desk. Little adrenalin rush there.

Anyway I had of course just made a backup, and only one order was placed while I was downloading it that I could restore by hand using the email that was also sent for each order, so no damage done. But I'll never make that mistake again :-)

View on HN

I think the "wrong shell incident" is quite common:

"Just need to load this dump to the testing database and I'm on my way to weekend...ok done...oh wait, if I look closer, this shell isn't the testing system...hell noooo, just nuked 8 hours of work each from 20 product managers."

Literally happend to me when I was a working student at a DAX 30 corporation. Never happend to me since then again.

It's common enough that I made sure our AWS boxes put the name of the current environment both in the bash prompt and rails console prompts (production is in bright red, staging is a nice soothing purple :).

I actually make a terminal-window-size PNG of 'PRODUCTION' and 'TEST' and set it as the background gif at like 10% alpha in the terminal windows, which also have different background colors.

I set the background colour in our team's shared iTerm config. Green for CI, blue for the test server, yellow for staging, and we don't have access to production!

The thing I like about our setup is that it's completely independent of individual developers' setups. Some of us use iterm, some of us are okay with terminal.app (iterm 2 ftw!). All of the backend capable folks (three of us) do have access to production, but we use it very responsibly (helped by the white on a red background production in the console prompt)

Oh boy, is it ever. I once had a Linux gateway without X installed on it, and I ssh'ed into a remote billing server. A friend called to see if I wanted to go out for lunch so I issued the shutdown -h now command.

I was trying to work out why the system wasn't responding right at the moment when I realised I hadn't logged out of the ssh session.

To prevent this I change the PS1 in the bash config. For servers I use a different prompt format.

For my local computer:

hollander@mypc 19:30:45 ~ >

For servers:

[hollander@server01 19:30:45 ~]

This all with coloring for name, servername, time and filepath, different for local computer and server. Root and username have different colors as well.

Local's are green, staging is orange, production is bright red.

Saved me a whole bunch of times when tired.

Can you paste in the shell codes you use?

Red

export PS1="\[\e[31m\]\u\[\e[m\]\[\e[31m\]@\[\e[m\]\[\e[31m\]\h\[\e[m\]\[\e[31m\]:\[\e[m\]\[\e[31m\]\W\[\e[m\] "

Orange

export PS1="\[\e[33m\]\u\[\e[m\]\[\e[33m\]@\[\e[m\]\[\e[33m\]\h\[\e[m\]\[\e[33m\]:\[\e[m\]\[\e[33m\]\W\[\e[m\] "

Green

export PS1="\[\e[32m\]\u\[\e[m\]\[\e[32m\]@\[\e[m\]\[\e[32m\]\h\[\e[m\]\[\e[32m\]:\[\e[m\]\[\e[32m\]\W\[\e[m\] "

As always you can tailor them if you prefer different things to be different colors, I'm often lazy and don't change local machines to green but always change staging to orange and production to red.

I had a developer who worked on my team at my last job make almost that exact mistake - I literally saw the blood drain from his face.

Fortunately we also had the DBAs in the same room - so HR database restored without anyone noticing. Poor chap - I think he got quite a scare.

Fun and games... :-)

We were a development team years ago (90s) working late on the production Oracle database writing some SQL (cringe). One junior dev deleted all records from one of the core tables and sat expressionless in total fright for a good 5 minutes. I let her sweat for a bit before I typed 'rollback;' Good thing she didnt commit and there was enough rollback space :) One of many close calls.

I did the same thing about 8 years ago but had to ask a more senior person to restore.

Reply on news.ycombinator.com