Hacker News Reader

Comment by rjbwork | Hacker News Reader

This is a really good reason to be careful about what you log to log analytics platforms. I just recently implemented an ETL system that has the credentials (along with other stuff about the job) for data access passed into it from a PaaS framework. While I want to log the information, I don't want to log my DB connection strings! It's very easy to overlook such things and produce them as part of application logging/exhaust without realizing it, especially now that we have mass adoption of things like Splunk, Logg.ly, CloudFront, Cortana Analytics, Elmah.IO, LogEntries, Seq, and a dozen others.

View on HN

> It's very easy to overlook such things and produce them as part of application logging/exhaust without realizing it

Very true. It is a constant battle between debuggability and not leaking credentials.

In Erlang for example, when processes crash they dump their state, their neighbours, links to other processes, and other such useful stuff. That's very helpful, however it means it could dump credentials as well.

Luckily there a custom function to format the state of a process http://erlang.org/doc/man/gen_server.html#Module:format_stat... which helps with that. But have to implement that each process which holds credentials.

Also some of those log ingesting services provide a pre-indexer credentials filtering. I know Splunk has it:

http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Anony...

Of course it is better if it is filtered out before that. But it could be a safety net perhaps.

>Very true. It is a constant battle between debuggability and not leaking credentials.

And there are so many ways of getting bitten.

Another anecdote for you:

I recently found out that an API wasn't logging errors anywhere.

Easy enough to fix, plug in a library and bam, everything is nicely logged to the database.

Including the HTTP headers.

Headers that include API auth tokens.

Ouch.

I'm actually fixing an issue with this right now. My logging framework is capturing user passwords, so I've delayed the release one day while I blacklist certain known-sensitive information from the capture.

> while I blacklist certain known-sensitive information

Is there any way you can take the opposite approach and whitelist stuff to include?

That way you don't have to worry about stuff sneaking in in the future.

(not that blacklisting has ever bitten me in the behind before, no siree...)

Generally a good approach when it comes to security- default to no access. Much safer in the even of programmer error, misconfiguration, etc.

Alternatively use managed service accounts. You shouldn't even know or be able to expose the db connection credentials - let AD manage that for you. This is Windows specific advice and obviously doesn't count if your services don't support this (log a bug in the product if it doesn't).

Also Linux kernel logs. Kernel ASLR requires kernel addresses to be secret, yet dmesg is full of them sometimes and you usually ship those out over unencrypted syslog.

Completely agree. Here's another example: BitDefender (antivirus) passes your email and MD5 of your password in the hash when you want to go to your dashboard. When I contacted them more than 1 year ago about it, their "senior e-threat analyst" said it's nothing to worry about. Absolutely ridiculous.

What to do when the company is ignorant and continues to use something as stupid as that?

For this reason, and others (version control, backups), I think we need to move to key authentication, ring of trust, etc.

Reply on news.ycombinator.com