Hacker News Reader

Comment by tenderlove | Hacker News Reader

I'm just commenting here so that people can have a central thread for love / hatred. ;-)

But seriously. This is extremely critical, please upgrade!

Thank you for your fast hard work on this - this is what makes the Rails community so great and why I love it over the other options out there.

I know this has to have been a stressful weekend. Is there a tip jar anywhere for beer money for the team that worked on this?

You'll have to ask @bitsweat. I only accept hugs. <3

hugs += 1

I'll throw in on Gittip: https://www.gittip.com/tenderlove/

Would also be happy to give you a hug ;-)

And you can give him hugs here: http://www.hugboard.com/e5f69b274a

hugs += 1

As there have been many exploits / issues recently realized in parameters parsing, why isn't there more of a focus on security here? Specifically, this is where users/hackers can put ANY DARN THING THEY WANT and your server has to deal with it.

As a simple solution, one could pass a signed auth-hash of the fields generated by form_for, and the server could re-hash the fields submitted to ensure the form data you asked for is what you get (this solves the primary issue with attr_accessible). I feel getting this right is crucial to Rails' future.

> why isn't there more of a focus on security here?

More compared to what, exactly? This vulnerability was responded to pretty damn quickly after it was reported, given that almost nobody is even paid to work on Rails. If you saw Aaron tweeting about "working over the weekend" a few days ago, well, now you know.

That said, you mention attr_accessible in your post: that's gone as of the next release of Rails. Basically everyone agrees that strong_parameters is a better approach, which is why a gem was released that works with 3.2: you can use that better approach now.

As was mentioned below, security is a process, not a result. Nobody wants these kinds of issues to happen, but they will happen, and do to every single framework that's used by lots of people.

If you've recently started a new Rails project (or are thinking about it), you should use strong_parameters right off the bat!

Agreed! That is what I teach when I do training.

Why is strong_parameters not the only way? Rails is pretty opinionated in many other respects...

It will be the default in rails 4... so soon : )

It was opinionated to use attr_accessible until a better approach came along. Beginning in 4.0 it will be opinionated to use strong_parameters, but they can't just take attr_accessible away because a lot of people are upgrading apps.

The downside of 'don't break APIs for minor version changes' means that you can't break APIs until the major version changes. ;)

I'm not 100% positive, but I don't think strong_parameters would do anything to protect you here.

It would not have helped with this vulnerability, no. But it is a better way of handling filtering appropriate parameters.

This is actually the first I'm seeing about strong_parameters, since I don't follow Rails 4 stuff much right now. Thanks for the tip!

Any time. Please let me know any issues/problems/thoughts you have with it.

+1 Steve

Part of being a good developer is understanding your framework and making sure that your app has the level of security you need. The framework cannot protect you from everything.

The team responded really quickly. Aaron is a really talented developer and a nice guy. We should all be thanking him. If you don't feel that enough emphasis is being put into security in rails beat away at it, find holes and then get involved in developing fixes for them. That is the beauty of open source.

Matz developed Ruby for adults. You get a lot of power, but also a lot of rope to hang yourself with. This is why testing in rails applications is so important.

http://blog.phusion.nl/2013/01/03/rails-sql-injection-vulner...

Here are some good and experienced Rails developers who apparently had no idea that Rails would automagically suck in XML and YAML and turn it into symbols instead of strings.

Clearly they aren't the only ones who didn't "understand the framework" or we wouldn't have gone a week with the impression that CVE-2012-5664 was only exploitable in specific circumstances.

That's not my point....

I made the "understand the framework" statement in response to some mistakes I have seen introduced by sloppy dev's on some rails projects I have worked on...That obviously wasn't the case here.......

If security is important you should periodically try to hack the system and consider incorporating automated penetration testing software against your application. I haven't had a need to go this far with my own recent apps so I cannot speak authoritatively on this, but I think there are some tools that can help with doing penetration testing etc..

In a past life I had to work on some pretty secure systems and did some crazy testing on things. I saw a lot of good developers introduce pretty big security holes....my favorite was when our /etc/password was served up by an application...and this was a well known team of craftsman that did this on a fairly large project. None of these have been limited to Ruby projects....they have included Java, C etc..

In my view security is a moving target. If the cost of a attack warrants the effort to protect against it then you do. If not then you don't. Even if the developers of the framework concentrate on security, there will always be ways to get around it. Safe's are rated on the amount of time it takes to break into them, if someone wants into bad enough they will get in. The same is true with software.

Should I be more aware of the security on my apps? probably. Should we as a community be better with it? Yup. But unless I've taken the time to really dig into it, offer constructive feedback and be willing to jump in to fix it I have no business criticizing the state of things.

And I say all of this as someone who loves Ruby.

I'm just saying Ruby is good at enabling DWIM and Rails has taken it and run with it to the point of magical thinking.

It's very hard for an app developer to test for vulnerabilities such as this one which seem to involve a combination of contributory factors. When magical stuff is constantly willing to help the app developer out in the background, it's very difficult to get a handle on what our true attack surface is.

Rails has definitely started to get complex, which is why many have chosen to go towards the simplicity of Sinatra, Rack etc.. Any time a framework get's complex this happens.

Look at big data and AI. There are loads of permutations. How do you prevent stuff from just happening? How do you make sure that things are correct etc.? Trust me I would love to do TDD on an AI / big data analysis project. But the reality is that no-one has figured out how to reliably test things....so TDD is not the right tool at this point. But there is also the potential to introduce vulnerabilities. That doesn't mean I am not going to use AI or do big data.

It's always a balance between tighter security/less complexity, or more complexity and less security etc...obviously there are other factors as well, but my point is to choose the right tool for the job.....sometimes it is Rails, and sometimes not......

And beyond that give the Rails team kudos for a taking care of things like this when they do find them.

Let us agree in giving the Rails secuity team many well-deserved thanks!

We on the sidelines primarily criticise as a stage of our shock and grief. That the patches keep flowing is unsettling in the short term but reassuring in the long term.

Thanks again everyone who dropped everything and worked so hard to get this set of fixes out!

I really hope this isn't the sentiment of the RoR community. There has to be a place for critique when it's warranted. At this scale it's not a joke any more, you ask for what you need and otherwise you do less.

I specifically asked them to elaborate on what could be made better.

Less magic.

The more magic, unexpected behavior you have when parsing untrusted input, the more likely you are to have security holes.

Instead of building up some complex object based on untrusted input, the author of the application should specify the values and types expected, and the parser should parse those and nothing more. This would lead to much simpler code paths, as the user never has an object that has unexpected keys, values, behaviors, etc. Don't parse the object using complex, general purpose code, then hand the user an object that they have to treat specially; if their form only expects 5 values of given types, then parse only those values and those types.

The problem is, all of this kind of magic is at the very heart of what Rails is. I don't know if you could eliminate it all and still have Rails be Rails.

This sounds a lot like strong_parameters, which (I believe) will be the default in Rails 4, and is only a gem install away from Rails 3.2 apps.

https://github.com/rails/strong_parameters

It will be the default, yes.

There are specific things that could be said about the bug in question, like not being secure by default, but this doesn't fix the underlying problem. The development team should recognize that security is an important part of the project and act accordingly.

Seems reasonable. How did they not 'act accordingly' in this case? What should be different about the security process currently in place?

Nothing gives me confidence in a platform like "almost nobody is even paid to work on Rails."

I'm sure some Core developers would love your sponsorship to give even more time to devote to Rails development.

Even without the money, this issue was dealt with swiftly. See the comments below about 'the Rails security team is the best vendor I've worked with.'

Yes, judging platforms by how many people are paid to work on them instead of track record or other verifiable merits is totally the best way to go.

I agree, this is a dangerous sentiment.

Whether money is involved or not, for a framework, developers should either be committed to their products or not.

If this was a different sort of product then that limitation of not getting paid might carry some weight, but when you encourage people to develop on top of your platform and when it is shown to have egregious flaws there is no excuse. You either get to work fixing them or you tell the world to stop using your framework because it's broken and not going to be fixed.

Fortunately, the Rails devs are seriously committed to their product, which is why these fixes came out so quickly.

Edit: The Rails team is certainly deserving of many thanks, but they don't get a pass on problems just 'cause they work for free. Similarly, if someone gives me a free car I will thank them, but if that car starts a fire in the garage and burns down my house I will curse them too.

I thought some come of the core committers had their work sponsored by their employers.

Aaron is the only person on core who is paid to work on Rails. (Among other things.)

I am a committer, and part of my job is to work on open source. Ish.

Other than that, it's everyone else's spare time, IIRC.

Wow, that actually gives me more confidence because we have people doing it because they feel passionate about it. Thanks for your hard work....

I've been busy with some other small OS projects (and stuff that pays the bills) but personally feel like I need to try to carve out some time this year to do something to contribute back to Rails.....

If I can help you help us somehow, please let me know.

Cost/benefit.

There are always going to be security holes in anything we make. We can be a bank and focus two feet ahead on making sure everything is as secure as possible, or stay aware of security (and not do anything stupid) while moving fast enough that any flaws are irrelevant/fixed when exposed.

It also highly depends on how much risk you're willing to accept. For the average rails app, absolute security is not as important as moving fast. Be an adult and make adult decisions about your tools and processes to suit your circumstances.

> As a simple solution, one could pass a signed auth-hash of the fields generated by form_for, and the server could re-hash the fields submitted to ensure the form data you asked for is what you get (this solves the primary issue with attr_accessible).

It does not solve the issue of javascript generated forms.

Sure, but as with all things, it could be turned off. The more I think about it, the more I like this idea. I may as well try it out and mock up a pull request.

If you turn it off then you're back to square one security-wise. Apps that have neither APIs nor JS are an increasingly small share these days. Also consider what is possible to sign. In most cases there will be some non-enumerable data in the field, leaving you with only being able to verify the field names, but there could be nested data and it seems like a 50/50 shot that whatever unforeseen vulnerability would not need to change the top-level params anyway. I don't think this would afford much of a security guarantee.

The only way to fix this by "more of a focus on security" would have been not to do clever things with parameters in the first place, but the clever things provide a lot of value, so the next best thing is security auditing and be on top of patching any vulnerabilities.

>why isn't there more of a focus on security here?

Because this is ruby we're talking about. A "Fun" language that has 100000 ways todo the same thing, so newbs find it fun and easy. You can almost guess how the language works and almost always be right. Thats cool, great for learning, makes you feel like a superstar when you're just getting started with programming... but its really not such a good thing when it comes to maintainability, and security.

This breads a community of people who arent very mindful of anything but having fun coding. (not always a bad thing, but certainly not conducive to good security)

The second you talk about "Multi-platform" or "security" to your average ruby user, their eyes glaze over. They just want to make cool stuff on their Mac, not worry about Security and best practices.

You could say the same thing about a lot of interpreted languages, but Ruby is especially bad.

Yes, you're right. There have never been any security problems with "serious" "nonfun" languages like C, or Java, or the .NET stack. :-/

This has nothing to do with the Ruby language, by the way, any more than a hole in IIS is a problem with .NET.

If you're going to talk smack, at least learn what you're talking about.

Hey, did you hear that scanf() will accept XML and YAML and automatically convert it into pointers handy for you to dereference?

Neither did I.

Or how about when you put that 256th character in a ruby string and end up overwriting heap metadata? Man I hate it when that happens.

There are known undefined behaviours and there are unknown defined behaviors.

Which is worse?

I'll take Ruby over C any day of the week.

No formal argument there.

How about informal?

Nope, just actual.

You misinterpreted my comment. I agree, the language is largely inconsequential. Its the culture around the language that is the problem. Ruby's is particularly bad. All I'm saying is the "1000 ways todo the same thing" nature of Ruby, has the unfortunate consequence of attracting newbs, and making 'best practices' hard to nail down.

Other languages tend to have more support in corporate/educational areas, tend to have more money backing them, tend to have more 'best practices', tend to have more rigorous testing and review. Ruby is the hipster hacker's language.... and the quality of code shows this. (in the core of the language, and by the individuals who use it)

Other languages absolutely have these problems too, but it has been my experience that Ruby is particularly bad. You are welcome to disagree with that part.

But IMO it makes sense. The quick and dirty 'million ways todo the same thing' nature of Ruby breads this kind of culture. I certainly didnt mean to single out interpreted languages tho, or imply other languages dont have security problems, or unique issues with their cultures.

PHP is probably about just as bad. I'd put Ruby and PHP high up on the 'fun to program in' list, and low on the 'secure, quality languages' list.

I still believe you may not understand the difference between a language and a framework.

The distinction is rather academic with ruby and rails. 90% of the answers to "how do I do this in ruby" on forums are actually "how to do this with rails" answers, but they never mention that little detail, because who'd ever write a ruby program without rails, right? Trying to find straight ruby answers is annoying as hell.

I've had frustrations with certain "ruby" libraries requiring methods like "blank?" (IIRC) that are provided only via Rails, not Ruby. It made developing on a machine that Ruby but not rails rather annoying.

"The distinction is rather academic with ruby and rails."

No, it isn't.

"90% of the answers to "how do I do this in ruby" on forums are actually "how to do this with rails" answers"

You're looking in the wrong places.

"who'd ever write a ruby program without rails, right?"

People who are writing shell scripts? People who are using Sinatra? People who are writing desktop Ruby programs?

I haven't seen many seasoned developers claiming PHP is a 'fun' language to program in. It was my first web development language and it was fun back then. But the honeymoon period gets over quickly once you realize the limitations of the language and see what other languages like Ruby has to offer.

"quality of code shows this. (in the core of the language"

Your examples of this would be?

Sounds like PHP >_<

I don't think so, a lot of ruby users are good at security

So far as I can tell, there are a seriously large number of people paid to work on Java. The security record of Java is way worse than Ruby.

I can tell you that "security" is a topic that, unless handled carefully, will make anyone's eyes glaze over.

Remember when release cycles where short in Java? Me neither

I'm curious. Is there any objective data apart from your anecdotal generalizations that can justify the claims?

Most Ruby developers primarily work with Rails. I presume this is the reason you are conflating the framework with the language.

That's an interesting suggestion but it has nothing to do with the problem at hand.

The problem here was a feature somewhat haphazardly added to Rails for ActiveResource was turned on-by-default and enabled features that should only be active for interactions with trusted clients (i.e. authenticated services running in your own infrastructure)

Your suggestion is not without merit, but this is a case of having to learn to walk before you learn to run. There are clearly much more egregious parameter parsing vulnerabilities which need to be solved before the things you're describing would ever make it into rails-core.

Hey, Everyone! Leave a note (and, if you want, a hug picture) to say thank to Aaron for his hard work. This isn't any easy responsibility. http://www.hugboard.com/e5f69b274a/contribute

Getting continuous errors on deploy during the bundle stage like so:

  /usr/lib/ruby/1.9.1/rubygems/remote_fetcher.rb:215:in `fetch_http': bad response Not Found 404 (http://bb-m.rubygems.org/quick/Marshal.4.8/activesupport-3.2.11.gemspec.rz)

Is this because rubygems.org is being nailed?

seems to be - mine is stuck at getting metadata from rubygems.org for 45 minutes. is there an alternative server to fetch this from?

It's been an hour, but you could also fetch straight from github with this in your Gemfile:

   gem "rails", :github => "rails/rails", :tag => 'v3.2.11'

For a less hammered server, can use http://bundler-api.herokuapp.com

I usually just use that one for my apps, much faster than rubygems.org

I'm getting exactly the same thing. Glad to know it's not just me.

For the time being I've rolled back the gem changes and applied the suggested hotfix (removing XML from the default params parsers). I've been trying for a solid hour to get a deploy out with the updated rails version, and it's just not having any of it.

Edit: finally got it out. This deploy model is completely screwed, though. It just shouldn't be normal to have a service like rubygems.org in the daily deploy loop. This is absolutely not a knock on the fantastic volunteers that run it - they simply shouldn't be dealing with this sort of load spike.

This is a mirror, so that's why it's probably out of date...can you try just production.cf.rubygems.org ? I have heard zero reports of downtime or other issues. :/

http://uptime.rubygems.org/131647

Hi - thanks for responding. My deploy's gone through now, but I'm curious how I ended up contacting a mirror - I was running the default bundler capistrano task, so (approx):

  $ cd <deploy-path> && bundle install --gemfile Gemfile --path <shared-path>/bundle --without development test

I haven't deliberately pointed anything at a mirror - do you have any idea how might my install have ended up doing so? This is with rubygems 1.8.11, bundler 1.0.21.

The Rails security core team and the awesome collaborators deserve a raise. Thanks for your continued vigilance and effort.

These are the commits that need to be pulled in, right?

https://github.com/rails/rails/commit/d5cd97baa44fa66dc68104...

https://github.com/rails/rails/commit/43109ecb986470ef023a7e...

Are there others?

3 links on HN frontpage for this same vulnerability proves the love of the community to warn each other tenderly.

There's actually two vulnerability announcements: https://groups.google.com/forum/?fromgroups=#!topic/rubyonra...

This one deals with problematic JSON parsing and affects only 3.x. It is dealt with in the release that fixes the other vulnerability

Thanks for your awesome work and continued vigilance Aaron!

LOVE FOR TENDERLOVE! <3 Thanks for the prompt action.

Aaron, thank you for all your hard work and making the Rails community fun to be a part of. ... upgrading now!

Agreed. Watching this develop from behind the scenes (at a distance), I'm not sure any core team could have done better managing the deployment and announcement in a more timely and careful fashion. Well played.

Could you confirm that upgrading to the fixed Rails versions means I don't have to add the line changing `ActionDispatch::ParamsParser::DEFAULT_PARSERS`? Or do I have to do both?

Thanks!

Just one or the other.

All kinds of love.

MUCHO LOVE, ALSO GORBY!

Are there any reports of attacks using this vulnerability?

Thank you Aaron and the rest of the Rails core team. We really appreciate the speed with which you addressed this!

Thank you for all your work!

<3++

love++

love += 1

Reply on news.ycombinator.com