Introducing the Rails API Project | Hacker News Reader

Introducing the Rails API Project

blog.steveklabnik.com

174 pointstomdale86 comments

Wouldn't this be better as a new lightweight generator for the existing rails project? i.e.

$ rails new api

instead of

$ rails new

My concern is that rails-api will diverge from rails and I'll be stuck having to decide which is the easiest starting point where I want features from both.

It already is: https://github.com/rails-api/rails-api#for-new-apps

This is not a fork of Rails, it's just a set of plugins that work together nicely. Luckily, all the work on Rails 3 means that Rails is super modular, and it's easy to break off just the bits that you want. See Crafting Rails Applications[1] for more on this topic.

1: http://plataformatec.com.br/crafting-rails-applications

A few days ago I found myself getting a rather strange git error while running bundle after upgrading to the latest ruby. Turns out the active record serializer gem was moved just hours earlier to this rails-api location without any reason or notice. It was referenced by a git revision, because we needed the latest changes and it was also encouraged in the readme (as it still is now).

My issue with all this is that we already had tagged revisions in our application and this basically broke our older builds on which we actually relied on should things go wrong (the application is a rails engine).

So I think it would had been nice if this was announced earlier.

So sorry about this. I was travelling back from Europe, and we had to coordinate between everyone who owned the various gems, so there was a bit of overlap there.

I hope to not be encouraging people to use HEAD soon.

It's ok. I think we should had fork the repo and reference that in the first place.

I'm not trying but be harsh but I've never really understood the need for this project. I mean, Rails just boils down to a collection of gems right? So wouldn't you be best just cherry-picking your own gems? Or using something barebones to begin with, like sinatra/padrino?

Rails loads ton of stuff you don't need if dealing with a simples REST API, for example... So yes, you could only start with sinatra/padrino and add the things you want, or do like these guys and start with rails and remove everything you do not need.

I think that you've hit the nail on the head, and highlighted 2 differing styles of building web apps within the Ruby community.

There is a great post about this here: http://rubysource.com/rails-or-sinatra-the-best-of-both-worl...

If you're in the mood to hand-roll your own stack and figure out, on your own, how to integrate the pieces, you are not the target market for this project.

This project aims to provide the integrated development experience of Rails targeted at an API or JavaScript-heavy use-case.

I personally use Rails extensively for these use-cases, which was my driving force behind creating the active_model_serializers gem with José Valim. José and I also worked together on a project at Strobe that used a stripped-down, more API-focused version of the runtime stack.

Since Rails 3, Rails has been flexible enough for anyone to do this sort of thing, but Rails is all about having good defaults and leveraging conventions. This project is an attempt to see whether a relentless focus on the API use-case will yield some conventions that benefit those of us who are not using Rails primarily to generate HTML.

> So wouldn't you be best just cherry-picking your own gems?

I skimmed through the README. This project is picking up a default set of components needed for API. You can add or subtract components based on your needs. So instead of you cherry picking all your gems, you choose either none or some. For many people, having a default working set with non-needed components removed is good.

> Or using something barebones to begin with, like sinatra/padrino?

I have used sinatra and though I like it, I much prefer rails. Sinatra has its charm, but it's too open ended that in a big project, you end up creating half of rails yourself. Some people mount sinatra apps in their rails application, but I prefer letting Rails handle the api.

Seems like a great initiative. As a django user, I was using tastypie, which sits on top of django for creating an API. I agree that it's nice to reduce some fat, if it's really not necessary.

On a side note, and sorry for going off-topic, but referring to this comment

> Security: Rails detects and thwarts IP spoofing attacks and handles cryptographic signatures in a timing attack aware way. Don't know what an IP spoofing attack or a timing attack is? Exactly.

It's not completely transparent to developers, or it shouldn't be. If you're not careful, your rails app might be vulnerable to IP spoofing even now.

See https://github.com/rails/rails/pull/7980 and http://blog.gingerlime.com/2012/rails-ip-spoofing-vulnerabil...

> It's not completely transparent to developers, or it shouldn't be.

It should be as transparent as possible. Frameworks like Rails should expose the highest level cryptographic features possible and enable them by default. Secure-by-default is just another flavor of convention over configuration.

Rails doesn't exactly have the greatest track record when it comes to crypto or security but especially since Rails 3 they've done a great job addressing the problems, IMO.

It was probably a wrong way to put it. I apologise.

I don't doubt the integrity, diligence and efforts the Rails project is putting into security as a whole.

Secure defaults are very important. Crypto is important, but not all security is crypto.

In this particular case, and I don't claim that it applies everywhere, this transparency has a potential cost.

I agree that it should be as transparent as possible, and also prefer convention over configuration. But some time the convention means some added risk, and particularly an unknown risk (because by following the convention I don't even have to think about this risk). In such a case, I think configuration might be preferable. Conversely, for any built-in security to be transparent, it really needs to be watertight and fit 100% of the users. Otherwise it puts some users at a risk they are completely unaware of.

That's the case wherever trust is involved, though. Your ideal is also unattainable.

> not all security is crypto

Yes, but you started off talking about crypto so that's what I'm going to address...

Unless you're a cryptographer (and even then), any time you're using any type of crypto the risk model in your head is probably going to be completely wrong. I recently blogged about this:

http://tonyarcieri.com/all-the-crypto-code-youve-ever-writte...

Unless you plan on becoming a crypto expert your best bet is leaving cryptography to the experts.

> Yes, but you started off talking about crypto so that's what I'm going to address...

I didn't talk about crypto at all, and I don't see any connection whatsoever to this subject. Admittedly, even my comment was off-topic. Now it really goes way off.

The good news is that the Rails project has a robust security process, and because we handle IP spoofing, there's a place to discuss this vulnerability.

(See http://rubyonrails.org/security ; as a member of the security list, I can verify that we take every report extremely seriously)

That means that long-term, the number of vulnerabilities in Rails apps decrease, even for users unaware of the specific vulnerabilities. Again, that doesn't mean that Rails apps will never be vulnerable, it just means that you're sharing much of the responsibility for securing your app with many others through a project that takes resolving security vulnerabilities extremely seriously.

That's really good to know. I don't doubt the rails project has a good security process. I wasn't aware of this page, but perhaps should have searched better.

I believe that the security coordinator (Michael Koziarski) was actually involved on the discussion around this on github, so I'm not sure whether this needs to be forwarded to the email address again?

I'm not trying to make this into a huge issue, which in most setups and apps most likely isn't. I do think it's important people are aware of this, and if they are vulnerable they can and should protect themselves. I have suggested a number of workarounds to address this issue on my post in hope that people use those, whether or not the rails project as a whole is going to address the issue.

We're using this to build internal services - I'm currently wondering if there's a best practice for taking querystring params and mapping them to an ActiveRecord query.

Obviously, it's possible to build a "Searchable" module/class, but I wondered if anyone has already solved this problem?

Eg, pagingation, querying on date-ranges, ordering, filtering etc.

There's nothing like that in rails-api yet, but I can see how that'd be useful, for sure.

Does the hypermedia support exist now, or is that a future goal? I couldn't tell from your blog post whether the use of future tense "will" about most of the features implies "when you use it" or "when we write it" ;)

I also didn't see much in the READMEs or open issues having to do with hypermedia at a quick glance.

Also, LOL at steveklabnik2 ;)

> Does the hypermedia support exist now, or is that a future goal?

Future goal. There's nothing at the moment, but it's a case that we're interested in. I'd love to talk about exactly what we need to do on the mailing list.

> Also, LOL at steveklabnik2 ;)

Yeah, steveklabnik would still be almost in top 100 to this day, I just looked. Sigh.

As someone creating a large Javascript application with a Rails API, this makes me very happy. Perfect timing!

Awesome. Please let me know how I can help you build your apps more effectively.

I've built quite a few Rails-based APIs recently, and one of the major pain points (for me) has been the performance of generating the actual JSON in the payloads. I've ended up with a modified JBuilder gem and using a subset of its render methods, but performance is still pretty lack-lustre. Compared to to_json, it's absolutely glacial!

I'm wondering how the serialization in rails-api performs in this regard? I'm assuming as this is a core part of its differentiation from rails base, and so it should be better than JBuilder. Has anyone run any benchmarks for JSON rendering?

Forgive me if this is a naive question, this is the first I've heard of rails-api and I haven't explored the source or tried it out as yet.

It uses (well, encourages the use of) AMS, which, as I understand, is way faster than JBuilder. Don't have a benchmark handy, though, as I haven't run into this problem personally.

We'll be taking this for a spin. I've been pushing for an API-driven site for awhile but the case for that model was cinched when we started adding non-web clients to our system.

A leaned-out Rails is a nice compliment to other bolt-on API options and fat GUI-based API builders. Personally I prefer to start with something even simpler, but the facts are you can cover more ground faster (and potentially safer) with something like this vs. building your HTTP stack from scratch.

The Hypermedia stuff is most exciting to me, as hand-rolling that is a hard-sell for many teams (if you have a system of any significant richness).

Awesome. Let me know how it goes, please. :)

> The Hypermedia stuff is most exciting to me,

Me too. There's nothing concrete here yet, but there's lots of space to explore and good things to build. I really want us to be a leader in this space.

I'm very excited about this as well. Any hints about what's on the roadmap?

Would love to see easy versioning with custom mime types and link headers for pagination and associated resources. Maybe that doesn't belong in core... but it would still be pretty cool.

Nothing specific so far. AMS should know how to put out some kind of hypermedia-enabled type. We want to build a bunch of apps and then pull an implementation out, not imagine what we think would be good and then try to build apps with it.

My testing shows that versionist works just fine with rails-api apps to provide versioning capabilities. If you run into any problems, please file an issue.

https://github.com/bploetz/versionist

thanks. i'll give it a try... it wasn't on my radar. :)

What's the easiest way to make this work well with the asset pipeline assuming I wanted to make a single page app hosted in the same project? Or should I just not do that?

edit: thanks, guys!

The asset pipline isn't removed, so you don't need to do anything special.

That said, I don't think that it's ideal for heavy JS apps. Lots of possible work that could be done in this area.

> That said, I don't think that it's ideal for heavy JS apps.

Rails-api churns out JSON, JS consumes it. I can think of performance being an issue.

What are the other issues?

I've responded to your sibling.

Why don't you? Can you please elaborate on that? Thank you!

Basically, as it stands, you have to put your {Ember,Backbone,..} app in `app/assets/javascripts/app/models`, for example, which is kinda crazy. I'd like to see JavaScript become an actual first-class citizen.

Also, the pipeline has been re-written for Rails 4, but historically it's been kinda slow and buggy.

For an example of what I'm thinking about, rake-pipeline-web is interesting.

We used http://middlemanapp.com/ to build http://pollev.com/ (view this on a android or iPhone). Our entire mobile website compiles out into 3 files: JS, CSS, and a tiny html file.

This works out awesomely for a few reasons. First, our deploys for our mobile app is just rsync. No fancy cap scripts. Second, its super easy to put the mobile app behind a CDN.

One wrinkle in this approach is how to deal with dependencies between the Rails API and a completely separate JS app. We hack around that be packaging up our mobile app as a rubygem so that we can manage the dependencies through bundler.

I highly recommend this approach. Its a great way to separate out an API team from a front-end app team. All of our future apps are going to be written this way, including an HTML5 rewrite of all of our visualizations (from flash).

We plan on writing up a bunch of blog posts on the workflow, so stay tuned :)

I recently used http://yeoman.io/ for developing a backbone.js app, you can easily set it up to automatically compile (it uses Grunt internally) the individual sources to your rails apps public folder for development, and then compile/minify/concatenate your files for production. Worked quite well for me.

This is great, getting started guide would be nice too.

Thanks! Do you need something more than what's in the README[1], or do you mean I should have linked it from my post?

1: https://github.com/rails-api/rails-api#railsapi

Just the link would have been nice, I'm not jumping on raiks api right away, but skimming the getting started guide of something new usually helps me figure out if I want to use it or not in the future.

Roger. I'll add one, thanks.

This has come at a time I'm looking at Grape mounted in a Rails app for an API layer. Is there a simple way to explain to those of us with less API building experience what the pros/cons are of using rails-api over something like the Grape gem?

It's the exact same as Sinatra vs. Rails: build it all yourself vs. using a framework that has it built for you.

We've had a lot of love recently for Goliath + Grape, e.g. see: https://github.com/postrank-labs/goliath/blob/master/example...

Goliath is it's own asynchronous app server, and it wraps around the nice Grape API DSL. Works really well for little projects I'd rather write in ruby than CoffeeScript + node.js

Goliath is not only relatively slow, it limits you to using libraries which have both an EventMachine implementation and an em-synchrony-style Fibered wrapper around them, which is basically a scant fraction of the total Ruby libraries available. This is pretty silly.

It might be worth noting that Grape is Rack based and will work very well alongside Rails or standalone on top of Rack (sample in https://github.com/dblock/grape-on-rack).

Yeah just async HTTP/TCP, databases, and system commands. Not anything anyone really uses

There is certainly not an EventMachine counterpart for every Ruby library available today. Furthermore, the EventMachine versions of most libraries are second class citizens compared to the synchronous versions of these libraries.

Goliath/em-synchrony make the situation even worse: now not only do you have to have an EventMachine version of a library (which is already probably less featureful and more poorly maintained than the synchronous version), you need a version of that EventMachine library which has been wrapped with Fibers. This means your pool of available libraries is even smaller and poorly maintained than what was available with EventMachine, let alone what's available with synchronous libraries. And all of this is to get you an API which resembles what the synchronous libraries would've given you in the first place.

Rails(-API) is designed to work with that rich ecosystem of well-maintained synchronous libraries. Due to its design, Goliath can't work with them.

So the question is: why would you forego the synchronous library ecosystem in Ruby to get "fake" synchronous libraries that only work with libraries that are 1) built from the ground up against EventMachine 2) have been wrapped to look like normal synchronous libraries em-synchrony style?

To have a ruby HTTP server with async request handling and semi-readable code?

This looks great - I'll be trying it out for a project this week.

I'm still searching around for a good solution to API "views" or presenters when I don't want to expose all of a model's attributes. Something like Rabl? What do other people use?

RABL is very good at this.

The organization includes ActiveModel::Serializers, which is really important for giving you normalized JavaScript output. Rabl is slow and manual, which is the opposite of the Rails philosophy. Convention over Configuration wins every time.

ActiveModel::Serializers is exactly what I was looking for, thanks!

Hi Tom, we've been using a combination of responders and Draper decorators for this. It has been in production for almost a year and has worked out pretty well so far. Simon, one of our developers, wrote a couple of articles outlining our API setup:

Part 1: http://techblog.tribesports.com/blog/2011/09/24/versioning-t... Part 2: http://techblog.tribesports.com/blog/2011/09/24/separating-a...

Rails-api isn't a new project, and the name of the project is misleading. It provides a no-frills controller. Why is this is another project and not part of actionpack?

ActionPack already has enough debris in it that it's starting to look like a typical suburban garage so crowded with boxes and nicknacks that there's no room for the car it was intended for.

This is better as a separate thing. Maybe it could absorb the under-utilized ActiveResource subsystem.

Agree that actionpack is a mess, but I don't like all of these new projects separate from Rails. If they are going to be versioned with Rails, just put them in rails/rails.

ActiveResource is being pulled out of Rails 4 and being put into its own gem.

The gem 'rails-api' existed before, yes, but this announcement is about the Rails API project.

1. rails-api (the gem) forms the core. 2. ActiveModel::Serializers is now under the same organization, and making them work well together is a big priority. 3. Other gems/JS libraries may be built as we see fit, and put under the project umbrella as well. 4. I am now in charge of it all. ;)

> Why is this is another project and not part of actionpack?

Both of these gems were originally in HEAD Rails, but David reverted them.

> vBoth of these gems were originally in HEAD Rails, but David reverted them.

Will serializers be in the Rails 4 Gemfile at least?

It will not.

There _is_ a library called ActiveModel::Serializers in Rails 4, but it's much worse than the implementation that we have in the rails-api organization.

Would be interesting to see some simple benchmarks comparing a base JSON only Rails app with Rails::API.

The problem with benchmarks is that they don't really correspond to reality. Your app is not a simple test app. Your site will not be running on my MacBook Air. Your users won't have the usage patterns my script does.

This is true, but they're better than nothing.

A bad benchmark is worse than nothing

What is bad about the OP'd suggest benchmark?

I'll just leave this here: http://www.youtube.com/watch?v=xEJ1n13soWU

Of course. It's not about showing some absolute numbers that users can expect. It's just about showing the potential overhead of a full Rails app vs Rails::API. I'd think this would be a reason why people would chose Rails::API over just Rails.

The README itself says Rails::API is faster, so not sure why a simple benchmark to back that up isn't a good idea.

edit: It really boils down to another way to visualize the overhead vs just listing what's been removed.

IMO you shouldn't do the benchmark. Let this project be a starting point for a Rails api project and/or a how to for people who want to customize Rails for their needs.

"We are building an API. Rails has extra components which aren't used but might cause overhead. Here is how to remove it."

I think that's a valid enough use-case.

I agree. In fact, what would be really cool is to have a Single Page App's front-end as a template spec against which to compare different framework's performances against, a bit like what TODO-MVC did with client-side JS frameworks.

I have been using rails-api for over a month in a large project, this is port of a older rails app, empirical evidence app loads faster and memory footprint is small, as dependencies are less. I have made some additional changes, i use it with RABL, test suite is now separate and uses curb/faraday as API client. I am also considering repacking all my gems to make bundle install faster and a smaller size for standalone installations

Whoa whoa whoa guys, timeout! I thought Rails enforced REST conventions on all your models and therefore Gives You An API For Free? Wasn't that its big selling point? Or of adhering to REST in general?

Rails doesn't enforce anything - but it does encourage you to follow a RESTful style (not necessarily REST itself).

But the point of this is to strip out the defaults in Rails that are included because they are convenient in full-blown apps, that aren't needed in apps that only serve APIs (reducing bloat and probably making things faster). How much of an issue those things are in a real application will vary.

From the article:

> we can remove many parts of Rails that aren't important in an API context: many middleware don't make sense, and all of ActionView can go away

...

> Similarly, the structure of ActionController::Base is interesting: it includes a ton of modules that implement various features. This means that we can build an alternate 'controller stack' that doesn't include all of the ones that are in Base.

Interesting project, but if you're only building an API, using a much more lightweight combo such as nodejs + express seems better.

Why?

Because nonblocking is the secret ingredient in the async sauce, of course. With it, you go fast. Without it, you go slow.

Without all that callback spaghetti, how are you supposed to scale to roflmillions of users?

We really didn't make the choice to prepare for millions of users. We like having complete control over our code, and nodejs+express gives us that. I'm sure we make some mistakes Rails would correct for us and we take a bit more time on development, but we also learn a lot.

That's a tradeoff, and overall we're happy with it.

That depends on what you need for building the said API. Rails does a lot of stuff. Lightweight frameworks leave out many things out of choice. You possibly get enhanced performance with lightweight frameworks, at the cost of having to build some components. The choice depends on your "performance gain" to "painfully building components rails provides" ratio. node ecosystem has grown a lot, and it might serve your needs, but that's something that varies from project to project.

You've got a crack team of Rails devs and a deadline?

I challenge you to build some non-toy API with Node.JS and then try the same thing in Rails.

Express is decent enough, but it's barely an equal to Merb.

Challenge accepted: we have an API up and running at tldr.io :) We don't have a gazillion hits everyday for sure, and it may not be as complex as Twitter but: - It works perfectly - It really wasn't that difficult to build

The second half of that challenge is to compare to how hard it would've been in Rails.

I can also answer that as I've a fair bit of Rails experience. This time, we built our API using nodejs/expressjs for the first time, so we had to learn it plus the ins and outs of handling user login, sessions, CORS and a lot of small things Rails does for you. So we wouls probably have been faster with Rails.

If we need to build another product from scratch again though, we'll definitely be faster with nodejs. I understand that nodejs programming seems strange/hard to do in a non-spagetthi way to some people, but really, after a few months it is not hard.

Reply on news.ycombinator.com