Hacker News Reader

Comment by old-gregg | Hacker News Reader

Absolutely my favourite recent development in the Python world. I won't be mistaken if I say that many of us have built half-arse versions of this on top of urllib(2) multiple times in the past. :-)

The support for keep-alive is particularly welcome, huge kudos to the author.

Question to users of other languages: do equivalents exist for PHP, Perl, C# or Java? Ruby folks rave about restclient, but is there anything else?

View on HN

For Perl, I've mentioned this elsewhere here but Mojo::UserAgent looks quite neat. This time with Mojolicious url - http://mojolicio.us/perldoc/Mojo/UserAgent

Interesting things can happen with M::UA in a single line:

  use Mojo::UserAgent;
  print Mojo::UserAgent->new->get('http://freegeoip.net/json/8.8.8.8')->res->json->{country_name};

Very powerful.

Indeed. And there are many more nice examples in the cookbook: http://mojolicio.us/perldoc/Mojolicious/Guides/Cookbook#USER...

Also check out the command line interface which allows you todo things like this:

  $ mojo get http://mojolicio.us 'head > title'

ref: http://mojolicio.us/perldoc/Mojolicious/Guides/Cookbook#Comm...

For Ruby, there's also Faraday https://github.com/technoweenie/faraday.

Which is used on the Twitter gem for ruby: https://github.com/jnunemaker/twitter/blob/master/twitter.ge...

For Ruby there's httparty, https://github.com/jnunemaker/httparty, curb, and a couple of others that escape me right now.

Here's the new one I'm working on:

https://github.com/tarcieri/http

httparty does a few of the same tricks, but I still find its API a bit obtrusive. At least it automatically parses JSON! (my library does this too, if an appropriate library is loaded)

In addition to a humane interface, it will soon be backed by a Ragel-generated HTTP parser:

https://github.com/tarcieri/http/tree/master/parser

Did you get to re-use any of Zed Shaw's excellent parser work from Mongrel?

Zed had two parsers: a server parser as part of Mongrel and a client parser as part of RFuzz. I actually used the client parser from RFuzz in my Rev/Cool.io libraries. I talked to Zed about his plans for these, he wanted to do a unified parser that can handle both HTTP requests and responses (since the only difference between the two is the first line) but he never got around to it. Both projects are now unmaintained.

Carl is working on a comprehensive event-driven HTTP (and Websockets) framework for Clojure. I'm just borrowing his parser and re-wrapping it in Ruby. I prefer to use Carl's as he's (at least for now) actively maintaining it, as opposed to using Zed's abandonware.

Looks very promising. I see it currently uses Net::HTTP under the hood - do you plan to keep it that way, like Httparty and Faraday, or are you going to do away with Net::HTTP altogether? (The comment saying "this is temporary" next to requiring 'net/https' is encouraging... :))

I plan on eliminating Net::HTTP entirely and writing my own client using Carl Lerche's parser for his Picard library which is written in Ragel:

https://github.com/tarcieri/http/tree/master/parser

Also Typhoeus and RestClient.

Probably others.

In the FAQ it says that support for keep-alive is on the way.

I do have a half-baked implementation on urllib2 that I hate. This is really cool.

Keep-alive has first-class support in the soon-to-be-released version!

Is it just me or does the FAQ break the back button?

I always use libcurl for PHP... don't really have any complaints.

C# / .NET had System.Net.WebRequest since 1.1. It's similar in principle to Requests.

No, no it's not. (Http)WebRequest IS AWFUL. I built RestSharp because HWR is so bad. WebClient is closer to requests, but still horribly awful if you know anything about HTTP.

The new System.Net.Http.HttpClient in .NET 4.5 is closer to Requests and not entirely bad, if a little rigid.

Just out of curiosity, what's so awful about WebRequest? I've been using it for years for various things and never really had a problem with it.

I'm not your parent poster but I personally dislike that you have to instantiate a separated Response object, and that you have to fetch a stream, even for tiny responses. In those Python/Ruby frameworks it's just one line. I always end up writing a wrapper just to change it.

And this one is a pet peeve, but I hate that verbose stateful OO idiom where you instantiate an object (with a factory) and then use setters to change properties, instead of setting it all in a single command. I don't know if other .NET HTTP libraries change that, though...

  - It's too low level for quick stuff
  - It handles HTTP basic auth wrong
  - It throws an exception, AN EXCEPTION, for any response that isn't 2xx
  - Async is an ugly mess
  - In love with streams
  - Horribly inconsistent across framework profiles
  - ServicePointManager.Expect100Continue
  - Inconsistent header handling and restrictions
  - Terrible cookie API

I think it's fine for simple scenarios (i.e. plain GET), but it breaks down in a hurry and the fact that you have to read out of the response stream is real overhead.

> I think it's fine for simple scenarios (i.e. plain GET)

So's python's urllib2, the point of `request` is that as soon as you go even slightly beyond very basic GET and POST, it breaks down quickly in a horrible mess.

It's not fine for a plain GET, it's awful.

I'm still surprised every time I use it that they've not released a modern library for it in .Net.

The async handling was the deal breaker for me. I gritted my teeth through the other stuff but finally gave up at that point.

timelines suggest that requests was inspired by https://github.com/mikeal/request from the land of node

I can assure you that it wasn't :)

Just about every library & language has a Request-like wrapper. It's nothing unique to Node, and certainly nothing unique to client-side JavaScript where native XHR is a nightmare.

Reply on news.ycombinator.com