Hacker News Reader

Comment by tlrobinson | Hacker News Reader

“requests” doesn’t support (or apparently want to support) Promises which are now widely used in the JavaScript community, thus it’s obsolete. Promises work best when all the async operations in your code use them, especially with async/await.

Interestingly early versions of Node had a limited form of Promises but they were quickly removed.

View on HN

Just to give a little bit more context, in contemporary javascscript you can write something like this (wrapped in a try-catch:

    async function http_request_new(url) { return new Promise(...) }
    const http_response = await http_request_new("https://www.google.com")
    ...

The await signals that the asynchronous http_request function is non-blocking, and we should continue execution once the non-blocking return value (promise) has resolved. Previously idiomatic javascript would have looked like this:

    function http_request_old(url, cb) { return ... }
    http_request_old("https://www.google.com", function(error, http_response) { ... })

Luckily node.js provides util.promisify that converts any old callback-style async function into one that returns a promise, so that await can be used:

    http_request_new = util.promisify(http_request_old)
    const http_response = await http_request_new("https://www.google.com")
    ...

Unless I'm mistaken, it's fairly trivial to promisify the old-style code

util.promisify won't work on request in most places because they don't use the defacto standards of callbacks (the first argument being an error argument, callback being the last argument, etc...). Funnily enough, i believe it predates the widespread use of that convention! But that means you need to either do it yourself (which admittedly is pretty trivial), or use custom bindings to "promisify" request (which do exist! and are even maintained by the original authors).

But promises are only one reason why request is more difficult to use.

Compare the node.js style streams with async iteration which could be possible if the library supported it:

node streams style:

      const readStream = fs.createReadStream(inputFilePath, { encoding: 'utf8', highWaterMark: 1024 });
      readStream.on('data', (chunk) => {
        console.log('>>> '+chunk);
      });
      readStream.on('end', () => {
        console.log('### DONE ###');
      });

async iteration streams:

    for await (const chunk of fs.createReadStream(inputFilePath, { encoding: 'utf8', highWaterMark: 1024 });) {
      console.log('>>> ' + chunk)
    }
    console.log('### DONE ###')

Not only is the latter easier to quickly grep and understand, but it also is easier to catch exceptions (a try/catch works, and it bubbles up! So it's harder to ignore errors by forgetting to add a `readStream.on('error'...)`), and it works correctly in async contexts so it doesn't also need to be wrapped by Promise constructors.

Then throw in writeable streams, transform streams, and tons more that all require more difficult setup, less standard ways of working, and are overall harder to get completely right without any bugs.

With async/await you have to wrap everything in try/catch, so it only looks better if you do not handle errors. It also makes it harder to do more advanced designs like rate limiting, loading bars, etc. I really like callback convention with error first. Something being async also means it can fail. The most important part of async code is error handling.

errors bubble in async contexts still, so this example will still catch errors correctly:

    async function foo () {
      throw new Error ('test')
    }
    
    async function bar () {
      return foo()
    }
    
    async function baz () {
      try {
        await bar()
      } catch (err) {
        console.error(err.message)
      }
    }

That's not the case in callback-based systems where it's more of a golang style "handle it right then, or it gets ignored forever" kind of thing.

But also unhandled promise rejection handlers are seeing much more widespread usage now to catch "unhandled promise rejections", which make it a bit easier to handle errors in async contexts, but i'll be the first to agree that there is still a lot that sucks about handling errors in async contexts in javascript...

> It also makes it harder to do more advanced designs like rate limiting, loading bars, etc.

It does, but you can also handle a lot of these by mixing promises and async/await (since a/a is basically a nicer syntax for Promises). And callbacks still exist for the cases that really need them (like you said, loading callbacks are still a hell of a lot easier to use than janky promise-based solutions in most cases in my opinion! Mostly because a promise is one-and-done, but callbacks can be re-called multiple times).

But the point is that if you stick to ONLY callbacks, EVERYTHING has to handle wrapping it. Whereas if you move on to using the new features where they make sense (the "easy path" can use async/await or async iterator streams), then only people who need those extra complicated features have to wrap and work with those more complicated setups.

I think there are use cases where Promise and async/await makes sense, for example transactional SQL, so you can break up the transaction into many simple queries instead of one huge complex query and without the added boilerplate from the callback pattern. But when you add Promises, they tend to spread. A middle ground is to have both Promise and callback, eg. if(!cb) return promise. Another gripe with Promise's is that "soft" errors are treated the same as catastrophic errors.

When you add asynchronous operations, whether they use promises or callbacks, they tend to spread beacause any consumer of an async operation probably needs to be async itself. That’s not specific to promises.

If you return a promise you don’t really need to support a callback because you can just call .then on the promise with a callback. i.e.

    func().then(result => {
      // success
    }, err => {
      / error
    })

Is equivalent to

    func((err, result) => {
      if (err) {
        // error
      } else {
        // success
      }
    })

IMHO promise-based APIs are strictly better than callback/“errback” style APIs for many reasons, async/await support in the language being the nail in the coffin.

While I agree that there is room for both (callbacks make sense in some areas, promises/async/await in others). I feel that promise-based async should be the default, since it provides so many additional benefits.

>A middle ground is to have both Promise and callback, eg. if(!cb) return promise.

This works in many cases, but becomes ugly in some. The 2 ways of working have subtle differences (callbacks and promises both trigger at different times, and there is some interplay with "microtasks" and queues and schedules and other things that make some situations hairy). There's also the problem of Promises being hard to use when additional closures are involved (it's why `.forEach` sees much less usage in async/await codebases, because it's a lot more cumbersome to "wait" on iterating over an entire array with `.forEach` because the inner `await` can't escape the function in `.forEach`.) So most users will end up choosing one or the other for the whole library, and at that point why not just have them be 2 different libraries?

It also makes documentation harder, and makes the implementation of the library a lot more complex (now it needs to do EVERYTHING in both a promise-friendly way, and a callback-friendly way), and some things just can't be easily split (how do you support both async iterators and node.js event streams at the same time in the same interface?)

One strategy is to be adapter friendly by sticking to conventions. By adapter friendly i mean promisify and event emitter to async iterator adapter. And vice versa. And by conventions i mean things like error first. Then it might be possible to just have a .promises property that expose the api via the adapter. So that you can support many paradigms, with minimal changes to the code base. With paradigms I mean different ways to work with async, there are probably 20 different ways that I know of if I also include ways how to run things in parallel. My personal opinion is that promises should have stayed in userland (not in ES spec) to give competing standards a chance to compete. Those wanting async/await patterns could have used co routines/generators.

> With async/await you have to wrap everything in try/catch, so it only looks better if you do not handle errors.

Better than having

    if (err) {
      callback(err);
      return;
    }

at the top of every function.

> It also makes it harder to do more advanced designs like rate limiting, loading bars, etc.

No it doesn't. Rate limiting is no harder with Promises or async/await. Loading bars can be handled with async iterators. At the worst, one only needs to fall back to callbacks in such situations, and abstract that in a Promise wrapper for use with async/await.

> I really like callback convention with error first.

It has completely different semantics from the rest of the language. You must handle an error wherever it's thrown, which is ridiculous.

> The most important part of async code is error handling.

Sure, and that handling should be done at the highest level possible.

The idea of exceptions is to catch them where you would handle them. If your code is catching to re-throw, catching to add a header and then rethrow, or catching to revert back to rcode-style error handling, then you aren't using try/catch to its most practical potential.

With this philosophy in mind I've never been overburdened by try/catch littering my code.

In the case of Request, there is already an official wrapper for the library that supports promises. https://github.com/request/promise-core

It is a couple hundred lines of code though.

> Unless I'm mistaken, it's fairly trivial to promisify the old-style code

Even if it was trivial to promisify request (though it seems it isn't according to sibling comments), the point is that the promisify utility is effectively a workaround for working with older libs & APIs in a modern idiomatic way. It's supposed to be a temporary shim to move into modern patterns, not something whereby you start a new project, choose new actively-developed lib, and expect to have to wrap that lib just to use it idiomatically.

Similarly, "upgrading" your popular lib to have a modern API just by wrapping it with the promisify util and leaving the internals using a legacy pattern not only adds overhead, but it also just a bit of a hack, and not very ideal. Especially when more modern alternatives to your lib exist which you could be recommending.

Exactly - for the longest time, the success / error callback signature was the primary and expected way to handle async code in Node modules, and there was a lot of objection to using promises - in part because at the time it was not stable, and in part because its performance was less compared to callback style.

You don't want to put performance bottlenecks in core and/or frequently used libraries just so it fits your preferred async style. If you REALLY want to pay that price, it's trivial to promisify a library like this - and it doesn't add complexity or size or (worst case) dependencies (like 3rd party promise libraries, as was the case for a long time) to your small, focused library.

i guess these are the downsides of a platform without much stdlib. consensus stdlib doesn't really work, and then things don't work together.

There is an [official] additional package "request-promise" that adds Promise support. I don't dabble in Node much, but is this a bad thing? Although on the other hand, most other libraries support promises out of the box (e.g. using the "pass no callback to get a Promise back" pattern).

Bolting on the newer features as additional packages has a few downsides in my experience:

* Increased complexity. So now in order to use request in a modern JS ecosystem, you need to install request, and request-promise-native, which itself installs request-promise-core, which all 3 then combine to use. Which do you look for to find documentation? Which do you look at when you have bugs? How does interop happen with other 3rd party libraries? Does every other lib that works with request have to handle all the official wrappers or just the main? And good god what happens when breaking changes are made to one of them? Everything is just a lot more complicated here.

* Only so much can be easily patched on. Things like streams are pretty complicated, and writing a complicated (and often slow) runtime transformation of a stream to an async iterator, both of which don't have complete 100% matches for all of their features, means that everything is now using a sub-par incomplete interface. The main application can't implement features which take advantage of the new syntax, and the wrapper can't always represent the old ways of working perfectly.

Basically, it's a leaky abstraction in most cases, which is slower, uses more memory, and is a lot more complex.

There's also axios, superagent, and a whole bunch of excellent node HTTP clients (also 'fetch' if you like encoding URLs and parsing JSON manually).

I use request-promise and it’s always worked perfectly fine. I normally do

  const response = await rp({
    request params
  });

and then continue on.

Reply on news.ycombinator.com