I think the general rule of thumb (which I don't think the op adheres to): if your code calls it, it's a library, if it calls your code, (as in you are fulfilling some interface) it's a framework.
Sticking with the example of express.js, for me it feels more like a library since I'm still in control of the program flow, even though I need to adhere to certain interfaces.
I would call react or backbone.js or angular etc frameworks, because you would have to completely restructure your codebase in most cases - or, at least, create compatible clones of those libraries.