Filesize has nothing to do with speed most of the time ... Try the PDF Reference document [1], it's pretty fast.
The project is very active: in the three month that we investigated it, many of the pdfs that were unusable (e.g. NASAs budget report, one of the worst written PDFs I have seen so far) when we started are now fast enough. Also almost all the effort that Opera puts into the project is about performance.
I suggest you check the file again in PDF.js and report the PDF so somebody can look at it.
[1] http://www.adobe.com/content/dam/Adobe/en/devnet/acrobat/pdf...