That's where a multi-format PDF would come in handy -- you could still have a human editor lay out the pages for 2 or 3 different page sizes, and then the reader would pick the appropriate one (given the constraints of the reader device and font size).
PDF spec actually supports different page sizes (crop box, trim box, bleed box, media box, etc) for different pages. But it seems like what you what is essentially two separate PDFs with different page sizes combined together, and the reader picks one based on the viewport. I'm not sure PDF spec natively supports that, but PDF does support JavaScript and I can imagine JavaScript querying the viewport and then hiding/showing pages.
A useful feature of a PDF reader on these devices could be to display only this area with a smaller, perhaps user defined, margin.
This at least would fix the 'huge margin' problem.