If those million items need to be read from different servers -- as they might well, in a distributed database -- it's definitely not just a few ms of computing power.
For reference, reading a million items of up to 1 kB each costs $0.125 with on-demand dynamodb.