How Scribd runs 150,000,000 polygon intersections a day
coding.scribd.com
coding.scribd.com
1. Oh, an interesting looking article! Hopefully it will be in HTML or PDF so I can read it!
2. (browser tab freezes) Oh shit, it says "scribd.com" in the URL bar.
3. Interface finally loads; the fonts are broken, scrolling doesn't work, and it's only barely readable.
4. I begin the frantic search for the "download" button. On most sites this is easy to spot and use, but on scribd it seems to move around from day to day. Sometimes it's big and green, others it's small, white, and hidden somewhere on the page.
5. Scribd demands I log in to download; it doesn't support OpenID, and I can never remember which throwaway account/password I used, so I just register again.
6. Finally, it lets me download and read the document.
----
How about this instead? Scribd should offer a "direct link" to the PDF, and then it would be easy for users to submit these links to external websites (HN, Reddit, etc).
First, most of them have wait timers. After opening their page, the user must wait 60-120 seconds before downloading.
Second, they're plastered with ads. And not Google text ads, but awful '90s-style popups / popunders / flashing seizure GIFs. Ad blockers are useless, because the sites are written to prevent the download link from working when ads aren't visible.
Third, most (all?) of them don't handle Unicode correctly. Upload a file in anything except US ASCII, and you'll be lucky if even the file extension comes across intact. For example, Преве́д.zip could come out "Ð�Ñ�евеÌ�д.zip" or "Преве́" or nearly anything else.
Lastly, sites like Rapidshare are known for imposing download limits, such as "2 files per hour" or "max 50 Kb/s". This is hugely annoying when trying to download a half-dozen 30MB files.
- first - wait timers: Megaupload has a timer which takes ~1min. to count down - that's quicker for me than dealing with scribd's registration.
- second - ads: I haven't noticed, I use adblock and never seen an ad on Megaupload. Not sure what's the current status of Scribd, but disabling scripts brought out a lot of spam words years ago (maybe they don't use it anymore), so annoyance is pretty much the same (or isn't there, depending on your view)
- third - unicode: We're talking about pdf-s. They have their own tags, for example for the title which makes the name pretty much irrelevant. Unless you're downloading a book with "Author - Title.pdf" name, you're most likely going to run into "some_serial_number", "ModelNumberDescription_code" for manuals, "thesis.pdf" or some other internal convention. Actually zipping the file allows you to see the original one.
- fourth - limits: Megaupload has none that I know of and I get >200kB/s most of the times.
So it might be very subjective, but even considering all the crap download pages give us, I'd take a Megaupload-ed .pdf over Scribd any time.
+ After writing this, I got back to scribd and tried to download something - it's MUCH better than it was before. I only had to scroll through 2 pages and find a small link at the bottom and it opened a new window. Right now, for me it's only as bad as MU.
Come to think of it HTML5/CSS3 + an open archive/compression format would suit this purpose nicely.
Almost any browser other than lynx has a buitin pdf reader, and pdf can be created by just about anything.
Finally pdf is a format that allow you to ensure that no matter where it is printed, it is exactly as you wanted it to be.
Edit: actually, there is an exception, and you mentioned it: printing a pdf, once you have it open, is almost always a good experience. Too bad it's the thing I want to do least often.
1) GPU development requires specific developer experience. Making performant GPU code is an even nastier and less intuitive problem than making performant CPU code. A naive implementation would be shockingly inefficient.
2) Leasing GPU hardware in a datacenter is very rare. You'd have to do it at the office, or build your own servers and install them yourself at a colo. Lots of time and effort.
Even if the GPU solution was 10x faster (it could be much more, but it depends on how much the CPU, disk, network is a bottleneck), if you're talking about reducing $50k in computer time to $5k in computer time, it's almost certainly not worth it. If you're talking about $2m to $200k, that's a completely different matter.
This could be very useful for running certain algorithms in web apps...
Below I've linked github urls and two blog posts that were done about them. Unfortunately right now webGL is kind of a moving target and I doubt these examples still work out of the box. However, if you are interested in GPGPU on webGL I would encourage you to get involved as I think there are some tweaks to the standard that would make GPGPU life easier.
Matrix Mutliply http://learningwebgl.com/blog/?p=1828 http://github.com/bunions1/matrixMultiplyGpu
Falling Sand http://learningwebgl.com/blog/?p=1471 http://github.com/bunions1/fallingsand-webgl
The important bit is at the end where they say "10% of the time is spent on polygon intersection" and the rest on image processing. We do GPU image processing on the cloud and this is (likely) where Scribd could likely obtain a significant speedup - many of our image processing operations run hundreds of times faster than on the CPU.
It's a complete apples to oranges comparison. Scribd is performing quite complex logic with an emphasis on correctness not speed. I feel like you're dismissing an interesting article for the sake of an amusing one liner, and that's pretty much the antithesis of HN.
It is a nicely-written and illustrated article, but it describes a problem that hasn't been considered 'interesting' for decades and can, in any case, be tackled with a caching scheme. Why would they need to run the same intersection logic over and over, when there's only a finite number of glyphs to render in any given document?
Actually, polygon operations on grids is still an active research area, with the latest papers on the topic less than two weeks old: http://www.sci.utah.edu/socg2010agenda.html
> Why would they need to run the same intersection logic over and over [...]?
We usually don't. It basically depends on the context in which the glyphs appear on the page. For a standard, say, LaTeX document consisting of mostly text and without weird graphic operations taking place around or on top of the text, a given glyph is just processed once.