co-founder of fileee.com here(we do that for a living :) ). I can tell you that in general adaptivethreshold is better, because it is less error prone against low contrast situations and missing edges. That said there are also cases where canny performs better. That's why we actually decided to use a machine learning approach to decide when to use what. There are even more things one can do to improve the detection(e.g. hough tranform to find edges or use variance or fft to assest whether possible "document" candidates are just garbage rectangles or real documents.)

canny and adaptive threshold can go hand in hand. i.e you can threshold an image and then apply canny. Adaptive threshold turns the image into binary (either white or black) and it works great when there are big shadows.

e.g

http://www.aforgenet.com/framework/docs/html/img/imaging/sam...

->

http://www.aforgenet.com/framework/docs/html/img/imaging/bra...