For these kind of tasks I always used an app written by myself using an embedded browser control (webkit engine or IE ActiveX control on Windows). So I load the page and just call the control methods (usually just to convert the output to plain text).
How are this tool (and similar tools) more efficient then the highly optimized browser engines? I am missing something here?