Yes, an API that is isomorphic to the website itself would be much easier to use. Our plan is for the next version of HN's API to simply serve a JSON version of every page.
In the meantime, if you scrape the HTML, please respect https://news.ycombinator.com/robots.txt and don't do it more than once per 30 seconds. HN's app server runs on a single core, so we don't have a lot of performance to spare! This is in the works to change as well.