You could at least read the examples on the git-annex page[1] before passing judgement that the use cases are at all different (they're not). Instead of using a new 'lfs' command that ties you to GitHub, you use an 'annex' command (along with a few others).
Git-annex does just fine "keeping track of larger objects inside your git project efficiently", and is no more divorced from your normal project workflow than GitHub's lfs.
I'm sure this will be much easier to use for the end user like other github products and will "just work" out of the box.
The documentation lays out the workflow: https://help.github.com/articles/configuring-large-file-stor...
As does the website: https://git-lfs.github.com/ (see: "Getting Started")
git annex add large_file
git commit
And with mixed (haven't experimented with that yet), I'm pretty sure you could:
git annex add large_file
git add small_file
git commit