Yes you could inspect the headers and such to determine what type to return, but the core problem remains, that the HTML in those is not reusable whereas as JSON would be.
How much overhead this is for you depends on your design of course, but it does mean some work might potentially be wasted if you go down the full SPA application later and even if not changing layouts and such might be a pain.
if it is greenfield development, it is much better to simply return SSR HTML and avoid that complexity
you can join the discord for more immediate help:
Web2py (being phased out by py4web which I haven't tried) made that very easy. If your api is accessed as api/hello.html, it can return HTML. According to the extension, hello.json, hello.csv, hello.txt, hello.pdf, etc. can return the adequate content.
Of course you can always return JSON regardless of the extension, but I found the same endpoints serving different formats very convenient.