The git-dit project (https://github.com/neithernut/git-dit) provides a distributed issue tracking functionality inside git.
It does this without cluttering the repository with unneeded files, gives the possibility for having tree-like conversation (including merging conversations), referencing issues, linking to commits and so on. It is even possible to host the issues in another repository as the code (if that is wanted) or having one repository of issues for several projects / moving issues from one repository to another.
There is only a CLI as of today. I hear that there is some effort on building a (view-only) web frontend for it, though I don't know much about its progress. Maybe asking the maintainer would be an idea.
What's also missing is a way to give users of the tool access to a repository where they can submit issues (which then could also be used by a web/gui frontend for the tool). This is not the domain of git-dit itself, but a solution needs to be found. One idea would be a publish repo (where everyone can push) which automatically does some sanity-verification on the issues and forwards them to the maintainers repository... or something like that. Also integration/mappers to/from other services (gitlab, github) are missing and so is mail->git-dit integration (posting issues from a mailinglist automagically into the issues repository).
Also, https://github.com/vitiral/artifact/ is a really nice tool to do planning of an application or library inside a git repository. I am currently starting using it in imag (https://imag-pim.org) and it is really wonderful. The author currently does a reimplementation of its core functionality to make it even more powerful.