With that being said, I believe it might be quite a learning curve for you. Not because it will be difficult, but just because there might be many topics at once to learn. Like you said, GraphQL, Postgres, Elixir (and a little bit about the BEAM ecosystem), Phoenix and CouchDB. If you are doing anything for the Web, you will inevitably touch JavaScript as well.
Not sure if you are open for suggestions, but one thing you could consider would be using JavaScript or TypeScript with NodeJS. I'm not the biggest fan, but you would be able to use the same programming language in the frontend for our Web app and also the backend with NodeJS and a full-stack framework like Remix[1] or NextJS[2]
This could reduce the overhead to start your side-project and will give you fewer puzzle parts to learn and put together.
[1] Remix: https://remix.run/ [2] NextJS: https://nextjs.org/