I thought you said you weren't using existing libraries? Or did you just mean for the server? You made it sound as if you thought writing a server for a custom binary protocol was harder than writing a HTTP server from scratch.

If you're aiming for an existing ecosystem, then sure, there's no reason not to use HTTP, assuming you make use of established libraries. But widespread use is HTTP's only real virtue; the protocol is considerably more difficult to implement correctly than it should be.