Websockets and HTTP/2 have not much to do with each other. Both are established on top of a TCP connection (which might have been used to speak HTTP/1.1 before, until an upgrade occurred). Websockets allow sending unsolicited websocket frames in both directions, HTTP/2 multiplexes several HTTP streams on top of a single TCP stream.
There is no websockets over HTTP/2 specification- at least the last time I had looked. It wouldn't really yield much benefit, one would just have the 2 layers of framing (web socket framing on top of HTTP/2 framing). With the current approach one would typically just have 2 TCP connections established from a browser to servers, if websockets are needed: One for websockets, and one for HTTP[/2].
Regarding SSE: Use it when single directional communication is good enough, when HTTP based authentication is required, for integration with other HTTP infrastructure (load balancing, proxies, etc), or when no websocket library is available. Websockets might be mostly preferable when stateful "realtime" communication in both directions is required.
On the server side most things are handled in the low-level HTTP handlers. There (e.g. in Netty/Jetty and Co) one needs to care about ALPN negotiation, websocket upgrades and Co. But higher up (J2EE, express, etc) things are mostly about HTTP semantics, and there isn't any handling about "upgrade" or even the "Connection" field in general anymore.
Yes, the line between HTTP application semantics and HTTP/1.1 transport is very thing and nobody exactly knows where things start and end. But I'm still not convinced we need web sockets over HTTP/2, e.g. in order to allow upgrades in the application layer.