I looked into a similar ffmpeg solution a couple of months ago and broadcast an libx264 video only stream with ~1 second latency to an rtsp server: https://gist.github.com/andrewmackrodt/88c2233fb9cc4797ada93...
However, I found this solution to be too CPU intensive as I was unsure if/how to instruct x11grab to interact with the GPU directly. Perhaps if I used my nvidia card for encoding it would have been more performant. However, OBS makes this entire process much easier.