Maybe you are just capped by bad wifi? I had the same issue and switching around wifi protocols did a 10x on my speed. Like from 20-30mbs to 300-350mbs.
I did upgrade my router in the fall and was surprised how big of a difference it made for my speeds around the house, but we are still nowhere near saturating the bandwidth we have available with 50 / 50.
Granted, we are a family of 5 with 3 preschoolers that don't do anything besides watch shows a couple hours every evening.