This key combo doesn't work
To manage the top sites (which Firefox 78 displays on address bar click or down arrow if enabled in the preferences [1]), you need to: 1) add the top sites on the home page, 2) hover over the page you want to remove, 3) click the "..." button in the top right ("Open Menu"), 4) click "unpin" if you don't want it to be kept there, but stil calculated in the ranking, 5) click "dismiss" if you don't want it displayed.
[1] This is a change in Firefox 78, as when pressing the down arrow key Firefox 77 displayed what Firefox 78 displays when pressing the space bar.
I've given that random person's blog thousands of hits over the last year...
I don't know of an easier way to do this.