There are a number of hobby projects that aim to do exactly that (provide the instant-on, boot to BASIC experience of old 8 bit micros) without just being a recreation of an old system.
One that looks particularly interesting is the Color Maximite 2, which meets your specs except for the HDMI out (it uses VGA):