First thing to realize is that Tokyo is not really a single city but a prefecture, it is like a bunch of smaller cities (wards) stuck together, and there are 23 of them. They all more or less independent and they have their own center, and they also have some kind of theme. All that to say that there is no "downtown Tokyo", and choosing what to do in Tokyo is essentially about the districts you will choose to visit.
Which one depends on your interests. Shinjuku which is a major business center at day and a shady (but still safe!) place at night, Shibuya for fashion and youth, Roppongi for foreigners and its nightlife, Asakusa for its temple, Yoyogi's park, Akihabara "Otaku" mekka, Odaiba's futuristic vibe, Ikebukuro's shopping centers and arcades, Ginza's luxury, etc... Tokyo-Edo museum is indeed awesome if you are interesting in the history of the city, and it certainly a "do-not-miss", but on a 24-hours stay, you have to make choices.
If I had to pick one to add to any list, maybe Shibuya crossing. It is one of the most famous sights of Tokyo and a symbol of the busy city it is. It is also right out of the station on the Yamanote line, so close to other notable places. Just north is Harajuku, Yoyogi, and Shinjuku, all worthwhile.
Even 20 years ago it felt like the pasts vision of the future and the malls already felt pretty has-been.
But I agree on going to the scramble. Maybe walk from there to Harajuku and drop by Tower Records on the way.
- Uneo Park and the Tokyo National Museum. - Tokyo Metropolitan Edo-Tokyo Museum (and I've heard great things about the open-air take on this museum in the suburbs ... which could be a better option since the former I just saw is being renovated).
And on the other side of the country, don't miss Kyoto. Just came back from there, and I'm not sure I've ever fallen in love with a city faster.
But above all, I adore just walking around any city, aimlessly, and exploring whatever I stumble upon. Turn into a temple, walk through a random neighborhood grocery store, delight in the ordinary-for-here that's not-so-ordinary for elsewhere.
Speaking of Shinjuku and videogames, if you've ever played any yakuza/like a dragon game, you owe it to yourself to go to Kabukicho and its big red gate.
In any case, whatever you choose to visit in Tokyo, it will be really nice, and a lot of it will still be waiting when you eventually come back.
Cheers!
Meiji jingu + walking around harajuku/omotesando gives you a great contrast and is very doable in half a day. Not original, but very "tokyo vibe".
Walking around Jimbocho, with lots of old books shops around
If you can pass the evening, I would recommend eating out in ebisu yokocho