They already are, across the river in Allston - much to the dismay of some the faculty, as the CS department is currently based just north of Harvard Yard. The entire school of engineering will be moving in the next few years.

Oddly enough, the current building the Harvard CS department is based in (Maxwell Dworkin) was itself funded by Ballmer as part of a previous gift with Bill Gates (Maxwell and Dworkin being their mothers' maiden names).