Because in the beginning SMS support was important and the 140 character limit worked well with the SMS 160 character limit.
With longer content, media and replies added in, it started closing itself in a bubble and becoming about Twitter being its own thing walled off from the rest of the world, a concept significantly different than Twitter started as. Maybe it also seems more monetizable, but there can be a debate on this.