To make the choice of L1 vs L2 a little more flexible there's also Elastic net regularization [0] which lets you combine both L1 and L2
Also agree with LDA, I've recently been using it for some unsupervised text classification and the topics it generates are impressively sane.