That aside, summarization is a relatively old field of study (in CS) and there is tons of good information to read and even many free libraries. My suggestion would be to try out an unsupervised learning algorithm such as LDA. You still need training data, but you don't need to label it with categories. The downside is that you will have no control over what it learns. Still, classic examples of LDA involve classifying news sources.
For standard linear classification, understanding Bayes is important, but for actual implementation look at something like liblinear and use logistic regression with regularization. The difference between Bayes and LR is that Bayes optimizes learning the underlying probability distribution while LR optimizes "getting the classification right", otherwise known as optimizing the expectation. Regularization controls for overfitting and there can be a big difference between the type you use (L1 vs L2) and the settings. Dont' make the mistake of treating it as a minor tweak.