The only time I've needed to build a custom footprint (not from a library) was for parts that I had to physically measure with calipers. For anything even remotely standard (transistor in standard package, integrated circuit, SMT resistor/capacitor/diode and such) the library symbols will be fine. Unless you don't like their style or whatever; you want rounder or sharper corners on pads or what have you.