A factory bean is a class that creates factories. You'll almost never need to create your own as you normally use the ones Spring provide for you. And with annotations, you'll likely never even notice them.
A proxy factory bean is used for Aspect Oriented Programming in the Spring framework. AOP is a fantastic way of doing OOP. Try reading up on it before you knock it. Spring uses AOP via proxies. By using proxies, you don't need to inject bytecode via CGLIB, which while a totally valid way of doing AOP, adds another dependency to your project which you may not want or even need.
Singletons are used by default by Spring, which is an IoC (Inversion Of Control) container. An IoC container is extremely useful as it fully decouples classes from one another - the container handles the coupling and you merely set things such as object creation up via annotations or configuration. One of the considerations needed to be taken into account when using Spring for object creation is to decide whether each object is created once and once only by the factory bean, or if multiple instances should be created. You usually use a singleton bean in Spring - otherwise you have to introduce tighter bean coupling with other beans, which kind of defeats the purpose of an IoC container.
A bean is a JavaBean, which is a very simple but powerful convention used to allow reusable components in Java.
So what does this do? It's a convenience class used to extend the Spring AOP framework should you need to do so without having to make changes within the Spring framework itself. Almost no application programmers will need to use it. However, if you have a very particular requirement for AOP proxy bean creation that is not currently handled by the Spring framework (unlikely) then you will indeed find it useful, and I dare say it is, indeed, convenient.
In short, ignorance is OK. But mocking something in ignorance is pretty foolish, and your foolishness is on full display here.
P.S. incidentally, Spring is now well past v2.5 and in fact the current version is 3.1.x. It helps to read the most up to date documentation when trying to understand something in a framework. The description of the class in the latest documentation now reads:
Convenient superclass for FactoryBean types that produce singleton-scoped
proxy objects