I can actually speak to this because I was involved with maintenance on a consultant's neural net built in raw java.

What I have to say is this: please don't build your own.