One example: In my parser I use tag embeddings. There are roughly two possibilities to do this (since Caffe does not have a lookup table a la Torch yet, though there is a pull request for this):

1. You train the embeddings on a large corpus and give them as part of the input.

2. You train the embeddings while training the classifier. You basically encode each tag (at various positions in the parser configuration) using a one-hot vector and direct this input to a linear layer (per position) that has the number of neurons corresponding to the requested embedding size, sharing weights between all such tag layers. After training, the n-th float of the embedding for the m-th tag is the m-th weight of the n-th neuron.

If you want the best of both worlds (train tag embeddings on a large corpus, finetune while training the classifier), you set up the topology for (2) and then you initialize the weights of the linear layer yourself by putting the learned embeddings as the initial weights.

There are also a completely different example for image recognition here:

http://nbviewer.ipython.org/github/BVLC/caffe/blob/dev/examp...

Edit for clarification: Caffe is quite different from the other libraries --- you do not set up a network programmatically, but specify the topology as data (via a protocol buffer file).