Unfortunately RL Glue is not nearly as good as it looks or sounds. I was thinking about using it to solve some problems I have (for example, I was interested in modeling embryo selection for complex traits as a POMPD to see what the optimal approaches might be rather than settling for a simple calculation as I did in
http://www.gwern.net/Embryo%20selection ), but the project is abandoned, and while it implements an interface/IPC glue, there are few implementations of any interesting environments or useful agents, so you would wind up writing everything yourself anyway. Once I got the simple tabular agent example working, I went looking at the more complex ones and was quite disappointed.
From the description in OP, it already far exceeds RL Glue's usefulness.