This is very dangerous.
With the average setup, anyone with root access on the middle box can borrow the ssh key of any user connecting through it.
See http://unixwiz.net/techtips/ssh-agent-forwarding.html#sec for more info.
In the normal case, this is done through a socket, which means that root on the machine can access it, and sign arbitrary things. A compromised machine is therefore dangerous. This is no different than getting root access on the local machine, as you can access your local ssh agent there as well.
However, with sshmux, the agent channel is not forwarded as a socket, but instead internally handled as the ssh channel it really is. This means that compromising the agent at sshmux requires arbitrary memory access, which is a considerably more difficult feat, and means that the machine is 100% compromised regardless.
I am root, I have arbitrary memory access on the jumphost. I could write a tool, like the heartbleed poc, to find the data I want.
I wrote it because, in the case that you have an evil user that scans all physical memory, you can't do anything on the computer and still be safe. ssh -W bypasses the host, and due to various cryptographic features of that setup, the only thing that could happen would be to have someone break the connection or present you with incorrect host keys, which the client would detect. I'm of course assuming that they don't have a SSH vulnerability, as that's a completely different story regardless of agent forwarding.
You can't use that machine for anything else and still be safe, and the presence of that machine on your network is a bad sign, most likely indicating that other things will be taken over as well in the same way that they got first one, or already is compromised, including some of your endpoints. While agent forwarding to the final host isn't necessary, your network is compromised, meaning that other important thing most likely got stolen, just not your private key. If your trumph card is that they don't have your private key, it's kind of like standing at a house that has been completely destroyed with all valuables in it, holding a key and saying "Well, at least they didn't get the key for the front door!". In this case, having private keys for different purposes would be a good way to isolate things, just in case. I personally separate work keys and private (as in spare time) keys.
I know I'm exaggerating the comparison a bit, but the setup required to take over agent forwarding on a network you control, means that you're screwed regardless. Agent forwarding should be used with care, ssh-add -c being absolutely necessary, but agent forwarding has its uses, and many of the cases where I'd be attacked means that I have more pressing matters to deal with.
Of course, without askpass, a properly patched evil sshmuxd could sign a separate session while still signing ths original.
As I have said many times, ssh -W ProxyCommand makes you immune to evil jump hosts, but if root on a machine in your network got compromised, as would be required, then you're going to have a bad day regardless.
This is yet another situation where Kerberos got it right years and years ago, but is almost impossible to actually /use/ even now after n years.
I've wondered from time to time why Kerberos more or less faded into obscurity. I guess we have the answer then.
Kerberos could have bad interface as well, but I don't know how that would work in this scenario. The protocol itself is secure, but just as with ssh agent forwarding, if you let everyone make authentication requests without requiring user interaction to verify it, it will have the same problems.
I'm curious, how did you get that set up?