It existed before widespread association with Apu.
Fren, in a certain context, is a 4channer shibboleth. But you are overstating it a lot here.
If someone joins your community and starts posting green frog comics and calls people fren a lot, there's a good chance they're doing the shibboleth you described. Outside of that context, I don't think it's often a racist term.