The conflict here is very weak, and most judges would not likely recuse in this case or feel shut pressure to do so. The husband worked for a 1000+ attorney firm that once had a client, and had no involvement in anything. The judge has no knowledge or involvement.
If every judge in a similar situation recused, you would likely have to move/transfer lots of cases around the country to try to find judges that could hear them.
I actually suspect the judge just may not want to deal with this high profile of a case. It can be really a mess to deal with (even before you get to the personal toll of increased death threats, etc)