The main, major issue being plant based full time is eating out with others: if you are plant based every single meal there will be comments and discussions about how you eat, why you eat something vs something else, how your interlocutor could be eating better but they don't have time/money/... to do so and on and on.
Some former plant-based people will tell you that one of the best side-effects of not being plant based anymore is that during means you can talk about "something else".
This is magnified if you have to regularly eat out for work / with clients, where you can easily be put in the situation that you have to decide if you want to be plant based full time or plant based in private only.
To me this is the main difficulty with transitioning to a full time plant based diet: unless your circle of friends is ok with that (even better if they are plant based themselves) and unless your work situation is compatible with that and unless your coworkers are ok with that, it is very likely that in the long run you will fail due to having to decide between eating how you want, and having an easy time in your social/work life.
In my opinion this is why health-based plant based folks stick with the diet less, because unless you truly have a deep belief in what you are doing, it is really easy to say "it's not worth it" and it's a lot more likely you can have a strong belief in the ethics of being plant based than in an abstract concept of healthy eating