I think it's mainly the change in behavior that could be viewed as concerning. As a user of a website, if the order of websites you visited after browser startup impacts the success or invalid certificate error I feel like it's a pretty confusing experience.

I definitely didn't know this before and if I saw this behavior I would be pretty confused.