If something is notoriously difficult, it isn't up to what one individual thinks. That's what makes it notorious.