Linus just says that he doesn't want to "act professional".
Remember that he may still feel that it is a hobby for him to work on Linux.
He started it in his spare time and, even if he is technically payed to work on it by the Linux Foundation, he has the right to do whatever he wants with this project, including still considering it is a hobby.
Would you like someone to tell you how you should speek with your friends outside work?
By the way, people sending him patches are not his coworkers, except perhaps Greg KH. He can see them as friends or strangers or even as business people wanting to sell him stuff.
You cannot compare his situation with the situation of people who "work in reasonable office environments".