The hardest part is motivation, not necessarily because the temptation to goof off is higher at home (sometimes it can be the opposite, if you are in an office where goofing off is common).
You don't get the human reactions to your work. For example you can't build some feature and just grab the nearest person to show and see their subtle reactions to it. You don't get to walk past another part of the building and see a room full of people using your software to get their work done or whatever the effect of what you are doing is.
Instead you end up sending an email at 7PM saying "I have completed item #232" and you get one back the following morning saying "ok, proceed with item #233".