Why should I trust the third party library was built by programmers that knew better? What do they know that I don't?
If I ever need to build something that people say others are better at - with no argument backing - I will just ignore them (I'll do due research as in other projects though).
Most programmers have a simplistic view of time, and start out writing some library that assumes simple arithmetic can calculate any duration, then they find their library is wrong in many cases, and problems are gradually fixed as they research and learn more about time. A robust time library that gets everything right is not going to be hacked together over a weekend.
Perhaps the parent comment was too strong as it sounded a bit like "Don't try to write a time library you idiot." Writing a time library is fine, if that is your aim. I think the suggestion is rather, don't bother writing a time library that is intended to be tacked onto your main project, as you'll either get it wrong, or waste time you could be working on your actual problem, when others have already solved the time problem.
This is exactly right and it has nothing really to do with time specifically. In any area with a lot of edge cases, you'll usually figure out how to get a lot of those edge cases right after you've got a lot of them wrong, so you should prefer to use something that's already gone through this process. Similar advice here [0]:
"Back to that two page function. Yes, I know, it's just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I'll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn't have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. That LoadLibrary call is ugly but it makes the code work on old versions of Windows 95.
Each of these bugs took weeks of real-world usage before they were found. The programmer might have spent a couple of days reproducing the bug in the lab and fixing it. If it's like a lot of bugs, the fix might be one line of code, or it might even be a couple of characters, but a lot of work and time went into those two characters."
[0] http://www.joelonsoftware.com/articles/fog0000000069.html
I once went, "Oh, this time conversion function is simple. I should have it done this afternoon." About a week later I was still writing test cases that broke it.
Using a well debugged time library is worth it. Spend your cycles writing tests for it, if you must.