MishraAnurag

Great post. Soft errors have a different meaning altogether in electronics. It's an interesting read.

https://en.m.wikipedia.org/wiki/Soft_error

Looks like a scam.. The app icon for Flappy Bird is nowhere to be found on that screen.

Using the relevant SQLite implementation directly cuts down that time in half to about 3.5-3.7 seconds over a few runs.

That's a fair point. NSDateFormatter is fast enough and you should never replace it for anything else unless you know for sure that it's a problem. Well that goes for any optimization. In my particular case, it was really a bottleneck, and the time difference of 100 seconds vs 7 seconds meant a user will not have to wait for 93 seconds during the initial import step. I am not suggesting we start getting rid of NSDateFormatter as it is a very valuable tool which we use for any and all date formatting ourselves, just not during massive imports anymore.

I'd suggest using timegm instead of mktime, or set the TZ environment variable to UTC to ensure all implementations return an identical date. I ran the same tests and found that the strptime was quite fast, but gmtime was taking most of the time. To speed that up, you could borrow SQLite's implementation. Checkout the computeJD function from SQLite's date.c - http://www.sqlite.org/src/doc/trunk/src/date.c

Are you using mktime to get the unix timestamp? That might be the slower part as opposed to strptime.

A link to the source code is present in the article. You can find it here - https://gist.github.com/AnuragMishra/6474321

The NSDateFormatter is already being cached. That was my first suspicion on finding this issue too. We are using one formatter per thread in the production code, but that doesn't apply for the code I've posted since everything is done on the main thread using a single formatter instance.

That's interesting. I'm not sure of the significance of that year or how that relates to the algorithm in Meeus' book. This web page talks about the date algorithms in Meeus' book in some detail but the math is beyond me - http://mysite.verizon.net/aesir_research/date/jdimp.htm. The Julian day conversion algorithm here is the same one used by SQLite.

That was to make a fair comparison between the two approaches since NSDateFormatter's dateFromString gives an NSDate, while SQLite was handing back an integer.

But you are right. In production, it makes more sense to let SQLite handle the conversion and insertion in the same statement.