Here's my breakdown of what took the most time/stress (I didn't use a stopwatch, so it's heavily impacted by how it felt):
2% creating a spreadsheet that did the simple arithmetic to add up sums for investment boxes and that did the final tax calculation
90% spent trying to understand/fix one stupid mistake with an HSA account that was very confusing in the tax code. I am nearly certain that TurboTax wouldn't have helped.
5% reviewing all our other accounts, saving copies of the statements, and inserting relevant values into a spreadsheet
3% copying and reviewing my copying of other numbers, including the long stupid EIN numbers from W-2s that I miscopied and caused the IRS to automatically reject my return
... and here are my conclusions:
* The thing I personally would most love but don't believe is easily possible is a way to avoid all the mistakes copying and to simplify some of the data collection and staring at 1099s from different accounts that arrange the boxes slightly differently, but I can see why this hasn't really been automated in an open source way. While some sort of image processing or by trying to scope out APIs with the financial institutions might help, it's hard to envision an open source project that would do this.
* I actually value tax time as a time to review accounts and finances. So even if the data collection stress was reduced, I would still want to have a good feel for the numbers.
* The most serious problems I seem to run into never feel like software problems. I have had problems like the HSA issue in the past with turbotax and the "helpful" UI can only do so much to answer tough questions.
* Last year I filled out the Qualified Dividends worksheet by hand, the spreadsheet is a million times better though. It's basically the same thing as those online calculators, but I wanted to be able to see what calculations it was doing and review it and also not put my tax info online.
* Overall, what I'm most inspired to do would be to share the spreadsheet or a similarly scoped program and document/continue to refine my process for entering and collecting data. Working towards an end-to-end solution feels to me like a loser, at least to start off. Modular seems like the way to go. Solve one problem at a time. There is already an https://opentaxsolver.sourceforge.net/. Haven't tried it out yet, but I feel like it isn't what I'm looking for.