Thanks for the encouragement! I might take another shot at it sometime. I had this server running for about 3 months with just my personal mail, sending probably 1 email per day on average. It's possible slightly higher volume would do a better job of "warming up" the IP.
I self-hosted from about 2007-2008 and 2011-2013, and had nowhere near as much trouble with deliverability. It came as a bit of a surprise how much more difficult it is these days.