If patent workers are overworked, why don't we replace them with a certificate of origination. You file it, it takes two weeks, it's dated, and it describes the work. Then, if you need to sue someone, you then have to pay to prove that they are infringing. Reverse the burden of proof and delay the cost of approving patents.
Second, my biggest problem with software patents is that you can't build the thing from the description. If you look at a patent for a device, there are schematics and descriptions of how it works. Software patents just say "the user can click a button and it automatically buys a book". Where are the detailed drawings of how the web server connects to the payment system which connects to the distribution center? How do you scale that up to millions of users? That's where the magic is.
Patents are supposed to give the inventor a reasonable period of time to profit from his work. After that time period is over, it becomes public knowledge, so everyone can do it for free. Software patents do not give away the knowledge that it actually takes to build the system. Instead, they typically just describe the tiny bit that is apparent to the user.