I think the delta between "contractor" and "part-time or temporary employee" is a bit steeper than you are claiming.
The tax issues are not minor, to begin with. If I have no property and no employees in a given State, I likely do not pay all sorts of taxes to that state which I otherwise might have to. I likely do not even file an income tax return with that State. As soon as I have an employee living in a State, then I definitely have to file taxes there. I also have to register with their labor department, there are probably state-specific labor laws which I could run afoul of, maybe state-specific postings and notices, etc. It's not simply a matter of the payroll company making the correct deductions.
I have to follow a different hiring practice to hire an employee than a contractor. There are likely to be more requirements on how I must make the job posting, and how I can conduct the interview process. There are additional government forms that must be filled out during the hiring process, and after the candidate is hired. The legal rights of a temporary employee are different from the legal rights of a contractor in important ways that I might not fully understand.
Once an employee is hired, I either have to pay them hourly, or a guaranteed monthly salary. I cannot offer to pay the employee a fixed amount for a certain deliverable. I have to pay the employee on a set time schedule, and not based on deliverables. I cannot withhold payment based on deliverables not being met.
I will have to update my worker's compensation policy. Depending on the size of my company and the state, I may have to enroll this person (and their family) to provide health benefits, life insurance, disability, etc. Also, certain states impose family and medical leave requirements on employees which would not apply to contractors. Federal discrimination regulations only apply to employees. I'm sure the list goes on and on...
And here I was, just wanting to get some help mocking up this new UI screen, or porting some code from one language to another, or making a new logo for my site, or updating the CSS to work with latest device X, or...
Over the last decade I've spent mid-6-figures on sites from Rent-a-Coder to Upwork and it's been a game changer for being able to quickly tackle big problems which arise due to a new R&D effort, or closing a new customer contract which our current feature set doesn't quite support, or just getting logos and wireframes because I'm not a designer and don't have nearly enough work to justify hiring one.
In short, contractors are super fucking important, and the "ABC" test seems to fundamentally mischaracterize a lot of tried and true contract work. Someone that I am hiring to perform a specific task, who is working remotely, who I will not be on-boarding, who will not be working closely with many employees, and who is paid upon completion of that task and then goes away, should be a contractor, even if they are doing work that is central to my business, and even if I closely monitor or specify how the work is to be completed.