Ultralights are governed by Part 103 and limited to: single-seat, day-only, VFR-only, weighing less than 254 pounds, carrying 5 US gallons or less, and having a top speed of not more than 55 knots [calibrated].
* - Technically "certificate", but practically the same.
The bigger issue is that drone pilots have no skin in the game. If they cause an mid air collision resulting in fatalities: opsie daisy!
for everyone else the result is usually a lot of sobbing.
that doesn't mean they're not blase about taking risks ahead of time. but that's more because they misevaluate the risk, and can't clearly imagine the negative outcomes.
"Is used or intended to be used for manned operation in the air by a single occupant;"
Since it sounds like you're doing so with the specific intention to use it as an unmanned craft, the Part 103 rules would not apply (and would not permit it). (The law is “is” rather than “was originally built to be”.)