It'd not fully solve your issue, but have you ever seen https://github.com/py2many/py2many ?