This is almost the exact approach I took for my project Neptunely (https://neptunely.com/). Working on bringing it to a VST at the moment so it's more portable.