Why call python from Rust? I think calling rust from python would make more sense. Use it to optimize python functions like how python libraries do with with C (i.e. numpy)
How would I go about doing that? I could put the Python code behind a little http API and call it that way, but that’s a bunch of overhead and extra stuff to maintain just to analyse some text. If I embed said Python tools in my Rust code then I can call those tools with significantly less overhead and complexity.
If you’re suggesting spawning things over shell/cmd line I’m of the opinion that this a generally bad idea.
Really, where does this fear of spawning things come from?
So the spawning of a short-lived subprocess approach has massive overhead, only suitable for multi-second workloads.
The simplest approach is via a pipe to the process' stdin/stdout I guess. Of course you have to (de)serialize your data, but you would have to to the same if you go the HTTP route, which seems far more complex. Furthermore the suggested solution probably has a nice wrapper in the language's stdlib (e.g. `check_output()` in Python land).