Oh oof. I thought removing that requirement would be the whole point of something named "FFmpeg-over-IP". Shared filesystem usually involves full trust between machines, bad network error handling, and setting things up by hand (different config on every distro)
No idea what problem this is trying to solve. Just seems like the user wasn’t familiar enough with how to use a NAS.
There’s a pretty well understood concept of what a NAS is; this isn’t a complicated philosophical problem.
A very common workflow in motion picture production is to use NAS for storage on a fast network, mount the SMB share, have a script/tool/app that monitors the ingest directory and writes to an output dir.
FWIW the key differentiator between a NAS and other types of network storage is the protocols they use.
If files is the main primitive, it’s a NAS; it’s its blocks it’s considered a SAN.
Sometimes SANs have NAS “heads” for clients that want file access or a block level storage device.
It would be helpful to understand your specific use cases in more detail, particularly for batch transcoding (my understanding is that this solution is not about stream transcoding).
If you're talking about how today's open source NAS software has a button for enabling NFS/SMB on a directory:
1. I built my NAS long before software like that was common. Some of my custom stuff (Eg tiered storage, storing the first few seconds of every video on flash, etc) would be a pain to migrate.
2. Some of my Windows machines are untrusted. Unlike the NAS, they have internet access. I can't give them read access to the entire NAS, but I still want to use their GPUs and CPUs to run ffmpeg on arbitrary files on the NAS.
3. I could spend a day writing more code to move files in and out of a shared directory every time I need to run ffmpeg on them. But I was hoping "FFmpeg-over-IP" would let me run a remote ffmpeg on a local file. Like calling an RPC
This has been available in Synology and QNAP devices for that long…
I used to own the tape library that is the Disney Vault. A common pattern for transcoding is to have a watched folder. Drop files you need transcoded in, get files you want in a diff directory when finished.
Neither the OP nor I are talking about offline transcoding.
The same server could also offer a download link for the transcoded video, and also receive URL parameters for the transcoding options. Or the transcoded video itself is returned after the subprocess finishes.
Something along the lines of:
Server
import asyncio
import subprocess
from aiohttp import web
async def transcode_video(request):
cmd = [
"ffmpeg",
"-i", "pipe:0",
"-f", "mp4",
"-preset", "fast",
"output_file.mp4"
]
process = await asyncio.create_subprocess_exec(
*cmd,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
async for chunk in request.content.iter_any():
process.stdin.write(chunk)
await process.stdin.drain()
await process.stdin.close()
await process.wait()
return web.Response(text="Video transcoded successfully!")
app = web.Application()
app.router.add_post('/upload', transcode_video)
if __name__ == '__main__':
web.run_app(app, port=8080)
curl -X POST http://<server_ip>:8080/upload \
--header "Content-Type: application/octet-stream" \
--data-binary @video.mp4
As for why the variable number of outputs exist in the first place, it’s because when you’re a media server, you dont want to wait until the full file is converted before serving it to the user. You want to start playing immediately so the output is chunked into multiple files
User space limits potential security impacts, and a restricted VFS could be used to prevent clients from accessing anything that they shouldn't.
(Although I'm not even pretending to know whether or not this is a remotely good idea - my guess is that it isn't, but I'd like to know just how bad an idea it actually is.)
There are ways to intercept writes without root and send them to another machine. Eg you could use LD_PRELOAD. But that's exactly the kind of pain in the ass that I was hoping a project named FFmpeg-over-IP would automate.