Containers aren't great security boundaries. To get the safety you'd really need, you should absolutely use a VM.