Best Practices for Deploying MinIO in Virtualized Environments | Hacker News Reader

Best Practices for Deploying MinIO in Virtualized Environments

blog.min.io

54 pointsedogrider17 comments

> Avoid “noisy neighbor” problems. In order to get the most from MinIO, it should be the only workload running on the hypervisor, the same as you would do for any high performance system in a virtualized environment. This will prevent loss of performance caused by resources being consumed by workloads as they run on other VM’s.

I understand their logic behind this but if MinIO is best used as the only VM on an entire physical system why bother with running it on a hypervisor? I guess "single pane of glass management" but why bother with the license cost of VMware, RHEL, etc.?

Maybe something like https://firecracker-microvm.github.io/ ?

You still get hardware redundancy by clustering the hypervisors together, so you could start your VM workload up on a healthy node.

You can backup the entire vm and shift it to new hardware when you have a hardware failure. If you are already doing a lot of VMs, you continue using the same monitoring and deployment for everything.

If I listened to all of my vendors when they say this I'd have 12 physical servers rather than 2. This is nonsense as long as you pay attention to your loads.

I am curious why people use Minio over S3/B2/R2/GCS. It seems like the hard part is not really the api of a key value store like this, but the reliability. S3 is so cheap (and I assume its competitors are even cheaper), it seems like giving up that reliability and running your own infra would be a really tough sell.

(For testing, I totally understand having something that matches the s3 api but is lighter weight)

At my company, we've considered switching to MinIO from S3 because it would allow us to host data on prem and potentially have higher upload speeds than S3 (although we haven't tested that). To your original point though, unless you have a compelling reason to not want to upload your data to servers you don't control, I agree that your best bet is just to use a cloud storage solution.

Good points, this makes a lot of sense. I have been doing cloud stuff so long I forgot many companies are leery of it (some justifiably, some not, but as a developer you have to work with the constraints)

I know this is a smaller and uncommon use case but in a previous job I used minio so that we could have a single storage implementation for an application that was most commonly deployed to offline scenarios but when deployed to online scenarios it could seamlessly use S3 by just updating the environment values.

Ah, ok I didn't realize you could use it as a wrapper around S3. That makes a lot of sense

It doesn't wrap Amazon's S3 it just provides an S3 compatible API. Our use case was to have application code that only interacted with file storage through a single S3-api compatible filesystem adapter. Then we'd either spin up minio in a docker container in those offline deployment scenarios and feed that container's info into the applications environment variables, or when deployed to the internet we'd feed it real AWS S3 connection info. The application code then never needed to know or care where the files lived which meant that code didn't need to have conditionals to use regular filesystem libraries in one code path and S3 adapters in a different path. This also naturally prevented some of our junior devs from trying to do things that instinctually made sense when working directly with a filesystem but is impossible to do on S3 (like trying to unzip a file on S3, or symlink or think that you have cheap and easy move operations).

MinIO can be backed by S3. I think the largest compelling point more for all in one solutions that want to support on-premise data storage in addition various cloud storages. I think I've only interacted with it directly through thanos/prometheus though...

> S3 is so cheap

Only if you're not accessing the data (outside aws)? S3 storage is cheap, but outgoing bandwidth is expensive?

Yeah, I don't understand that either, people pushing S3.. is everyone on AWS here? If you are using VPS Hosters or Bare Metal machines and connect that to S3, it's not just expensive, but also slow. While your own minio cluster is cheaper and faster. Within AWS I totally get that S3 is a no-brainer.

Because it adds a compatibility layer between your application and cloud services (plus local) by providing a common S3 API.

e.g. https://docs.min.io/docs/minio-gateway-for-azure.html

Core minio is such a solid, well thought-out product.

The Kubernetes operator, a little less so, and the helm chart, less again. I think they rightly focus on the core and rely on the community for some of the tangentials.

We might be hacking on that chart a little to contribute.

Agreed, the k8 operator is terrible. I deployed it at version like .4 and there is 0 upgrade path to the current version. I'm stuck with basically doing a completely new deployment and migrating all of the data from the buckets, ACLs, users, etc.

Reply on news.ycombinator.com