Finding thousands of exposed Ollama instances using Shodan
blogs.cisco.com
blogs.cisco.com
For my own purposes I either restrict ollama's ports in the firewall, or I put some proxy in front of it that blocks access of some header with some predefined api key is not present. Kind of clunky, but it works.
Fortunately it’s an easy fix. Just front it with nginx or caddy and expect a bearer token (that would be your api key)
Even if those services had some access protection, I simply must assume that the service has some security leak that allows unauthorized access and the first line of defense against that is not having it on the public internet.
I don't think it's sensible to expect every project like Ollama to ship their own half-broken authentication and especially anything resembling a "zero trust" implementation. You can easily front Ollama with a reverse proxy which does those things if you'd like. Each component should do one thing well.
I trust Nginx to verify client certificates correctly so I can be confident that only traffic from trusted users is able to reach whatever insecure POS is hiding behind it.
Or the worms that scan for vulnerable services and install persistent threats.
If you want to remove the password on a service, that’s your choice. The default should have a password though and then people can decide.
Defense in depth is essential in an age of unreliable software supply chain.
The default install only binds to loopback, so I am sure it is pretty common to just slap OLLAMA_HOST=0.0.0.0 and move on to other things. I know I did at first, but my host isn't publicly routable and I went back the same night and added IPAddressDeny/Allow rules (among other standard/easy hardening).
Problem is people don’t know that it’s a good solution.
Definitely not credible to speak about ML stuff and of course - Ollama has never been production-ready in the sense iOS (Cisco’s) was.
Must be a good time to be in security space with this sort of stuff plus the inevitable vibe code security carnage
That's not security by obscurity.
If the "uuid looking" part is generated using a csprng and has enough entropy, it has the same security properties as any other secret.
There's other issues with having the secret in the URL.
I know it's commonplace, but is this unauthorized access in terms of the CMA (UK) or CFAA (USA)?
Is this thanks to everyone thinking they can code now and not understanding what they’re doing.
Make it make sense
What's likely happening here is that people are renting VMs and one-line some docker-compose up thing from a tutorial. And because it's a tutorial and people can't be bothered to tunnel their own traffic, most likely those tutorials are binding on 0.0.0.0.
Plenty of ways to footgun yourself with c/p something from a tutorial, even if you somewhat know what you're doing. No need to bring "everyone thinking they can code" into this. This is a tale as old as the Internet.
Another thing is that docker, being the helpful little thing that it is, in its default config will alter your firewall and open up ports even if you have a rule to drop everything you're not specifically using. So, yeah. That's probably what's happening.
Servers that shouldn't be made public are made public, a cyber tale as old as time.
The new problem is if the LLMs are connected to tooling.
There's been plenty of examples showing that with subtle changes to the prompt you can jailbreak the LLM to execute tooling in wildly different ways from what was intended.
They're trying to paper over this by having the LLM call regular code just so they can sure all steps of the workflow are actually executed reliably every time.
Even the same prompt can give different results depending on the temperate used. How security teams are able to sign these things off is beyond me.
Seriously, this is extremely mild as far as issues go. There's basically no incentive to fix this problem, because I bet even the people who lost a few pennies of electricity would still prefer the convenience of ollama not having auth.
Plus, that's the worst case scenario, in real life even if some black hat found an exposed ollama service, they have no interest in generating tokens for <insert random LLM here at 4 bit quant> at a slow speed of <50tok/sec.
Don't expose services to the public internet unless they have been battle hardened to be exposed to the public internet, e.g. Nginx as an authenticating reverse proxy.
Even this one would be remedied by not running ollama as root and not have its binaries owned by the user it is running as (though overwriting executables/libraries that are being mmapped as executables is usually not possible), which I hope would be the standard mode of its setup.
Either way my point is that software contains vulnerabilities, especially software that hasn't been hardened to be exposed to the public internet. Exposing it to the public internet anyway is a display of bad judgement, doubly so when the person responsible seems to believe that the worst thing that can happen is someone using the software as intended. Details of specific vulnerabilities are really beside the point here.
Assuming that the happy path is the worst that can happen is simply naive, there's no two ways about it.
But sure, it's always possible to be more innovative about how to go about enabling RCEs, like the log4j case demonstrates..
If you deploy a power plug outside your house, is it the fault of the power plug designer if people steal your power?
Put it behind a webserver with basic auth or whatever you fancy, done.
Yes, this goes right along with the tried and true Unix philosophy: do everything, poorly. Wait what?
This is a program that very different people want or need to try out that just so happens to involve a client-server architecture.
So at least 1,100.