hoosieree

Yeah, aggregating seems technically challenging, but doable. But I wonder if ActivityPub or some other "open social" protocol would suffice. I'm torn between "support open protocols in particular" versus "support as many things as possible".

What does it say about me that I know this xkcd by number? I propose there should be a new ISO 927 to rule them all.

Would you mind elaborating a little on where you draw the line? In my mind "SEO" means adding the correct meta tags to the html, but I suspect that's because I have approximately zero wisdom on the subject.

That's a good resource, thank you for compiling it and sharing it here. Do you happen to know if there's any that allows for one user to comment with service A, and another user to comment with service B?

Updated for 2023: if you attempt to implement this algorithm in a language or service area not approved by Dijkstra, it will return a scary copyright notice and refuse to work.

[edit] I've been told that this is only true in Poland, where reverse Polish notation is interpreted literally.

Paid streamed content should do something like mobile virtual network operators, so people can pay for what they use.

Currently only piracy provides this service, and paid content services are choosing legislation over competition.

The established players really should be working on this, because it's only a matter of time before some pirate service turns legit and eats all their lunches.

"A Generic Approach" seems like a good starting point for a classical approach: building a set of reusable components and heuristics to recognize idioms, etc.

Might also be worth considering an approach integrating LLMs for summarizing code. Maybe you could fine-tune a pretrained model that already "understands" source code to associate sources with generated code? If going this route I would still probably use a disassembler to preprocess, and maybe also extract basic blocks to use as my "target" domain for fine-tuning.

As for Tigress, I used it extensively and found it to be really great most of the time. There are some limitations to be aware of: it only works with C code, and you have to turn your multi-file projects into a single file with a main() function. Also, its C parser (CIL) has some limitations (e.g. doesn't recognize the word static in "struct foo x[static 1]") so you might need to translate your C code first. I translated manually because it was a really rare issue for the code I started with. I also had mixed results using Virtualize and JIT. Sometimes they would emit invalid code, so I ended up just throwing out that data.

In my view, the up-and-coming Tigress challenger is obfuscator-llvm. I think it is very promising for future work because it inherently supports more languages than only C. But currently obfuscator-llvm is much more limited (~3 transformations compared to ~48). So if you're using C, today I would pick Tigress.

That better be a Bowie reference and not an Oasis reference.

Short answer: not where it counts.

My work focuses on recognizing known functions in obfuscated binaries, but there are some papers you might want to check out related to deobfuscation, if not necessarily using ML for deobfuscation or decompilation.

My take is that ML can soundly defeat the "easy" and more static obfuscation types (encodings, control flow flattening, splitting functions). It's low hanging fruit, and it's what I worked on most, but adoption is slow. On the other hand, "hard" obfuscations like virtualized functions or programs which embed JIT compilers to obfuscate at runtime... as far as I know, those are still unsolved problems.

This is a good overview of the subject, but pretty old and doesn't cover "hard" obfuscations: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=1566145.

https://www.jinyier.me/papers/DATE19_Obf.pdf uses deobfuscation for RTL logic (FGPA/ASIC domain) with SAT solvers. Might be useful for a point of view from a fairly different domain.

https://advising.cs.arizona.edu/~debray/Publications/generic... uses "semantics-preserving transformations" to shed obfuscation. I think this approach is the way to go, especially when combined with dynamic/symbolic analysis to mitigate virt/jit types of transformations.

I'll mention this one as a cautionary tale: https://dl.acm.org/doi/pdf/10.1145/2886012 has some good general info but glosses over the machine learning approach. It considers Hex-rays' FLIRT to be "machine learning", but FLIRT just hashes signatures, can be spoofed (i.e. https://siliconpr0n.org/uv/issues_with_flirt_aware_malware.p...), and is useless against obfuscation.

Eventually I think SBOM tools like Black Duck[1] and SLSA[2] will incorporate ML to improve the accuracy of even figuring out what dependencies a piece of software actually has.

[1]: https://www.synopsys.com/software-integrity/software-composi...

[2]: https://slsa.dev/

Still better than YAML.

Wow, I really could have used this for my Ph.D. research (deep learning for obfuscated code).

I ditched Ghidra in my experiments in favor of angr early on because Ghidra did not play nicely with multiprocessing and I had a lot of data to process. Well maybe it does but it was much easier for me to achieve the same thing with angr.

Love the name! Although I feel compelled to point out that Compiler Explorer is the name of the project and Godbolt is its author's last name, but I suppose if people are to the point of using Godbolt as a verb the ship has sailed.

That's a good one! Might replace my current favorite which is:

Let x = 0.99...

Then 10*x = 9.99...

And if we subtract x from both sides, we get:

10x - x = 9.99... - x

And since we already defined x=0.99... when we subtract it from 9.99..., we get

9x = 9

So we can finally divide both sides by 9:

x = 1

All of it, probably. Next question!

    “how much of it fits in one page” matters. This doesn’t mean that cramming everything into tight subsequent paragraphs, like a serious book, is a good idea: code isn’t supposed to be primarily read paragraph-by-paragraph. On the other hand, a two-words-per-line, twenty-words-per-page nursery rhyme-like layout means that one might need to scroll through dozens of pages to get “what’s this all about.”

I disagree that code is "supposed to" be any particular way, but in general I agree with this point. And I think it's gradually being forgotten. Automatic code formatters and mandatory style guides are becoming more common, and while they do raise the floor (by preventing some interpretation of "bad" style) they also lower the ceiling. Uniformity isn't bad, but sometimes you really need certain things

to

stand

OUT!

Inclined planes form a column. Unguided inclined planes: knives, axes. Guided: planes, scissors. Inclined planes wrapped around cylinders: drills, screws.

Blunt objects form a column: Hammers, presses, brakes.

I agree with your categorizations involving tools that rely on heat, but I guess the problem with any such taxonomy is the next person comes along and goes "well I would do this differently, and that..."

what fresh pop-up/pop-under hell even is this?!? I specifically asked for not that

D-K effect effect is almost as entertaining as the Butterfly effect effect[1].

[1]: Which is the far-away effect attributed to having watched the movie The Butterfly Effect.

Now if we could just get the world to run on DC, then HVDC transmission lines would make distribution more efficient.

That's a good question. There are Tigress transformations [1,2] that seem highly relevant to this goal, but they're harder to work with because the resulting C code isn't always compilable without errors.

In my work I'm not looking for intentional spoofing, but the obfuscations I do use [3,4,5,6,7] end up building very similar control flow structures for different functions. Maybe that fits the spirit of your question... Let me know if not.

So far I'm doing purely static analysis and control flow, but the broader field of reverse engineering includes dynamic/symbolic analysis where you track values through a running/simulated program. Great results but very costly to run.

I've been focusing on making cheap/static analysis better, so I haven't explored the dynamic/symbolic side at all yet.

[1]: https://tigress.wtf/virtualize.html

[2]: https://tigress.wtf/jitter.html

[3]: https://tigress.wtf/flatten.html

[4]: https://tigress.wtf/split.html

[5]: https://tigress.wtf/merge.html

[6]: https://tigress.wtf/encodeArithmetic.html

[7]: https://tigress.wtf/inline.html

My research is about detecting semantically similar executable code inside obfuscated and stripped programs. I don't know what commercial antivirus or vulnerability scanners use internally, but it's possible to generate similarity scores between an obfuscated/stripped unknown binary and a bunch of known binaries. I suspect commercial scanners use a lot of heuristics. I know IDA Pro has a plugin for "fingerprinting" but it's based on hashes of byte sequences and can be spoofed.

My approach is basically: train a model on a large obfuscated dataset seeded with "known" examples. While you can't say with certainty what an unknown sample contains, you can determine how similar it is to a known sample, so you can spend more of your time analyzing the really weird stuff.

The hardest part in my opinion is generating the training data. You need a good source code obfuscator for your language. I've seen a lot of papers that use obfuscator-llvm[1] to obfuscate the IR during compilation. I use Tigress[2] to obfuscate the source code because it provides more diversity, but it only supports C.

[1]: https://github.com/obfuscator-llvm/obfuscator/wiki/Installat...

[2]: https://tigress.wtf/

no need to fight over it, compromise is both elegant and simple:

    YMYY-YD-DM

Jeep owners will be upset if they can't take the backdoors off.

Precedence was a mistake and new language designs should throw it in the garbage.

Left to right is fine. So is right to left. Just pick one and stick with it.

“But PEMDAS!” Ok, YOU can use parens.

If not for Wilhoit's law, we wouldn't have any laws at all!

2026: npm in pure css

Even sticking within the confines of json there's low hanging fruit, e.g. if your data is typically like:

    [{"k1": true, "k2": [2,3,4]}, {"k1": false, "k2": []}, ...]

You can amortize the overhead of the keys by turning this from an array of structs (AoS) into a struct of arrays (SoA):

    {"k1": [true, false, ...], "k2": [[2,3,4], [], ...]}

Then you only have to read "k1" and "k2" once, instead of once per record. Presumably there will be the odd record that contains something like {"k3": 0} but you can use mini batches of SoA and tune their size according to your desired latency/throughput tradeoff.

Or if your data is 99.999% of the time just pairs of k1 and k2, turn them into tuples:

    {"k1k2": [true,[2,3,4],false,[], ...]}

And then 0.001% of the time you send a lone k3 message:

    {"k3": 2}

Even if your endpoints can't change their schema, you can still trade latency for throughput by doing the SoA conversion, transmitting, then converting back to AoS at the receiver. Maybe worthwhile if you have to forward the message many times but only decode it once.

Awesome, the workaround worked for me. Thanks for the quick reply (and your amazing work on cosmopolitan and the rest)!

Am I doing something wrong? I tried this on MacOS 12.6.9 and every binary I've tried so far gives a similar "zsh: exec format error":

    $ unzip cosmos-3.0.1.zip
    $ cd bin
    $ ./python -c 'print(2+3)'
    zsh: exec format error: ./python

That settles it, I need to watch this movie.

see: Jexi

https://www.imdb.com/title/tt9354944/