CyberChef – The Cyber Swiss Army Knife | Hacker News Reader

CyberChef – The Cyber Swiss Army Knife

gchq.github.io

392 pointssimonpure54 comments

I'm a big fan of CyberChef. One of its most useful features is "magic" and turning on "intensive mode". This will automatically detect the encoding used and can often detect 2-3 levels of encoding.

Sometimes it can detect quite a lot more than 2-3 levels! The "magic" function really feels like it earns its name.

If you like that, try FTFY https://ftfy.readthedocs.io/en/latest/ which can automatically repair a huge range of ways that Unicode text can be broken by re-encoding. This would be a great addition to CyberChef if they could reproduce it in JS.

Classic example from the docs:

> ftfy can fix multiple layers of mojibake simultaneously:

    >>> ftfy.fix_text('The Mona Lisa doesnÃƒÂ¢Ã¢â€šÂ¬Ã¢â€žÂ¢t have eyebrows.')
    "The Mona Lisa doesn't have eyebrows."

protip: Open the JS console (F12 / inspect) and start the CyberChef challenges!

43 6f 6e 67 72 61 74 75 6c 61 74 69 6f 6e 73 2c 20 79 6f 75 20 68 61 76 65 20 63 6f 6d 70 6c 65 74 65 64 20 43 79 62 65 72 43 68 65 66 20 63 68 61 6c 6c 65 6e 67 65 20 23 31 21 0a 0a 54 68 69 73 20 63 68 61 6c 6c 65 6e 67 65 20 65 78 70 6c 6f 72 65 64 20 68 65 78 61 64 65 63 69 6d 61 6c 20 65 6e 63 6f 64 69 6e 67 2e 20 54 6f 20 6c 65 61 72 6e 20 6d 6f 72 65 2c 20 76 69 73 69 74 20 77 69 6b 69 70 65 64 69 61 2e 6f 72 67 2f 77 69 6b 69 2f 48 65 78 61 64 65 63 69 6d 61 6c 2e 0a 0a 54 68 65 20 63 6f 64 65 20 66 6f 72 20 74 68 69 73 20 63 68 61 6c 6c 65 6e 67 65 20 69 73 20 39 64 34 63 62 63 65 66 2d 62 65 35 32 2d 34 37 35 31 2d 61 32 62 32 2d 38 33 33 38 65 36 34 30 39 34 31 36 20 28 6b 65 65 70 20 74 68 69 73 20 70 72 69 76 61 74 65 29 2e 0a 0a 54 68 65 20 6e 65 78 74 20 63 68 61 6c 6c 65 6e 67 65 20 63 61 6e 20 62 65 20 66 6f 75 6e 64 20 61 74 20 68 74 74 70 73 3a 2f 2f 70 61 73 74 65 62 69 6e 2e 63 6f 6d 2f 47 53 6e 54 41 6d 6b 56 2e

Any hints on challenge #5? FromHex returns something that looks like it has the bz2 compression header. Trying to decompress w/ bz2 doesn't seem to work though.

42 5a 68 39 31 41 59 26 53 59 34 3d 45 44 3d 31 37 3d 44 45 3d 30 30 3d 30 30 28 3d 44 46 3d 38 30 3d 30 30 3d 31 30 68 3d 30 37 3d 46 37 3d 46 30 3d 30 43 3d 30 30 66 3d 30 30 3d 33 46 3d 45 46 3d 44 46 3d 46 30 30 3d 30 30 3d 0d 0a 3d 44 41 3d 38 31 48 3d 43 34 3d 43 38 3d 44 30 68 3d 30 33 43 3d 30 38 3d 30 30 3d 30 30 3d 43 30 3d 43 38 32 3d 30 30 3d 30 43 46 3d 38 33 21 3d 39 30 3d 31 38 35 4f 3d 31 34 3d 46 34 46 3d 39 34 3d 46 43 52 6f 42 3d 0d 0a 3d 39 45 3d 41 37 3d 41 36 6a 46 3d 39 41 63 3d 31 32 6d 46 3d 30 34 3a 32 3d 43 38 52 5d 52 7a 68 3d 31 30 3d 31 31 3d 30 31 3d 41 41 3d 31 46 3d 38 36 3d 31 45 3d 42 30 52 3d 42 41 3d 30 42 5e 3d 31 36 25 3d 41 46 3d 30 37 3d 0d 0a 3d 46 31 3d 44 37 38 74 3d 43 46 3d 42 37 3d 31 36 3d 39 35 3d 38 42 3d 30 35 3d 45 41 3d 39 34 30 44 3d 46 31 3d 31 34 3d 43 33 3d 32 32 62 6a 3d 32 32 3d 38 43 6d 3d 41 41 3d 45 35 3d 41 34 3d 44 45 3d 39 34 53 3d 42 34 2e 3d 0d 0a 3d 41 36 3d 42 33 3d 41 43 3d 31 42 3d 38 30 3d 30 38 3d 33 46 46 3d 44 41 2c 49 3d 43 35 3d 42 38 09 28 3d 41 32 72 35 3d 30 38 40 3d 46 41 2a 3d 31 45 61 3d 44 45 5e 3d 39 45 3d 44 42 3d 31 41 3d 43 33 3d 38 42 26 3d 0d 0a 3d 43 44 3d 39 32 3d 44 30 3d 43 32 3d 31 32 3d 41 41 2a 3d 30 33 48 3d 46 38 45 3d 45 38 30 3d 41 35 3d 43 35 40 3d 44 45 3d 46 30 3d 31 37 3d 43 38 3d 44 34 3d 41 30 3d 39 46 3d 43 45 3d 46 34 20 76 3d 30 36 2c 3d 0d 0a 43 6e 3d 45 37 3d 39 30 3d 44 39 3d 43 31 64 3d 30 35 3d 42 39 3d 30 30 58 3d 44 32 48 3d 46 41 3d 41 46 3d 41 44 3d 38 38 3d 38 30 30 3d 31 38 3d 46 37 3d 39 30 3d 31 32 4d 3c 3e 76 3d 30 45 6b 3d 38 42 5b 3d 42 46 3d 0d 0a 3d 45 36 3d 31 30 3d 38 46 3d 44 31 3d 44 31 3d 45 41 3d 45 42 3d 39 44 44 3d 44 30 3d 44 31 3d 43 34 74 3d 38 43 2b 3d 41 38 5e 3d 43 42 31 3a 50 3d 42 37 3d 30 38 3d 42 43 09 3d 0d 0a 3d 41 32 3d 31 37 3d 41 37 3d 31 46 3d 46 31 77 24 53 3d 38 35 09 3d 30 33 4e 3d 44 31 7d 3d 45 30

I just got through this one and was having the same block as you, took me a while to connect the dots. There's a step missing, take a look at some possible encodings before you decompress. Hope that helps without giving too much away, good luck!

I think the landing page is more informative personally:

https://github.com/gchq/CyberChef

Love how the contributors all have what seems like dedicated accounts with no 'legal name'. Take for example this: https://github.com/gchq/CyberChef/commits?author=n1474335

    n1474335

Great name. In typical GCHQ style they have good OPSEC.

You can find more contributors here:

https://github.com/gchq/CyberChef/graphs/contributors

    d98762625
    s2224834

Among others :)

afaik those are user IDs of Chinese(?) messengers. Devs or others there like to use those as usernames because the are (of course) mostly unique and also enable others to easily contact them.

I don't think in this case it'll be messenger IDs (and if it is definitely not chinese) as GCHQ is the UK's equivalent of the NSA.

They're a pretty secretive bunch, when they present in person they don't use real names, and if you go to their headquarters you have to leave all electronics at the door (did a talk there once and had to buy myself an analog watch for the day!)

Why did you buy an analog watch as opposed to just go without?

I wanted to know what time it was for things like how long till my talk starts and how I was doing for timing. :)

They're defo not Chinese. GCHQ is part of UK Intelligence. They also have the $id@publicemail.provider to hide their emails too.

Chinese messenger ID-as-username mostly starts from q(QQ - counterpart FB Messenger) and s(Sina Weibo - counterpart FB/Twitter). Others(a - admob, b - bilibili etc.) are quite rare tbh so I don't think that's the case here.

Interesting point: They just use Gmail.[1]

[1]: https://github.com/gchq/CyberChef/commit/c423de545fd0d27aabe...

You can put anything you want in the author and committer email so this doesn't guarantee that they actually use or even created those gmail accounts.

I recently started building an offline, desktop app in Flutter as a collection of dev utilities: https://github.com/nileshtrivedi/devtoolbox

Flutter was a promising choice as it'd give me Windows, Mac OS and Linux build from a single codebase - and even the possibility of orgs running this internally as a self-hosted webapp. But one of the ideas I had was to bundle CyberChef with my app and open in a webview. Turns out, Flutter doesn't support webviews on desktop platforms at all. https://github.com/nileshtrivedi/devtoolbox/issues/4

That looks nice, I also worked on something similar but CLI based. It's having both TUI as well as CLI modes.

https://github.com/abhimanyu003/sttr

Since it's written in GO, you can run it pretty much anywhere you like :) ( cross platform )

Yes, it support file inputs as well.

You should check out the WebViewX plugin, for simple websites it works without any drawbacks, even though it's pretty hard to find by just searching for it

If this is the right package, this too doesn't seem to support desktop platforms: https://pub.dev/packages/webviewx

Is anyone aware of cli tools with similar functionality (besides chepy[0])?

I appreciate the magnitude of cyberchefs operations, but having to switch to the browser for these tasks can be cumbersome at times.

[0] https://chepy.readthedocs.io/en/latest/

Edit: I had missed that cyberchef offers a node API exposing most of its operations: https://github.com/gchq/CyberChef/wiki/Node-API

https://github.com/Ciphey/Ciphey

Past discussions

https://news.ycombinator.com/item?id=20767183

https://news.ycombinator.com/item?id=13056254

Thanks! Macroexpanded:

CyberChef – Cyber Swiss Army Knife - https://news.ycombinator.com/item?id=20767183 - Aug 2019 (59 comments)

CyberChef - The Cyber Swiss Army Knife - https://news.ycombinator.com/item?id=20543810 - July 2019 (1 comment)

CyberChef – The Cyber Swiss Army Knife - https://news.ycombinator.com/item?id=13099687 - Dec 2016 (1 comment)

CyberChef – A Cyber Swiss Army Knife - https://news.ycombinator.com/item?id=13056254 - Nov 2016 (139 comments)

This is perhaps the first repost where using "The" has been successful

We usually take out "The" when it is used as a marketing trope. I missed it this time.

I was hoping this might be about the Victorinox Cyber Tool, which is actually a Swiss Army Knife: https://www.victorinox.com/global/en/Products/Swiss-Army-Kni...

Because I knew it had to exist. Thanks for not letting me down, Victorinox. https://www.swissknifeshop.com/collections/usb-flash-drive-s...

I have one of its predecessors (2GB USB 2.0 flash drives). Costed around $60 back then and it was worth it. There's hardly a day i don't use it.

After a very long time of daily usage one of the knifes broke. Totally my own fault. I sent Victorinox an email to thank them for their excellent product that lasted a long time of abuse. They replied to mail the thing to them so they could get it repaired for free. Everything was replaced, except the flash drive. Awesome thing, awesome service!

It's amazing that some companies can still do that. But then you get things like L.L. Bean ending policies because assholes are buying at Goodwill and then sending them in to be refurbed for free.

After a recommendation on HN, I carry one of these on my keychain: https://www.amazon.com/Samsung-BAR-Plus-128GB-MUF-128BE3/dp/... . Essentially indestructible, as everything is solid metal casing.

I'm so lame, I was hoping they'd have wrapped this in an Electron app for the "Download" option.

It's a static website. If you want to use it as an application you can download the compiled page (https://gchq.github.io/CyberChef/CyberChef_v9.32.3.zip) and create a shortcut to the 'Cyberchef.html' file contained within.

I run it in my "homelab". It's all very well self contained. The UX could use some work, but I think it's a case of function over form.

I'm still looking for my ultimate hex editor (TUI/GUI) for offline use. Not been impressed with any of the many options I've tried on Linux.

ht-editor was a fantastic editor[1], similar to hiew[2] on windows. Unfortunately ht-editor codebase is a bit hard to extend and it's based on really old modified binutils headers. I was trying to update it, but it's probably better to just write it from scratch, it's still much more straightforward to use that most modern cli hex editors.

What is with russians and their love for advanced windows cli tools? IDA Pro, hiew and far manager[3] come to mind.

[1] http://hte.sourceforge.net/

[2] https://www.hiew.ru/

[3] https://www.farmanager.com/

Reminds me of the old "Sam Spade."

This is actually very useful. I've been spending the past few weeks working on a cross-platform, native dev assistant app like this written with GTK3 and Rust. It's been a really good exercise in designing meaningful GUIs and, well, usable code. I don't have anything to show right now, but when I get it to an MVP state I'll be sure to share it with everyone!

I'm curious to hear what things people want to see out of an app like this. What utilities are you constantly Googling for that you'd rather have on your desktop?

Something I run into a lot is a JSON-like blob of text I'd like to be formatted as JSON (new lines and indentation). Most JSON formatters choke on improper JSON (understandably). It would be great to have a tool that was more lax. Like browser support for terrible HTML lax haha. If I could paste JSON-like strings into a text area and have it fix and format it as best as can be, that would be great. Some examples of non-JSON syntax to handle would be like single quotes instead of double, arbitrary JSON nodes (not necessarily wrapped in `{}`), some pre or post text (some non-JSON text at the beginning or end), comments amongst the JSON, etc. Another JSON aspect that would be useful is something to escape/unescape JSON (specifically double quotes). I deal a lot with JSON that includes escaped JSON in values and it would be great to have some better way to visualize and process those blobs. Sublime Text has a nice plugin to handle some of the escaping/unescaping[1].

[1] https://github.com/Nadock/json_stringify

Thanks for the suggestion! I'll look into that as it seems particularly feasible, Rust has notoriously good JSON deserialization libraries.

I feel like I am forever having to hack together things to parse json to csv. It feels like there is never an easy path (indeed jq is frustrating, I usually end up just solving in python)

Not a fan, I have tried say formatting a large JSON API results (with no formatting) and it just runs out of memory because: 1) the string is huge like 64mb+ 2) it tries to do syntax highlighting and blows up big time on the resulting formatted value.

For a utility tool it's awful finnicky on real world data and I think tries to be pretty at times rather than useful, or perhaps a case where a browser makes a poor Editor.

We might need to try fine tuning some limits. If CyberChef thinks it will have an issue rendering some text, we'll stop trying to display it and offer the user the ability to render only a part of the result or download the file.

Here's an example of me trying to format a 100mb file.

[0] https://imgur.com/a/3DtqnuL

Very similar to DevUtils https://devutils.app/

CyberChef is free though

The author/s of Dev Utils have actually got a comparison! https://devutils.app/devutils_vs_cyberchef/

I really apricated the honestly about the pros and cons of each.

I always wondered if any of the online tools like JSONlint or any other tools that manipulate data would save it or try to extract data as it is often used to indent or decode potentially sensitive data.

it took me far too long to realize you have to drag the operations to recipe area and not just click them. /facepalm

I'm a great fan of cyberchef, really easy for quick chaining of decoding techniques to look at various types of data.

This sounds like a useful tool based on the comments, but the site is completely unusable on my iPhone. :/

lol

this tool is extremely useful to have in infosec CTFs!

It is very useful. I have issues with how both GCHQ and NSA operate, but I also really like the tools they provide like Cyberchef and Ghidra as they automate a lot of workflows.

The 'magic' command in cyberchef is pretty magic, especially for obscure (to me) utf and language encodings, though I have certainly run down more than a few rabbit holes because it detects the file magic for 'inflate' compression in pretty much everything, which would be perfect for hiding embedded files because of the number of false positives that appear when you're looking for them, and it's just the algorithm someone of a certain vintage who was serious about hiding something from everyone but someone else of that era would use.

The image analysis stuff is great. I use palette randomization for detecting embeds, and the entropy analyzer/visualizer is great fast method for detecting encrypted and zipped payloads. I like that it's fast, and it's there on the web so I can use it on anything.

The loading page alone was worth the click.

Anyone worked at gchq?

Reply on news.ycombinator.com