Stegasuras: Neural Linguistic Steganography
steganography.live
steganography.live
Anyway, really cool work.
Computers can detect stenographically with statistics.
That's part of what this project/paper is about, they created a model that not only produces text that passes as human, but it's significantly less detectable than previous encoding methods.
Steganography. Stenography is something else unrelated.
E.g. One sentence of the 'cover text' I got read:
"Then, on September 8, 1816, Sherman surrounded Army headquarters and killed three emissaries and surrendered with large supplies and safety pins."
Sherman wasn't born until 1820.
I used this LM Context: "This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this E-mail. Please notify the sender immediately by E-mail if you have received this E-mail by mistake and delete this E-mail from your system."
To encrypt this fake, but real looking PII data: "Jeffery,Gourlay,2/2/55,495-24-1236"
And produced this output cover text: "Disclaimer: An individual may be reminded of this email at any time by telephone from a different address, e.g. e-mail. However, online resources and other media (e.g. e-mailed stories, articles, etc.) that include original content, are not covered under this policy. The information contained herein"
Malware or a bad actor could transmit this past enterprise InfoSec controls all day without easy detection.
If I understand correctly, this approach can generate steganographic output that is virtually indistinguishable from the output generated by the language model used. Quoting from section 4.3 of the paper:
"Most striking, arithmetic coding with the unmodulated language model induces a q distribution with a KL of 4e-8 nats. This indicates that, consistent with theory (Sallee, 2004), arithmetic coding enables generative steganography matching the exact distribution of the language model used."[a]
Regardless of theory, I can't help but think, holy cow, the K-L divergence between the distribution of tokens in the steganographic output versus the output of a GPT-2 model is a minuscule 0.00000004 nats!
Did I understand correctly? Given that language models are getting even larger and the distribution of their output is getting harder and harder to distinguish from that of human output[b], I find this incredibly powerful.
Anyway, congratulations and thank you for sharing on HN!
On https://steganography.live/encrypt leave the LM context intact and type your secret message into the secret message box. Then click Encrypt at the bottom.
Select the output cover text and copy it to your clipboard.
Now go to https://steganography.live/decrypt and paste the output cover text in and click Decrypt to recover your secret message.
----
On January 25, 1798, Washington was elected President of the Confederate States of America. He later led the United States in the battle of Fort Sumter, North Carolina, in which he secured the passage of the Union Army, defeated the enemy and secured its last major victory. Washington was awarded the Order of the Pacific on December 27, 1798. His wife, Bea, was born in 1833 and settled in the Virginia river valley at East Mary Street, Bowersville. In 1855, Bea had married. Shortly thereafter, General John F. Washington was elected Governor of Virginia and commissioned to serve a year in captivity. He became enthralled with the Union and persuaded Henry P. Morgan, to meet his need for volunteers. Washington volunteered with the Federal troops using a French silver coin and with their aid the Americans located and captured the greatest concentration of troops there. After her capture, Bea was sent to Washington's camp in Richmond, Richmond and had a farewell ring to the American flag. It was this ring that Washington had set aside for himself with his death and a letter from his wife, who was
----
EDIT: decrypting that message twice (i.e. decrypting the result of decrypting the above message) apparently "works" and produces some interesting results. It gets even more wacky as I continue decrypting more and more outputs.
Type the text to encode with in the top box, and the text to encode in the bottom box and press the bottom at the bottom. Easiest is to just click on the right like Gettysburg Address or Apologies by Plato to fill it with a preset text, but if you fill it yourself you need some words that follow other words to work, like at a minimum "a a b a".
Edit: I can't get it to spit out anything no matter what I do.
Both the context and secrete message change the encrypted message
The closest I came was putting the text in a gray coded word per token form, then using GPT-2 as the error metric in the encoder, but the resulting bitrate was very low.
I wrote a steganographic tool in this same spirit back in 2011, called Plainsight.
Back then, we didn't have deep learning, and the "Imagenet moment for NLP" had yet to arrive.
My Python code, with examples, is here: https://github.com/rw/plainsight
Unlike the OP, my Plainsight algorithm is 100% invertible by construction, and accepts binary input. (I verified the inversion process with "roundtrip fuzzing", a technique I still use today.)
Plainsight uses each bit of the input message to generate tokens. Bits are used to decide how to traverse a Huffman-style n-gram tree, weighted by frequency. This tree of n-grams is the model used in both the encoding and decoding steps. The drawbacks to my method are that the output 1) can be verbose and 2) does not convince a human that it's plausible, except for short messages.
Stegasuras has orders-of-magnitude better output, and seems to solve the problems I couldn't solve eight years ago. I would venture that their new result has as much to do with advances in language modeling, as it does with the particulars of their encoding and decoding algorithms.
I'll also note that I'm glad these researchers were able to use grant money to do this work. As a non-academic, I applied for an AI Grant to support me in upgrading Plainsight to use deep learning, but I was turned away at the time.
Finally, one of the ideas I picked up back then is that spam can be used to contain secret messages. Send enough gibberish to enough people, with your intended recipient included, and you'll look like a spammer--not a spy:
$ wget https://spamassassin.apache.org/publiccorpus/20030228_spam.tar.bz2
$ tar -jxvf 20030228_spam.tar.bz2
$ cat spam/0* > spam-corpus.txt
$ echo "The Magic Words are Squeamish Ossifrage" | plainsight -m encipher -f spam-corpus.txt > spam_ciphertext
$ cat spam_ciphertext
(8.11.6/8.11.6) 3 (Normal) Internet can send e-mails until to transfer 26 10 [127.0.0.1]
also include address from the most logical, mail business for your Car have a many our
portals ESMTP Thu, 29 1.0 this letter on internet, <a style=3D"color: 0px; text/plain;
cellspacing=3D"0" how quoted-printable about receiving you would like width=3D"15%"
width=3D"15%" border="0" width="511" Date: Tue, 27 Thu, 19 26 because
zzzz@localhost.spamassassin.taint.org for
$ cat spam_ciphertext | plainsight -m decipher -f spam-corpus.txt
Adding models:
Model: spam-corpus.txt added in 2.57s (context == 2)
input is "<stdin>", output is "<stdout>"
deciphering: 100% | 543.84 B/s | Time: 0:00:00
The Magic Words are Squeamish OssifrageEdit: nevermind, found it. For those looking, it's at https://github.com/harvardnlp/NeuralSteganography
?