Hacker News Reader

Comment by jonas21 | Hacker News Reader

Regardless of what you think about the lawsuit, you have to give them credit for their watermarking method:

If I was going to scrape this data and re-purpose it, I would've absolutely cleaned up those apostrophes. The pivoting between straight and curly would certainly be a pet peeve. Unless there's a semantic difference between the two I'm unaware of.

The semantic difference would be important in a song like Baby Got Back by Sir Mix-a-lot, which includes both speech quotes and imperial measurements.

Imperial measurements should use the prime symbol and not straight quotes: https://en.wikipedia.org/wiki/Prime_%28symbol%29#Designation...

The song was released almost a year before UTF-8 was first presented at USENIX, so I think it is reasonable for the lyrics to be expressed using the commonly available technology available of the time.

For lyrics, you should just transcribe the pronunciation, eg "six foot two".

I for one would impose metric... it may be harder to rap, but true talent steps up to a challenge...

"But I would walk 804 kilometres, And I would walk 804 more, Just to be the man who walks a 1,608 kilometres, To fall down at your door..."

It's so catchy! Perhaps: "16093.44 meters high" by "0.2286 meter nails"...

1.608 megametres :^)

The units are not spoken, but for clarity they should be notated in the lyrics.

TIL about prime symbols. Thank you!

My anaconda don't want none of your poor semantics.

I queried for this song's lyrics, and I found only a single quoting symbol (') used entirely throughout.

Taking a layperson's reading of the lyrics, I didn't find anything off due to quoting issues. If there was any disruption, it was minimal and unnoticed.

Sir Mix-a-lot is an American rapper so the measurements in the song are presumably in US customary system units and not imperial units.

Ahh, but is it US customary or US survey?

Could you try not to be so pedantic regarding my pedantry.

Yeah, makes sense, but this is still a pretty good approach. Inserting invisible or unusual Unicode symbols would prompt the scraper to carefully cleanup the read files (maybe even fixing these apostrophes as a result). Unusual whitespace is also likely to be removed and cleaned up. On the other hand, these alternating apostrophes have a chance to stay unnoticed (or neglected), falling through the cracks.

There is a semantic difference between the two. The straight quote is a superset of the curly one.

So "rock 'n' roll" is correct. And "rock ’n’ roll" is correct. But "rock ‘n’ roll" is not correct, since the wrong apostrophe is used. We're not quoting the letter n, we're showing that the letter a was removed.

The vast majority of content consumed by this scraper is never closely inspected by a human, though.

On this quantity of data, you wouldn't be able to do this manually.

If you hope to avoid being caught this way, I'm going to assume you noticed this without the benefit of hindsight and plan to correct all out-of-place Unicode characters automatically. How will you avoid over-correcting?

There's also no reason to believe this is the only fingerprinting Genius has done (they only need to publish the most obvious fail). For example, I can use the same fingerprinting technique but switch between American and British spellings.

This is not a straightforward problem.

I think if you had multiple corpuses of lyrics, you could cross-check for anomalies of any variety (odd quoting, switching between american/british english), etc.

The fingerprinting isn't likely applied to every song, to prevent obvious detection. If you went through multiple databases, you might see N prevailing copies of a song's lyrics, and 1 that seemed different. The one that's different has the anomaly.

I'm not disputing that they proved their point, but this is triggering one of my pet peeves about common misunderstandings of Morse code.

Timing is critical in Morse code. You can't just write out a bunch of dashes and dots to transcribe it without clearly transcribing the rests between dots and dashes as well. They haven't given us the rests at all, so all the info they end up having is:

dot dash dot dot dash dot dot dot dot dot dot dot dash dash dot dash dot dot dot dash dot dot

And that can be interpreted in any number of different possible ways besides "REDHANDED". E.g. it could also be "AU5EWRFE", or any of thousands of different interpretations (actually probably a lot more than that; this would be a fun programming problem). They should have used a binary encoding; 22 bits (all they have given us) is not enough information to uniquely encode the string "REDHANDED". Once you include the short rests that are needed, we're talking 44 binary bits or 22 ternary bits. And if you want the long rests to distinguish properly the spaces between words, then 22 ternary bits won't do it; you need the full 44 binary bits.

>They should have used a binary encoding; 22 bits (all they have given us) is not enough information to uniquely encode the string "REDHANDED".

The fact that the sequence can be interpreted as REDHANDED with a particular way of grouping the input is just being cute. Regardless of the grouping, it is a binary encoding of a 22-bit number, and so would have a one-in-2^22 chance of being reproduced at random.

Edit: To clarify: You're saying they should've mentioned 22-bits in the context of binary digits without mentioning Morse code, and if they did want to bring up Morse code they should've used trits or more bits to encode the stops. I'm saying that the fact that their 22-bit sequence can be interpreted in Morse code as a relevant word is just dressing, and does not detract from the point that the sequence was likely copied. Put another way, if someone tried to counter by saying their sequence could've been generated independently because "AU5EWRFE" and many other strings also encode to the same sequence, it would not affect the facts at all.

I think you missed the first sentence of my post.

Nope, I didn't. Your first sentence says:

>but this is triggering one of my pet peeves about common misunderstandings of Morse code.

and my post is about how there is no misunderstanding.

There is a misunderstanding because anyone who properly understood Morse code would never represent it in this manner where it's indistinguishable from a large number of other possible strings.

There is no misunderstanding because it was never the intention to use the bitstring to represent a unique word in Morse code in the first place. Instead, Morse code was merely used as a convenient method to devise a bitstring from a relevant word.

You'd think Google would be wise to that since they do that themselves.

E.G. When they caught Bing copying them... https://www.wired.com/2011/02/bing-copies-google/

And they definitely do it with maps. There is a tiny little village I visit in rural Roscommon each year. Each year a new major retailer appears to have opened in this 500 population village, well according to Google Maps that is. At the moment there is a branch of New Look situated on a farm down a single track country lane.

This is a variation on the trap street https://en.wikipedia.org/wiki/Trap_street

I recall coming across this in my travels. There was a named "town" at the intersection of two streets - upon passing through there, nothing. Later wondering where the town went I found that it was not ever there and was just present to identify people copying that map.

Called a Paper town.

This is very clever! Also, take a look at Claim 2 of this patent[1]. Do you think these are similar enough to constitute infringement?

[1]: https://patents.google.com/patent/US9881516B1/en

(Software patents should be abolished. I just like to point out their absurdity and how it's easy to independently develop a technique (steganography in a search engine result) that someone has already grubbed a "patent" on.)

This bit at the end is the best part, I think:

> while particular embodiments and applications have been illustrated and described, it is to be understood that the disclosed embodiments are not limited to the precise construction and components disclosed herein. _Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the method and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims._

If I'm reading this correctly, this patent is claiming things that are "apparent" (obvious?) to those "in the know". Computer or not, how did this get granted?

If it's not implemented in PHP, it's not infringing!

The patent claim requires the program to "load available PHP server header information"

That is just so bizarre to me too. That one would be able to copy the exact core of the idea, but write it in a different language, and suddenly it's not infringing, even though it causes the computer to do the same thing.

How these patents even pass the sniff test, I'll never understand.

Reminds me of a much older practice for the same reason: https://en.wikipedia.org/wiki/Agloe,_New_York

That's pretty clever! How the tables have turned since hiybbprqag

Wasn’t sure if you were having a seizure there, so for anyone else who was wondering:

https://www.urbandictionary.com/define.php?term=hiybbprqag

https://www.cbsnews.com/news/hiybbprqag-how-google-tripped-u...

I posted about this the other day and someone who used to work at Microsoft made a really interesting/helpful response - worth checking out: https://news.ycombinator.com/item?id=24112418

Warning: The Word of the Day for Aug 11 at Urban Dictionary is pretty NSFW.

Thanks, sorry, I should have noted that anyway just for linking UD.

pgnttrp!

Does anyone know if ebooks are watermarked in a similar way?

They almost certainly are for some sources. And probably with your unique user id to work out who leaked the copy on to torrent sites.

The way to check this is pretty easy though, get 2 users to bit for bit compare their books to work out if they are identical.

Open the book in Evince, Print to PDF. Bam, your book is now anonymous.

I am not following your line of reasoning correctly.

The watermarking is done using characters that look different from each other. Printing the book will not change those characters, and therefore it is still possible to extract the original information. Harder, but not impossible.

For this information to be lost, Evince would have to print the book in a crappy font that uses a single symbol for both. Which is possible, but I think it's not what you were going for.

The hash won't be the same, which is what matters the most.

They won't have people manually reviewing the punctuation to catch infringement, and it's highly probable that Evince printing to PDF will mess the whole internal structure, so it's hard to automate it.

If you just print it then that defeats the point. They don't care about individual copies, they want to find who is stripping DRM and posting books on torrent trackers.

How likely are you to notice if your resulting PDF had a specific unique pattern of zero-width spaces embedded with it? Or if a few curly quotes aren't curly? Let alone scenarios like the cover JPEG having stenographic info in it...

Absolutely. It usually tends to be either more visible, or less visible though - some pdf files have a literal watermark on the pages, while other formats like epub contain a guid or other watermark content in the source (epub is zipped xhtml)

https://gizmodo.com/harpercollins-is-now-using-digital-water...

https://goodereader.com/blog/e-book-news/everything-you-need...

Amusingly, I believe if you use calibre's ebook conversion to replace stylesheets and add toc, it may also actually remove those markers that have no actual content and only exist to provide a unique ID.

Not ebooks, but interesting technique used to find who is leaking your sensitive documents: https://en.wikipedia.org/wiki/Canary_trap

Not sure about ebooks, but I assume every workplace email sent to all does this to catch leaks.

Can someone explain to me the benefit here? Is it making it less likely for the google scrape to get a search hit?

There's no "benefit", they were just looking for a nice unique way of watermarking textual content, to prove that what shows up on a Google search is indeed sources from them and not some other transcription of the lyrics.

Watermarking, while similar is not the correct word, though the purpose of watermarking and this steganographic embedding is the same.

You have to admit, it's genius!

Reply on news.ycombinator.com