The Technology Behind Making the Internet Archive into the Worlds Worst DVR | Hacker News Reader

The Technology Behind Making the Internet Archive into the Worlds Worst DVR

slifty.com

101 pointsknowtheory14 comments

This is such a good post about:

a. how the internet archive is building its political archive

b. proving to other cultural heritage institutions that they can get their act together and preserve culture today and make it relevant today

Ok, so Ted Cruz ads are a particular part of culture, but bravo.

I used to work at The New York Public Library's Labs and we had to fight to do anything with anything that wasn't blatantly in the public domain. Meanwhile, one of my colleagues there, Leonard Richardson, would go off and create massive archives like the Minecraft Geological Survey (http://mcgs.crummy.com/#mcgs) in the evenings.

Seriously, we need more folks working in these big organizations that are very good at surviving for more than a hundred years, acting as arks of human culture, that can collect and make useful today's materials. It's just our ability to remember today at stake.

So much culture is going to be lost because people are shell shocked by copyright lawyers and the constant drumbeat of expansive copyright enforcement. We are destroying ourselves to protect the perpetual income streams of publishers.

History only qualifies as a profit center to them, If it didn't happen when nitrate film was burning, it will take a lot of organization and legislation to make it happen with bits and bytes.

I like how the Center For Public Integrity just used "Duplitron5k" discovery of a supposedly "Internet-exempt" hybrid super PAC political ad garnering free time in cable news.

(https://www.publicintegrity.org/2016/05/12/19677/news-media-...)

The PAC asserts because the ad is a “Web video” they are exempt from the kinds of public disclosures applied to paid political communications broadcast over the airwaves.

Well the project is bizarre and whimsical and I guess pretty expensive, but the way it's written about here makes me an enthusiastic fan. Well worth a read.

Looking at the site, it looks like this focuses on US politics. Being a US citizen, I'm totally fine with this. An easily searchable archive of political ads should make it easier to call BS on our politicians when necessary.

I find myself wondering, however, what it would take to extend this effort to other countries of the world. I have a few friends in Canada and in Europe, and speaking with them, I get the impression that the politics of their respective countries are just as susceptible to shenanigans as that of the US. I think someone politically active in say, Sweden, would want to be able to call out his or her leaders for supporting puppy-kicking or baby-eating just as much as someone in the US would.

You are correct -- this is only US Politics right now.

The biggest limitation we face is simply getting reliable access source material / TV content. In order to count airings of ads in another country we would need access to video streams.

We do have a few international channels in our TV Archive (although we don't use them for the Political TV Ad Archive). We'd love to expand that some day...

Many countries also just don't allow political advertising on TV. Sweden allows it, but almost all the TV channels are based in the UK which bans it.

We do have explicit "and now a party political broadcast by the {partyname} party." In the uk.

When I was an undergrad (a few years ago), we had a visiting lecturer[1] who taught journalism classes mostly for the newspaper folks on campus. Well prior to consumer VCRs, he had students tape a wide range of political ads and other political-related video. It was really interesting having access to all this stuff in a pre-YouTube and Internet Archive era.

[1] http://www.nytimes.com/1997/07/12/nyregion/edwin-diamond-a-w...

I picked a random candidate and then clicked on play on all of the videos at the same time. That was a special kind of insane.

tl;dr they are archiving political ads off TV and try recognising them in software by their sound using http://labrosa.ee.columbia.edu/matlab/audfprint/ I think.

They don't say how political ads sound different from other TV - I guess it works? Or maybe they are just using it to check if they have that ad already like how Shazam works?

It's all about the FREQUENCIES! Actually though, this system can be used to find copies of any sound -- sound bytes, commercials, music, whatever. We just happen to be using it for politics because we apparently hate fun.

We're using the python port of audfprint though: https://github.com/dpwe/audfprint

And yes -- we have a big bucket of known ads, and a big bucket of archived programming, and we're comparing new content with the appropriate buckets.

NOTE: we are talking about ways to release the fingerprints of all the TV we archive so anybody can "search" TV for copies of sounds.

There are possibly some obvious and unique heuristics. For example, the FEC requires that broadcast commercials must either an explicit audio statement of who authorized the commercial:

http://www.fec.gov/pages/brochures/notices.shtml#appearance

e.g.

"I'm Ted Cruz and I approve this message"

"This message paid for by the Americans for A Better Tomorrow Tomorrow"

There are very, very few non-political commercials that have this kind of phrasing.

edit: I'm only referring to how political commercials are different from non-political commercials. Obviously, the above heuristic has not much value in determining whether a particular ad is the same one as one already recorded and identified by the Archive.

Reply on news.ycombinator.com