Hacker News Reader

Im wondering how the already preserved and publicly available film content is being considered being made freed ? Theres very clear information on rights, reproduction restrictions, and the preservation efforts that have already been made here: https://www.archives.gov/research/order/film-sound-video-dc....

I wonder what this material is. Are these commercial films, historical footage, or what?

The OP says they're public domain, so I'd guess it's films produced by the government.

...or past the copyright window.

> Years ago in partnership with the National Archives Amazon.com digitized a tremendous amount of film for the National Archives, with the catch that the material cannot be freely disseminated online by the National Archives until Amazon breaks even on their digitization investment. It’s been years now and for a variety of reasons - many of which are Amazon’s fault - there still exist a solid number of discs that Amazon hasn’t sold even one of. (In part because amazon hasn’t even had them all available to sell. Pretty ridiculous).

> The flipside of the catch is that the DVDs can be viewed and even copied for $0 on site by researchers. I am doing research there and I have have a research pass. I was looking at many of these titles yesterday. It’s time to set these national treasures free.

-- https://old.reddit.com/r/DataHoarder/comments/a6fkpm/i_have_...

That doesn’t answer what type of media is on the disks.

It says film. So, it could be movies or images. What else would be possible? Are you thinking of quickest ways to rip the DVDs if they are movies?

Yes, but "film" could refer to a lot of different content. Movies, TV shows, documentaries, commercials, interviews, news reports etc.

GP was just wondering out loud about the content itself, not the format of said content.

A little off-topic, but it's interesting to see the shift in meaning of the "reel of film" icon over the decades; a long time ago, it would signify any video, but now it's more associated with an actual cinema film.

So... it's already been 3 days, right?

My guess is they're the films described in the CustomFlix partnership announced here:

https://www.archives.gov/press/press-releases/2007/nr07-122....

For all the helpful advice offered both here and on Reddit about how to do this, I wish there had been more time asking if this was something the OP should be doing in the first place.

I completely understand the frustration if this is indeed the above set of films and it's been over eleven years since the digitization agreement and files have not yet been made freely available to the public.

That said, planning to use a researcher's pass to "set these treasures free" and blaming Amazon for not having generated more revenue from these films makes it sound like the OP thinks he knows better than the staff of the National Archives how to best care for these assets and that he knows better than the folks at Amazon how to turn a profit. To me, it sounds more than a little arrogant.

A big reason the National Archives enters into agreements like this with companies is that digitization, especially on their scale, is expensive. If it weren't for agreements like this, Amazon would only want to digitize the films for which they knew they could turn a profit and the vast majority of the collection would sit un-digitized and be at risk of loss. The tradeoff is between the immediacy of access vs the number of assets digitized and the team at NARA made the decision that it was better for the American people to have more assets digitized.

I'm worried that the next time NARA is in talks with someone about a digitization agreement (for example, if there's a large number of early jazz audio recordings on 1/4" reels and Spotify is interesting in paying the cost of digitization in exchange for 2-3 years of exclusivity) that the company will point to this example and say "didn't you just let a researcher publish the entire collection Amazon digitized? How can you assure us the same thing won't happen with these recordings?" The result will be the National Archives clamping down on researcher access. I think that would be a net loss for everyone.

> The result will be the National Archives clamping down on researcher access.

Or maybe the National Archives will stop making deals where free distribution of the material is unlikely to happen for decades and will instead open up greater access to researchers.

Or they could make the same deals but put a deadline on them to prevent the exclusivity from sitting in limbo forever. That would at least give Amazon an incentive to distribute the material rather than just squatting on its monopoly.

A reasoning that doesnt require (too much) arrogance: contracts and business rules hold despite changes in context. Plans change, people leave, things are forgotten. But the contract, and its limitations, go on.

Since amazon was only looking to get its investment back, but not a profit, then clearly the intent of both parties was to eventually have it release to the public. Whoever on amazon’s side pushed the deal presumably thought they would be sold, but it clearly didn’t happen. It’s much more likely this has been sitting on a backburner somewhere, left as some forgotten plan, than some kind of long-term strategy to induce sales.

Thus, by virtue of an ill-written contract, we’ve entered a situation that no one wants. Amaxon no longer cares about it, the museum presumably prefers releasing it, and the public can only benefit from it.

By virtue of that same contract, there’s an escape hatch that might bring us back to a state where everyone is content.

It would make sense to exploit it.

Ofc, this is assuming Amazon doesn’t care. But amazon is a company, and the larger a company is, the less distinguishable it is from a government. And governments certainly have control over things it has collectively forgotten about, as it only rarely operates a single, like-minded, cooperative organism.

And amazon is indeed a very large company. It would hardly be unsurprising for Amazon to not even be aware this contract still exists.

> A big reason the National Archives enters into agreements like this with companies is that digitization, especially on their scale, is expensive. If it weren't for agreements like this, Amazon would only want to digitize the films for which they knew they could turn a profit and the vast majority of the collection would sit un-digitized and be at risk of loss.

And yet there are researchers willing to do this quickly and for free, so why do we even need Amazon in the loop?

Seems counter-intuitive to Amazon's own public dataset program / policy: https://aws.amazon.com/opendata/public-datasets/

>Democratize access to data by making it available for analysis on AWS.

>AWS evaluates applications to the AWS Public Dataset Program every three months.

>If we bring your dataset into the AWS Public Dataset Program, we will cover the costs of storage and data transfer for a period of two years

yeah i don't think that's public,more like a rental service. i wouldn't bother working with that unless i had free AWS access through corporate

Easy: get a bunch of DVD drives and a ~1 Gbps uplink. 2000 4.7GB single layer DVDs in three working days is about 870Mbps. Upload them as you rip to S3 and then send a pull request!

https://github.com/awslabs/open-data-registry/

So the 3 days is for figuring out how to use git, then?

lmao

obligatory xkcd: https://xkcd.com/1597/

If anything is out of your few commands, just save work elsewhere, delete the project, and download a fresh copy.

I've, frankly, done that many times at this point... when your merge request gets too wonky, it's just easier. I do try to remember to squash, then rebase generally, though there are times I just forget or mess up.

When in doubt, "how does the team here handle these conflicts? At my old job we just did <made up thing> to solve it, I've forgotten the process here."

Easy: get a bunch of DVD drives and a ~1 Gbps uplink.

Can you detail how one easily gets a 1 gigabit uplink installed temporarily in the National Archives?

I've had telcos drop fiber into the middle of fields in the middle of nowhere, so it's probably possible. But it feels like there's a lot of complicated bureaucracy to navigate, and the researcher OP is time-restricted.

Sneaker shuttle the SSDs from the National Archives to your friends apartment with the uplink.

sigh

No one got the joke. I’m suggesting that the DVD’s should be submitted as an AWS public dataset, because Amazon is the company that digitized them in the first place, and the content is apparently public domain.

>In part because amazon hasn’t even had them all available to sell. Pretty ridiculous

not ridiculous, good lawyer preparing the deal

Not ridiculous, but very sad

What's "good" about deliberately keeping public domain content out of the public domain?

Not "good" as in "good vs. evil", but rather "good" as in "proficient at the skill of lawyering".