American scientists unearth lost 1960s polar satellite images | Hacker News Reader

American scientists unearth lost 1960s polar satellite images

barentsobserver.com

228 pointstrauco49 comments

About a year ago the Avaneya project released software that can decode the Viking Lander tapes. A few tapes conatain images that were (as far as I know) never released.

Samples: https://gist.github.com/jakeogh/fa995a3277d500ab59b1

https://directory.fsf.org/wiki/Avaneya:_Viking_Lander_Remast...

Can someone explain how they were able to take and send such high quality digital pictures back then?

This is amazing!

According to Wikipedia they used a movable mirror that reflected the light to 12 photo diodes for different wave lengths and then they just scanned the environment from left to right (or maybe the other way round) with five vertical lines of 512 pixels per second. The 300° panoramas are made out of 9150 (vertical) lines.

[1] http://en.wikipedia.org/wiki/Viking_program#Camera.2FImaging...

www.jpl.nasa.gov/news/fact_sheets/viking.pdf

UHF 1Kb/sec to the orbiter, also could use S-Band direct to earth at a lower bandwidth. The orbiter could buffer 40 Mb. Not sure how fast its link to the DSN was.

ed: 16Kb/sec in there too. Example DSN doc:

ipnpr.jpl.nasa.gov/progress_report2/42-33/33C.PDF

My memory was everything was first shown grayscale, and then color. Not sure if that was a ground processing thing or the lander/orbiter sent two streams. Maybe the orbiter buffered the deeper bits and sent 8bit for near-realtime enjoyment.

A 40 MB buffer on an interplanetary spacecraft in 1975 is pretty impressive.

Looks like the buffer was on tape - it would record at the speed of acquisition, and then when it was time for transmission it would spool the tape at whatever rate was appropriate for the quality of the radio link.

And it's small b, so 5 Megabytes on the tape.

(Educated) guess: the monochrome image was simply the first wavelength band that came down. The later color image could be composed after the other bands were received.

I don't know if a single wavelength monochrome image would look very good, especially if it were, say, blue, instead of red.

Creating the color images was a lot harder than we might imagine, in a world where Photoshop does CMYK separations at the click of a mouse. Since I'm old enough to remember the Viking landing, they first reported on TV when the first color pictures were processed that Mars had a blue sky just like Earth. However, this was quickly corrected once the color pictures were correctly calibrated. Of course, interestingly enough you can see blue in the Martian sky at dusk or dawn.

Of course it depends on the wavelength, but single band images look a lot like any grayscale image. It's typically only when different single-band images are compared side by side that the differences really jump out. Which is, of course, why they bother with multiple bands. I worked on the lunar reconnaissance orbiter camera, and was involved with image processing and calibration for the wide angle camera, which is multiband (5 bands in visible, 2 in UV).

I find old data strangely fascinating. I actually enjoy working with legacy systems every now and then - the older the better. There's something therapeutic about carefully making sense out of old and/or poorly maintained data.

I wonder if there's a business opportunity in this area, especially for digital data. Something like a consultancy specialized in extracting legacy data and migrating it to modern databases where it could easily queried.

I think it's not so strange -- digging through old data in archives is what historians do. If you compiled your findings into a book, you'd be writing bona fide history.

History is stupendously fascinating. What dawned on me in college is that history is much more than retelling stories that have already been told. By doing archival research, you're making it possible to tell entirely new stories, because only after events are over is the data available to to compile and synthesize what actually occurred in fine-grained detail. These stories can be unknown to the participants. It's tremendously important work.

In the sense that history attaches meaning to otherwise meaningless human events, history is nothing less than the search for meaning itself. Fascinating indeed.

In school, in science class and history class, I always felt we focused too much on "what we know", and not enough on "how we know it".

I got more of a sense of how research is done in college as opposed to high school. I think the simple fact is that there's a lot more information to teach that falls in the "what" category. The actual hands-on work of historians is not something you really study in detail unless you're a history major, I think (I wasn't). In the case of science, I think virtually all children learn about how science experiments are done and the scientific method early in their science education -- perhaps that wasn't quite what you meant. You might be right that the how doesn't get enough focus. But I think that if a student is interested enough in science to be a scientist, that person will not be deterred by missing out on that.

You might enjoy Vernor Vinge's Zones of Thought series. The recurring protagonist-ish character, Pham Nuwen, decides to become a "programmer archaeologist" - the most lucrative job in the universe and also the job that's considered to maintain the fabric of civilization itself :)

My favorite quote is about unix and time().

http://akkartik.name/post/deepness

For more in this line, check out Newton's Wake, by Ken MacLeod. In that book, there are combat archeologists: people who go places where a singularity blew through, leaving all manner of post-singularity technology lying around, and they have to make sense of it to use them or steal ideas from them, only they have to do this in the midst of armed conflict over ownership of those things.

I think there a strong case of prescience there. It is not unreasonable that we will have systems whose time tracks to the beginning of the computing epoch in 1970 long after we forget what 1970 was (assuming we don't kill ourselves first). Just think of all the layers above that today and add a few thousand years.

You bet there is. There are millions (possibly billions) of dollars a year spent on maintaining outdated systems and equipment.

I have some pretty bad memories of transferring the information from a lot of columnar pads into a database.

(We kept the pads)

It's pretty amazing that something pushing the absolute limit of technology in the 1960s is now something that you could bundle up, throw together as a torrent, and publish for anyone curious enough to look at it.

Not really. Moore's Law over 50 years.

http://observationdeck.io9.com/comparison-of-lunar-module-na...

Too bad other fields haven't progressed. Commercial supersonic flight, for example, would be nice. In 1970, I'm sure they would have predicted 2-3-5 the speed of sound by now.

To deny that Moore's law is amazing is amazingly ignorant.

Huh, didn't realize that's what I did. Are you sure that you didn't just misread what I wrote?

I thought by this point that it was well known, well documented, and well established. So, any technology influenced by it should come with certain expectations that shouldn't surprise anyone reading HN.

Now, try not to be amazed by Moore's law but by saddened by the lack of progress in other fields. There's a lot of work to do.

How is the data "worth billions"? That may have been the original cost, but the original cost isn't the same as the value today.

"Cost billions" would have be correct. Their value is "priceless" as the are clearly very valuable but determining that value is impossible. What is a ride on a time machine worth?

My cynical anti-govt title: "Priceless climate data that cost taxpayers billions found unopened at government climate center" Ha!

I know you're being cynical, but these weren't the originals--they were photographic images of the originals displayed on a TV monitor (!). So though what they received were unopened boxes, they were unopened boxes of poor copies. In addition, the original data wasn't used for climate research, but rather weather monitoring and forecasting. One of the innovative things about rescuing this data was to put this data to a different use than its original purpose.

I say this just to clarify that it is not the case that we launched these satellites and then never bothered to use the data (i.e., in case readers had an image of the Ark of the Covenant being wheeled into a warehouse).

>in case readers had an image of the Ark of the Covenant being wheeled into a warehouse

Damn, you got me.

Yeah. I hate the title. Apparently the editors felt the story's importance wasn't self-evident enough, and what readers would be capable of understanding was a pricetag. In truth, the data is priceless in every sense of that word. Unearth is also strange word choice, considering it was unearthed in a warehouse.

For HN, a briefer version of the subtitle would make a better title IMO.

Linkbait title.

From the end:

"The original cost to American taxpayers to gather the images was in the billions (in today’s dollars). “For a few hundred thousand dollars I could get it all back,” he says."

I do think, though, that as lake99 pointed out there could be TREMENDOUS value in this. Extending our image record a full 17 years is has enormous scientific value. It gave examples of imagery of the Aral Sea, which allows for significant study there, and 17 years more of full polar ice records. All of that has pretty big implications for a number of fields.

It's not really extending it by 17 years though since they don't have data for the full 17 years. They have snapshots from 17 years prior to the start of actual data collection.

I haven't actually poured through the data set linked in the article, but skimming it gives the impression that it includes a variety of images from 1964-70, which seems to me like a lot more than just a snapshot. It even mentions in the article that they were able to see multiple years of polar ice.

That would still be a snapshot. There is no monthly data, or even yearly on the same date by the sounds of things, nor does it backfill up to 1979(when we started actually collecting data)

When I read the title, I did not interpret "worth billions" as referring to the cost of producing them. I took it as its potential value to current science. In fact, the blurb below the title says, "pushing back the modern satellite record of sea ice extent by 17 years". This, indeed, may be worth billions to us now. Imagine having to deduce that data by other means available to us now.

In order to price something, you have to find a buyer who is willing to put up the money. Only the US government could pay billions for it, and, in fact, they already did -- back in the 60s. The price it could fetch now is utterly irrelevant -- it's one of a kind and has no replacement value, because it's irreplaceable. Luckily the government didn't throw it away.

The way it is "worth billions" would be computing the cost to reacquire it which is impossible. It is a stretch of course, and it depends on a strained definition of value.

That said, any reliable data from earlier is always good for filling out our understanding of how the planet has changed over time.

The additional data point really makes obvious how steady the Arctic sea ice was up to somewhere between 1990 and 2000: http://www.tjradcliffe.com/?p=1634 (just a quick replotting and refitting of the data, as the graph in the paper linked from the article doesn't really give a clear sense of how far in the past this new data point is).

A bit more detail here including picture of the "data" (i.e. metal film cans):

http://nsidc.org/monthlyhighlights/2013/04/glimpses-of-sea-i...

Which linked to this paper in the journal "The Cryosphere":

http://www.the-cryosphere.net/7/699/2013/tc-7-699-2013.pdf

16-bit? Did they mean 4-bit?

>16-bit? Did they mean 4-bit?

This batch of recovered Nimbus images was converted from film to digital; there was another [1] where the analog data from magentic tape was digitized.

[1] http://www.moonviews.com/2010/03/dumpster-diving-for-science...

Interesting. I was thrown off by their characterization of the images as "grainy 16-bit grayscale satellite pictures" as the word grainy seems to connote poor image quality, while 16 bits of gray is excellent. Perhaps what they are (confusingly) trying to say is a very good scan of grainy film.

I'm confused by this bit:

"...the scientists had recorded the images on magnetic tape, played it back on a TV monitor, then snapped photographs of the TV monitor. What he had were those images, sporadically placed along rolls of film as long as the wingspan of a Boeing 787."

Can someone elaborate on this?

They did the same thing for the video from the moon. The feed from the lander was displayed on a TV in the control room,and then that TV was being filmed by a camera pointed at it. If I remember correctly it was to do with a different format than used by the US television, and lack of equipment which could covert such material in real time.

Sounds like Kinescope http://en.wikipedia.org/wiki/Kinescope - But instead of capturing the full framerate of the original video like you would for TV, they only captured one frame per visible slice of land to save space

A similar stash of data data is in the libraries of the agencies that did the research for the various over-the-horizon-radars around the world. Australia's DSTO has rooms full of magnetic tapes with decades worth of ionospheric measurements on them.

These data are available from the National Snow and Ice Data Center (NSIDC): http://nsidc.org/data/nimbus/index.html

the reference to dark data is interesting - I didn't know there was still so much data out there that hadn't been digitized! This kind of stuff would be great for a kickstarter campaign though..

Yes, exactly... someone kickstart this man kickstarting this "dark data" organization.

They could easily make large glossy prints of that satellite photo for rewards and make the top tier something like "Fly out to have lunch with the recovery team and get an original film tape (already recovered)".

Google should use this data as an overlay for google maps :) I would love to watch that data on that way. Any google employees reading this? ;)

Reply on news.ycombinator.com