717 Gigapixel Image of Rembrandt's Nightwatch | Hacker News Reader

717 Gigapixel Image of Rembrandt's Nightwatch

hyper-resolution.org

565 pointsgaws134 comments

Seeing Night Watch at the Rijksmuseum a decade or so ago totally changed my view of seeing things in a museum vs. seeing them online. I'm a child of the internet and had this view that seeing it on my screen was good enough, but wow is Night Watch incredible up close and in person. Overwhelming, almost. A totally different experience.

That said, this image is amazing, and lets you see a lot more detail than you can easily manage at the museum.

Our brains just love the tactile. Knowing you are feet from the threads and paint of a master. That you can connect with this long dead artist so closely, in 3 dimensions. It’s very human.

Oil paintings look quite different depending on the angle you view them from too, that's something that I don't think can be recreated without some kind of VR / head tracking

I did early interactive multimedia art history documentaries, with voyages into paintings through voiced over descriptions and closeup montages. It was captivating work, each documentary being 7 hours of material, in 8 languages. Because Philips and Sony were trying to prove CD-ROMs to the consumer back then, 42 years ago, they spent money. We got access to museum archives, worked on some of the first digital scans of a good number of well known pieces, and had celebrities reading the voiced over narratives. We had these weird Sony News Unix workstations with newspaper layout monitors - huge grey scale displays, spoiled me until 4K monitors finally arrived. That was '89-'94, the end of the dawn of all this digital media we take for granted today.

That must have been quite a fun ride! It really boggles my mind how you can get such cheap light high quality monitors now. And now we just waste them displaying these ugly orange web pages and Skittles colored TypeScript code.

I remember hearing about the Sony News Unix workstation, and being dismayed that it didn't run Sun's NeWS window system. ;)

https://en.wikipedia.org/wiki/Sony_NEWS

https://en.wikipedia.org/wiki/NeWS

Sounds really interesting. Is this still available somewhere?

The platform no longer exists, it was "CD-I", a pre-internet hybrid CD-ROM format with extensions for real time streaming. Intimate experience with that early real time streaming got me on the original Sony PlayStation OS team in Tokyo.

Very true! See my previous comment on Paul Debevec's Light Stage and Rouen Revisited, and check out some of the vidoes of Paul Debevec talking about and demonstrating it -- he's great at explaining and showing how it works, and it's useful in many different ways, including AR/VR, video games, movie production, art appreciation, online museums, historical preservation, 3D portraits (they captured Obama!):

https://news.ycombinator.com/item?id=30069100

Scanning and Printing a 3D Portrait of President Barack Obama:

https://vgl.ict.usc.edu/Research/PresidentialPortrait/

I’m a religious person, but we don’t need this religious mumbo-jumbo to explain why oil paintings are better in person. They just provide more experience in person. A photo gives you one angle and flattens the depth. The real object is designed to be viewed from lots of angles. It looks different from close up, as it does from far away. It interacts with the ambient light.

If you walk a couple buildings over from the Night Watch, you get a whole museum dedicated to me droog, Vincent van. And Van Goghs are so highly textured with impasto that they are far more radiant, more life-like, in person.

That said the quality of museum experience is important. If you get rushed through the queue to see the Mona Lisa for fifteen seconds (and it’s pretty small), you’ll probably enjoy a print more.

The Getty museum in California uses natural light in their galleries which was pretty special.

Your right about the mono Lisa. When I went 1 year before the pandemic you couldn’t get close. and it was the only painting in its own room. Luckily that museum has no shortages of good paintings.

There are a couple of VanGoghs at the Rodin museum in France which are pretty amazing and not to busy.

The Getty is a great museum except for the art. Not that it’s bad, it’s just that the collection is not the great artists at their best. If you want a better art experience, head over to the Norton Simon in Pasadena. Simon was an art collector who was rich, while J. Paul Getty was a rich person who collected art, and the difference really shows in the collection quality. Both Getty museums are really good architecturally, and worth a visit.

Good tip on the Louvre, though. The best art is not the most famous, but whatever emotionally connects to you. And most museums have plenty of pieces that can do so.

It isn‘t “religious mumbo jumbo”. I didn’t even mention religion. Information encoded using multiple senses will form different and more lasting memories for most humans. I think you managed to agree with what I wrote and dismiss it at the same time.

In a past life in philosophy, a professor of mine worked in aesthetics, and specifically wrote on the significance of genuineness or authenticity, and its special connection to the sense of touch. I don't remember it well but your remarks reminded me of it. Here is a review which summarizes it: <https://ndpr.nd.edu/reviews/things-in-touch-with-the-past/>

An excerpt: > Touch is often practically impossible for ordinary viewers of many genuine specimens of very valuable things, such as a Gutenberg Bible or Michelangelo's David. Protected as they often are by glass encasings, barriers and surveillance technology, these objects are for the most part currently only sensorily accessible through vision.

> However, Korsmeyer rejoins, the sense of touch is still at work given the viewers' physical closeness to these objects and the subsequent in-principle possibility of touching them. Korsmeyer's connection between touch and genuineness explicitly resonates with recent claims about the role of contagion in experiences of the authentic. Cognitive scientists Paul Bloom and George Newman, among others, have argued that our valuing authenticity in objects is explained by our implicitly accepting a magical law of contagion (see, e.g., Newman and Bloom 2012). According to such a law, desirable or odious qualities can be transmitted by contact. So, for example, the value that we attribute to the original David kept in the Gallery of the Academy of Florence (and that we do not attribute to the 20th century copy currently visible in the Piazza della Signoria) is explained by the possibility of touching an object that Michelangelo himself touched. Korsmeyer refers to this feature of our implicit reasoning patterns as 'the transitivity of touch'.

I had the same experience with the Mona Lisa too. Even going in not expecting to get much out of the experience, it really popped, even on the other side of a crowded room.

I wonder if it has a bit to do with it being 3D too, where the edges of paint strokes might catch the light a bit differently than under a scanner.

The Mona Lisa was the most disappointing art work I've ever seen. Size, color, everything. And I was lucky enough to see it up close unlike the arrangement now.

So very true.

Particularly true about the Sistine Chapel. This virtual view is outstanding, but can't possibly come anything close to seeing it in person. https://www.vatican.va/various/cappelle/sistina_vr/index.htm...

I'm going to be the contrarian: Unless you're getting an exclusive viewing and you wake fresh with a singular goal, seeing the great art of the world is likely to be a compromised experience. When overwhelming beauty and art surrounds you, and a hectic schedule dogs you, everything is dulled and the experience becomes almost a fog. My experience going through the Sistine Chapel was almost an aside to just days of overwhelming artistry, so I gave it barely the respect it deserved. I've enjoyed and experienced it virtually magnitudes more.

A couple of years ago I did a trip to Belgium and France. Saw all of the sights, loads of museums, and did a tonne of wandering. It was a great time, but was overwhelming. I came home and maybe a week later we were browsing YouTube on the TV and came across a channel that just walks around neighbourhoods of Japan (e.g. Shibuya, Tokyo, among others. The channel is Virtual Japan). A couple of hours of walking a stabilized camera through the streets of a Japanese city. My son came in and watched with me while we looked at storefronts, read restaurant menus, walked through malls, virtually participated in pedestrian scrambles, etc). The weirdest thing is that days later my "trip" to Japan felt much more real than my trip to Belgium and France (or any prior trip I'd ever taken). Absent all of the worries and hustle and overwhelming inputs, somehow this completely not real experience felt much more real, and to this day I feel like I've been to Japan, while so many other countries that I've physically been to and experienced for weeks seem like almost a dream. It really was a fascinating experience for me.

It made me wonder if there is a business in on-demand telepresence for this sort of virtual travel. "Uber" someone technologically enabled to walk around an area, look at things you want and follow directions. Add some dystopian elements to it and soon they're getting in fights at your request.

It sounds like you've discovered the secret to travel. You don't have to see all the sights or cram everything into each day. Usually you get more out of it if you take things slowly, be intentional about what you really want to do, and savor each experience. With two small kids now, we usually pick one city per trip and plan (at most) one morning activity and one afternoon/evening activity. Sometimes it's one activity per day, sometimes it's none (maybe we just meander around a neighborhood between meals, like your Virtual Japan experience). Like you said, if you see 10 different Roman churches in one day, the Sistine Chapel becomes just another fresco.

I realize my comment gave perhaps an inaccurate impression, but even with an incredibly sparse itinerary, and bounteous amounts of time (I spent weeks in Italy. I went through Belgium and France over two weeks. I spent a day just wandering around Paris with no itinerary, in addition to days more doing a variety of things), being completely inundated with new and unknown -- even the most benign door or building -- added to staying in a strange places, eating at strange places, and being detached from your normal life, for me at least made the experience far less...persistent.

The Sistine Chapel experience, for instance, aside from being just a mass of people being shuffled through one of the largest tourist draws in the world, was on a day that started with incidentally seeing Pope JP2 give an address, and ended with seeing the Dalai Lama at Tivoli when going there for a dinner (two spiritual leaders in one day! Yet I remain an agnostic). In the end the Chapel got filed away as "neat some stuff painted on a ceiling". That is an extreme example, but for my "a 100 year old is historic" North American sensibilities, virtually everything in places like Belgium, France and Italy is overwhelming, from the weird little waffle shop in Ypres to the sound of bells, the stones on the street, etc, everything just becomes an onslaught of overwhelming experience.

I spent 3 days in Venice; you can't even scratch the surface in 3 days, but I'd say 3 days is about enough for one visit. If you want to see more, book another visit.

Quite a lot of fine art in Venice isn't in crowded museums; some of it's in little churches, away from the crowds, and some is in smaller private museums, away from the day-tripper trail.

I've never been to Florence. I hear that's overwhelming.

>I've never been to Florence. I hear that's overwhelming.

JFYI, Stendhal Syndrome (the term was born in Florence, by a psychoilogist that observed cases among tourists):

https://www.medicalnewstoday.com/articles/stendhal-syndrome-...

Not really-really proved to be an actual illness, and not common, still ...

I think you made some really interesting points and did a great job of describing that experience. And I do agree with you about the sort of overwhelming experience that results in a kind of fog. Before I took my family to Europe in 2017, my wife and I planned our trip meticulously. Her instinct was to see and do as much as we possibly could. I relentlessly pulled in the other direction. I argued that choosing fewer things to see and having "down time" was essential. If you try to cram as many experiences as possible into a visit, you get overloaded and burned out and none of them will be as meaningful as they should.

Having said that, before visiting the Sistine Chapel, I had indeed studied the imagery on my computer, poring over many of the details ahead of time. I knew that giving my brain some time to ingest, digest, and comprehend the enormity of what I was about to see would be beneficial. And I suppose in much the same way you studied the streets of a Japanese city, I had a similar experience with the Sistine Chapel.

And perhaps here's my somewhat contrarian point to your contrarian point... :-) Seeing it in person lit up the right side of my brain whereas the virtual tour on the computer was more of a left-brained experience. Standing in the same room Michaelangelo stood (and lay on his back) for four years was indescribable. The light that bounced off that ceiling and into my eyes was different than the light emitted from my computer's LCD monitor. Visually exploring it with my 12-year-old daughter and sharing it through her eyes was great. It was a fantastic experience, one I will always treasure. I hope to see it again someday.

Although I wasn't rushing I never really thought about the Sistine Chapel. I’m reading this like “I’ve been there” and I am generally fascinated by microstates but the significance of this isn't nearly as high as you all are putting it, to me.

Makes me think I should look it up more, because I absolutely did not.

Some parts were crowded, other parts were not, I enjoyed the art but hmmm. To me the Swiss Guard tradition was more interesting and fascinating, and some of the graves had fascinating designs on them.

Dating people from other developed nations than the US forced me to never do a rushed vacation.

They have weeks off of work and expect you to as well. They don't save up for 5 day trips across 3 day weekends to rush rush rush. They* just dip out and live in the different place for a while, take classes, get to know locals, etc. (*not everywhere has this privilege, but it is very common)

I’m never going back to the other way I just hang out with richer Americans. More people have been doing something equivalent over the last year, they're usually also richer Americans just still career focused as well, compared to trust funders.

> not everywhere has this privilege, but it is very common

Lol this is not very common at all. You thinking that's common tells me you don't actually know how normal people from European countries actually live. Might be helpful to actually meet these "locals" that you talk about.

I just didn’t want to write an essay about who does what by country and class

I’m just glad I was exposed to it and was inspired and quickly able to adapt

What you refer to is people relocating for work or study. It happens but it's uncommon.

Much more common is boarding a low cost flight on Friday night or Saturday morning, flying between 60 to 120 minutes and coming back on Sunday night. Add to that a two/three weeks vacation once per year.

I’m referring to statutory 5 weeks of vacation actually. Narrowing it down to developed nations with that feature.

I realize that I didn't explicitly state that I was writing about my experience in Europe, as a European living there. The data points are me and my friends of multiple countries but mainly Italians.

There are (working) people who move to another place for many weeks but the vast majority of us does weekend tourism and 2 or 3 weeks vacations. Companies are not particularly keen to let one person go for 4 o 5 weeks. My personal experience when I was not self employed is that they don't like even 3 weeks in a single stint. The two longest vacations I had were one month long after I went self employed (hi Australia!)

I expect that working from home will make staying abroad for a long term more common, but it costs more money that staying at home (you're probably still paying a rent or mortgage) and it's usually not for families.

I know this will get booed but I found the Sistine Chapel underwhelming. Part of it has to do with, like you mentioned, being overwhelmed or maybe touristed out. The previous year I'd spent 4 months in Europe and after the 3rd or 4th city I just got tired of churches and museums. While it was over a year later, when I made it to Rome, the Sistine Chapel just didn't push any buttons for me.

I'm not saying anyone should agree with me. Only that I don't find it shocking if some people say the Sistine Chapel didn't trigger any strong lasting memories.

Same, a few days ago I chanced upon a photo I took of it (you're not supposed to take photos) and I was surprised I didn't remember it basically at all. I do clearly remember being in its fairly small room with a billion other people, though, and only having X minutes to look at it before the next group has to come through.

I also remember being disgusted at the vast, obscene wealth that the Vatican has amassed and regretting giving them more money.

I visited Paris for a few weeks a couple of years ago. The way I did it, I had a rough schedule that was my fallback for the day. In the morning I'd usually hit one thing but otherwise spent my time mostly wandering around.

Didn't even go up the Eiffel tower because I didn't feel like waiting in line.

I enjoyed travelling this way. But I admit it's probably easier without a family to accommodate.

I disagree. If you have ever been there you're getting bumped by smelly tourists left and right while someone spills water on your foot and another person tries to shoo you out as soon as possible.

Mona Lisa? Good luck even looking at it if you're short, unless you want to be stuck with your face in someone's armpit while they are taking a selfie.

Yes, my first visit to the Met in NYC really opened my eyes to how impressive these works are in person and how much a photo on a screen does not convey. Not just the masters, but some of the portraits of people I've never heard of and by relatively unknown artists in styles I didn't particularly appreciate.

Same, in a similar note I’ve been a big fan of Alphonse Mucha and Art Nouveau all my life and I had no idea that he used gold paint!

Having seen so many jpegs and pictures in books of his works, it did not convey or describe this at all!

I passed one in a gallery one time (not looking for one) and the sheen from the lights reflecting totally changed everything in a more amazing way!

An RGB value per pixel is not all of the information you need to render a painting accurately! To capture the effect you describe, paintings should be scanned with something that can capture a BDRF. Then viewing them in VR could give a very accurate experience.

i had to google BDRF: Bidirectional reflectance distribution function

https://en.wikipedia.org/wiki/Bidirectional_reflectance_dist...

Shouldn't that include the color spectrum, and perhaps the polarization of the light?

Yes, if you want to go all the way. There are many simplified formulations of BDRFs that are commonly used that ignore polarization and even wavelength dependent effects (e.g. metallic/roughness used in physically based rendering for games which has only two parameters on top of the base color), but they would still be a huge improvement over not capturing any BDRF data at all.

Gold is tricky! ;) The paper I linked to in another comment (about Paul Debevec's Light Stage) describes automatically identifying and digitizing "The Golden Haggadah"'s gold foil:

https://news.ycombinator.com/item?id=30069100

https://www.osapublishing.org/oe/fulltext.cfm?uri=oe-27-8-11...

>Multifocus HDR VIS/NIR hyperspectral imaging and its application to works of art

>[...] To demonstrate the potential interest of this processing strategy for on-site analysis of artworks, we applied it to the study of a vintage copy of the famous painting “Transfiguration” by Raphael, as well as a facsimile of “The Golden Haggadah” from the British Library of London. The second piece has been studied for the identification of highly reflective gold-foil covered areas. [...]

>[...] The sharpness index, as well as the color and spectral metrics show that it is possible to achieve good quality spectral reflectance images using a hyperspectral scanner in non-controlled illumination conditions. Moreover, as an example application, highly reflective golden material has been segmented from a facsimile. [...]

>4.3. Additional example of application: identification of golden foil in a facsimile As a final experiment, an original facsimile from the British Library of London has been captured using the proposed framework. This facsimile (see Fig. 8), presents areas of golden highly reflective material. These kinds of materials always represent a challenge for image capturing systems. The problem is that depending on the illumination/observation geometry, the capturing device may receive specular reflections from the sample. If this happens, these areas would most probably saturate the sensor when using the exposure times needed to correctly capture the rest of the scene. Even if the illumination could be controlled, if the samples are not perfectly flat (as is the case in many artworks and illuminated manuscripts which have irregular texture), avoiding the saturated areas would not be possible by only manipulating the light sources. Therefore, even in controlled illumination laboratory conditions, the high dynamic range may still be a problem for this kind of samples. As in previous sections with the art painting, two spectral reflectance cubes of the facsimile were captured (one LDR and one HDR). These cubes were used for the automatic detection of those areas of the facsimile containing the highly reflective golden material. For such a purpose, the best results were found using the spectral metric goodness of fit coefficient (GFC) in both cases, in the near infrared spectral range from 700 to 1000 nm. The segmentations of the HDR and LDR cubes were compared with a manually segmented ground truth (shown in 8 center of bottom row). Since there are only two possible labels for each pixel (0 for non-golden material and 1 for golden material), the performance was compared as the percentage of matches between the automatic segmentations and the ground truth. [...]

>5. Conclusions and future work: In this study, a complete framework is introduced for the hyperspectral reflectance capture of a painting in situ, and under high dynamic range conditions. Both the high dynamic range and the focusing problem due to chromatic aberrations have been overcome by using multiple captures with different focus positions and exposure times. A final hyperspectral reflectance cube has been computed using weighting maps calculated for both sample and flat fields and the quality of this cube has been tested and compared with a spectral cube captured in the usual LDR and single focus way. Our results show that the proposed method outperforms the best low dynamic range capture acquired. The sharpness index, as well as the color and spectral metrics show that it is possible to achieve good quality spectral reflectance images using a hyperspectral scanner in non-controlled illumination conditions. Moreover, as an example application, highly reflective golden material has been segmented from a facsimile. Our results show that by applying the proposed framework for capturing and processing, those areas which saturate the sensor in the usual capturingway, can be correctly exposed and segmented using the HDR multifocus capture. In future research, a new version of this framework will be developed including piecewise cube stitching for blending different cubes captured in different regions of big paintings. This will allow us to get closer to the painting and retrieve higher spatial resolution data, whilst still maintaining the spectral resolution and performance achieved in this study. Moreover, we will use the spectral reflectance images computed in this study, together with X-ray fluorescence measurements for the non-invasive pigment identification, in order to help the dating of ancient paintings and other works of art.

Never been to Amsterdam, but I felt the same way at other major art museums. Although I think the best aspect of going to a museum is not necessarily seeing the most popular painting. For instance, when I went to the Louvre, which --- and I say this as a person who is not a huge fan of tourist traps --- is one of the best things to visit in Paris, of course there are the most famous works (e.g., the Mona Lisa), which are good (don't get me wrong on this), but 90% of the stuff in the museum is not really being looked at closely by everyone. One of the best things about a really big museum is just going and looking at art that you never knew about before. Just taking a closer look at pieces and getting a new new favorites. I took a few notes when I was visiting!

Side note on going to the Louvre. I went last summer and, thanks to Covid, had to reserve my (free --- thanks EU!) spot (which I did only a few hours before). As a result, getting into the museum was very quick. Surely, this had to do with the reduced number of people in Paris, too, but I now think that entrance by reservation is a pretty good system generally for such popular spots.

Prado in Madrid is an interesting place to visit too. Most of the religious art didn't resonate with me, some other pieces more, but Goya - Goya was from another world. I am not sure how many talents comparable to Goya ever walked the Earth.

The Louvre is one of the few places where I actually felt like I could get lost in a building. It simply goes on forever. The sheer amount of works, objects, and information, presented to a visitor is astounding and hypnotic. For modern paintings, I like the Musee d'Orsay slightly more; but for sheer art history the Louvre is truly one of the greatest institutions in existence and a must visit for anyone.

They will recommend that you plan on not "seeing everything" in a lot of guides. Believe that. Making some choices for things you want to see beforehand will cut down on the exhaustion that often comes over you after you end up in a room filled with dazzling tea cups from 16th century France and yet you're just desperately trying to figure out how to get out. It happens to the best of us.

Paul Debevec created the Light Stage to capture high dynamic range reflectance fields (including clean high resolution normal/bump/gloss/texture maps) of human faces.

It uses hundreds polarized LED lights and cameras, plus lots of image processing, to separate the lighting effects of specular reflectance (glossy shine) from subsurface scattering (glowing skin), so you can reconstruct the 3D image and relight it under different conditions, environments, and viewing angles.

https://en.wikipedia.org/wiki/Paul_Debevec

https://www.pauldebevec.com/

https://en.wikipedia.org/wiki/Light_stage

"The Light Stage With Paul Debevec" - 360 Video (captured with JauntVR panoramic camera):

https://www.youtube.com/watch?v=xujwI4dimDA

Digitizing Photorealistic Humans Inside USC's Light Stage:

https://www.youtube.com/watch?v=c6QJT5CXl3o

Paul Debevec: Light Fields, Light Stages, and the Future of Virtual Production:

https://www.youtube.com/watch?v=bAe2dUJxe3w

A Light Stage was featured in the 2013 film "The Congress", which is a 2013 film adaptation of Stanislaw Lem's book, "The Futurological Congress", directed by Ari Folman:

https://en.wikipedia.org/wiki/The_Congress_(2013_film)

https://en.wikipedia.org/wiki/The_Futurological_Congress

I really love that movie and the book it was based on, which both raised some interesting issues: Like Blade Runner's relationship to Do Androids Dream of Electric Sheep, it was a lot different than the book, but shares some deep ideas, and stands on its own as a great movie.

The Congress Official Trailer (2014) Robin Wright, Jon Hamm HD:

https://www.youtube.com/watch?v=1rNSTizOsws

The scan scene in the Light Stage at USC ICT's motion capture studio was emotionally riveting and technically realistic, with Robin Wright playing a partially fictionalized version of herself, with Harvey Keitel playing her agent, baring her face and soul to the sparkling panoptic all encompassing emotion capturing machine.

The Congress (2013) Scan Scene:

https://www.youtube.com/watch?v=pPAl5GwvdY8

"Rouen Revisited" is an earlier interactive kiosk project that Paul Debevec and Golan Levin created in 1996 at Interval Research Corporation, based on photogrammetric modeling techniques he developed at UCB:

https://www.youtube.com/watch?v=Ao3kf0YQ31c

https://acg.media.mit.edu/people/golan/rouen/

>Between 1892 and 1894, the French Impressionist Claude Monet produced nearly 30 oil paintings of the main façade of the Rouen Cathedral in Normandy. Fascinated by the play of light and atmosphere over the Gothic church, Monet systematically painted the cathedral at different times of day, from slightly different angles, and in varied weather conditions. Each painting, quickly executed, offers a glimpse into a narrow slice of time and mood. We are interested in widening these slices, extending and connecting the dots occupied by Monet's paintings in the multidimensional space of turn-of-the-century Rouen. In Rouen Revisited, we present an interactive kiosk in which users are invited to explore the façade of the Rouen Cathedral, as Monet might have painted it, from any angle, time of day, and degree of atmospheric haze. Users can contrast these re-rendered paintings with similar views synthesized from century-old archival photographs, as well as from recent photographs that reveal the scars of a century of weathering and war.

>Rouen Revisited is our homage to the hundredth anniversary of Monet's cathedral paintings. Like Monet's series, our installation is a constellation of impressions, a document of moments and percepts played out over space and time. In our homage, we extend the scope of Monet's study to where he could not go, bringing forth his object of fascination from a hundred feet in the air and across a hundred years of history.

Here's a paper about "Multifocus HDR VIS/NIR hyperspectral imaging and its application to works of art" that references his work, about how you can capture the 3D texture and hyperspectral reflectance field of artwork in a way that you could dynamically relight in different conditions and environments, interactively view in VR, use in high quality computer games and renderings, etc:

https://www.osapublishing.org/oe/fulltext.cfm?uri=oe-27-8-11...

>Multifocus HDR VIS/NIR hyperspectral imaging and its application to works of art

>Abstract: This paper presents a complete framework for capturing and processing hyperspectral reflectance images of artworks in situ, using a hyperspectral line scanner. These capturing systems are commonly used in laboratory conditions synchronized with scanning stages specifically designed for planar surfaces. However, when the intended application domain does not allow for image capture in these controlled conditions, achieving useful spectral reflectance image data can be a very challenging task (due to uncontrolled illumination, high-dynamic range (HDR) conditions in the scene, and the influence of chromatic aberration on the image quality, among other factors). We show, for the first time, all the necessary steps in the image capturing and post-processing in order to obtain high-quality HDR-based reflectance in the visible and near infrared, directly from the data captured by using a hyperspectral line scanner coupled to a rotating tripod. Our results show that the proposed method outperforms the normal capturing process in terms of dynamic range, color and spectral accuracy. To demonstrate the potential interest of this processing strategy for on-site analysis of artworks, we applied it to the study of a vintage copy of the famous painting “Transfiguration” by Raphael, as well as a facsimile of “The Golden Haggadah” from the British Library of London. The second piece has been studied for the identification of highly reflective gold-foil covered areas.

[...]

https://en.wikipedia.org/wiki/Hyperspectral_imaging

I noticed this most vividly when seeing Van Gogh's Sunflowers (the 4th, from the National Gallery in London). A high-res scan such as on Wikipedia [1] does no justice to just how textured the painting is. He's clearly applied quite thick strokes and, under proper lighting, the texture really pops in a way that screens simply cannot reproduce. It brings an entirely new dimension (pun intended) to the painting and really changes the whole expeirence.

Proper, museum/gallery lighting really transforms art in a way that even prosumer screens simply cannot replicate, because you can move around the piece and the interplay of light against it does so much.

[1]: https://en.wikipedia.org/wiki/Sunflowers_(Van_Gogh_series)

I'd love to see a dynamic raytracing of a high resolution 3d scan of this picture or another masterpiece. I doubt nobody has attempted to do this before.

Many of Van Gogh's, when seen in person, are almost more like relief sculptures because of the incredible depth of his strokes. I loved going right to the sides of the frames and looking across the paintings to see how each little valley in his strokes caught the light and cast tiny shadows. I think it's also what adds a lot to the kaleidoscopic feel of his works, alongside his color palettes and stroke work; the extra dimension makes his paintings vibrate with restrained energy in interplay with the surrounding light.

My daughter and I went to an art supply store in December and we were surprised at the variety of paint bases. I had no idea. Many of them would be mistaken for sculptural material, almost like play doh or some like that, if you didn't know what they were. I had no idea.

The Neue Pinakothek in Munich had a sunflower painting and I noticed the same thing. The textures really pop and there’s so much paint it is protruding from the canvas.

Extremely relevant to this is the excellent program Ways of Seeing by John Berger:

https://youtu.be/0pDE4VX_9Kk

I had exactly the same experience when I saw Caravaggio's "The Taking of Christ" in person recently. I was familiar with the image, but I was blown away by how incredible the painting really is.

Agreed - photos diminish it - nothing can capture the intensity of the original.

I think part of that is the architecture and environment of the museum. They spend tons of attention to the high ceilings, sense of awe (a bit like a cathedral), and that can make all the difference compared to viewing something in your bedroom in the dark.

Is there any distortion/difference seeing the painting in real-life at an off-angle that cannot be replicated through current scans?

And are there any neurological studies about seeing/feeling physical objects versus virtual? And would VR help close the gap?

Me and the wife visited the RM in Nov 2019. The painting is rather large, so 717 Giga pixels doesn't go as far as you would think but you get quite close to the textures that you would see close up.

Whilst you are there, please note there is rather a lot more to see than the one picture. The architecture is worth noting too. My overriding impression of the piccies on display is: "sumptuous". That's probably not in the fine arts's lexicon of superlatives but my personal experience.

The model boaty exhibition is not as jarring as it might sound, alongside some very serious artwork exhibitions and is great fun. It doesn't seem to really belong there at first blush but I do understand that naval prowess is rather important to the Netherlander psyche, so perhaps that legitimises it. It does give you a bit of relief after having rather a lot of important art thrown at you.

All in all, I cannot recommend the Rijks' highly enough. If you go anywhere near the Low Countries, then slap it on your itinerary. The coffee is quite reasonable in the entrance cafe, too.

I missed the RM by two days. We traveled to Amsterdam in March 2020, saw the Van Gogh museum, went to Barcelona for a few days and were planning on going to RM on our return leg. Except when we got back to Amsterdam, everything was shutting down and all we could do was eat and walk around.

> The painting is rather large, so 717 Giga pixels doesn't go as far as you would think

A billion is rather large, too. The pixels are 5 micrometers (1/200th of a millimeter). That is a lot smaller than the naked eye can see (about 1/10th of a millimeter)

Apparently at some point in the past three pieces were removed from the sides of the painting.

> That’s why we’ve had so-called artificial neural networks working overtime on reconstructing the original appearance of the painting. Hour upon hour, the computers have been whirring away, recreating these missing pieces on the basis of a 17th-century copy.

Interactive comparison of the missing pieces:

https://www.rijksmuseum.nl/en/stories/operation-night-watch/...

Zoomable image with the missing pieces added back in:

https://i.micr.io/jfYIf/en/nw-met-missende-stukken-clean

I seem to remember they were removed to fit through the door!

I knew from this image that I have sadly not yet been able to see in person.

There is a song by artist "Ayreon" called "The Shooting Company of Captain Frans B. Cocq" [1].

It is part of a concept album that describes the first person experience of the last human on mars that experiences artificial "dreams" induced by a machine traveling through history.

To quote Wikipedia [2] regarding the title of the painting as context:

> Militia Company of District II under the Command of Captain Frans Banninck Cocq, also known as The Shooting Company of Frans Banning Cocq and Willem van Ruytenburch, but commonly referred to as The Night Watch (Dutch: De Nachtwacht), is a 1642 painting by Rembrandt van Rijn.

[1]: https://youtu.be/EApiD5JQAH0 [2]: https://en.wikipedia.org/wiki/The_Night_Watch

Wow, thanks for that. It’s so cool to find out about more modern proggy space rock like this.

I guess this is a better place than any to mention the King Crimson song "The Night Watch", which remarks on the painting and its imagined context.

https://www.youtube.com/watch?v=-qCXHghedxw

Glad I'm not alone in loving that song and those 2 albums (Universal Migrator). Absolutely amazing.

That song in particular really captures the feel of walking around old Amsterdam for me, going to see the painting, and having the Dutch Golden Age come to life. Amsterdam is one of my favourite cities, and Ayreon has a lot to do with that.

I am astonished Ayreon is not a lot more mainstream popular.

What's a good way of showing extremely high resolution images like this?

I did a few absolutely spectacular renders some years ago of a buddhabrot fractal, I don't remember the resolution but probably at least 100k x 100k, but after weeks of rendering I couldn't show it to anyone. I could only view it piecewise myself, as it didn't fit in RAM.

It's been a bit of a white whale for me.

I also have a high-res animated 4D rotation render that's half a dozen Gb, but can't be streamed online because compression algorithms absolutely massacre the details, and nobody wants to download files that big.

I did a bit of looking around at 4k matrix hardware, and it's pricey. There's probably someone making 8k by now, so it's just a matter of paying the high cost. Or get 1:16 1080p or something.

Other than that, 4k HDR TVs are cheap. Obviously 4k is too few, but at least you can make it 75" for under $1000...

As far as animations go, ffmpeg can work miracles.

Apart from that needing to be a matrix of over 1200 4k TVs or over 300 8k TVs the problem here is opening and viewing the image not displaying it in full all at once. ffmpeg would run into the same RAM issues, the only approach to viewing such images is to split the image up into separate pieces and make multiple scale levels then feed it into a viewer that understands which pieces at which scale to load.

Ipad has a nice high red and bright screen I suppose.

I don't think you quite understand the resolution of these images. If you were to arrange a 25x25 grid of 4K displays, perhaps. But that's like the size of an apartment building. And the whole point is that there is mesmerizing detail even close up. Zooming out doesn't do it justice.

There's https://en.wikipedia.org/wiki/Deep_Zoom for example.

> A DZI has two parts: a DZI file (with either a .dzi or .xml extension) and a subdirectory of image folders. Each folder in the image subdirectory is labeled with its level of resolution. Higher numbers correspond to a higher resolution level;[6] inside each folder are the image tiles corresponding to that level of resolution, numbered consecutively in columns from top left to bottom right.

Yeah, something like that is what I've been looking for.

This is what this site uses: https://openseadragon.github.io/

Do they always take the image from straight on, front? Is the camera panned around with a narrow FOV so that it doesn't see portions of the canvas from an angle? Does the light source move with the camera?

Or, I'm thinking, it's often really interesting to see the 3D texture of a painting -- is that ever something desired to be recorded? For example, with impressionist / pointillist paintings I've seen in person, looking at the canvas from a very low angle is even more interesting to be able to see the brush strokes than seeing it directly face-on.

No idea about these images, but in reproduction work it's not uncommon to work with the film plane shifted so that the camera sees the work piece from an angle (but still a flat perspective) to avoid reflections that otherwise be seen flat on.

I took a geology class in college where we studied a lot of high res photos of both individual rocks and larger rock formations. They used some sort of moving bed and a microscope for the rocks so you could get really close up while also seeing larger features. The formations we're photographed using a GigaPan mount that would allow you to attach a camera with a telephoto lens and it would step in both X and Y axis to create hundreds of images it would later stitch into a composite.

This explains all the technical details - it is far more advanced than you would imagine:

https://www.youtube.com/watch?v=z_hm5oX7ZlE

The implication is that the image tiles are overlapped, so the edge of one tile falls near the center of another. That might provide enough parallax information to do a 3D reconstruction. I wonder if the raw images and metadata are available?

It might be enough. Would be a hard algorithm to code though, don't you think?

Do you think they should have just used stereo cameras in the first place?

Wow, that's a lot of work! And yet it seems a shame to not capture more information than a single RGB color value per pixel. I can think of three other types of information you could capture: 3D information (which they did capture but only at a coarse level), the BDRF, and hyperspectral information. Of these the BDRF seems most important for viewing of paintings, and I'm surprised that there's no consideration of that given the thoroughness of the rest of the project.

With the BDRF you would be able to show the painting as it would appear under any lighting condition and any viewing angle. This would make the painting appear much more realistic in a VR rendering, for example. The current stitched photo assumes uniform lighting and a straight-on view, and if you simply mapped it to a texture on a plane in a 3D environment it would look flat and lifeless.

Is the viewer open source? I saw links to give credit to various open source libraries/utils but I didn't see a link to the actual system.

It works really well. I'd love to use this for one of my projects.

It's OpenSeaDragon plus addons.

looks to be using open sea dragon [1] This is just a tiled map so any web mapping software would also work OpenLayers, Leaflet, Maplibre etc.

[1] https://openseadragon.github.io/

Interesting, I've used leaflet a lot in the past (love it!) for GIS but never thought about tiling a more generic image to it. Good call.

Indeed! The pinch-zooming on mobile is so smooth I for a second thought there were no library involved at all.

Some interesting facts from Rijksmuseum's abstract of a talk from 2018 - not related to the painting of this post:

The stitched and registered images, each exceeding 6 gigapixels, were then visualized using the “curtain viewer”, an internet-based image viewing technology developed by Erdmann for the Bosch research and conservation project.

For this high resolution photography the museum faced several challenges. For the overall images of these large paintings, a total of 242 images were required. This amounts to approximately 70 Gb per painting per imaging modality. The maximum storage capacity of the Rijksmuseum’s Digital Asset Management (DAM) software is currently only 2 Gb per file.

It was also an enormous undertaking to stitch such a large number of composite images and register them for use in the curtain viewers, enabling the conservators to fluidly switch from an overall image to the micro level and back using only the mouse wheel.

https://aics46thannualmeeting2018.sched.com/event/Cz5i/imagi...

Very interesting to zoom all the way in and note that some of the pigments had a tendency to crack as they dried, and others did not.

I imagine the paints were selected intentionally with this in mind, as the cracking where it exists seems to enhance the painting more than take away (dark background, hats, etc) while lighter colors that should not be cracked, are indeed less so (skin, armor).

Sounds a bit like reading too far into it without actual evidence this was the intent. Could just be the way the layers were done, could be accidental, could be an artifact of the way it has been handled over the years, could be an artifact of the capture. Could be intentional sure but to jump to "the pigments were designed to crack a certain way so the painting looks like it does nearly 500 years later" seems to be a product of being told it's a great painting by a great artist so everything about it therefore must be great than the cracking actually being great.

Could reddit's r/AskHistorians sort this out? I've never heard anything like it, but there really doesn't seem to be a lot of cracking that takes away from the effect.

It’s called craquelure. (Forgive my short explanation. I have to get up in 4 hours.) In addition to applying thick layers of paint Rembrandt used varnish. All of this dries at different times, which leads to the cracking effect. It’s more a condition of how it was painted and the media the artists used and less about trying to achieve the effect outright.

Source: I have a phd in art history, and publish and teach in this field

Fun fact: the Night Watch is an inaccurate title. The title is actually Militia Company of District II under the Command of Captain Frans Banninck Cocq or The Shooting Company of Frans Banning Cocq and Willem van Ruytenburch. It’s known as the Night Watch because of dirt and varnish—the varnish yellows over time. This changed after extensive restoration. Also, the painting has been vandalized several times. In one unfortunate event in the ‘70s a person took a knife to it.

For more on craquelure see: https://en.m.wikipedia.org/wiki/Craquelure.

(Edited to clarify that varnish yellows over time)

It’s more likely that the darker pigment contributed to the cracking rather than the other way around or whatever.

lots of restoration

Another interesting aspect mentioned in an earlier (dutch) article: the original painting had far more blue tones, which would contribute to a color temperature much closer to daylight (5600K), instead of the current yellowish tone.

For some reason, the blues degrade, I don't know why.

Earlier discussion: https://news.ycombinator.com/item?id=29778166

In 2001 I flew across the world to see this painting. Got lost in the museum trying to find it and came in through a side door. Upon seeing it, I needed a seat, there conveniently was one.

The in person experience was so different than seeing this online because of the scale and detail.

The 'z' key allows you to zoom to an arbitrary box, including magnifications that are not possible with scrolling. 'u' provides a URL for direct linking to a region. More keys are in the scripts, but these seem to be the useful ones.

Similar view on the Ghent Altar Piece by Van Eyck: http://closertovaneyck.kikirpa.be/ghentaltarpiece/#home

Does anyone know how big the image would be in file size? This website downloads everything in tiles so its hard to tell.

Uncompressed 717 gigapixels x 24 bits / 8 = 2.151TB

5.6TB

You can read up on all the details on the Rijksmuseum's site dedicated to the nightwatch. https://www.rijksmuseum.nl/en/whats-on/exhibitions/operation...

Interesting... this implies 64 bits per pixel. What format would that be, 16 bits per color RGBA?

Maybe that size includes the redundancy with all tiling layers of different resolution?

edit: reading the link in the GP closer, they seem to refer to the full imagine, but it looks like they extracted the drawing underneath the painting. So I'd guess something must be from outside the visible spectrum.

In this video [1] he mentions that each 100mp image is around 600MB, which aligns with the size of the 4-shot multi-shot mode of the H6D-400C MS. 8439 photos 579MB would be around 4.7TB in raw format.

[1] https://www.youtube.com/watch?v=z_hm5oX7ZlE [2] https://www.hasselblad.com/h-system/h6d-400c-multi-shot/

Another document that is cool to zoom in on (not nearly as deep as this) is the great Isaiah scroll:

http://dss.collections.imj.org.il/isaiah

Just seeing something created so long ago in detail is moving.

I'm just noticing the shadow of the hand on the jacket of the man holding the spear. What an interesting detail to add! I don't know if anyone would have noticed that not being there. I also notice the perfect use of color on the blue tassel of the spear- how did he get the shade of paint in the light and dark parts of the tassel so exactly correct? It's almost like this was drawn from a photograph- but that can't be. I wonder if he had live models standing in to draw from?

Also, I notice that the boots they are wearing actually look very practical- they're shoes when you need shoes, and long boots when you need boots. What a good idea.

Have you found Waldo ? (Just kidding ;) )

What I'd like to have next is a way to change the viewing direction and the lighting dynamically.

Paint color is not like a screen pixel. It changes depending on the light color, and direction.

Nightwatch maybe was intended to be watched lit by a dancing flame.

I am very hopeful for new rendering techniques like Nerfies, for this type of application.

It also has the potential to reverse the effect time had on the painting, so we could experience this it like it was painted.

Wow, that is an amazing resolution. It would be interesting to create such image every year, and see the changes over time (if any).

wow Rembrandt hid a map of the matterhorn and it's surroundings in the left eye... amazing what these cameras can find!

Which person?

I wanna see the street view here :) - https://hyper-resolution.org/view.html?pointer=0.393,0.002&r...

Such good detail! Wow!

Really beautiful. It's impressive how seemless the experience is considering it's constantly fetching jpegs.

Hyper resolutions are nice but for paintings especially the interesting stuff is in 3d - the paint strokes, layers of paint etc. I remeber seeing cool 3d recreation of painting being used in Fake or Fortune but can't remember the name of the startup that did it.

Why not link to the original?

https://www.rijksmuseum.nl/en/stories/operation-night-watch/...

The linked website is owned by the scientist that worked on the project and works for the museum

The "little white pixels" in each corner appear to be hand-drawn registration marks. Curious - all the precision of the image but these hand-drawn tags seem quite out-of-place.

Is it possible to download the actual image file? I'm aware that most conventional software will probably choke on it (which is kind of why I want to try it).

if you zoom in you can see the text in the mirror at the top middle, anyone know what it says?

Here's the 44.8 gigapixel image from the museum as comparison : https://www.rijksmuseum.nl/en/stories/operation-night-watch/...

According to wikipedia there are 18 names, and following the source/link it seems they were members of a local militia.

https://intransit.blogs.nytimes.com/2009/03/23/putting-names...

amazing that you can see all the cracks in the paint when zoomed in. Preserving this clearly is of importance, but digitizing allows us a snapshot for comparative restoration. Hope whoever is in charge of restoring this does not end up like the botched restoration of Jesus Fresco in Spain.

Don't worry, it is being taken care of very seriously: https://www.rijksmuseum.nl/en/stories/operation-night-watch

Finally an NFT I would happily pay for

yes sir. This would be an NFT that would be worth paying for!

that's amazing. Although I had to chuckle because five minutes in I basically realized that my experience of looking at the painting resembled this scene: https://youtu.be/QOlPNZzneGw

Anyone know if it's possible to load something like this up on a VR headset, like an Oculus?

3.63 m x 4.37 m in size.

11.9 feet x 14.3 feet.

Wish you can disable "overshooting" when panning with a mouse.

What kind of camera was used to take this?

Seminar on how it was done here by the author https://mobile.twitter.com/erdmann/status/147794699638431334...

https://youtu.be/z_hm5oX7ZlE

97 rows × 87 cols (8 439) @Hasselblad H6D-400c MS images Size: 925 000 px × 775 000 px (717 GP) Res: 5 µm (5080 ppi)

It's a £43k camera on a huge automated gantry system controlled by python software

They didn't need such an expensive camera if they were going to do it this way. A Sony A7R would have produced an equal result.

if you have a priceless work of art, cost is no object when scanning it

The cost of the camera is a small drop in the bucket versus the rig and work gone into this.

They used a Hasselblad H6D mounted on a rig

Hasselblad

One of my favorite paintings of all time.

717 Gigapicels?

They should have titled this installation Rambrandt

I'll, erm, show myself out.

We should just send this out into space and call it done. Human civilization: fulfilled.

Reply on news.ycombinator.com