The Night Watch: The Missing Pieces
rijksmuseum.nl
rijksmuseum.nl
My grandfather, a taciturn, unsentimental Quaker mechanical engineer, was given the job of preparing a Methodist church to be a Quaker meetinghouse. The church had tall stained glass windows, but Quaker meetings are un- or minimally adorned. Well, Dan was a competent, efficient fellow. In short order the meeting was fit for occupancy again. The stained glass windows were gone except for a rose window. Some time after some of the Methodists came by asking what had become of the tall stained glass windows. They were memorial windows and they wanted to keep them as memorials. No one knew. Questions went around. Eventually someone asked Dan. He said they were under the gravel in the parking lot. What! Why are they there!? He just said, "they were ugly."
I hope the spare pieces of the Night Watch weren't ugly, and if they were, the job of fitting them to the space wasn't given to some 18th century Dan.
https://en.wikipedia.org/wiki/Ecce_Homo_(Mart%C3%ADnez_and_G...
How art is appreciated changes with the times.
Anyone interested in the recreation of famous incomplete paintings, should check out Manet's The Execution of Emperor Maximilian. This is a painting Manet gave up on, and chopped up into smaller studies.
https://www.nationalgallery.org.uk/media/33986/n-3294-00-000...
There are sketches that indicate the format he was aiming for, but Manet was a famous experimenter, so we will probably never know.
My guess, even that I hope not, is that people thinks that AI is going to be more impartial, more accurate, less biased. The reality is that an AI, as used in this example, is going to be over-fitting.
An artist can understand Lunden and Rembrant art and use that knowledge to produce something closer to the intention of the artist and the copy.
Cool project, but a dangerous precedent if people thinks that AI provides unbiased reality, that using AI you can show "how it really looked" as something more accurate that any human con achieve. Constructing something from missing data is always guesswork, whenever the intelligence is natural or artificial.
That statement rubs me the wrong way a bit though, because whether you ask a person or a computer model to copy Rembrandt's style, in each case it's still a copy, and I would argue in each case there's still a(n) artist(s) mimicking Rembrandt, just using different tools.
> My guess, even that I hope not, is that people thinks that AI is going to be more impartial, more accurate, less biased. The reality is that an AI, as used in this example, is going to be over-fitting. > Cool project, but a dangerous precedent if people thinks that AI provides unbiased reality, that using AI you can show "how it really looked" as something more accurate that any human con achieve. Constructing something from missing data is always guesswork, whenever the intelligence is natural or artificial.
I think this is an excellent take.
I see it as just an old-meets-new mashup, craftfully executed. Any artist/technique you use isn't going to be the original Rembrant, so the choice of what artist/technique you use is just a question of aesthetics and taste.
Here we're just exploring a new technique of the 21st century.
this is about the use of AI to reconstruct missing pieces of a Rembrandt painting ("Night Watch") using a contemporaneous copy from Lundens. After the Lundens painting was made, the Rembrandt original was cropped aggressively (~30 cm on a side) in order to fit onto a particular wall where it hung.
Now that it is considered a masterpiece, they are trying to computationally estimate what those cropped sections looked like using Lunden's copy.
Link: https://www.youtube.com/watch?v=z_hm5oX7ZlE&list=PL2Uw4_HvXq...
I saw on the left side of the main gallery (this is from memory, 30 years back) a small painting (say, 40x20 cm) which had a name similar (or the same) as The Night Watch and I thought that it is a great deal for such a nice but average painting.
Then I saw some people standing in the middle of the gallery watching something and I discovered the painting :)
If you enter from the room to the right hand side of the Night Watch it's almost easy to miss, not least because the ship models in the rooms that precede it are quite impressive, too.
From the Great Hall and the Eregalerij (Gallery of Honour) the effect is positively awe-inspiring. I've once heard someone describe the Rijksmuseum as a cathedral of Enlightenment. The Night Watch is the centrepiece that cathedral was built around.
I think there was also an exhibition on the history of doll houses
So the thing is, the completion is a modern artifact, and fundamentally in both cases it captures the expertise of the reconstructor, and not some kind of essence of the painting.
What sort of NN would be used to warp two compositionally similar images to match each other in structure? In the example the copy painting had almost all of the same general components, laid out with fairly significant differences, and they warped it to match quite precisely.
The basic idea is:
1. Find features in images A and B (SURF / SIFT are common). However, maybe a NN could find interesting features here?
2. Match features so you know what feature from A is where in B. Again, a NN could potentially do this "better".
3. Compute a homography (translation between planes of features) or procrustes transform (or other types of transformations)
4. For each pixel (p) in the target frame (B), take the inverse of the transformation to find the pixel of the original frame (A) (which is usually a fractional pixel because it won't line up)
5. interpolate between nearby pixels in (A) to get the "true" pixel value at (p) in (B). Perhaps a NN could improve this interpolation? We've seen this in super resolution, for example.
You can easily reproduce these steps with just a few function calls in OpenCV. Try it out. https://opencv.org/
Up to step 3 is very helpful to answer questions like: "How much did the camera move between captures of image A and B. So now you can solve structure from motion, visual odometry, etc. This is typically how quadrotors navigate with cameras.
I'll see myself out..