Hacker News Reader

Comment by acchow | Hacker News Reader

Mindblowing that this technique is patentable

Keep in mind that video cameras of that era were mostly not bayer pattern sensors, at least for professional television cameras. They were three, full raster sensors, typically with an interference-based prism block shunting blue, green, and red light to different sensors. These were so-called "three chip" cameras, the optics of which dates back to analog cameras that used tubes.

Bayer pattern sensors were more common in DSLRs, because they were an extension of 35mm film, and you had to have a single, flat imaging device to work with film optics.

In the film industry, the only digital cameras were the Sony F900 series (iirc) which I believe had the three-chip design, the Thomson Viper (again, memory foggy), and maybe the Panasonic Varicam... I don't know if the Viper or Varicam were single sensor, but they evidently weren't Bayer pattern or didn't compress the Bayer pattern. (I think at least one had RGB stripe layout, which is just inherently inferior.)

So yeah, it's obvious now. But it was patentable then, because nobody (or very few) had ever made a Bayer pattern sensor for a video camera. It's easy to forget how long ago 2007 was, at least until I try to get out of bed in the mornings.

There's a good wikipedia page for anyone interested in how the 3 sensors were implemented: https://en.wikipedia.org/wiki/Three-CCD_camera

There's a fascinating detail about 3CCD/3CMOS that's not mentioned in that article, namely that the sensors aren't actually in the same plane, but slightly offset in depth (probably because of the filter films between the prisms). Broadcast lenses are specifically corrected for being either 3CCD or 1CCD, the former having a back-focal distance that varies with wavelength, so that red is focused in the red's sensor plane and so on. You cannot reasonably use one kind of lens on the other kind of system. Almost all broadcast lenses that shore up on eBay are for 3CCD - so they're pretty useless for use with "normal" small-format cameras, even if their specs are otherwise enticing.

Yeah, they used telecentric lenses; for the prism block to work properly, you have to have each principal ray across the lens perpendicular to the filter. This puts very rough optical constraints on the lens; a typical video camera lens would have dozens of elements arranged in many groups, while an equivalent 'cine' lens might have half or fewer elements.

And on top of that, the lenses produced ugly images. Going to single chip designs was a big upgrade in image quality, mostly for optical reasons.

> Bayer pattern sensors were more common in DSLRs, because they were an extension of 35mm film, and you had to have a single, flat imaging device to work with film optics.

A lot of people were shooting video with their DSLRs before more dedicated devices became available (myself included). Is there actually a fundamental difference between a DSLR and video camera at that point?

>A lot of people were shooting video with their DSLRs before more dedicated devices became available

I doubt it. DSLRs didn't have video shooting capabilities till quite later on (late 00's?) compared to phones and point and shoot digital cameras which did much earlier.

Yeah you're right - I was off by a few years. Seems like it wasn't common until 2009

There was a Bayer-filter camcorder on the market in 1997: the Canon Optura (or MV1, outside America).

That was such a great camera! That and my Nikon 990 were my transition to a full digital video and still image workflow.

But if you had a raw image from a camera with a bayer pattern, and you were tasked to compress it, anyone with decent knowledge about compression and the technology they're working with should be able to come up with compressing before de-bayering. Is something really patentable just because it applies to non-mainstream technology?

From the sounds of it, the patent literally seems to be about... compressing the data you get off of the image sensor. And that's it. Or are there important details I'm missing?

> But if you had a raw image from a camera with a bayer pattern, and you were tasked to compress it, anyone with decent knowledge about compression and the technology they're working with should be able to come up with compressing before de-bayering. Is something really patentable just because it applies to non-mainstream technology?

Actually this exact reasoning is textbook grounds for rejecting a patent application. If a “person having ordinary skill in the art” would trivially come up with the claimed invention, a patent application can be rejected without a reference of prior art by citing obviousness of the invention. If what you’re saying is true, the examiner that originally granted the parent was probably not familiar with the subject matter(which is common).

Source: was a patent examiner a good while back

Yes that's the theory, but in practice patent offices massively err on the side of "eh it's probably novel, we'll let the courts sort it out later, oh and by the way here's our fee".

Absolutely. Well, first "let me issue a vague rejection and force a slight amendment to show due diligence, then issue an allowance on the amended claims". And then let the courts sort it out, as you said.

A lot of great inventions can seem obvious in retrospect once it's been explained to you, so it's a tricky razor to apply.

Perhaps, but this line is more often used to pretend that obvious inventions are more novel than they really are.

While I intuitively agree with you, how do you tell?

And more importantly, how do you tell in the limited time window the patent examiner has ([1] claims it's as low as 19 hours.)

edit: [2] has a more in-depth look at the problem

[1] https://www.oecd.org/site/stipatents/5_4_Frakes_Wasserman.pd... - "on average, an examiner spends only nineteen hours reviewing an application, including reading the patent application, searching for prior art, comparing the prior art with the patent application, writing a rejection, responding to the patent applicant’s arguments, and often conducting an interview with the applicant’s attorney"

[2] https://www.patentprogress.org/2018/03/06/granted-19-hours/

The problem is that, as you say, examiners usually lack ordinary skill in the art. Judges likewise. And there is never a shortage of "experts" who will testify (for a fee, of course) to how hard this was in retrospect.

I see a lot of people say that software shouldn't be patentable, but I actually think the real problem is the "ordinary skill in the art" test seems to be entirely broken / almost absent in that domain.

A lot of high value software patents seem to be things that would be in the realm of a solid candidate to come up with on the spot in a 30 minute interview for a programming job, and not because the technique is "known". Compressing individual channels is definitely something that sounds in the realm of good intern candidate to be able to come up with the idea on the spot in an interview.

So the problem isn't really software parents intrinsically, it's that they are so absurdly out of line with the "invention costs" such that "something an average programming intern would be able to come up with independently has a high chance of actually being patented". If the parents were only actually for "nonobvious to someone who hasn't been exposed to it before" things then they wouldn't really be a problem.

It's not either/or but rather both. Software (like all other math) shouldn't be patentable, and the "ordinary skill in the art" test is entirely broken / almost absent in that domain.

Sure the idea of compressing before de-bayering might be obvious. I'm not so sure the actual technique for doing the compression in a manner that also made it easy to de-compress while de-bayering is quite so obvious.

It probably took quite a bit of experimentation to come up with a number of different ways of separating the colours and compressing the individual colours, and identifying the approach that played best with the de-bayering process. So the basic "compress before de-bayering" idea might not be patentable, but the actual algorithm and specific approach(s) sounds reasonably difficult to refine, and thus patentable.

My understanding is that all lossless on-camera raw compression is patent encumbered, and so companies use fancy proprietary mostly-lossless compression eg. blackmagic raw.

> So yeah, it's obvious now. But it was patentable then, because nobody (or very few) had ever made a Bayer pattern sensor for a video camera. It's easy to forget how long ago 2007 was, at least until I try to get out of bed in the mornings.

yeah really this is a commentary on just how poorly-suited a 28-year exclusivity period is given the rate at which modern technology advances.

I fully agree, this is something that definitely would have been novel at the time, it's a good patent, but even 14 years later it's grating that fairly basic/obvious stuff to contemporary eyes is still patent-protected.

There were Bayer sensors before Red.

If I remember correctly, three-chip cameras became impractical at higher resolutions because of the focusing problems that resulted from refraction. Red, green, and blue wavelengths refract at different rates... which was a problem when they're all coming through the same lens. I don't remember details beyond that.

I think it had more to do with the requirement that three chip camera lenses be telecentric (see my earlier comment). Tends to make worse lenses that are more expensive.

Until tested in court, many, many things are patentable...

It's been tested in court.

https://www.engadget.com/2019-11-11-apple-prores-raw-red-pat...

Major patent reform is required.

We actually have had major patent reform! Look into IPR review, the Alice Supreme Court decision, and the patent reform act of ... was it 2012? 2013?

The IP landscape is wildly different than it was when these patents were first filed. That they've held up under scrutiny implies they're not _total_ junk.

Also, it probably still needs reform. But my understanding from talking to lawyers (who I was paying, to be fair), it's better than it was.

Apple is really good at losing patent cases in court, even when someone sues using a patent copied verbatim from Apple itself (S3TC - road pizza video codec)

Seriously. I zip my raw files for storage on my NAS. Would that be an infringement?

Reply on news.ycombinator.com