RED is suing Nikon for infringing on its video compression patents | Hacker News Reader

RED is suing Nikon for infringing on its video compression patents

nikonrumors.com

I haven't pulled the patents, but they probably have to do with compressing the pre-debayered image. Apple sends Red a big check every year for the same patents; I believe they're part of ProRes but not sure.

Essentially, by compressing the raw RGB data coming off the sensor (where each pixel only has R, or G, or B) you avoid the data explosion of interpolating color values at each pixel. It's a technique for compressing raw files, and Red was early to the game.

Perhaps someone on here has more detailed technical info, I haven't actually read the patents, just talked to engineers close to the project.

Mindblowing that this technique is patentable

Until tested in court, many, many things are patentable...

It's been tested in court.

https://www.engadget.com/2019-11-11-apple-prores-raw-red-pat...

Major patent reform is required.

We actually have had major patent reform! Look into IPR review, the Alice Supreme Court decision, and the patent reform act of ... was it 2012? 2013?

The IP landscape is wildly different than it was when these patents were first filed. That they've held up under scrutiny implies they're not _total_ junk.

Also, it probably still needs reform. But my understanding from talking to lawyers (who I was paying, to be fair), it's better than it was.

Apple is really good at losing patent cases in court, even when someone sues using a patent copied verbatim from Apple itself (S3TC - road pizza video codec)

Keep in mind that video cameras of that era were mostly not bayer pattern sensors, at least for professional television cameras. They were three, full raster sensors, typically with an interference-based prism block shunting blue, green, and red light to different sensors. These were so-called "three chip" cameras, the optics of which dates back to analog cameras that used tubes.

Bayer pattern sensors were more common in DSLRs, because they were an extension of 35mm film, and you had to have a single, flat imaging device to work with film optics.

In the film industry, the only digital cameras were the Sony F900 series (iirc) which I believe had the three-chip design, the Thomson Viper (again, memory foggy), and maybe the Panasonic Varicam... I don't know if the Viper or Varicam were single sensor, but they evidently weren't Bayer pattern or didn't compress the Bayer pattern. (I think at least one had RGB stripe layout, which is just inherently inferior.)

So yeah, it's obvious now. But it was patentable then, because nobody (or very few) had ever made a Bayer pattern sensor for a video camera. It's easy to forget how long ago 2007 was, at least until I try to get out of bed in the mornings.

There's a good wikipedia page for anyone interested in how the 3 sensors were implemented: https://en.wikipedia.org/wiki/Three-CCD_camera

There's a fascinating detail about 3CCD/3CMOS that's not mentioned in that article, namely that the sensors aren't actually in the same plane, but slightly offset in depth (probably because of the filter films between the prisms). Broadcast lenses are specifically corrected for being either 3CCD or 1CCD, the former having a back-focal distance that varies with wavelength, so that red is focused in the red's sensor plane and so on. You cannot reasonably use one kind of lens on the other kind of system. Almost all broadcast lenses that shore up on eBay are for 3CCD - so they're pretty useless for use with "normal" small-format cameras, even if their specs are otherwise enticing.

Yeah, they used telecentric lenses; for the prism block to work properly, you have to have each principal ray across the lens perpendicular to the filter. This puts very rough optical constraints on the lens; a typical video camera lens would have dozens of elements arranged in many groups, while an equivalent 'cine' lens might have half or fewer elements.

And on top of that, the lenses produced ugly images. Going to single chip designs was a big upgrade in image quality, mostly for optical reasons.

> Bayer pattern sensors were more common in DSLRs, because they were an extension of 35mm film, and you had to have a single, flat imaging device to work with film optics.

A lot of people were shooting video with their DSLRs before more dedicated devices became available (myself included). Is there actually a fundamental difference between a DSLR and video camera at that point?

>A lot of people were shooting video with their DSLRs before more dedicated devices became available

I doubt it. DSLRs didn't have video shooting capabilities till quite later on (late 00's?) compared to phones and point and shoot digital cameras which did much earlier.

Yeah you're right - I was off by a few years. Seems like it wasn't common until 2009

There was a Bayer-filter camcorder on the market in 1997: the Canon Optura (or MV1, outside America).

That was such a great camera! That and my Nikon 990 were my transition to a full digital video and still image workflow.

But if you had a raw image from a camera with a bayer pattern, and you were tasked to compress it, anyone with decent knowledge about compression and the technology they're working with should be able to come up with compressing before de-bayering. Is something really patentable just because it applies to non-mainstream technology?

From the sounds of it, the patent literally seems to be about... compressing the data you get off of the image sensor. And that's it. Or are there important details I'm missing?

> But if you had a raw image from a camera with a bayer pattern, and you were tasked to compress it, anyone with decent knowledge about compression and the technology they're working with should be able to come up with compressing before de-bayering. Is something really patentable just because it applies to non-mainstream technology?

Actually this exact reasoning is textbook grounds for rejecting a patent application. If a “person having ordinary skill in the art” would trivially come up with the claimed invention, a patent application can be rejected without a reference of prior art by citing obviousness of the invention. If what you’re saying is true, the examiner that originally granted the parent was probably not familiar with the subject matter(which is common).

Source: was a patent examiner a good while back

Yes that's the theory, but in practice patent offices massively err on the side of "eh it's probably novel, we'll let the courts sort it out later, oh and by the way here's our fee".

Absolutely. Well, first "let me issue a vague rejection and force a slight amendment to show due diligence, then issue an allowance on the amended claims". And then let the courts sort it out, as you said.

A lot of great inventions can seem obvious in retrospect once it's been explained to you, so it's a tricky razor to apply.

Perhaps, but this line is more often used to pretend that obvious inventions are more novel than they really are.

While I intuitively agree with you, how do you tell?

And more importantly, how do you tell in the limited time window the patent examiner has ([1] claims it's as low as 19 hours.)

edit: [2] has a more in-depth look at the problem

[1] https://www.oecd.org/site/stipatents/5_4_Frakes_Wasserman.pd... - "on average, an examiner spends only nineteen hours reviewing an application, including reading the patent application, searching for prior art, comparing the prior art with the patent application, writing a rejection, responding to the patent applicant’s arguments, and often conducting an interview with the applicant’s attorney"

[2] https://www.patentprogress.org/2018/03/06/granted-19-hours/

The problem is that, as you say, examiners usually lack ordinary skill in the art. Judges likewise. And there is never a shortage of "experts" who will testify (for a fee, of course) to how hard this was in retrospect.

I see a lot of people say that software shouldn't be patentable, but I actually think the real problem is the "ordinary skill in the art" test seems to be entirely broken / almost absent in that domain.

A lot of high value software patents seem to be things that would be in the realm of a solid candidate to come up with on the spot in a 30 minute interview for a programming job, and not because the technique is "known". Compressing individual channels is definitely something that sounds in the realm of good intern candidate to be able to come up with the idea on the spot in an interview.

So the problem isn't really software parents intrinsically, it's that they are so absurdly out of line with the "invention costs" such that "something an average programming intern would be able to come up with independently has a high chance of actually being patented". If the parents were only actually for "nonobvious to someone who hasn't been exposed to it before" things then they wouldn't really be a problem.

It's not either/or but rather both. Software (like all other math) shouldn't be patentable, and the "ordinary skill in the art" test is entirely broken / almost absent in that domain.

Sure the idea of compressing before de-bayering might be obvious. I'm not so sure the actual technique for doing the compression in a manner that also made it easy to de-compress while de-bayering is quite so obvious.

It probably took quite a bit of experimentation to come up with a number of different ways of separating the colours and compressing the individual colours, and identifying the approach that played best with the de-bayering process. So the basic "compress before de-bayering" idea might not be patentable, but the actual algorithm and specific approach(s) sounds reasonably difficult to refine, and thus patentable.

My understanding is that all lossless on-camera raw compression is patent encumbered, and so companies use fancy proprietary mostly-lossless compression eg. blackmagic raw.

> So yeah, it's obvious now. But it was patentable then, because nobody (or very few) had ever made a Bayer pattern sensor for a video camera. It's easy to forget how long ago 2007 was, at least until I try to get out of bed in the mornings.

yeah really this is a commentary on just how poorly-suited a 28-year exclusivity period is given the rate at which modern technology advances.

I fully agree, this is something that definitely would have been novel at the time, it's a good patent, but even 14 years later it's grating that fairly basic/obvious stuff to contemporary eyes is still patent-protected.

There were Bayer sensors before Red.

If I remember correctly, three-chip cameras became impractical at higher resolutions because of the focusing problems that resulted from refraction. Red, green, and blue wavelengths refract at different rates... which was a problem when they're all coming through the same lens. I don't remember details beyond that.

I think it had more to do with the requirement that three chip camera lenses be telecentric (see my earlier comment). Tends to make worse lenses that are more expensive.

Seriously. I zip my raw files for storage on my NAS. Would that be an infringement?

Apple actually tried (and failed) to break RAW from the RED royalties in 2019, so I believe you are correct.

https://www.engadget.com/2019-11-11-apple-prores-raw-red-pat...

> Essentially, by compressing the raw RGB data coming off the sensor (where each pixel only has R, or G, or B) you avoid the data explosion of interpolating color values at each pixel. It's a technique for compressing raw files, and Red was early to the game.

So the novelty is that they separate R,G and B planes before compression?

There are other compression algorithms that work with other colorspaces, giving even better compression. E.g. splitting luminance and chromatic components. I think these algorithms might also be older.

Digital camera image sensors only have one image plane. Each pixel is recorded as a grayscale value where the light has passed through a red, green, or blue filter. To get a complete pixel (with values for all 3 colours) requires software to interpolate a pixel with its neighbours.

Well... yeah, that's the crux of it.

But the patented part is treating each channel separately; the actual compression algorithm isn't at issue (which, in Red's case, is JPEG2000).

That said, as soon as you get into colorspace conversions or color sampling conversions (like YUV, etc), you're baking in choices and dealing with quantization and roundoff errors. If you want the most pristine image possible for dealing with it in post, you want the raw photosite information. And if you want that, without deBayering (which introduces those same errors!), you get to pay Red a royalty check.

I don't make the rules, I'm just pointing out what the stakes are here. Yeah, it's pretty fundamental and obvious now, but at the time no one was selling a professional video camera with a single chip for that price. It was a different world.

> So the novelty is that they separate R,G and B planes before compression?

> Yeah, it's pretty fundamental and obvious now, but at the time no one was selling a professional video camera with a single chip for that price. It was a different world.

I don't see how no such cameras being on the market makes the software any less obvious, unless we are totally abdicating the meaning of "obvious" to reduce it to "non-novel". But if that were the law's intent, it wouldn't require non-obviousness and novelty - just one or the other would suffice.

The obviousness test seems to vary by jurisdiction, but typically requires that someone with ordinary skill in the field would not likely come upon that solution[0]. It must also be novel[1].

[0]https://en.wikipedia.org/wiki/Inventive_step_and_non-obvious...

[1] https://en.wikipedia.org/wiki/Novelty_(patent)

> someone with ordinary skill in the field would not likely come upon that solution

This requirement often appears to be effectively ignored, as in this case. Compressing the raw data (i.e., before transformations like de-bayering) is a pretty obvious thing to try out. This sort of run-of-the-mill development does not deserve a 20-year monopoly.

Amazon's one-click patent is probably the most famous example of this. Anyone with any database programming experience can figure out "store the customer's payment details and shipping address and use them later to complete purchases". That was also true in 1999.

The patent office seems to evaluate the requirements in a manner very favorable to patent applicants/holders. I think that's a bad thing; the criteria for what counts as an invention should be stricter.

It was obvious then too. Way back when I was taking the raw grayscale data before debayering and I immediately saw I'd need to treat each color's pixels separately. No background in compression or signal processing or anything, just a tinker's mindset and some experience faffing around in Photoshop/GIMP

On top of that the patent requires the Jpeg2000 compression algorithm to be useful.

If the choice is between the current state of affairs and no software patents, then no software patents is the way to go.

> So the novelty is that they separate R,G and B planes before compression?

No. The novelty is that they compress before de-mosaicing (aka debayering).

In other words, rather rebuilding the image from the signals and compressing that image, it compresses the signals themselves (not even seperated I think).

I'm just guessing it makes more sense to separate them. In my naive no knowledge version. if it was

   RGRGRGRG
   GBGBGBGB
   RGRGRGRG
   GBGBGBGB

Then, if I compress losslessly just grayscale I know at the other end I just interpret the first pixel as Red, next as green, red, green, etc., Then on the next row I interpret the first pixels as green, next as blue, next as green etc...

BUT, if I compress lossily then the unrelated colors next to each other are going to bleed into each other while compressing. VS if I separate them out so

   RGRGRGRG
   GBGBGBGB
   RGRGRGRG
   GBGBGBGB

becomes

   RRRR BBBB  
   RRRR BBBB

   GGGG
   GGGG
   GGGG
   GGGG

The compression would make more sense and less bleeding of the signals from each color into other colors.

It would also seem to compress better. Imagine you're filming something pure green. Exaggerating but you'd get this

Which after separation is

That's just a guess though.

A different comment suggested they compress the green channel, and then compress the red and blue channel as differences against the green channel.

This makes a lot of sense to me, since green is sort of 'luminance'. It is like a poor-mans LAB color space that exists before demosaicing.

This is a very good guess, and I believe it's precisely how it's done. Doing anything diff-ing the color channels still requires a lossy transformation step, so it's not quite as "raw."

And you're right, that treating it as a single monochrome image makes it less compressible, because you get lots of high frequency information between adjacent pixels with different color. Which is sort of the opposite of what you want.

Note also that, despite calling it "raw," Red's implementation is lossily compressed. You can even tune the compression ratio of their "raw" codec, and back in the very early days, there was certain subject matter that just looked terrible coming out of the camera, even as raw -- I remember one particular tree that an early user would film to test new firmware updates; in the earliest firmware builds, it actually crashed the camera (!). The images it did produce showed tons of wavelet compression artifacts.

For those who may not be aware: digital cameras capture a grayscale image with a single value per pixel. The image sensor is covered by an array of colour filters (red, green, blue) in a pattern called the Bayer pattern. Typically, one row of filters will alternate RGRGRGRGRG and the next row will alternate GBGBGBGBGB. "Debayering" involves interpolating the colour-filtered grayscale values of neighbouring pixels into each pixel, and then often applying some software filtering to remove dead pixels, noise, and sharpen the image.

Aren't there still plenty of cameras using 3 separate sensors and an optical beam splitter, instead of a Bayer filter array?

I don't know of any consumer cameras that do this. Some manufacturers use a different filter on some models, Fuji uses an X-Trans filter on some models which in the past can cause issues with some software demosaicing.

X-trans is still a mosaic and the underlying tech is mostly the same (color filter array with monochrome "buckets"). Instead of a 2x2 pattern, it uses a 6x6 pattern.

Goes something like: GRBGBR BGGRGG RGGBGG

Instead of: RG GB

Consumers camera use a Bayer colour filter array because it is cheaper and more compact. Demosaicing is performant enough for most consumer applications that it is hard to justify for anything than scientific applications. As a matter of fact, even Motion Picture cameras, e.g. ARRI LF, are using a Bayer CFA.

If they do, that has to be limited to "broadcast" cameras. Digital cinema cameras all pretty prominently use a single sensor.

There is one model still sold by sony using these! https://pro.sony/en_GB/products/shoulder-camcorders/pxw-z750

All the others have now switched to bayer sensors.

Only $3k from Adorama; are pro cameras really that cheap these days?

[edit] That was google shopping popping up the Z150, which is presumably less expensive. The Z450 is 20k, body only, and the Z750 is "in store purchase only"

Yes, it's everything else that's expensive — a set of cinema lenses will set you back many multiples of that. Then you need to pay the people to operate them.

Here's a deep dive with an expert from Arri (big movie glass company) here: https://www.youtube.com/watch?v=q1n2DR6H7mk

There’s also a stacked sensor version.

https://en.m.wikipedia.org/wiki/Foveon_X3_sensor

Very surprised to see them still kicking, and preparing a 5th generation sensor by themselves.

https://m.dpreview.com/news/1885231343/sigma-releases-update...

They are really dinosaurs of electronic imaging. Sans Sony, I can say ALL sensor makers from the dawn of the era died off, or were sold/resold a million times.

Canon still makes their own sensors, don't they?

Canon is a long story, they been in, and out of the sensor game over the years

They eventually made a CMOS sensor development from scratch

Sigma had their Foveon sensor in APS sized version several years ago but they're currently working on the 35mm version. https://www.dpreview.com/news/1885231343/sigma-releases-upda...

It's taking forever though, they are encountering many issues to scale to 35mm.

Not plenty, “3CCD” cameras exist but are rare and mainly used for industrial inspections and TV broadcasts. Some old home video cameras did use it but not anymore.

What is the advantage in these last applications?

"Image quality" seems to be the most common answer... It eliminates the need for color filter deposited on the sensor, and also seem to require cares taken in lens design to have optical parameters converged for all three image planes in respective wavelengths(can't have one color being magnified or out of focus than others).

I suppose whether those points are pros or cons depends on design/manufacturing capability, and considering rarity of modern 3CCD designs, likely decisions are difficulties outweigh advantages today.

Nope, Bayer is the norm, but there's still some Foveon cameras that are typically rather slow to produce images.

No, nobody uses a 3-strip/3-sensor approach anymore. It was used in the early Technicolor process, and was revived during the digital era with cameras like the Minolta RD-175, and it lingered on in cine cameras for a while longer, but nowadays almost everyone uses a bayer-filter approach (or a very similar one - there are several proprietary alternatives for the mosaic layout that claim various quality improvements).

The only similar thing nowadays is Foveon sensors where you have 3 color sites stacked on top of each other, but Foveon didn't really have the money to keep up with the arms race and nowadays the modern sensors are far better even though they do have to apply a low-pass filter to avoid mosaicing. When you have many, many, many nodes of improvement past the Foveon sensor, that stops being a big deal.

It was also never particularly widely adopted, because none of these things are new - it was a tough sell to market a "really high color-resolution!" 5mp camera going up against 20mp filtered sensors and 50mp unfiltered sensors even 10 years ago.

Interestingly though, the Foveon approach is similar to film. Film has multiple layers and all are exposed at once. There's no denying bayer is sort of weird. It's just very expensive to be the odd man out doing your own sensors. "Good enough" usually wins, Foveon may be betamax but bayer is VHS - and I use that analogy knowingly, in the sense of Betamax also doing key features worse in areas that consumers care a lot about.

> It was also never particularly widely adopted

I think you must be ignoring television and video here? Basically everything that was shot on digital video for broadcast for decades was shot on 3-sensor cameras, and every "serious" video operation from sports to news all the way down to wedding photographers and high school videographers with any scratch at all were using 3-ccd setups. Sony's current (and only) 8K TV studio camera uses 3 sensors.

They meant Foveon was never widely adopted...

This means that the true resolution of digital cameras is roughly one third of the advertised megapixel value, depending on the exact Bayer matrix design.

Well that’s sort of true but the algorithms that decide on which pixel has which colour are better than just the number of underlying RGB pixels…

It is roughly half of the pixel count, as the Bayer interpolation gives better resolution than a naive conversion. Also, a lot of detail is in the green channel, which is registered by half of the total pixels.

Only if you consider raw pixel count in a single channel as the only "true" measure.

The true resolution of digital cameras isn't measured in pixels, it's measured in cycles per pixel and then related to the sensor size and pixel pitch[0]. Camera sensor pixels are not squares like the pixels in a digital image, they are point samples. You can then interpolate from the point samples onto an arbitrary number of digital pixels, so what matters is basically how close you can still see edges between lines, which depends on more things than the sensor size. When megapixels are quoted for a digital camera, they are either quoting sensor size (resolution) (like on real engineering documentation) or they are just quoting the highest resolution digital image they think they can get away with the software producing (marketing, advertisements), which is arbitrary, which is part of why you do hear not to just select a camera with the most megapixels.

It's a pretty deep topic and people do get degrees in basically just digital imaging. Not me though! I don't really understand most of this stuff...

[0] https://www.image-engineering.de/library/technotes/742-units...

It's only an approximation that cameras do point sampling, as the sensors are obviously not points.

That camera sensors mean a different thing for a pixel than what's typical for an image or an electronic display is a matter of convention.

True that interpolation is possible, but that's no reason to use different conventions as noted above.

I have seen a display manufacturer manipulate the same to artificially boost their resolution specifications.

Doesn't that mean green is over represented per area? Why not something like

    RGBRGBRGB  
    GBRGBRGBR  
    BRGBRGBRG

The human eye is particularly sensitive to green. This is also why the 5:6:5 encoding of RGB has its focus on green.

Also why Pentile OLEDs exist

Green is in the middle of the spectrum so it is the best for detecting amount of light. Color is determined by the G-R and G-B ratios.

There are sensors with more esoteric Bayer filters, like to better capture in low light there are sensors which replace one of the G's with an N (neutral light / "no" color filter)

Apple is paying RED for #9,230,299. A quick look at the claim doesn't seem to have this patent listed.

I believe there are two things to keep in mind here:

1. There is the idea of compressing each Bayer color.

2. There is the compression method.

I believe this issue is about point 2, the compression. But it seems that Nikon is using a different compression so I am not sure Red will win this.

Edit: from the legal document, I believe it is about this part:

"The camera can be configured to transform blue and red image data in a manner that enhances the compressibility of the data. The data can then be compressed and stored in this form. This allows a user to reconstruct the red and blue data to obtain the original raw data for a modified version of the original raw data that is visually lossless when demosaiced. Additionally, the data can be processed so the green image elements are demosaiced first and then the red and blue elements are reconstructed based on values of the demosaiced green image elements."

That quote sounds a lot like 1. (the idea of compressing each Bayer color), but I might misunderstand it.

Sounds a lot like converting "left-right" audio into "mid-side" audio for the same benefits.

2 is RED obfuscating ordinary JPEG2000

Edit 2: Nikon is using the TicoRAW compression from intoPIX.

So I wonder why Red isn't suing intoPIX.

I have a guess, but I haven't actually researched this specific case to be sure.

intoPIX seems to be based in Belgium. I'm not sure exactly what type of patents RED has, but there are types like software patents which don't exist in the EU. Or perhaps it isn't about the type and RED just straight up doesn't have EU patents even if they theoretically could. Combine that with intoPIX (and I haven't verified this) only operating inside the EU, or at least outside of the US. You end up in a situation where RED would have difficulty suing intoPIX, because intoPIX doesn't operate in a jurisdiction where RED holds the relevant patents.

Once Nikon enters the picture, things change. Nikon clearly has US presence.

Of course it could also just be that RED doesn't care about small fish and only started caring now because Nikon has the potential for a lot of revenue.

for people accustomed to looking at highly compressed video data (like mpeg2, H264 or H265/HEVC file, VP9, AV1, etc), what we're looking at here is a whole other category of barely-compressed high bitrate video used by professional cameras. It saves a lot of SSD recording storage space vs truly raw video but also leaves the video in a state to be further edited and cut together, then compressed into a final output product much later in the work flow.

The ordinary consumer will never see these video file formats, unless they go buy at least a $2500 blackmagic pocket 6k pro and add another $2500 of lenses and accessories to make it useful.

https://patents.justia.com/inventor/thomas-graeme-nattress

This is really a clinic on the dereliction of the USPTO and erroneous court rulings that have led to software patents.

But... the fact is that Red ripped off JPEG2000 and are now suing over it. And why not, when they're abetted by a corrupt patent regime.

They got Kinefinity and DJI with this patent collection already, too, forcing them to drop ProRes RAW support.

It's quite ridiculous, really, basically any camera which internally records compressed lossless sensor data prior to debayering (any "internal RAW" format) is vulnerable to RED patent trolling.

When will this patent expire?

The main patent in play, https://patents.google.com/patent/US9245314B2/en , expires in 2028.

I'm not well versed with patents. So when this patent expires in 2028, can RED "renew" it or it will be available for everyone to use w/o paying RED anything?

You can't renew a patent. Utility patents get a 20 year term, design patents get 15 years.

When a patent can take up to four years to be granted, and two years in most cases, then the current model is not sufficient.

Instead it ought to be a number of years from the day the patent is granted.

For software that should be around 5 years, if we really need software patents.

My Terra 4k is not receiving a software update for this exact reason. Fantastic RAW image, not losing that.

This is a common misconception, but the patent is not specific to internal recording.

Apple and Atomos co-invented ProRes Raw, and this nonsense has hampered Atomos's external recorders as well. Apple and Atomos are paying Red's extortion and... that's how bullshit patents are used to steal people's work in the USA.

The patents in question are a new level of terribad. So bad that they can be trivially invalidated by prior art. Nikon's going to spend a lot of money on lawyers but is likely to win (if they don't settle out of court...)

(When reading patents, skip to the claims, they are the only part that actually matters).

7,830,967's independent claim is a 2k+ resolution video camera that records sRGB or rec709 gamma. That's it.

8,174,560 is even more ridiculous. It's a 2k+ resolution camera that records at a 6:1 or more compression ratio. No actual method or anything, just the concept of compressing at least 6:1 compression ratio. Check out indepedent claim #1.

Both of these are very likely to have very easy prior art available. Finding any camera that records high resolution video, made before the priority date would do it.

Again, the patent to compress Bayer pattern data before de-Bayering is the pivotal one. There's a reason Apple is still paying royalties, and it ain't because they don't have good lawyers.

It seems like there should be some kind of penalty for filing troll patents if they can be invalidated like that.

There are legal remedies, like being forced to pay the other side's legal fees, and even worse consequences than that.

Difficult to win, though.

> if they don't settle out of court

Nikon never settles out of court. They have excellent IP lawyers.

That said, the video/imaging arena is a minefield of highly-enforceable patents. Very old ones. I don't miss writing that kind of software.

They’re also known to quietly imply that you may also be infringing on some of THEIR patents for which they have an enormous pile of.

Yup.

Like I said, I don’t miss that field too much.

Like dancing in an iron maiden.

Apple and Sony have both tried to invalidate this patent collection and failed.

> the concept of compressing at least 6:1 compression

The level of bullshit that this industry gets away with is absolutely fascinating. I'm a gamer and I was part of the crowd who were galvanized by TotalBuiscuit, rest his soul. I am not alone, and part of an ever increasing amount of people, that seriously question the anti-consumerism (hence, anti-capitalist, if you're very patriotic) practices employed by the status quo. It's patently obvious to do so in hindsight, but I look at a lot more through that lens nowadays.

Red is that 6-times distilled, 10-times cold filtered, 50 times purified, embodiment of anti-consumer practices and it's maddening.

Red isn't a consumer company. How are they anti-consumer?

I'm not defending their IP practices, but this is not as stupid as it seems. Nikon knew about the patents when they designed their cameras. The patents were not dumb in 2007-2008.

Claims like the one excerpted are part of larger claim sets; if any claim gets invalidated, the rest that rely on it are useless. For the sake of being explicit, but also robust, lawyers will take a technical description and devolve it into a series of nested claims. Those individual claims, in isolation, sound dumb. But they're usually contingent on earlier claims that might reveal the novelty.

(I have not read the patent.)

There's a conversation to be had about how much money Red is making from selling luxury goods to amateurs but this thread probably isn't the best place.

> Red isn't a consumer company. How are they anti-consumer?

Really?

I think you know like everyone else here what consumer means in this context, it's not literally about consuming food, or the disease known as consumption, and that entities that aren't individual consumers in the nightly news sense of the word are called 'consumers' as well.

In this instance semantics isn't insightful.

Because anti-competition is anti-consumer.

isn't the whole basis of capitalism that capital drives competition which drives innovation and consumerism?

> Red isn't a consumer company. How are they anti-consumer?

Everything is consumer these days. CNC machines, milling machines, soldering stations, tablesaws, planers, and, yes, high-end cinematic cameras.

You're no longer talking about Hollywood and maybe 4 or 5 other big studios leveraging this stuff. Marques Brownlee, Linus Tech Tips, Corridor, and tons of other Youtubers (and I'm sure many more from other platforms) eat these things like candy. If you or I wanted to get into this stuff, we reasonably could with a little financial discipline (and maybe bargaining with our significant other :D).

They are unimaginably vulnerable to someone coming along with a platform that accepts any, say, NVME, with some form of "works best with" certification system. That way, if your risk is low ( enthused hobbyist) go with your MLC dumpster diving special, if your risk is high then go with OEM, certified, or better (because there is better storage than what Red offers). Keep in mind that Red have supplanted a stupidly broken industry with a vastly less broken industry, but on the absolute scale they are still pretty fucking stupid. They are living on the time it takes for someone to realize that, given a huge amount of R&D into a sensor (because Red is likely the leading sensor), a more capitalist/competitive approach exists.

> Marques Brownlee, Linus Tech Tips, Corridor,

are not consumers but media businesses.

When did the consumer stop being the ones who consume the product?

I’m sure many farmers would consider John Deere’s practices regarding DIY repairs, anti-consumer, as they are the consumers. Any person who might reasonable be considered an end-user of a product, or might buy a type of product might be considered a consumer of that type of product.

Anti-consumerism and protectionism do not even require that you be a customer of the company in question, some practices extend beyond the product itself when the company seeks to stifle competition.

That line between "business" and "consumer" is becoming increasingly indistinct. This is the very nature of "influencers." We have every-day-joes/janes setting up multi-thousand dollar machines in their own (or rented) garage for their own, or followers', enjoyment.

> are not consumers but media businesses.

This idea of yours is extremely out of touch with reality, just like Red are. Those "media businesses" would be nowhere without an engaged audience. The modern audience are participators, not watchers. Commodity 3D printers have succeeded for a reason.

People aren't as one-dimensional as they used to be. They take their interests to an extreme degree. It might take a videography/cinematography enthusiast 2 years to save up for a Red, but they will.

Influencers are businesses, plain and simple. They buy the camera to make a profit.

Compare this to someone like me, who also owns expensive cameras, but I have them to take pictures and show my friends. I plan to make $0 from this endeavor. I'm a consumer, not a business.

And you don't care about the immensely expensive Red SSDs?

Oh, I shoot stills on Sony equipment, so I just use good old SD cards. And film when I need high resolution ;)

I agree that RED is cost-prohibitive for individuals that aren't running a business because of things like the 500% markup on SSDs. That is always the risk of a proprietary ecosystem; kind of like how you can buy 128GB of RAM for your PC for a few hundred dollars, but the same RAM in an Apple computer is $1600. That's just the price of "we guarantee that it will work", and for business use, it makes a lot of sense. For consumers, it kind of sucks, because you feel so close to being able to afford something really cool, but you just can't make the math work.

I do understand the pushback; people want to pay for the impressive sensor and not the mind-numbingly-boring SSDs, but they want to make money on both. I am not sure that's strictly consumer unfriendly, but just how they do the financial engineering.

Can I ask you a question?

Did you actually believe, or even think about the position you're now taking prior to reading the comment thread about it?

I'm highly skeptical that someone genuinely holds the position "businesses aren't consumers."

I think it's more likely that you're just arbitrarily taking a contrarian position to argue online to pass the time.

RED have some very nice people.

However, the product was/is okay-ishm but the fan boys are utterly toxic pricks.

the charged for everything, made special names for all sorts, and had tesla levels of bullshit when it came to deadlines and capabilities.

For example, the "RED ray" laser projector, which supposedly was a 4k native laser based projector, except that it wasn't, never ran at 4k during NAB and if I recall correctly at some point was replaced by a barco. (https://www.extremetech.com/electronics/126579-red-ray-4k-ci...)

They made "redrockets" that were supposedly super high tech decoding cards, but were in reality rebadged from another company. They were extra ordinarily fragile (I personally accidentally killed 3 in just one event) and fucking expensive (£4k each)

It was me, apparently that was the problem, not the design. In the same event I replaced graphics cards, a boat load of Fusion IoDrives, none of them died.

So yes, they were anti-consumer to a certain extent. However if they liked you, they'd invite you to the ranch, you'd hang out with other RED people and generally have a good time.

Don't forget about "REDmags", which are bog-standard off-the-shelf mSATA-SSDs sold at a markup so high it would make Apple execs blush.

Yeah this is all accurate... there is a lot of marketing at that company, which is unsurprising given that it was founded by Jim Jannard, who made his money selling consumer fashion goods at Oakley.

> if any claim gets invalidated, the rest that rely on it are useless.

This misunderstanding came up in another thread. If a dependent claim is invalidated, then it invalidates the claims it depends from, but not the other way around. For example, consider the following claims:

1. A method comprising steps A, B, and C.

2. The method of claim 1, further comprising step D.

If the prior art only teaches claim 1, then claim 2 could still be valid if the addition of step D is non-obvious over the prior art. If the prior art teaches claim 2, however, then claim 1 is also invalid.

“I am not alone, and part of an ever increasing amount of people, that seriously question the anti-consumerism (hence, anti-capitalist, if you're very patriotic) practices employed by the status quo.”

this is one of the most bizarre sentences ever written

Why?

How many people on this forum, specifically, suggest that they use Apple devices because of, ultimately, good-will garnered by Apple? There should be no doubt that Apple manufactured the best laptop hardware on the planet, for the general use-case, between 2014 and 2020. Yet, many people continue to perceive Apple based on historic merits as opposed to current merits. You could always get much more than Apple for much less, but they had the quality nailed down: it is certainly not rare (especially on HN) to come across someone still using their MBP from 2014 and reluctantly considering an upgrade. Following the golden days of Apple hardware, we still have people defaulting to that prior perspective, even in the face of the the keyboard "don't use it in an environment with dust" problems of recent history.

Consumer psychology is extremely fascinating, and until you look at the overall behavior of consumers (which includes your own stupid self), you just don't realize how you've monkey brain has been exploited.

That sentence is only bizarre to people who haven't realized their own bizarre behavior.

Edit: I include myself in monkey-brain analogies, I am a human/stupid/exploitable just like you.

> Edit: I include myself in monkey-brain analogies, I am a human/stupid/exploitable just like you.

Yeah, I mean, kinda right back atcha here ;) A lot of people are still stuck in the "apple is dumb Jony Ives design with no ports and super bad performance and overheating" era too, despite Apple actually putting ports back on laptops and having an in-house ARM core that is far and away the most performant laptop you can buy with multiples of the battery life of similar x86 laptops too.

A more productive take on the topic is that a lot of times these things come down to values differences between various consumers: you value repairability and open ecosystem, Apple customers value high-quality OEM parts/service and the all-around build quality (other laptops have a few of the selling points, but it's hard to check all the boxes that Apple does). You highly value the keyboard, Apple customers are willing to tolerate it even if it's not ideal because of the rest of the package. Apple customers value the battery life, you are willing to tolerate plugging in more often even if it's not ideal because of the rest of the package. Etc etc. Those are values differences, not in the sense of moral differences but different customers have different needs and preferences and a lot of time is spent arguing about stuff that ultimately comes down to "I value different things than you do".

It's fairly rare that things actually come down to what I'll call "alternative facts" scenarios - sometimes it does, like the broad disagreement over how to interpret M1 performance numbers (but the differences in performance/battery life that people observe tend to lead to a pretty obvious conclusion imo). But it's generally a lot rarer than the "I like different stuff than you" spats.

I'm saying this as someone who has an x86 apple laptop for work and is not particularly pleased with it, but who is looking at the M1 as a prospect for a personal laptop, but also considering a few particular x86 laptops as well. People get super weird on the x86 side with the "nothing apple can ever be good" shit too, or the Android people getting super weird about how sideloading is a must-have feature (and ignore how that sidesteps the app-review process and leads to facebook/netflix/other must-have apps forcing their way to escalated privileges). It's generally a lot of values arguments, people value different things and that leads them one way or the other.

Of course marketing does play a key role in telling people what they should value, and I am no different from anyone else in that I am affected by marketing even when I think I'm making rational, logically-supported choices ;)

I read your post at least 3 times and still don't understand how those sentences fit together into a coherent thought.

What does TotalBiscuit have to do with video compression? What does anti-capitalism have to do with being patriotic? How is TotalBiscuit related to anti-consumerism?

The only thing that somehow makes sense is that Red employs some anti-consumer practices through overly broad patents, thus restricting innovation.

Is that what you were trying to say?

I haven’t looked at the claims here, but the independent claims are the most broad claims of a patent. The dependent claims are always necessarily narrower. So, even if an independent claim is invalidated, it’s dependent claim may survive if it is novel and non-obvious over the prior art.

> even if an independent claim is invalidated, its dependent claim may survive if it is novel and non-obvious over the prior art.

False. Let's say Claim 1 has elements A, B, and C. Claim 2 adds element D. Thus Claim 2 really has A, B, C, and D.

If 'A' is invalidated, then any claim that includes it, i.e. that depends from Claim 2, is also invalidated.

This is not true. Dependent claims are often viewed as "backup" claims in case the independent claims are rejected in prosecution by the patent examiner or found invalid by a court.

It's not hard to find online sources to back this up, for example:

https://patentlyo.com/patent/2008/05/theory-of-depen.html

https://www.natlawreview.com/article/patent-owner-tip-12-sur...

> False.

You have it backwards. If a dependent claim is invalid, then the independent claim from which it depends is also invalid. It doesn't make sense the other way around.

I'm not sure who "you" refers to here.

Your Patently Obvious article refers to patent prosecution, which is different from re-examination. Let's keep referring back to my original 1(A, B, C) and 2(D) example.

A dependent claim is "backup" because the examiner may find that (A, B, C) is not patentable, but coupled with D, you get (A, B, C, D) which is.

This happens all the time in prosecution. So let's say (A, B, C, D) is allowed and issued.

Note that you did not win the right to sell a product with (A, B, C, D), because someone else (Call them CompanyX) has patented (A, B, C) and you are infringing it.

All you've got is the right to exclude anyone, including CompanyX, from selling (A, B, C, D).

If someone else infringes your patent and you sue them, and they're able to prove that 'A' was known in the art, then CompanyX's patent and yours are both invalidated.

Consider the following claims:

1. A fly-swatter comprising (A) a shaft, (B) a handle attached to one end of the shaft, and (C) a perforated paddle attached to the other end of the shaft.

2. A fly-swatter according to claim 1, further comprising (D) a robot attached to the handle and configured to identify and kill a fly by applying force to the handle so as to cause the perforated paddle to strike the fly.

Here, if claim 1 is invalidated (*) by a prior art fly-swatter, then claim 2 may remain valid if the combination with element (D) is novel, non-obvious, and the other requirements are satisfied (written description, enablement, etc). If claim 2 was invalidated by a prior art fly-swatter, however, then claim 1 would also be invalidated.

(*) Note: This would only happen in a trial or IPR because during prosecution if the examiner objects to a claim then it must be withdrawn or amended to be allowable.

A good test question there (I realize I was wrong about the (A, B, C) example. You have to knock out all three.

You have two contradictory hypotheses: "if claim 1 was invalidated by a prior art fly-swatter" and "If claim 2 was invalidated by a prior art fly-swatter"

If fly-swatters are prior art (as they are), then you would claim your robot thingie as :

"An automated fly destroyer, comprising:

a fly swatting device, and a robot configured to etc. etc."

And you would say in your spec, "the fly-swatting device can include, in one embodiment, a commercial fly-swatter, but could be any device capable of killing a fly, including newspapers, rolled up magazine, board, book, and many more."

However, if your claim enumerated the features of a fly-swatter, then it would certainly be questioned. You should not bother listing things that are known in the art, and furthermore, you'd have failed to claim the book, newspaper, magazine, etc. methods of killing the fly.

On further editing: your idea of knocking out a dependent claim by knocking out all the elements is theoretically correct. However, that's not the way things are usually done.

[1] explains the issues around this. IPRs almost always go after the broadest claims, and the patentee needs to respond "even if claim 1 is invalidated, claim 2 is still valid because blah blah blah."

Patents almost always put the heavy novelty in claim 1, like I did for your robot fly-swatter invention. The dependent claims are narrower because they add additional elements which often aren't very novel. However, you're right that they can still survive and be useful.

[1] https://www.natlawreview.com/article/patent-owner-tip-12-sur...

> Awaiting a patent lawyer who does this every day

I am a registered patent attorney (*), but I don't practice in patent prosecution.

> a fly swatting device ... And you would say in your spec ...

You're alluding to means-plus-function claiming. "An element in a claim for a combination may be expressed as a means or step for performing a specified function ... and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof." (35 USC 112(f) or pre-AIA 35 USC 112 sixth paragraph).

> You should not have claim elements that are known in the art, and furthermore, you'd have failed to claim the book, newspaper, magazine, etc. methods of killing the fly.

As a strategic matter, I agree because you usually want to claim the broadest, most essential form of an invention, and that's a good reason to use means-plus-function claiming. The presence of prior art elements in a claim, however, do not invalidate the claim because (1) for a prior art reference to anticipate a claim, all elements must be present in the prior art reference (all elements rule), and (2) for non-anticipating prior art references to render a claim obvious, the claim must be considered as a whole, not element-by-element. (35 USC 103, "...such that the claimed invention as a whole would have been obvious...").

(*) Not your attorney, not legal advice, etc.

OK, you're an attorney, I also passed the Patent Bar (and am a Patent Agent). But it was before AIA. I don't think the 2004 Patent Bar had anything about re-examining claims, but it sure as hell did cover PCT, reissues, death of the inventor, and other crucial points of law :)

Now that we've managed to alienate all the civilians, would you say

"means for killing flies" or

"fly-killing device" where you say in the spec what that includes? Or have dependent claims that name the most common fly-killing devices.

> would you say "means for killing flies" or "fly-killing device"

I don't know. Again, I passed the patent bar, but I don't practice in prosecution. So, by my estimate, you have infinitely more experience than me. The answer probably depends on whatever the Federal Circuit most recently said about 112(f).

You might be confusing infringement with patentability.

If an independent claim isn't infringed, then by extension none of the dependent claims are infringed and they don't have to be considered. So the easiest route to defending a patent lawsuit can be proving the independent claims don't apply.

There are two main defenses: invalidity and non-infringement. You're correct, and you're talking about non-infringement.

Invalidity can be proven outside of a full-blown lawsuit (via re-exam) and so it's cheaper. Asking a jury to judge invalidity is asking for trouble, because they usually figure they're not competent to do it.

I actually watched a video of a mock jury, where they debated this. It's pretty shocking how little they understand the law. One juror actually said, "well, that prior art reference -- it's not fair to expect the PTO to have heard of that."

I realize I was wrong about this. Sorry. Too many years away from this stuff.

To invalidate a claim, you have to knock out ALL the elements, not just one.

More comments below.

The F900s that Lucas shot Episode 1 on recorded at sub-1080p resolution, _before_ being letterboxed. That was a long time ago.

> 2k+ resolution camera that records at a 6:1 or more compression ratio.

How does that even get granted? Does the patent office not have domain experts?

USPTO is entirely funded through patent fees. They get more fees if they approve a patent.

No, I am not joking: https://www.uspto.gov/about-us/performance-and-planning/budg...

There's your problem.

That is not the problem. Just because an agency is funded by fees does not mean that it has perverse incentives nor corruption (though it can be the case). After all, the FDA has to assess all of Pfizer's drug applications -- do you want to pay for all that out of your income tax, or should Pfizer be paying it? Furthermore, the USPTO has had the same issues described here long before they were self-funded by congress.

The problem is that that the USPTO doesn't have enough money to hire inspectors and officials. They are woefully, desperately underfunded. The same problems that plague the USPTO also plague the IRS, and for the same reason. The USPTO would do much better if they were permitted to double the fees, or more, for patent application.

> That is not the problem. Just because an agency is funded by fees does not mean that it has perverse incentives nor corruption (though it can be the case). After all, the FDA has to assess all of Pfizer's drug applications -- do you want to pay for all that out of your income tax, or should Pfizer be paying it?

The FDA gets billions in funding from the government and the USPTO does not. I'm not saying you are right or wrong about the problems regarding the USPTO but to compare its funding to the FDA is disingenious and wrong.

Former patent examiner here. The main problem with quality is as you describe: patent examiners don't get enough time. I've written about this before on HN: https://news.ycombinator.com/item?id=31197809

Note that increasing fees alone won't necessarily change anything if USPTO management doesn't give examiners more time in response. For example, the USPTO charges applicants extra if they have more than a certain number of claims (20, I believe), but examiners only get 1 hour extra if there are more than that number. I once had an application with 45 claims... I estimate that I only got around 23 hours total for the first "office action" on that one (most of the work), and that includes the extra hour I got. So I got 5% more time for 125% more work!

Edit: Here's the application with 45 claims that I worked on: https://patents.google.com/patent/US20190160529A1/en

The FDA used to not be funded by the pharmaceutical industry, and instead be directly funded. There are routine questions about how the current arrangement influences FDA decision making in a way that negatively affects their ability to independently review drugs. IOW: the current setup causes the FDA to approve things they probably should not have approved, because they are incentivized to "get along" with the companies paying them.

The U.S. encourages carpet-bombing with trivial patents claiming to be bonafide inventions, because it gives opportunities for market protectionism.

Patent lawyers consider the "not obvious to someone skilled in the art" test done by merely looking at prior art. If it's not previously explicitly patented, it's valid.

According to them. This is exactly what a patent lawyer told me when I asked about how they test for that requirement.

Anything available to the public can be used in a prior art rejection, not just patents. (I'm a former patent examiner and I frequently used "non-patent literature" as it's called.)

That's very cool. Were you ever obligated to grant one of these ridiculous patents, or were you able to use your discretion?

I wasn't around long enough for many to get to the point where they could be granted. I think I only granted around 3 or so. And those ones were unlikely to cause anyone trouble.

I've heard a second-hand story of one examiner who granted a patent that was later strongly criticized. They specifically requested more time from their supervisor to work on it, but their supervisor denied the request. Since they didn't have any valid reason to reject the application, they allowed it. The examiner suspected that it wasn't valid, but they really didn't have a choice as they have to meet a quota and "docket management" requirements. So it's not a matter of an examiner using their own discretion. The examiner knew it was likely a problem but they couldn't find any evidence which was legally enforceable.

Why do patent examiners sometimes grant invalid patents? For the same reason software developers sometimes release buggy software: Management doesn't give them enough time or resources to do a quality job.

I checked what you said about the 967 patent. I'm afraid it's just wrong. You quoted some of the initial part and missed out the bit which actually describes what's happening:

"an image processing system configured to perform a pre-emphasis function on the digital raw image data, to compress the digital raw image data after performing the pre-emphasis function such that the digital raw image data remains substantially visually lossless upon decompression, and to store the compressed digital raw image data in the memory device at a rate of at least about 23 frames per second, wherein the pre-emphasis function comprises a curve defined by the function y=(x+c)^g, where 0.01 <g<1 and c is an offset."

That "pre-empahsis" is a description of Rec 709 gamma. (the dependent claims go on to give the 709 constants for it).

Sure, I can't say - but when was that made part of Rec 709? I suppose that may well be what it invented...

Edit - to clarify better that what I'm trying to convey is that the time and context matters too, and the claims have to be read in light of the description, so suggesting that claims are trivially invalid is something to be cautious about.

Rec 709 was published in _1990_ (and had that gamma structure then, of course!).

All you can say red is doing is reciting the rather obvious combination of a 2k+ camera with rec 709, 23+ fps, etc.

Maybe it's the case that someone can't find a 2k camera being offered to the public early enough to invalidate on a prior art basis-- but I can't see how anyone could conclude from this that a red success would be indicative of anything but a deeply flawed patent system in dire need of reform.

How do prior art pieces invalidate it if they weren't compressed with it?

If you really believe they can be trivially invalidated, then it sounds like you should probably help out the patent office and initiate a patent challenge showing the prior art, so they can invalidate the RED patent?

Hit up https://www.uspto.gov/web/offices/pac/mpep/s2206.html so that the USPTO can actually do the work necessary here because you prompted them to (they're not going to invalidate a patent until someone shows they should. They are backlogged for years on their regular queues already).

If Apple and Sony couldn't fight RED on this, I doubt Nikon can either on the merits of these patents.

Of course, Nikon likely has an arsenal of camera patents with which to negotiate with RED so that might be different.

AIUI (and I ANAL), these patents are also behind the Z-Camera ZRAW format change as of Z-Cam firmware 0.94.1:

https://www.personal-view.com/talks/discussion/25023/z-cam-i...

> "As [the original post] said, ZRAW stores a full RAW, albeit compressed (Huffman and Golomb). This means that the original information about YUV is not correct. In the code, they called the channels that way, indeed, but the channels themselves, as it turned out, store the usual RGB CFA Bayer."

https://www.personal-view.com/talks/discussion/25185/zraw-is...

> "ZRAW is not RAW anymore. After firmware v0.94.1 it's an AES-encrypted MOV-container-corrupted overXORred shit, that contains only HEVC bitstream."

These are the patents:

7,830,967: Applying a gamma correction before compression

8,174,560: Above with a transformation to take a delta between green and the red and blue channels to enhance compressability. Claims are extremely vague on this one.

9,245,314: Lossy compression prior to moving sensor data into memory

9,436,976: Just looks like a combination of the three previous patents

Kodak had prior art on a lot of this for their satellite image processing which isn't technically video but the data rates are in the same realm.

All of which are patents on MATH, which is supposed to be unpatentable.

Talk about bullshit.

Note: I haven't read these patents.

Mathematics by itself is not patentable, but its usage certainly is. (In fact, if that were not the case, perhaps nothing would be patentable given how mathematical nature is.)

Red joins the ever-growing horde of U.S. American patent trolls. It's a real shame.

Yeah, real shame. Nikon's market hardly overlaps with RED's, and never will. This is not "free market competition", but a cheap attempt to suck out money from other companies. Probably the company has already reached the end of its own potential.

I’m not saying that it’s good, but it’s not actually “patent trolling” by definition. That is patent action launched by a non-practicing entity who makes money by lawsuits instead of actually making products, and patent trolls have usually bought patents from someone else (like a bankrupt company) instead of filing them themselves.

These patents are actually core parts of RED’s products (and compressing raw sensor data pre-debayer was actually fairly unique back in 2007 when they started doing it).

What would be the consequences of throwing out all patents that are purely an information theory thing?

These seem to have the biggest impact on our ability to interoperate with complex technologies at scale.

I do find myself wondering what fraction of Red's revenue is IP vs camera sales. No particular reason, just curious -- these patents sound rather profitable.

AFAIK almost no cameras are paying royalties to Red. They prefer to just not implement raw.

But they are collecting royalties from Apple and others...

One would have thought that Nikon has at least one enforceable patent against Red and therefore cross-licensing is the eventual outcome.

This would be hard & confusing to optimize well, make energy efficient but... ideally... we should just sell camera hardware, without the software.

Let end users install & set up their own data-pipelines.

You'd need good application processors with honky gpus, and probably a lot of weird image processing hard blocks sprinkled in.

But relying on companys to navigate the software seas, especially with shitty ass patent minefields for dumb-as-fuck obvious-shit (todays: compress raw before you combine) littering the field, is just sad. This is like the invert of the much desired non-smart TV, one which is good hardware but doesnt rot, doesnt decay, doesnt get crappy revamps. When vendors make software it is rarely "soft" for users, it is fixed & immutable. So just sell the hardware. Have the world be free to figure out & explore as it will what to do with the hardware. Unbundle the software.

(And for god sake someome do something about the piece of shit degitimizing trashfire of ruin & obstruction that is the patent system. Software patents are almost always completely stupid, stop patenting information theory.)

Alleged violations of patents:

- 7,830,967 "Video Camera"

- 8,174,560 "Video Camera"

- 9,245,314 "Video Camera"

- 9,436,976 "Video Camera"

- 9,521,384 "Green Average Subtraction in Image Data"

- 9,716,866 "Green Image Data Processing"

- 10,582,168 "Green Image Data Processing"

Almost all patents use the following generic description or variations thereof:

"Embodiments provide a video camera that can be configured to highly compress video data in a visually lossless manner. The camera can be configured to transform blue, red, and/or green image data in a manner that enhances the compressibility of the data. The camera can be configured to transform at least a portion of the green image data in a manner that enhances the compressibility of the data. The data can then be compressed and stored in this form. This allows a user to reconstruct the red, blue, and/or green image data to obtain the original raw data or a modified version of the original raw data that is visually lossless when demosacied. Additionally, the data can be processed in a manner in which at least some of the green image elements are demosaiced first and then the red, blue, and/or some green elements are reconstructed based on values of the demosaiced green image elements."

IANAL, but this legal description is gobbledygook describing a video camera, that can compress video, in a visually lossless manner, by rearranging RGB data to be more compressible, and then being able to obtain original data from a "demosaiced" version of the RGB data.

RED clearly doesn't like Nikon's use of GREEN.

From what I understand we should we looking at the patent claims to see what the patent really covers.

I think this is straightforward technical language being translated into patent language. But you should always read the claims; the language is usually more precise there.

I believe the technique at issue is a way to compress the data -- using standard codecs like JPEG2000 -- without first de-bayering the image. I imagine it was pretty novel when the patent was filed, which would've been the mid '00s. (The Red One used this technique, and it launched in ~2007.)

The fundamental problem seems to be that in practice, mere novelty is enough to get a patent.

I don't think a company should be granted a 20-year monopoly just because they were the first to write an idea down. I don't see how patents on run-of-the-mill innovations further innovation in the economy as a whole.

Even if there were no such thing as patents, companies would have begun compressing individual channels prior to de-bayering, because it makes sense and is a fairly obvious thing to try. The patent system didn't spur innovation in this case. It just allowed one company to demand a rent from everyone else.

Completely agreed, for the way the patent system actually works in practice.

There is a theory that patent rights can encourage inventors to publish their invention, so that everybody benefits in the long run. This is a nice theory, except it doesn't hold up in practice (at least in the tech industry) because (1) patent language is so obfuscated that it is not a useful source of knowledge; and (2) the threat of triple damages means that no (honest) practitioner in their right mind would ever read a patent other than their own.

It is possible that in other, slower moving industries, people do get some minor value from reading expired patents. Though somehow I doubt it: in our industry, the useful inventions spread through other channels, and I suspect the same is true everywhere.

The application to images may have been novel, but this is just a trivial application of a rule that has been well known since the first uses of compression methods.

Whenever there is some data which passes through several conversion stages, there is a certain point in the chain of transformations where compression is more efficient than in the other points, so it is always necessary to choose carefully where to insert a compression transformation.

For example, if you aggregate some files into an archive file and then you encrypt the archive file, a compression step must be inserted between the concatenation of the input files and the encryption step.

Inserting the compression before concatenation or after encryption will give much worse compression ratios.

The same is true for any chain of transformations. It is always necessary to identify the point where the compression must be inserted, which for image processing in a camera happens to be before debayering.

Absolutely anyone who would receive the task to add compression to an image processing chain of algorithms would start by making tests to determine where to insert the compression.

Discovering the claim of the patent does not require any kind of creativity or any other special skill in the domain. It would have happened automatically to the first one who happened to work at this problem.

You mean corporations don't actually want to innovate once they have significant market share?

Apparently using jpeg2000 to store video (which is redcode) is enough to violate these patents. Too bad Apple couldn't demolish this patent.

Nikon has 100s or 1000s of their own patents, I would guess that they will counter sue for random violatations by RED.

As usual, the lawyers win.

AFAIK the patent is on lossy raw video; JPEG2000 has nothing to do with it.

Nikon will most likely end up having to remove internal RAW support for the Z9. That was a big selling point for me, I received one of the first in Z9 preorders (I'm a professional photographer who does a lot of video production work as well). I work with RED cameras fairly regularly.

RED, while protecting their first to market realm and making sure they get the check, have really stifled where the camera industry can go. Apple, Sony, Kinfinity, all lost out to RED on similar cases.

Assuming Nikon isn't completely ignorant (they've been protecting patents for a long ass time), introducing internal RAW in the Z9 was a red flag on announcement day for this very topic. It gave them the splash they needed to launch their flagship camera, so it's very possible they figured they'd take the win while they can, even if it came with a hard slap for everyone after the fact.

And it's a thin, but very legal argument to say they compete in the same space. No Z9 owner in a professional space is looking to buy a RED really. If there are, it's probably less than 10 in the world.

A huge part of me wishes people just wouldn't buy RED cameras in protest to how big of a road block they've created and the business practices they've employed themselves.

And Maybe the rest of the industry just collectively gangs up on them to apply pressure. One can dream. Right now, the only way this ends is if people talk with their money, or the patents expire.

I cannot think of a better way to actually impede humanity's progress than to put up artificial barriers to techniques developed to help spur innovation and growth. The patent system is mind numbingly stupid as it absolutely STIFLES innovation.

On the other hand, who would research anything or spend anything on R&D if it was not protected? It's a two-way street. There are good patents, bad patents, and everything in between.

Most of R&D in the medical/pharmacological field is done by government grants as an example.

Do you have a source for 'most'? Pharma R&D spending is very high compared to any other industry.

https://www.cbo.gov/publication/57126

https://www.ineteconomics.org/perspectives/blog/us-tax-dolla...

The raw dollar value R&D in the private sector is higher, but that's accounting gymnastics for tax purposes, not actual R&D.

Then the competitors that don't care about lack of patents will take over the market, creating the impetus for innovation again.

Not really, there will be small innovation on obvious matters but any R&D that actual takes capital to validate or does not have immediate result will simply be dropped.

Or kept purely secret

I thought competition is what keeps markets working efficiently. Without competition, what's the motivation to actually share innovation when it is not needed to maximize profits, as innovation is much more costly than maintaining the status quo?

> who would research anything or spend anything on R&D if it was not protected?

Most people that do real R&D do it because they want to, not because they're rewarded with patents. They want to make a better camera, a better camera format, and sell it. And nobody learns how to make something by reading the patents, in the past 50+ years they're practically unreadable legalese. Once upon a time they had diagrams for mechanical devices or chemical processes, but now they're just ideas that many people faced with similar challenges and available technologies would have considered, except the first to bother to patent a general idea like "compress raw sensor data before debayering" now gets to be a massive pain in the ass to people actually doing R&D to make things.

I think this is true on an individual level, but a lot of high tech innovation is only possible with hundreds of engineers working together full time for years. Nobody is inventing EUV lithography in their garage, someone had to spend hundreds of millions of dollars assembling the necessary talent and providing the resources needed to get it done. That someone expects a return on their investment, otherwise they will find something else to do with their capital.

What we really need is much more strict requirements for what qualifies as a truly unique innovation worthy of patent protection. None of this BS about virtual shopping carts on eCommerce sites, or playable minigames on video game loading screens.

https://www.businessinsider.com/how-a-22-year-old-makes-micr....

> Sam Zeloof, a 22-year-old American, produced a chip with 1,200 transistors

This is an impressive feat to be sure, but to compare it to modern EUV lithography is like comparing a crossbow to an ICBM

Why can I not reply to the person below me?

Regardless, anyone could do this type of fabrication in their garage given removing the legal and technical barriers that keep EUV technology behind lock and key.

What that guy is doing in his parents garage is not EUV lithography. He's doing what early IC manufacturers first accomplished in the 60s, only with the benefit of 50 added years of knowledge in the space and decades old equipment that never would have been affordable to the average tinkerer when this technology was cutting edge.

The idea that the risk of patent infringement is the only thing preventing him from building a chip comparable to a modern Intel or AMD CPU in his garage today is absurd.

That’s not actually true. Setting aside the fact that to build an EUV lithography system costs over one hundred million dollars (a single system ships from ASML in 40 shipping containers and has over 100,000 parts) [1], Sam Zeloof could legally build one tomorrow if he was able - he just wouldn’t be able to sell it unless he licensed patents.

1. https://www.cnbc.com/amp/2021/12/10/asmls-high-na-euv-lithog...

Exactly. It's not like it's literal magic. It's just knowledge and resources, which should be getting more abundant and cheap, not scarce and expensive.

I also think a shorter duration and/or a cap on total royalties would be useful. Like 10 years and 10x initial r&d investment, instead of "forever" and "infinite".

> I also think a shorter duration and/or a cap on total royalties would be useful. Like 10 years and 10x initial r&d investment, instead of "forever" and "infinite".

For what it's worth, patents are limited to a 20 year duration. This is (was) fine in some slower moving industries, but completely ridiculous for something like consumer electronics or software.

> Most people that do real R&D do it because they want to, not because they're rewarded with patents.

Everybody's gotta eat. Would you work your job if you weren't getting paid?

Your question shows a lack of understanding. Many billionaires do not need to work. What keeps them showing up to "jobs"?

Of course - billionaires, the quintessential example of the motivations of the common man. When I think, "a person under the same set of constraints as Joe in R&D", I think of Jeff Bezos.

So this statement then admits that there are differences in how people are treated and what they are able to do based on religion? No, race? Again, missing the mark?

Ahh, MONEY! Which is as made up as it gets. This person is worth more than that person because this computer says so and it has to be TRUE, or else society wouldn't work. We are about to find out real quick how the world works again, and I am warning everyone here, if you think you are safe because of money, the world is going to humble you.

Sheer...human...curiosity. When everyone can afford a comfortable lifestyle, what else do you think humanity will do? Drink themselves to death? If that were true, we would already be long gone.

It seems to me there was a whole hell of a lot of invention and progress prior to the advent of patents. Perhaps the same things that spurred that innovation?

It is more like what should be allowed to be patented and what should not, how long should patents last, etc.

Besides, patents are not the only way to protect R&D.

There has been mindboggling investment in machine learning over the last few decades, yet almost nothing is protected by patent (even where patents exist, they're purely defensive). Tesla's patents were voluntarily neutered, so anyone can use them - hasn't stopped Tesla from investing in R&D. Microsoft has done something similar.

It's simply not true to say that innovation or R&D would grind to a halt if patents ceased to exist. If anything, we might even see the complete opposite, because it massively lowers the barriers to entry.

Bad patents and their negative impact on the world far, far outweigh the positive impact from the good ones (if there is such a thing). Scrap the whole damn system.

It's worse than that. By allowing patents that are supposed to be illegal (like those on math and things that are obvious), the USPTO abets companies in stealing people's work.

Why not use BlackMagics cameras?

cause they aren't reliable, or ever available.

Did RED or Nikon get them from Pied Piper?

Just use QOI and be done with it. https://qoiformat.org/

QOI is patent free, simple to implement and fast.

Previous discussions here https://hn.algolia.com/?q=qoi+image

The amount of human waste generated by patents and the resulting machinery surrounding them, entire ecosystems of support that self perpetuates this insanity. The human race cannot advance until ideas are exchanged freely and scientifically and not via combat.

BTW my parent comment was at -2 when I posted this.

Should have put their patent infringing software behind some DRM. RED finds out Nikon is violating their patent, sues Nikon, Nikon makes a strategic campaign donation, some people at RED end up being felons. God bless America.

Seriously, if I infringe a patent in some DRMed software, can anyone ever legally discover that fact?

>"Seriously, if I infringe a patent in some DRMed software, can anyone ever legally discover that fact? "

Yes, RED can file a suit, and 'discover' the software. The legal system is not some arcane and buggy piece of code.

Last gasps of a dying industry, even if they don't know it yet.

Future movies will be filmed in mocap + multisensor photogrammetry and scaled up with AI techniques. (My startup does this.) Cost and time savings of not doing location scouting, set dec, lighting, and blocking are huge. Plus you can edit way more than a static grid of photons in post.

Consumer smart phones use computational photography and sensor arrays to work magic.

$100,000 glass optics are dying. They'll maybe find future use in industry and research, but the entertainment field will move on.

That sounds like a terrible film experience.

Where can I read more? Also, can you point to a movie/scene that used mocap? Would love to watch that.

I guess we’re pushing the point with modern blockbusters (Marvel/Star Wars) where more of the movie is animated then not. And Avatar is inverse Who Framed Roger Rabbit at this point.

Arn't AI poses estimates more likely to become the future rather than the white balls motion cap suits from the 90s ?

I don't know. I think there's a big space for AI assisted XYZ everywhere in life, but to me that doesn't mean it would replace the more 'genuine'/'analog' stuff like good lenses. Even for entertainment and personal use.

You do understand that this is why people don't like contemporary movies, right?

I thought it was because the obnoxious rise of sequels, reboots and overdone cliche series.

People do like contemporary movies though. You might not, but if people didn't like them they'd lose money and studios would go back to what made money.

> You do understand that this is why people don't like contemporary movies, right?

Though I strongly prefer independent films, people "vote with dollars". Take a look at Disney.

The problems with filmmaking are the institutional capital, planning, and complicated logistics required to create a film. There's only so much money to make movies, not everyone gets a shot. This is why generic "for the masses" films get made and get theatrical release placement. This is why we have more Twitch streamers than filmmakers.

It will change.

Yeah, as much I think this guy sounds crazy, this is absolutely a fair point.

Reducing the need for films to spend $10m+ on production would mean all kinds of cool shit could be feasibly created.

Look at the video game space where we have indies producing all kinds of complex, 3D games through tools like Unity.

Interesting take that I haven’t heard before, outside of like Diamond Age.

If you were a betting man, what would your timeline be? When does computer generated imagery surpass camera footage when representing reality? When does a drama shooting in Toronto pretending to be New York just render the whole thing? Finally, do you think performances will be noticeably worse and we’ll all just live with it, or is the whole idea of actors having a harder time acting with greenscreens/mocap sets overblown?

Do you have any background in movie production? There is more to that industry than VFX tech. Yes, virtual production etc are ascendent, and growing fast -- but most of the production spending is still using fancy optics and traditional production. (And even for virtual production, the actors are still people.) I think this is a typical straight-line projection of the change in the industry that fails to account for the actual underlying dynamics.

But I wish your startup well. I hope you've hired filmmakers, directors, producers so someone on your team understands how movies are made today.

(Plus: there's lots of money to be made in downmarket production. More money is being spent on video production today than ten years ago. The industry is not dying.)

> there's lots of money to be made in downmarket production

This is almost certainly what the commenter is targeting. I'd guess that and/or the adult film industry.

I don't think so -- I think it's a reference to Gollum, the apes in "Planet of the Apes," the de-aged Moss Tarkin (sp?)... basically CG people. That work is almost entirely at the high end, because it takes a lot of digital art work to make them look realistic.

This is moving down market, fwiw, but it's not disrupting traditional moviemaking. It's a fast-growing but tiny niche that I believe will saturate before it comes close to replacing the entire industry. (Full disclosure: I work with cameras professionally, so maybe I'm not an unbiased observer of the industry.)

[ edit for clarity ]

> Do you have any background in movie production?

Yes, but that shouldn't matter.

> most of the production spending is still using fancy optics and traditional production.

It used to be shot and physically spliced together celluloid, too.

Optics are punch card technology. The art, and indeed the end goal itself, is storytelling. The technology is just a means to an end. A grid of photons can be crafted and manipulated much easier with new techniques than by trying to put those photons in the correct places by hand. We're going to look back at optics and glass as the dinosaur age.

> And even for virtual production, the actors are still people

For now. Models can generate animated poses. This is moving faster than you're giving it credit.

(Even when robots replace us, people will still want to act - no question. The technology will still afford that. But just like ADR can now be model-based, so can posture correction.)

Your average ten year olds will be making their own Star Wars films by the end of the decade.

>Cost and time savings of not doing location scouting, set dec, lighting, and blocking are huge

Yes this worked awesome on the Star Wars prequels. Some of the most natural and real acting I've ever seen. /s