Comment by ktizo | Hacker News Reader

Comment by ktizo | Hacker News Reader

Parent Full thread

All the cited apple patents on multitouch gestures appear to be filed since Jeff Han's TED talk - http://www.youtube.com/watch?v=QKh1Rv0PlOQ - which clearly demonstrates many of the patented techniques. I find it really hard to understand how any of these patents were awarded.

[edit] Some of the quotes from his 2006 presentation are quite relevant to this...

... "Now, multi-touch sensing isn't anything- isn't completely new, I mean, people like Bill Buxton have been playing around with it in the '80s." ...

... "Now this is a photographer's light box application. Again, I can use both of my hands to kind of interact and move photos around. But, what's even cooler-

(uses fingers to 'grab' two corners of one of the photos and 'pulls' it to full screen size)

is that, if I have two fingers, I can actually grab a photo and then stretch it out like that really easily. I can pan, zoom, and rotate it effortlessly.

(slides piles of photos around)

I can do that grossly with both of my hands,

(pulls photo out of stack & enlarges it)

or if I can do it just with two fingers on each of my hands together.

(grabs empty space around photos & zooms in and out of canvas)

If I grab the canvas I can kind of do the same thing- stretch it out- I can do it simultaneously, where I'm holding this down-

(holds pile of photos down while pulling out another)

-and gripping on another one, stretching this out like this.

Again, the interface just disappears here. There's no manual. This is exactly what you kind of expect, especially if you haven't interacted with a computer before." ...

Which sort of begs the question that if an expert in the field thinks the gesture is exactly what you would expect, even if you had no expertise whatsoever, then how does that not qualify as obvious?

“All the cited apple patents on multitouch gestures appear to be filed since Jeff Han's TED talk”

FingerWorks, a pioneer in multitouch gestures, filed for (and received) boatloads of patents. Apple bought FingerWorks in 2005, inheriting its intellectual property. The scientists came to work for Apple and continued to file patents for the research that they had done since the nineties.

“In June 2005, FingerWorks officially announced they were no longer in business. The founders continued to file and process patents for their work through late 2007. And as of August 2008 they still filed patents for Apple, Inc.” [1]

[1] http://en.wikipedia.org/wiki/FingerWorks

Beyond that, we need to consider priority dates when determining when something was 'invented' when it comes to prior art.

Note the application referenced downthread (20060238520) is a continuation of a 2004 filing, itself a continuation of a 2001 filing, itself a divisional of a 1999 filing which claimed priority based on a provisional 1998 filing. [1]

So to challenge that application, you'd need to find art circa at least 1998. And even then you have to consider the USPTO's (now legacy) practice of recognizing date of invention, rather than filing. Which means you may have to find art even older than 1998, to get that bit invalidated.

[1] Patent applications are rarely straightforward affairs. The process of mapping what we colloquially think of as "an invention" (awesome new widget) to patent applications often entails the creation of a family of patent applications. It gets very messy, very quickly.

So this expires 1998+20=2018?

No, it expires based on when it was granted not the priority date. However, different parts of the same patent can have different priority dates if they where not in the initial filing. So, some prior art after 1998 may still be relevant.

This is why a lot of patent trolls would drag their feet as long as possible in the hopes that someone would actually research, prototype and bring to market what they've "invented". These so-called submarine patents then surface to sink the company who's done all the work.

It's my understanding that the USPTO is trying to limit the ability of companies to do this though.

^ Wayne Westerman, 1999. Hand Tracking, Finger Identification, and Chordic Manipulation on a Multi-Touch Surface

http://www.ece.udel.edu/~westerma/main.pdf

Edit:

Ref, note [1]

http://en.wikipedia.org/wiki/FingerWorks

Wayne Westerman is one of the identified Inventors. Published papers/articles by the inventors can count against their being able to get a patent, dependent upon the character of the disclosure (did they tell a couple friends or write a widely circulated article? etc) and how long before the filing that disclosure was made.

Since this appears to be dated after the priority date, I don't think it would pose any problem, even if it were published in a professional journal.

Software patents like this should never be granted. Any kind of gesture we naturally do in the real world quite obviously can be rendered in software. There's nothing "invented" here. Microsoft was similarly granted a patent for grasping an object in 3d. If every possible human action can be granted a software patent, then computing in the future is going to become awfully ridiculous.

I was curious about the difference between Han's technology, and the iPhone's. In Han's words, his device is pressure based: http://www.youtube.com/watch?v=DZv1nBM8zsI (he also believes that the scale of the devices puts them into completely different categories.) I read up a little on the differences between pressure and capacitive devices, and one that stands out is that capacitive surfaces can emit more light, because less layers are required beneath the glass.

(Edit: I meant to post this as a reply to the grandparent comment, not the parent post about FingerWorks. Whoops.)

Hans device is light based and makes use of frustrated total internal reflection of infra-red light being shone into the edge of the touch surface that is then picked up by an infra-red camera that is sitting next to a projector underneath the table.

Just had a look at their patents, and found this line in their gesture patent -

Identification and classification of intuitive hand configurations and motions - http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Se...

How is the use of intuitive gestures not obvious, entirely by definition?

When looking at patents you need to go beyond the title and read the full patent. It rarely broadly covers the title description.

There is a big difference between dreaming about something and actually doing it. For example, everyone knew what a flying machine was, centuries before it was invented. Clearly this doesn't mean that creating that machine was actually obvious. With computing it is similar, there are lots of "inventions" dreamed about by science fiction writers that are simply not possible today. When such innovations become possible, they will need to be protected.

And yet the wright brothers were not granted a patent on "things that fly". They were granted patents on some specific machines they invented that could fly. But the patents were about the mechanical machines themselves. If someone else had come along and invented a totally different means of flying (like a helicopter) they would not have been infringing on their patents. This is a patent on "things that fly".

Apples patents are clearly not patents on "smartphones with a rectangular touchscreen and no external UI buttons", but they cover many inventions that make such a phone usable.

Likewise, apart from a totally different kind of flying machine like a helicopter, there probably isn't any better method of controlling an airplane than the method covered by Wrights patents (since it specifically does not exclude non-wing warping control designs), which is why the basic mode of control is still in use today. So Wrights patents at the time might as well be called a patents on "things that fly". It was very difficult to make a controllable plane without violating the patent.

Likewise, unless you equip a phone with dedicated zoom controls, there probably isn't a better or more intuitive way of zooming than using pinch-to-zoom or double tapping.

> Apples patents are clearly not patents on "smartphones with a rectangular touchscreen and no external UI buttons", but they cover many inventions that make such a phone usable.

Actually, per my understanding, Apple's design patents are exactly that (if you throw in beveled corners). It's true, though, that those aren't the patents at issue in this thread.

Yes, but design patents are a different animal, and AFAIK much easier to work around. So if they changed just one aspect of the design (making it less likely to be confused), Apple would have a harder time enforcing it.

Lots of devices do not infringe on Apple's iPhone trade dress (ie, design patents). Devices that sell millions of units a quarter.

>When such innovations become possible, they will need to be protected.

The underlying implementation details should be protected, sure, but not the idea itself. Just because you're the first to successfully make a flying machine doesn't mean someone competing with you, implementing it in a different underlying way, shouldn't be able to make a flying machine.

Under that argument shouldn't the patents apply to the capacitive touch surfaces and not the gestures?

"Method for transportation by flying vehicle" is very different from "Construction method of flux capacitor that provides enough energy to power flying vehicles". The first is obvious and there is prior art even if people could not build it. The concepts of pinch-to-zoom or slide-to-unlock are obvious to even people without Phds in touch surface technologies.

Anything intuitive by definition should not be patentable.

A multi-touch screen, like a design for an airplane should be patentable. It actually takes serious R&D and discovers things that were not understood before. Obvious things you can do with multi-touch screen or airplane should not be patenable. Any programmer that had a multi-touch screen to play with would come up with pinch to zoom. Allowing pinch to zoom to be patented is like patenting aerial photography, an obvious use of an airplane.

I think the important point is that "obvious" has a very precise meaning for patents, and it's not what you think it means. . There was a great IAmA on reddit with a patent examiner that's worth reading: http://www.reddit.com/r/Android/comments/ww982/iama_patent_e...

This needs many upvotes. That IAmA is rather worth reading, with the in-depth responses of the two patent examiners to questions. For example, I think most people here wouldn't know that the "obviousness" of a idea is determined by whether previous patents or prior art can be combined to produce that idea (and not by whether a professional in the field would judge it to be obvious, or any other standard). Furthermore, prior art can include products/art from anywhere in the world, patented or not.

Some great questions and comments there too. I like the one by futurespacetraveler, apparently also of HN, who said this:

---

The price we pay to incentivize an inventor to disclose their invention is with decades of protection from competition. For a utility patent, that's 20 years from the date of the patent grant. And we always pay the same price, regardless of the underlying value of the invention to our society. So whether you invent a new method of swinging in a swing or a new drug to fight cancer, we pay the same price to know how it works.

And I think that's one of the key perspectives that often gets overlooked. We are paying for knowledge. The inventor is supposed to have figured something out that even skilled practitioners in the art hadn't thought of. We are paying for the secret sauce, so to speak. I think sofware "inventions", for many of us skilled in the art, seem so obviously non-insightful that we can't believe society has paid for such "knowledge" with a patent. Many of us realize that we don't even need to read most software patents to learn how to do what it explains. We're paying for "I was here first", rather than "I finally solved the problem no one else could". Yet we pay the same price regardless. That's why the Defend Innovation site wants to limit software patent terms to 5 years. Patent protection should be proportional to the value of the invention. But I believe that should be true for any invention.

Patent protection should be proportional to the value of the invention

He was making sense up until that point. The value of patent protection to society is determined by the potential harm done to the progress of science and the useful arts that would result if the IP had been kept as a trade secret. In the case of pinch-and-zoom gestures the net benefit to society gained by allowing Apple to own the idea is demonstrably nonexistent.

Apple sold a hundred million iPhones before they ever set foot in a courtroom. Clearly patent protection was not necessary to allow them to recoup their investment a thousand times over. Meanwhile, the harm done to competition by granting them a 20-year monopoly on the basic UI paradigm is easily estimable in the trillions of dollars, if they succeed in enjoining Samsung and other manufacturers.

Trillions of dollars ? Are you serious ?

I can think of at least 4 different ways to implement zooming without using pinch to zoom. So this idea that it is worth trillions is bizarre.

I, too, can think of at least 4 different ways to implement zooming that would suck and not sell very well.

You seriously don't think that Samsung's line of smartphones would bring in a trillion dollars' revenue over 20 years, if left unmolested by Apple?

I believe the patent is for a specific implementation of identifying those intuitive gestures and generating corresponding user interface commands. That's what follows in the description of the algorithm and circuitry.

It isn't. Which is why it has this line in it -

[0046] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.

The only things that matter with regards to patents, are the claims.

That line, and the general and detailed descriptions can be ignored entirely.

The claims for the pinch-to-zoom patent are broad enough to cover pinch to zoom on any touchscreen.

The claims as-filed, sure. But the USPTO doesn't generally update the published application to match the current document (including changed and struck claims) during the process.

So it's entirely possible that wording didn't survive.

In any event, it's hardly fair to judge a system based on the snapshot of a submission from before it had even gone through that system yet.

You can easily read all of that in the public PAIR database, it's not hard to find.

Btw the wording did indeed survive, the patent as it stands is sufficiently broad to cover all pinch-to-zoom implementations on any kind of touchscreen.

Many patents are "obvious" - after the fact.

Many patents are "obvious" before the fact, thus prior art.

Show us a patent that is not obvious.

You're kidding right? Sure, I'll dig through the patent lists for a few hours digging up the hundreds of examples of patents that were creative new ideas for their time. Very little of them will be software patents.

But this isn't a discussion about patents like that. This is a discussion about patents being granted for ideas that are simply innovations and/or copies of pre-existing ideas. That's completely different.

Not every patent is of equal creativity, importance, and value.

Being obvious doesn't make it prior art alone. If that were the case, Shazam would owe me millions.

I'm saying the patent was obvious before the fact because of prior art. Just because someone was the first to patent something doesn't mean that they can claim it was not obvious before their patent was issued.

If you had prior art that invalidates a patent it doesn't necessarily mean the owner of the patent owes you money.

If "intuitive" is so damn easy why are so many gadgets and appliances so hard to use?

Perhaps partially because you can patent intuitive.

It’s easy to use a good UI—not to create one.

Don't finger print readers violate this patent?

Please correct me if I'm wrong.

Apple probably got a patent on using those gestures on a handheld device, which no-one had demonstrated before because the touchscreen technology required to achieve it wasn't available yet. Apple didn't invent that technology, any more than they invented the gestures themselves, but because they were the first ones to take the obvious step of combining the two they get to patent it and have a monopoly on it for the next two decades.

I suppose it is a bit like wondering how Boeing got to own using the moon's gravity for orbital maneuvers. - http://www.space-travel.com/reports/Boeing_Patent_Shuts_Down...

I should stop looking for signs of any kind of sanity in the decision making processes of faceless bureaucracies, as it is entirely pointless and only causes me stress.

Industry sources have told SpaceDaily that the patent is regarded as legal "trite", as basic physics has been rebranded as a "process", and that the patent wouldn't stand up to any significant level of court scrutiny and was only registered at the time as "the patent office was incompetent when it came to space matters".

thanks for posting this. quoted text from the link.

I'm surprised Samsung didn't use this at the lawsuit. Or did they? Because I find this evidence of prior art better than that touch table they showed.

The jury completely ignored prior art anyway, because it was "bogging them down."

That's breathless headline BS. The actual statement is quite clear that to keep making progress they'd move on from what was bogging them down and come back to it later.

The actual statement is ambiguous:

After we debated that first patent -- what was prior art --because we had a hard time believing there was no prior art." "In fact we skipped that one," Ilagan continued, "so we could go on faster. It was bogging us down."

Unsurprisingly, the ambiguity leads to people reading what they want from it.

Granted, this excerpt seems ambiguous at first. A different quote had the phrase about coming back to it later. There's more context here though:

"We weren't impatient," Ilagan said. "We wanted to do the right thing, and not skip any evidence. I think we were thorough... we took our time. We didn't rush. We had a debate before we made a decision. Sometimes it was getting heated."

Even without context, what you quoted is not really that ambiguous. He says they debated it, so much debate that it was "bogging them down". That's not skipping it in the sense most people think skipping it means.

So "skipped it" in his words seems to just mean they moved on. Then since the question was answered on the verdict, they did clearly did come back to it to form a group consensus later.

I'm interested in this other quote that has the phrase about coming back to it later... Do you have a link?

>> A different quote had the phrase about coming back to it later.

Which other quote?

More detail: http://news.cnet.com/8301-13579_3-57500358-37/exclusive-appl...

http://en.wikipedia.org/wiki/Begging_the_question#Definition

Sorry but I'm an asshole. You are using it wrong.

It depends on whether or not he meant it in the definition of a logical fallacy or in colloquial terms. Colloquial language, if properly understood in context, is not incorrect. In one episode of The Simpsons, someone says something to Fat Tony that he doesn't immediately understand, but (to paraphrase) he replies with "In context, it is clear what you mean."

Using "begs the question" to mean "raises the question" is a perfectly cromulant application. Latin may be a dead language, but English is alive and ever-changing.

Probably, but it still sort of fits this way round and it all seems to work. Let me know if any bits fall off and I'll give it a push and see what it does.

This is just a feeble attempt at discrediting Apple, pathetic really. "how does that not qualify as obvious?" If it was indeed so obvious for so long why did we have to wait until Apple did it for every one to realise it? Either Apple innovated or everyone else is an idiot for not doing what was obvious.

In either case the me-too idiots didn't realise how to implement control interfaces for touch screens until Apple showed them how to, at which point they tried to copy what the bright kid had done.

The point isn't that Apple didn't innovate.

The point is whether Apple's ideas are worth granting a monopoly over.

I recall playing with a Microsoft Surface the year before the iPhone got released, and lots of the mechanics were very similar. Pinch-and-zoom is the 'one-click-shopping', or the 'press tab to move to next selectable UI element' (also patented) of the touch interface; if you're being paid to develop it, you'd think about that concept in about five minutes.

Furthermore, do you really want to live in a world where Apple gets a monopoly on touch gestures? Describe a different zoom-in mechanic using only two fingers.

So you're basically saying that Apple innovated but it shouldn't be granted a protective patent. Isn't this what the patent system was created to do? I see a lot of complaints about when patents are wrongfully used, but this is the first time I see someone complaining when they are correctly used.

The story here seems to be that Apple did the right thing buying a company that developed an innovate technology. In other words, the small company the created the technology was compensated by its development. Now, you think that a company like Samsung can just say "no, this is obvious" and start using that same innovation without any compensation or agreement? This is what seems crazy to me.

> Isn't this what the patent system was created to do? I see a lot of complaints about when patents are wrongfully used, but this is the first time I see someone complaining when they are correctly used.

I'm saying the system is broken - not an uncommon argument here on HN - because I don't think these are ideas worth protecting.

Did you spend $10 million dollars investigating the right kind of chemical coating to get glass to respond to finger tips? Maybe you can get a monopoly on that.

Did you have an engineer dick around in a lab for a week? Why are we protecting this? We're basically rewarding incumbents.

Being a software engineer I obviously can't comment on other fields. But it strikes me that a huge proportion of these patents were awarded on incredibly specious grounds. I know innovation is obvious in hindsight but… I think these kinds of wide-swath patents destroy competition which harms all of us.

Patents as they stand today destroy more social value than they produce.

You are creating a purely rhetorical distinction here. There is nothing that can be so complex that it cannot be replicated by someone else -- that is the whole definition of engineering.

What you're saying that is that if it is innovation in materials research then it is valuable (probably because you and me don't know anything about it). But software engineering is not, probably because you could (in theory) do something similar. Let me tell you: if you were the one doing the design of a motion sensing system, I can bet it wouldn't be "easy". And you would want some kind of protection for your efforts. It is too simple to disregard somebody else's effort when making a hypothetical point.

> if you were the one doing the design of a motion sensing system, I can bet it wouldn't be "easy"

WELL… I never took any visual recognition courses in university. But there is a large amount of literature on the subject. I think there may have been more innovation producing cheap infrared 3d sensors that work reliably than converting that vector field into a bit of math you can pattern match - but what do I know?

We're disagreeing on the "obviousness" of these patents.

>The aspect of a patentable invention most difficult to judge is obviousness. An invention is patentable only if it is not obvious to a person who is typically educated or trained in the field of the invention.

http://www.uspatlaw.com/contant.php?id=3

I'm trained in the art and I think pinch to zoom is obvious. It would be the first thing you try right after scrolling.

At the end of the day, I think the USPTO has been far too lax in what they consider obvious or not obvious. And yeah, frankly, I think a stunningly large percentage of all software patents are 'too obvious'.

None of the examples of patent trolling we've seen are really 'misuses' of the system.

Let me tell you: if you were the one doing the design of a motion sensing system, I can bet it wouldn't be "easy".

This sentence makes no sense. The patent in question is not on a single technical implementation of pinch-to-zoom; it is on pinch-to-zoom in general. It does not merely protect the (easy) engineering needed to detect two fingers pinching, it protects the idea of "two fingers pinching = zoom". This is an obvious idea, and should not be protected by patents.

>So you're basically saying that Apple innovated but it shouldn't be granted a protective patent. Isn't this what the patent system was created to do?

It was created to encourage R&D by granting those who come up with complex implementations a window in which they can recoup the expenses. It wasn't created so the first person to implement an idea in general can have a competitive advantage.

If companies had used the patent system to create artificial monopolies in this fashion throughout the development of personal computing, we'd be living in much less functional world.

The very existence of personal computing disproves your point. The patent system is not a new invention, and there are thousands of patents in every step of this development. Each company creates patents and then licenses them to other companies interested in the technology.

>The very existence of personal computing disproves your point.

Hardly. Of course the patent system isn't new, Apple just isn't behaving like personal computing companies have historically behaved. Licensing complex things like algorithmic implementations hasn't been uncommon, but licensing obvious UI ideas has been.

Apple licenses 1-click shopping from Amazon... see how the game is played? It's real simple.

I did further down: http://news.ycombinator.com/item?id=4443294

Personally I think it's better too.

A bit more like, Apple buying the rights from a group who was patenting what was being done in the field at around the same time as other people were also building similar systems.

Reply on news.ycombinator.com