Project Gameface launches on Android | Hacker News Reader

Project Gameface launches on Android

developers.googleblog.com

185 pointsxnx66 comments

Even as a currently able-bodied person it's interesting to think about applications for this. I downloaded the Windows app and did a quick binding of mouth left and mouth right to my arrow keys so I can effortlessly flip through image galleries. (Couldn't figure out how to turn off cursor movement entirely though.)

Absolutely. DCS is a plane combat simulator, hevely relies on head tracking for immersion, and the UX is literally you clicking buttons and some are below the HUD and left and right so you need all the head degree of freedom just to look around

That doesn't leave much space to zoom in and out which is a issue because visual acuity is greater than monitor fidelity so some degree of zoom is required to keep tab of adversaries

It'd be awesome to have a "squint" gesture to zoom in

Accessibility usually benefits lots of able-bodied people. A lift or long enough ramp helps pregnant people, those with broken legs or the elderly instead of just wheelchair users. Decent UI design with good colours helps not only the visually impaired, and decent keyboard navigation can empower your power users instead of just blind people.

It's a shame so many people still see accessibility through a rather narrow lens.

Issue #1 is still an interesting albeit sad read https://github.com/google/project-gameface/issues/1

With Laurence's response it doesn't sound like he stole anything at all.

Yes, people should read the whole thread instead of jumping to conclusions after the first post.

But it does seem like there was an opportunity to collaborate?

I think there's a gap between "opportunity to collaborate" and "my life is hell and you stole my idea"? If something is open source (and licensed appropriately) then even collaboration becomes moot. There are a lot of takers in the world, and not many givers.

To what end though? They’d exist at completely different ends of the technical stack to the point where it’s likely very unusable for the final product. Even though they do similar things, it’s very doubtful collaboration would have netted anything.

> Since the issue was raised on HackerNews -- my life has been hell. Not an hour goes past without some kind of personal attack that ranges from a shaming to an outright physical threat. It has affected my livelihood, my career, and my health. It destroyed a family vacation when I was supposed to unplug. Nobody deserves that, and certainly not someone who is innocent of what they are accused of. Please be aware of that should you ever feel emotional again.

We are toxic and we're not seeing it. We should stop posting a link to that thread imo.

Oh he very much deserved it. We need to discourage the culture in tech where people set up VC-style "conversations" with the intention of scoping out ideas.

The Winget product manager, who stole the Appget product idea, on the other hand probably does deserve the blowback

https://news.ycombinator.com/item?id=23331287

I agree to the sentiment, but not that he should be harassed for it. It seems that too many of us are happy to hide behind the corporate veil and be less than human when it comes to treating others fairly. Google KNEW of this guys vulnerable position, had him consult on an issue, then released a (very similar) competing app (and we all know where it will be in 3 years - at least its open source this time). I get that these things happen in the corporate world every day, but that doesn't make it right.

If COVID taught me anything it's that it doesn't matter where you go looking on the political spectrum, be it left or right, you'll always find enough people that are fine to be the judge, jury and executioner.

I think lockdowns deteriorated a lot of peoples mental health and we will only see the consequences of not addressing it over the coming decades.

As it stands now, that thread is a very clear cautionary tale. It is posing links to the next ongoing, emotionally charged, accusation we need to be cautious of.

That's exactly an example of how our world becomes immediate and binary.

In a world that tries to be pluralistic we end up judging in a binary way. The classical good v.s. evil, where there's middle ground (to the story).

It's really complex to judge and we're in an era that everything becomes judgmental quickly.

There are many horrible stories of stolen ideas by big tech companies. Some are very nasty.

On the other hand, even at my current workplace, sometimes at trade-shows or emails, someone approaches and suggests something that's similar to what we're working for many years (but yet to release).

Even Alexander Graham Bell had a patent conflict when first registering his invention, not to mention Tesla :)

Exceptionally so when you realise that the end result was him deleting his projects and his account from github.

This is just another example of google bulldozing into a space, and eradicating the competition. And for what? Another product that people will depend upon that will be in the graveyard within a few years.

Sure, this isn't lmoroney's fault per se, but this WAS a consequence of his actions as a googler. He is WORKING in a position which has great power (he is not an individual releasing a product here, he is google), he should exercise greater responsibility - especially when consulting with someone who he already KNEW was in a vulnerable situation and then using your massive backing to release a competing product (which just so happens to be a block based facial gesture workflow builder). The imbalance of power here is colossal. Its literally individual homeless guy vs google.

We should all do better. Hiding behind the corporate veil shouldn't be an excuse to be less human.

I sincerely hope the poster is getting help with his mental health and is in a better position than he was last year.

here are the key facts of what happened:

1. Oz Ramos developed an open source project called Handsfree.js starting in 2018 to help make computing more accessible, inspired by helping someone at a homeless shelter recovering from a stroke.

2. Around 1-2 years ago, Laurence Moroney from Google asked Oz about using Handsfree.js for accessibility. Oz showed him demos of Handsfree.js and related projects like a gesture mapper.

3. Oz asked Laurence for support and collaboration, but felt Laurence ghosted him after getting information about his projects.

4. Google later released Project Gameface, which has similarities to Oz's work in using computer vision for accessibility. Oz felt Google stole his ideas instead of supporting him.

5. Oz posted about this publicly on GitHub, accusing Google and Laurence of stealing his work when they could have supported his open source efforts while he struggled with poverty and mental health issues.

6. The post gained attention on social media and Hacker News. Many criticized Google based on Oz's account.

7. Laurence responded stating Project Gameface was inspired by Lance Carr, not Oz's work. He explained Handsfree.js did not solve a specific technical problem he inquired about.

8. Laurence said he could not directly financially support Oz the way he requested, and directed him to Google's formal processes for charitable giving and support.

9. Oz apologized for accusing Laurence of literal theft and the emotional tone of his posts. He explained he was struggling and felt validation-seeking from their interaction.

10. Laurence accepted the apology but noted the severe reputational damage and harassment he faced from the public accusation.

11. Oz deleted his repos and expressed regret over the harm caused. A professor retracted his earlier criticism of Laurence.

3 seems to be a bit wrong based on Laurence's post on github https://github.com/google/project-gameface/issues/1#issuecom...

  So why didn't you help and why did you ghost me when I asked? <<

  As I explained at the time, my team isn't set up to help individuals in the way that you asked. I brought your case before those that can within Google, and left it to them to reach out to you. Indeed, you mentioned (in your first post in this thread) that they already gave you a computer -- I am not sure when that happened, but those are the folks that do that kind of payment/donation, and not my team. I asked you, at the time, if you had a charity or other organization that you work with, because that would be the only way we would be able to do anything, and maybe not even then, and the answer was 'no'. That's where the conversation ended.

Without being judgemental, OP in that thread has clearly some kind of mental disorder, or was/is under a severe stress condition.

while I'm hopeful google will be responsible long-term maintainers of this accessibility feature, given their "ship and drop" attitude (music, reader, podcasts, stadia...) I'm glad they've chosen to open source this. In any case, an exciting project!

Yea given that it's open source, I'm hopeful that it'll keep going as an accessibility tool for a long time. https://github.com/google/project-gameface

Was instantly confused as this is already a name of a product (Coherent Gameface for making a game UI in HTMl5 for multiple engines).

Seems this might well be useful, possibly even for normal people. But if I was suddenly full-body paralyzed, I would do everything and anything to get a Neuralink after seeing the first human user of that get ~80% of my score in a test measuring fast and accurate mouse movements.

> Neuralink

Yeah, but that's not a realistic hope even among fairly wealthy Americans right now, and its many orders of magnitude less accessible to most of the world. Considering the smartphone market penetration in some profoundly impoverished areas, this could be really huge for a whole lot of people.

Besides that project is just "some sucker is gonna come and figure it out for me" bounty in form of a company, and importantly built on a known dead end

It’s a good day for accessibility between this and the stuff apple announced.

Tomorrow is Global Accessibility Awareness Day: https://accessibility.day/

Hence the announcements, I assume.

macOS has a lightweight version of something similar built-in: https://eshop.macsales.com/blog/64948-control-mac-with-head-...

You can move your head around to move the cursor, smile to click, raise your eyebrows to right-click, and fart to make it to type the embarrassed emoji (not really, lol).

But it's pretty cool.

Also reminds me of the Dasher app, an eye-tracking typing app with text prediction. You stare at the a letter, one at a time in an ever-zooming tree, in order to form words: https://www.inference.org.uk/dasher/DasherSummary2.html

The biggest issue I see with such things is lack of standarization.

For us lucky, "regular" human beings, the interfaces has some standardisation.

- Touchpad - Touchscreen - Keyboard

We can complain on some minor differences but it's not like going from one device to the other the controller becomes that much different.

Ideally, if big tech players are into really making such things useful (eye-tracking, face gestures, etc), there should be standardisations.

So if I need those accessible interfaces, they're agnostic as possible to the platform itself.

As tech giants delve deeper into markets like gaming, do they enhance the industry by driving innovation and offering new platforms, or could this stifle smaller developers and lead to less diversity in the gaming landscape?

Gaming industry couldn't have any worse-er giants

If it's Google, the answer seems to be: no. Nothing happens except Google pays a bunch of developers a bunch of money.

Yet to see how well this tech works but congratulations to Google engineers for making this open source.

This could be another dimension to gestures that adds even more convenience for end users.

ex) Swipe right on Tinder by winking, swipe left by simply blinking.

Off the bat I can see on-device assistants be more capable & allow customized handsfree browser/PDF interaction

Can this be used without internet connections or is it just continuously streaming your face video back to Google?

The model runs locally on device, you can find the mediapipe model in the repo.

Awesome news, can’t wait to try it out

Last release and code changes on Aug 3, 2023 ?

entire android directory added yest https://github.com/google/project-gameface/tree/main/Android

In the near future, people can control appliances with purely their own consciousness, and the only prerequisite is that it requires a minimum consiciousness level, which is reachable for most of the human being. Lastly, we usually think that people living in the Stone age are primitive people. Are they?

https://twitter.com/Michell51304

It’s interesting this tech is for disabled people when Nintendo just assumed their players were disabled to begin with. They always played their games with their left hand to simulate it.

Software developers are pretty terrible in applying this kind of empathy in our UX.

Assume your app is unusable and the user doesn’t know how to read or have hands. Hah.

Left-handedness is a disability? What?

If you’re right handed, then yeah, playing with your left simulates a handicap. They designed a lot of their games around such handicaps.

They just had to swap out their tester every 6 weeks when they became ambidextrous

That isn't what they said, and it's probably not what they meant.

What they probably meant was that Nintendo testers use their non-dominant hands to simulate someone who has no prior experience with video game controls. (Though I'm not sure exactly what that would mean. Maybe OP meant they use "backwards" controllers with all the buttons swapped?)

It could be a reminder to developers that they should not assume that they should develop for one kind of player.

I remember watching an episode of MKBHD where he played a racing game, and the hands on the wheel were white male hands (he is black). He just looked at the camera and said nothing, but sighed. I thought it was well done.

personally I hate PC games that assume WASD, or don't let you remap keys, or make other asumptions that are pretty easy to figure out you shouldn't do.

should not assume that they should develop for one kind of player.

Or even better, if they accommodate the lowest common denominator, you actually can end up with better experiences for even experienced players.

I’m looking at this tech in reverse. Sure it’s going to help the disabled, but it’s also going to enable entire new things in games. I fly around a helicopter a lot in Arma, and I’d love some simple solutions for head and eye tracking just with my web cam and it looks like most of this r&d is happening in the accessibility space, not in the game design space.

>I remember watching an episode of MKBHD where he played a racing game, and the hands on the wheel were white male hands (he is black).

Is this actually a problem? Should every character you play be the same race and gender as you? Personally, I would not care.

It depends how far they go. There are games where they insist I must play as a big titted whore even though I don’t identify as a woman and even if I was a woman, I’d opt for more subtle whoreness not the full whoreness some of these games insist I must role play.

Some examples of games where you have to play as a "big titted whore"?

Sure, Black Desert Online.

They insist you play a big titted whore.

With magic and shit, for clarity’s sake.

I've never seen this game before, but looking into it, they don't seem to insist on you playing as a "big titted whore", and to be honest, I wouldn't really care to play as a voluptuous woman ahah, I hope you don't call women whores just because of that.

I mean, they desensitized it to me too. I play the girl class if there’s no option. The arc of justice doesn’t bend toward truth, it bends toward revenge. It’s zero sum.

I guess the bitches were tired of playing bulky men that they don’t identify with and now force the men to play whores we don’t identify with.

I’m just saying

But if I had to get really meta:

I personally don’t believe there are enough women to force that macro change in gaming. It’s self flagellation of hyper aware males.

If I had to get double meta:

It’s self flagellation by guys that don’t even get laid.

Tricky business

I'm pretty sure the girls in skimpy outfits thing in BDO is because that's how Korea and Japan rolls with their character designs, and men want to play skimpy dressed lady characters. I don't think it's a DEI thing for women that want to play big tiddy women lol.

That's like arguing that you can't drive a beat-up pickup truck in a F1 racing sim, or a Cessna in a combat flight sim.

The character design is part of the game that you chose to play.

Is it ok that some people do care?

I stand corrected. He just said "not exactly what my hands would look like, but (laugh)"

youtu.be/kFz9afj8lu0 ~ 7:00

As a lefty who has written on whiteboards and blackboards I'd say yes in possibly the smallest way possible.

I think games are fundamentally different from other software in many ways. For example, games tend to revel in selectively sharing information (Tunic for example) in a way that regular software doesn't. It's hard to apply lessons from one to the other.

I can't recall which standup comedian it was, but he had a bit about games where they're the only form of entertainment that tests you. A song doesn't make you dance along to continue. A book doesn't quiz you on its themes and slam shut if you're wrong. For some games, being hard to play is their point. That's why accessibility is great to make games where that isn't their point more, uh, accessible. And maybe even those hard games should have accessibility, I haven't given that front enough thought yet.

>It’s interesting this tech is for disabled people when Nintendo just assumed their players were disabled to begin with.

oh, that's why the N64 controller was designed that way, to accommodate those of us with three hands -- I get it now.

Reply on news.ycombinator.com