Show HN: Spatially aware group video calls from the browser | Hacker News Reader

Show HN: Spatially aware group video calls from the browser

party.mookerj.ee

163 pointssmoo41 comments

Made this to try and simulate real world interactions - normally I found on large video calls, only one person gets to speak. With this you can dip in and out of conversations with groups of people just by moving your avatar towards or away from them!

It also uses modern web technologies and does not even need a login, just show it your face and it will try and use emojis to tell people what you're doing even if you're not in a call with them

I’m not affiliated with them and i realize it’s different from what you did here. But might be interesting for you to check out sococo.com

I like the concept. It was not immediately clear that you tap/click to walk. This may have been explained in instructions but who reads those?

saw the same problem and been working on something similar. Up for a chat? jiwanix@gmail.com

Something similar to this that works better IMO is Online Town, which was posted yesterday on HN: https://hn.town.siempre.io/

https://news.ycombinator.com/item?id=22818300

A 3d version of this concept is well executed by Mozilla Hubs, but the lack of echo cancellation makes headphones a necessity when using Hubs which is a downside.

A lot of this can be solved with push-to-talk, like what Virtual Airwaves is doing. They simulate radio in that the closer you are to someone, the "stronger" the signal is. (Take a look at their patents). https://virtualairwaves.com/ or try it at https://cb.virtualairwaves.com/ There are usually conversations on Channel 1 each evening, UK timezone.

Echo cancellation works in Firefox, we are landing a fix for chrome this week. It turned out to be quite involved to do spatialized audio and echo cancellation:

https://github.com/mozilla/hubs/pull/2361

Nice! This could elevate virtual museum tours, which are currently lacking in interactivity.

I tried with my camera shutter closed and it didn't work at all. Just said "waiting to detect your face" with a smiley-face.

Very creepy.

What’s creepy? The smiley face or that it requires a face? requiring a face is a good call

The fact that if I hadn't used a physical barrier then my face would have been immediately fed into a detection algorithm without any explanation or warning.

I get that it's just a tech demo but a small disclaimer about what will be done with the data before requesting the permission would go a long way.

The facial detection all runs locally in your browser using tensorflow.js and it's crazy that we live in a time that's even possible :D

But I do appreciate the feedback and will be sure to add a disclaimer and make it clear what's going on!

Yeah it doesn't need to be long (short is actually better as more chance of people reading it through)

1. The app requests permission to view your camera

2. Your video stream is displayed and a local js script will run facial recognition (not against a database, just a binary "is this a face") and only then will it allow you to join the chat.

3. When you join the chat, your video and audio will be transmitted but will not be stored.

some people likely won't even read it and just click anyway, but for the curious and/or paranoid it will definitely be reassuring.

thanks for sharing your work.

The prompt is to allow use of your camera. Do you expect it to make you waffles?

What exactly do you think will happen other than _using your camera_?

Maybe, but it makes it impossible for me to test it at the moment as my webcam doesn't capture enough light to get the face detection working apparently.

It uses your face to make the little avatar to show to people as you move around the map! (as well as to animate the little emoji that follows you around!) I honestly don't know if a show-your-face policy is the best to prevent people being mean to each other online - I'd love to hear your thoughts! You can read my code though, nothing gets saved anywhere and the video calling functionality is all peer-to-peer!

i think show-your-face is a reasonable burden that will help improve conversation quality

Firefox asked me for permission to use my video camera, so unless that didn't happen for you not sure why you would be surprised that its using your camera feed.

If you don't want to show your face, you can use the parameters "--use-fake-device-for-media-stream --use-file-for-fake-video-capture=/path/to/y4m_or_mjpeg" on Chromium. You can get y4ms from xiph.org[1].

[1]: https://media.xiph.org/video/derf/

Love the idea, but the other party couldn't hear me even though my mic was working, and the video seemed to be stacked in a strange way where my own camera was generally half off the screen. Tried both Firefox and Chrome. Would certainly love this or something like this for virtual social gatherings though!

That’s so cool. You could add background chatter too if you haven’t already, you’d just have to amplify the local conversation the user wants to tune into

Edit: by background chatter I mean the noise from all the other conversations happening in the moment nearby

Very nice! I had exactly this idea a couple weeks ago when on a 25+ person “zoom party”. We needed a way to break into little groups. This is a cool way to do it, obviously with little improvements. I think requiring a face is good too! You could have not map in the background too, it’s not really necessary, just the radar is fine!

zoom also has breakout rooms for what its worth, might be a paid feature, but you can specify whom and how long.

Tried this out, but it picked the wrong webcam. You should provide an option to select the correct one.

Super fun take on spatial software with video calling. But made me kind of uncomfortable haha

Also, I'd try to make it easy for a person to leave the chat room. Couldn't find an exit. Not good UX

I'm sure you'll improve it along the way. Good stuff :)

This is a really cool idea I've been thinking about as well. Schools/churches/clubs/etc. looking for an online equivalent to informal meeting spaces need something like this. Zoom/Hangouts/etc. break down with lots of people. The map is creative, but another metaphor I've thought of was just tables. Show a handful of tables in a room, maybe with avatars on them? I also had an issue where I couldn't easily pick which camera to use.

I'm right next to people but can't see or hear them? Is there something else I need to do?

Edit: I got it. My roommate's browser was not working for it in general.

Very cool @smoo. How did you build it? And does everyone meet in London or are they distributed to the centre of their nearest city?

Thanks! I've only added in a map for London at the moment but will be sure to add different places soon! I've actually also made it open source too at https://github.com/souramoo/party - it uses standard ecmascript and webpack for the client, nodejs for the backend hosted on heroku and webrtc peer-to-peer for video :)

Would be nice if we can host a meeting and pre-select the locations in the background. I think this can be an excellent solution for virtual meetups during the pandemic - much better than Zoom or chat rooms.

That's really cool! Audio didn't work on FF mobile.

We are working on something similar: https://www.joinvoiceplace.com

Interesting would love to talk to you guys jiwanix@gmail.com

Great idea. I say that as a pat on my own back, given Ive been hacking on basically this exact same thing for the past few days.

I think this has a lot of potential.

Go to game rooms, and play some game online. Go to a club, and listen to the same Spotify track.

Would be helpful to have a screenshot or something so I can see what's up before approving camera/mic access!

I agree. Even after approving the camera and mic, the way it was presented makes it seem like the camera is only going to be used for controlling the emoji. But that isn't the case, the camera video is shown to other people.

I feel like there shouldn't be a #public room. It is basically Chatroulette.

Thanks for the feedback! I will update it with a screenshot (you can see ones I've put on my github at https://raw.githubusercontent.com/souramoo/party/master/publ...

You can turn your camera off once you're in a room too - I actually made this to try and host my birthday party next month where it would be nice to see everyone :)

I think the idea is really great for private parties. Zoom and similar conferencing systems don't work well because either everyone is the same volume and it is just loud noise, or it mutes everyone but the loudest person, which kills lively conversations.

I really like the idea of being able to mingle, and "walk" around to chat with different groups, all at the same party.

Didn't someone made something similar using jitsi meet?

That was me: https://meet.primrosevr.com

There have been a lot of changes since I made the first post a week ago. Check out the github, I'm looking for contributors!

Just a yellow box for me, not functional. Chrome Mac OS.

Reply on news.ycombinator.com