Show HN: ThingsTwin – 3D House Plan Based UI for Smart Home | Hacker News Reader

Show HN: ThingsTwin – 3D House Plan Based UI for Smart Home

thingstwin.com

62 pointsCan0nC18 comments

As someone who works in the professional smart home business and in a comapny with close to 15 years of experience, using "3D" based house plans are usually a waste of resources. In most cases, this leads to the following problems:

- Having to create a 3D house plan from scratch

- Having to add textures, objects to it, as otherwise it would look bland

We only use black/white 2D house plans, which bring all the positive aspects of a house plan without the negative one from 3D, as a flat 2D plan does not need any objects in it to look good.

Here is an example screenshot of my apartment https://i.imgur.com/99qos7l.png

Your floorplan example has very little in the form of indications or metrics, seemingly nothing beyond switch on off status and maybe light intensity level.

I feel like the next frontier for home automation is going to be sensors and monitoring. Early forms of this data gathering being proximity or motion detection for triggering actions such as lights or displays, including thermostats, which also monitor temperatures indoors and from local weather stations. While outlets with power monitoring are available, wall switches don't seem common yet, which is odd as they'd have the awesome application of detecting bulb burnouts. Individually customized replacement bulbs delivered automatically as a service

Air handling seems also like an easy area for improvement, humidity based switching exists, but coordinating bathroom fans exhausting with an intake running can efficiently recycle hot summer air with cool night air. If the fans were somehow reversible, bathrooms could intakes and exhaust on opposite sides of a building, while also monitoring air temps and humidity to detect with thermostats when they've sufficiently cycled through outside air.

Plumbing especially needs innovation, it would be nice if there was accurate flow metering, something ubiquitous you could put on a compression valve supply line that could measure usage, detecting drip leaks and left on taps, ideally with a valve inside for regulating flow. Can we get a power line over plumbing (PLoP) standard?

While not necessarily hard to display in 2D, certain aspects, like building airflow, might require 3D stimulation. It seems like 2D floorplan or walkthrough video to 3D model would be an ideal automation workflow, with generic textures and object models sufficing.

Yes, you don't have any detailed information in this view. By pressing any Icon you toggle it, by long-pressing anything with a "+" you open a popup that displays light levels etc.

For example, we have this for anything heating-based: https://i.imgur.com/evaKP07.png

The same case here:

Ventilation is toggle based, temperature is connected to a sensor and pressing the icon opens a popup to change the temperature threshold.

What you're suggesting for ventilation efficiency is a solved problem - new buildings with ventilation systems all lead to a single input and output. By regulations it's mandated to use a heat exchanger. So the hot air you want to suck out during cooking loses a big of it's heat and gives it to the air being sucked in.

For older buildings it's in the realm of possibilities. As soon as any integration is possible (Be that IP based or trough switches), any logic can be programmed. The fact is that most people don't care about those technical details, they just want it to "work". You don't spend money on such an infrastructure to be adjusting everything yourself - albeit for the customers that specifically demand it, making these options accessible is always a possibility.

The only thing I can say is that 3D simulations for airflow are something you'd do on dedicated software, as the aspect of simulating such a thing:

- requires a lot of computational power - requires a larger amount of detail avaliable for each room

This means you'd have to place all the objects you have in your house in the 3D plan, which can be a lot of work as there won't be fitting objects avaliable to each variant.

That screenshot looks on the one hand very useful and on the other hand somewhat user-hostile since you have to build your own mental mapping from the architectural-style floor plan to the rooms in your house without any hints - I can imagine it being much easier to get used to or share with guests if there were sufficient icons or furniture shown in each room to make it obvious which room it is and which direction it's facing without having to consider the measurements and position relative to the other rooms.

I have to disagree a bit. For new users it will always take a bit of time to get used to it - be it in 2D or 3D. But after some time, the usability of 2D and 3D will be the same. And guests will have that problem either way - for that reason, one could just render their usable rooms on a separate UI.

If I were to use the 2d floor plan I’d look at it for a moment, and then I’d flip a couple of lights so that I could tell which room is which. It’s not complicated imo.

Creating a 3D house plan from scratch and adding textures and objects is a solved problem.

The Sims was published 21 years ago, and it makes it quite easy for kids and even adults to build, decorate and furnish models of their own homes and families.

See my other posting in this thread about "Towards a mixed reality intelligent campus".

https://news.ycombinator.com/item?id=27174053

Here's an even earlier pre-release version of The Sims that we released internally at EA in 1998 to The Sims Steering Committee, which convinced them to let us finish and release the game in 2000.

https://www.youtube.com/watch?v=zC52jE60KjY&ab_channel=DonHo...

That pre-release version is quite crude, with ugly graphics and a klunky user interface, but you can look at youtube videos of The Sims 4 to see how far the state of the art has advanced in the 23 years since that pre-release demo of The Sims 1.

Here's a slightly more recent demo of the architectural editing tools, pie menus, and visual programming tools that I developed in The Sims 1 (this is the released version, but with the unreleased internal "Edith" object editing and "SimAntics" visual programming tools built in):

https://www.youtube.com/watch?v=-exdu4ETscs&ab_channel=DonHo...

>The Sims, Pie Menus, Edith Editing, and SimAntics Visual Programming Demo. This is a demonstration of the pie menus, architectural editing tools, and Edith visual programming tools that I developed for The Sims with Will Wright at Maxis and Electronic Arts.

Here's one of LGR's reviews of The Sims 4 from 2014:

https://www.youtube.com/watch?v=ohwbklm3w_E&list=PLXtVXmQQAA...

>LGR - The Sims 4 Review

>Gameplay and overview of The Sims 4. What are the new features? How does it compare to The Sims 3? Is it worth buying? Questions, answers, snark!

He's published many other reviews of the Sims 4 expansion and stuff packs, too:

https://www.youtube.com/playlist?list=PLbBZM9aUMsjH0ugWCCkcF...

This one from 2014 focuses on build mode and the architectural editing tools:

https://www.youtube.com/watch?v=PstU5SsEAX0&list=PLbBZM9aUMs...

>LGR - The Sims 4 Build Mode Gameplay Revealed

>Commentary on the latest gameplay from The Sims 4, showing the trailer footage for build mode. New scalability, wall options, movable rooms, and all that!

I'm not saying it's not a solved problem. It is, and we don't even have to take Sims for that. Close to all, if not all, architects around here already create 3D simulations of the buildings to show to customers.

What I am implying is that the total gain you get from fabricating, setting up and rendering a 3D model of a house plan is barely more than the one you get from a 2D version

Advances in miniaturization and better economics for lidar definitely are making floor plan creation and 3D room scanning far more accessible. For work I've been experimenting over the last few months with "RoomScan LIDAR" and "magicplan", using both an iPhone 12 Pro's built-in lidar and the ability to link with a bluetooth laser distance/angle tool. While the market is definitely still in its infancy and a number of things remain frustrating (as does dealing with more complex room geometries), the potential is obviously there and they've already been useful tools.

Being able to throw together decent floor layouts with fairly decent 3D point clouds for an entire building in a short period with just a few tools you have with you anyway can be fairly magical already. And there is a clear path for far more with another year or two of maturation. In some ways the biggest issue right now is lack of ability for other tools that could in principle make a lot of use of imported plans/point clouds to import them. Generating is getting better and better, but the pipeline after that is a weak point.

And case in point, there's this very post. Why can't I just give it my PLY|OBJ|XYZ cloud, or at least USDZ|DAE model? Why is there a need to reinvent the wheel here? For 3D House Plan stuff to take off, there has to be amortization of the work of generating something in the first place. While it's more accessible than ever, there is still time/money cost to throw something together, and it's silly to need it done more than once (with necessary updates after major renovations, but those are unusual events).

Of course for me this entire thing is an instant write-off as soon as I see any sort of cloud requirement. Optional maybe ok, another vertical silo/3rd party dependency? Hard no. And lack of leverage is an issue throughout for an effort like this, anything that doesn't work through existing Home Assistant or other ecosystems is also immediately on the back foot.

If only I could host my own storage for this, I might actually be interested...

You know what’s even better UX? Light switches. The Hue dimmer switches are great.

The most value I get out of them being “smart” is that I can program a double tap of the “off” button to turn off all lights in the house, or multiple taps of the “on” button to cycle between scenes.

Yeah, exactly. I have lots of "smart" stuff in my house, and it's almost totally controlled either with light switches or automatically. Controlling from a phone is possible, but rarely used because it's not convenient compared to pressing a button on the wall or literally doing nothing.

In my case, most of my house is Insteon switches and keypads, though most I've added in the last couple years is LED/RGB stuff (with tasmota).

There's a keypad in the kitchen, with buttons "bright", "dim" and "off" that do expected things. It takes a second to press as you walk by, and visitors figure it out in 5 seconds.

There's some automations hidden though: at night, the lights automatically turn off. If nothing is on at dusk, some automatically turn on, the exact config depending on if the home is "occupied" or not (based on motion and control presses). There's some RGB lights on top of the cabinets that pick a random color when you press the buttons. "Off" actually leaves those on (dim) before midnight, though you can double-press for actual off.

All this stuff is largely hidden, but you notice if it's missing (eg: we never come home to a dark house). To me is the point of "home automation" or "smart home" (whatever we're calling it this decade), that all the phone-based and control-from-internet stuff totally misses.

My office is one exception right now. Since the pandemic I changed the lighting to all RGBW bulbs and strips, and have a circadian program running (changes color temp and brightness slowly over the day) which is awesome. I currently control it with either the home assistant app or voice; it's ok but noticeably inconvenient compared to a switch (I just haven't got around to fixing that yet).

I definitely would not remove physical controls from the wall in each room, and if you have that, I'm not really sure what you need a floorplan view for. It looks neat, don't get me wrong, but I just don't see the practical need.

Systems like HomeAssistant or Apple’s Home app seem to become much more usable when home devices are properly organized or automated—but doing so is extremely tedious or difficult. When long device lists are not organized by function or room, it is prohibitively difficult to explain how to use these systems to guests, or less technical users.

There’s a floor plan add-on[1] for HomeAssistant, but creating a floor plan or rendering for your home is a lot of work with difficult tools.

I’m excited about this project because it makes creating high-quality floor plans more trivial and closely linked to the device organization and setup phase.

[1]: https://experiencelovelace.github.io/ha-floorplan/

Open source floorplan based home automation seems to be in it's infancy.

As far as I'm aware, even if you have scans of interior floor plans there's no straightforward way to drop and place 2D indicators or icons for devices in respective rooms that can be saved into an openly used format.

I wonder if 3D models in game engines will be the approach that's adopted. Maybe with a top down isomorphic minimap being generated, but otherwise using presence detection to track a person model through a house, starting automation much like game events are triggered by players passing invisible barriers

In 2008, four years after I published it during the 2004 election, I discovered that some research scientists incuding Marc Davies at the University of Essex Digital Lifestyles Centre forked and cloned (or "hacked into") my open source "Dumbold Voting Machine for The Sims 1" virtual agitprop, and reprogrammed it for remotely controlling ubiquitous mixed reality home automation!

It's easy to make a 3d model of your home in The Sims -- that's one of the main points of the game! And it's gotten even easier and more realistic in the 21 years since it was first released in 2000.

https://donhopkins.medium.com/dumbold-voting-machine-for-the...

>Dumbold Voting Machine for The Sims 1

>Just before the 2004 presidential election, I created and programmed a downloadable Dumbold Voting Machine object for The Sims 1, as interactive agitprop to raise awareness about the problems of voting machines. I wrote this article about it, which I’ve now updated and included some new links and videos!

[...]

>Epilogue

>When you put something out there as free open source software, there’s no telling what people will use it for! Flabbergastingly enough, I’m delighted to discover that somebody actually repurposed and reprogrammed the Dumbold Voting Machine for their own nefarious purposes.

https://www.researchgate.net/publication/4370926_Towards_a_m...

>Towards a mixed reality intelligent campus

>By Marc Davies, Victor Callaghan, Michael Gardner, Digital Lifestyles Centre, University of Essex, UK.

>Abstract: This work-in progress paper summarises our research towards the vision for creating an intelligent university campus (iCampus) based on a mixed reality technology and network based education. The paper brings together earlier work aimed at exploring how simulators and other virtual augmentation can utilised by scientists to enhance the development, testing and demonstration of new ubiquitous technologies and environments with our latest work aimed at creating a simulation of a classroom based on the MPK-20 Project Wonderland Virtual Meeting Office developed by Sun Microsystems. We conclude by outlining our future plans.

>Fig. 4: Sims Object Remote Control Menu

https://www.researchgate.net/figure/Sims-Object-Remote-Contr...

>Sims Object Remote Control Menu

>The iSpace simulation was created by modifying an off- the-shelf copy of the Sims computer game, (Maxis/EA Games, 2000). Apart from the 3D graphics and supporting tools, a particularly attractive feature in the Sims was the fairly realistic behaviour of environment inhabitants.

>The simulation consisted of a five room environment modelled on the iSpace [5][6][14]. Each object and person was controlled by at least one thread, placed on a stack and run in sequence by the game. Object threads were used to regulate the animation displayed by the game’s virtual machine [9]. Most objects could only access their own threads, so for example a television couldn’t access information contained in a thread for a lamp.

>To create a Sims-based simulation, the original program code had to be modified so objects could access threads for other devices and any required information contained within. For this stage of the project the most efficient way to achieve this was to program a single Sims object to act as a ‘remote-control’ for other pervasive devices in the environment. The ‘Dumbold Voting Machine’ [12] an add-on device available online, was modified to act as a remote-interface, usable by Sims avatars in the environment. Within the code, for the re-programmed voting machine, the current state of each pervasive device in the environment was stored to memory.

>Agent code, added to specially created classes, ran from the voting machine thread, prompting state changes to objects in the environment as required. Agents determined when to make changes using sensor settings coming into the voting machine thread on each cycle.

>The menus, from the voting machine, were re-programmed to provide a manual interface for researchers, (see Fig. 4). This menu was used to force the priority of actions performed by a Sims avatar.

>The original program allowed a player to design, build and furnish a house to their own specifications, using numerous pre-programmed materials and objects available in the game libraries. Using a game to create a digital home simulator introduced several advanced features that provide a higher level of realism to the environment. These features include avatars that randomly visit the virtual home. Another benefit of using computer games is that researchers can take advantage of the popularity of the original product, as a level of familiarity with the environment could be established in the minds of the observers. Additionally, popular games often spawn myriad online fan-sites, often offering freeware add-ons and/or modifications. This could be exploited by a researcher/developer to further expand the realism or capabilities of their simulation.

>The Sims was programmed using a bespoke language, created by the original game developers which served the needs of an in-house development team very well. […]

Interestingly enough, the University of Essex is where Roy Trubshaw and Richard Bartle developed the original MUD1 (Multi User Dungeon) on a PDP-10 in 1978!

https://en.wikipedia.org/wiki/MUD1

More related publications by Mark Davies:

https://www.researchgate.net/profile/Marc-Davies-2

iWorlds: Generating artificial control systems for simulated humans using virtual worlds and intelligent environments

https://www.researchgate.net/publication/220497559_IWorlds_G...

iWorlds: Building Mixed Reality Intelligent Environments Using Customisable 3D Virtual Worlds

https://www.researchgate.net/publication/224207355_iWorlds_B...

Modelling Pervasive Environments Using Bespoke and Commercial Game-Based Simulators

https://www.researchgate.net/publication/221093410_Modelling...

Decloaking Big Brother: Demonstrating Intelligent Environments

https://www.researchgate.net/publication/224207398_Decloakin...

Towards MMO Intelligent Environments

https://www.researchgate.net/publication/266617707_Towards_M...

Social Presence in Immersive 3D Virtual Learning Environments

https://www.researchgate.net/publication/270881554_Social_Pr...

I haven't used that add-on specifically, but the tooling for making a floor plan is very good these days. SweetHome3D is absolutely brilliant and with only a 5 minute video tutorial to guide them, I'm sure any computer-literate person could grab a tape measure and make a great floor plan of their apartment in less than an hour.

I learned myself that a “house plan” whether it’s 2d or 3D or 4d or whatever is a horrible interface for a smart home. I prefer a much simpler “card” or page per room.

Would be great if this could sit on top of home assistant which is pretty awesome.

Reply on news.ycombinator.com