In-Flight Entertainment Challenge | Hacker News Reader

In-Flight Entertainment Challenge

blog.mand3l.com

127 pointsjonaraphael80 comments

Recreating Delta's in-flight entertainment UI in ReactJS between on a flight between NYC and SFO. [Post by a friend of mine]

Open articleView on HN

> Rapidly translating UIs into code is a core skill for UX engineers

Maybe I'm confusing the terms, but in our company UX engineers main job is to "figure out" the user flow. It's kinda UI but with mostly boxes (colors, fonts, animations will be added by designers/coders later).

It's closer to product management than programming actually.

Are they confusing titles UX designers with UX (software) engineers? I mean, I guess, then its just a matter of everyone calling the role something different. My wife is a UXD, but she has never been called an engineer before.

FAANGs have UX Engineers whos main goal is to typically prototype ideas. The code they write is not always submitted to source. So a UXD works with a UXE to make ideas happen quickly to vet them out.

Interesting, do those roles pay well? I’ve never heard of a role that would fit me at a FAANG and didn’t think it existed but that’s exactly what I do.

As SWE I'd pay for UX the most of the team. If they did a good job, then coding it is simple. Like 50% of typical SWE work are to figure out the UX.

I was a UX prototyper at Microsoft myself, that’s how I met my wife who was working as a UX designer in the same studio. We didn’t have a name for it, my official title was just SWE. I haven’t heard of UXE being a real title at the current FAANG I work for, but I no longer work in a design studio.

Perhaps you are. Replace Engineer with developer and you have some one who is basically a Front-end Dev. But the scale here is, are you front of the front? Or back of the front? Or all of the front. I think with UX Engineers you are all of the front. Which makes you a weapon.

Idk, those two terms are as different to me as URL vs URI. Searching it will yield a long explanation about the differences, but IRL people will use them interchangeably.

It might seem that way to you, as a url is a subset of uri.

I've never actually been in a company that has UX engineers though. It's usually UX or UX designers which work with figma or similar tools and delegate the coding to non-specific software/frontend engineers

I think it's not a strict subset, there are URIs that aren't URLs (like ISBN numbers), but there are also URLs that aren't exactly URIs, like http://example-blog.com/latest

But in practice, sure, the 2 are pretty much used interchangeably

edit: Perhaps I'm wrong, all resources I can find classify all URLs as a type of URI.

Maybe my understanding of URIs is flawed then, but from your example it looks like an URI for blog articles sorted by publishing date. List<type> was a valid resource type as far as I remember.

People do this a lot, but those of us who actually do the work instead of sit and talk in meetings about the work know that there is a huge difference between writing React components (UI) and designing flows in Figma (UX).

It's just outsiders who say UI/UX as if they are the same thing.

Yet I’ve seen people use the opposite definitions professionally. IMO it’s not the kind of thing that’s worth trying to get people to change.

I'm a backend SWE. How are those two roles not just UI designers and UI SWEs? Or frontend as our team would say. I get that "experience" suggests something grander than "interface," but that sounds like how I call myself a SWE just to sound better than programmer (which nobody calls themselves anymore, so there's no real difference). Maybe instead of backend we can say "cloud-end" cause "RESTful JSON API" lost its zing.

In our company:

Writing React components -- frontend engineers (no "UI").

Designing flows -- UX (no separation on UXD/UXE, and they actually have to sit in meetings a lot, to figure out what users want and how others do it).

And we use "UI" for folks in design industry: images, colors, fonts, look-and-feel, modern fashion, transition animations, etc. Maybe some nice interactivity that UX didn't care enough for.

(Doesn't help that uri and url look very similar in printing)

Here’s another challenge:

Create fun games to play in these flights.

The games available are usually poor and slow.

Is it a technical limitation such as too many screens and not enough processing power to go around?

Or maybe no good HID (Human Interface Device) controllers? Touch-screens should be okay, especially for turn-based games, but maybe continuously pushing your finger against the screen would make the person seated in front angry after some time?

Perhaps Management doesn’t realize that video games are a popular thing?

Or is this just too much of a first-world problem and not worth the effort from the airline companies?

I’d love to see games like Rogue, X-Com, Sopwith, Jagged Alliance, Space Invaders, Asteroids, Myst, Vampire Survivors, and so on, available to play at 30,000 feet.

Maybe the airline could have these games hosted on their wifi network for downloading and playing in the browser of your personal pocket computer instead?

I don't think adding a nice controller and cool games to play would be prohibitedly expensive. But it is an investment and someone needs to take the risk. It's entirely possible that they spend all this money creating this thing and no one uses and just watch films the whole trip.

You're probably better off getting a switch or a steam deck. It has the added benefit that you can finish your game afterwards, after the flight.

> I don't think adding a nice controller and cool games to play would be prohibitedly expensive.

Yeah, but it would be more expensive than dirt cheap.

I see it all the time; you wouldn't think a huge company would have a problem spending money on something. But, at the end of the day, it's all about that department's budgets and priorities.

>I don't think adding a nice controller and cool games to play would be prohibitedly expensive.

Everything like that on an aircraft (nice controller) has to be certified for every specific type, etc. Lots of process, paperwork, testing, etc.

I used to fly on planes that had “fancy” controllers - they had lots of buttons (full QWERTY keyboard, ABXY and d-pad, special buttons for controlling the lights and calling the attendants, etc.) and slotted into a latched recess right below the IFE. Something like this: https://www.vecteezy.com/photo/11024929

There’s no analog stick and the ergonomics aren’t that great, but it’s definitely usable as a controller.

Yes, it used to happen. I think though, that the trend of providing just wifi and the customer supplies their own device will spread. It's cheaper.

Chances of someone not having a screen with them on a plane these days are slim. I'd rather they used whatever budget they have for this into better in-flight internet speeds with lower prices. Then there's almost zero need for screens behind seats as everyone can just use whatever they prefer to consume whatever media they normally would.

I mostly agree, but I still use the in-seat screen for the flight map.

You could have that available on the inflight wifi would work fine, local image/tile server to a certain level globally, cache the area under the projected flightpath before departure, then update the cache as time goes on.

Some planes have external cameras too which can be interesting - you could have those on the inflight wifi too.

I’m sure that’s exactly the sort of complicated technology product that airlines would excel at delivering.

For the first part you can also just go on flightradar on your phone on the flight wifi.

Flight Map, movies and better/bigger screen a right position. Or chit-chat with a stranger, enjoy the view or read a book ;)

I could use my laptop but it is better suited for other tasks.

Myst would be a perfect game to play on a long flight. Sorting the right controller setup might be tricky though.

I was on a Japan Air flight about 10 years ago that had Street Fighter 2. It was on a crappy remote controller with vaguely SNES colored buttons. It was alright but your finger would inevitably slip and hit a game exiting button by mistake.

Still, it’s got to the be the most niche SF2 port ever made. Probably easy openings for the top spot.

Compete with fellow travelers. Game players and observers.

Travel socially+locally connected.

I remember being on a long haul with a colleague, and playing a trivia game on the inflight entertainment system, with other flyers. It was fun to hear the roar of victory from another passenger

It probably was a SNES: https://snescentral.com/article.php?id=1019

The trend is just to have you use an iPad or iPhone with local video and games available from the plane's Wifi. This pretty normal on US shorthaul US flights now, since those things are way better than anything built into the headrest.

The trend is to cut costs on domestic carriers; what you describe is just the excuse. When a flight has both IFE and this "iPad experience," I almost never see anyone using their own device. Even without the IFE, when people are entertaining themselves on their own devices, I'm skeptical that many of them are actually using whatever WiFi-driven entertainment monstrosity is available, as opposed to e.g. predownloaded Netflix content or Kindle books. In my experience it has never worked very well.

Watching TV/movies on the airplane screens is better than your phone/ipad because the screens are sometimes bigger, the content may be different (sometimes newly released movies show up earlier than netflix), and the screen is positioned in a more comfortable place for viewing while leaning back into your seat.

Playing games on those screens sucks: the controls are bad, the input latency is bad, the screens are usually pretty low fps (30 in the best case), and the hardware running the games is underpowered.

Playing games on your phone is better than the screen, watching stuff is better on the screen, so while this may be a cost cutting measure I'm fine with it for the most part.

> When a flight has both IFE and this "iPad experience," I almost never see anyone using their own device.

I don't like the IFE screens because of terrible hardware (unresponsive screens) and software. I always use my own device.

But you're right -- I generally don't even want to interface with the IFE and either leave it in map mode or off, if that's possible. Occasionally there's a film I'd like to watch, but never enough to watch on the seatback. On my ipad, OK.

Built in IFE's are pretty expensive, due to the required certification for each a/c type, periodic checks and fixes, and eventual need to be removed or replaced. And most of that has to happen with the plane not making money, in a hangar. Then also weight + fuel burn. It seems like a reasonable excuse to me.

played full on modern Tetris (probably an ipad port) on a flight recently, it was pretty decent

I think portable gaming has been available for so long that as far as airplanes are concerned, people who want to play games on the flight will bring their own.

Longest Flight I was ever on was Toronto to Munich, but my travelling was from Victoria.

I brought my 3DS with Zelda Oracle of Ages and Oracle of Seasons installed. Beat one on the trip their, one on the trip back. It was great for keeping myself entertained on the flight.

Nowadays I travel with my Switch

Back in the day Virgin Atlantic (and others?) hosted a version of the SNES. I played Link to the Past on a flight from London to Cape Town and, as the first long haul flight I'd been on, I thought it was normal. By far the best in-flight gaming experience I've had [0].

Pretty much all carriers have D-pad controllers, though the inputs tend to be very spongy. I don't think people use them much anyway because the touch screen is more intuitive and orienting the controller with respect to the screen is a surprisingly bad user experience (some are vertical, some are horizontal, wrapping never works well).

The games are improving a bit because the hardware is actually quite good now (some of the newer aircraft even have per-seat bluetooth). American has Plants vs Zombies, for example. I've played Luxor somwhere, and there are occasionally good puzzlers like Sokoban which I rarely play anywhere else. Annoying that the game has to cache and reload every time there's an announcement. The mid 00s systems are pretty terrible, but newer seat back displays are pretty great in terms of resolution/quality for what they are.

[0]https://nintendo.fandom.com/wiki/Nintendo_Gateway_System

I was once on an international flight which had an endless trivia game, where the questions were broadcast to every seat. You could pick a name, and there was a flight-wide leaderboard.

I think that could be a good direction: simple shared experiences that you can drop in and out of. Airlines are too cheap to spring for fast hardware, and the touch displays are chosen for durability more than responsiveness.

It would be neat if you could play small "MMO" games with the other passengers on the flight. Or just a chat room or Twitter clone. I suppose it wouldn't get used all that much.

> maybe continuously pushing your finger against the screen would make the person seated in front angry after some time

The tech crowd here is stuck on the technology limits, but this right here is main reason. It would drive me nuts if someone was trying to play a fast interactive game on a touch screen behind me while I was trying to sleep. It’s bad enough when people jam the screen when picking a movie.

A few years when my children were young enough , I was shopping around for couple of those backseat LCD screens to plug a dvd player or media player into for long distance trips. Eventually I used a Raspberry pi and HDMI splitter back in those days to run Kodi. I spent quite a bit of time editing XML on a Kodi theme to disable some elements and make the UI as simple as possible. A kodi or other media player UI theme like this would've been great. 10-12 inch touch screen displays are relatively cheap now. Finding a Raspberry Pi 4 at reasonable price might be hard right now. But there are cheaper alternatives to that as well.

Related hot take: your dev environment should work in airplane mode. It's a forcing function for better test encapsulation and being more deliberate about external dependencies.

This is how they should do interviews. Hand a blank computer at nyc and get the results in sf. Job offer in hand or not.

No offer, no worries. Catch the next interview back.

Except, unless you are springing for business class, you are selecting against anybody who is tall, and probably anybody sitting behind somebody who puts their seat back.

I know some people can work on planes, but I sure as hell cannot without contorting myself. If the person in front of me puts their seat back then I’m either looking over the top of their headrest, or putting my own seat back, and neither really works for using a laptop.

A thing I have ben pondering on is whether in-flight entertainment is dead.

I have not used it for several years. I get a much more consistent and higher quality experience using my phone and my Nintendo Switch along with my noise reducing headset.

i am sure it still has value for some, but the value must be diminishing as more and more people can supply this entertainment themselves.

And I believe it is not cheap to fit and maintain an entertainment system on an airplane.

It's the only convenient way to watch movies in-flight due to the lack of internet, and I usually see like half the plane using it. Guess you could pre-download from iTunes or less legit methods, but I doubt very many people maintain offline portable movie libraries.

Oh, some flights also have wifi and a local website to load a similar UI on your own device, but that can be unreliable. Last time I was using that on my laptop, I found out from the passenger next to me that it doesn't work on iPads.

At least that way, your show doesn't get interrupted by announcements.

both HBO and Netflix support downloads. that is what I use.

I find it useful and always use it on long flights.

For those of us who refuse to check in luggage unless it's strictly necessary, packing a console tends to be an unaffordable luxury.

I could use my phone, of course. But for gaming, the state of mobile gaming is so atrocious with all the exploitative pay2win crap that I'd often rather play one of the pedestrian but innocent in-flight games. And for movies, at least in my view, the phone is uncomfortable with the small screen and the need to somehow hold it, I'd rather play a martial arts movie on the plane screen and relax.

Of course, the fact that people find it useful doesn't mean that it cannot die - many of us also find having decent legroom useful and it's a thing of the past on most airlines...

Just got off a long flight - I'd say about 80% used the in-flight system, 20% had their own devices.

> Slick draggable scrolling is a key part of many screens in Delta’s in-flight entertainment system (complete with momentum-based flicks, over-scroll, page-snapping, etc). Because I didn’t have access to a component library and hadn’t thought to pre-install a good drag scroll library, I ended up having to build my own.

Shouldn’t have been using any drag scroll library. This is very strongly something that you should never do on the web, because it doesn’t have the right primitives to make it anything other than bad. Scrolljacking is always bad.

The proper solution is to just use normal scroll containers. Seriously.

That provides what you want for touch-based scrolling, without ruining click-and-drag or scroll-wheel or keyboard scrolling. It gets you your momentum-based flicks, leaves overscroll behaviour up to the user agent (which will be bounce on some platforms like Apple’s I believe, a glowing indicator on others like Android I believe, and nothing on some if you’ve also hidden the scrollbar with `scrollbar-width: none`), and you can use CSS Scroll Snap to control snapping if you want (though frankly I think it’s almost always better not to try to control that stuff).

Fun challenge, but reminds me of my major pet peeve with the Delta inflight UI:

The landing page looks empty!

I frequently think it's broken or still loading. All it features is the "New Releases - View" box in the bottom. I would expect to see something else there. Images of the first three featured movies perhaps? Anything other than the blank blue background.

(Dear delta engineers on HN, please add some preview images to this page).

> (Dear delta engineers on HN, please add some preview images to this page).

Delta (most likely) doesn't have actual engineers that handle the day to day work on the IFE systems, most of it is outsourced.

I can't remember if they were a client or a not, but a lot of IFE systems (even byod ones) are sold by Panasonic and they basically run on a system that somehow felt worse than a RPI 3, and the bugs.... while working on a system I saw the same IP assigned multiple times to client devices.

Also, a fun fact I recall. Last I heard (as of about 2018) media updates on planes required a tech to meet the plane at an airport and hot swap a HDD in a special enclosure.

https://www.panasonic.aero/our-offerings/

Delta's latest IFE is developed in-house and is actually quite nice. Their previous IFE (currently on most planes) is Panasonic.

I love that Fearless is one of the in-flight movies.

https://github.com/paulmand3l/delta-inflight/blob/main/src/a...

Here's a fun idea: ML model which takes an image of a UI (or mockup e.g. Figma) and recreates that UI in code. The ML model would infer callback/binding names, and any missing info like resize constraints, text / images / icons, buttons, etc.

Already somewhat possible. At least the UI generation part. Someone over on the r/chatgpt had a post where they took a SVG exported layout of a web UI and pasted the SVG markup in chatgpt. The prompt just asked it to convert svg into html divs. It in return generated the necessary html and css. Which came out to be remarkably identical to the original SVG. Amazing. Maybe with a little effort, someone could even get it to generate the callback/bindings.

EDIT: Screencap of the reddit post here: https://i.redd.it/x7yza5pgftna1.png

html width 100% height 100% overflow: hidden

Please don't do this. I would love to try it, but it's unusable on phone even on desktop mode.

Considering it was made in less than 7 hours with no access to the internet, I'd excuse hacks.

Edit: Especially since it's called out as a constraint:

> Don’t worry about production quality, just re-create as much of the look and feel of the UI as quickly as possible. No accessibility, responsive design, localization, etc; you just want something you can run through a user test.

Maybe should have spent less time to not have time to write overflow hidden then.

I mean they did it so they didn’t have to deal with a whole class of problems (making it look and work well on a phone).

For a 7-hour hackathon that seems entirely reasonable.

Realistic constraints too, since the original UI was designed to work with specific devices at specific screen sizes, not as responsive.

For a very _very_ amateur web developer, why would you not want to design a webpage like this? What difficulties would you run into if you designed a web-app in this way?

This is awesome. Looks great. Let's push it to production.

-management of company I work at.

We have an agreement. We keep our POCs so minimal they convey that they will break on everything except the happy path.

As for designs, I believe we use Figma.

I think the minor gripe I have would be the demo does not handle the D-pad, which will be part of the user interface. D-pad will bring the extra challenge of performing spatial navigation.

This is a great idea for a challenge. I generally like to burn an hour or two on a cross country flight making a simple computer simulation. Some good ones include: a triple pendulum, tidal forces, 3 body problem, sand piles, traffic jams...).

The UI actually comes together quite nicely when it is built with a powerful computer. On the low end inflight devices it is always so slow and clunky. A reminder to design for the actual device and not for the MacBook Pro running Figma.

> Slick draggable scrolling is a key part of many screens in Delta’s in-flight entertainment system (complete with momentum-based flicks, over-scroll, page-snapping, etc).

Then I'm going to have to fly Delta next month -- all the systems I've used have had such terrible latency that none of that was possible. I assumed it was crappy hardware.

Don't get your hopes up too much..

The last time I flew Singapore Airlines (maybe 3 years ago?) for an LAX->NRT flight and back, their system felt smoother than anything else I'd used on a plane before. IIRC it seemed to be built with Android or similar. Thought that was a pretty decent UI considering how bad in-flight systems usually are.

Something about the torture of flights makes this exercise sound as fun as playing coal miner in my backyard.

I can’t work on my laptop on a plane. The veiwing angle is atrocious. Can’t get the computer out away from me enough.

I would have been absolutely dead without access to Google, for CSS

Install Kiwix and a dump of MDN: https://library.kiwix.org/?lang=&q=MDN

> No accessibility, ... you just want something you can run through a user test.

As long as those users aren't disabled.

No access to Copilot is also a big constraint.

Reply on news.ycombinator.com