Analysis and reverse-engineering of the original Starlink router
olegkutkov.me
olegkutkov.me
This is typically done in order to be able to safely update the u-boot bootloader, retaining the ability to boot the previous version if the upgrade fails. Also called A/B firmware update. The previous bootloader stage ("SBL" or "TZ", depending on secure boot mechanism) might check a flag in a configuration partition to see which loader should be started and whether a previous boot to that loader was attempted, and will revert back to the previously active one after an upgrade that fails to boot up, or fails to pass secure boot integrity checks.
A failure could for instance happen if the power drops while writing the update to the u-boot partition. Without such a mechanism, an update failure would brick the device. Alternatives are "never update the bootloader", or "hope for the best".
You can even boot and update your BIOS without a working processor. Did this over Christmas when the mainboard/CPU combo I got for a family member wasn't compatible. Just put in a USB drive with the correct file name and press a button on the board and it updated and I was good to go.
A little of topic, but just wanted to state how much better these things are than the old days where it was easy to brick a board.
End consumers normally do not have access to some source code or any documentation about these chips, sometimes even the competitors are getting access to source code to integrate their solution better. I do not know whom they protect against, probably all people who did not sign a NDA.
These thinks are only shipped to end customers in binary only version to protect the IP from someone. Often it is pretty easy to tell the system it is now operating in a different country (e.g. setting in Web UI) and then it will not comply to the local radio regulatory requirements any more. From my experience it is not the FCC which really demands it, please blame the chip vendors for binary only driver first.
The worst hacker for a silicon vendor, where they normally protect most against, is some guy like the author of this blog post who analyses his device in much detail and tries to run own software on it which was not verified by the vendor first. Such software modifications could cause worse customer experience because it is performing worse (which is funny if they have to reboot the vendor software every 20 days) or could cause extra support effort.
No. They are jus that hard to work with.
MTK, Qcom, BCom are all terrible with software support, no matter who you are.
Apple always demands to be in control of what is running on their devices, no matter what - even where it's pure software that's concerned, like with the NVidia fallout - and they (usually) insist on quality... there's a reason why Apple hardware feels so painless to interoperate, compared to the Windows or Linux world.
Silicon vendors often do not care about small customers, small is probably less than 250k units per year. If you are a customer which is expected to generate more than 5% of the revenue for the complete product line or the business unit then you get good support from these companies.
The proprietary Broadcom Wifi driver was leaked multiple times by some companies which did not run the script to clean a GPL tar, but did something on their own and forgot to remove the Boardcom wifi driver source code. Last time I saw this was already 8 years ago. The Broadcom proprietary Wifi driver also contained the source code of the firmware running on the ARM chip inside the Wifi IP, at least it did this some time ago.
For the Qualcomm 5G modem and the graphics driver this is probably different.
Sometimes, while you go back and forth with the vendor in a hapless effort to repro anomalous bugs, the best thing for the customer is reboot-every-N-days instead of degraded-performance-forever.
It sucks, but its commonplace.
(N.B. I work in this space and its definitely been a pain point that we've had to accept to be able to make progress)
> This app is written in Go, and it’s really hard to disassemble. Shouldn't it be easier than an app written in C/C++? As far as I know Go embeds reflection metadata, including type information, which should help with decompiling. I have never tried though, so I might be wrong.
* MT7622 SoC (2x Cortex-A53, including 4x4 ieee80211n wifi integrated in MT7622 and supported by upstream kernel too)
* MT7915 Wifi 6 (ieee80211ax) 4x4
* MT7530 5 port 1G switch (2.5G uplink)
* Linksys E8450 / Belkin RT3200 (same device just different name and color of casing)
* Ubiquiti UniFi 6 LR
* Also some others
Edit: Maybe not? https://publish.twitter.com/?query=https%3A%2F%2Ftwitter.com... ... though it is already "later this year".
Also how do they update hardware on a satellite? Or are there other problems keeping them from establishing intersat links for now?
This is the only reason I didn’t get a starlink subscription. Seems like a bullshit limitation. Reminds me kinda of Tesla stink.
1) The Starlink Constellation only looks for your base station in it's allocated cell and isn't setup to scan and discover mobile stations
2) Right now there is no communication inside the constellation (they might have launched a few with the laser links but it's not turned on broadly) so the satellites are linking down to another base station that has access out to the wider internet. They need to provision access on those so they don't get overwhelmed.
3) The base station itself needs to know where it is well enough to hit the active satellite for it's cell. You might be able to do that without GPS but it is an issue you have to address.
Both of these are supposed to be solved eventually and they've said they are planning to allow roaming base stations eventually.
First and primarily, the legal front: All regulated electromagnetic spectrum transmission is country-bound, since the regulators are by country. Private entities cannot legally transmit in any given country without authorization, and there is a fairly in-depth set of national and international legal frameworks for this that are pretty well respected. In principle SpaceX could negotiate the proper licenses and agreements internationally to allow roaming, and they certainly will eventually for certain applications like maritime and aircraft usage. But it's perfectly understandable in their bootstrapping phase of pure fixed point service (which is the easiest for a host of reasons) that they just put a blanket ban on it. Those who want mobile just have to wait.
Second, there are technical reasons. Starlink will always face some fundamental density limitations in how small a cell any sat can talk to and how much bandwidth there will be per cell and for the overall sat. And in its initial version, satellites also act purely as "bent pipes", mediating directly between a ground station which then talks to the internet at large and the end customer CPE. And SpaceX is being careful and deliberate about building up all the infra needed to scale its network, and wants to ensure that everyone joining gets a good experience. So they're also being fairly careful about allowing anyone out of their assigned cell.
Eventually a number of developments will alleviate this to some extent. Intersat optical links are now launching on new birds, and IIRC this year (maybe early next?) they'll have the in-space routing network up. That will dramatically increase their flexibility in routing traffic and dealing with chokepoints (as well as allowing Starlink usage out of range of ground stations, again critical for ships/aircraft). Simple increasing raw numbers of sats (and particularly VLEO v-band with Starship) will improve their density margins a bit as well. And then there is just plain old software infra stuff. Portal for customers to update addresses, backend to deal with handling allocations and looking for cell overload including across varying regulatory regimes, figuring out pricing at the margins and what to allocate for fixed vs mobile usage in a given area, letting people see where there is room and where there isn't, and putting a nice GUI over all of that is work that isn't needed for a 1.0 MVP but will be important to getting stuff to work right.
>Seems like a bullshit limitation. Reminds me kinda of Tesla stink.
No, this one at least is pretty reasonable. Easy to forget that Starlink is doing something very different, new, with more to come, and is very, very early. And still a chance it could all fail, they haven't actually hit their core minimum economic targets yet (that was what Elon's "bankruptcy" warning last month was about). It's much more capex intensive and opex intensive then traditional offerings. They've got an excellent product though and are bootstrapping it impressively but they've got to be very careful nonetheless. Plenty of fundamentally incredibly promising businesses have foundered on the shoals of scaling.
1. It gets really hot. Like, it kinda makes the plastic routers seem well-ventilated.
2. Congestion is, sadly, a real concern here. Too many devices will definitely cause the router to choke a bit, and if you're going to share it with a family that's just something to be aware of.
3. Latency is fine wired, but a little shaky over WiFi. Especially recently; I'm not sure if this is a firmware issue or what, but there's some funky QoS issues that I've noticed lately where latency is wildly unpredictable. It almost shut down a Zoom interview the other day...
4. You get one LAN port. That's it.
Besides those gripes, it's fine, but it leaves a lot to improve on for the next iterations. Hopefully Starlink is taking notes here, because as nice of a device as it is, it does feel a little rushed for the purpose of getting it out the door.
https://www.esat.kuleuven.be/cosic/blog/dumping-and-extracti...
It would need adaptation at the application layer, but the data management needed is minimal. It would also need some attention at the central terminus / edge node serving the area, to identify duplicate requested streams and shift a corresponding stream to the broadcast channel. In turn, this would typically depend on cooperation from the original source of a stream, typically mediated by their contract with an edge cache service.
The extremum of usefulness for this facility would be when hundreds or thousands of terminals in an area are used to watch the same World Cup football match. In this case, the cache storage footprint would be minimal, as almost all viewers would be watching at the same point in the broadcast.
More commonly, a few blockbuster movies and local news might dominate broadcast traffic. But no advance planning or personal attention would be needed; any stream found being fed to more than one downlink could be shifted to broadcast according to the amount of bandwidth that would thereby be saved.
As laser inter-satellite links are added to the constellation, a single uplink stream could serve simultaneous broadcast from multiple satellites in a chain.
Multicast routing uses, exclusively, point-to-point links. So, a packet could go up to the satellite once, but would need to be copied down to every terminal individually.
But internet video is in any case not distributed by multicast UDP. Even if it were, you still would need caching for viewers not watching identically the same frames at the same time, because a multicast router does not do time-shifting: all copies of each packet are queued to send immediately.
Of course the transport link would have to cooperate and open a group-encrypted channel for every multicast group a client is subscribed to. Basically how regular satellite TV works, but with dynamically allocated bandwidth.
Once you get to radio spectrum transmission, each pair of endpoints competes with every other pair in range, in a strictly limited pool of bandwidth. Adding another terminal takes away capacity from all the others. A multicast stream going to your satellite and then distributed to 100 terminals by multicast burns 100x the fixed available bandwidth as each terminal gets.
The only way to get any benefit from batching delivery over radio spectrum is via broadcast. For a packet network, you have the extra chore of making those broadcast packets actually useful to as many terminals as possible. In practice that requires caching of broadcast content in the terminal, so that when it is actually asked for it is already there.
Multicast is indeed problematic for wireless networks, even beyond what you describe (e.g. FEC, ACK storming, retransmits, etc). But if the phy layer is using OFDMA or similar, its entirely plausible to dedicate a subcarrier for multicast traffic. This doesn't solve the O(N) terminal problem you describe, but it mitigates the impact for concurrent transmission of unrelated unicast traffic while the multicast delivery is happening.
There's a fairly large academic body of work on this specific topic re: OFDMA multicast. And though I think your note on caching is a necessary component of this (a la Open Connect), I still think there is a role that multicast can play here.
ACK storms aren't a thing with multicast; if FEC doesn't get you what you need, you wait a random time and send a NAK up the line, and the sender re-sends the requested bit. If the bit you were about to request shows up again before you get to asking, you discard the NAK you would have sent. So, if a lot of receivers miss the same packet, only one NAK usually goes back.
The Starlink transceivers are SDRs and extremely versatile. So, instead of playing games with modulation and subcarriers, it probably uses its whole band on each packet, effectively broadcasting all packets, and receivers just pick off whichever packets are meaningful to them. The ones actually intended for more than one terminal are the "broadcast" packets, differing only in how they are processed by the terminal.
The actually tricky thing is that broadcast streams cannot be like the regular end-to-end packet streams, where identical images are delivered in absolutely unlike packets. You need a higher-level negotiation identifying streams of interest by name and playback position, so that the uplink terminal knows when it is sending (or will have sent) the same stuff twice. If it decides some of its traffic should be broadcast instead, it can substitute packets from an equivalent, shared stream. The receiving terminal, meanwhile, is told that the stream it was getting is ended, and it should start extracting and caching content from the substituted broadcast stream.
The usual DRM nonsense will make this annoyingly more complicated: the participating broadcast receivers get a copy of a key to decrypt the broadcast traffic. Everybody else throws it away as indistinguishable from noise, like all traffic not meant for them.
Interesting. This has not been my experience during my time so far in building a 802.11-over-mmWave ISP for the last 7 years. BUM traffic is routinely an issue, one that we measure frequently, and multicast ACK storming is absolutely a side effect of times when things are bad. This is usually because multicast, at least with 802.11, has to be sent at the lowest possible MCS because you don't know the rate capabilities of the STAs on the other side of the fence (also true for broadcasts) and as such the delivery is inherently unreliable. When things are bad, ACK storms are the measurable sine qua non of these events.
Its possible that, for Starlink, this isn't an issue for them because its not a conventional PMP infrastructure as you point out. They know the rates of the terminals on the other side and they're not using non-proprietary modulation.
FEC (or even network coding) can certainly help in these situations, but it has been far from a silver bullet in most testing I've been a part of.
> The Starlink transceivers are SDRs and extremely versatile. So, instead of playing games with modulation and subcarriers, it probably uses its whole band on each packet, effectively broadcasting all packets, and receivers just pick off whichever packets are meaningful to them.
Even if its an SDR, and presumably using something like phase-key shifting for modulation at the PHY layer, there is still MAC layer multiplexing it could use to offset the impact of multicast. E.g. in 802.11ax with MU-MIMO, you could dedicate an entire spatial stream to one kind of traffic, or alternatively you could dynamically put STAs into MU groups based the bulk of the traffic they're sending to blunt their impact on the shared spectrum. This is even more interesting in the SDR context where you can build a more adaptive algorithm for grouping (e.g. like using geospatial context to ensure minimal interference etc).
>The actually tricky thing is that broadcast streams cannot be like the regular end-to-end packet streams, where identical images are delivered in absolutely unlike packets. You need a higher-level negotiation identifying streams of interest by name and playback position, so that the uplink terminal knows when it is sending (or will have sent) the same stuff twice
I think this is where your point about the proprietary SDR is the most interesting (and maybe even for the DRM case as well). Would be interesting to see how they could handle this being in the middle of it all, without causing a major performance impact to end users. But thats mostly an implementation detail.
Starlink will not have that problem, anyway. They could invent their own horror story, and then have no one to blame but themselves.
A more complicated modulation scheme could enable cheap hardware in the terminals to see only a fraction of traffic, which might be worth the RF complication. That probably depends on how much of the protocol work they are willing to do in the terminal's FPGA. Certainly the CPU cores in the terminal won't be up to sorting through the whole pile.
An FPGA configured to watch for a small number of packet headers of interest, and decrypt and deliver just those packets, ought to be able to relieve the CPU core enough.
But it’s free and it works!
I understand some Canadian arctic cities want to do what you say because they depend on x$/gb satellite links, but not big enough for a Netflix or whatever appliance. But every time there’s an iPhone or windows update, everyone is downloading it independently. It would also make sense to load a box of SD cards with their supply flights and put them on-net. Dunno if a formal system exists for that.
Unfortunately, even though its receiver was supposed to enter production in November, it's still not available. Ordinarily I'd blame 'rona/supply chain/whatever, but according to Wikipedia, it failed to deliver a previous kickstarter.
When it's available, I'll buy one. But I won't be the first, in case this turns into vaporware.