Reverse-engineering the UniFi inform protocol | Hacker News Reader

Reverse-engineering the UniFi inform protocol

tamarack.cloud

175 pointsbaconomatic76 comments

Nice trick. Just a heads up that I had to whitelist your domain as NextDNS blocked it for being newly registered.

Given this thread will probably attract other Unifi users... has anyone had success migrating from MongoDB to something like FerretDB?

I played around with getting this to work a few weeks ago and found that day-to-day it works without issue, but restoring a backup will error since it relies on some unsupported Mongo semantics (renaming collections iirc).

What does an admin do about NextDNS blocks?

If you subscribe to the mindset of "new domains are likely to be bad" you just deal with a steady stream of allowlist requests from your users until the end of time. There will be new domains until the end of time, and site owners shouldn't be doing anything extra (imo) to justify their existence to admins. If you use a firewall voluntarily and that firewall blocks sites that are legitimate, that's on you, not the site owner.

We get this a lot at my job, where many customers' admins block s3 buckets by default. We give our customers a list of hostnames to allowlist and if they can't figure it out, that's on them.

>If you subscribe to the mindset of "new domains are likely to be bad" you just deal with a steady stream of allowlist requests from your users until the end of time.

Newly-registered domains are not generally an issue with enterprise users. However, they are overrepresented in malicious traffic due to domain-generation algorithms (DGAs).

> Newly-registered domains are not generally an issue with enterprise users.

I take it this means enterprise users are not generally needing to do anything legit-for-work on a newly registered domain.

Enterprise clicks on newly registered domains tend to be (a) being phished or smished or cryptomined or whatever, or (b) someone reading X or Bsky or HN or ProductHunt's vibe code of the date -- things the enterprise would also like to have blocked.

Consider the CloudFlare/Proofpoint/NextDNS/etc. domain block on new domains much like updating one's HN home page to https://news.ycombinator.com/classic …

Sounds like a massive waste of your time for NextDNS admins and a poor UX for end users. If your security relies on trusting old domains, then you need to rethink your security. Also, I bet it's just as easy for you to accidentally whitelist a bad actors as to blacklist a good one. What am I missing here?

I don't disagree. The idea seems to be that newly registered domains are far more likely to be malicious (and not present on domain blocklists yet).

How are you performing backup of FerratDB? Are you using MongoDB tools, or are you using PostgreSQL-specific tools?

It seems like a pretty tall order, but I really want an open source access point controller daemon that knows how to provision and manage a wide variety of APs from different manufacturers.

So you'd have one services that can provision Ubiquity, MikroTik, TPLink and other APs and manage the clients.

Now that would be interesting! Multi-vendor support is on the radar, but haven't started looking into it much yet.

Alternately, run OpenWRT on the APs themselves, and then you just need one provisioning protocol.

Does it support seamless roaming of clients between group of APs?

Last time I've tried, it was not supported by any open source solution.

Well AFAIK the core seamless roaming in Unifi is using hostapd, which is the same AP software you use on OpenWrt. See 802.11r Fast Transition.

I think it should even be possible to get seamless roaming between Unifi and OpenWrt with correct configuration of hostapd.

Seamless WiFi roaming is mostly a client decision. The best you can do on AP is to:

a) optimize signal strength for coverage (stronger signals aren't always better in multi-AP deployment);

b) provide hints via 802.11k/v/r to help clients make, hopefully, better decisions;

c) forcefully drop and disassociate clients when signal is weak enough.

But if the client has bad WiFi implementation, there's nothing much you could do.

OpenWRT currently supports 802.11k/v/r, but optimizing coverage by adjusting signal strength and channels is left for experienced users to deal with manually. There is the are where some commercial offerings will do, but the result greatly varies. AFAIK there's no ideal system anyway coz physics is hard.

I would love to have some way to configure the USW Flex Minis without the controller software. I can't find any other small PoE powered switches for a similar cost.

> ("TNBU" is "UNBT" backwards, presumably UniFi Broadcast Technology.)

This seems like an odd misunderstanding, especially because the correct inversion “UBNT” is the default login name for most UniFi web UIs.

You might have a bit of dyslexia, OP!

ubnt has been the ubiquiti default login at least back to 2010 when I started using their products, before UniFi was a brand. I always assumed it was short for Ubiquiti Networks.

Sure, but the parent was saying this part was odd:

> "TNBU" is "UNBT" backwards

TNBU is clearly NOT uNbt backwards.

Using the network byte ordering (big endian) of UBNT as the magic number in the protocol is a nice touch.

I believe they used MIPS processors in their early gear, so that makes sense.

A lot of companies in that space did then. I was at a robotics company at the time and we experimented with mikrotik routerboards + the various long-range Ubiquiti wifi modules, some of which are even still listed on the website: https://techspecs.ui.com/uisp/accessory-tech/xr (though not the 900 MHz XR9, which was arguably one of the most interesting for long range comms)

You might be onto something there! But yes, good catch, I'll get that updated.

UBNT stands for UBiquiti NeTworks.

Do I miss something? How do you adopt the device in the first place? If you have to SSH into the device and set the inform URL manually could't you just route the request based on the request hostname?

Yep, once you set-inform the host header handles the routing. This in particular is most useful for things like DHCP Option 43, where devices only get an IP.

But if you only got that IP and a MAC-Address - how do you know which tenant is supposed to adopt the device?

We support two approaches, you can either pre-register MAC-Addresses or you can add source IP's to assist with that mapping. There is more information in our docs about this: https://tamarack.cloud/docs/migration

that's the key information to understand the whole thing.

it would be great if that could be in the article in the first place. (I'm assuming you are the author)

Good call, I should have made that more clear. Article updated!

The default inform address is unifi:8080.

Every network I've set up (which is not many) has a dns search suffix handed out by dhcp. So the wap will resolve, e.g. unifi.branch.megacorp.net with zero config needed.

"The whole proxy is maybe 200 lines of Go with an in-memory MAC-to-tenant lookup table." Is it possible to have the source code or is it private ?

The proxy is part of our platform so the source isn't public, but the concept is straightforward and the blog post covers the core logic. Happy to answer questions about the approach if you're building something similar!

I think you can even use standard haproxy acl rules and tables to extract few bytes from the request and route them correctly. No need for Go.

This is very cool.

I wonder if there's a way to control routing client side and remove the list of mac addresses. Eg manage DNS for customers (upsell ad blocking!) and CNAME the unifi entry to a customer specific vhost.

Thank you! DNS-based adoption works well for this. You point the unifi hostname at the tenant's subdomain and the Host header handles routing from there. We also have a DHCP Option 43 generator for setups where DNS isn't practical.

I don't quite get the reason for sniffing the packets. Wouldn't it be simpler to just run multiple VMs on one host to be multi tenant?

It would definitely be simpler, however the routing issue still stands. You would need to have a public IP for every VM, which is getting less practical. The MAC-based proxy makes it so we only need one IP and we can worry about the routing within our platform instead.

but it is http! why can't you do virtual hosting on a reverse proxy?

You definitely can once the device has a hostname set in set-inform. The MAC proxy is just for initial adoption where a device may only know the IP. Such as when you use DHCP Option 43.

You can after the initial discovery step, the article mentions this. The MAC routing is for the first step where the device is reaching out to try and find a controller and signal it's available for adoption, which uses an IP address at least in the scheme that is relevant for hosted operators. After that initial channel is established, the controller uses it to tell the device what its hostname is, and you can switch to more normal HTTP proxy routing thereafter.

Yep, that's exactly right!

Bit of a thread-jack, but has anyone reverse-engineered the UniFi camera adoption protocol? I was surprised to discover that, unlike the APs, the cameras can't be adopted through the Unifi Software Controller that you can just throw into a Docker container. You're supposed to do that through their NVR appliance (Unifi Protect). I was hoping to just use them with my open-source NVR. They seem to be about the only option for a reasonably priced, larger image sensor camera that is not made by a company participating in the Uyghur genocide (Hikvision, Dahua, Univision, Huawei).

I found https://community.home-assistant.io/t/unifi-cameras-without-... in which someone sshed in, edited some config files by hand, and got streaming to work for the current boot. One could probably take that a bit further and, you know, save the config to flash. But it'd be nice to just do it the way their controller does and know it's going to work for future firmware updates and such.

They also stream by connecting to your NVR with modified version of flv, rather than you connecting to them with RTSP, which is annoying but can be worked around.

If you want to bypass Unifi Protect, what sort of "adoption" are you thinking of? AFAIK, "adoption" is a Unifi Protect thing. Otherwise it's just a device on your network that you can configure Frigate etc. to connect to and pull streams.

Changing the credentials for web access (firmware upgrade, janky jpeg-based live stream, etc.) and ssh access from the default ubnt:ubnt. Surprisingly, I don't see a page for this in the web UI, and the `password` command in the CLI is ineffective. I haven't looked around the filesystem.

Setting where it sends the video stream.

Configuring video settings, zone detections, etc. I found a video going through them here: <https://youtu.be/URam5XSFzuM?si=8WK4Yghh9kidZe6c&t=279> Just about any other camera lets you change this stuff through the camera's built-in web interface and/or ONVIF. Ubiquitis apparently don't.

> Otherwise it's just a device on your network that you can configure Frigate etc. to connect to and pull streams.

No, it connects to you!

You want to change the credentials of the camera, so Frigate can log into it while it is connected to your Unifi network?

I did that for 5 different cameras yesterday, you're saying Unifi's cameras doesn't allow user management? That sucks!

> No, it connects to you!

I thought frigate connects to the camera's RTSP stream (maybe with ONVIF in the mix)?

> I thought frigate connects to the camera's RTSP stream (maybe with ONVIF in the mix)?

Right, that's the expectation of Frigate, my own Moonfire NVR, and basically every other NVR out there. Ubiquiti decided to think different.

Well thanks for the heads-up to avoid their cameras.

I hear you, but on the other hand, I'd take a bit of interop pain over supporting genocide any day. It looks like with the hints from moonlighter and from https://github.com/keshavdv/unifi-cam-proxy I'll be able to get this to work.

Honestly it might be less work than some other cameras that (allegedly) speak RTSP. You'd be shocked how low-quality these implementations are. Never advancing timestamps, setting the RTP MARK bit arbitrarily, writing uninitialized memory framed as audio packets (on cameras that don't have microphones), closing file descriptors then writing data to them anyway (and so having it show up on the next accepted connection to be assigned that fd even pre-auth), etc.

FWIW, there are multiple other camera brands who don't manufacture in Xinjiang (or China for that matter), like Axis or Vivotek.

I've eyed Axis cameras but they're pricy (particularly for large sensors) and don't seem to come in the turret form factor I prefer. E.g. the AXIS M4317-PLVE is a dome and $717 at newegg. Kinda a weird model actually—180/360 degree view which isn't what I'd want. But I haven't found anything that is at a price I'd like to pay myself, let alone recommend to others for home use.

Vivotek's a bit more reasonable but still. The (brand new?) Vivotek VIT04A-W is the closest I've found—1/1.8" sensor, 4MP, turret, $535 on jmac.com.

These Ubiquiti models seem really nice in terms of hardware specs and and very reasonably priced. $200 for a 1/1.8" sensor turret, $479 for a 1/1.2" sensor turret with extra AI features. Same general price bracket as Dahua, I think.

Check secondhand. I'm finding quad sensor cams that retailed for $2k purchased off craigslist for $100. Wipe em, flash new firmware, deny all internet access at the network level, and you're good to go

Arecont Vision is another good brand. I’ve got a friend that got a bunch of Arecont domes stupid cheap and they have amusing modes like “casino mode” (guaranteed 30fps recording for various gaming regulations).

https://www.arecontvision.com/news/arecont-vision-adds-casin...

(eBay deal sniping sometimes gets you some funny deals but YMMV — I picked up an Axis Q1700-LE license plate camera for under $200 for some experiments.)

> writing uninitialized memory framed as audio packets..., closing file descriptors then writing data to them anyway...

Thanks for the reassurance that I'm not such an incompetent dev as I feel.

Funny how companies tend to be competent at either devices or software, and rarely both. This sounds vaguely like the automotive industry.

Unifi cams don't stream RTSP, they stream FLV v1 (FlashVideo) on 3 streams over plain TCP on port 7550, one per quality channel. And yes, they stream that TO the NVR who adopted them only... then the NVR recodes and sends RTSP (configurable).

For the adoption stage, UniFi cameras broadcast on UDP port 10001 using a proprietary TLV (Type-Length-Value) protocol. The Protect console listens on this port and picks up new cameras immediately. 4 bytes `\x01\x00\x00\x00` sent as UDP broadcast to `255.255.255.255:10001`

The response then contains these fields:

  | Hex Code | Field | Data |
  |----------|-------|------|
  | `0x01` | MAC Address | 6-byte hardware address |
  | `0x02` | MAC + IP | Combined MAC and IPv4 address |
  | `0x03` | Firmware Version | String |
  | `0x0B` | Hostname | String |
  | `0x0C` | Platform (Short Model) | String |
  | `0x0A` | Uptime | 64-bit integer |
  | `0x13` | Serial | String |
  | `0x14` | Model (Full) | String |
  | `0x17` | Is Default | Boolean (adopted vs unmanaged) |

After discovery, the Protect console: 1. Connects to the camera via SSH (default credentials) 2. Configures the Inform URL (TCP 8080) 3. Camera registers with the controller

So conceivably at step 2 you could use your own modified URL to point to your own NVR and then grab the FLV streams from there.

Thanks!

> 1. Connects to the camera via SSH (default credentials) 2. Configures the Inform URL (TCP 8080)

Not what I expected but okay. Looks like there's a `set-inform` command. It looks like it opens a TLS connection, doesn't check the certificate, and tries to opens a websocket:

    GET /camera/1.0/ws HTTP/1.1
    Pragma: no-cache
    Cache-Control: no-cache
    Host: ...
    Origin: http://ws_camera_proto_secure_transfer
    Upgrade: websocket
    Connection: close, Upgrade
    Sec-WebSocket-Key: ...
    Sec-WebSocket-Protocol: secure_transfer
    Sec-WebSocket-Version: 13
    Camera-MAC: ...
    Camera-IP: ...
    Camera-Model: 0xa601
    Camera-Firmware: 5.0.83
    Device-ID: ...
    Adopted: false
    x-guid: be9d8e45-62a8-ae84-8b23-71723c7decaf

I might try accepting the websocket but I have a feeling I'll get stuck about there without knowing what the server is supposed to send over it. I'm debating if I'm willing to buy a Unifi Protect device or not.

...then again I did a search for a couple strings and ran across https://github.com/keshavdv/unifi-cam-proxy . It's the opposite direction of what I want (makes a standard camera work with Unifi Protect) but maybe contains the protocol details I'm looking for...

> ...then again I did a search for a couple strings and ran across https://github.com/keshavdv/unifi-cam-proxy . It's the opposite direction of what I want (makes a standard camera work with Unifi Protect) but maybe contains the protocol details I'm looking for...

Actually, yes. I got lazy and just asked Claude Code to write a server, using that as a reference...and it worked. It was able to change the password and have it start streaming flv video. Not exactly a production-quality implementation but as a proof-of-concept it's quite successful.

There you go! I wrote a proxy server to deal with Unifi cameras and also dewarp their 360 camera streams... and used Claude Code to reverse-engineer most of what's going on. Sniff the entire network traffic between their NVRs and cams via Wireshark/TShark, grabbing the NVR's web socket streams, and also had it write a custom METAL shader pipeline native to Apple silicon to replace ffmpeg which was way too slow to deal with 5K 360 streams and dewarp them. All in a matter of hours. Amazing times ;)

I actually just registered here to comment the first time after lurking for years ;) I had the displeasure to upgrade from a broken g3 flex to a g5 flex, and then finding out they cut out the rtsp stream functionality for unknown (probably business) reasons. I dont plan to buy a protect appliance, and frigate should still handle my nvr stuff, so this comment thread comes really handy right now. Is there any possibility for you to publish this proxy to github or similar?

If I'm understanding correctly moonlighter's proxy speaks Ubiquiti's protocol on both ends, "just" altering the video stream along the way. Which is pretty cool but not what you're looking for.

I'd like to make a production-quality version of what you're looking for: acts as an RTSP server, adopts Ubiquiti cameras. As a single-binary thing that you could run anywhere, maybe even on the camera itself. (self-adoption? emancipation, as in "emancipated minor"?) But it'll take me a bit. My RTSP library right now is client-only, so it needs a bit of expansion to do this. Server support is in my TODO list along with several other changes. https://github.com/scottlamb/retina/issues/89

Here's that YouTube link without the creepy Google tracking component:

https://youtu.be/URam5XSFzuM?t=279

Im using a g3 flex with frigate without use of unifi protect, works fine.

I think newer models like g4 flex dont support this thou.

Nice hack! Would hosting controllers on individual IPv6 addresses on the same VM solve this too?

Yeah IPv6 would solve this because we wouldn't have to go through the proxy at all, since each VM could get their own. Unfortunately IPv6 isn't quite as supported as we'd like.

Ubiquity in general has had garbage support for IPv6 since the beginning.

They claimed most of their customers aren't asking for it when I pushed them on the issue years ago.

Yup, absolutely. I'm so annoyed by this. I haven't been able to get either my Comcast IPv6 address pool set up and usable throughout my home network (three switches and two Unifi Pro 6 WAPs, across a few VLANs), nor my HE.net tunnel.

So annoyed.

I use Ubnt for layer 1 and 2. Their switches are nice if your needs can be met by their spog. Same with their access points although I haven’t see a good one from then since WiFi 5. Their latest WiFi 7 ap is top dollar but used last generation chip and has a cooling fan :[

Dont use unit for routing, though.

Is it just me that pretty much cannot read most of the text in the "Reading the MAC" code block? I don't know if it's because I use dark mode, but some of the text is #24292E on top of #141A16, which for me at least is practically invisible

Sorry about that, I typically use light mode, fixed and deployed!

A million times better, thanks =)

Thanks for calling it out!

Controller uses way to much RAM compared to OpenWISP and good luck if a device is EOL. Lots of $10 USG-3P's out there.

FWIW, the USG-3P is listed as supported by OpenWRT.

I must admit, it's a better choice than a Raspberry Pi.

I'm glad the payload was usable and the author has fixed their problem, it's an interesting challenge.

However, there are other approaches. A public IP per client isn't going to be nearly as expensive as a VM per client, and lets you route your clients by target. Or you could route by source IP: either by having the client register their IPs, or with some combination with seeing where folk log in from.

Neither is necessary, though, given inspection does appear to work.

I don’t get this comment. Inspection does work but the suggested alternatives don’t.

Having the client register their IPs isn’t tenable for most folks. What’s my IP at the shop? (No idea) Will it change? (Yes) now it’s broken.

Seeing where folks log in from isn’t nearly the same as where their UniFi networks are located. (Store vs home.) Broken.

So neither of the those are robust approaches whereas the author’s solution is bulletproof and simply works in all cases.

No offense, but why suggest “other approaches” that have such major holes? Why not just cheer on the solution that works all the time?

The author framed his issue as a choice between separate VMs (with high cost) per user or decoding the messages. As he, you, and I all say: what he's got does work. I'm absolutely not saying that now he's solved the problem he should do something else. But the choice wasn't between only those two extremes.

This protocol was amenable to inspection, the next might not be.

I use NextDNS, one of the features it provides is letting you register a source IP so requests from your network "just work". It might not be a mainstream consumer feature, but neither NextDNS nor managed Unifi controllers are mainstream consumer products.

Reply on news.ycombinator.com