I definitely agree with the rest of the comment. It's a good solution. I inherited a software product that worked like this at a past job, and it was great. (And now I'm trying to convince my current job that we want to move to this model for normal servers in datacenters, very much not IoT.)
A couple of complications:
- You should think about the fact that this means your root partition changes. Either you want to structure your system with separate read and write partitions and bind-mount the relevant directories from the write partition, or you want to make it completely read-only / stateless. Remember that /var/log is traditionally on your local disk, so if you don't do anything special, you'll even get two /var/logs on each device, which may or may not be what you want.
- You do want a management server, as this document suggests, to track which devices have actually updated and which haven't, so you can manually send people after devices that are just behind a terrible internet connection.
- You want some mechanism for detecting if the new version doesn't work and rolling back; this is basically as simple as setting a "I just tried partition X, if it doesn't work don't try it again" flag in the bootloader on boot, and clearing it once userspace is up (and when the partition gets rewritten with a new version).
- The updates should be signed etc. as described in the document. Depending on your threat model, you might want to prevent replay attacks that cause an attacker-controlled downgrade by giving it a higher-versioned filename; either use HTTPS to an update server you control, or use signed metadata files with timestamps.
The fact that you get image-based deployments instead of dealing with apt upgrades from arbitrarily-old versions (and thus inevitably slightly drifting configs on devices installed at different times) is fantastic.