Microdot: A Web Framework for Microcontrollers
lwn.net
lwn.net
I would think Lua or even C based web framework for MCU and restricted embedded boards will be a lot more resource efficient.
Typically you can just run a few cgi scripts with bootstrap as the UI and jquery as the javascript, but a simple MVC will be really cool.
I understand making them more approachable for developers at a higher level, but the resulting solutions need an order of magnitude more resources to accomplish the same tasks when compared to writing an equivalent system in C.
Assuming these kinds of tools are being used temporarily for proof-of-concept, I think that it is a great fit; however, we all know that there's nothing more permanent than a temporary solution.
> the resulting solutions need an order of magnitude more resources to accomplish the same tasks when compared to writing an equivalent system in C.
I mean, yeah... but that's true when you're writing Python code on a normal computer as well. It's not really an argument against writing Python except in cases where you're actually running into resource constraints.
If you can get the resources you need in a microcontroller you can afford, and you prefer programming in Python... why not?
Particularly when you consider that the "bloat" here has no secondary side-effects: it doesn't waste resources that other software could be using, because no other software is running.
I am a long-time web developer who has always intellectually understood the interest in microcontrollers but never had much of the need. I do now.
I'm not even a fan of Python but MicroPython is a surprisingly productive development experience and I've learned a lot in a very short time with a Pico W.
I will probably use TinyGo at some point in the future, particularly as I try to use smaller devices, and I'm not averse to porting to C, but I really rate the MicroPython ecosystem highly in terms of actual coverage, though mainly just the intent of it, which is unimpeachable IMO.
> it doesn't waste resources that other software could be using, because no other software is running.
While this is interesting observation it is not always true. A lot of devices are low power, and depend on battery.
It really depends here. The lower the volume, tighter the iteration time, etc the more sense it makes. I know oil rigs, for example, have a bunch of rpis running on them, because the manufacturer is selling a $100k box and any amount of time spent shaving $10 off the BOM is really not worth it at all.
"Embedded" development on a highly specced SBC running Linux (like a Raspberry Pi) is a long ways away from what MicroPython is targeting.
Still, even so, I decided to give MicroPython a go for a new design; an industrial product we would ship in the tens of thousands of units.
This will count as one of the many choices I will regret making in my business history. I won’t bore you with details. We shipped 250 units. We are busy porting the code to C as fast as humanly possible.
At least I can say we tried. I have coded embedded systems with assembler, Forth, C and C++. Any one of these would be better than Python.
Yes, I use Python extensively for desktop, data science and web work. It simply isn’t a good fit for embedded work.
Things that are trivial to code in a performant way in C are impossible to code in Python. A simple example of this is a cyclic redundancy routine. We had to write those in assembler. Also, garbage collection was problematic, even controlling it manually. If I remember correctly, it took out a 5 us chunk of time.
We had to resort to such things as allocating large buffers globally to avoid allocation/de-allocation issues, etc. This was done mostly by declaring global variables as tuples (due to the simple memory structure) and then accessing them with assembler routines. The immutability of tuples only exists within the confines of Python, outside of that, it’s just memory. This approach resulted in massive performance and memory utilization (reduction) gains.
Beyond that, the entire codebase was a single function running a state machine, this to avoid costly (time and resource) function calls.
Eventually the thing starts to look like a typical bare metal embedded system where you create a round robin scheduler, etc. At that point it is very hard to justify using Python for anything, in fact C is far easier.
Code like that quickly becomes something that is hard to maintain, enhance and evolve. It became very obvious that we had to switch the product to C as soon as possible.
I had an nrf52(840) microcontroller and it took me hours to get "hello world" out of that thing with nRF Connect/Zephyr. Granted, I had no idea what I was doing, but accomplishing the same with equally-foreign-to-me Circuitpython took five minutes - just plug it in to a USB port and drop the bootloader and then your .py into file explorer. I _want_ to do things "properly" and efficiently, but for a small project I can't justify the time investment to get there and so the -pythons provide a tradeoff I'm willing to make.
Okay, and? It's 2024, even a $2.50 toy devboard had multi-megabit flash & ram. I really don't care if my code takes up 10k or 100k. If Python makes it easier to develop and eliminates entire categories of vulnerabilities in the process, it doesn't make any sense to use bare C instead. I paid for the entire chip, why shouldn't I use the entire chip?
C is fine if you have to do high-speed bitbanging or have to squeeze every last byte out of a chip, but for most applications that's simply not the case anymore. A decent fraction of embedded systems is absolutely trivial on today's hardware. It's time we start treating them like the mini-computers they are, because remaining in a 2000s mindset is going to result in a lot of compromised network-connected devices due to some developer fat-fingering a buffer size.
"multi" is "2 or more"
So that's 256KB+ or RAM and 256KB+ of flash. Please send me a link to a $2.50 board with those specs. I could use one at that price point.
[1]: https://imgur.com/HUt2qLY [2]: https://www.aliexpress.us/item/3256804697715980.html?
I’m coming at it from opposite direction. I’ve been learning C to program ESP32 devices and it’s such a frustrating experience how much that language is lacking or has to be DIYed. I could easily use python and move 10x faster, but I’m trying to force myself to learn C in the process, otherwise I have no idea what the benefit would be.
Keep at it; the benefits to learning C/C++ are immense. They have become the underlying "lingua franca" of the software industry allowing you to study/program applications/libraries/system tools/OS kernels/bare-metal MCUs etc.
For lots of C code to study for MCU programming, see the books (some are available for free) by Michael Pont of SafeTTy Systems here - https://www.safetty.net/publications Though they are for 8051 and not for ESP32 you should be able to understand the techniques involved and use them with your choice of MCU family.
I had a python-based game of life that drove a 16x16 LED panel but the frame rate wasn't very good so I rewrote it in C++.
I gather other microcontrollers have had programmable I/O state machine cores in the past but I'd expect to see a lot more of them (or more documentation and coverage of the ones that exist)
At high school I was taught Arduino and C++ but younger students have rPi Pico's and CircuitPython (Adafruit version of MicroPython), to use the same language as the CS syllabus. I guess the "easy" features of Python (many traps once programs get complex) make it attractive as a first programming language.
One project I've worked on for 3 years with a C++ codebase is going to be passed on to mostly Python programmers... I'll just have to make sure the hardware documentation is good.
Very few people are doing microcontroller programming these days, and so beginner-friendly microcontroller resources are far less plentiful than those for mainstream languages. Just teach kids Python on a normal laptop (or in the cloud if you're in the US and give them Chromebooks).
I guess the ability to interact with the physical world (with servos and such) might make the subject more attractive to some, but I'm sure there are ways to plug those into a real computer, or at least a Pi.
There are definitely ways to plug servos, etc. into a normal computer, and some of them are already branded as teaching tools. See SAMLabs, etc.
This should make you happy :)
1980s embedded engineers please stand up.
-------
But BasicStamp was a thing even decades ago (but after 1980s). There are plenty of advantages to interpreted languages, and they were a bigger deal back when 8052 or whatever we're EPROM and harder to program. I mean, physically harder, with UV to erase the flash.
Having a portion of the code execute out of dataspace thanks to an embedded interpreter was worthwhile, even with all the extra CPU cycles and RAM wasted.
I see Micropython as just an extension of that. Python displaced BASIC a long time ago for good reasons.
Even lua needs ~100K IIRC, and that's more than the entire address space (including RAM, ROM and IO) of a 6502.
So there's a wide number of under $5 boards available that can run MicroPython, providing an easy start to embedded programmers.
------------
Of course, 8-bits are needed for minimal power consumption and maximum efficiency, with an ability to work off of 128kB of ROM and 8kB of RAM at under $1.
But more importantly, 8-bitters are more complete systems. Better ADCs, often DACs, OpAmps, and other advanced features that RP2040 (and other 32-bit systems) got rid of to make room for RAM or ROM.
That's the thing: 8-bits are more convenient and easier to use not because of code issues, but because they come with all the tools you need on one platform.
---------
But beginners don't know how to use those tools *anyway* and are instead focused on blinking lights or basic I2C / SPI controls. I think grabbing a 32-bit Cortex M0+ (like RP2030) or similar system to run MicroPython isn't a bad thing.
Its inefficient and better designs exist. But beginners shouldn't focus on optimizing yet. Its just like BasicStamp or BASIC-for-8052 back in the day IMO. You need a beginner platform so that young teenagers can experiment with this stuff, and young teenagers aren't going to read the 300+ page manuals that come with an AVR64DD32 to fully understand an 8-bitter's operation.
But even a teenager can plug in a USB port, click a few buttons on an IDE and load Micropython to a RP2040 / ESP32 / whatever.
Been using esp32-rs recently (esp32 is an admittedly beefy platform/soc) and its been real pleasant.
https://github.com/pimoroni/phew
and this rather clever start-as-access-point-to-join-network scheme implemented on it:
https://github.com/simonprickett/phewap
which works a treat!
Microdot: Yet another Python web framework - https://news.ycombinator.com/item?id=38834156 - Jan 2024 (26 comments)
Microdot - https://news.ycombinator.com/item?id=38156060 - Nov 2023 (26 comments)
I'm curious to hear what other similar products are out there that others are using?
Edit: Sounds like Microdot runs the entire web framework completely on-device? My initial understanding was wrong in that case; it's an apples-to-oranges comparison
[1] It seems like Microdot handles more of the stack and is probably more hobbyist-friendly based on the fact that it targets MicroPython. Or maybe it's just better all-around; I haven't tried MicroPython so I can't compare.