CI/CD with KiCad and Gitlab
sschueller.github.io
sschueller.github.io
I had it set up as a CI job for a while, but I ended up disabling it for a few reasons, the main one being that I just don't think it's a good idea to constantly create build artifacts for circuit board designs. It has created problems when people have circulated intermediate schematics generated by CI that don't match the hardware we built.
You really only need to archive fab packages (by which I mean schematic, gerbers, BOM, assembly drawing, fab notes, etc) at the points in time where you fab and assemble a board.
Creating more artifacts just creates the possibility for more noise. It's not like software CI as there's not really a way to automate hardware testing in a CI server. Very little incremental value in continuous artifact generation as a result.
I've used kicad professionally for about three years now. That brief phase of using CI was in my current employer and we had some confusing weeks where no one knew which sch version was the right one.
Being able to share out autogenerated schematics is one of those rare cases where CI hinders you rather than helps you.
I'm sort of tunneling in on this one facet though. This is great work and I'll be reading it more closely when I get off the plane. I'll probably grab some of your kibot configs for myself!
Is it hardware testing at the simulation/emulation level as a "digital twin" or hardware as testing as in hardware-in-the-loop (HLL)?
This, very likely.
It's possible to set up simulations of certain sections of the hardware using some Spice software, but it's quite time consuming and I'm not sure if anyone's built a way to compare the output of the simulation to expected output.
Simulating any entire circuit as a whole is impossible, since lots of parts just cannot be simulated.
There's a semi-standardized file format for simulation models, but there are still differences between simulators. And every manufacturer's website is a different kind of poorly-organized, so it's hard to even find the models.
The other thing is things where high frequencies exist, like switching regulators. It's possible to simulate them, but there's all sorts of parasitics (imperfections from the circuit being made out of real stuff rather than lines on paper) that impact the circuit. I've only used KiCad, so I don't know if the commercial software here can model these parasitics, but they can have a big on performance.
Imagine being tasked with simulating an entire Raspberri Pi board. You would probably need dozens of engineers and likely more than a year just to write and verify the simulation code. And, of course, a supercomputer to run it.
Hardware isn't about isolated components. It's an arrangement of components on a physical circuit board with interconnecting traces and physical characteristics that interact with the finished board. And, all of this, exists in the context of environmental requirements (temperature, humidity, vibration, RF susceptibility and emissions, thermal requirements, etc.).
A "simple" DDR memory design consists of memory chips, the transmission line (traces) connecting to the controller (FPGA, processor, etc.), the PCB stackup (at speed, this matters a lot), decoupling capacitors, the power distribution system feeding them, thermal management and more.
And that's just two chips talking to each other in a fairly structured way.
and only upload gerber zips as release artifacts
I do keep a bom.csv in the repo to link right in the readme. I consider the specific part selection part of the reference not entirely documented in the sch, and so it's source.
Same for .stl and other output meshes from openscad and freecad sources.
Which is kind of a pity because github has a neat stl diff viewr that's essentially useless if you don't want to check in stls.
Still, this heavily sped up prototyping, as anybody could always fetch the data required to release or produce stuff. Shifting left works for electronics just as well as it does for software. :)
Kicad is on a good track to become as capable as Altium, if we judge over the last 3-4 years of the progress. I made it a habit to give them money annually via the Donate button on their front page: https://www.kicad.org.
IMO other tools mostly excel at special tasks. Be it having a good component library, or less rocky integration with external tools, for example for FPGA/ASIC configuration or layout, so the stuff that is heavily driven by commercial interest in increasing an engineer's speed or lowering defect rate.
(Disclaimer: Haven't been doing electronics for two years, though, as I switched to Software.)
I also want to make the switch from embedded software to higher level software but all employers scoff at my lack of modern SW experience.
Also around Germany.
My motivation: the higher level stuff seems easier with up to twice higher hourly rates, but most importantly: it seems way easier to get fully remote jobs in the higher level programming areas. I would even take a pay-cut in exchange for fully remote: it's my nr 1 motivator to switch to something else.
What's your motivation if I may ask? Just curious.
Same. Mostly the possibility of WFH which is absent if you ever need to touch lower level HW. Pay I can take it or leave it.
> the higher level stuff seems easier
It's not really easier though. I had a 3 month stint doing some backend contract work and the amount of framework fuckery and the speed of development on live web systems that customers use, can be crazy stressful compared to embedded.
It's like changing the components of a car, while driving it. Shit breaks much more often. Embedded felt much less stressful since any of my commits to production didn't instantly impact thousands of customers at once like on the web, with the risk of fucking their data or experience.
For reference, the public part of what my employer sells: https://work-microwave.com/work-microwave-products/ . There's a lot more that's not public, though.
(If you're interested, feel free to have a look at the careers page. If you plan to switch and are a good fit otherwise, I'm certain that could be accommodated. In case you choose to apply, drop a note that you have a recommendation from an employee, it should speed things up. We are definitely looking for many new engineers as we're growing quickly. ;) )
However, this is a small subset of the actual list. Chances are, there's many companies working with proprietary PCB designers that were discontinued 20 years ago or just maintained for a few companies.
And then we ourselves got acquired by a company that could afford Altium.
Last I used KiCad for much (ca 2018), I found the workflow to be kind of clunky so I never expended much effort to get really good with it. For instance, their footprint assignment scheme caused so many errors because you couldn't, at that time, assign footprints to a given part in the component editor. Thus, you could spend a lot of time getting the footprint just so, then not notice you fat fingered the footprint assignment unless you did the old trick of printing out your solder mask and hand populating your board.
But whatever KiCad's current flaws, its not Fritzing, so that helps.
Sadly, it's not like KiCad excels there either. At least to my knowledge it's not possible to replicate a routed part of your module like it's already possible to reuse schematics sheets, is it?
This stuff you still need to discuss. It's just that the basics are often enough. But, in my experience, if you want less than 0.3 mm tolerance on your PCB outline or a component needs a certain soldering cycle or method, you better tell your PCB manufacturer and you better do it on the phone, they may or may not look at the drawings. Once we only accidentally discovered that an isolator in a medical device was soldered using a temperature profile that exceeded its maximum temperature and thus the isolation was no longer guaranteed by the manufacturer.
ECAD is way more standardized than MCAD for information sharing and version control, and at least has industry standard best practices. MCAD is still a wild west, with mostly proprietary tools used for version control (several thousand dollars per user per year) with no hope of interoperability. And each vendor that offers automated quoting uses a proprietary system, with a lot of manual inputs.
Granted, mechanical parts can be much more complex and it's easy to make unmanufacturable parts in MCAD, whereas ECAD is basically a stack of 2D layers. But still, as a mechanical engineer I am amazed at how easy it is to order a PCB or even an assembled PCB compared to ordering the simplest machined part.
I give it 2 years tops :p
I haven't used any of them though. I don't think you need them unless you're doing really pro stuff (e.g. a PCIe card or motherboard) or RF design.
On the free side they mostly suck. KiCad is powerful but has AWFUL usability. Don't bother unless you're willing to fight the software and spend hours on tutorials for things that should be trivial. Think early Blender but worse.
The other free options are:
* Eagle: Terrible. Do not bother. No idea why anyone uses this. Doesn't even get copy and paste right.
* gEDA: Awful. Do not use.
* DesignSpark PCB. This is very Windows 95 and a bit odd in some ways, but it's probably the easiest to use software and it's not limited like some free software (frequently there are board size or component count limits). Very decent option. If you're new to PCB design and use Windows I think I would maybe recommend this. Either this or...
* Horizon EDA: A new option that uses the KiCad routing engine but is way more usable. I would say this is the best free option at the moment, and it's open source. Except maybe LibrePCB because...
* LibrePCB: I haven't actually gotten around to trying this yet so I have no idea. May be good, may be awful.
v6 has been very stable for me.
Early v7 builds were a crap shoot. I may try again later this year but I'm really happy with v6 for now.
However, it was not uncommon. I always had a debug build of KiCad on my machine in order to replicate crashes or odd behavior others saw.
It should be noted that we're generally talking about PCBs that could get pretty big and complicated. We certainly uncovered edge cases that were hard to foresee that way.
I've been looking at upstreaming what I can and what would even make sense. Ultimately I've switched to mostly using non-KiCad python (dealing with the messy nature of the bundled python wasted 3 solid days, don't use it unless you must) to generate and manage a parts DB + vector drawings that are then processed into the KiCad formats.
It works, but a lot of KiCad internals were simply never designed to handle this scenario. Even multiple PCBs are a pain. It uses multiple proprietary vector representations, some for rendering and some for analysis, etc. The file formats themselves are ok to deal with but aren't well-specified so they will change on you.
Fortunately, the end Gerbers are well defined and it's often easier to simply generate those directly vs going through KiCad if the DRC are guaranteed by construction to never be violated
Does anybody know something like a translator tool between all the different `yaml`s different CI/CD services use? Or maybe some metalanguage specifically made for that that can run instead of whatever hell dash/bash is.
I wish I could just jump ships altering a `script` tag in whatever `yaml` from whichever service.
The render on my page is however a bit more work. I graft textures over the parts and set material properties such as reflections. I will do a write up on it when I get a chance.
Snapeda and componentsearchengine are also getting better.
One thing that makes me wonder all the time are CI/CD runs that commit back to the repository. That feels weird for some reason I cannot articulate very well yet, but makes me think there should be a better way. Especially if the results are artefacts like that. Any experience by others to weigh in?
It allows for a build that is closer to the vanilla KiCad GUI experience and can be built locally with the normal install using ‘make’ - something I wanted.
Awesome job.
Can you make a base gitlab project so anyone can fork it easily when anyone wants to create a new project?
Kibot is really long on functionality - the maintainer is someone in South America who does a pretty incredible job of maintaining it and integrating commonly used plug-ins.
The one thing it really lacks is a comprehensive starting example of how to use it and structure a YAML file.
This would be a great example.
Do you have any write ups on the blender rendering and animation? If that could be automated I know a lot of people who would love to use it.