Show HN: Cantools – an extensive set of CAN bus tools | Hacker News Reader

Show HN: Cantools – an extensive set of CAN bus tools

github.com

220 pointseerimoq80 comments

Thank you for making cantools. I'm using it daily and enjoying how easy and powerful it is.

Two other projects also deserve to be mentioned. The opendbc is attempt to make sense of CAN traffic in different cars, they reverse-engineer messages and provide descriptions to each. It's part of comma.ai open source autonomous driving effort. The other project is CANboat which does similar thing for NMEA2000 protocol used on boats.

The industry so far is very closed and rejects concept of interoperability. It's nice that somebody is taking steps in right direction.

I'm glad you find it useful.

> The industry so far is very closed and rejects concept of interoperability. It's nice that somebody is taking steps in right direction.

Sadly very true. I've worked in the industry, and while there has been a lot of standardization, there's definitely not a lot of interoperability. Car makers and tier 1 suppliers in my experience are very insistent on keeping their knowledge in-house, distrustful of open source tools and extremely unwilling to share any technical information that could lead to greater interoperability.

Throwing out there that Wireshark has CAN parsing ability. For any else of y'all that have been stuck with a legacy non standard CAN protocol running on industrial automation, writing a Lua protocol parser and a little script to convert whatever internal format you've been using to pcap-ng will be an afternoon or two very well spent.

In industrial automation there are mainly 2 standard protocols, which can obtain easily: DeviceNet and CANopen.

There's also an ton of shops using literally no standard above the 8bytes+IDbits that CAN controllers spit out and accept.

If you work with J1939 there is a can-j1939 kernel module which was mainlined in the 5.4 Linux kernel. It handles address negotiation and other time-sensitive matters which would have had to be done in user space previously.

https://github.com/linux-can/can-utils/blob/master/can-j1939...

Welp this makes my day-job a touch simpler now. Thank you for the heads up!

I would like to suggest that in your "About" suggestion you provide a short description what a "CAN bus" is (with links to https://en.wikipedia.org/wiki/CAN_bus - or whatever you feel is appropriate ).

I'm a very technical person, and I honestly don't think I'd ever heard of it. I spent a decent bit of time looking on the site trying to figure out what it was before resorting to Google.

It's not a huge deal, but seems to me like it would be helpful, and worth the effort.

If you saw a library for say Image Processing or cattle husbandry for Java/JVM would you insist that the project provide a description of “Java”? I’m going to guess if you’re being honest you’d say no. CAN really is not an obscure thing at all. Also it is ones of those things that if you don’t know what it is, you probably don’t need this. Otherwise the google burden is minimal for the curious.

Obscure or not, I think the latter part of this comment is the important part. If you don't know what it is, you are not the target audience and you won't become the target audience by suddenly knowing what a CAN bus is.

So no, I'd say it's actually not worth the effort to add an explanation. It may even give real users the wrong idea.

If you don't know what it is, you are not the target audience and you won't become the target audience by suddenly knowing what a CAN bus is.

So you've never visited a web site, read a bit about a topic you knew nothing about, and then thought, "Wow, that sounds like something I'd like to get into?"

There are a lot of people out there who are curious about the world around them and enjoy expanding what they know instead of stagnating in a knowledge silo.

Sure, but I don't think this is meant to be pedagogical tool to teach you about CAN. You're always free to look up anything you don't know, yourself.

Changing the first appearance of the string “CAN” into a Wikipedia link is hardly going to break the flow of the README.

And the exact same thing can be said about “Java” or in this instance “Python” or any other jargon. Why does no one ask for those terms highlighted? All technical documentation is not a Wikipedia article. It’s just not appropriate imo.

It's on GitHub, so you can assume people know what Java is (if not where Java is) and what Python is (if not the family of said genus).

But not what a CAN bus is. There's no harm in making the word blue.

I'd posit that the user base of java, python, etc, huge topics in their own right, vastly outnumbers the users of CAN. It should just come down to simple numbers. I'd welcome that linkability too, makes bridging out to new unfamiliar territories go a lot smoother for the uninitiated. Call it improving the discoverability of things.

I didn't "insist". I offered a suggestion. In my understanding that's one of the reasons people put up "Show HN" posts. It was intended as entirely constructive, and I still think it is a good suggestion. CAN is obscure enough that I'd never heard of it, and yet it is something that I would have an interest in.

You seem to think I was nitpicking the post (unless I'm reading you wrong), and that was not at all my intention.

> I'm a very technical person, and I honestly don't think I'd ever heard of it.

> CAN is obscure enough that I’d never heard of it.

These types of statements are why I said “insist”.

There is really is almost no way to make statements like these without sounding foolish. You’re asking us to posit some compelling reason that anything you don’t know is “obscure”!

As someone doing quite a bit of technical writing I’m sensitive to this topic because it really is a type of bikeshedding, which is a patently destructive criticism (even if usually unintentional). It might not be bikeshedding if one has some stronger evidence. My evidence is that CAN is in the marketing material for any automotive car scanner from $10 to $10000. It’s literally as ubiquitous as Ethernet in the US and Europe. As someone else mentioned this really doesn’t matter, what matters is if the term is obscure to the intended users/audience.

Tech writing is about economy, and there are thoughtful discussions to be had about what needs to be described, and what is assumed from the audience and providing an ideal user experience. Appeals to ones own experience unless you’re a member of a special audience are usually the worst.

I think the issue is that HN doesn't allow blurbs on link stories. A section like that doesn't make sense for a project that is targeted at an audience that already knows the basic jargon. OTOH, a HN post will reach a much wider audience and needs a description.

CAN is almost as old as Ethernet. While Ethernet where introduced in the 1980 and standardized in 1983, CAN where introduced in 1986 and standardized in 1992. So its interesting that a "very technical person" does not know established communication systems.

See xkcd: Ten Thousand https://www.xkcd.com/1053/ and be enlightened

I don't find that interesting or unexpected at all. If you don't work in the auto industry (or where it's used elsewhere), your only introduction may well be a single slide in University. You can be a "very technical person" and not know what Docker or TCP/IP is either. Having no clue what ethernet is would be a bit odd though.

I only know of CAN because some of the pwnage is somewhere between amusing and terrifying.

I just joined a company in the automotive industry as have several others. Everyone who has changed industries to work for us has never heard of CAN or only know very superficial things about CAN. Improving the linkage to that body of knowledge is always welcome, including at my work.

That's to be expected. Even people within the auto industry only know superficial things about it. You can use can pretty heavily and not know much more than: "There's two wires, and you need a bus terminator"

There are a ton of communications systems, just to name a few out of my head: RS232, RS485, USB, PS/2, Centronic, Firewire, I2C, SPI, JTAG, SPDIF, LIN, FlexRay, AS-i, Component, IO-Link.

CAN is also used outside of Automotive. Even Google is using it for their servers, or better for the redundant power supplies for their servers. HP is using it in their printers and copy-machines.

The thing is, there is always a technology you don't know. There is your preferred search engine and Wikipedia to help you with that knowledge, these days.

PS: These days almost everybody miss the difference between Ethernet and TCP/IP. While everybody talks about Ethernet most of the time TCP/IP is meant, not Ethernet.

I agree completely

In this case, you could argue that if you don't know what CAN is, this tool is useless to you. Not all technical documents detail this kind of basic information. I routinely see tools for some technology that don't explain what that technology is. Maybe this is a bad idea, but if you don't work with widgets, then you probably don't need wiget-tools. CAN is a really big deal. It needs no more introduction than USB or TCP/IP, though maybe it would be a good idea to describe what those do too. The fact that you've never heard of it, doesn't make it some niche thing. I constantly see stuff I've never heard of on hacker news, that someone in another industry would assume "everybody knows about".

Here is another project for working with CAN:

https://github.com/hardbyte/python-can

I used it (as well as Cantools) for importing various CAN-specific data formats.

Has anyone taken canbus and deployed it on a greenfield non-automotive system? If so, what was the thought process to justify use of this protocol vs. something like ethernet? The selling point seems to be 'save copper' but the associated overheads (tooling, training, reduced throughput[0], larger connectors, multidrop bus debug drawbacks, etc.) seem like they would far outweigh any nominal savings in all but the highest volume production scenarios.

[0] https://en.wikipedia.org/wiki/ELM327#Protocols_supported_by_...

A decent number micros support CAN natively without a PHY, (STM32), it works well over short to medium length link, and is just _vastly simpler_ than ethernet in both protocol and backend implementation. Nothing so complicated TCP/IP stack to deal with, programming for it is more akin to using a serial peripheral on your device.

I agree it is simpler from an MCU programming perspective.

Yes.

It was used as an industrial communications bus to coordinate slave devices from a master controller. Drop-length was a non-issue as the majority of our devices used an in-rail (DIN mounted) interconnect bus. The larger "drops" were often a under 10" and just jumped one DIN rail to the next DIN rail in the cabinet.

This was selected for the following reasons:

1. These devices were installed in HEAVY EM environments (think in close proximity to very large electric motors) and the electrical characteristics

2. The system topology physically supported multi-drop

3. We had a good deal of talent with CAN experience

4. Our device was already supporting a bunch of wired and wireless protocols (rs422, rs485, ethernet, btle, wifi, etc) and tbh CAN support was a gimme on the SOM we used as the device's core, so initially it made it into consideration by chance

Interesting. I wasn't aware of any EMI benefit. I wonder if this is true with shielded ethernet cables at lower data rates (eg. 10BaseT)? It seems maximum CAN drop length is far shorter than ethernet, which I guess could become a consideration in industrial deployments.

The EMI resilience is one of the big reasons why it's used for automotive applications.

Many (all?) CAN standards transmit on redundant copper, in opposing polarities, making it really easy to identify a signal vs noise. https://en.wikipedia.org/wiki/CAN_bus#/media/File:CAN-Bus-fr...

It's like a one-wire serial signal, but redundant.

yeah, differential data lines; common in serial applications where you want to protect from noise. a lot of the higher level dialects are built off something like (if not the) rs485 electrical layer.

yeah, i have seen properly shielded cat5 ethernet based implementations in similar environments. i would have to say that i've seen far more industrial / heavy-commercial applications that used cat5 because "thats whats on the truck" that have had to be re-pulled. in a lot of instances things would "work" for a bit, until the operating environment changed (plant was expanded, circuits re-wired, etc). one of my favorite instances was an electrical submetering system where all the meters would go down at the same time every night. it started happening after the system had been active for a few months. the culprit turned out to be a cell modem the system owner had added. the modem had been mounted inside the NEMA enclosure that housed a datalogger. the datalogger would query values from the submeters over a serial bus (rs485) every minute, aggregate it and upload it to "the cloud"...once a day. the cabling the electricians had used was some generic roll of cat5. it wasn't shielded (and thus the shield wasnt grounded). the modem would broadcast, and that event would "crash" the serial bus, cutting off communication with the submeters. we (not really "me", but the EE i sent to the site) removed the modem relocated it outside of the enclosure so the system could operate while the correct cable was pulled. we had built a number of systems with integrated modems (we had SKUs with that configuration), so the issue really was the wrong cable for the application being used because it was common and on hand (and cheap; legit industrial serial cable can cost you upwards of $1/foot).

It supports dead simple hard real time bus arbitration of messages for one.

As I see it there are two properties of a networked system this can possibly enhance: (1) predictability, and (2) a reduction in overall latency. On the former it is only a small part of any goal or guarantee of total predictability. On the latter we should note that it is of significantly decreasing importance as bandwidth increases, which Ethernet enhances greatly, particularly on low node systems with relatively low bandwidth requirements, which covers most industrial deployments. Therefore it's sort of a 'meh' feature outside of specific requirements.

Which is why you see it heavily used in distributed real time systems where those guarantees are valuable, even at the expense of raw throughout.

And that why you see more and more Gigabit Ethernet replacing stone-aged protocols such CAN, Profibus or Firewire, because a simple Ethernet Chip is dead cheap, >100x faster and comes with a modern memory-mapped driver, the others not. It makes hard real time latency and throughput much easier. With CAN you cannot get very far, and the typical CAN multiplexing tricks get dirty very soon.

You can get CAN to go pretty far.

And real time latency is not "dead simple". A low priority message will always lose out to a higher priority message on CAN. With ethernet a low priority task can flood the bus if the circumstances are right.

Also, which MCUs come with gigabit ethernet?

Formula 1 only so far, but the idea was to get beefier ones sooner or later.

I'm sorry, I don't quite follow. Can you reiterate?

Q was which MCU's come with Gigabit Ethernet.

I answered the Formula 1 MCU's (Motor Control Unit). Maybe the official one from Illmore already (I think), and some of the internal ones used for testing (which I worked with).

The NASA Space Shuttle also had several dSpace controllers on board (RTLinux with special IO drivers) and I guess modern rockets and planes also will switch to Ethernet also. It's 1MHz vs 1GHz. It makes a difference when you can afford a fast CPU, i.e. high speed controller and you don't want to wait for CAN messages coming in every 1000 cycles.

Yes. Motion controlling a Mammography device of one of the largest healthcare companies with CANopen in the early 2000s where one of the selling point over Ethernet was that CAN allowed message prioritisation without collisions, so for example the “emergency stop” message would have the highest priority.

Take a look at UAVCAN[0], it's a protocol for using CAN in other applications.

[0] https://new.uavcan.org/

Any (reliable) system you use is using CAN: medical devices, elevators, airplanes, robots, drohnes …

Google is using it internally for their servers. Many rack providers have implemented it for server rack monitoring, …

CAN is cheap. You get it in almost every micro-controller. Requires less overhead in micro-controllers than Ethernet. Is much more reliable than Ethernet if you leave controlled room conditions. It is mostly cheaper if you have to factor in all-over costs like cabling, switches, management.

It is everywhere where you don't see it, where you just care that the system needs to work with less management as possible.

This looks well designed and thought out. Does anyone know if it has been applied to NMEA-2000 (a marine electronics flavor of CANbus)?

I was thinking the same question. I could only find C++ implementation at https://github.com/ttlappalainen/NMEA2000 which I assume you have found as well.

Yes. And that ttlappalainen project generally inherits from the CANBoat project.

Different project, but maybe this helps.

https://github.com/canboat/canboat

NMEA2000 is the marine electronics flavored J1939. Everything but the data dictionary is J1939.

You can just switch on the J1939 protocol in WireShark.

Thanks for all your hard work! We used this constantly at $oldjob and it saved us a ton of time and effort.

I'm very interested in beginning to monitor some information on my vehicle. Is the usual workflow involve pulling a dbc file off the vehicle with a CAN bus reader and using software to look into it? Or is it more complicated? Anyone have recommendations as to the reader itself?

I don't think you can pull a DBC file off of a vehicle. I think you have to build the DBC file yourself by reverse engineering everything--you'd monitor the CAN network, and then turn the light on and off, sit in the driver set, etc. The DBC file is the car's secret sauce.

There's also at least one repo [1] of DBC files that have already been reverse-engineered by the community

[1] https://github.com/commaai/opendbc

You can get away with ELM327 based readers (the cheap bluetooth/Wi-Fi OBDII readers you see on Amazon/eBay) if you want fairly standard data like speed, and some manufacturer specific data in the form of PIDs (https://en.wikipedia.org/wiki/OBD-II_PIDs)

If you want more involved data, or to send messages, it's mostly sniffing, doing an action, and seeing what changes.

`can-utils` has a nice UI for this, which lets you filter messages that don't change easily.

(Also, I've actually never seen a `.dbc` file before, but it looks like a map of PIDs, usually you [or the community for your car] have to reverse engineer those with the sniffing method, afaik)

The ELM327 can technically stream all the CAN messages on the bus and do that type of sniffing, but it has a very small buffer and gets completely hosed with chatty cars.

I personally use the MCP2515 with a Raspberry PI for more involved car hacking.

https://www.raspberrypi.org/forums/viewtopic.php?t=141052

The electrical modification mentioned is needed for the most common MCP2515 modules because the PI isn't 5v tolerant, but it's an easy enough fix

I have two of them hooked up to a Raspberry PI (one for each of my car's CAN busses, some cars have more), and that with `can-utils` has gotten me pretty far.

People should be aware that many of the cheapo "ELM327" readers on Amazon/eBay are counterfeit chips that merely identify themselves as ELM327. They do not perform as well as the real chip, but may be good enough for reading your check-engine code. For example, I had a fake one that choked if I tried querying engine RPM in a loop, while the real deal had no issues.

I've had success with clone ELM327 for reading stuff like RPM in real time, but it is hit or miss.

They're just generally slower and usually missing features of the newest ELM327 revisions.

The problem is if you just search for ELM327, most of them will be clones, so I just use the MCP2515 when I need lots of data, and use the ELM327 for basic stuff

If it's in the budget, I would highly recommend the Macchina M2[1], especially if you need to access more than one bus at a time. It's based on the Arduino Due, with dual CAN transceivers (and LIN). It has a lot of bells and whistles - but can be simplified down to just a can sniffer if needed. I'm currently using one with SavvyCan to reverse engineer a vehicle for a project, and it has no problem keeping up with the high-speed network.

[1]https://www.macchina.cc/m2-introduction

The car doesn't define its own interface, so you're stuck using the Internet.

Overall the easiest place to start is a forum for your specific vehicle. You may get lucky with predefined services or someone else having done the reverse engineering for you.

If you don't get lucky, it depends on the manufacturer. In the case of European manufacturers you are best off if you can acquire the ASAM ODX material for your vehicle (sadly there is not an easy "clear market" way to do this - reverse engineering the manufacturer's diagnostic tool is often easiest). This will define in a very complete manner the available services on your vehicle. Some aspects will still be proprietary, usually specified as DLL files referenced in the ODX (for example, the Seed/Key authentication algorithm for flashing VW vehicles). These you will have to find references to on forums or reverse engineer yourself.

For those interested in CAN bus for UAVs, there is https://github.com/uavcan as well.

Ugh, where was this when I needed it in 2013? At $OLDJOB my boss and I wrote pretty much this entire thing, in an unholy mix of bash and Tcl, much more poorly.

Any chance this unholy mix is open sourced?

Sorry, no. Very proprietary medical device. (And, really, you wouldn't want it. Really. shudders)

I cant tell you how many times I've seen this type of thing done at different companies. Many of those projects died.

The last one was a LIN tool written in python. My coworker wrote it. He wanted to open source it, so I talked our boss, the local IP guy and eventually the legal department. Legal thought I wanted them to write a license. No, that's not useful, I needed to know which of GPL, MIT, BSD the company would approve. They came with some of the misguided concerns companies tend to have. The other question I could not get answered is who in management needed to OK it. Oh and would the copyright belong to the company or the guy who wrote it. None of my questions could seem to find definitive answers. The guy who wrote the tool quit and it became abandon ware inside the company. I had mentioned that nobody stays around forever and the author would have done some maintenance after he's gone. I said this well before he left as an argument in favor of open sourcing it.

Anyway, I hope a lot of people bring this to life so we can all stop reinventing this wheel or paying certain companies to rent theirs.

Related: Busmaster, but its Windows only.

> Legal thought I wanted them to write a license.

I'm not surprised. I once had a conversation with some business lawyers that were specialized on what is easiest described as "IT law" [1] (mostly intellectual property, licensing, contract law, data privacy) who were working in the field of IT/software. You could talk with them about client-server and p2p systems or even basic encryption and data security, so they definitely knew a bit about technology.

They knew about open source software but I was surprised that they had never heard about source licenses nor about the creative common license. I tried to explain the basics to them but they didn't seem to get the concept of a license that could be reused by more than one licenser and were confused why a license would need a name (like 'MIT license'). In their view, licenses were always individual drafted documents with terms that needed to be negotiated with business partners or were imposed on customers. They didn't believe me that you don't create your own custom license for open source projects and thought that I was talking about copy&pasting bad license templates from the internet. When I told them the basic concept of the GPL they thought I was bullshitting them.

[1]: https://en.wikipedia.org/wiki/IT_law

Whats that saying about a person not being able to learn something that threatens their livlihood?

We have been using proprietary CANoe for well over 6 years. Will evaluate this. But don't know if corporate will like it or not.

Different use cases in my opinion.

As it is described this would more likely meet the use-cases of CANalizer (sniffing, logging, lightweight scripting) but not CANoe (bus simulation, node simulation full on HIL Simulation).

This seems to work with a candump format input, does it works with canalyzer *.asc format ?

Yeah, using -L you can shell redirect to a workable log file. I've done this a few times. They also build the asc2bin conversion utility to convert any logs to the more common *.bin format.

Beautiful - I’ve been using Intrepid Vehicle Spy for awhile but the cost is hard to justify for a hobbyist.

What’s missing is a can bus sim

I've recently become interested in a can bus sim, actually. If you know of any resources I would appreciate any pointers.

There are some basic simulators out there that use single board computers. I've been meaning to try to build one at some point.

https://github.com/carloop/simulator (RasPi)

https://github.com/limiter121/esp32-obd2-emulator (ESP32)

Thanks. I'll check them out.

Perhaps CANdevStudio would be what you are looking for?

https://github.com/GENIVI/CANdevStudio

Reply on news.ycombinator.com