Here's a fun bug we recently had: we had to ban substractions in our program (replacing them with an __asm__ macro) because of a bug in linux kernel 5.7.0 to 5.10.10, which had the (indirect) impact of not properly tracking the valid min/max values in the verifier[0]. The worst part is, it didn't cause the verifier to reject our program outright - instead, it used that information to optimize out some branches it thought were never reachable, making for some really wonky to debug situation where the program was running an impossible control-flow[1], resulting in it returning garbage to user-space.
All this to say, CORE is really only half the problem. Supporting every kernel in existance is still a huge effort. Still worth it compared to the alternative of writing a linux kernel driver though!
[0]: https://github.com/torvalds/linux/commit/bc895e8b2a64e502fbb...
[1]: https://github.com/torvalds/linux/blob/bc895e8b2a64e502fbba7...
Yes, each kernel version might have different features between then and now. You have to pick a minimum supported version and write against that.
I really wish customers would update to a newer distro, but I also understand why they don't. So it's up to me to adapt.
> You have to pick a minimum supported version and write against that.
What we end up doing is progressively enabling features based on what's available in the kernel. Every eBPF we write is compiled multiple times with a couple of different flags to enable/disable certain features. It works decently well, and allows using the most capable datastructure/helpers based on the kernel version.