Hacker News Reader

Comment by centimeter | Hacker News Reader

Absolutely. As an embedded engineer, 90% of ARM boards (including and especially the popular Raspberry Pi) are complete dogshit. Barely (or not) functional, absurd design decisions, etc.

A hilarious example - to deal with the poor reliability of the RPi, you might think to use its watchdog timer to reset the device if it locks up. But guess what? The WDT in the crappy Broadcom chip on the RPi doesn't work! It's supposed to be there, and it's exposed to the kernel, but it doesn't actually do anything.

View on HN

Come on, you need to at least read the datasheet and errata of the Broadcom SoC. Oh... There isn't one... Unless you're purchasing millions of chips from them and signing an NDA.

Seriously, why the hacker community even touches the RPi is beyond me. It is a ridiculously closed product.

Yeah RPi is a great toy but you should treat it like a chemistry set. Use it as a tool for learning but don’t build a product with it. The difference between a $24 board and a $200 board is often the level of support you get. The boards I’ve used in the past all have manuals down to the chip and I won’t touch anything that doesn’t.

And with a 8x smaller price it's perfect for throw-away units. I've heard of people who use it for projects were you place things out in the environment where you really have no control over vandalism, animals,etc so just replacing a cheap unit here or there isn't a big deal.

On the other hand, a surprising number of companies decide they don't need that kind of dcoumentation and just ship Pis, and that often works fine too. Not for things with deep hardware integration, and not for mass-scale, but plenty things just want "linux in a box with some simple peripherals".

As the person tasked with debugging one of those "linux in a box with some simple peripherals", I'm not even 100% convinced they sell them with more than the intent of the consumer turning it on once, playing with for a few minutes, then forgetting it exists.

Oh, not just talking about consumer devices: if anything, I would expect more non-consumer cases (but that could be a wrong estimation)

Because it boots linux and starts at $5

Even then i'm not sure Broadcom would give you that. They're very aware that they're your only source for $chip and will treat you accordingly.

(A few employers ago, they were the only source for ethernet PHYs for a carrier-grade network product we were building where we supplied our own MAC)

You could buy 30 of them, at least one would be more reliable than the board discussed in the article.

But which one? :)

For 30x price you could buy an overpowered x86 and run QEMU.

Are you aware of the datasheets published by rpi? https://www.raspberrypi.org/documentation/hardware/raspberry...

Those are very far from complete datasheets, covering only a small part of the SoC superficially.

Yep, the 200 pages you get from there is a bad joke. Even a relatively simple ARM microcontroller (the STM32F1 series) has over a thousand pages of documentation in the reference manual. A full set of reference documentation for a SoC like the BCM2835 would probably come out to 5-10k+ pages.

It's the "maker" community, who are more than satisfied to glue together half-working things with no real knowledge --- or intent to learn --- about how they work.

In fact I'd bet most of them would be scared off by the several thousand pages of documentation an SoC like that usually would have.

> It's the "maker" community, who are more than satisfied to glue together half-working things with no real knowledge --- or intent to learn --- about how they work.

Is that any different from all that stack overflow copypasta you find all over source code, where the paster clearly had no idea how that code worked? And I’m talking about people who are being paid to ship code.

shudder. It’s a wonder anything functions at all these days.

Can you blame them, though?

Not to pick on anyone, but I recently installed a stock storybook setup. Out of curiosity I did a count of all the node_modules (top level + nested). There were >2000 packages installed.

Now I know that if I follow the rules of a textbook professional (one that takes security seriously), then I'm obliged to audit all 2000 packages. But between you and the rest of the gazing internet, I don't do this. That's my dirty little secret. I don't give a shit. Gluing orphaned half-assed npm packages together and pretending everything is okay is my job.

The ironic part is how many hours we are murdering worrying about TypeScript or unit tests or integration tests and trying very very very hard not to notice that landfill we have placed under our massive rug.

There are too many technologies and not enough time to master them. Every time I dive into a popular node package I become more and more worried for the human race. We get things fundamentally wrong and it lives like that. For decades. And people come along and just obliviously use it like that and don't question it. Take something simple, like JWT. Or nonces. Or what people believe and implement around random numbers and how random numbers work (or, often, don't work). Bad information becomes bad code that sits around for a lifetime.

A fairly honest assessment of node here. People that use it aren't usually so candid about it though. Kudos!

Makers would be way better served by basically anything except the Raspberry Pi. Unfortunately RPF/Broadcom marketing is quite pervasive.

In what way?

Seriously, it's easy to crap on this stuff when you're willing to recommend boards that cost "just 20 dollars more", or have one-hundrenth the community because most of the end users are embedded engineers or the anything-but-a-pi sub crowd.

To me it just shows a lack of ability to consider requirements in evaluation. Even something as simple as the fact the rPI form factor has been fairly consistent for a while is a huge boon to people just trying to start with this stuff.

I could give my mom a rPI and a $20 starter kit and she could probably get to the point of a working PC.

I wouldn't dream of that with anything else. The fact it's Broadcom based is a little unfortunate but that ship has sailed and the ecosystem around it now trounces any benefit of alternatives for the actual people using them. Not embedded engineers who say it's crap because they wouldn't make a custom product based on that SoC.

This is like when people point out that for half the price of an Arduino Uno you can get an STM32 board that will run circles around a dinky ATmega328.

It's not about the power or "objective goodness", even the header layout of the Uno lets beginners use an insane wealth of peripherals designed for it

> This is like when people point out that for half the price of an Arduino Uno you can get an STM32 board that will run circles around a dinky ATmega328.

There is an Arduino port to STM32 (Blue Pill in particular), for those whose idea of embedded programming involves a bunch of delay() statements scattered around the library code, making sure no other peripherals can be used while you’re polling for the next UART char..

I'm well aware of how many places the Arduino core is ported to, it doesn't change the fact an actual Uno has the defining form factor and MCU.

There's a million and one peripherals designed for that specific combination that don't work seamlessly with other boards (even within the "official" Arduino family)

I mean your complaint about the quality of the library code, it's perfectly fine that the code sucks if it lets people do 1 million and one things they couldn't otherwise.

Do you really think someone programming an Arduino for controlling their cosplay LEDs cares if the library they used is poorly written? As long as it works it works.

They were never going to read the datasheet for an MCU and start figuring out bit masks for pin initialization, so it's literally a case of something is better than nothing, and there's no reason to gatekeep the environment that lets them use embedded systems

I'll bite.

I started on ARMv5 back in the days of Sheevaplug/Dreamplug. At least that had JTAG, even if the cable did cost $70.

I have tried:

* Espressobin: While Marvell's hardware is pretty good, they've had trouble with the v7 board, resulting in the small community being somewhat bifurcated. I still don't think it runs a mainline kernel properly. At least the PCIe slot is standards-compliant. u-boot+ATF will always be a 2018 (?) Marvell fork, no one's ever going to update that.

* Firefly RK3399. Lol. https://lkml.org/lkml/2020/4/6/320. Although, if you can get ARM Trusted Firmware working it's a reasonably competent board.

* Vocore2. Support is reasonably helpful but insists that I use GCC 3.4.2 to compile the bootloader. Remember to back up the wifi calibration data - you can't ever get it back if you erase it. You will need to break out the magnet wire if you want to unbrick it. Shame, it's a nice little MIPS board.

* Anything Libreboard - use this, they've put a stunning amount of work in to get things upstreamed.

When I want to just work on something not on a PC, I keep coming back to the Pi. The Pi 4 is competent with an SSD attached over USB 3, and the Pi 0W is great for very simple tasks that need Linux. I wish it were better, I wish Broadcom was more open, but the Pi has the biggest community I've seen in any "hobbyist" ARM space.

This sounds disparaging of the "makers" but if your options for a hobby project are:

1) glue shit together

2) read several thousand pages of documentation

who is going to choose the latter?

This is a market that's being poorly served (maybe it's not big enough to justify investment in developing non-shitty products for them?) but blaming the users seems distasteful.

SoC manuals are not novels, they are not meant to be read front-to-back. You read as much of the documentation as necessary.

It’s still a crap show. I’m working on a beaglebone, i just decided to stay in the PRU for now, because connecting a timer to a DMA to spit bit patterns to a GPIO port without using a processor would involve connecting sections of the manual that are thousands of pages apart.

While the rpi does not have as open a foundation as we would want in an ideal world, it seems like the alternatives are not much better. Do you have any alternative boards that you would recommend to the hacker community?

Not sure why you’re downvoted. The Pi boards are terrible and the majority of the market objective appears to be trying to undercut them.

Commercial embedded stuff is better but not without warts. A former colleague of mine sent me a rant a couple of weeks about one which I won’t name, but the board vendor couldn’t even get a Linux kernel to boot reliably on it when it was shipping to customers.

> The Pi boards are terrible

Engineers and power users tend to forget that the Raspberry Pi wasn't designed for them.

The target market for the Raspberry Pi doesn't care about reading the SoC datasheet or connecting the latest NVMe drive that costs several times more than the SBC. They just want a cheap, simple board that lets them start learning and tinkering with ample resources available on the internet. The $35 Raspberry Pi excels in this regard.

> one which I won’t name, but the board vendor couldn’t even get a Linux kernel to boot reliably on it when it was shipping to customers

Please do! It would be very good to know that your vendor is doing that.

The only reason an engineer wouldn't want to tell us who the vendor is, is if the vendor is their employer. ;)

Or any other form of NDA, which are extremely common in this space.

No not at all. NDA.

Whistle blowing anonymously for a technical person isn't complicated.

Lots of people would regard it as unethical however.

Whistle blowing? Isn't it ethical by definition? What could be an example of unethical whistle blowing?

Most instances of whistle blowing are situations where a person has promised not to reveal something and then decided to. Situations like that are necessarily an ethical trade off - the person doing the whistle blowing has decided that keeping their word is less important ethically than revealing the secret but this is a very subjective decision that different people (and ethical systems) can and do have different opinions on.

There are reliable choices like boards using SoMs (from NXP or NVIDIA), or even the Raspberry Pi Compute Module.

For example the Honeycomb LX2K at https://www.solid-run.com/nxp-lx2160a-family/honeycomb-works... or the NVIDIA Jetsons at https://developer.nvidia.com/embedded-computing.

Those options have the advantage of using SoMs so that you can customise the boards for a higher-volume run too.

The regular RPi on that point isn't exactly ideal, but depending on the use might be quite _okay_.

Here in Germany, Toradex ARM SoMs are quite popular for industrial applications. The hardware is reliable and the software for it (Windows whatever CE is called now and Linux BSPs for Yocto) is pretty good on release and well maintained for many years afterwards. The modules are not exactly cheap for the performance but also not super expensive, generally in the 50-150€ range.

Variscite has a great line-up of NXP based SoMs, at $25 and up. Quality BSPs and good support as well.

I've worked on a product that was based on Orange pi. The documentation was non existent and reliability was abysmal. The product was used in hospitals non the less. I still have nightmares about it. Granted, the functionality was non essential (it wasn't anything remotely related to life support. But still, it was used in a freaking hospital). I tried convincing the management to at least switch to RPi, but RPi was three times more expensive and according to my tests was not reliable enough to convince them. Cheap ARM boards are at best interesting toys and you should not even try to make any products with them. Ignoring this will be your downfall. The company was bleeding money left and right for fixing the already sold products. That's were I gtfo

Really? Although I agree the OragePi far from great, I have one currently with an uptime of 116 days running various services and I have no issue for the pas 4 years outlasting the power supply. I'm wondering if most of the issues with Pi boards comes from bad SD cards and power supplies. I have also an Odroid C2 and a Raspberry Pi 1st gen running 24/7 with no issue.

I'm glad you've had better experience.

It was around 3 years ago. I haven't touched any ARM board ever since. We specifically used OrangePi zero. The armbian at the time had thermal management problems to the point the boards would straight up fry to a brick. I remember there was an update that fixed the thermal problem but it would corrupt the SD card once in about five reboots. And it really didn't solve the thermal problems. They'd get hot enough to be unresponsive. So the clever decision by management was to add a fully self designed avr based board to periodically poll the device on one of the GPIO pins and somehow reboot the OrangePi if it didn't respond. So reboots were common and SD cards only lasted 1 month tops.

Do you have any more info about the watchdog timer not working (anymore)? I used to rely on it (before it became bcm2835-wdt) and it did work.

Nothing specific - I noticed that it was broken, and then I did a google search and I found a bunch of people running into the same problem, with no one conclusively getting it working.

Right, I see. I know the old supporting Debian systemd service was broken in various ways for a while. In my experience the hardware works fine in this instance, at least on the older chip (and I doubt very much they changed it).

Do you have any recommendations about good stuff?

Seconded. I have a rpi4 for a small home-automation project that I haven't really started.

If anyone knows of an alternative that's in the $100 (US) price range, runs linux, and will make my life easier, I'm grateful for suggestions.

Oh, and it needs to play nicely with a USB-to-ZWave dongle and one of the major open-source home-automation packages that can do lighting controls.

EDIT: I forgot to mention that I want it to reside in a closet that has very little ventilation. So I'd be leery of deploying a cheap used x86 system, even if the price way right.

EDIT2: Holy crap I'm being high-maintenance here. I should just do a damn Google search like everyone else.

Beaglebone Black ($43) or even Beaglebone AI if you're willing to go to $127.

The power system doesn't suck like the RPi series. The chips are all well-documented with the possible exception of graphics. It has eMMC so you don't have the microSD idiocies if you don't want them.

And if you need genuine real time, it has two RISC cores (4 on the AI) that operate in actual real time as opposed to Linux kinda/sorta/when-it-works/maybe real time.

"But the RPi is cheeeeeeaaaper!" <grumble>

> The power system doesn't suck like the RPi series

Bullshit. The beaglebone can be permanently damaged (not just filesystem corruption!) BY BEING UNPLUGGED: https://elinux.org/Beagleboard:BeagleBoneBlack#Improper_Powe...

Never had crap like that happen to a raspberry pi.

> Never had crap like that happen to a raspberry pi.

Uh, yeah. A failure mode so rare that they can't isolate it.

RPi--the system that overdraws every USB interface on the planet and requires special power supplies. RPi--the system that couldn't even get USB-C right. RPi--the system that shuts down when you fire off a flashbulb (not their fault--but still a power failure). RPi--the compact system that needs a heatsink or it shuts down.

RPi has lots of things going for it. The power system is NOT one of them.

(If you want to pick on Beaglebone power systems, pick on the PocketBeagle. It has some truly stupid failure modes.)

for similar money your could have the PCEngines APU2? why would you pick one over the other (GPIOs might be one reason?)

GPIOs and functional units. Also total power.

The Beaglebone series really does lots of I/Os well.

I use the Beaglebone Green and Beaglebone Green Wireless due lots of projects at work. They're nice boards.

My main frustration with them is that there's a ton of outdated documentation around how some of the software stack works. It's changed quite a bit over the last 5 years and you can easily go down pointless rabbit holes.

There is so much x86 stuff alone that will easily run linux and outperform a Pi4 for that budget, used and new.

Pi 4 as home server value proposition gets bad fast for me at least once cases, power supplies, storage etc factored in.

I would love to hear suggestions for x86 stuff in that budget. Bonus points for PoE.

APU2 is slightly above that price range, but incredibly good in basically every respect.

APU2 is 3x the cost, which isn't really "slightly" in the space that the raspberry pis play in.

second vote for APU2. or APU4.

I've got more than a thousand of the previous unit in production. just ramping up on APU units. yes they are higher power than a Pi, but still very low. easy to develop for (amd64). tons of Ethernet ports. easy to use reliable storage. USB3, SATA, mpcie ports, SIM slots. openly available schematics

Not a lot not to like!

Check eBay for corporate standard desktops which are dumped on there regularly. Dell OptiPlex is a good starting point, scale the price with age and you'll still wind up with a lot of compute for not a lot of money.

Dont listen to the hype. Get your home automation setup with what you have today and get something working. You don’t need a Jetson or and Intel NUC to get started. My uptime was 100% with my home automation until a hurricane came.

Worry about a better product for your next build.

There's also the Pine64 stuff nobody seems to be mentioning.

Also, like the sibling said, just do it. If every option is ARM, there shouldn't really be much of a difference if you swap the underlying system.

Cheap used x86 is often the best answer.

Couple years ago I bought a pile of barebones ex-Datto Alto, NUC-style, AMD GX-415GA systems on eBay for sub-$5/ea and I reach for one of those for pretty much anything that doesn't Actually Need an RPi. RAM runs $5-$10 for 4GB, $20 for a 2.5" SATA SSD. They boot much faster than a Pi, idle around 7w, and run fine in my "network closet" that gets over 100F in summer.

Those exact systems aren't plentiful on eBay any more but there are tons of NUC-alikes, actual NUCs, and Thin Clients in the $40-$75 range with low-TDP CPUs of comparable performance.

Honestly, the Raspberry Pi 4 is just fine for what it's meant to do. It's a lot of CPU for the money and it's easy to find resources on the internet.

You could consider the more expensive Jetson Nano if you need a powerful GPU or if you just want to try something different. People tend to assume it's faster, but the Raspberry Pi 4 CPU actually beats it in most cases ( https://syonyk.blogspot.com/2019/11/battle-of-boards-jetson-... )

We're rebooting rpis every 3 hours... in production...

Client bought too many to change.

Why?

I know of a museum with a huge Pi deployment and they don't have issues like that at all.

Museums tend to be climate controlled; factories... not so much.

It's not like they overheat and lock up in 4 hours, it's more like randomly in a day or two. But since they're just monitoring stuff every couple min, it probably felt safer to reboot all the time.

Not sure exactly about the decision process that went there, I just did part of the software running on them. But they definitely weren't stable long term.

There was a parking kiosk machine near my house that was using a raspberry pi. I only know it was a raspberry pi because it's often stuck in the boot screen with the raspberry pi logo at the top. My pihole runs for months without issue, but this kiosk was down really often. I wonder why, maybe the sd card is not reliable enough, or maybe it's the climate fluctuation like you mentioned because the kiosk is outdoor?

What I've noticed is they can run off 500 mA usb if you run them headless, but need the full 2A from their power adapter if you run X and usb peripherals.

So your pihole doesn't heat as much as the pi in that kiosk even given the same room temperature. And then, the temps inside an outdoor box in the sun can go way up too.

When I worked at Garmin, a guiding principle was “reset, and reset often”. Embedded products that you have to push out the door quickly have too much internal state to track.

And Garmin has some volume; I do custom industrial stuff that amortizes the software cost over a couple hundred devices at best.

Reply on news.ycombinator.com