Raspberry Pi 4 PCIe bridge “chip”
blog.zakkemble.net
blog.zakkemble.net
If it's for the adventure, cool. If it's because you are pissed at the RPi folks for not including it then holy crap widen your horizons....
The "let's hack this into a Beowulf cluster!" stuff was secondary. Now it seems to be primary and the education stuff is, um, not even mentioned anymore?
The outcome couldn’t have been better.
There's a group of guys I build hardware/firmware for. They are experienced engineers, but their expertise is in machine tools.
We met because they made a post on an Arduino forum a few years ago basically saying "we need to do X. We know it has to be possible in a cost-effective way because of all the cheap Arduino, Raspberry Pi, etc. hardware we keep seeing for sale but we don't have any experience on those platforms. Can anyone help us?"
This illustrates a point that a lot of people don't see: while Arduino or RPi were released for a specific purpose, the absolutely massive ecosystem that grew up around them enabled a slew of entirely new classes of applications that weren't feasible before due to cost.
Back to "my" guys. If I quoted them an upfront design for what they needed, starting from scratch, it would have reached into the five figures easily. Instead I was able to say "hey, I'm working on something that has a lot in common with that. I'll build you a prototype of what you need for the cost of the hardware" and we went from there. Now that project makes me a few $k per year.
Except at the very top of the website?
*really wants to know, not trolling
There's no other board in this range because the other chipmakers aren't subsidizing their boards or ecosystems in this fashion. So there's no answer to that.
But I know what my time is worth and I'd rather spend $150-$200 more on a Beagle X-15 or Boundary Nitrogen or Toradex Colibri and get the documentation and schematics and all the source I need to make it do things like eMMC and PCIe. If I spent more than 2-3 hours on hacking chips into the board I've already lost.
Besides TFA's chip requires desoldering and resoldering an SMT chip from the RPi's tiny and densely populated motherboard, not exactly Soldering 101.
Moreover if you're really just starting with low level programming, want to mess with basic electronics and things like that then don't get into PCIe, actually don't even get into Linux. Just get a cheap-o ARM microcontroller board, a multimeter and a $10 saleae digital analyzer clone from ebay and play with that, you'll learn a lot more that way.
That's why the Raspberry Pi is so popular.
I agree with everything you said about using cheap microcontrollers to experiment with basic electronics projects.
Or if you want to build a cluster, like this guy:
The Beagles at least have mainline kernel support, but things like pine64 and odroid still rely on a lot of patches or binary blobs.
Thanks for pointing out the X-15, though.. I've not looked at Beagles since the Beaglebone Black. It looks like an impressive board with mainline kernel support.
Can you elaborate on this? Who is subsidising the Pi, and why?
Edit: Not sure why downvoted. Broadcom very much is able to write off a ton as donations to charity, since the rpi foundation is a British charity. They also get sweet heart deals from the UK based on that. The UK gets the ability to put a passable personal computer on every child's desk, like they've been attempting to do since the BBC micro days.
If I donate $1000 dollars so that I can "write it off," do I come out ahead financially?
Which I'm not counting as a negative. This is successful example of charitable giving where everyone wins.
Just because Kramer didn't know the process behind writing things off, doesn't mean that there isn't a real process and it isn't being used in this case.
This is the whole reason why the rpi foundation is a charity, to allow these financial structures.
I'm not saying it isn't a real process. I'm saying it isn't magic. Reducing your taxable income by $1,000 doesn't mean you save $1,000.
But marking your engineers' spare time as donated since your contract owns everything they produce anyway, and selling chips at cost and marking the profit as a donation when you were probably going to take a loss on them does in fact net you ahead.
The closest I’ve seen to what you describe was a BBC Microbit my son was given as part of holiday club but that was sponsored by Maxell, not the British government, Broadcom, or anyone else.
Also the Microbit was designed by BBC R&D with government grants. Maxell is just the distributor.
The Raspberry Pi isn't a giveaway item and so schools need to make an investment in hardware or obtain sponsorship to kit out their STEM resources with them.
1M BBC micro:bits were distributed free to Yr 7 pupils as part of a project in 2016 - schools had to register/apply at the time, so if your school did not participate their pupils would not have received anything, however, as you have mentioned, other sponsorship also made units available.
Best have a word with your school's STEM/Technology people to see how they are supplementing your children's curriculum with technology resources.
Also, the heterogeneous 2xA72+4xA53 CPU setup is ehh.. :/ What's really cool about the RPi4 is that there's four A72 cores. And now 8GB RAM. And https://rpi4-uefi.dev.
I have an STM32-based controller board next to me (I'm a firmware developer) and I had originally suggested that instead of us spinning up a custom board, we use a BeagleBone or RPi Compute Module and build out the I/O and take advantage of all the OSS out there. I got a lot of pushback that I wasn't willing to fight (I said it a couple years ago with the same results), but if we had gone that route, the entire project would have been done months ago. Same would have been true the first time I brought it up on that other project.
Sure the Pi's are cheap and attractive, but it's the ecosystem that provides the real value. And these days, with software usually being the main cost of the project, you can't beat OSS.
There’s a lot to be said for having a commercial product with multiple revisions to avoid painful guesswork.
So the answer is "No, there is no alternative in the same price range."
https://shop.popcorncomputer.com/ https://store.pine64.org/
That's assuming you don't want to buy a cheap phone and dismantle it (comes with battery, charging circuitry, touchscreen, audio, flash storage, etc).
And, as a plus, everything is documented. And TI will talk to you.
It's true that RPi owners don't know about a lot of that other stuff, but there are reasons they don't know about them.
What's the best right now in terms of openness and support for someone working at the driver level?
I don't work as much with TI but I've used Panda in the past for OMAP4 and some personal work with Beagle/AM335x and it's solid stuff. I'd like to think Beagle's community is as good as RPi's, just a little more low-key (and the chipmaker participates as well)
The iMX6 used to be single, dual, (solo/duallite), quad. Done. Now there's what - 5 other variants? 7?
With RaspPi you can download an SD card image fully set up with a simple "wpa_supplicant.conf" for WiFi and a Bounjour discovery service to register a hostname like 'pi.local'. Takes like an hour with a crappy connection and then another hour at most for apt-get update. Need to hook a security camera? Download a prebuilkt MotionEyeOS image. Need to control a CNC with GCode and a .Net core API? Grab the Duet 3 and a raspberry pi.
I have some good memories of using TI embedded hardware (MSP430) and the documentation being pretty good. That's a different paradigm than a whole Unix system, though, even if it's a system that's going to be used for embedded stuff. It's nice to hear the Beagle situation is good. I remember those boards getting a hearty endorsement from Andrew Tanenbaum but otherwise I don't know too much about them.
And then when I knew more I found aliexpress and the Orange Pi Zero LTS.
I run NetBSD, and the same exact software that runs on a Pi will also run on a RockPro64, just like the drivers for a zillion PCIe devices which work on other platforms will also work on a RockPro64.
GNU/Linux really should share more between distros instead of always needing distros to be able to differentiate themselves from each other.
Drivers are open source and part of the kernel. In that case, if a distro hasn't enabled support in their kernel build, you can build your own. Or compile it in with dkms if it's out of tree.
Or it's a proprietary driver, and then it might only work with certain distros, but in this case I seriously doubt the HW would work with NetBSD either.
> It's true that RPi owners don't know about a lot of that other stuff, but there are reasons they don't know about them.
You're just trying to explain away ignorance, and prejudice really.
The other nice thing is the massive community means that software gets tailored to the Pi’s capabilities - things like octoPi have been invaluable to me. Sure, I could assemble the same thing myself, but having somebody else do the heavy lifting is really nice.
The final nice thing is that rPis are easy to buy. With other boards I have to pay for shipping from the states or wait for shipping from China. With the rPi, I have one distributor that’s local to me, and a second where shipping is pretty cheap.
If I wanted to configure display, read thermals, cpufreq I had to use some rpi specific tools in /opt, intstead of using standard kernel interfaces.
Trying to add a TFT display to RPi involved googling and finding a bunch of outdated/contradictory advice and then still having to figure it out myself with experimentation.
Locally bought Rpi2 at the time was also 3x the price of the better specced Orange Pi PC. And if I wanted to use video decoding I had to pay even more.
I mean, it's great it works for you. My interest was in learning, and it felt like the things I would learn about Rpi would not help me that much with any other boards, due to quirkiness of Rpi's SoC design, and nonstandard OS interfaces. So I bailed early, and focused on something else. I also found the SoC to be rather closed.
However, there's two big issues: price and availability.
Price is obvious. If you want to spend $200 or more you have a massive number of options. However, this is a far cry from the $35-75 for a Pi, plus for both you still need a power supply, case, etc.
I can also order a Pi from a couple different sites, and probably pick one up locally in a pinch, too. It seems like there's lot of other boards where you can't actually buy one. Some are in the category of "how many thousand would you like?" It's just a totally different world than the Pi's distribution and ecosystem, to the point it's not really fair to consider for the average hobbyist.
There's dozens of readily available case options and other add-ons for the Pi.
I also know if I build software for the Pi I'll still be able to buy another one anytime I want for the next few years, and due to its popularity chances are for the next decade (if not a real Pi, someone will make a clone).
It's great there's options, but the fact a product merely exists doesn't instantly make it a viable alternative.
https://dietpi.com/ (warning: puke inducing site colors! :-)
I'd call that color scheme Liquorice-Absinthmeister :-)
Yes, I find it extremely uncomfortable to look at; literally a punch in the eye.
Which means my country’s very good consumer protections don’t apply, which immediately rules them out for me.
Seen pragmatically your "protection" in buying such small things, where the transport costs more than the things themselves is denying the seller a good rating, until one item that is functional arrives. AND/OR disputing via whatever payment service, or simply writing it off.
In countries where consumer protections are a complete joke, and a bad purchase is as good as a write-off, I could perhaps understand this sentiment.
However, I don’t have money to splash out on things that might be broken on arrival. I’ve already bought a few Raspberry Pis that needed returning and doing so was hassle free as soon as I quoted the Consumer Guarantees and Fair Trading Acts. At most places, I don’t need to bother.
(Another benefit is I never need to buy extended warranties; legal protections here tend to last longer than those anyway!)
If I can get the product in New Zealand via a local supplier, and I tend to prefer to support local businesses anyway, I can have just as much fun without any risk.
I just wrote that because of my experience in Germany, where I'm from, one tends to see the sourcing of gadgets from China like that.
I once ordered a NanoPi Neo2 with 1GB RAM, some 'Sata-Hat' for it, and a fitting metal case made of aluminum. Was about 50 US$ with the wall plug, directly from the vendor, waiting some weeks for delivery, and the possibility of hassle with customs, though unlikely for something of that price.
So i applied your reasoning and sourced it locally. BANG! 99EUR!!! Didn't make that much sense anymore, economically, but i wanted it. Then the delivery service fucked up, it got returned/canceled by the delivery service because I complained about their reason "could not find address", while having seen the driver speeding away at the end of his tour from my window, which was obvious from the online tracking with real time map which they offered.
Had to tell the vendor I did NOT cancel, please send again, this time with a more reliable delivery service, if possible.
Finally I got it! (That was the 'fun', the rest I could have done without)
Would it break now, i'd have no consumer protection at all because i bought it way more than half a year ago, which is how long ours last.
Since I'm no power shopper, this made me think about my/your reasoning, and the conclusion that if you want something from "there", either the 'no risk, no fun' thing is the way to go, or it doesn't exist, since sourcing it locally at least doubles the price, or it isn't available at all.
Ordered a few things since then, had no hassle, everything was good, and at least 3 times cheaper.
Even have Alipay connected to my bank/checking account and identity card, which was possible to set up via browser at the time.
The idea of supporting local businesses is appealing in theory, but seems to hail from bygone ages. Again, as seen from Germany, because nowadays many here are either clueless and/or scammy, which takes the 'fun' out of it.
So, coming back to the little expendable cash, the only remaining question for me is: Does it make sense to have some man in the middle delivering my cheap things for at least 50% more?
For now, nothing beats the RPi 0 on price, as well as features-per-dollar. The closest is an Onion M2, IIRC. It's $9 before shipping, but only has 64MB memory. I'm prototyping /piloting on RPi Zeros, but if I end up scaling to 1000's of sites (which is why I cant use a $200 SBC per site. It's a self-funded passion project), I'll likely have to switch to the Onions due to the limited availability of RPis you mentioned
Pine64 has a number of SBCs and also other products that build around them. https://store.pine64.org/product-category/pine-h64-boards/
Both offer things in similar price ranges as Raspberry Pi. I have some of each (if you consider the Popcorn to be a true successor to the CHiP as the seller does).
There is a great difference between a board which uses mainline kernel and is supported by major distros, and a board which can only run specific old version of OpenWRT and a patched 2.6.x kernel.
Sure, the 2gb version is cheap, but it's also borderline useless if you want put a pi4 to use as a computer instead of fancy arduino.
And while they're both "computer-board-things", we can't really compare a LattePanda to a Pi: the only reason LattePanda boards exist is because they allow the same kind of prototyping/embedding as the pi, but specifically for windows. There is very little reason to buy one unless you _need_ a board that runs regular windows (because you need to run closed source, windows-only tools, for instance).
I have been happily using my RPi2 as a home dns, file and media server for at least a year now. While I agree it is constrained by memory, IO and compute, it is adequate for my purposes. Though I am upgrading to an RPi4 4GB, it is a nice upgrade, not a requirement (for me).
I presume there are some people who wish to run x86 binaries on Linux or Windows, for which a LattePanda might be a good fit at a modest cost. I wish them success, which is more fun than failure.
I remember when EMACS was "Eight Megabytes And Constantly Swapping"; there is something deeply fucking wrong with modern software if two thousand megabytes isn't enough for it to run smoothly.
These kids are truly spoiled. Even with 1GB today could be enough if you know how to set Unbound + a light browser.
If you need a pi because you want to run something on it that'll run just as well on a pi 2, then sure 2GB is perfectly fine. But if you actually need a pi 4, you need it because it is vastly more powerful and offers actually useful amounts of memory.
> a home use license for Mathematica is $300+
Is no SandyBridge either.
The devices are wildly different, with the amount of RAM available being far less important than the fact that your GF's laptop, with a vastly more complex motherboard, actually lets the OS efficiently use a swapfile. 2GB on a laptop goes a lot farther than 2GB on an Raspberry Pi.
Also, did you forget you're on HackerNews? Sometimes people do things just because it's interesting or they're curious.
I mean, I have the strong suspicion that the vast majority of RPi owners wouldn't be able to cite the name of a single alternative. Not a one. RPi's popularity eclipsed everything else (like Arduino in its heyday).
And even in your comment you haven't mentioned any alternative yourself, only that they exist. If you care about people using alternatives you should start with that. Actually, I should too, but the only Embedded Linux boards with a PCIe interface I've used are super expensive FPGA-based ones, so not really in the same category.
Though must say TI AM3* boards (beagle’s) have good mainline Linux support and they appear to do all their work on public got repos. Now the iMX* boards seem nice but I haven’t really found a maker board variant.
For most embedded work which needs to be reliable, the RPi is to much designed to a price, with enough tradeoffs making it an unattractive and ultimately more expensive option. In a PoC we did, we had to replace RPi's weekly in the field, racking up intervention costs. This completely disappeared the moment we switched to beaglebones.
I quite like the RPi for what it is, a cheap & fun platform offering a lot of possibilities to the masses, but I wouldn't consider it to be a serious embedded platform to target you can depend on. But I've used it as a video/slideshow player in real-world situations, where if they'd drop out, there aren't any real consequences (which most of the time were cards failing, and yes, the root-fs was read-only...)
I’ve got two dead pi’s here ceased to function while in unattended operation.
But I'm pissing into the wind at this point. Got my cheap ARM board, fuck you. And that's fine. Enjoy the adventure. I'm all for hacking, but reinventing the wheel isn't my thing.
If there is a new version or my existing one breaks, I'm relatively certain I can get a replacement, and if I need to upgrade, it will be painless.
That said, there is nothing wrong with trying to shoehorn features into something you've got already, vs buying something unknown.
But that's not really the case.
Sure, for any one project an alternative _may_ be the right solution, especially if you’re spending serious time on it, but when you’re constantly creating little projects and proto-projects, many of which involve little more than copying from a few blog posts, writing a bit of code for an evening or two, and deciding not to go much further, it is just SO much less risky in terms of time and probability of major hiccups to start out on an RPi. Plus, if you’re like me there’s probably one lying around somewhere so you get instant gratification and zero incremental cost to give it a try, which is often the difference between scratching the itch and not on that particular project.
Every time I look at one of the alternatives to the RPi, I just don't have confidence in the tools and community. I don't want to have to do what I used to do when I worked on embedded hardware for a living. It was fiddly and annoying (though it did have its truimphs), and mainly only tolerable because I was getting paid to do it.
I want a stable, working, well-supported, upstreamed platform that I can build on without too much fuss, and also feel confident that I can keep upgrading the OS and kernel to whatever is the latest, for the next 5-10 years, even if the company behind it disappears.
I play with this stuff in my spare time, and fighting with a chip vendor's nearly-always-shitty BSP is not what I consider a good use of my time. You (being someone who currently works on this stuff day-in and day-out) are in a very unique position that you can leverage to make it easy and productive for you to build on random hardware platforms, especially the ones that aren't even mainstream RPi competitors; most people posting here about this sort of thing are... not in that position.
Recently doing another project and ran into the same limitations on the pi4. This time tried the NanoPi m4v2. Wow is it amazing. Native PCIe slot with a NVMe/SATA/USB3 riser hats are fantastic: https://www.friendlyarm.com/image/catalog/description/M4v2_e... - with the SATA hat you can build the equivalent of Synology DS918+ NAS for 1/10th the price.
Won’t be going back to Raspberry. Specifically this is for the use case of a few hundred SBC's deployed in the field for commercial/light industrial - but if you’re a casual Raspberry user do reevaluate the NanoPi. (edit: or the OrangePi apparently, see below!) I’ve found the Raspberry Pi software support to be an ocean wide and a frisbee deep. For anything more involved than a casual weekend project you’re going to have to bushwhack either way.
Here is the status matrix: https://linux-sunxi.org/Linux_mainlining_effort#Status_Matri...
And other things raise big red flags for me, like:
> sunxi-musb driver lacks DMA support (with current driver, USB gadgets are limited to PIO, limiting speed to 10MiB/s and causing large CPU)
That... seems like a pretty big deal for anyone who wants to do anything with USB.
It's also telling that it appears that all the upstreaming efforts seem to be community-driven, which is an even bigger red flag to me; the main page of the wiki you linked calls out Allwinner for their hostile attitude toward open source and the GPL[0]. It's fantastic that such a community has formed and works hard to fill in the gaps, but I'm not sure I want to (for example) build a NAS out of an OrangePi when I'm not confident that in 5-10 years I'll be able to run a future-current OS and kernel on it.
The RPi's support isn't perfect, and not everything has been upstreamed, but most of the critical components are, and everything else is either in progress or at least has an out-of-tree open source driver.
[0] Not that Broadcom wins any points here...
Good to hear that the M4V2 lives up to its billing.
I also have a few Orange Pis. For me the sweet spot for them is in low power ethernet connected: the Orange Pi One. Maybe as a GPS-based NTP server, or some such.
[1] https://www.wlanpi.com/ [2] https://github.com/WLAN-Pi/wlanpi/releases
I guess the question here is one I have with every non-RPi board: if the company behind NanoPi disappears tomorrow, can I continue to run a current version of a distro, forever, using a vanilla Linux kernel? If not... I'll pass, thanks.
While the RPi Foundation has their own fork of Debian and the Linux kernel, you can run stock Debian with a stock kernel on it just fine, and some of the missing drivers are being upstreamed all the time. The community has built images based on various other distros as well.
I recently went through the difficult process of taking an old (commercial) NAS that was running Debian squeeze and an ancient 2.6.31 kernel, and upgrading it to Debian buster and a 5.6 kernel. I was incredibly lucky that a hobbyist had upstreamed support for its hardware a bunch of years ago. If it hadn't been for that, I would have been stuck. I expect that sort of thing from a commercial appliance, but for a board I'm buying for its hackability, I expect to not be tied to the company that makes it for software updates.
This PCIe on the Raspberry Pi is a neat hack, but still a way of from being a real desktop.
SolidRun HoneyComb LX2K has a bigger full size slot and an M.2 slot. I don't think bifurcation works but you do have two slots, you could run a GPU and a NIC or HBA at the same time, using an M.2 to PCIe-slot adapter. And/or use extension boards with PLX switches if you need more cards?
https://store.pine64.org/product/rockpro64-2gb-single-board-...
If there are convenient alternatives to rpi, especially if they're cheaper for comparable performance, I'm all ears!
The eco system around the pi is really amazing and I feel like its more alive. Which makes sense since its a learning platform.
With the RPi, I know that enough stuff is upstreamed that even if the RPi Foundation were to dissolve, I could still run a stock Debian with a stock kernel on it and get most of the functionality working (where in my case, "most" is everything that I personally have a need for). While the RPi isn't 100% open hardware-wise (stupid Broadcom), the stuff that is open is good enough for me.
But I don't know how to get similar confidence about NanoPi, RockPi, OrangePi, CubieBoard, etc. How open is the hardware? Is support for everything I need upstreamed? What's the community like? A cursory look at their various websites didn't give me enough information to feel comfortable trying one out. I have a limited amount of time for this stuff, and I'd like to have a pretty high level of confidence before buying one.
I'll admit that my hesitance is maybe a little unfair and colored by a single bad experience: a few years ago I bought a ReSpeaker (the original v1 hardware). It was, frankly, garbage. The hardware itself was designed poorly and was buggy (attempting to run anything off the SD card would result in random no-reason SIGILL/SIGBUS/SIGSEGV, for one thing; that's a sign of broken hardware design). Support was minimal[0], and after they started working on the v2 hardware, they more or less stopped caring about v1. I actually threw the board away because it was useless. I wasted a good 20+ hours trying to get the ReSpeaker to do something useful, and I'm never getting that time back. I don't want a repeat.
In contrast, the RPi has always been a reliable platform to build on, with a great, responsive community.
[0] And the only suggestion was "only run things off the built-in flash"... which, IIRC, was double-digit-megabyte tiny.
Note that most PCB houses want Gerber files. PCBWay has an online converter for brd files. It does require creating a userid on their site, but it will give you the Gerber file without buying anything...so you're free the buy the PCB from anywhere you want.
Off topic: When I heard about Apple silicon on iMac, I was wondering whether it will retain PCIe or will Apple will 'courageously' kill PCIe on iMac (/s).
Fun fact, System76 now runs slightly modified EDK2 (as a Coreboot payload) with a custom setup UI written in Rust: https://github.com/system76/firmware-setup
Edit: apparently not, but how is the Ethernet connected on the pi4? Is it on the BCM SoC?
So maybe it's a separate chip?
In theory you could rip the PHY chip out and replace it with a fibre PHY, so you'd get a fibre Pi.
How many PCIe lanes are on this connector? Is this a full 16x slot? I assume since it has a very limited amount of PINs I'd assume it's just a 1x.
The next obvious step is hooking up this to a PLX chip. Duplicating the number of lanes available even if there's no additional bandwidth could really open up doors.
I guess it would be possible to design an ATX size motherboard with a RPi4 compute module slot (once CM4 becomes available). The PCIe bus would allow all sorts of peripherals. Don't expect any performance though.
You could build a fully blown Linux desktop peripheral wise, just don't expect desktop performance.
I did this with an external PCIe 'splitter'/switch board: https://twitter.com/q3k/status/1225560443927891971
It worked, kinda. After a kernel rebuild with the drivers I wanted and dtb bumps to allow for more PCIe MEMBAR space i was able to get multiple devices (IIRC 10GbE and SATA controller) to work simulataneously, just very slowly.
huh! huh! can you have enough BAR space for an AMD GPU to run?
Rockchip RK3399 is infamous for having too little for GPUs..