Oracle built a 1,060 node Raspberry Pi cluster
servethehome.com
servethehome.com
- heard years ago from a friend. Not sure of the origination of the quote.
Is the process "Google vs. Oracle about Java" (I understood that it's about the functions/methods declarations and not about the code itself) similar to a theoretical similar situation where DB XXXX would provide the same functions as the ones provided by the Oracle DB?
It was a photo of Larry Ellison's yacht.
And suddenly the intended audience of the article came into focus.
Java SE is priced per processor. $17.50 per month at that CPU count. And so, a relative bargain per year at 17.50 x 12 x 1060 = ~$222k / year.
Great model to attack the incumbent [transport / transit] carrier costs - end run around the middle-men and drop your CDN boxes directly at the edge (last-mile) ISPs.
On top of that Digital Ocean's API was so much better than Azure. So I spent a week and switched to Digital Ocean. Needed to scale the instances less, cost less, so more profit!
Unfortunately after 2 months of development, and half a year of marketing it, we only managed to attract a handful of customers, and had to shut it down.
The technology was basically an a beefy server for NFS, a beefy MySql server, and a beefy load balancer. When you started hosting with us, you paid $50+/mo and got what felt like a standard WordPress install on a dedicated digital ocean droplet, when metrics showed that you were hitting the scale threshold, we would create a new droplet run an initialization script to mount NFS, set up the nginx server, and then attach it to the loadbalancer.
When the combined CPU or RAM droped below the initial scale threshold, it would drop one VM from the load balancer at a time keeping it provisioned for an hour in case it needed to scale again.
Worked amazingly well. Sure the load balancer was a single point of failure, but in the 10 or so months that it was running, we never had a second of outage based on pinging the websites.
What was it written in?
I used the Digital Ocean API to provision infrastructure, and SSH to remotely login and run the initialization bash script. The init script configured nginx, NFS (maybe it was SMB, i dont rmember now), a cron job to minutely heartbeat statistics back to the MVC app.
There are so many better ways to do it now, though. Off the shelf load balancers with letsencrypt and an api, hosted dns, managed & clustered databases, monitoring api. It would be a lot simpler today than it was just 2 years ago, let alone 7.
The real question here is did they use it to run some heavy applications on the cluster and how did it perform? Basically do they have a purpose for it or was it simply a "because we can" type of thing?
The real selling point for me is energy efficiency and price, but the reason I could never make it happen is because most of the applications I want to use are x86 only and I don't have the resources to be a trailblazer in that area. In addition sourcing becomes an issue with raspberry pis when you aren't a big company like Oracle. It's hard to buy more than a handful as an individual.
That's not to say this isn't cool engineering for engineering's sake. Just that if you are considering practical applications for this, maybe think twice.
I haven't checked recently, but at least that's what I remember.
Really? What software in this space (headless server cluster) is still x86 only?
I worked on two top-50 supercomputer clusters for science applications. You learn a lot by doing. You find out what happens to your electrical power, TFTP, DHCP, and syslog servers, and other infrastructure when those hundreds/thousands of nodes boot up all at once. Your Cisco switch reloads when you add a secondary IP address to all of your network interfaces because it's CAM table overflows. You learn how to automate without stuff like Ansible and Chef because they don't exist yet.
I have a file server running AMD Phenom II 955 that I'm thinking of replacing with a RPi. Is there a practical way to have RPis manage a software RAID? Performance is not a consideration. Is it even worth looking at RPi in this scenario? The advantages I can think of is small footprint, lower electricity cost.
I can't think of any good reason to do use software RAID, let alone doing so on a Raspberry Pi. I would recommend hardware RAID, ZFS, or a Synology appliance.
If performance is truly not a concern, yes it is possible to do this with a Pi but I don't think you are going to be super impressed.
Most would argue RAIDZ is better, but many of these reasons are functionality based over and about just redundancy, i.e. Copy on Write, et all.
Bite the bullet of the cost of N*2 mirrors.
I looked at NAS appliances, but I think it's still cheaper to create one from scratch (assuming existing knowhow, and "free" time :)
I made a nas with one 3tb drive and zfs. Zfs was a bit difficult to install but now it works really well and the power consumption is low.
You have to use the Linux images mades by ayufan (a third party) though. The manufacturer just list a bunch of distro that run on the cards and most of them seem to be made by some volunteer. I'm not a big fan of that.
It's hard to see the advantage over a dedicated NAS device, since the savings on the controller gets eaten up by needing to buy a whole bunch of drive enclosures and power supplies.
This is not trivial advice :-) Requiring 4 drives in RAID10 has different consequences from 2 drivers in RAID1. Requiring a price < 100$ is different from < 200$. Hacked storage setup (eg. USB drives) versus proper (eg. storage controller with multiple SATA ports) is different.
For example, if you're willing to spend more, but not so much more, than say, 80$, why limiting to the RPi? The Odroid H2, for example, gives a lot of options.
Among the other things, I use it as file server with a COW-filesystem on two (cheap) SSD disks in RAID1. The board has two SATA ports, so it's a solid hardware setup. It has two ethernet ports, so it's ready to work as router. It's also x86, so it will have software support for a very long time¹. And it also runs at an incredibly cool temperature - I bought the fan, but it rarely spins. To be fair though, the H2 cases are expensive trash.
You can certainly do pretty much everything on a RPi4, as much as you can commute 50 km a day with a bike, which is... not ideal :-)
¹=RPi have an enthusiastic community, but ARM boards are second class citizens when it comes to Linux support. While Linux is formally compatible, it's still adviced to run the specific ARM distro.
This system is going to be set up with software RAID.
I expect to have higher compute power, more RAM, and much better IO throughput than a RPi solution. Also, the laptop battery (which is still good) serves as a built-in UPS.
It all depends - do you want a fileserver or a crash course in how to build a fileserver?
I did this using an rpi2. It is non-trivial in terms of effort.
In a practical sense the rpi is limited in terms of power to the USB ports, so any build with 2+ HDDs and some sort of wireless USB keyboard and mouse dongle will use up too much power.
In other words, I had to buy a powered USB hub.
I used mdadm to do a software RAID, which more or less worked out of the box but took a while to clone the original drive. Throughput wasn't amazing for anything except occasional pulls and puts; I wasn't going to stream 4K video off of this, though I did have some luck with Subversion and music streaming.
It was clunky enough that I got a Synology box a year and a half later and never looked back.
As someone not familiar with Oracle events, I'm rather confused about Oracle Code One. I found a FAQ [1] which says:
> Oracle Code One is the ultimate developer event where we will cover the technologies and programming languages developers have come to love over the years. Expect talks on Go, Rust, Python, JavaScript, and R and SQL along with more of the latest Java technical content.
but looking at the photos, I see in the background quite a few arcade video games. Is this some sort of break area for people to hang out in between talks, or something like that?
Perhaps the arcade games are also running on Java and Oracle is simply trying to "show off" a variety of uses of Java or something.
For example, I remember hearing about Joyent's port of KVM to Illumos. They were testing the Illumos KVM against Linux KVM, and the Illumos KVM was faster. After a lot of debugging, they realized the Illumos machine was turboboosting better than the Linux machine, since it was in a colder part of the data center.
Power wise each Pi going flat out burns around 5W, so all told you're burning ~6kW, which is quite modest for a big computing setup.
https://www.arnabkumardas.com/platforms/nvidia/nvidia-jetson...
https://raspi.tv/2019/how-much-power-does-the-pi4b-use-power...
There is also the matter of how much RAM and disk can be attached to each board, and the speed of the NIC. Gigabit ethernet is a little slow these days.
Edit: It is conceivable that a raspi has the oomph to max out a GbE link but is limited to 300 Mbps due to the use of a USB 2.0 bridge in between the ethernet interface and the SoC. My mistake.
Surprised they didn't just bridge 5V to GPIO pins.
Weird flex but okay, I'll just carry on with my ARM64 Cavium kubernetes cluster that blows this toy out of the ocean.