Mono on AIX and IBM i | Hacker News Reader

Mono on AIX and IBM i

mono-project.com

80 pointslewurm49 comments

Seems like IBM is really increasing its investment in open source software and hardware for PowerPC recently. I've noticed them offering bounties for PowerPC related work on Rust, there's this recent work on Mono, and OpenPOWER has led to the development of truly open servers and workstations like the Talos II.

They were also contributing to Swift on Linux and Go was ported to their platforms by them.

It does seem kind of odd though. The price/performance of PowerPC compared to Intel is still quite bad. To me it seems they’d be better off giving up on this term and focusing on other things. Maybe they could start with making Watson an actual thing instead of just a marketing term for consulting services.

Yes, price/performance for small systems is bad. But vertical scaling is relatively cheap/goes farther. If your database can't run in a cluster, you might need a very beefy machine to run it. That is what you use Power9 for.

The $1300 18-core / 72-thread Power9 seems like a good deal IMO. That's severely undercutting Intel's i9-7980XE (18-core / 36-thread), and offering grossly superior specs to boot, like wtf 90MB L3 cache, 8x DDR4 controllers, and dual-socket support.

In contrast, the i9-7980XE is MSRP of $2000, only has 4x DDR4 controllers and is single-socket only.

The 18-core Xeon Gold 6140 is $2400. Only 6x DDR4 controllers (although it seems to go up to quad-socket for what its worth).

AMD EPYC offers the single-socket 24c / 48t 7401p in and around $1300 (I'm seeing it ~$1100 and $1200). 8x DDR4, so its far more comparable to the Power9 machine. But Power9 is likely faster on a single-core basis, has a unified mesh instead of the 4x NUMA configuration of EPYC. EPYC doesn't have a unified L3 cache: tasks only effectively have 8MB of L3 (but there are lots of 8MB L3 caches scattered throughout EPYC).

So EPYC vs Power9 is a fair comparison at ~$1300, but Intel is severely overpriced in comparison. Actually, I'm not liking any of Intel's higher end options this generation, EPYC and Power9 seem superior on paper... as long as you don't need AVX-512.

I think the main problem is that there are no low-cost POWER systems for people to tinker with like there are for x86 and ARM. You are limited to expensive high-end servers, and that leads to a lack of software support.

https://www.digikey.com/product-detail/en/nxp-usa-inc/TWR-P1...

Power certainly exists, its just less popular. The above board is under $250 and seems to be good enough for tinkering usage.

Software support is certainly weaker. I think Power9 primarily exists for people who are planning to write their own application servers, or otherwise are willing to spend time recompiling OSS over to the Power9 architecture (Gentoo style or similar).

A "real" server application will want to buy a Talos II Workstation, for $3000+. If you're worried about performance, you have to develop and test on the big stuff.

If you're planning on a high-performance database optimizations (ie: lets say you wanted to improve PostgreSQL's performance), you wouldn't test on a Rasp. Pi or Intel i7. You'd buy a Thunderx2 ARM, Intel Gold, or AMD EPYC to test on. The architecture of "big boy" chips is grossly different than the smaller scale stuff.

----------------

And the way you "test" on big-boy machines, is through cloud services. You don't necessarily have to buy a full machine if you're testing (although local access is certainly useful).

Raptor does have a sub-$2000 machine on pre-order: https://www.raptorcs.com/content/TL1BC1/intro.html

That is starting to approach the price point where it's worth it if you want to tinker, but with something that is a lot closer to a real machine than a $250 embedded board.

I tried taking a look for cloud services that offer POWER machines, but couldn't find anything that I could just sign up for and try for relatively cheap. There are a couple of universities that have free POWER clusters that you can request access to, but for tinkering I'd actually rather be able to pay for what I'd need rather than having to manually request access.

Do you have references to cloud services that would allow you to easily get access to POWER9 systems by the hour? IBM's own cloud services only seem to offer bare-metal POWER8 servers by the month; as far as I can tell (from their fairly confusing pricing page and docs), all of their virtual servers or hourly servers are Intel.

I remember hearing of some Power cloud offerings, but unfortunately I can't find any these days.

I find old-press releases going to broken web-pages: https://www.ovh.com/world/news/cp1606.world_exclusive_ovhcom...

But that's not really helpful. Hmm, I guess Power8 / Power9 cloud systems seem to have disappeared. IBM really needs to work on getting cloud-instances ready for tinkering.

> Raptor does have a sub-$2000 machine on pre-order: https://www.raptorcs.com/content/TL1BC1/intro.html

With RAM and storage, it will be $3000+.

With "price/performance" parts, like the $1300 18-core CPU (something that would make the entire purchase worth it IMO), you'd be above $4k+.

Oh, yeah, looks like you also have to add the CPU, that's not included in the base price.

Although, for RAM and storage you can purchase that separately at cheaper prices.

I just tried to sign up for IBM's cloud, to see if I could use the limited free trial or $200 credit to try out one of the bare-metal systems.

Nope. The limited free trial only seems to apply to a few of their SaaS offerings; and the $200 credit also doesn't apply to their "infrastructure services" like VMs, bare metals servers, storage, etc.

Additionally, it looks like you need to get manual approval for you account to be able to use the VMs; even once I added my credit card info, they tell me my account needs to be reviewed by someone before I can actually rent server time.

On AWS or GCS, you just sign up and spin up VMs. I've used that many times for quick compat testing against different operating systems, some quick extra CPU cycles, deploying test servers, and so on.

But on IBM's cloud, you need someone to actually manually approve your account before you can do anything but use their SaaS offerings.

They really don't seem to want to attract people casually trying things out.

edit: after a bit, they did approve my account for infrastructure access. But the bare-metal POWER servers can't be rented by the hour, only by the month, and they start at $1000 per month. Still not really helpful for casual exploration. They also have such wonderful usability features as requiring you to allow popups in order to configure and order servers or VMs.

Oregon State University's Open Source Lab offers development hosting on their POWER8 OpenStack cluster.[1]

[1] http://osuosl.org/services/powerdev/

Yes, but they require you to fill out a long, detailed form about what project you're affiliated with, exactly what resources you need and for how long, an IBM advocate (?), and so on.

This is quite inconvenient for someone just wanting to do a little bit of experimentation before committing.

For instance, I was interested in experimenting with AltiVec on POWER for the Rust bug bounty. I wanted to get an idea of the scope of the problem, see how things work in C and C++, do a little bit of quick microbenchmarking in C and C++ to see how it compares to SSE.

If it turns out that it was something I was interested in pursuing further, I might try to work with the Rust project to apply for this kind of access officially for a CI system for the Power architecture. But that's a lot of trouble to go through if I just want to do some quick experimentation before deciding on whether the scope of the project is worth it.

That board is a 32 bit power board, not really much use for ppc64 development.

And the Rasp. Pi 3 uses a small 4-core in-order ARM-A53 instead of the out-of-order bigger core ARM-A57 that's behind the ThunderX2.

Point being: if you want to really develop for big things (like ThunderX2 or Power9), you buy the big thing. I'm not entirely sure if the "small embedded" stuff, like Rasp. Pi, really help.

QorIQ is moving to ARM though, so don't expect that segment to continue. Also there's a pretty big difference between embedded PPC that has lost to ARM and POWER9.

It really isn't very bad; Intel is very overpriced relatively if you're talking about e.g. $/core or $/thread at a similar TDP. The bandwidth in P9 chips is pretty high, and they have huge amounts of L3 cache compared to any price-comparable Intel purchase. They also have a significantly higher thread count (SMT4), and comparable PCIe lane count for peripherials vs up-to $8,000 xeons. The individual cores are also very fast: it's not like those ARM64 servers that tried to win by having many slow cores. The cores are fast, and there are a lot of them. Things like COTS networking blades and big-ass database servers are going to be their prime turf, I think.

I actually have an Excel spreadsheet open right now with some preliminary numbers (evaluating some random specs from a recent Anandtech article, along with Sforza CPUs, out of curiosity), and in terms of $/thread, the Sforza CP9M06, an 8-core 4SMT chip for $600, is only beaten in $/thread by a Cavium ThunderX2 9980-2200, at $18.5/core vs $14.02/core, thanks to 128 threads. But, at a price of $1,800, with a significantly lower base/turbo clock frequency (3.5GHz+ vs 2.2+) and less than 1/2th the L3 cache.

Granted, these numbers are more or less impossible to verify at this moment and unbelievably fuzzy, and it's incomplete without filling in all the data on a wider array of Xeons, EPYC machines, and real benchmarks. But the back of the envelope numbers come off as pretty solid, and my experience with POWER8 makes me think these machines will perform very well.

There is also the fact that IBM is quite open about POWER this time around, all the way to making all their firmware/boot/BMC code open, and encouraging patches, under FOSS licenses, and making all the architecture/firmware documentation freely available. Might not matter for hyperscale data centers that negotiate themselves, but certainly a nice bonus.

The biggest price-related barrier at the low end is really the cost of the mobos, though. That's where most of the TALOS II's price goes to. The CPUs are relatively cheap in comparison.

Keep in mind that almost all that work will be Linux-based. This announcement was regarding AIX & i, which is off in la-la land compared to what you expect with a Linux-based system.

Oh, yes, I'm well aware of the difference.

It's all POWER though, and I seem to be seeing more of it around recently. I had thought that it was going to just keep on shrinking into a more niche market once Macs, Xboxes, and Playstations stopped using POWER/PowerPC processors, and Intel was dominating in the cloud, but it seems like they're actually investing again in trying to get out of the relatively small niche they're in now.

Maybe it's just my perception though; I happen to have noticed POWER related things happening in a couple of different areas recently. I haven't been following it very closely.

I was never paid by IBM to do anything; this is a hobbyist endeavour.

Really? Wow. I just assumed it was paid because I wouldn't have expected a hobbyist endeavor to port software to a proprietary Unix and mainframe OS running on expensive hardware.

Do you develop for AIX and IBM i for work and have access to hardware and inspiration to do this for that reason? If not, how did you wind up picking this as a hobbyist endeavor?

I know a friend who's employer was throwing out an E4A. OS/400 is also pretty interesting from an architecture perspective too.

Yeah, it might be kind of interesting to play around with IBM i/OS/400.

Oh, and I guess I shouldn't have called it a mainframe operating system; that's z/OS. IBM i is their minicomputer operating system.

Hardware-wise, IBM i can scale up from basically a thick pizza box to something relatively monstrous in size. The architecture and OS are still relatively advanced, too, even by today's standards. Those of us who know its history sometimes think of it as "IBM: The Next Generation", since it was based at least in part on the Future Systems project, a replacement computer architecture that never quite came to complete fruition. IBM has never marketed it as a mainframe-class system, though.

I used an AS/400 as operator back in 1994, was mostly taking care of doing backups.

Sadly I never got to program into them, but at least .NET (on Windows), Android seem to share some of the ideas.

I'm more impressed that you were able to acquire the unobtanium that is AIX on POWER, let alone System i (the real iOS, not the one from the fruit company)

I thought the real IOS was from company with the bridge logo that rhymes with Crisco lard, especially since IBM iSeries is just a renamed AS/400 - OS/400 from 2006.

IBM partners to offer AIX and System i VMs for open-source people to use to port their software. Links that were posted elsewhere:

http://osuosl.org/services/powerdev

http://openpower.ic.unicamp.br/minicloud/

https://fit-rhlab.rhcloud.com/powerlinux-openpower-developme...

https://ptopenlab.com/cloudlabconsole/

So you have a link to those bounties? I haven’t seen them but would be very interested!

https://www.bountysource.com/issues/46345753-altivec-vsx-sup...

I looked into doing this briefly, and a couple blockers were that I couldn't find any easy cloud services offering POWER that I could use for testing, the cheapest dev system I could find, the Talos II Development System, is itself $5000 once you add RAM and storage (https://www.raptorcs.com/content/TL2DS1/intro.html, they do have a cheaper system available in pre-order: https://www.raptorcs.com/content/TL1BC1/intro.html), and the C intrinsics that they offer work with special overloaded functions with a vector type extension to C, making it a lot harder to use the approach of just offering wrappers for basic intrinsics and then working on higher-level support on top of that.

Looks like gnzlbg put more effort into it, but ran into similar issues: https://github.com/rust-lang/rust/issues/42743#issuecomment-...

Cool, thanks!

https://www.bountysource.com/teams/ibm/bounties

Well there’s free as in beer, free as in speech, and free as in working for IBM without getting paid.

They just laid off thousands of people from their Healthcare group. On top of thousands and thousands more in other groups recently. Couldn’t some of them have been reassigned to work on this stuff if it’s of value to IBM?

Are or were any of them experts on the things that IBM wants done? Are the involved in the communities that IBM wants work done in? If not, then they're still a net drain while they come up to speed, and engaging these communities and spreading out some money in the form of bounties makes sense.

they're still a net drain while they come up to speed

And we wonder why companies expect candidates to have years of experience in the exact stack they use and don’t offer training to anyone with other experience, just not buzzword-compliant.

All these people were smart enough to get hired in the first place remember.

Yes, but you need to look at the ROI. If it costs more to keep these people on and train them than it would to get the community to do these things then IBM is rationally externalizing these costs. They're not a charity.

That said, I agree that if they need support for these things internally, long term, and consistently, then it makes more sense to train up staff internally.

Healthcare tech is quite different from porting software to AIX/IBM i. I would be surprised if 1% of the people let go were actually willing to transfer into that sort of work.

IBM has been doing more lately with open source, but not really to a level that matches the endeavors of others. OpenPOWER may be gathering some real momentum, though, so I wouldn't be at all surprised if we start hearing more about it in the future.

As for Mono itself, this particular implementation isn't an IBM project, and when I personally discussed the matter with them some ten or so years ago they said they had discussed it internally but had no particular desire to pursue it at the time. They gave me some good reasons for this but I don't remember the details.

IBM people, if you're reading this and ported Mono to work with XCOFF, how about adding an XCOFF back-end to LLVM? Hell, I'd even take being able to bootstrap GCC 8 from source using your compiler.

Wow seems like a lot of work for something that won’t run fast (until PASE is cut out of the picture), and likely has a very small user base. Who is likely to use it?

Actual IBM users, according to the post.

Huh. I honestly thought AIX was dead by now. I remember porting something to AIX 20 years ago, it wasn't fun. I guess IBM stuff doesn't die, it just gets more expensive.

I mean... it's Unix. That's like saying "I ported something to Linux 20 years ago, shouldn't it be dead by now?" and also like saying "I ported something to OSX or Solaris, it wasn't fun" because AIX is Unix exactly like OSX or Solaris.

Cause every Unix-like OS is exactly the same?

At the end of the 90s they had spent like 15+ years diverging from each other. Some more gracefully than others.

But it's not Unix-like. It's Unix. It's actually Unix, as in UNIX™.

But at any rate, your comment implied it should be dead because it's so old. Which is a silly statement, considering just how old UNIX™/Unix systems actually are.

I think the implication was that he thought it sucked 20 years ago, so it is surprising that it has continued to suck for 20 more years

Actually it was much better experience than HP-UX, where aCC was stuck between K&R and ANSI C, during the time I happened to do heavy UNIX development.

The whole hosting environment for the project I am on went from Solaris to AIX.

a frying pan to fire situation, huh?

I wouldn't be surprised if there's more i shops using newer software than AIX shops, nowadays. IBM's own ports of things like nginx, node, and Ruby to "AIX" were all to support i.

Reply on news.ycombinator.com