OpenBSD: Use the space freed up by sparc and zaurus to import LLVM | Hacker News Reader

OpenBSD: Use the space freed up by sparc and zaurus to import LLVM

marc.info

80 pointsprotomyth52 comments

So, of the list of current platforms:

  alpha     Digital Alpha-based systems
  amd64     AMD64-based systems
  armv7     ARM-based devices, such as BeagleBone, BeagleBoard, PandaBoard ES, Cubox-i, SABRE Lite, Nitrogen6x and Wandboard
  hppa      Hewlett-Packard Precision Architecture (PA-RISC) systems
  i386      Standard PC and clones based on the Intel i386 architecture and compatible processors
  landisk   IO-DATA Landisk systems (such as USL-5P) based on the SH4 cpu
  loongson  Loongson 2E- and 2F-based systems, such as the Lemote Fuloong and Yeeloong, Gdium Liberty, etc.
  luna88k   Omron LUNA-88K and LUNA-88K2 workstations
  macppc    Apple New World PowerPC-based machines, from the iMac onwards
  octeon    Cavium Octeon-based MIPS64 systems
  sgi       SGI MIPS-based workstations
  socppc    Freescale PowerPC SoC-based machines
  sparc64   Sun UltraSPARC and Fujitsu SPARC64 systems

There still seem to be a few that aren't supported by LLVM

OpenBSD has long included multiple system compilers as required for supported platforms, at one point gcc 4.2.1, gcc 3.3.6 and gcc 2.95.3 (!) were in the tree. OpenBSD 5.4 was the last version to include gcc2 for a.out/static lib platforms like vax. After 5.5 most platforms were ported to ELF and gcc3 or gcc4 (..if not memory constrained) by Miod Vallat, even when it meant writing compiler backends. At that point gcc2 was finally removed (..along with a lot of old a.out/static architecture cruft that had accumulated).

Now that LLVM/Clang is in, the potentially viable platforms (i386/amd64/armv7?) can work toward switching, while gcc4 remains for the ones that can't yet. I believe m88k is only gcc3 platforms left, but there's a very passionate guy in Japan interested in keeping his OMRON LUNA-88K and LUNA-88K2 workstations alive.

This may all seem strange, but OpenBSD has always strongly been against cross-compiling beyond perhaps initial platform bring up. This means being able to compile a release and packages on the physical machine must work.

mips, ppc and sparc64 are in theory fine on llvm already, in practice there are issues but nothing that can't be worked through, especially given past work OpenBSD has shown on gcc. All 3 are commercially viable for at least a decade.

m88k and sh4 are the odd ducks, and if there were a backend wow that would be painful to build llvm on natively :)

What is the value in continuing to support Alpha, SGI, and PA-RISC?

Were they cool in their day? Sure. But the consumer router you use to hook up an O2 or an Octane to the internet has a more powerful MIPS CPU.

There have been many advantages to running on different platforms, which have various quirks.. different endians, page sizes, strict alignment requirements, stack-grows-up machines (hppa).

It also helps to have systems that are slow or have low memory, occasionally they hit bugs that are unnoticed on modern hardware.

Alpha may have been interesting in that it had some peripheral similarities to PC machines, but were actual 64-bit computers in the 90s. For kernel driver hackers, popping in a PCI network card that was probably never tested on anything besides a x86 PC, getting it to pass packets, and then making it your home firewall, was a thing of beauty.

you interested in one? I got openbsd and openvms disks for you and the machine itself. You can have that beauty again. You can put it next to the HPPA you'll be getting as the door prize.

That's tempting, but I'm probably not nearby. If you're really offering hardware, SCSI disks etc. OpenBSD developers surely might jump at the chance.

http://www.openbsd.org/want.html

There have been many advantages to running on different platforms, which have various quirks.. different endians, page sizes, strict alignment requirements, stack-grows-up machines (hppa).

I think that trying 'oddball' platforms indeed help shaking out bugs. But if that's the goal (rather than supporting the five users that still use such hardware), wouldn't it be more attractive to build pseudo-platforms in an emulator that simulate other architecture choices, and run it on fast and easy-to-get x86_64 hardware?

You don't find races and bugs on emulators the same way as on real hardware. Also, if you do find a bug in your emulated environment, how much time would you spend wondering if the emulator is bad or your code? Also, there is a real chance that the emulator would let stuff pass (like allowing code make memory or bus access on odd address or some other restraint) which really shouldn't be allowed and you would not notice it.

I think of machines like these (disparate architectures (endianess, buses, processors)) as sort canaries, or watchmen for Good Code. NetBSD (my preferred OS) is the same in supporting multiple architectures with one codebase, and I like to think it strengthens the operating system for all platforms.

> But the consumer router you use to hook up an O2 or an Octane to the internet has a more powerful MIPS CPU.

Which consumer routers have 64-bit MIPS CPUs? I thought they'd mostly settled on mediocre 32-bit CPUs.

The Ubiquiti "EdgeRouter" series features Cavium Octeons. They're moderately fast MIPS64 SoCs with hardware offload for a lot of common network operations.

OpenBSD does have a port for the ER series, but it's separate from the SGI MIPS ports:

https://www.openbsd.org/octeon.html

How good is that edgerouter hardware? The price is pretty attractive.

(I just realized that's a pretty subjective question but I actually am interested in the opinion of someone who has interest in the odd corners of openbsd)

I have one for my gigabit connection at home and quite like it. It's able to hit about 933mbps to WAN->LAN which is important.

Note: I don't use OpenBSD on it, just the default OS which is a custom roll of Debian.

It's worth noting that hitting that performance relies on hardware acceleration not available on OpenBSD. I've read it can still manage ~100 mbps on OpenBSD which should be good enough for most people. But keep that in mind if you want serious routing performance and OpenBSD.

Is that a hardware acceleration of some portion of the network stack that's not carrying over to OpenBSD?

It's certainly not a problem for me if that's really just a limit on bandwidth, and not some kind of additional CPU load or network latency.

I believe the router relies on a binary blob that lets EdgeOS (based on Vyatta which is based on Debian) offload routing to some specialized hardware.

I run a OpenBSD router in a PC Engine Apu1d and it's more than fast enough. I can hit 700+ mbps on iperf with a basic pf rule set. Since OpenBSD is in the middle of updating their network stack to enable SMP I'd bet that number will rise with the next couple releases. I found mailing list posts from a year ago that have the iperf benchmarks of the apu1d right at 500 mbps. I'm interested to see if the network changes that landed in 6.0 will mean a measurable difference. This is entirely academic on a 30 mbps connection but I like it when numbers change.

An Octane is still much more powerful than your average consumer router. Don't underestimate the value of memory bandwidth.

That said, the real reason these platforms continue to exist because volunteers continue to maintain them. You would have to ask those individuals what value they derive from the hobby.

There are a lot of 10+ year old pa-risc machines out there than cannot be replaced. I remember an optic fiber network currently in use that had embedded pa-risc in underwater equipment, which means they had to be under maintainable for the 25 years of useable life of the cable.

If anyone knows some similar example please let us know.

Usually this comes coupled with a period-correct OS, i.e. HP-UX.

Yes, you could do an image loader for HP-UX bins on another OS, but that level of technical competence is rare.

The reason why OS people do ports - portable software is good software. Avoiding ISA monoculture avoids shortcuts in kernel architecture. Some ISAs (alpha, sparc64) easily trigger invalid assumptions on ordering, alignment, endianness, etc improving overall quality. Alternative archs are also a great way to learn about areas of the system that are "done", so you can go fix and improve the things that were assumed to be "done" once you have requisite background.

The value is probably that there are OpenBSD developers who want to continue supporting these platforms.

Running on such systems keeps the developers honest. No mass produced CPU implements as aggressive visible memory access reordering as Alpha AXP CPUs and SPARC faults on unaligned memory accesses while other platforms "just" take a performance hit.

I have a soft spot for Zaurus as it was my first "smartphone". Compared to the smartphones of today, the UI was horrible, X11 used through a stylus with applications that were barely any different from desktop apps. However you could compile Linux programs [small ones, at least] and run them on your phone!

I don't remember my Zaurus -ever- acting like a phone (it would have been cool if it had!).

Everything else you write, I completely agree with.

What's the deal with "Use the space freed up"? Does this mean there are disk space limits to the OpenBSD distro, and that something has to leave to make space for something new? Or is it a joke I'm not getting?

It's those line deletion/addition ratios man, it has to even out.. or preferably favor the left side.

But seriously, it's often said that the tree is now full, you need to remove something unmaintained before you add may something new.

..more seriously, while it may not be noticeable by DVCS users, working directory sizes for CVS/SVN repositories can actually /decrease/ when code is removed.

Guess: OpenBSD has wide embedded use, so they may want to keep the base image as small as possible.

Why under src/gnu ? It's not a GNU project, it's not under GPL.

Check the README :-)

http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/gnu/README?rev=...

Seems to me that it should be renamed.

CVS versions files, not directories.. and it also doesn't support renames or moves, not without some repository hackery. The historic purpose of src/gnu/ is irrelevant now, it's just a common place for "Gigantic and Nasty, but Unavoidable" software.

So the more appropriate question is, why is someone still using CVS? In 2016? For a collaborative open source project?

I understand that a certain degree of conservatism in software can be beneficial, but given just how limited CVS is, it would be kinda like writing code in ed (now I hope no-one actually does that).

Because it works for the people who are doing the work. Many outsiders have suggested that they switch, but that advice never seems to take hold.

There are a lot of things that "work". If you need to edit files, "ed" works to edit files, for example. But if someone was using that outside of scripts today, I think it's not unreasonable to ask why.

The question is, why, knowing that there are things that work vastly better (I'm not speaking hypothetically here - I used CVS for many years myself, and I remember all the painful limitations, and the nearly universal joy on the team when we finally migrated to Subversion), they still stick to that old stuff?

Sure, there's some cost to migration, but it's been done thousands of times before by other people, so there are a lot of tools that automate all that, and vast knowledge about how to do it right. So the cost really low compared to the objective benefits that they would get - such as, well, being able to rename things without wiping history, or tracking changes across several files etc.

So, is it really just the "that's how we did that in the good old days, and that's how we like it - now get off my lawn" sort of thing, or are there some technical reasons why they consider CVS good enough? I'm hesitant to assume the former until I can definitely ascertain that it's not the latter.

Well, your ed comparison could aswell be made for vi -vs- emacs (or Eclipse spit). As long as the code being written is entered as fast as someone can type on the keyboard, it really doesn't matter which editor they use. Sure, vim and emacs and bloaty-stuff-in-java can help in various ways, but since they aren't really writing the code for you, if the limit is how fast you can write the code, the amount of java or elisp or vim-scripts around your text editor won't have a large impact on how good network code or how safe your string handling is.

I won't argue that the opposite is true, but the general speed limit in openbsd is not the tooling or even the meta-tooling around the source tree, but rather how many decent coders is active at this moment and how much useful code can they get reviewed and committed.

I don't think I've ever seen something along the lines of "phew, if I hadn't had $weird-dvcs in place, I could not have invented this provably correct parallel AVX-accelerated version of SHA768" or something to that effect.

But it isn't vim vs Emacs, that's the whole point! If we were talking about two different apps with comparable feature sets, at least for common operations, sure. But CVS is really vastly inferior even to Subversion, to which e.g. FreeBSD migrated 7 years ago.

Just to make it clear, I'm not complaining about OpenBSD using CVS in general, or claiming that they are too slow or missing features because of it. I'm just wondering why, in this day and age, anyone would use CVS when so many clearly better alternatives exist. I know I would be frustrated if I had to code in a CVS repo, and would push to get it migrated to at least SVN just to make my own life more comfortable. So it naturally makes me wonder why other people in that situation, who presumably know at least as much as me, do not.

I think the reason is that the versioning tooling is 1% of the work, so making it 0.5% would not be "twice as effective", it would move from 100% to 99.5% work to be an openbsd dev.

Diffs mailed between devs still look the same, oks for diffs are still a social construct that needs persons and not programs.

No, the real question is why certain people on HN that likely won't even use OpenBSD let alone contribute to its source code are complaining about what VCS it's using, or what the directory structure of its source repository is like.

You seem to be making an awful lot of assumptions about someone you don't know.

I was actually picking a BSD to install on a laptop recently. OpenBSD got dropped off the list when I found out that they still use CVS for base system and ports (because, yes, I do intend to hack on those).

In any case, your basic premise is wrong. If you see someone riding a horse buggy on a freeway, it's not unreasonable to ask why they're doing that.

I thank you for your wise decision. The question gets old, but CVS fits their development style and release cycle fine. Yes it's ancient but the-known-evil (against the unknown-evil), and it's the whole CVS, AnonCVS, Patch, Mail, Build cycle that would need changing.

> If you see someone riding a horse buggy on a freeway, it's not unreasonable to ask why they're doing that.

Surely it must work for them I'd be thinking. If you use a tool that still works and has worked, changing for the sake of it seems unnecessary.

I really dig that the entire source tree is navigable and viewable via a familiar FS-like tree layout: http://cvsweb.openbsd.org/cgi-bin/cvsweb/#dirlist

And wish github provided navigation that was as easy and lightweight as this.

See the README file in src/gnu/

http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/gnu/...

src/gnu already has Perl in it, which isn't a GNU project or under the GPL either.

Perl is dual-licensed under the GPL & the Artistic license: https://github.com/Perl/perl5/blob/blead/README#L83

No such message.

What the heck? Its there one minute then deletes when I refresh. Ok, an article link is: https://twitter.com/bob_beck/status/772205499949481985 and http://www.phoronix.com/scan.php?page=news_item&px=OpenBSD-A...

[edit: mods can you update?]

The mark.info link works just fine. Here are the first few lines, if it doesn't load from somewhere else:

    [prev in list] [next in list] [prev in thread] [next in thread] 

    List:       openbsd-cvs
    Subject:    CVS: cvs.openbsd.org: src
    From:       Pascal Stumpf <pascal () openbsd ! org>
    Date:       2016-09-03 22:47:02
    Message-ID: 9a28502641f17805 () openbsd ! org
    [Download message RAW]

    CVSROOT:    /cvs
    Module name:    src
    Changes by: pascal@cvs.openbsd.org  2016/09/03 16:47:02

    Log message:
        Use the space freed up by sparc and zaurus to import LLVM.
        
        ok hackroom@
        
        Status:
        
        Vendor Tag: LLVM
        Release Tags:   LLVM_3_8_1
        
        N src/gnu/llvm/Makefile.common
        N src/gnu/llvm/llvm.spec.in
        N src/gnu/llvm/configure
        N src/gnu/llvm/LICENSE.TXT
        N src/gnu/llvm/README.txt
        N src/gnu/llvm/Makefile.config.in
        N src/gnu/llvm/.clang-format
        N src/gnu/llvm/LLVMBuild.txt

... 6200 lines later ...

        N src/gnu/llvm/lib/Linker/LinkModules.cpp
        N src/gnu/llvm/resources/windows_version_resource.rc
        
        No conflicts created by this import


    [prev in list] [next in list] [prev in thread] [next in thread] 


    Configure | About | News | Add a list | Sponsored by KoreLogic

From what I hear, one of two DNS servers is down, so if you get lucky, you see the page, and if you get unlucky, you don't.

Google cache is here: https://webcache.googleusercontent.com/search?q=cache:https:...

Currently there are 3 DNS servers and no problems as far as I can see. (Ignoring the MX record anomaly)

http://www.intodns.com/marc.info

It's actually not a DNS issue, the servers marc.info points to are occasionally out of synch. It would be great if someone told them to fix that..

wut it's back now.

Reply on news.ycombinator.com