Hacker News Reader

Comment by ansimionescu | Hacker News Reader

Good ol' gzip download, because fuck source control.

edit: FWIW, a friend who works there told me flat-out to just avoid them (I'm job-hunting)

This is almost a universal problem with firmware and chip engineers, who are generally older and more experienced, and thus apparently can't be taught new tricks. There are also issues with bureaucracies in each and every semiconductor company (which are on average decades old) so firmware is usually hot potatoed around by management and ends up lighting the customers' projects on fire almost every damn time. I don't have enough fingers to count how many times I've encountered problems in a project and wasted weeks treading water, eventually to find out that the code examples weren't tested, a parameter that closely effects very sensitive silicon isn't documented, or even that I can't so much as get the documentation for a licensed USB peripheral that is non EHCI compliant (who knew such a thing even existed anymore).

Everyone from Broadcom to Qualcomm (and many more) are guilty of this and it is a very rare treat to find a chip that is well documented with firmware and code examples that work and give you a full picture of what you can do and where the pitfalls are.

They also have to live in fear of patent suits. Open-sourcing a modern chipset driver is a pretty ballsy thing to do.

IANAL and I don't have much experience with the legal side of silicon IP but from some cursory conversations with people in the industry, if you're making a chip as advanced as a VideoCore, you're certainly already cross licensing a lot of IP cores. If you license, for example, a EHCI compliant USB core or an MMU or cache pipeline that you get in the form of a VHDL or Verilog file (poor you), that license automatically transfers a right to the patents (as long as the legal chain of contracts were worked on by competent negotiators and lawyers) and rarely do those extend to firmware source code from my understanding.

There could still be problems with the documentation of the interfaces to different cores or whatever, but that's peanuts to a company like Broadcom. I'd guess they already have very liberal agreements for important IP and the stuff you'd sue over is mostly the closed source silicon.

> generally older and more experienced, and thus apparently can't be taught new tricks.

Ah yes, experience, the bane of all technical projects.

I have worked at lots of hardware companies over the years, and to varying degrees, they universally treat software as the red headed stepchild -- an unfortunate necessity akin to paying taxes, or worse.

Hahaha! Ha! Haa...

And that's why I no longer work for a semiconductor company.

Wow, armchair engineer, you sound like you really know what you're talking about.

When this old engineer worked on embedded MIPS cores and 802.11[abgn] at Broadcom, everything was under source code control. This was pre-git, so I think we were using cvs. How the hell do you conclude that releasing a zip file means the engineers don't use source code control?

The fashion of the day is, for better or worse, "if you don't have a github you ain't shit."

git is overkill for facilities with small groups working in a common area (not hard to imagine if you're in the hardware biz) and all the engineers are used to cvs command syntax..

The best VCS is the one you use.

unless it's CVS, or god forbid, Visual SourceSafe.

There _are_ differences, but yeah - you're arguing about the ranking of the 99.91% - 99.95% better solutions compared to no version control. (Seriously - even CVS and SourceSafe are 99.9% closer to "perfect" than doing nothing)

Overkill until the network goes down or the server dies or somebody needs to work remotely and the VPN isn't working. I don't know how fast git is wrt cvs, but I remember being pleasantly surprised coming from svn.

You don't have to have a distributed workflow to occasionally reap the benefits of working with git.

I'm struggling to think of a situation where Git could be overkill for anything... I've personally used it for hardware and software projects where it was just me, where it was used for small teams (3-5 people), and with hundreds of developers doing on a large code base. It worked well on al of them.

The only thing I would not use version control is if you have files that don't change. Everything else should be in some form of VC (sometimes Git is not the best choice, but it's good for lots of things)

Semiconductors, especially VLSI, have a different workflow from most software projects. There are supposed to be as few releases as humanly possibly due to the enormous cost of each release. There are not expected to be a community of developer/designers, each with their own vector of revisions available for development and testing. It is prized to arrive at a single image which is reproduced as many times as possible in practice, rather than in variation, as the lithographic process favors identical reproductions.

In my estimate, Broadcom is on the cusp of reaching a sublime point I've seen in other, unnamed semiconductor companies, at which the number of combinations and revisions of IP within a family of chips becomes difficult to account for.

So, it's really not even a question of git being overkill - it's a question of whether its source code will persist over many times the length of the proejct. Git will be considered for substitution by those deeply committed to CVS simply after a matter of time, once its institutional bugs have been discovered and patched.

On my one man team, the very first command I type on a project after mkdir is git init. Everything is logged and timestamped, and it is far from overkill. Git is easy to learn after working with CVS

A one man team is very different from a 20-man team (which I still think counts as small).

Hey, I like git too. Respecting the needs of others not to spend mental load on learning something they don't want to is often important too.

Is it really worth it to spend time re-training the 15/20 engineers that don't already know git? I don't think so.

Agreed. I write lots of small apps by myself, and I use version control to save me from, well, myself.

Those kids on HN think that everything cool was invented last summer by some new college grad, funny thing is most of stuff was already invented in 70's but who would care about prehistory of computing.

Yea. I totally remember Web Browsers, Big Data, and Terabyte hard drives back in the 70's. It's too bad that these things were lost to the ages, only to be re-invented in the present.

New names for old ideas.

Based on that, SpaceX isn't a big deal, because of the Apollo program, right?

Especially because of the Apollo program. SpaceX for all its efficiency, just goes to show how far the present and past few generations have dropped the ball.

well as a many time Broadcom customer I've certainly seen the best and the worst of it - I've had large (1000 file) software releases for Broadcom chips that mutated their directory structure release to release, bug fixes would arrive and revert themselves, then after complaints return - integrating a new software drop into our VCS was always fraught and took weeks longer than we expected - people would fight not to be stuck with doing the next Broadcom integration and eventually we'd say "enough! we'll live with it" and refuse to take any new stuff - at one point we got the FE to admit that there was no one master tree inside Broadcom, just personal trees that were passed around - on the other hand other code from them was well maintained and obviously under good source code control.

I think the way to understand Broadcom is to realise that they are really a whole bunch of smaller startup companies, more are continually being bought and brought into the fold, this messes with their internal company culture, chips may have functional units in them that came from 5 different companies, each with their own coding standards, VCSs, documentation standards etc etc - as a result it's pretty hit and miss and may take a generation or two for a new technology to settle down - I kind of get the impression that there's ongoing friction between the remnants of these companies as they find their way in the new organisation

Let me guess... BCM43xx? If so, what's your email? This armchair engineer has an eratta shit list a mile long for you. You know, yeahs after the first revision.

I hear you. It's so much worse than you can imagine. By the time the code gets out the door, whether through a dump on the net or even worse, through a channel partner and onto a shelf at Fry's, all the engineers are working on new projects. There is sometimes a six-month lag from the time the code is compiled until it shows up on a store. By then the technology has changed no one gives shit about the old stuff. There is no incentive and certainly no corporate push to go fix the bugs that got shipped. I know I always felt like it would be a simple thing to go patch the code, but it was never a priority for the management.

I'll tell you the really screwy thing about BCM43xx development. We sold reference designs to companies like 3Com and Linksys (before and after acquisition by Cisco), Apple, D-Link, etc. All of them got the same reference design, in theory, but if one of those companies reported a bug in our code, that company would be the only one to get the patched code. As a consequence our code was littered with #ifdef (COMS) or #ifdef (LINKSYS) with specific patches enabled. The other companies, those that hadn't yet discovered the bug, would get updates that we knew were broken. It was terrible for the companies, even worse for the customers of those companies.

Plus there were source releases to those companies and they weren't allowed to know about patches that they didn't receive. So we had a step in our build process called the code transmogrifier. That would go through the code and expand certain C preprocessor symbols so the code no longer contained #ifdef (COMS) or #ifdef (LNIKSYS). That way every company got a different source code release as well. We would do separate compiles of the transmogrified code for each customer.

Then we had all the customization because those companies had fired all their own engineers and instead wanted us to produce a product that was branded "Cisco" or "Linksys". We had to take the exact same code an rebrand copyright strings to make it look like it was written by our customers.

I voted with my feet and left the industry.

akislev: I think you paint with too broad a brush - I've worked as a chip designer, embedded systems programmer, kernel hack - it really depends on the team - also most chip teams I've worked with have been relatively young (most in their 20s)- most big silicon projects depend on good version control software because you are doing continual integrations and practice tapeouts through the last half of a project - you may have 2-3 different versions of the code you're working on passing through different portions of the process at any one time

As the guy on the chip team who understood software well (or the guy on the software team who understood the gates) I do think there's a somewhat different issue that has to do with schedules: as a chip designer you get to do maybe 1 month a year of creative design and 11 months meeting timing and making damn sure that silicon will really work the first time - by the time you tape out you're really looking forward to that creative bit and about the time you've up to your armpits in it the silicon comes back and the firmware guys are doing bringup - you don't have a lot of time for the firmware guys because they're getting in the way of the cool part of your job, and besides you are sooo done with that chip you slaved over last year

I don't know what the answer is other than making sure you have firmware running on your logic simulator long before tapeout (you should be doing that anyway)

Ooops yeah I chose the wrong words and painted way too broad. I meant version control as far as the firmware libraries and public facing code. I can't imagine going from FPGA to silicon synthesis is even possible without extremely tight version control of some sort or another (I think they used to use basic transperencies for comparing masks)

I stand corrected as far as the average age of employees but every time I have interfaced with any level of a silicon company the average age distribution has been skewed pretty high.

22 year old working in firmware department of major silicon IP company here (you can probably guess who - obviously standard disclaimer about I don't represent them).

My company selling IP rather than hardware makes it a bit easier, but: management is realistic and reasonably versatile. Attitude to proper technologies (e.g. using Github instead of releasing tar bombs over FTP) is good. Documentation is good, bar "don't document that or we will be sued" situations, which are too common. Testing sucks in certain areas but is good in most.

Obviously at my age I don't have much experience, but I thought the alternative view might be nice.

Don't worry, they're going to have to release it again. This zip contains many files with copyright (c) Broadcom (and other companies) missing the 3-clause BSD license.

On an unrelated note, x86 looks like RISC compared to this chip assembly language :)

   vmul  -,H(1,0),3 CLRA ACC
   vadd  HX(3,0),H(0,0),2 ACC
   
   vasr  H(0++,0),HX(2++,0),2 REP 2
   ;vmov  H(0++,0),0 REP 2
   vst  H(0++,0),(r0+=r3) REP 2 
   add   r0,16
   addcmpblt r5,1,r1,loopacross

Looks like SSE2 or SSE3 with a different syntax and vector support.

The vector register file is a block of registers 64 bytes across by 64 down which can be accessed horizontally (H) or vertically (V), and in units of 16 bytes (H8), 16 shorts (H16 or HX), or 16 32-bit words (H32).

Each processor has an accumulator that can be cleared (CLRA) or used (ACC).

This code is basically equivalent to the following C code:

    uint8 in[2][16];
    int16 temp[2][16];
    int32 r1,r3,r5;
    uint8 *r0;
    do {
        ... code omitted
	// vmul  -,H(1,0),3 CLRA ACC
	// vadd  HX(3,0),H(0,0),2 ACC
	for(x=0;x<16;x++) {
	  temp[x]=inb[1][x]*3+in[0][x]+2;
	}
	//vasr  H(0++,0),HX(2++,0),2 REP 2
	//vst  H(0++,0),(r0+=r3) REP 2 
	for(y=0;y<2;y++) {
	  for(x=0;x<16;x++) {
		in[y][x] = temp[y][x]>>2;
		r0[y*r3+x] = in[y][x];
	  }
	}   
	//   add   r0,16
	//   addcmpblt r5,1,r1,loopacross
	r0 += 16;
	r5 += 1;
    } while (r5<r1);

This is code for the vector processor. This is not the same as the GPU cores which use a different architecture and instruction set (based around floating point calculations).

I made a back of the envelop design of a arch that used the same technique in college. Way cool to see it in practice.

This project of yours look really cool, https://github.com/peterderivaz/pymeta3

What do you think about http://www.myhdl.org/doku.php ?

Fancy meeting you here :)

Wow, a selectable row-major or column-major register file! I would have never imagined such a thing. =)

What/where is the assembler thar is used compile this code?

Why would a large company release the commit history of source code when it could open them up to all kinds of lawsuits? It seems perfectly reasonable for them to release it as a zip.

why would the history of the commit open them up to law suits, but not the final product?

A trivial example would be a commit that shows something was implemented and then later removed because of a patent/IP violation.

    1-jan-2003 - bob: added new feature X
    5-jan-2003 - bob: Rev 1.0
    8-oct-2012 - frank: Turns our X was patented in 2001 but we didn't realize it. Replace feature X with something that doesn't violate patent ######

Look gift horses in the mouth much?

Why "avoid" them?

It sounds like ansimionescu might have been told that Broadcom isn't a good place to work.

It is open source. You can put it on Github if you want to.

Generally speaking, that's not what "open source" means. Reading the license agreement is always required to determine if that's actually okay to do. In this case it's okay as long as you reproduce the license in any redistributions (e.g. hosting it on GitHub).

The original article states that the code is licensed under a 3-clause BSD license. The 3-clause BSD license permits redistribution: https://en.wikipedia.org/wiki/BSD_licenses

That's pretty much what I just said. "In this case it's okay as long as you reproduce the license in any redistributions (e.g. hosting it on GitHub)."

So how is that "not what 'open source' means"?

There appears to be a common confusion that Open Source means "source you can see". Instead Open Source is a made-up word which was created to avoid the confusion associated with Free Software, as in the Free Software Foundation.

That is to say: an "open source" license which does not let you redistribute is not an Open Source license at all. Even Microsoft has not tried to mis-use the term instead using "Share Source" for their, thing.

Reply on news.ycombinator.com