Hacker News Reader

Comment by sammcf | Hacker News Reader

I had a tremendously embarrassing failure in a sheet metal part designed in Solidworks due to a bug in the way bend allowance is calculated. The long story short is that the flat pattern developed by SW was about 10% dimensionally inaccurate which resulted in a rotating part binding up in the field and going blammo. This was years and years ago as a fresh young designer, so I reported the issue to my reseller and the response was essentially "yeah, you're gonna have to take that into account." I've since learned that this sort of idiosyncratic difference in calculating certain values, particularly things like bend radii and bend allowance, things like beam extensions on structural frames, etc, is par for the course for CAD packages. Pretty much every engineer I know has learnt this lesson the hard way.

I guess it's not really a bug per se, just hidden behaviour that you have to learn about and take into consideration.

View on HN

> "yeah, you're gonna have to take that into account."

What? But the software was supposed to take that into account, which is why it it supports bend allowance calculations in it!

(I have a tiny bit of experience with this; I designed some sheet metal thing. I somehow clued into bend allowances before doing it, even though it was my first and only one! I figured, you can't just draw the naive shape; the metal has a bend radius. How the hell does that work? [half a day of Googling] Oh. I used a 2D drafting program that had no clue about this, so I manually drew the pattern with the allowance in it already, based on the thickness of the aluminum.)

> But the software was supposed to take that into account, which is why it it supports bend allowance calculations in it!

Structural engineers aren't allowed to treat software as a black box and trust that it works correctly. They're legally responsible for choosing their tools and verifying that they work correctly.

I.e. the "that" in "that into account" means the tool being possibly wrong.

There is no, "sorry the TV blew up in your face; my scope was out of calibration". :)

Uh, I don't think there is a better example of closed source than CAD packages.

I don't think he was talking about verifying the software itself. Even if the software was open source, would it have been formally verified to make sure it has zero bugs? It is better to check the calculations by hand to make sure that no critical errors make it into the final design.

If you are checking the calculations by hand, then it seems to me you are verifying the software. It's not a bad thing to do, but I wonder why it's considered acceptable when the results don't match your own? If your results are correct, then you have a malfunctioning tool...

Wouldn't it be helpful to be able to look at the code to see what it is actually doing behind the scenes?

Not if you can't code, which most engineers cannot (nothing wrong with that). In fact, I'd say without a team of highly trained software engineers and a whole lot of time and money, having the code for a CAD program is almost useless. In civil engineering, it's my understanding that it's standard procedure to check everything CAD tools output. If a bug leads to failure in the field, at least one person, but probably multiple people, has failed horribly and he will rightfully be held responsible.

There are many problems where finding a solution is much harder than verifying a solution.

A well known one used all the time in crypto being finding the prime factorisation of a number versus verifying that prime factorisation.

So verify the solution you give me!

If the whole point if verification is that you don't trust your software, then that's not a very good idea. So you use the ultimate slow-but-reliable computer, the trained engineer.

Yet another example that software engineering isn't.

I think think this varies very much between different "cultures", such as different countries and different fields.

I make a structural engineering program which is used by pretty senior engineering types in Scandinavia and Germany, who can certainly be trusted to know that with some nonsense input, the output may also be nonsense.

The requirements for the UK are completely different - they want the program to be used not as a tool (like a fancy calculator) but as a simple black box that a novice engineer or even a non-engineer can use.

I remember meeting an engineer over easter brunch a few years back and I asked him what his actual day looked like, you know day to day activities. Then someone started making a toast so he quickly summarised 'I make calculations, and then I add a giant margin of error that's totally unnecessary if our model was completely correct, just to make sure' haha.

I don't get this though...

Like if the software calculates bend radii that will break things, isn't that a bug? What's the explanation for that even happening? Is it just that the formula involved means that you can't always get a "right" answer, so the software will be like "well, this is the best we can do, figure out the rest"?

>Like if the software calculates bend radii that will break things, isn't that a bug?

Well, it's not like all companies fixes their bugs or care about them. If, as the parent says, all CAD packages have such issue, what you gonna do?

The difference in LibreOffice not rendering an EMF correctly and a bug that causes wrong dimensions on a drawing in a CAD tool is that LibreOffice gives you a bad looking document and a CAD program costs you money when the material starts to be cut up and it all had to be thrown away as scrap.

Mostly I can fix the LibreOffice bug and make the document render correctly (unless it physically corrupts the document when saving), once physical material gets an error it's often impossible to fix.

So what are you gonna do? Move to another CAD package? Take a look at the reasonably large number of CAD packages listed on Wikioedua here:

https://en.m.wikipedia.org/wiki/Comparison_of_computer-aided...

Virtually all of them are proprietary. They are too expensive to move away from, and every one of them has flaws that firms workaround and build there workflows on. Retraining, proprietary file formats and the disruption of migrating an essential part of most businesses is prohibitive.

Frankly, IMO the CAD marketplace is an example of how closed source really bites an industry. Funnily enough, most engineers - who would never accept inaccurate tolerances on items like screws or bolts - seem to accept this is the way it is and despite the stories we read here let the situation continue.

I'm frankly amazed there aren't more failures in infrastructure from bugs in CAD software. That's more a testament to the fact that engineers are engineers though - when they realise there are problems, they dig in, figure out where the problem is and find a solution, and in the case of CAD software I really think they build or design their way around the problems.

Overall, CAD software is, despite all the flaws of being coded source, still pretty amazing and vendors do fix issues eventually. We can do so many more things in society because of it, but as we evolve as a society it becomes apparent there are limitations. Eventually CAD software vendors will be disrupted, but we're a long way from that point right now!

I mean sure, things have bugs, right? I get that.

Sometimes autocorrect messes up things I type. It's not really a bug per-se, more just a side-effect of how things like autocorrect work, right? But sometimes Chrome crashes, and that's a bug...

The OP seems to say that the bend radius thing is more of the first case than the second, and I'm wondering how true that is. I'm not really sure of the science behind it (maybe it's an impossible constraints problem).

Oh, no doubt. The issue is that the tools don't appear to document their limitations and constraints, and there is often no way of knowing until you get bitten.

At least with OSS you can look at the code behind the program to work out how it calculates things like the bend radius.

I'm no engineer, so I had to look it up (Wikipedia was no help whatsoever on this occasion) but I located this page which gives the formula and an explanation:

"Calculating the correct flat pattern layout is crucial to getting a good quality finished part from your press brake. Yet, many CAD and CNC programmers have no idea how to calculate the required values. Years ago, the real experts created cheat sheets and tacked them to the wall. They only taught the new apprentice how to apply the results shown on the cheat sheet, not how to calculate the numbers. Well, now those experts have retired and it's time for a new generation to learn the right way to do the calculate the correct flat pattern layout."

http://www.sheetmetalguy.com/bend-allowance.htm

Now admittedly the guy does put the responsibility on the operator, but it appears CAD packages can indeed calculate this for you. So long as the right info was input into the program, the formula should be programmed correctly and the operator should be able to have confidence in the results!

>Now admittedly the guy does put the responsibility on the operator, but it appears CAD packages can indeed calculate this for you. So long as the right info was input into the program, the formula should be programmed correctly and the operator should be able to have confidence in the results!

"Should" is the operative term. I can tell you with that having hand checked the automated results of several CAD packages at developing sheet metal flat patterns and finding unexplained variability and errors too many times to count that if you want confidence in the dimensional accuracy of a sheet metal part, you need to develop it by hand. Especially for anything over 5mm in sheet thickness. As far as I'm concerned you really can't engage in sheet metal design without being able to manually verify the development results, so in a sense it is up to the designer regardless of the capabilities of the CAD software.

To be fair, I'm picking on one of the hardest physical manufacturing processes to actually get right - even CNC brake presses have a great amount of variance in calibration (e.g. the amount of bend allowance you need to make for various sheet thicknesses...) across different suppliers, materials, etc to produce physically identical parts. The valuable lesson for a fresh designer is don't trust your 3d representation of the part, don't trust the manufacturer specs on ANYTHING and always prototype. I'm guessing this is why people tolerate it - the world with 3d CAD is still a vastly nicer place to be an engineer than otherwise, so we just rationalise away the shitty parts as a reasonable price to pay.

I'm guessing this is why people tolerate it - the world with 3d CAD is still a vastly nicer place to be an engineer than otherwise, so we just rationalise away the shitty parts as a reasonable price to pay.

Yeah, I know. In my experience, engineers are normally immensely pragmatic people. My dad was an engineer, so I have a lot of respect for the profession. It would just be nice if CAD would be more open.

> Eventually CAD software vendors will be disrupted, but we're a long way from that point right now!

Not so sure we're that far off. Nobody really has much incentive to innovate in this space because it's so niche. If you're a believer in the version of the future where 3D printers are as commonplace as 2D printers are today, then this may all change.

Suddenly, 3D modelling skills are today's word processing skills (either for work or for fun), and there are a whole bunch of people working on coming up with innovative CAD solutions for the masses.

Your analogy would work better if wordprocessor vendors had been disrupted by open source alternatives that took the world by storm.

I would argue that SolidWorks has been doing a good bit of innovation. The fact that its been stealing more and more market share over time speaks to it.

Each new version of SW introduces new "features" (some genuinely useful, some shiny marketing gimmicks) and a host of new bugs. Software churn at it's best. Gotta release a new version to make money, just fixing bugs isn't profitable. Oh, and make sure file formats are not backwards compatible, even if for no good reason but to force upgrades.

>Nobody really has much incentive to innovate in this space because it's so niche.

http://www.pwc.com/gx/en/industries/technology/publications/...

Siemens makes CAD packages. Dassault too. Autodesk does nothing but CAD packages. All in the top 20 of "software companies by revenue". PTC is number 42 and Bentley is 70 again CAD packages. It might be a niche, but it's pretty expensive niche then. Pure CAD companies Autodesk, PTC and Bentley made 2,5 billion dollars of revenue in 2014.

There is a common file file format for CAD as well as other areas of engineering [1], most proprietary CAD tools support it. I am the editor of a small part of it.

[1] https://en.wikipedia.org/wiki/ISO_10303

That said, in the real world there are plenty of people who don't want to use it, for whatever reason. I've got customers who insist that they'd rather use a not-yet-written reverse-engineered Parasolid X_T exporter than a well-tested STEP exporter that's been around a decade.

We also get people coming along every few years who argue that because there are some people not using STEP we should start again from scratch. They are very enthusiastic for a while until they realize how much work is involved and they give up, having wasted a fair bit of time in standards meetings.

I think there is scope for some disruption from small companies willing to use STEP as the internal data model for their software but just working on small parts of the picture.

It is a formula but you need to know the physical characteristics of your metal.

Or it's on purpose. This way you can sell training courses and each user is locked on it's cad implementation due to the cost of retraining on another tool

What all the non mechanical engineers don't realize is the bend allowance is different for types of material (steel, aluminum), grade or strength (high tensile steel will stretch less than mild carbon steel), and even the tool (mandrill vs brake bender). Software like Solidworks and Inventor come with default values. You need to know to go in and change them based on your circumstances.

I don't know how expensive is the stuff you were making but if it was just about cutting metal it can't be that expensive. Isn't testing samples the 101 of engineering or have we gone 100% trusting CAD now? It looks to me that there are so many things that can go wrong in real life like deformations because of temperature, different calibrations on different models of machine, different composition of the material used, etc...

Metal isn't that expensive I guess but still, several hundred dollars and a turnaround time increase of a week+ minimum isn't always on the cards for every project. Sure, if I had unlimited time and budget, I'd physically prototype and field test everything. Client demands tend to take precedence. Part of being a good engineer is balancing those concerns, learning which parts of the design and manufacturing process can be trusted implicitly and which assumptions must be rigorously tested - suffice to say I've been a lot more stringent about sheet metal design in the years since.

> I reported the issue to my reseller and the response was essentially "yeah, you're gonna have to take that into account."

> I guess it's not really a bug per se, just hidden behaviour that you have to learn about and take into consideration.

I'm not that kind of engineer, so I may be missing something, but it sounds like the reseller snowed you into blaming yourself for their faulty software.

I get what others have said, that a structural engineer is responsible for the things they produce and for doing whatever is necessary to verify correctness, but that doesn't mean that software engineers get to pass off responsibility for the things they produce.

Usually there is a lire in the eula for this purpose. I have no idea how this would plan in front of à judge though

I mean, it's a really easy issue to work around (if occasionally time consuming), most people who use the software are familiar with the issue, it's not enough to make me want to change CAD package given the massive sunk costs one rapidly accumulates using a particular package...I guess there's no real incentive for the software guys to devote time to fixing this when they could be working on new features or whatever.

In a better world, I'd love for bug fixing, or even just polishing workflows for existing features, to be as high of a priority for CAD companies as new feature. However, only one of those things is seen as growing the install base...

Reply on news.ycombinator.com