Hacker News Reader

Comment by dekhn | Hacker News Reader

There's a whole subquestion the article mentioned mostly as an aside: version control systems don't facilitate multi-way merging of binary/complex file formats. I don't know that anybody has developed a good solution to this (that scales to large numbers of different formats), and "just use a text file format" doesn't really work for complex models where changes don't appear like minimal diffs.

I invested fully in FreeCAD, it does support pointers and instances for many ops, but the reality is that you have to instantiate all the instances in 3-space if you want to do complex geometry conflict checking.

View on HN

I'm a mechanical engineer.

The version control software that we use (I'm most familiar with Windchill, and just a wee bit familiar with SolidWorks PDM) is dumb. It's a B2B market with fat margins that is ripe to be disrupted.

Typically in Windchill, a part has a part number, and can be checked out and checked in, iterated, and revised, in operations that are non-intuitive and difficult to reverse. If you ever wanted to build an assembly using older versions of current parts, the process to figure it out might take 100 clicks, or might not be possible depending on how your system administrator set things up.

Merging (in the style of git) is generally a completely foreign concept, and engineers generally avoid collaborating on a single part or assembly file for that reason. Dividing up the interior of a vehicle's engine bay, for example, is best done as separate assembly files that are only later brought together as a parent assembly. Communicating about the volumetric boundaries of these assemblies is complicated.

I'm often aware that I could be more productive and adaptable using a git repo (or similar) containing my parts, assemblies, and drawings than I currently am with Windchill's specialized system. Haven't ever seen it in the wild, though.

I've used Windchill extensively and I've set up a Vault instance (the Autodesk equivalent). I completely agree with what you are saying. At most companies, there is a "Emperor's Clothes" scenario where even the expert users and administrators have no idea what's going on under the hood, or how to execute advanced operations. (granted, Git has the same issue sometimes, let's face it) To make matters worse, Windchill has quite possibly the worst UI of any software I use --- first off, it runs in an instance of Internet Explorer embedded in the Creo CAD package. Few icons are labeled, and the UI is not discoverable at all. Google is often no help, since it is relatively niche BTB software. Doing what the parent suggests, using old versions of parts in new assemblies, is only reliable when saving the old versions out as a "dumb solid" like a STEP and re-importing them into Windchill. Part of the issue is that version control and databased file systems as a mental model do not align very well with how the human brain operates. The hierarchical structure rules the day in the end, because it aligns well with how we think.

I met the founder of Grabcad when he made a trip to Shenzhen China around 2012. I took away that he was trying to build the "github for CAD". We tried the Grabcad system out in our office for a while and it was useful but didn't really meet that vision. I haven't tried it since the company was acquired by Stratasys in like 2014 but a quick login looks like it hasn't changed much.

Completely agree, there is a lot of opportunity here. I have extensive experience using Solidworks, its PDM and another vault software (Adept, don't get me started). Collaboration on reasonably complex assemblies is rarely worth the effort, it's easier to communicate in other channels and link up at the end. File versioning is needlessly complicated and often frustrating, especially for less tech-savvy designers.

I deployed PDM at my company. It works fine, and even for people who don’t really know what version control is.

Multiple downsides though… cost, only really wants Solidworks files (although does handle everything as binaries), but the worst and most unforgivable issue…

They keep full versions of every file, at every check in… forever.

If you have a 500MB file, change the color, and check it it, you now have 1GB of file space taken up, with no clear way to cut that down! Coupled with the fact that deleted files don’t delete until someone manually does a destroy operation, it’s a storage space murderer.

You can setup a cold storage scheme or enable vault compression for old versions to save space on your archive server to save space.

There was something that was a super pain in the ass about their cold storage solution, but I haven’t ever seen a vault compression. I wonder if it’s new or I just missed it.

I’ve spent thousands of hours on pro|e and sw. the past few years it’s been mostly onshape. Their implementation for version control is the best I’ve seen, and is continuously improving. Def check it out if you haven’t

I'm not sure what method it's using (if any) but Perforce is the prevalent version control system for most industries with large files. I think most programmers have heard about it from it's integration with the Unreal engine but as a hardware guy I know it's used at a lot of semiconductor companies for things like managing versions of the various CAD required (layout, schematic, HDL, etc.)

It has the great advantage of being a company you can pay to fix problems. I don't know who you can pay to implement some custom git system for handling huge files and repos. GitHub/Atlassian/GitLab are not interested in working on your company specific workflow.

This might be possible if the binary format encodes a sequence / tree of CSG operations. A binary blob might be difficult to merge / conflict-resolve for humans.

Even if you had that representation, resolving merge conflicts would be a huge headache. Imagine resolving code conflicts by resolving things at the AST level.

Forget CAD, we don't even have a good way to version control images, something programmers work with every day, besides "use Git LFS and pretend an image is just a pointer".

edit: just saw GumTree mentioned above. Guess I have something to check out...

Of course not possible with images as a binary blob but it could be possible with a series of operations on a (vector) image ("event log").

>Imagine resolving code conflicts by resolving things at the AST level.

This sounds extremely nice actually. The more semantically aware the version control system is, the more likely it can gracefully handle merge conflicts. A textual diff is a very low level transformation primitive; composing them doesn't guarantee very much at the higher level.

Coming at it from the other end, you may or may not already be aware that one of realthunder's branches (can't remember which right now but probably LinkStage3 since that's where the action seems to be) lets you save the .FCstd file as a directory so that you can commit it to git. I've not tried it myself yet to see how useful the diffs are, but it's not unfeasible to think there's some value there.

GumTree does AST level diffing, hypothetically one could build VCS on top of that. That would work for binary files as long as they are parseable to some sort of sensible AST.

https://github.com/GumTreeDiff/gumtree

I worked on this type of problem in the context of structured data in a version managed database, and you need more than just an AST.

These types of systems have lots of extra cross dependencies and complexities so to resolve changes you need enough to effectively replay changes from one branch onto another. You’ll find points where this will not work and at that point you need to be able to present the user with self consistent alternatives to choose between, and a way to safely save and recover the other data while they do this. That goes through the entire application stack and UI.

The stuff I worked on all worked, but every application and customer that built on our core needed to develop custom rules for their work flows, and I’m not sure many ever did.

> you need to be able to present the user with self consistent alternatives to choose between

That seems very high bar to aim for compared to git (and other traditional VCS), which do not ensure that the end-result of a merge (or rebase) is in any way sensible.

It is a high bar, but consider what happens when you get an inconstant merge. With text representing a program you get a compiler or interpreter error and can view and edit the text to fix things up. With something like a CAD drawing you may not be able to open the file, or it might be almost incomprehensible to the user (if you broke coordinate transforms or accidentally combined sub assemblies or something). If you are really unlucky you end up with something that looks okay but subtly breaks the application in a way you will not notice for a long time, and then it’s going to be hard to fix.

Oh, and you need to implement this for every file type.

Lots of programs have an "undo" feature, perhaps the history of this could be saved along with the files.

EDIT: that is, saving all user inputs used to generate the binary files in a text format.

Reply on news.ycombinator.com