A History of Source Control Systems: SCCS and RCS
experimentalworks.net
experimentalworks.net
Yep, here's the first delta in one of the SCCS dirs...
^As 00031/00000/00000
^Ad D 1.1 98/04/27 21:02:41 dhd 1 0
^Ac date and time created 98/04/27 21:02:41 by dhd
http://www.exnet.com/About_Us.html
And I have found some there from 1995, which is about when we will have taken the HTTP server live...
I am super curious about your experience with SCCS! I have never really used it for personal projects, so the blog post is purely based on research and a bit of trying. Really curious what oyu loved and what annoyed you about SCCS!
Collections of revisions were self managed as you say.
RCS came along and I think that’s where the $Id$ construct came in. This would expand to the version number of that particular file - if you const char * const idstr = “$Id$” then you could “strings” your binary and see what components were there. Of course, fileguard #ifdef’s would affect this.
Finally, CVS (maybe) had the idea of tagging, which would associate a single label across a current snapshot of different files and their versions.
In 1993-94, ClearCase came on the scene and tried monetizing this concept. A site license cost $30k in 1994 dollars and required a kernel patch for it to work. Try explaining that to your DEC, HP or Sun representative. The software implemented its own filesystem - you would set some kind of version label on a directory and then magically all files and sub folders would appear as of that date. This included libraries, binaries, data files, even crash dumps.
This is all from memory, someone can fact check and correct the finer details.
Nice blog post!
Systems like Clearcase is where “release manager” type roles were born. One poor guy would become the expert trying to bang a release out of it over weeks or months.
RCS had tags but they were single file tags; the two big things CVS gave you were "this collection of RCS files means something as a whole" (which made tags and branches meaningful) and "you can sort-of-efficiently update from a repo on the other side of the country". (The former was the entire point: CVS was written so Solbourne could maintain their multi-cpu-SunOS fork and still pull changes from upstream; the latter was from Cygnus having a similar upstream+local problem and developers distributed across the planet.)
I also ran across SCCS for Linux somewhere, I forgot where I saw it. Maybe you can look for it and play with it.
We switched to PVCS which made it easier to have isolated work spaces, plus a bunch of scripts to build those so developers could effectively emulate feature branches. Merging was a bit painful but at least you could work without holding other people up.
I’ve used other tools since but git is damn good compared to those older tools. I do miss being able to embed version strings but in reality tagging is superior if you have disciplined release control.
I have been a solid user of SCCS and successors since SCCS became available out of the box on SunOS.
The delget/deledit pessimistic model of SCCS (and RCS, and CVS?) were poor compared to newer VCSes such as SVN and git.
The pain of getting SCCS (and even RCS) to work for multiple networked users was ... well ... profitable for me at my hourly consultancy rate...
But even SCCS saved me/others from many traumas and detected file corruption (even a failing SCSI disc connection once, IIRC).
Getting groups to move to CVS was hard. Devs were terrified of conflict resolution if they edited the same file concurrently as another dev. Took patient explaining that it would only be an issue if both devs modified the same lines, and even then it would guide you through the experience.
But they did let me, a young engineer, convert them from SCCS to CVS, so yay!
This is fairly accurate according to my experience. It seemed absolutely crazy to me when I started computer science at uni in the early 2000s that nobody was using version control. It wasn't even mentioned as a thing we might want to look into. I had to go and look for something and found SVN. I ran my own server and thought it would be perfect for when we had a group piece of work, but I was unable to convince the rest of the group to install the weird software and cooperate. By '09 I'd discovered git and it was a breath of fresh air.
> SCCS stored each version’s delta along with metadata ... In principle, it is how most version control systems work today.
I think this is one of the most misleading and confusing things people seem to get taught about version control systems. Whether they store deltas or not underneath is actually an irrelevant implementation detail. But surely it's easier to think of version control systems as storing version? It's right there in the name! It's not called a delta control system.
Post author here!
I agree! One of the fascinating bits of SCCS is that it actually exposed the idea of deltas right away to the user. Of course in the 70s, the amount of users was limited, pretty much exclusively programmers familiar with the idea, and probably all people who knew each other :).
Nowdays a VCS should just care about snapshotting your work. In my experience, only Git is so confusing that people need to understand the underpinnings of the system (whats a GC, whats a tree, etc). Systems like SVN, Mercurial & Co all seem to mostly do the right thing for the user with minimal knowledge on the user side.
Fun side note: I had a heated argument with Olivia Mackall, the original author of Mercurial, when I wrote Mercurial bookmarks. I initially called it refs, since they were models after git refs. Olivia insisted to call it bookmarks, since it's the more intuitive english name for what it's doing (bookmarking a part if a book of 'history'). Nowdays I much prefer the name bookmarks, since it's an analogy that makes it easy to reason about what it's doing, rather than a technical term that needs to be learned.
It's really interesting that those early systems really did deal with deltas. Unbelievable that ideas from systems that have been and gone before people's lifetimes still prevail today. When was the last time users needed to know about deltas? Possibly skipping ahead impatiently here...
I have an another blog idea for you: what are these "right things"? I have used Mercurial a bit when I was new to programming, and back then Git and Mercurial seemed more or less the same with different command names. Today, I almost exclusively use (and hate) Git, but I find it hard to see what alternatives there would be in DVCS space, other than something lie Darcs/Pijul (perhaps my imagination has been stunted by too much exposure to Git). It will be great to have someone with the knowledge lay it out comprehensively and explicitly, so that the next generation of VCS developers will be able to build upon it.
I do have thoughts about waht the next generation of source control systems should look like! It hopefully will be the last post in the full series. So estimated arrival time for that post, by interpolation of how long it took me to write this post, is roughly 1-2 years :P
1. The SCSS/Git way, aka the hacker way: look at what we can do with existing stuff, and use that to build something that can do the job. For example if you're one of the world's expert on filesystem implementations, you can actually produce a fantastic tool like Git.
2. The mathematician's way: start by a model of what collaboration is, and expand from there. If your model is simple enough, you may have to use complex algorithms, but there is a hope that the UI will match the initial intuition even for non-technical users. Darcs did this, using the model that work produces diffs, and conflicts are when diffs can't be reordered. Unfortunately this is slow and not too scalable. Pijul does almost the same, but doesn't restrict itself to just functions, using also points in the computation, which makes it much faster (but way harder to implement and a bit less flexible, no free lunch).
3. The Hooli way: take an arbitrary existing VCS, say Git. Get one of your company's user interviewer, and try to please interviewees by tweaking the command names and arguments.
The tradeoff between 1 and 2 is that 1 is much more likely to produce a new usable and scalable system fast, but may result in leaky abstractions, bad merges and hacks everywhere, while 2 may have robust abstractions if the project goes to completion, but that may take years. OTOH, method 3 is the fastest and safest method, but may not produce anything new.
So, I am the main author of Pijul, and I also don't quite see how to do much better (I'm definitely working on improvements, but not technically radical). But the causal relationship isn't the one you may be thinking: it is because I thought this was the ultimate thing we could have that I started the project, not the other way around.
In retrospect VCS had a long way to go, but going from nothing to RCS was huge.
The BSD archives published by Kirk McKusick include the source repository in sccs, so I have used sccs a little to drill down into nuggets of history there.
In a former job I inherited some sccs repositories which I converted to git. (I believe in preserving history) https://dotat.at/@/2014-11-27-uplift-from-sccs-to-git.html https://dotat.at/@/2016-07-19-uplift-from-sccs-to-git-again.... (sadly the links to the scripts are broken; I wonder if I can get permission to republish them…)
More weirdly we had an ancient and janky terminal-based user interface to allow colleagues to edit configuration files in a controlled manner. It avoided concurrency problems by using rcs under the cover to implement locking.
Many years later I bought the book "Applying RCS and SCCS: From Source Control to Project Control", half-read it and decided I still didn't feel it was something useful to me. It wasn't until 1998 or 1999 that I started using CVS, then a few years later switched to SVN, an finally to Git (between SVN and Git I spent a few years using Unity's Asset Server).
A strange case I've run across from the SCCS and RCS era is Plan 9. The history of the Plan 9 kernel is stored as ed scripts[0], which produce revisions per file, essentially like ad hoc SCCS deltas. I'm not sure if it was assembled as such after the fact or recorded like that all along. That method seems to have only been used for the kernel and the rest (such as the libraries) was snapshotted daily on a file server starting in 2002.
[0]: https://9p.io/sources/extra/9hist/README (To look around, just pop off README.)
Another weird one is a distributed version control system over UUCP [1]. Although not distributed in the modern sense
[1] O'Donovan, Brian, and Jane B. Grimson. "A distributed version control system for wide area networks." Software Engineering Journal 5.5 (1990): 255-262.
All subsystem maintainers had their own way of working with this. Some used svn, some used bespoke scripts.
The main reason for creating bk and git was that none of the existing version control systems matched the workflow of sending patches via a mailing list. This is quite clearly reflected in the design of git, a tool for quickly wiping and recreating a whole project directory.
Linux development history was imported to git, and you can follow it back in time since long before the introduction of git, so in that sense some form of version control can be said to have existed.
My understanding is that even during the BK years, some maintainers refused to use it and would continue to use tarballs + patches.
We've used version control system based on SCCS. Like CVS is based on RCS format, this was based on SCCS.
But it was distributed! Only way to clone repo or push or pill was direct file system access (so, NFS was used, of course), without any protocol over HTTP, SSH or TCP.
It was very cumbersome and difficult to understand & master. I could say, it was more complicated than git, because it felt alien. I don't know why and cannot pinpoint reasons for it.
It had generator for web reviews (like modern pull reviews), but again, generator produced static HTMLs with different diffs, and you need some place to host them and they were non-interactive, nothing like in-line comments and such.
At the very end, right before acquisition, we converted repos to Mercurial, but tools for web reviews was ported to hg with exactly same output. And then I left Sun, errr, now Oracle.
There exists an abundance of resources related to TeamWare. From online documentation [2], to patents [3]...
[1] https://en.wikipedia.org/wiki/Sun_WorkShop_TeamWare
[2] https://docs.oracle.com/cd/E19957-01/806-3573/TeamWareTOC.ht...
Joke's on y'all, our "release lock" in first semester introduction to programming for team assignments was the name of our ZIP file and whoever cc-blasted every member of the team with "team07_assignment03_v07.zip" called dibs on it :)
It goes without saying, that "it works on my machine was out mantra". Those were the days!
Manager: The client says the program is not working.
Programmer: I don't know whats wrong, it works on my machine.
Manager: Fine, then we will ship your machine
And thus docker was born.
(1) Lock the file(s) as soon as you start coding. Commit when finished coding.
(2) Do all your coding against unlocked files. When finished coding, then lock the files, and copy over your edits to them. Then commit.
Style 1 is fine if there is low contention for the files, like a team with few members or a part of the codebase that is rarely changed. You never have to manually copy your edits to an updated file. If everyone uses this style, there's no such thing as a conflict. It's not even possible.
You also don't necessarily have to coordinate offline with anyone to prevent situations where people unknowingly proceeding in incompatible directions with the same file. Coordination is still better, but locks naturally prevent this to some extent.
You do need to be responsive to emails to avoid blocking others. And it's possible to forget you have something locked when you don't even need it to be, like if you were just starting to code and paused or abandoned that work.
You may have disagreements about whose work is more important and whether you should release a lock or make the other person wait.
Style 2 is helpful if there is high contention. You need to plan and coordinate more since otherwise it's possible two people could make incompatible changes to the same file without knowing it until the last moment.
If the base version of a file changed (since you started coding against it), you have to manually copy over your changes. This definitely involves tedium and may require resolving conflicts.
This style is a lot like what we're accustomed to today with Git (and Subversion and CVS before that). But updates and merging are totally manual instead of mostly automated. Back in ancient times, codebases were smaller, so although that was definitely a burden, it was doable.
edit: typo
Of course, RCS was freely available, while SCCS needed to be licensed from Bell Labs, so RCS was much more widely used.
Super interesting! I heard rumors about it!
Do you know if that was just a matter of implementation or a matter of data structure usage? In my mind, the RCS approach makes more sense for common operations such as checking out the most recent version, particularly if history is deep. (Writing would still be same, given the most recent version is in top in RCS (and not just append only).
Would love to get more flavor on the his, and happy to amend the post with more information!
I still use RCS for my personal projects. I had moved to github, but M/S ended that experiment with their changes, so back to RCS and anon ftp on sdf. One thing I really like about RCS is it will update tags in source during checkin, easy to find the version of a binary. I would have stuck with git if git did that.
Anyway, I was working at a fortune 500 company (which I just left), the group I was in had no SC until I was hired. I forced them to use RCS, and once we got a server I had them move to CVS.
Then a couple of years ago, we moved to git after corporate got serious about SC (Source Code Control). They contracted with github soon after the Microsoft purchase, so I moved our CVS to github. FWIW, when I left, many people were still strongly resisting SC.
I know some source was lost, and I wonder if the attitudes have finally changed.
https://www.logicbig.com/tutorials/misc/git/accessing-remote...
You can also use gitea or forgejo for a similar experience to GitHub but self-hosted.
If git had something like that I would have stuck with it.
My editor can tell me committed-on if I want it to, so it's a bit of a head-scratcher for me.
* Tags in the binary which can be listed using ident(1)
* If I have a object checked out, no one can check in their changes without talking to me. Most other SC deals with this automatically, but I am old school :)
* works nice with emacs, but on OpenBSD the emacs interface is broken, you need to install GNU RCS for OpenBSD :(
One thing I really like about git is gpg signing on commit, but on the BSDs pinentry* does not prompt when used from inside emacs, so signing fails when using gpg
The inability to lock checkouts is one of the things that a co-worker of mine fought tooth and nail about when we moved from RCS to CVS.
Alternatively, CVS and SVN have RCS-style keyword expansion built in. Either of these is an obvious improvement over RCS, so I don't think there are any good reasons not to upgrade.
I can not overstate what a mistake this is. The fact that Git is effectively incapable of storing binary data is an utterly terrible thing. Our systems, tools, and pipelines are so much more fragile and complicated than they need to be because of this limitation.
My blog post going into further details: https://www.forrestthewoods.com/blog/dependencies-belong-in-...
I work with a couple source available pieces of software that have their dependencies all packaged in the repo and it's a joy to work with compared to trying to compile a typical open source package.
> it's a joy to work with compared to trying to compile a typical open source package.
I know right. Once you experience it and realize how delightful building a project can be it’s hard to go back. I swear people don’t realize that things don’t have to be as painful as they are!
> Note that all RCS files are usually stored as the filename with an appended suffix ,v.
RCS also supports storage of them in an RCS subdirectory if it exists for the file (much like SCCS and probably inspired by same).
Another utility which initially shipped with SCCS in use to this day is what[0].
0 - https://man.freebsd.org/cgi/man.cgi?query=what&apropos=0&sek...
It works really well for those. None of this extra overhead of sticking your entire $HOME in a git directory, or weird symlinking to some monorepo or whatever. Nice, intuitive, easy to manage, easy to back up and sync, perfect for local condig files you're not looking to turn into your next big project, just a file you want to potentially roll back from time to time.
The bespoke layers I saw were usually kludgey, but sometimes they worked well enough, in a "worse is better" kind of way. Other times, people were greatly relieved to eventually get a better off-the-shelf SCM system.
This building atop SCCS is an example of starting with the minimum and adding what you need. It has different pros&cons, compared to a more typical contemporary approach of starting with the maximum off-the-shelf framework/platform/service, and figuring out how to get it to do the simple thing you actually need.
There's a sweet spot in between the two, of sometimes using some big/medium-complexity off-the-shelf thing, and other times keeping it simpler, while making better decisions than flipping a coin. Though that tends to get lost in the reality of resume-driven-development, when the incentive of a decision-influencer to add a keyword to resume isn't aligned with the goals of the current project or current company. (But companies only have themselves to blame for this, when they hire by the keyword. Of course, selecting for and rewarding pursuit of keywords, that's what they'll get. Similar with hiring for Leetcode.)
It offered:
* role based access (administrator, integratore, reviewer, developer)
* code review
* continous integration
The original paper is from the past century https://web.archive.org/web/20151018170817/http://aegis.sour...
https://aegis.ifkee.de at least has a copy that builds on contemporary Linux.
The SCCS interleave format lived in BitKeeper and PVCS/Dimensions.
Thankfully at the time, I chose SVN over CVS and although SVN was only at v0.8 my small team never lost any data. I never regretted my decision.
i really wonder what this means, i am not aware of anything by that name. I sure hope he follows through with this seemingly quite ambitious series.
Realistically knowing my track record, the series will probably take 2-never years to finish.
He was by far the best known professor in computer science, much loved by his students: https://www.reddit.com/r/KaIT/search/?q=Tichy . Guess the memes are only half as funny if you don't understand German.