The advantages of an email-driven Git workflow
drewdevault.com
drewdevault.com
> A very large body of email-related software exists and is equally reliable and well-understood. You can interact with email using only open source software and customize your workflow at every level of the stack - filtering, organizing, forwarding, replying, and so on; in any manner you choose.
Using email software in general is well-understood, but using it for Git is much harder than using GitHub. Customizing the workflow is also something you probably shouldn't want to do unless you're really working with a massive project such as the Linux kernel.
And whatever the advantages of this workflow are, there are also disadvantages, or at least unanswered questions. How do you switch from inline diffs to side-by-side, toggle whitespace, integrate the results of a linter/static analyzer, etc? It's a good idea to build your collaboration tools on top of plain text/simple interfaces, but there are also a lot of needs that push you in the direction of a complex/powerful app.
I was interested in the idea of alternative git tooling, but I don't think this article really delivers.
Your perception might be that they're addressed, because you see your vision of how things should work. But to me, I can't see more than "trust me, I'm working on it". The value of that assurance depends on how well you see the limitations of the older workflows, not just their advantages.
If you build it, that will assuage most worries, but talking about what you need in addition to the pure email workflow, and showing how it can be built would also help your case.
There's more tangible stuff here than "trust me, I'm working on it", I hope. A lot of the non-mail-related work exists and I'm getting started on mail stuff now, so I'm starting to write about it to spurn discussion and start paying attention to people's concerns with the approach.
You can also see some of the specific email-related stuff I have going if you want to dig some more:
https://git.sr.ht/~sircmpwn/lists.sr.ht/
https://git.sr.ht/~emersion/python-emailthreads/
The latter in particular could give some good insight into how this stuff is going to take shape. Would love to hear more of your concerns, keep them coming!
The article is a good complement to one that I wrote about specifically the advantages - https://begriffs.com/posts/2018-06-05-mailing-list-vs-github...
>reviewing patches is quite easy with the email approach as well [...] It’s just email - you can reply, forward the message, Cc interested parties, start several chains of discussion, and so on
The quote you gave also misses an important point directly preceeding it, which is that email is decentralized and federated. I also wouldn't understate the importance of all of it being open source.
This chore is very real, and very time consuming. Reviewers need to make sure that new versions of a patch are responsive to past [accepted] commentary. Authors need to find all commentary when updating patches in order to be responsive to it.
Not that code review tools from GitHub, GitLab, and others, are all that good. I've yet to find a code review tool that really works for me and the people I work with.
Whether you use email as a transport and/or interface, or something more weby, or whatever, at the end of the day, for me, the real win lies in code review and commentary management and UIs. Because I'm a TUI person, my approach would be to build a code/patch+commentary format, something like (designing on the fly here):
- foo.c
| /* This is code (or patch) */
? jane ^^^^^^^^^^^^
? jon The above is code, yes, but this is commentary!
| while (!review_done)
| ...;
- 0001-This-is-a.patch
| diff ...
...
An XML version wouldn't be a bad idea, provided a) that the schema is trivial, b) that there are tools for converting between a simple text format (e.g., like the above), c) preferably things round-trip. Obviously, with XSLT/XPath, (b) should be feasible, especially if we heed (a). And (c) should be possible if the XML schema is kept simple enough (but see below).
What's nice about this is that as long as there is version information we can merge commentary from many emails into one document (per file/patch), which then could make automation of commentary collation possible.
This needs lots more thought, I grant that. For example: how to cut-down quoted text without losing the ability to trivially merge commentary? Also, it would be very tempting to add lots of metadata, which would drive one to XML. I'm not a huge fan of XML, mostly due to lack of TUI editors for it... but XML does have the advantage of XSLT/XPath.
It seems to me that this is a problem that can be solved reasonably well, and that this is a problem worth solving. If we can solve it in a way that works well with TUIs, it will also work well with email, so this should be of interest to you (I hope so as you seem to have the time and energy to work on this!).
Thoughts?
I'm the author of emailthreads, mentionned in the article. I believe the library is already able to do something pretty close to what you want. See for example this output: https://github.com/emersion/python-emailthreads/blob/master/... (you can find source messages in the parent directory)
The goal is to eventually build a web UI around this library.
I've looked at the examples you linked, and... I like that the review format for review authoring is just plain trivial. Not sure how it works with quote trimming... I'm not sure I like the output format, but I'll go think about it.
Yeah, this library just parses email threads and the goal isn't to define a new format. This particular output format is just for debugging and unit testing, it's not meant to e actually used as is. The library user just gets a tree of text fragments with metadata, it's up to him how the output will look like.
https://pypi.org/project/mail-parser/
Since there isn't any documentation, I'm just trying to figure out what's it's actually doing (without reading the source).
The key thing about any tool that's trying to improve an email-based workflow is that it has to interoperate with people still using the traditional "just an email client" tools. Better web UIs for people who want to use web UIs are something I'd love to see, but a big-bang switchover to a completely different format won't work for large projects, where mandating changes of tooling is a really hard sell. All the three tools I list above take the "interoperate with the existing patches-in-email transport" approach.
The real challenge is that the GUI MUAs often make it hard for users to use even a bit of convention in plain text... I'm thinking of Outlook in particular, but it's not the only one. Even with an MUA that does angle-bracket quoting the right way and encourages users to trim quotes and bottom-post, the fact is that editing plain text is difficult and boring for most non-vim/emacs users out there, and that is most users. This problem can't be solved by the author of TFA, but if the author of TFA solves the problems that are in their bailiwick, then that could add pressure on MUA developers.
The chores of this approach - finding and collating feedback, sheparding a patch through the flow, and so on - are all things I'm acutely aware of. I don't think we need to throw the baby out with the bathwater like GitHub does, though, and I see these as places to improve our tools rather than replace them.
Take a look at this here:
https://git.sr.ht/~emersion/python-emailthreads/tree/test/da...
The specific output format you see here is just a testing tool, this isn't the final user-facing product. The important part is ingesting a bunch of emails and creating a thread where you can associate all of the responses over time with the particular parts of the patch. Combining this with a gerrit-like tool which tracks new patches over time and avoids duplicating review work is planned.
Another thing is that with an approach to email code review that solves the collation problem you might... be able to seamlessly bridge email and web interfaces. For example, GitHub does actually have an email interface -- a very poor interface (markdown won't be interpreted, yo wth GH?) and only for issues, not code review. But your service could provide a superior email interface to code review... that also plays well with the web. I'm glad you're already thinking of this.
(GH doesn't care about email integration because with a REST API and BUI they can cover all their users' needs (except email-centrism!) without having to deal with the challenge of email issues (like this collation issue). GH has chosen to not bother with email, while you're taking up the challenge. As a lover of email, I say good for you (and for the rest of us)!)
Again it's tempting to think of XML. If you want to convert to a plain text format, XSLT can do it. And if you want to convert it to HTML, XSLT can do it. Or you can fetch the XML with XHR. Or convert it to JSON for fetching with XHR... again, XSLT can do it (well, I've never written XSLT code to produce JSON, and I'm guessing doing it right isn't easy, but maybe newer XSLT iterations have support for JSON? idk, I've not followed it). If you use JSON you could use jq, natch, but JSON is a poor format for what is really document-like data, so I'd stay away from JSON. Even if you choose not to use XML (I wouldn't blame you! XML is for masochists), thinking about this problem with XML in mind might help you design a non-XML format that covers all the things you need to cover.
I'm not just a TUI power user. I'm also a jq and XSLT power user, meaning that APIs (meaning, nowadays, "RESTful" APIs) are very convenient. Structured document formats add a lot of value, but in an email context you just can't access that value due to lack of a) a notion of "email API", b) MUA integration with editors for different MIME types, c) a total dearth of editors for those formats lying in between vim on the TUI side and Word on the GUI side. In this particular context you have the benefit that you can consumer plain text and extract from it commentary due to the traditional email angle-bracket quoting style, then merge that content into a structured document from which you can render plain text again if desired.
Anyways, really, best of luck. If you want to exchange further thoughts off HN, I'm sure you can find my email address.
The inputs are plaintext replies (or HTML formatted as plaintext).
The original document is immutable.
There is no state (because it can be collated from scratch every time). EDIT: If updates are allowed in any non-email based way (e.g., via a BUI, REST APIs) then email must be generated and sent so that all the state is in the email archives and all participants can collate the same state.
But the result of the collation is more like a database and less like a document. In particular there's not much need for marking up commentary.
Multiple collation output formats will be needed: plain text, XML, XHTML, JSON. These formats/schemas are only needed for display purposes, not interchange, as the collation results can be computed independently by all who have access to the emails. This means there is nothing to standardize!
Thus collation state can be kept any way you want. XML, SQLite3, some NoSQL, any in-memory representation you like -- whatever suits you. Still, I'd use anything that has a nice, declarative query language. XML (XSLT/XPath), JSON (JSONPath, jq), or an RDBMS (SQL) -- all will do. For you, and given your emphasis on open source, SQLite3 or PostgreSQL will work best, I think. You can generate all of the output formats you need from such a database.
True enough. Not repeat that analysis in 17 years. Archives of LKML go back to the list's creation over two decades ago. How long do you expect your github page to last before the format changes or the project gets dropped or pickled?
That's a very backwards attitude that will turn away people.
HTML in e-mails is a useful tool. I encourage it in the dev mailing lists that I operate.
There is no conflict between HTML use and all the various dev-related use cases like posting patches or pull requests or whatever.
In a HTML message, you can have an indented code block which is not physically indented. When it is copied and pasted, it has no indent.
Hyperlinks and images can be used in HTML messages, which is convenient.
Color is possible: syntax colored code can be incorporated into an HTML e-mail.
Also, I use an mailing list archiver called Lurker, which I customized such that it allows HTML messages to be incorporated into the archive. So poor archiving of HTML mails isn't any objection.
Bottom posting is so useful that even Outlook users reinvent it, badly of course, using different text colours etc.
It's been decades now, why have none of the advocates of HTML emails fixed this really basic problem?
[0] https://lwn.net/Articles/702177/
[1] https://news.ycombinator.com/item?id=15370620
[2] https://www.reddit.com/r/programming/comments/73gpys/why_lin...
> Git was created by Linus Torvalds in 2005 for development of the Linux kernel, with other kernel developers contributing to its initial development.
Also, Drew doesn't really delve into what makes email clients "bad." Email is so much more accessible than it used to be thanks to Outlook, Hotmail, Gmail and the like. HTML allows people and companies to be creative/abusive with their email. Plain text is quite restrictive.
Which is a good thing, most of the time.
- merge/collate commentary (from many emails, say)
- display source with collapsed/collapsable commentary
- track commentary accepted/rejected/extant/addressed status
- track commentary metadata (who, when)
I mean, I use GUI/web-y code review tools because I find them much easier to use than email, but only because the problem they solve is: keeping track of all commentary. Collating comments (and status) from many emails is a time-consuming chore -- I hate it, and it's the only reason I don't want to do code review over email, but it's the only reason I have for not wanting to do code review over email. If we can solve this problem, then I'd be ecstatic to do code review over email.
You need look no further than the versionX->versionX+1 logs some of the kernel developers do to understand the problem. From one patch set to another the conversation around a piece of code gets lost, so much so that for long running patch series people will frequently show up and make the same request someone earlier in the series made because they missed the 20 emails talking about the pros/cons of doing something a certain way 6 months earlier.
(disclosure: I'm an occasional Patchwork developer)
Nothing else syncs to my phone in a manner that "just works" and doesn't require excessive fiddling, or using k9mail which appears to quite simply not work at all.
So with that said, how does a person use git via email with the Gmail ui?
* for sending patches, use git-send-email from the command line
* for reading, use a greasemonkey script to make gmail use a fixed width font
* when I need to apply a patchset locally I use the "patches" tool https://github.com/stefanha/patches
I also have a couple of gmail "canned responses" set up for things like "write my reviewed-by tag into my email reply".
It's not fantastically ergonomic, but it's good enough that I haven't really felt the need to use something other than gmail web interface.
Generate an app-specific token in Gmail and use it to configure git for sending mail. For receiving and commenting on patches, you might be able beat the Gmail interface into submission but it's not going to be convenient, if only for the browser restrictions on calling external commands (like git). Maybe with a browser extension.
In any case, you can use a real mail client in parallel to Gmail (the UI) and use whatever is the most suitable tool for the task at hand.
Before I switched to iOS devices, I used k9mail as I was looking for an email client that did "push" IMAP as I've self-hosted forever and a day. From that experience I'd say that k9mail is more a symptom of the Android ecosystem than an indication that Gmail is just that good. k9 worked for me, but was finnicky if I remember.
On iOS IMAP is very much a second-class citizen. But... with the stock mail client IMAP has been more reliable than Exchange.
> So with that said, how does a person use git via email with the Gmail ui?
Unsure, but if you can you can definitely use Gmail with other clients (for the time being).
k9mail is a dumpster fire in my experience.
$ git config format.subjectprefix 'PATCH foobar'While I like git in general, this particular argument's logic is completely backwards. If something works well for 15 million lines of code and thousands of developers, then most likely it will be a gigantic overkill for your small project with a team of four, and overhead that goes unnoticed at Linux's scale will be crushing for you.
At least, that's what I would think if I didn't know git first-hand. The fact that git turned out to be useful for small project is very atypical for a solution born out of needs of a huge one.
Regardless of whether it was a good idea or not, this ship sailed many years ago. In the 21st century, email is rendered in HTML, and nobody but a few engineers would even contemplate taking away the ability to render links, formatting and images in email.
Personally, I would argue that email is on its way out. Personal communications have largely transitioned to things like SMS, Snapchat, Facebook Messenger, iMessage, Twitter, etc. Transactional email notices are increasingly being supplanted by mobile push notifications. Businesses are making heavy use of hybrid push-pull channel systems like Slack.
IMO, the decentralized, federated nature of email was its downfall. It became a cesspool of spams, scams and unwanted notices. The research I've seen strongly suggests that it only takes a small amount of unwanted noise messages for people to go longer between reading email and reduce the amount of email they send.
Of course these are problems that can be solved, but my prediction is that they will be solved by email's successors. Of course, technology predictions are hardly worth the bandwidth that delivers them, but that doesn't stop people from making them. ;-)
git is a tool for engineers. I could reiterate this comment for most of your points. We're not here to debate plaintext email for end-users, we're engineers talking about engineering tools.
Sadly, centralized web services like GitHub considerably lower the effort needed to do that first contribution.
Also, incidentally to your point, it's a lot easier to control spam within a company email system. As a broad brush, you can just set your client to automatically relegate any email from outside your organization to another folder.
Email clients, like Thunderbird, do can render links and formatting in plain text emails
Tl;DR? My problem with code review over email is that keeping track of all commentary and metadata is difficult when reviewing over email. This seems like a solvable problem with a text format for interleaving source and commentary that is suitable for post-processing, particularly one that makes merging commentary easy, and preferably too that makes changing commentary metadata (e.g., comment accepted/deferred/rejected/addressed status) possible while retaining the ability to merge commentary.
https://blog.ffwll.ch/2017/08/github-why-cant-host-the-kerne...
Tell that to any big team in a big company or even a medium-sized company using a monorepo. The Linux project has a very high bar for entry and it's not an example of a busy repo.
Can you back up your statements with evidence? I have no reason to believe that email wouldn't work in a large monorepo-like setting. The less-than-10-in-total companies which have crazy Google-tier monorepos may run into issues with this (honestly, though, I doubt it), but they are not the norm.
Most projects are more accessible to beginners, and GitHub makes it much easier for them to do their first contribution to an open source project.
Do you mean the protocols, the clients, the servers, or something else?
Your email reader is supposed to help you read and reply to message threads.
> not being able to browse code
You browse code by doing a git clone/checkout, and then using your regular programming tools.
> no convenient links
Your email client should help you navigate and search message threads. Here is one of many that helps you do this: http://www.djcbsoftware.nl/code/mu/mu4e.html
> no formatting
This is what you editor is supposed to do.
> no branch visualizations
This is what tools like magit are supposed to do: https://magit.vc/
> release notes being easily accessible
They go in a text file in the repository. You open the text file with your editor. Your editor should also help you write release notes: https://www.gnu.org/software/emacs/manual/html_node/emacs/Ch...
> having to set up a bunch of fragile text based filters.
Why?
I see arguments like yours about email and git a lot. Why do you think people disagree with you? You have to pay attention to what you are really saying. You are not listing advantages of web-based git interfaces. You are listing "disadvantages" of existing tools. And all of the "disadvantages" you list are really areas where you do not understand how to use Unix programming tools effectively. Getting off of webmail and onto a good text-based email client is one of the best things you can do as a computer user.
I am not stating an opinion. I am making a judgement. I can do that because I have the requisite knowledge and experience. You do not - as you yourself stated, you "don't want to spend time configuring or fixing [your] email solution." (another tell-tale sign is referring to "the terminal.") Stop trying to make virtue out of your choice to be ignorant. Stating that most people will prefer GUIs is not an argument for anything ("X is popular" is never a good argument for anything) - most people have not had any exposure to alternatives like text-based tools and therefore cannot have any meaningful or informed opinion on the matter, which you pretend to do. Your choice to be ignorant of something does not entitle you to make judgement calls about the things you are ignorant about. You can say "X is difficult to learn/do" (true), but when you argue that "X is an inferior tool" with someone that has some expertise in the subject and in X, you just come across as a fool.
I can see how it would be inconvenient for people who use an inflexible email client or editor, though. (I'm trying to make this workflow a little less intimidating for Guix by working on a more convenient web interface on top of debbugs.)