Welcome Yari: MDN Web Docs has a new platform
hacks.mozilla.org
hacks.mozilla.org
I've said it before, but I think this move was a mistake. They've thrown away the benefits of a incredible wiki-based platform (where changes were pretty much instantaneous and any of us could easily make a difference to the docs!), the (actually pretty decent) WYSIWYG editor and an overall frictionless editing experience and replaced it with what, some (not even Markdown) document files in a GitHub repository. I honestly believe this will dissuade people from making small, quick fixes and ultimately drive away contributors.
When I last brought this up, I was told that there were a number of contributors put off by the wiki-based nature of the previous iteration and they will now be excited to be able to contribute using Git.
I'd honestly be interested to hear from any of these people. Given you could seamlessly login with your GitHub account, easily author your documentation and one-click preview your changes, what exactly are you gaining from having the editor gone and now requiring a local docs environment to do all of the preview/edit/commit steps manually?
Compare: https://web.archive.org/web/20200113175409/https://developer...
To: https://github.com/mdn/content/blob/main/files/en-us/web/sec...
Where has all the content gone? And of course there's no history anymore due to how the content has been migrated.
Isn't your entire article here? The history is there...
Also, at the footer of every document is a link to `$0/contributors.txt`. Your attribution is there: https://developer.mozilla.org/en-US/docs/Web/Security/Certif...
There will be a record of every contributor who submitted a revision to the non-archived content in all locales.
Btw: now you need to clone the entire repository and run a dev server just to edit and preview a single page (let’s not get into super involved ways to clone only part of a git repo). You can avoid cloning by editing with GitHub’s web editor, but you can’t preview that way.
They could've cut it down to e.g. the last n changes to a page if this was a concern.
Honestly the whole migration feels like it has happened before things had been properly thought through. There are pages lying around with {{IncludeSubnav{"")}} directives in the text that don't render properly - and they put this into production.
Lets be real: internet points in my github profile contribution heatmap
Kidding aside tho, as a first-time contributor back then the non-markdown editor was clunky for me, the spaces the highlights were just un-natty. You're right about the non-instantaneous fix visibility tho, but for extra QA maybe it's a fair trade-off.
I don't know if I hit an exception, but I added a small change (a single clarifying example) to the RegEx page, which was listed on the dev environment's history for ages before actually becoming active.
Also keep in mind that, given MDN's target audience, especially the target contributors, GitHub is less of a barrier than it would be for other platforms.
But yes, of course it'd still be good to keep an eye on the contribution statistics to validate those assumptions.
Ideally, you want to cater to both the CLI centric workflows of developers/build-systems and the WebUI centric workflows of everyone else (the old system). The Mozilla blog posts mention the JAMStack so I'm guessing that the intent was to port the server-side Django apps/editors to a browser-side framework like React or one of its alternatives.
If the design and implementation of the new platform was perfectly executed then only the new functionality would be apparent; both systems are essentially "wiki-based". Platform transitions are rarely seamless as your lost content/metadata clearly shows.
We are currently doing the same thing at Our World in Data and discussing this exactly so would be great to learn more about the design decisions.
This is not to say there aren't advantages to the git-based workflow and it might be a valid choice for a technical audience like this, but to claim it as "huge advantage in terms of contribution workflow" seems questionable.
2. GitHub and GitLab are a lot better nowadays at allowing you to edit markdown docs via the web and preview the results in a few clicks. Not quite as frictionless as 1 click, however if you are an editor and are constantly bombarded by spam, probably saves a lot of time as you know the person who is making the change and they are forced to add an explanation via commit message.
2. Yeah, it's really really convenient to submit a quick PR entirely in the GitHub Web UI. But I admit, we have some work to do make the previewing experience a bit better.
I'm only a sample of 1, but I once tried to edit a page and just couldn't find out how to do it. So I'm not sure their change will be any worse.
Yeah, I really agree with this. Having built a website which relied on people submitting PRs to add their content on GitHub - I wouldn't do it again. It's a lot of friction for both the maintainers and people contributing.
Anecdotally, for the project where I contribute most... the issue about fewer contributors is real. But you do still get contributors, and PR-workflow makes other aspects easier (like broad clean-ups/reorgs/scheduling/versioning). For contributors who come, you also get the opportunity to socialize/engage them during review. Overall, there are drawbacks/risks, but I'd say it was a net-improvement (wrt quality/clarity of the final docs).
Maybe look at it this way: Both workflows provide a way to organize open/community docs. Both workflows have positive role-models. In both, you need capacity+interest for editing, for socialization, etc. If you tend to these, you can do good. But if there's neglect... then that's where you'll see the starkest differences:
* In wiki-workflow, the likely symptoms of neglect are draft-quality content, bad prose, quirky TOC, drive-by edits that are out-of-place, etc.
* In PR-workflow, the likely symptoms of neglect are slow review/feedback, older content, would-be contributors who can't assimilate to the workflow, etc.
The problem was that our documentation was bit rotting in the wiki and the previous mediaWiki wasn't working so well for having consistent formatting across pages, reviewing changes and organizing the content.
Using the new Hugo based software, we now have a few very helpful macros to add links to Doxygen based documentation, include snippets of code from the library repository and automatic generation of navbars and the navigation.
And we also get with Gitlab web editor an easy way for people to contribute. It is still like before one click away to edit the content.
I probably should blog about it one day, but if someone want to check it out: https://invent.kde.org/documentation/develop-kde-org.
I suppose if the actual source file changes then the page would be updated as well which is neat, but if you're specifying certain lines to read from then that could easily become a problem.
Considering how few shortcodes you're using I just felt like there must be a good reason to use it in this way and I was curious why.
More interesting are the doxysnippet shortcode[1] that provides the extraction from the source code or the custom rendering of links[2] that allows this sort of markdown links: [Overlay](docs:kirigami2;OverlayDrawer)
[0]: https://invent.kde.org/documentation/develop-kde-org/-/archi...
[1]: https://invent.kde.org/documentation/develop-kde-org/-/blob/...
[2]: https://invent.kde.org/documentation/develop-kde-org/-/blob/...
Since 2017 I've moved all CMS I am responsible for to Git from MySQL and haven't looked back. You can still preserve any frontend experiences you want, but get to drop a huge dependency, a lot of code, and gain a ton of new capabilities (faster syncing, easier backups, forks, patches, full audit trails, etc). You never need to worry about migrations again or build your own change control logic.
But my efforts are finally paying off and we saw an increase of 33% in the visitor count of the website and the current codebase made it so much easier to work with and add new content. And this time, I blogged about it: https://carlschwan.eu/2020/10/30/kde-org-hugo.html
When I click "Edit this page" I am asked to sign in. Is there a reason for that? I'd expect to see the source and be able to submit a PR as a normal repo on Gitlab.
I now added a direct link to the source code too.
Not a big fan of the logo, it is cool, but doesn't really inspire my inner web documentation. (edit) Actually no, I think its the size and style of the logo. Singling out the top of the spear and using that would be cool, but it reminds me too much of a fighting game character as is.
I think this is why a lot of sites take markdown, then add their own extensions, like how there is "Github Markdown" among many other flavors. That's definitely one route, but I see something like ReStructuredText or Asciidoc as more mature and interoperable, while still being relatively easy to master in the same way as Markdown. Since they can both produce docbook output, vastly easing any migrations in the future by adhering to an industry standard.
with this arrangement, the character is leaning away from and pointing away from the words, and scaling down the character has made it difficult to make out the details of their pose (is that a hand? is the face empty? etc.).
The previous backend was codenamed "Kuma" and was represented with a bear, but wasn't displayed anywhere on the actual MDN platform.
Yari is a codename for the effort to move MDN closer to a static site architecture.
Having said that, https://docs.microsoft.com 's flavor of Markdown does allow for embedded HTML, like GitHub's and enough pages still use that feature that the conversion to Markdown is arguably incomplete. It isn't a big issue in practice, however; you can update markup from HTML to Markdown along with your other changes.
* As a recent-past engineer on the Windows team, I have a rather lower opinion of Windows API docs than you. :) The Windows developer platform has not had enough dedicated technical writers for years; our developer content teams are mainly editors of engineer- and PM-written original docs, which can lead to API doc sets with badly written pages, important missing information, or references to Windows-internal developer tools. I tried to channel my frustrations into correcting and extending my coworkers' writings, or into gently asking them to fix their omissions when I didn't have the free time to spend on the needed research.
Other things that are critical features for development documentation, like tables, are extensions which may or may not completely break if you ever change your Markdown renderer.
If you need more typesetting control than what Markdown allows, dropping down to HTML is always an option.
To the second point, while it's possible that content could render differently if you change renderers, I would presume the folks at Mozilla are aware of this and won't do it unless it's absolutely necessary.
Do we actually know which renderer MDN is using? I don't see this mentioned in the post. I would argue that although alternate implementations exist, Gruber's `markdown.pl` is Markdown (whatever it does, warts and all) and deviating from it is generally a bad idea if you can avoid it.
Is it? In the public setting? HTML comes with things missing from Markdown that you want, sure, but it also comes with a full scripting environment, the ability to arbitrarily inject code and so on.
If you can drop to HTML, you have to have a sanitiser process. So then you're dropping to some-unknown-subset of HTML if the process is automatic - or you've failed to reduce the amount of effort being put on the editors if it's a human one.
For a place like MDN, this really isn't a problem.
> If you can drop to HTML, you have to have a sanitiser process. So then you're dropping to some-unknown-subset of HTML if the process is automatic - or you've failed to reduce the amount of effort being put on the editors if it's a human one.
I presume Kuma had - and Yari will have, if it doesn't already - some way to prevent unsafe injection and other dangerous things.
This isn't to say that the issue doesn't exist. Instead, I would expect it to be the exception rather than the rule. After all, Markdown and its derivatives owe their existence to this very phenomenon.
The language of last resort, if you will.
Any form of structured information, for example API functions and their argument types, properties of a class, or tables of historical compatibility, is already stepping outside Markdown.
You're left with either writing everything out in text and having error-prone parsers cross-check pages against each other, or badly cross-referenced information that goes undetected, or using something that is Markdown plus additional markup for semantic data.
This is where is becomes useful to have tools that can generalize things. For example, say you link to another page in your documentation repository, in Markdown, you create a link either with an absolute or relative path, specifying the filename and optionally anchor on that page. Now later down the road, you edit the folder hierarchy, or rename a page, you will now need to find all references and update them manually.
Something like ReStructuredText has the "interlink" module, which allows you to modularize your pages, and use symbolic names instead of the relative or absolute path. Now, there are pros and cons to each approach, i.e. if you have a good set of tools, doing a global search and replace across documents can deal with this too. But having the flexibility of things like symbolic names and macros can make things much more manageable.
Of course, this is a double edged sword, in that you can customize, create macros, until you now have a monster in of itself, but that can be said of any tool.
I tend to see Markdown as perfect for standalone documents, and its especially good for formatting internet comments and the likes.
Tools like DocBook and other XML processors attempt to provide the maximum amount flexibility, and the cost of a steep learning cliff and lots of boilerplate, but if implemented well, it can allow things like conditionally including parts of documentation based on tags, or output formats, but definitely requires extensive tooling as opposed to Markdown and other formats that are meant to be readable in their source form.
I'd suggest they start with a toy I made a while back, Dumbdown as a base, and evolve from there: https://jtree.treenotation.org/designer/#standard%20dumbdown
- how is it acceptable to run off github for a project that wants to encourage authoring sites on a supposedly federated medium (the web)?
- do we really need an enormous "web documentation project" for something entirely man-made, and which once set out to facilitate easy self-publishing?
Even thinking about these and similar questions means that, rather than attempting to document the craptastic overcomplicated web, we're probably better off to leave the web behind us for good, and start to concentrate on defining HTML+CSS subsets (such as for purely static docs, for light content apps, and so on), to distribute simple static text via alternate p2p protocols. W3C and WHATWG have failed miserably to do so.
You may have a point here in reality, but "technically" it should be very easy to push to multiple backends (GitLab, for example). So I could see lock-in to GitHub not ever being more than a hypothetical problem.
What point are you trying to make? Should Oracle throw out its docs for Java because it's a large language that's man-made and designed for easily writing software—but lots of folks here think it's not good?
> start to concentrate on defining HTML+CSS subsets (such as for purely static docs, for light content apps, and so on), to distribute simple static text via alternate p2p protocols
And this will still require docs, and tutorials, and information about the P2P protocol, and how it's kept secure and ~anonymous, and how to facilitate discovery, and how to avoid a network partition, and how to build non-static tools like search engines, and how to securely interact with those tools, and how to do archival work, and a million other things that make that system an ecosystem...it's not easy to build a resilient distributed network that actually works.
The fact of the matter is that people use the web whether you think it's good or not, and having decent docs for it is important. Nothing is stopping you or anyone else from building an alternate web, but so far there hasn't been anything that people actually care to use.
That maybe our time is better spent by taking the chance to simplify the languages of the web along use cases rather than document the whole of it in an encyclopedia. The Java example is a good one, since IME tools such as javadoc, and just writing down the purpose of something and why and when it was introduced, immensely helps cleaning up your API surface. The web as it is is way too complicated when it really doesn't have to be, as witnessed by browsers vanishing.
In the end we did use Git for documentation (pressure from devs), and in practice the non-devs tried to stay involved but ended up letting "technical" people do all the Git and diffing stuff, so it remained a practical barrier to involvement in the documents.
People who are used to Git and development in general tend to forget that a lot of tools we take for granted, including text editors, terminals and command lines, are completely alien to non-developers and not everyone wants to learn that stuff for just one project.
I've known developers struggle with Git too (when they don't use it often, or it's only used for 1 out of 10 of their projects), so I agree it can be a fairly high barrier.
Anything closer to having to make a manual Github pull-request would be incredibly unergonomic for someone without an existing CLI setup for Github (ie. most developers).
The big upshot is that all the hard problems like merges and history/diffs and whatnot are solved for you by the VCS. I don't think MediaWiki allows you to "blame" anything for example; something I've often wished for. It's just a whole class of problems you don't have to think about.
Also since it's based on markdown, would hosting MDN locally be possible now?
The raw content is not Markdown. It's HTML with some macros (called kumascript macros).
And yes, you can "host MDN locally" now. Before you had to write a web scraper, now you can just iterate over the files after a `git clone`. But you might need Yari to build the raw HTML to fully formed HTML that you can open in your browser.
Maybe this bold new world is better. Maybe I'm just old and brittle. Or maybe we really have lost something that was good.
No. MDN is using a Microsoft platform as a front-end to attract contributors to their git repo, as most open-source projects do. If a company uses Gmail, does Google control the company? Github is a contractor for ICE, does Microsoft control US immigration enforcement?
I don’t like Microsoft owning Github but there is a broad, easily-seen line between reasonable objection and this absurd overstatement.
If the only way to interact with coworkers, customers, investors, etc. is through Gmail, then, well. yes. If you can't be an employee without a Google login, then, ummm, yes. If Google can suspend your account and lock you out of your workplace, then yes. If a Google outage means you can't work, then yes. If Google loses some email and it means you can't continue a product launch... If a Google breach means that your company has been compromised...
On the otherhand, if Github is just one avenue for contribution, well ok. Is this mirrored on Gitlab or git.mozilla.org or somewhere else that people can contribute without a Microsoft account? If Github fails is MDN just a flip-of-a-switch or less away from routing around it?
Edit: typo
I have been migrating all my projects to PostgreSQL because of personal believes (most of the time) and haven’t had any issues so I’m just wondering on the thought process.
I’m guess the downvotes is the community just getting a bit toxic on DBs.