Building a Text Editor for a Digital-First Newsroom
open.nytimes.com
open.nytimes.com
It was truly an awakening to the hidden wonders of the world.
https://www.gnu.org/software/emacs/manual/html_node/elisp/Te...
(Also I think - from occasional messing with XML in .docx files - but can't be sure that Word flattens styling like that as well.)
I prefer vim though..
Maybe the author has overcome these issues, but I'd be inclined to go with a different editor. Also, ProseMirror's architecture is extremely close to DraftJS, which I've had much fewer issues with.
I also don't think the concept of fairness applies since I'm accounting the details of my personal experience and the conclusions I personally drew from it. Some of those are subjective (code quality) while others are purely objective (I got a lot of errors). What I infer from that may be entirely different from what you do, but that does not make your characterization of things more fair than mine.
The documentation is still no good. The underlying tech looks super promising, but the plugin system makes no sense.
If all they did was update their documentation so you can clearly see how to use it, perhaps it would be great.
Just wanted to offer some feedback. Prosemirror's docs are woefully incomplete. The way controls are added is extremely confusing. Why do we need some node module called "prose-mirror-example-setup" ?
I know I'd personally prefer being able to add all the markup and style for the editor, and then just use prosemirror underneath the hood for state management.
Another thing that I found quite difficult, was programmatically inserting images at the cursor.
Finally, storing state as markdown has turned out to be a nightmare. Not really your fault, but perhaps something you should be aware of.
We then spent 2-3 weeks getting all of the formatting, tooltips and key mappings to handle some basic formatting (bold, italics, underline, bullet lists, numbers). Eventually, once there are more examples on the web of more complex formatting for the community I think it will become much easier to use.
It's the first editor I've used that felt like it made sense at a conceptual level but implementing "simple features" can feel very hard at times. I don't think that's a knock on the library though. There's a lot to learn and understand and it just took us a lot of time to get it all.
Splitting editors into an abstracted structure for finer control over data and presentation is something that's been done for a decade now in various text editors. Medium and Basecamp started similar projects and there's Mobiledoc which is trying to standardize the format. It'd be nice if all this effort focused on a single project so we could all just get a decent editor instead of different versions with various states of functionality.
FYI, my running list of editors: https://gist.github.com/manigandham/65543a0bc2bf7006a487
This is talking about using ProseMirror to build some more features into their CMS, but it sounds like more of just a use of ProseMirror rather than a new editor entirely.
Quill vs (CKEditor and TinyMCE, Draft, ProseMirror, Trix): https://quilljs.com/guides/comparison-with-other-rich-text-e...
Slate vs (Draft, ProseMirror, Quill): https://github.com/ianstormtaylor/slate#why
I personally went with Slate because I needed first-class React support but not as low level as Draft.
Have you done anything with collaborative editing with Slate? If so, how are you finding it?
Your document state is always represented by a list of deltas. Two people trying to change the same document would just have their individual list of ops "rebased" together using OT before being concatenated into the existing list of ops, forming a new document that is still just a list of ops.
Sounds like Latex which has been around much longer.
https://github.com/the-grid/ed
Built on top of Prosemirror.
On projects where I've had to load up multiple instances, I've had to move away form it as it kept reloading stylesheets and js files for each editor on the page.
Now I usually use quilljs, which works really well for some basic markup.
Personally tried both, working with Slate feels much smoother, plugin system easier than prosemirror in my opinion.
And, there is ckeditor5-beta. You might wanna check out.
I think to fully experience all three of those editors, you should build the following: something that generates a rich text input using the editor, updates a data store (can be just in-memory if you want), and then re-presents the rich text in a read-only format.
With Quill in particular, there's two ways to approach the second part - you could dump the data into another instance of the quill editor, but one flagged as read-only, and let quill handle rendering out all the elements. I don't do that, myself, as it's a bit heavy and relies on the DOM to be present to create the end result (meaning rendering rich text into emails on the server side can be tricky). Instead I use a library (that I wrote) that converts the quill representation into an HTML string that can be later injected into whatever place it needs to go.
In general, I think that's where the one of the important pain points is for any major modern text editor exist - converting the internal data representation into a canonicalized HTML string. This gets further complicated by your framework - it's easy enough to inject an HTML string, and angular 1 lets you even compile that so you can have custom elements there, but angular 2 has no such facility (compiling; you can still inject). So rendering custom elements into angular 2 is feasible but quite tricky.
You'll need to figure out how to handle custom elements too, like at-mentions and so on; those are really domain-specific. How I handle at-mentions in quill (with a pop-up autocomplete) depends on the structure of the overall system as well as the framework I'm in - angular 1, 2 and react all suggest different approaches for this.
Anyway, this is all a long response that means "it depends". Each editor has its own strengths and weaknesses that are exposed by the specific data and presentation needs of your project and the other frameworks in play. But building an atmention or other custom element and converting from the internal data structure to HTML is a good test of functions for all of those.
I carry the personal view that when someone says "text editor" they are implicitly implying plain text. As such the leading paragraph of the article makes me twinge a little when "text editor" is used rather than "word processor".
The article then continues on to describe the editor which sounds very much like a WYSIWYG editor. Indeed ProseMirror[1] homepage says the following.
ProseMirror tries to bridge the gap between Markdown text editing and
classical WYSIWYG editors.
It does this by implementing a WYSIWYG-style editing interface for
documents more constrained and structured than plain HTML
So whilst I've made a point of it reading oddly (and I would have thought this would also be the view of most others on HN) I am not trying to disparage the authors use of the term "text editor". But rather I am curious as to why such nomenclature happened to be used here as I seldom see it outside of descriptions of things such as Vim and Emacs.FWIW ProseMirror does not bill itself as a text editor:
> An ideal content editor ... bridg[ing] the gap between Markdown text editing and classical WYSIWYG editors.
> It does this by implementing a WYSIWYG-style editing interface for documents more constrained and structured than plain HTML...
(At least that's how I understand text editors; maybe I'm the ignorant one.)
If you’re like most people in America, you use a text editor nearly every day. Whether it’s your basic Apple Notes, or something more advanced like Google Docs, Microsoft Word or Medium, our text editors allow us to record and render our important thoughts and information, enabling us to tell stories in the most engaging ways.
Also, it could be argued that all word processors are also text editors but with added features. The fact that they allow the presentation to be edited does not mean that they stop allowing text to be edited.Viewing the header illustration for more than a few seconds crashes my iPad Safari tab. Scrolling until it is off the viewport prevents this.
EDIT: Dev Tools reports 74.8 MB transferred with an ad blocker. The illustration is a single 69.2 MB gif.
EDIT: 101 MB in total when you scroll to the bottom. Three more gifs: 18.1 MB, 7.1 MB, 4.7 MB.
Why the biggest newspaper in America can't run their own hosted blog is a bigger question.
(Note: I'm not dissing them, I know a very talented developer who works there and says he has a great team)
Which is bizarre. The comments section of tech blogs are a great place to discuss and ask questions, and the support for comments on Medium is abismal, looks like they deliberately made it suck.
https://open.nytimes.com/introducing-the-new-open-blog-23eba...
Whether that actually works is up for debate...
https://itunes.apple.com/us/app/unobstruct/id1255281426?mt=8
> Why the biggest newspaper in America can't run their own hosted blog is a bigger question.
Kind of ironic given that the post is about building their custom web-based text editor... I wonder if they use it to draft their posts for Medium too.
/joke
There are several reasons for why the GIF is so large that all contribute to the size. The GIF's resolution is quite large for a GIF, at 1004px by 788px, this is larger then they display it on the page. And the GIF is pretty long at 336 frames.
Since GIFs on like just about every video format except MJPEG has no interframe compression. So each frame only has to average 32 KB to make up that 10 MB (336 * 32.6KB = 10.7 MB).
This is an example of how extremely good our modern video codecs are.
https://www.instapaper.com/text?u=https%3A%2F%2Fopen.nytimes...
(This may require a free Instapaper account to view.)
BBC Studios is the commercial arm of the BBC for licencing and distributing media outside of the United Kingdom.
I hope they manage to find the right combination to stay around, I loathe the dystopian future where Buzzfeed and Breitbart are the only news outlets.
In any event, what I meant was that I don't want a future where we only have sources of "news" that serve to reinforce our pre-existing beliefs, with everybody siloed into their own particular little bubbles.
Their stock is at an 11 year high. Sales are growing again. Profitability looks set to surpass where they were several years ago. If it weren't for some stray costs & adjustments in Q4, they might have hit $100m+ in profit in 2017, the highest since 2010. Their balance sheet is solid, long-term debt is modest ($250m), and they have plenty of cash ($490m). Overall, it should be a good decade coming up for the NYT.
They're a broadcaster.
Kind of like how all of the best programmers have some kind of log or blog that they post in. While this could very well be some form of survivor bias, I think that just having to put effort into articulating your thoughts in a coherent way now and then improves other parts of your thinking.
I worked on the Graun’s website/publishing backend in the late 90’s. Lots of Tcl/Tk.
For backend specifically, on the CMS team there's this job https://nytimes.wd5.myworkdayjobs.com/en-US/Tech/job/New-Yor... or a fun one with the cooking team https://nytimes.wd5.myworkdayjobs.com/en-US/Tech/job/New-Yor...
https://nytimes.wd5.myworkdayjobs.com/en-US/Tech/job/New-Yor...
https://nytimes.wd5.myworkdayjobs.com/en-US/Tech/job/New-Yor...
https://nytimes.wd5.myworkdayjobs.com/en-US/Tech/job/New-Yor...
There’s a lot of value in building tools that fit the domain and workflow perfectly well. It’s expensive in the short term but in the long term saves a lot of time and money.
It takes a 'team' to provide any sort of professional level of support for an organization of their size. If the one-person "CMS/tech wizard" decides to stop working, or die, or get sick, or take an extra 4 hours to be with their family, the entirety of the NYT operations should just say "oh well, we'll just wait around for a while until we find our next one person to be our tech saviour"?
Absolutely, which is why I didn't stop at "I am a developer". That "who is good at Python" bit is definitely important.
However, seeing yourself as a "Python developer" rather than "a developer who is good at Python" makes it look like you see every problem as something you'll solve using Python. That's bad. That will put a lot of companies off hiring you (companies that believe in a 'best tool for the job' approach anyway, and those are the ones you probably want to work for).
If a company didn't hire me because they idiotically interpreted "I am a python developer" to mean "I would try to build a rendering engine in python" then I'd say that's probably a bullet dodged.
I'd interpret it as "I don't build rendering engines but I build other stuff for which python is a good fit", but maybe that's just me being weird.
The trope about great programmers being people who reverse binary trees in their sleep and being able to swap ecosystems at the drop of a hat needs to die. Great programmers specialise.
You aren't honestly trying to sift out clueless prospective employers out by their ability to parse a title, are you?
The advice here is to put focus on ability to write software, and experience using python as a tool to do so, rather than implying that your ability is limited to python.
It should be.
Picking up language X is very easy for most values of X (more esoteric ones excluded). After you know a few (and the more diverse the set, the less new concepts you'll see), picking up a new one is quick. If it is something like Go, it's a weekend's worth.
Using the language in idiomatic ways and knowing the most useful libraries and frameworks does take more significant ramp-up time. If you are a solo developer, or if the entire team is ramping up at the same time, then it is bad.
But a new, experienced team member, with zero knowledge in the most used language in the team? Sure, why not? They may even bring new skills to an otherwise homogeneous group.
I always let the interviewee pick the language during my interviews. Usually it is something like Javascript or Python. Sometimes it is Bash. I keep waiting for the day someone will offer to do it in Scheme, Haskell, Elixir, Prolog, D, Rust, even Ada. Pseudocode is actually fine too, unless I have a reason to think the individual lacks programming experience.
Whatever you do, DO NOT brand yourself as a X developer, if X is a programming language. I can understand specializations like machine learning. But languages? It's a mere tool, and you need more than one in your toolbox.
I disagree. If you know Y and you're learning X, it depends much more on the value of Y. If Y is Lisp you will be quite successful at learning any X. If Y is something like C or assembly language, then you will not (it's always easy to spot the C guy who just started using Python etc.)
Yeah. As in years. This stuff also changes all the time so requires perpetual learning.
>But languages? It's a mere tool, and you need more than one in your toolbox.
It's not a mere tool and languages are more than just trivially learned syntax. They are the umbrella under which a panoply of tools, approaches, ideas, priorities and culture aggregate.
Sure, you specialization isn't necessarily always the optimal approach and sure there are benefits to the cross fertilization of ideas across community lines. However, being a jack of all programming languages, master of none probably means you'll suck more than a specialist at a lot of stuff.
Moreover, some ecosystems plain suck and avoidance is the sign of a good programmer. Statistically, you're probably a worse programmer if you've used a lot of PHP and better if you've learned rust/F#.
It really depends on how much time you have to wait for a new hire to ramp up before they are useful. If you need immediate help, then hiring someone who doesn't even have experience with your programming language is a complete non-starter. If, on the other hand, you are hiring for the long term and can afford to pay a ~3-month tax for a good engineer for (hopefully) several years, then you are correct and the ramp-up cost becomes pretty insignificant. But don't make the mistake of thinking that this is an exception to the rule—what languages you know absolutely does matter to any company needing to get work done now.
What parent was trying to explain is that an individual is not defined by a specific subset of his/her experience.
Just as fluency in Spanish does not make me Hispanic, experience in python does not make me a "python developer". That title puts focus in the wrong place.
That said, aren't you using C++ as a proxy for other needed skills though? Many Python programmers are not exposed to concepts like vtables, or they do not know how a linker works(or why it is even needed). Heck, many won't have done manual memory allocation or manipulated pointers.
Funny that you mention linkers, as they’re a bit of a minefield. Lots of the behaviour of linkers is platform (and implementation) dependent, I.e. are you building a shared library or an exe on windows or linux, and there are different gotchas for all of them. We wouldn’t expect people to know al of those things, but it definitely helps when it comes up, and you don’t have the same problem (ODR violations come to mibd) in many other languages...
A shop that says it requires years of experience in a particular language is probably making their own hiring situation more difficult than they need to.
C++ for example is definitely a sharp language that takes some time to adjust to, and experience in C will help.
However: I think any competent programmer with a few years of experience would be able to gain basic proficiency in a month or two and probably daily-level-mastery in 6-12 months.
I say this as someone who had 10+ years in Java and only passive experience in C. I've managed to become mostly proficient in C++14 in about a month of full-time learning. After 10+ years of programming, I can pretty quickly adapt to the C++ way of thinking about things. I don't think I'm alone in this style of learning a language, and many hiring teams seem to agree.
I work in games on gameplay and engine code.
No language is inherently "better" than another. They are each useful for specific cases, and understanding more languages means understanding their contextual strengths and weaknesses.
Filter bubble detected.
Can't imagine why parent is getting downvotes - it's an important point, and doesn't seek to invalidate the whole piece.
You mean like Google Docs or Quip?
This part made me sad. Imagine how much more powerful our tools could be if we didn't insist on jamming them all into web browsers!
I mean the web is awesome but it’s far from perfect from a programming point of view but that of course doesn’t matter for other reasons.
Yes, but it's easier to download a version of the browser that works. The alternative is for the developer to build native apps for each OS they want to support. In my experience, it's easier to build cross-browser than it is to build cross-platform.
Instead of spending engineering $$$ building, maintaining and operating a complicated editor?
>> why not just teach them markdown and git?
I myself am surprised they don't just use markdown. Git is another story. > writers are not professional coders.
But I have heard of writers clinging to old versions of WordPerfect, which were not WYSIWYG. They preferred the keyboard shortcuts to clicking around with the mouse.Furthermore if you're just writing the text of the article, you don't even need markdown, just a textarea. They used to do these on typewriters after all. Newspaper articles traditionally lacked anything but plain text. No bold, italics, or subheadings.
> Markdown covers the basic text markups
> but does not define how to format all these embedded media
> a modern newspaper has
Traditionally the jobs were separate --- and I assume they still are. There are the reporters, who research and write the text of the article. Then there is the layout team who lay it out, decorate it, etc. I can see how markdown would be no good for them. They would want a graphical user interface.Is this really a good assumption? Journalists are making complex documents with central interactive features and lots of data visualization. It seems a lot like classic waterfall thinking to assume that reporters could perfectly envision that and toss it over the wall for a separate team.
1. Most articles have simple layouts, like this one, https://www.nytimes.com/2018/04/12/us/politics/trump-trans-p...
2. Most reporters were journalism majors, which teaches research, interviewing, and writing. Graphic design is a separate discipline. It is possible for one person to be good at both, but rare.
3. Companies tend to cut up work, especially big companies, like the New York Times.
4. Complex articles like this one, https://www.washingtonpost.com/graphics/2018/entertainment/v..., aren't tossed over the wall. But that doesn't mean that one person did it all either. I'm sure there was back and forth between the writer and designers, and likely an editor oversaw it all.
On the graphics desk we know how to use git— we use it for js/css—but still write words in google docs.
Why is the journalist responsible for how it's presented? Wouldn't it make more sense to have some kind of product manager or designer handle this? And why is the journalist responsible for programming the look and feel of the their article? Why not have a front end dev do that?
Style can convey as much information as text.
My wife was, and all of her friends still are, journalists. They constantly worry and complain about all of the tasks that were once the domain of trained professionals--photographers, editors, designers--being put on the journalist due to budgetary constraints.
They develop the underlying technology to let journalists do what they want without needing to hire more employees.
No one is "only" hammering a nail here.