English Syntax Highlighting
evanhahn.github.io
evanhahn.github.io
The consequent highlighted parts reminds me a lot of Chinese, where essentially every word is "important" and there are few filler words, and hence there is often a lot of contextual information.
1. http://nlp.stanford.edu/pubs/CICLing2011-manning-tagging.pdf
More generally, for true syntax highlighting I think you do need the parse tree, and parsers (as opposed to taggers) definitely aren't at 97%.
If you are interested only in the syntactic tag, not in the structure of the tree, the number is somewhat higher.
The 80% figure was quoted from some talk I heard a few years ago, so I concede it's almost certainly improved since then.
Graded as a school exercise, I think 97% wouldn't be that good.
And as POS tagging is usually only done as preprocessing for some other task like syntactically parsing a text (which itself is usually preprocessing for yet another task), 97 % accuracy per word is not as good as it sounds. Parsers need to work with wrong data for every second or third sentence.
(This surprises me. I would expect accuracy for different words in a sentence to be correlated, you either make no errors or several.)
There's basically no part of NLP that's a solved problem. Even something as superficially simple as segmenting running text into sentences and tokens is decidedly non-trivial.
not quite what you're proposing but useful nonetheless
I don't know what the rational is for greying them out is, but that's the category.
It's really just a way to get people to visit his blog by making a bold statement.
* http://www.ruediger-weingarten.de/Texte/Capitalization.pdf (pg 4ff, last paragraph)
* https://mindmodeling.org/cogsci2013/papers/0462/paper0462.pd...
* http://linguistics.stackexchange.com/questions/699/does-capi...
We the People of the United States, in Order to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity, do ordain and establish this Constitution for the United States of America
I.
All legislative Powers herein granted shall be vested in a Congress of the United States, which shall consist of a Senate and House of Representatives.
The House of Representatives shall be composed of Members chosen every second Year by the People of the several States, and the Electors in each State shall have the Qualifications requisite for Electors of the most numerous Branch of the State Legislature.
etc.
At least, it is not a typo from ubernostrum, it's spelled "defence" in the original [1].
Moreover that's the British spelling of defense (is there a link with the lack of capital?).
Also note that, instead of "Blessings", it is "Bleſsings" in the original. But they are roughly equivalent if you apply a compatibility decomposition in your unicode normalization (NFKD/NFKC).
So where is the edit button for that constitution? Or do they only accept pull requests?
[1] http://www.archives.gov/exhibits/charters/charters_downloads...
https://www.quora.com/Why-is-the-word-defence-the-only-uncap...
but at least in my browser, I can't see the actual answer (it says "2 Answers" but I can only see one, which just confirms that the word is not capitalized).
So, an essay title might be "The Man and his Dog" rather than "The man and his dog".
Personally, though, I find many US newspapers' headlines jarring due to excessive capitalisation.
final String id = leader(NAMES_AND_SCORES);
Given a global list of names and scores, determine the leader's
id. (Ensure that id is a string of characters.) I'll use "id" to
refer to that leader throughout this paragraph.
Or in other words, does the analogy being proposed really hold?
Or another take -- speed readers take in whole sentences at a time. Colorizing parts-of-speech this way would only seem to slow them down whereas syntax highlighting code speeds my reading. I'm sure there's an analysis here; final, String, leader() are not parts-of-speech; each is a separate semantic statement.
I think it stems from the way we read Sherlock Holmes as an experience whereas we read a program as an explanation. You cannot substitute an explanation for an experience.
It's not an analogy, it IS syntax highlighting. You just encode a lot of semantics into syntax in java, so syntax highlighting is more useful for determining the semantics.
I don't think the OP was claiming this is useful for english in the way syntax highlighting is useful for java.
Parsing is the act of taking a linear string of tokens and building a tree out of them. That means reading in a string of tokens and applying the parsing rules (which may be encoded as a set of fuzzy correlations when humans learn those rules). When the rules are not solidly codified or slow to apply due to unfamiliarity, it helps to have hints to orient/validate yourself.
You do this parsing routine with your own natural language, too. You're just much more comfortable doing so and do not need hinting on what each word's role is. Just like a lot of old-school unix guys of lore are more comfortable reading code without the spectra of colors we commonly apply today. I could see natural language syntax highlighting being very useful for language learners, though. Color is used in Chinese language learning to indicate tonalities for learners, since most have no native/intuitive way to transcribe the pitch contours. I'm not convinced that the syntax highlighting presented in the article is really what you'd want, but I'm interested in the direction it's headed.
As an aside, speed readers don't take in a whole sentence at a time. An entire sentence simply doesn't fit within your fovea, but they have optimized their eye tracking to boost their speed. I do imagine that having lots of colors would disrupt and distract from the text and harm their speed / comprehension, but it may be possible that a different highlighting scheme could work for them.
For example:
The old man the boat.
{SUBJECT}[The old] {VERB}man {OBJECT}[the boat]
The purpose of a syntax highlighter is so that the human knows that the computer agrees on what the sentence/code structure is.
As someone else pointed out, different colors for different protagonist talking would be neat as well. But there is not much you can't do without deforming the original intent of the writer.
Another thing that could be interesting would be to give these tools to the writer instead of highlighting the text automatically, but then as someone else pointed out, these tools already exists and are rarely used because of the noise they add (bold, italic, underlined). There are also quotes, quads, uppercase, ... There are many ways to help the reader follow the text.
Are we trying to color differently verbs, nouns, adverbs and prepositions? So that the "goal" is to properly decide if "lie" is a verb or a noun?
Or are we trying to colour subject, verb, object and other elements of the sentence? So that "Rome destroyed Carthago" will have a different color for Rome then the sentence "Hannibal tried to destroy Rome"?
In general, code has "reserved words" and the rest is either a "name" (variables, constants, literals) or a ... Well... "Verb". Like functions, procedures, methods.
In some rare case (function pointers, closures) you have "verbs" that can be used as "nouns" but you completely lack concepts like dative, accusative and so on.
I think that this really breaks down as an analogy when you try to adapt syntax parsing to natural language.
By the way, syntax highlighting works for code only as long as you see the same color scheme. If the scheme changes, the benefit is lost.
very minor spoilers
…as it descends into madness (blank pages, words in spirals, backwards characters, single-character pages, overlaid paragraphs…). The very first unusual formatting in the book, and spit-take surprising to me as I wasn't expecting anything unusual at all, was simply printing the word “house” in blue. A fun read, thanks for reminding me of it!
I'm even doing some html-css-js-m4 work for a relatives business website nowadays, and I did not miss highlighting even with such complex mess, instead, I'm happier.
Larger blocks of cited text might work.
I can't help to think that this was done by someone that does not read that much.
Younger people seems to prefer videos instead of READMEs, and I have not really understood why until I saw somewhere that speedreading skills apparently have been falling drastically among young people. I mean why look at a video for 30 minutes to see if a tool or framework is worth trying, when you can scan the equivalent text in 30 seconds.
I would actually be interested to read a novel which did something similar: one color for narration (probably black, as it will be most common), and then a different color for each person speaking. That would be useful in a similar way that I find syntax highlighting useful: I could instantly look at text, and without even reading it, know who said it.
Narration text could also take on different colors, similar to in "The Neverending Story". How it is done, and how obvious its meaning, could even be a part of the art. That would be far more interesting to me than English syntax highlighting, to the point that if anyone knows of a book that does this, please tell me, because I would read it just to experience it. The point here is that in fiction, it's not the parts of speech that matter to readers, that's just a means to tell the story. What matters are the elements of the story, and communicating those elements visually could be interesting and useful.
I switched to a gray scale theme (emacs tao theme) two weeks ago and it is so much better.
He said, "The first sentence.
"The second sentence," he continued.
Does anyone have some background?
I found this because someone was using it as an argument against syntax highlighting for code, but I actually find that it lets me read significantly faster.
I've written a chrome extension for highlighting keywords, too bad I don't currently have time to give it the love it deserves:
https://chrome.google.com/webstore/detail/highlit/cooahmcpma...
I use it sometimes, it works pretty well but occasionally gets confused.
One thing that worked well for me was shifting from "reading" to "interrogating". Don't just try to read through a text, think carefully (and jot down) what questions you need to answer from the text, and jump around the text as necessary to answer the questions. If you don't have a sense of what questions to ask, do a quick skim of the text and any other relevant material to get the questions first, then dive in. Iterate and refine your questions and answers.
I'm very curious.