Hacker News Reader

> A lot of the content on HN/Reddit/forums is of low quality grammatically and often written by non-native English speakers (such as myself).

Corrected:

> A lot of the content on the HN/Reddit/forums is of the quality grammatically and non-native written by written English speakers (such myself). as UNK

Yeah. It's got a long way to go. No idea where "as UNK" came from.

UNK usually comes from the final sampling step: the distribution of words contains a special token UNK to represent all words with insufficient statistics in the training corpus.

It's like the equivalent of NaN; what you put when you don't know what should be there.

coincidentally, the op corrected this as: It's like the equivalent of NaN

Yep, it definitely has room to improve. The work thus far has primarily been a proof-of-concept for the methodology used to generate training samples (i.e. starting with grammatically correct text and introducing errors). Next step is to try to include more high quality data, after which I may try out comment data from HN, etc. I think it would be interesting to see what the effect of somewhat noisier data like that would have on the model.

Programmatically generating incorrect grammar from correct sentences must be really tough. There are so many more ways to incorrectly structure a sentence than there are correct ones.

Random idea: what happens if you use Google translate to generate the incorrect sentence, I.e. Translate it to other languages and then back again. If the resulting sentence doesn't match the original, add it to the dataset.

>> But a tool that could understand my intent and then propose restructured or similar sentences and words that convey the same meaning could be a true killer application.

Yeah but that's hard though. Who knows what is your intent when you're generating an utterance, written or spoken? Sometimes you yourself may even read what you wrote (or hear what you said) and wonder what you meant.

Not to mention what an absolute nightmare it would be, trying to compile data on the linguistic intent behind utterances! How do you even start to collect that? Ask people to say things, then ask them what they meant... but what did they mean when they explain what they meant in the first instance?

The worse thing is that the very notion of what is grammatical changes with context. For instance, to go back to dropping the "the"'s: imagine you read the phrase "eat soup with spoon". Is that an ungrammatical form of "eat the soop with the spoon", or is it an instruction, perhaps something you'd find in a soup-eating manual, which therefore is perfectly valid as a terse form of English?

What you intend an utterance to mean affects whether it is grammatical and the grammaticality of the utterance affects its meaning. Nice, eh?

You make a great point.

Presumably biasing the training sets to those built from corrected sources would help, but what about using n-grams to throw out poor examples of ungrammatical sentences? It wouldn't be perfect, but some kind of score based on a much lower prevlence of n-grams from the generated sentence vs the original might indicate acceptable cases and those where they were fairly similar might indicate the generated sentence could in fact be valid (and thus should be discarded)

> A tool that could understand my intent and then propose structure or similar sentences and words

I'm working on this exact thing right now fwiw, in my app called Prompts. I imagine machine assisted apps are going to percolate up over the coming years, or months, in a way we haven't seen yet. It's pretty exciting, if we can get it right. Written language seems like a decent place to start.

Dropping `in the fridge` works.

"Alex went to kitchen to store the milk" corrects to "Alex went to the kitchen to store the milk"

On the other hand, it never removes unnecessary articles, demonstrating one of the deficiencies of the training set.

Couldn't that be rectified with a Parts of Speech tagger? Or am I overlooking something?

The training data never says to remove any articles, so a PoS tagger won't help.

I also got some weird results in the small sample of testing I did.

> The player chose to go to the good team rather than the sucky one.

> The player chose to go to the good team than rather the sucky one.

And it seems to like duplicating adjectives:

> Being worse off than even worse off makes one feel sad.

> Being worse worse than even worse worse makes one feel sad.

Interesting example, seems the algo cannot infer whether 'store' is the verb or the noun. It would need to understand the meaning and context of the sentence to do this.

Maybe it's also to do with how they generate the training data. Author did say removal of articles was one thing they used to generate the 'incorrect' sentence in the training data. If the film data uses 'the store' much more than 'store', you can imagine how the algo could get biased to thinking store is always preceded with 'the'.

Technical documentation may be dry, but it is generally well read and edited, grammatically correct, and easily obtained.