Voyages in sentence space | Hacker News Reader

Voyages in sentence space

robinsloan.com

90 pointsbkudria16 comments

The voyage looks somewhat random, yet inspiring. I can imagine using it to grab ideas for the overarching story arc in some storytelling.

> I am waiting at the entrance of the fantasy kingdom.

...

> I have a revelation about reality.

Main problem is, it doesn't look at all robust. A small change in a single word at the start or end sentence produces a whole different result. That doesn't look like a "space" at all, as insights about closeness do not work in it.

Ok, I've played with the "Look around" example and it does behave as a space, with the "Closer to home" option producing somewhat similar results to the starting point, and the slider producing concepts that look farther away from it.

Maybe the "Gimme a gradient" option step is too large for sentences that are far away, and an option to produce more steps, closer between them, would produce more intuitive results?

I guess that can happen sometimes in actual spaces too. The shortest path from a point near the north pole to the south pole can change dramatically if you slightly alter its endpoints. Given a high-dimensional space it could be that most possible paths behave in this weird way.

How useful is the spatial analogy if the space in question defies all our spatial intuitions?

I can see it has mathematical meaning, but isn't the whole post about having intuitive understanding of sentence space?

I think it's less about intuitive understanding, and more about having a flat, numerical representation of a sentence that (qualitatively) satisfies some properties.

This is essentially the reason that differentiable methods alone don't work that well for NLP. Language is a discrete arbitrary symbol system, so small changes may have large effects at many levels. ("Dog bites man" is not like "Man bites dog", discrete is not like discreet), etc.

I feel like the "middle" sentence is not intuitively what you'd normally think of something in between. Even for sentences that are quite close. "Eve walked home slowly" -> "Eve walked home quickly" gives a middle result of "As white hand smiled."

Robin is an author that dabbles in machine learning and farming olives for making olive oil. It's not surprising that his examples aren't robust, but I always enjoy reading about his endeavors because I end up in a rabbit hole on whatever subject he's writing about.

_"You can also increase or decrease the distance you peer into sentence space from your initial location; as you increase it, the results get more diverse. [...] What you’re seeing is the transition from the richness of arbitrary text to the regularity of this particular sentence space."_

There's a huge gap in here. Mr Sloan has worked on the idea of a gradient in language. He's worked to realate the craft of colors to sentence creativity.

But color craft on a computer is just an abstraction for picking colors out of the gammut of screen displays. And this gammut is a subset of the visible spectrum, which is a subset of the electromatnetic spectrum.

He didn't work at all on how this project relates to language itself. I guess I have to read the papers.

What exactly is 'sentence space'?

>> What exactly is 'sentence space'?

It's a multi-dimensional cartesian space where each sentence occupies a single point (i.e. a set of coordinates).

The point is that once you have such a sentence space, you can then do geometry over it. Like calculate distance between sentences etc. The hope is of course that the sentence space is constrtucted so as to encode the semantic relation between sentences, so for instance sentences that are close to each other encode similar meaning etc.

Sorry if I wasn't clear--What's in there? You've mentioned 'each sentence' and a 'multi-dimensional cartesian space', what _constitutes_ the 'space'? Each sentence from where? What determines a set of words is a sentence?

Is it a WordNet space? Is it a sampling from a corpus? A neural network trained on what? the internet?

One tree? What's the forest? Old growth? Jungle? Alpine above the tree line?!

Ah, OK, you were joking. Sorry for missing it, I had my humour removed at birth :)

The glass is half-empty, or half-full is not a joke, but a an ambiguity of language. “Sentence space” could refer to both the sentence itself, numerically quantified, and the context which makes a sentence grammatically correct, composed of senseable words which go together. Colors don’t walk. Right?

My response is just a comment, but even if you disagree with my example, the question remains—what constitutes the space the alternatives are drawn from?

Can this be used for style transfer?

I'm looking for a solution to create a consistent personality for a chatbot, where the content all is created by different team members.

For a similar tool in the somewhat simpler-to-understand ‘word space’, see Travis Hoppe’s ‘Transorthogonal Linquistics’:

https://github.com/thoppe/transorthogonal-linguistics?files=...

Shayne Miel has observed that its results for the space from ‘sociology’ to ‘mathematics’ closely match an earlier XKCD comic:

https://groups.google.com/d/msg/gensim/a8PF8CInRKk/6XWcp87WB

Warmer. _"English dump of Wikipedia that was sentence and word tokenized by NLTK."_ Wikipedia would certainly cover the morphology of a 'space'. I'm curious how the linguistic relations are derived from 'context', which I gather is what word2vec does. Although NLTK does have an interface to WordNet.

Reply on news.ycombinator.com