Comment by genmon | Hacker News Reader

Comment by genmon | Hacker News Reader

Some unlinked features...

If you put the Dewey division in the URL, the directory auto-opens. e.g. here are episodes about prehistoric life (my current jumping-off point)

https://genmon.github.io/braggoscope/directory#560

There's a visual map of episodes. After principal component analysis of the episode embedding vectors, these are the most significant two components as the x,y

https://genmon.github.io/braggoscope/map.html

(it's not super useful tbh -- e.g. the Manhattan Project and the Cambrian Explosion have the same x,y... presumably because they are both about explosions?)

Many episodes have a reading list, and these are all linked to Google Books (so you can purchase/check out from a library), e.g. this episode page

https://genmon.github.io/braggoscope/2022/10/20/the-fishtetr...

There are ~4,600 books, and I have ~88% coverage on getting a Google Books page from the original data. Any ideas about what to do with this big list of academic-recommended books v welcome!

This is great. I'll certainly be thinking of "classification" uses for ChatGPT in the future.

Thinking out loud: Add the experts, not just the reading lists. They're a jumping-off point into academic-paper-space. What have they published? In what journals? Who have they collaborated with?

Here’s a powerful use - content moderation. Today we literally traumatize content moderators with the dregs of the human mind. Chatgpt is fairly good at identifying the classification of content on many dimensions, including the ones it’s actively screens for. Regardless of how you personally feel about content moderation, I would be happy to see humans not have to be actively involved in it and face the traumas they must live with for where the moderation happens. I’m sure it’ll get things wrong, but humans do too.

I'm sure I remember Zuckerberg talking about this very thing in one of the Congress interviews he had.

I don't disagree with you, the content producers are always going to outnumber the moderators by a massive margin. It makes reasonable moderation very difficult.

OpenAI have a content moderation endpoint for this too.

It's good at it because OpenAI outsources the bad stuff to Kenya https://time.com/6247678/openai-chatgpt-kenya-workers/

People still have to look at this horrendous stuff.

To bootstrap, yes. But once you’ve built the machine, the people aren’t necessary.

Automating your moderation policies is a cop-out. It isolates you even further from the peons whose lives you're controlling.

European humans have the right for any automated decision affecting them to be reviewed by a human. We do not want a world where a tiny handful of people, getting fewer by the day if it's Twitter, have untrammeled control over what billions of people can say, see or do. We can't let businesses do what they like, change their policies on a whim and have it instantaneously applied by automatons, if it's bringing misery to millions of people.

Remember those old-fashioned things called laws? Courts? Judges? Legislators? The things we used to use for deciding what people could/couldn't do, before tiny companies with billions of users applied their own de-facto laws with no oversight, governance or accountability. Move fast and break things!

1) you can have human reviewed appeal and have automated moderation 2) a provider of a service is not a provider of a public commons, nor is the service being confined to a specific type of discourse preventing anyone from doing anything in their life. You don’t have the right to go to a movie theatre and use it to practice your singing recital during the movie - the theatre owner can and will kick you out. This doesn’t trample on your rights - you can still practicing singing, just not there. Likewise you can still spew racism and talk about child sexual abuse in your home all you want. But no one is obliged to provide you a platform for that. 3) I see nothing superior to a capricious and biased human judging my content vs a capricious and biased model judging my content. The human, however, will be traumatized by the “millions of people” made miserable by their inability to spew hate and share child porn on Facebook. 4) we still do depend on judges and laws. Today the judges as laws say you not only have the right but the obligation to create a space relatively safe. In your European utopia speech and thought is even more limited by law than in the US, but in both locales forum providers are held to a standard. Right now in the US there’s a major case in front of the Supreme Court which will determine whether these companies can be held liable directly for failing to sufficiently moderate - if it passes, they’ll be even more obliged to “purify” their platforms of anything that anyone might feel compelled to sue them over. But even now all providers have to conform to the laws and judgements of the lands they operate in, and those lands say it’s not only ok to moderate, but it’s mandatory. So, your final point really, truly, makes no sense.

Excellent! I'd love to see a script that sends a list of "descriptions" (1-100 words) to ChatGPT and directly gives you back a ready-made (embedding vectors closeness) map in a (textual) graph/chart format (like your above map or your plot https://interconnected.org/more/2023/02/in_our_time-PCA-plot...)

It turns out that "closeness" is usually hard to visualise/explore when you're dealing with a 1,000-dimensional space... and PCA has the failures mentioned above.

It's weird -- it's locally useful to navigate, and at a high level kinda useful, but only if you squint and don't look at the problems. So I feel like a fisheye visualisation would be appropriate? That's something that I'm exploring in other projects.

I wouldn't necessarily reach for PCA. No reason to think that the first two principal components necessarily encode anything particularly interesting. If you want to lay out each point in 2D in a way which keeps similar points nearby, something like t-SNE is worth a try - visualizing embeddings is what it was invented for.

Excellent, new to me and I'll give it a go, thanks!

I gravitate to PCA for terrible reasons (undergrad so it's what I think of first) and like you say, it's beguiling yet disappointing, the components rarely have any human meaning.

I'd suggest trying t-SNE [1] instead; you'll be losing almost all of the variance by projecting onto the first two eigenvectors produced by PCA.

[1] http://karpathy.github.io/2014/07/02/visualizing-top-tweeps-...

TSNE or UMAP (as others have mentioned) is good but also take a look at the tensorflow projector. You can host your own and/or pass custom data to it.

The word game of 'semantle' (https://semantle.com/) is a nice way to get an intuitive grasp of how unintuitive the closeness in a highdimensional space is, as you're required to guess a word based on it's semantic similarity (according to classic word2vec metric) to your previous guesses.

> https://genmon.github.io/braggoscope/directory#560

Shouldn't https://www.bbc.co.uk/programmes/b00lh2s3 (Ediacara Biota) be in there?

Love the plot, for high dimensional embeddings try UMAP for a better result, maybe T-sne too if you're interested in clusters.

Love the visual map. What does color mean? Any way to do a 3rd PC, and put the visualization in a cube one can toy around with?

Colour is the 3rd component -- I wanted to see the difference between overlapping episodes.

As for the 3D plot... here you go!

https://interconnected.org/more/2023/03/in_our_time-PCA-3D-p...

Basic PCA + Plotly is actually in OpenAI's official Python library (in `embedding_utils`) -- this plot is just the output from that.

:D this made my day.

Your PCA is awesome. The horizontal axis seem to go from people (high level nature?) on the left to physics (low level nature?) on the right, while the vertical axis seems to go from individuals and particles (small things?) at the bottom, up to civilizations and the universe (big things?) at the top. Maybe the Manhattan Project and Cambrian Period (not explosion) are together because they're both big things in their fields and physics and evolution are near each other horizontally?

Reply on news.ycombinator.com