Hacker News Reader

Comment by jsdalton | Hacker News Reader

Simon do you have a page somewhere that shows _all_ of the penguins created by various models across time?

You’ve been doing this public service for so long (well, for so long in “AI hype” years anyway) that it’d be fascinating to see the evolution of this artifact across time.

View on HN

For the moment the best place to see them all is to browse the tag on my blog - 140 posts now! https://simonwillison.net/tags/pelican-riding-a-bicycle/

I'm running out of excuses not to build a proper comparison site though. Maybe I'll have Astra do rhat.

Would be great to spend some money (other people's money) on paying some illustrators and artists to do the same for comparison. It could be straight images converted to SVG or direct vector graphics.

Doesn't that somewhat miss the point? The models can't see and check their work, for instance.

I'm not clear what you are saying. I am curious about what humans can do in the same context. It's becoming a standard I think for some tasks. E.g. how does a Gen AI stack up against a human in quality and "cost". Perhaps I'm missing what you are asking.

They can. Ask them to generate SVG , call a renderer and let them inspect the bitmap/png. They can learn about it.

They can, but for this specifc test, they don't. It's a oneshot with no feedback.

Technically you would pay programmers to write SVG code with no visual feedback if you want a straight 1:1 comparison.

Kinda. It's the difference between measuring the end product vs how it's made.

FWIW, I had Astra do it as you suggested: https://pelican-model-gallery.alienfluid.chatgpt.site

I used Light mode and it used up all my limits for the day and had to continue the following day.

This is great! I can see some of the more abstract ones ending up on the wall at SF-MoMA.

Missing GLM-5.3, though.

The GLM 5.1 one where the pelican is missing because it’s broken would be a great candidate. “A pelican enjoying a sunny ride!” is very “this is not a pipe”

You could also have it create a viewer competition selection; show two images without attribution and let the use select which one. Could be cool to see the wisdom of the crowds; could also have price ranges to have subcategories. I bet Astra could do that in a half hour.

Loving the pelican silliness. My ChatGPT is over the moon about it. Gonna miss it when it really is done. Though maybe a pelican riding a bike game could become the benchmark in a year.

Reply on news.ycombinator.com