The difference between this and Simon's pelican is that with Simon, I get the prompt.
Last I checked, I did not see the prompt for this really cool thing, so it is not reproducible.
Did I miss the prompt somewhere?
Last I checked, I did not see the prompt for this really cool thing, so it is not reproducible.
Did I miss the prompt somewhere?
this guy seems to have taken that idea and got something similar/better, so likely the prompt isn't too special
In either case, I still don't see how I could reproduce this to test against various models, which is the entire point of Simon's pelican.
This is a bigger difference to the pelican than simple reproducibility steps. The pelican is intentionally esoteric, and thus open ended. The LotR is mundane and has a "correct" answer, aka, copy the movie.
It makes it a really awful test of capabilities. The pelican isn't a slop test. This crap is.