281 karma · joined May 3, 2021
Good library systems have a good percentage of DVD released films readily available for free.
There are far more where the verified audience score is way higher and they tend to either be religious films or films with very low numbers of audience reviews (aka selection bias)
Doesn't happen as much anymore since they require proof of ticket purchase to review, but it still will on occasion. And it happened enough previously that they specifically had to put the proof of purchase condition in place.
At the moment I'm mostly just planning on sharing it with my brother, but who knows. I generally think I'll eventually share the data somewhere possibly on a website. I can't commercialise it if I understand the New York Times license properly since I scraped the Timesmachine for the listing pages.
Regardless, it's been super fun, and the data is very cool. Like 21 thousand unique films. At the moment I'm mostly trying to decide if I want to continue adding bits, like seeing all the listings a specific actor was in ... Mostly to test the hypothesis purported by PCU that there is always a Michael Caine or Gene Hackman film on television (I'm joking obviously there isn't although it is closer than you'd think). At the moment you can view an individual date or all of the listings for a specific film.
To me this is starting to feel almost like a Wikipedia thing or internet archive. This is, in a way, a specific ground truth of when television programs played in the New York area. I just have no idea who or what would be interested in it.
As for the others, padel is far far far more athletic than pickleball (and paddle tennis for that matter), and I would argue even more so than tennis. I grew up playing tennis and I love it, but padel is more finesse and also scratched that itch of learning a whole new sport. I learned it in the UK 3-5 years ago, and I do very much miss it. Literally I think there is one court in all of New England, it is crazy how much of a not-thing it is considering it was invented in Mexico.
There is at least 100 different films from every year from 1931 to 1999 it looks like (obviously many more from the 80s and 90s). But even just considering that, that's 7K as an absolute minimum. With straight-to-video and TV films the 90s peaked with ~850 unique films from 1995 alone playing on television. And to just give an idea of the level of obscurity, the median IMDb vote count for 1995 is 500, so half are on the level of something like https://www.imdb.com/title/tt0122968/ (which yes, played 7 times, mostly on TMC in March of 1996). Also once you get very obscure things get all muddled. Like with https://www.imdb.com/title/tt0112363/ which is marked as 1995 but certainly appears to have played once on television in 1991 so IMDb is wrong in this case. And of course there are a number of mistakes due to titles matching which I'm slowly correcting.
There are some caveats. This guy: https://www.imdb.com/title/tt0222755/, played 6 times, but on channel 41 (which at the time was Spanish language maybe?), so it has 0 votes on IMDb, but played 6 times on television, which is odd but not unheard of. But you do have to consider that this is using a slightly larger range of channels (some specific to the New York region, like CUNY and MSG) than one might expect. I would guess though that maybe 40% of films play on TCM, SHO, MAX, or HBO ultimately though. Most channels didn't play movies ever outside of primetime.
If you message me privately I'd be happy to share the data. The git repos are:
https://github.com/patsmad/nyt-listings https://github.com/patsmad/nyt-listings-app
I use them for curation at the moment so the READMEs leave ... something to be desired. I hope by the end of August to have a read-only version up and running, although without a wikipedia-like effort I don't see how I would curate it fully so it'll probably always be a little touch and go as to what data is available.
The stats I have from curating are: 369345 individual movie "listing boxes" (I would guess around 98% accuracy, although if I were to field a guess the actual number there should be is probably 400K) of which 321308 are matched to a movie, and 296941 of those are for sure unique. And overall 202203 have channel + time + duration matched up using the VCR listings (which the New York Times conveniently published from around November 20th 1990, and the internet archive very nicely has the program the VCRs used to encode/decode those codes). There are 21530 unique movies at the moment.
If I understand the New York Times correctly, then none of this can be commercialized since I scraped the core data (the pages themselves) from the TimesMachine, so this really is a personal project, which I'm happy to share. I've made a few Letterboxd lists from the corresponding data, for example a series of lists with all of the movies (and play times) for films playing on September 1 in particular e.g. https://letterboxd.com/patsmad/list/television-films-septemb... It is rather consistent, around 100 films a day, for 1990-1999 it was 106, 118, 74, 74, 89, 99, 98, 110, 97, 93. As is obvious I can talk about this for days.
I'm not sure the best way to do private messages, my email is associated with this account, but I have no idea if you can see that. I usually just lurk on HN.
It is pretty comprehensive and obviously it keeps the same market (NY). On your last point yeah the amazing thing is just how variable the length of movies are. Also just how many there were! Over 26 thousand unique films played on the 60ish channels in the New York Times in the 90s. Around 100 a day. Some films have like 40 votes on IMDb and played 20 times on television.
I have a small personal project of cataloging all the movies that played on television in the 90s. There are tons of television shows that are not only not available on DVD or VHS but also seemingly no one has it. Double goes for cartoons, tons just totally unavailable. It is sad.
To me being the best quarterback at your school and playing varsity football for four years should basically be considered the same way, as you point out, being the top bassoonist would. And my guess is that for the vast vast majority of cases extracurricular activities are pretty much all created equal unless you do it to a degree that makes it notable (like being an all-state athlete, or playing with you regional symphony or something)
I knew a varsity crew and varsity sailing athlete and despite both being nationally notable in their sports my understanding was the coaches could do nothing to "recruit" them. It of course helps in admission as much as playing a music instrument at a high level or being a published author might. If you have that plus elite academic achievements then you probably have a decent chance at admission to a school of your choosing.
To give an example. I gave it an image of a television listing from some time in 1996. It was for a television film fashioned from a two part episode of The Commish called Father Image. It correctly got all the information from the image, identified it as a TV movie based on an episode of a television show, and most impressively I think claimed that the tv show played on December 7, 1995. That wasn't when the listing was from, but that is the air date of the episode of the television show.
It is clearly using other bits of information (IMDb information about air dates I imagine) and effectively doing multiple db fetches / searches in the background to flesh out the information it is giving back. That is precisely what I want from a "new" style of search. Give it something simple and give me a bunch of curated information which otherwise would have taken multiple Google searches / diving into links to determine. Fun stuff.
But I've never coded in Rust. I could have written the exact same blog (excepting maybe the last bit) and called it "writing Python like it's Scala".
Additionally they don't require the SATs at although prior to COVID they also had a minimum score for that.
Wikidata is incredibly useful for things that I would considered valuable (e.g. the tMDb link for a movie) but due to the curation imposed upon Wikipedia itself isn't typically available for very many pages. An LLM won't help with that but another bit of information like where films are set would be a perfect candidate for an LLM to try and determine and fill in automatically with a flag for manual confirmation.
Impressions: was very underwhelmed previously with charGPTs ability to tell be plot summaries for obscure films. It almost always made it up, even when prompted with a source (e.g. Wikipedia). Asking vague questions like "I can't remember but I'm looking for a film like war games but with a tiny helicopter" chatGPT always presented me with very popular films and could never quite key in on the "toy" or "small" bit of the query.
Bard seemed better in that while it couldn't quite get the movie right (for the record google itself gets the movie easily, Defense Play) it did eventually give me the right plot. Same with Ghost Fever which no matter what I did ChatGPT would always hallucinate the plot of. Unlike ChatGPT, Bard gave me the specific Wikipedia link quite often as well. It did make up the people and box office of obscure films pretty much universally.
Bard was actually really impressive for returning obscure results (which ChatGPT always seemed reluctant to do). I asked for another ghost film from 1987, and it happily returned Ghost Riders a film with a scant 200 votes on IMDb. Whenever I asked ChatGPT to give me bad or obscure films it would give me movies like Top Gun no matter how much I insisted I wanted something obscure.
I wouldn't trust either really, but Bard seems slightly closer to what I would hope a search/GPT integration felt like. But again I'm probably using it much differently than is intended to how others might interact with it.
Just for example. I was trying to remember a film that is basically a knock off of war games involving two teenagers using a scientists small helicopter to do stuff. I typed into Google "film with small helicopter" and it was not only the first result, it was in a little cut out section highlighted at the top. The film is called Defense Play and it has a grand total of 144 votes on IMDb. Incredibly obscure.
I thought it would be fun to ask ChatGPT roughly the same question and at first it just gave me a list of very popular movies with large helicopters. So I said, no this has a small helicopter or a toy helicopter in it. And it gave me a list of films with helicopters but three in Toy Soldiers, Small Soldiers (which probably does have a small helicopter in it) and GI Joe (which is based off a toy and has a helicopter probably).
No matter how much I insisted that this movie is obscure it can't quite figure out how to do anything about returning things that match something as vague as "obscurity" even though there are measures (e.g. IMBd votes) available that could reasonably give it such a notion. It probably does know about the movie, but whenever I ask about obscure movies while it often can give me a year and the first few actors in it it does seem to make up the plot even when asked to just paraphrase whatever plot synopsis it has available to it.
I do see it improving but I also wonder if a model which is heavily weighted towards guessing the next section of text will have difficulty with the unlikely scenario of someone being specifically interested in an obscure film a handful of people have ever bothered to watch. Seems really useful for searches where a person doesn't really know what they want, and very not useful in searches where the person knows they need a specific and obscure piece of information potentially.
It was similar when I applied to grad school (this was engineering). They told us pretty unequivocally that getting a perfect GRE math score was pretty much required, but that the verbal went right in the bin, so don't worry about it.
It makes sense for engineering focused schools or those applying to engineering schools. I'm still less convinced places like Harvard where a good chunk of the students won't take a ultra-difficult math class during their time there has much use in the SATs. At least in the "I got a 1580 on my SATs but someone with a 1570 got in over me, how is that fair?" Kind of way.
This character is a conservative infotainment host. And so when reality ultimately flies in the face of his conservative viewpoints the joke is that he merely dismisses reality because "reality has a liberal bias".
Within this context I imagine the argument is that in many cases what is perceived as a "political bias" by some might just be considered "reality" by others. For example there was the issue from the early 2000s of "teach the controversy" where some conservatives would have argued answering truthfully about the scientific consensus on evolution or global warming was "leftwing bias".
The most important thing I think is that it is a joke that simply rings true with a lot of people as time has passed.
(2) Counter. I asked it the other day "how many movies were Tom Hanks and Meg Ryan in together" and the answer ChatGPT gave was 2 ... not only is that wrong it is astonishingly wrong (IMO). You could be forgiven for forgetting Ithaca from 2015. I could forgive ChatGPT for forgetting that one. But You've Got Mail? That's a very odd omission. So much so I'm genuinely curious how it could possible get the answer wrong in that way. And for the record, Google presents the correct answer (4) in a cut out segment right at the top, a result and presentation very close to what one would expect from ChatGPT.
I don't know about other use cases like generating stories (or tangentially art of any kind) for inspiration, etc. But as a search engine things like ChatGPT NEED to have attributions. If I ask the question "Does a submarine appear in the movie Battlefield Earth?" it will confidently answer "no". I _think_ that answer is right, but I'm not really all that confident it is right. It needs to present the reasons it thinks that is right. Something like "No. I believe this because (1) the keyword submarine doesn't appear in the IMDb keywords (<source>), (2) the word submarine doesn't appear in the wikipedia plot synopsis (<source>), (3) the film takes place in Denver (<source>) which is landlocked making it unlikely a submarine would be found in that location during the course of the film."
The Tom Hanks / Meg Ryan question/answer would at least more interesting if it explained how it managed to be so uniquely incorrect. That question will haunt me though ... there's some rule about this right? Asking about something you have above average knowledge in and watching someone confidently answer it incorrectly. How am I supposed to ever trust ChatGPT again about movie queries?
I suppose both "young people are generally drinking less alcohol" and "more young people than ever are abusing alcohol" could both be true statements.
Just an example to explain what I mean. The film Bard Wire has an alphabetical cast list (so Pamela Anderson Lee, as she went by at the time, is mid way down the credits). Interestingly the "three actors" listed by IMDb has Anderson first, but then the next two are just the top two on the credits. It would be useful, to a degree, to try and figure out who those second and third people should be based on some metric.