33 karma · joined March 4, 2020
I lost count of how many runs I had until I was satisfied with the results. I'd say for the final books I spent +- $10, so some millions of tokens!
What would you recommend? The cool kids seem to be using pure webGPU
Which thinker is that from? Foucault? Marcuse?
The ideia is to use AI to detect from the text books citing another books, and to make it "historical" by assigning the correct publication date to the target and source of each citation and graph everything!
The technical aspect is explained on this medium post[2](paywall free link!)
If you have any suggestion of improvements I will be happy to hear them :)
[1] https://news.ycombinator.com/item?id=28595967 [2] https://medium.com/mlearning-ai/graphing-citations-between-b...
I did something similar with RoBERTa and my own Kindle library to graph (with D3.js) all mentions/citations between my books (which books cites another books I have). I sorted the final graph by publication date to see some cool historical patterns of books citing another older books [1]
I also manually annotated ~1000 book mentions, but I combine RoBERTa with string search (I list all titles I want to search a priori) to reduce the number of false positives. I also augumented the dataset with thousands of books titles and metadata from goodreads.
I explain all the process on a blog post[2]
[1] https://thiagolira.blot.im/_projects/book_graph/main.html [2] https://medium.com/mlearning-ai/graphing-citations-between-b...
At the end of this 40 minute nightmare I had a bizarre homework to do at home that involved writing a list of uses to a paperclip.
They took a month to send me an email saying I wasn't accepted.
It's not that big of an achievement since I'm Brazilian and I already speak Italian, the romance languages are somewhat similar. But I'm happy and I want to keep learning new languages because that proved to be a rewarding experience all steps of the way.
I've used Lingq and Memrise, if anyone is interested :)