I would love to read anything you have written about the topic at length. Thanks for your contribution.
59 karma · joined January 30, 2022
I can easily falsify the accusation that, "people underestimate transformers and don't see that they are actually intelligent," by defeating the best open-source transformer-based word embedding (at the time) with a simple TF-DF based detector (this was back in September).
https://www.patdel.com/plagiarism-detector/
No, these things are not, "emergent," they are just rearranging numbers. You don't have to use a transformer or neural network at all to re-arrange numbers and create something that is even more, "artificially intelligent," than one that does use transformers it turns out!
I think most laypeople understand the simple statement, "it's a parrot."
I had the original author of this paper reach out to me about my plagiarism video on Mastodon:
https://dl.acm.org/doi/10.1145/3442188.3445922
The idea of a lossy JPEG/Parrot helps capture the idea that there are dangers and opportunities in LLM's. You can have fake or doctored images spread, you can have a Parrot swear at someone and cause un-needed conflict - but they can also be great tools and/or cute and helpful companions, as long as we understand their limitations.
* [On the question of replacing Engineers](https://www.youtube.com/watch?v=GMmIol4mnLo)
* [On AI Plagiarism](https://www.youtube.com/watch?v=whbNCSZb3c8)
The consensus seems to be building now on HackerNews that there is a huge over-hype. Hopefully these two videos help see some of the nuance behind why it's an over-hype.
That being said, being that language generation is probabilistic, a given language model which is transformer based can either be trained or fine-tuned to have fewer errors in a particular domain - so this is all far from settled.
Long-term, I think we're going to see something closer to human intelligence from CNN's and other forms of neural networks than from transformers, which are really a poor man's NN. As hardware advances and NN's inevitably become cheaper to run, we will continue to see scarier and scarier A.I. -- I'm talking over a 10-20 year timeframe.
https://news.ycombinator.com/item?id=32985848
YouTube video goes through and explains more of the math, at least from a hand-wavy perspective, as well as some of the attack vectors.
https://github.com/pwdel/codepractice/tree/main/PlagiarismDe...
* Mentimeter - seems to be the same type of, "sector," e.g. real-time engagement as opposed to long-term, "time-capsuling" of documents.
1. Start a free Dropbox account, create a public upload folder and share said upload link with everyone in the group. 2. Start a throw-away Gmail account, schedule an email containing the username and password to the Dropbox folder to all of the club users for a future date, X weeks in the future. 3. Logout of the dropbox and Google account, throw away the passwords to both.
As long as the email from the throw-away gmail account goes through, then everyone gets the time-capsuled password to Dropbox at the same time and can then log-in to Dropbox at their leisure to view the documents.
Obviously this solution is for a small, very-trusted group and doesn't scale, and has other security problems and the risk of losing everything.
Edit: Also found this, a bit old-school, Windows-based, PC-based solution, not really a modern tool designed to work over the web/internet it seems. https://www.jensscheffler.de/filelocker
There also appears to be a standard way of locking files in bash, which could be used to cobble together a solution where perhaps a file could be maybe scp'd into a server, locked upon upload, and then perhaps the directory unlocked at a pre-determined date for all users, but that seems like it would require a lot of thinking and building and sys-admin manual labor. https://stackoverflow.com/questions/1715137/what-is-the-best...
> Cannot parse Overpass API response https://backofyourhand.com/build/bundle.js:6:1039277 asyncFunctionResume@[native code] [native code] promiseReactionJobWithoutPromise@[native code] promiseReactionJob@[native code]
I don't feel discouraged at all and I appreciate your feedback and thank you wholeheartedly for it. First off, I have been told already on another platform that I did actually refer to Expected Value in the book prior to introducing it - so yes, that is absolutely in the change order for the next version on Tau Day.
I was in a rush to publish this last week for Pi Day. I had reorganized the chapters at one point and moved Expected Value way to the back because I saw it more in line with making a prediction rather than it being a parameter. Originally I had this book organized like a traditional textbook, but one of the points I would like to drive through is the, "workflow" of using probability to make a prediction, meaning, you 1) Calculate Parameters then 2) Find relationships between Sets and finally 3) Then and only then, make predictions, which involve a value and a relationship statement.
Part of this is because there is such a huge ongoing trend in wanting to make predictions, gain more certainty over the future, work higher end jobs leveraging statistical computing and machine learning, etc. - so I'm thinking there are a lot of folks out there who want probability described from yet another standpoint. That being said, if I have just repeated what has already been said, then I have failed.
On the bright side, with LeanPub I can just clean it up and release a new version of the book.
In most textbooks, Expected Value is taught way at the beginning, I think because it's, "easier" and perhaps there isn't a lot of thought put into the workflow I have described above. What it sounds like I should do is comb through and remove references to Expected Value earlier on in the book. I could also certainly clean up any technical terms like that as well...not sure if you had any other ones you identified?
Yes you are right about stretching the lemur abstraction. To a certain degree, we really can't teach lemurs any of this stuff, it's more of a motif for making things visual for humans, not lemurs. On the other hand, I do offer somewhat of a rebuttal to the idea that, "lemurs could never learn X," in this YouTube video I produced a few months ago on Bayesian Statistics.
As to your criticism about statistics vs. probability - to be honest that one confuses me a bit. What are you referring to specifically? What do you see as being statistics and what do you see as probability and where do you draw the line?
It would be really great to get people's feedback, but you don't have to comment here - I realize it may take some time to digest since it's 200+ pages. Remember if you subscribe you can just send me an email at any point if you come across an error or something that wasn't quite clear, or just any other idea of how I could improve.
thanks!