Serverless Video Transcription inspired by Cyberpunk 2077
github.com
github.com
Just using APIs also misses out on the learning, community, and collaboration of open source software.
On your new car, it’s a series of black boxes, cables, and tubes - you don’t know and don’t care what happens inside the boxes, just what goes in, and what comes out. It’s heavily knowledge based - you need to know an awful lot about the boxes and how to work with them - but nobody expects you to machine your own parts. If one isn’t made that suits your purposes, you have to go to an old car guy.
The skillsets are similar but different. To be clear, I really am talking about cars and mechanics here - you won’t find many who can fix up both a 1960 and a 2021 motor. You’ll find plenty who can do one or the other, however.
And yeah - the same applies to developers.
I specialised in new vehicles and then you have specialise in brand to actually know whats going on and how to fix said black boxes and systems.
My reason for training to become a software engineer was mainly because I felt like the knowledge and on-going training you need to retain/constantly learn was not worth the money earned.
Instead now I earn my bread from what was my hobby, essentially the same work ethic as far as knowledge is concerned just less back breaking to put into practice.
I blew a tire on my car and due to its unconventional size and delays I had to wait 3 weeks for new tires to arrive.
Then, during those three weeks, wife's car blew a tire too. Gigantic hole in the side of the tire: probably a defect back from when we bought the car that eventually made the tire blow (I kinda noticed it when we bought the car but the official dealership told me the car had been inspected and the tires were fine).
Thankfully it's a BMW with "run flat" tires: even if you blow a tire you can still drive up to 50 mph for up to 80 miles (officially, you can always try your luck too) on that blow / totally deflated tire.
But yup, even something which was as normal as having a spare tire in your car and being able to swap it yourself (something I've done several times in my life: for me and to help others) is a long gone skill.
Infotainment etc impact both equally so it’s clear EV’s will just be simpler than ICE’s coming out in the same year.
This seems like the really hard part. maybe if there was a way to find the time lips moves for a face. or guess gender and age of both face and voice.. or If the audio is a stereo mix, using relative position
I don't think people understand yet that this stuff is inevitable; whether it takes 20 years or 50 years eventually we (or the next generation) will _all_ be wearing these. So why not do it now?
For all the bad it could do, it could do so much more good/be useful. & if people are worried about the cameras on them; they should be worried about the camera that they _can't_ see, like the OmniVision OV6948.
I almost want to try and implement this in OpenCVjs and see if it is possible to build a browser extension to do this in the browser itself
To say nothing of all the myriad nuances of language like context changing word meaning and colloquial meanings of words.
In the game Cyberpunk, when doing the inline translations between english and japanese it is actually swapping fully translated sentences out, so not word for word, the effect just looks similar.
Here you have a technology that would work in a way totally different to the way people assume... But someone put the effort into making it look like it does something else because the skeuomorphic expectations of the human makes that inaccurate representation feel more correct.
That gap between the way tech works and the way our perceptions and psychology drive us to assume it works is very real, and it underpins piles of tech we use everyday, from the icons for making calls and hanging up to whether browsers blank the screen when navigating to another site.
Has our idea of the human/technology interface shifted far enough that the idea of information technology implants will seem a sensible optimization of day to day life? Will skuemorphisms change over time? Maybe the symbol for a neural implant phone call will look like a smartphone (I think it does in Cyberpunk 2077 but I can't recall)
I do wonder if they’ll be more amenable to the 2035 version of Google Glass though.
It's sweeping each sentence at a fixed number of milliseconds per character, set so that both languages finish simultaneously. Look at when "Donald Tusk" appears during the first line. Much sooner in English than in Polish, and both are happening before he actually says the name.
The example from the game is doing entire sentences too.
As mentioned in another post chuck a OmniVision OV6948 or similar in the glasses and nobody will bat an eye because they simply won't know.
I want to jump in, badly. How would one go about picking up the skills needed to create stuff like this? As pragmatically, concretely and efficiently as possible without getting sidetracked in overly theoretical distractions?
I’m a fullstack engineer and have an MS in CS and pretty good math chops, but I sadly only took 1 machine learning course in all of my formal education.
How do I get into this (gpt-3, chatgpt are also on my mind)? Please, any books, moocs, etc
> I want to jump in, badly. How would one go about picking up the skills needed to create stuff like this?
Have you tried to make any of your ideas? If not, pick one and try to make it.
I want to build a benevolent, kind, caring and nurturing AI that one can go to for learning, understanding and support in navigating this world and being human.
I want to dump key knowledge into it that I have been fortunate enough to discern during my lifetime and knowledge from others that I regard highly. Kind of like what characterai is doing, but highly fined tuned and customized.
What skills do I need to do this? What tech do I need to learn? What books, moocs, etc. or even just words to look into? Just looking to be pointed in the right direction, I’ll take care of the rest.
Definitely not trying to reinvent gpt3 or anything like that, but looking to leverage these tools and customize and fine tune them highly
In a sense instead of training a machine you could steer yourself to become that person. Call it being a life coach or a pastor. That's a lot less hassle than actual, true artificial intelligence (which needs to be invented as of yet).
I'm genuinely curious, not trying to piss on your parade or anything.
Here is my thinking, can’t be too elaborate because I’m typing on a phone keyboard, but here it is:
We live in the age of information. The amount of information out there is limitless.
This being the case, the importance lies in how a human being interfaces with this vast amount of information and which information out of the vast ocean of information is presented. Therefore curation is important. User experience is important. Discernment of what is actually valuable for a human being is also important. And how the user interacts with it is also important.
Chatgpt is one form of crafting this user interface/experience. There are other ways to go about it. The feel of the interface itself, how it presents itself to the human, and what exactly it presents are all key.
This is very different than say, an encyclopedia in which there is no discernment/bias of what is actually important for a human to know + very bad ui/ux.
And also, most importantly, a human will die. An ai model will not. A human can only speak to one person at a time. An ai model could speak to countless.
"Information" in the current sense (stats, news, (inter)national "politics") is a commodity that's being generated by the boatloads. Just because there's lots of it doesn't mean it's valuable in the sense that it actually pertains to you and your life.
So from my perspective you have the right ideas but wanting to satiate others with "information" is a bottomless pit (again, by design).
The information that is actually actionable is still right in front of us (at least in the west): it's local media (newspapers, radio, not TV). The slow, boring, local, tedious stuff. Apart from that it happens in associations, sport clubs, regional politics, NGO's.
For that, you don't need an AI.
Just fyi: I'm a doomscroller and news junkie so I'm not in a position to judge you. :)
Humans need to get information/transmission of a few key points that leads to their own wisdom and discernment flourishing.
This bit of information would be built into my ideal ai, and it would convey it to human beings in the most beneficial way possible. It would even be built into its character.
And humans need a supportive and nurturing friend for the rocky road that is life.
Those are my intentions
By the way, the benevolent ai would advise you to stop taking in so much new info every day (doomscrolling, too much news), as it leads to disorientation, chaos, confusion, and discombobulation in one’s mind :)
We are also trying to do something similar[0], but still a lot of work remaining. Idea is to allow real time processing of any video using just a CPU. [0] https://www.youtube.com/watch?v=E7UPj9blnWc
Some of the students have built similar systems, for example chaining Whisper and ChatGPT or translation or sentiment analysis of transcribed text, such as here (transcribe Swedish and tell me the sentiment of the text): https://huggingface.co/spaces/Chrysoula/voice_to_text_swedis...
To me "putting all the work on remote machines you don't manage" seems pretty cloudy, and I naively expected "serverless" to mean something like runs locally, not "uses a shit town of someone else's servers".
What did you use to make that entity diagram?
edit: answered below
Think of it as YouTube's auto-generated captions, but on a browser level.
Coming soon to a phone near you, and in realtime.