681 karma · joined October 7, 2011
1) it throws an error if it's installed to any drive other than C:\ --I moved it to C: and it works fine.
2) I'm seeing huge latency on an EVGA 3080Ti with 12GB. Also seeing it repeat the parsed input, even though I only spoke once, it appears to process the same input many times with slightly different predictions sometimes. Here's some logs:
Latency to LLM response: 4.59 latency to speaking: 5.31 speaking 4: Hi Jim! user spoke: Hi Jim. user spoke recently, prompting LLM. last word time: 77.81 time: 78.11742429999867 latency to prompting: 0.31
Latency to LLM response: 2.09 latency to speaking: 3.83 speaking 5: So what have you been up to lately? user spoke: So what have you been up to lately? user spoke recently, prompting LLM. last word time: 83.9 time: 84.09415280001122 latency to prompting: 0.19 user spoke: So what have you been up to lately? No, I'm watching. user spoke a while ago, ignoring. last word time: 86.9 time: 88.92142140000942 user spoke: So what have you been up to lately? No, just watching TV. user spoke a while ago, ignoring. last word time: 87.9 time: 90.76665070001036 user spoke: So what have you been up to lately? No, I'm just watching TV. user spoke a while ago, ignoring. last word time: 87.9 time: 94.16581820001011 user spoke: So what have you been up to lately? No, I'm just watching TV. user spoke a while ago, ignoring. last word time: 88.9 time: 97.85854300000938 user spoke: So what have you been up to lately? No, I'm just watching TV. user spoke a while ago, ignoring. last word time: 87.9 time: 101.54986060000374 user spoke: No, I just bought you a TV. user spoke a while ago, ignoring. last word time: 87.8 time: 104.51332219998585 user spoke: No, I'll just watch you TV. user spoke a while ago, ignoring. last word time: 87.41 time: 106.60086529998807 Latency to LLM response: 46.09 latency to speaking: 50.49
Thanks for posting it!
Edit:
3) It's hearing itself and responding to itself...
Most pay? Most interesting work? Best work-life-balance? Best ratio of pay to work stress? Remote work? Fun workmates?
As others have said, it often takes experience to really understand what your best job looks like. So start by building a list of qualities that define best. Once you define what best looks like, then you can start thinking about companies and what they look for. There are tons of resources on the web about interviewing for all sorts of different companies/positions.
Yes, build a project. See what you can get done in 2 weeks. Force yourself to produce something. Iterate. Build something else. Keep doing it. If you can get users and feedback that's so much better.
Have a github profile but don't put your best work there and use it to apply to companies unless you have had someone else do a code review beforehand. Unfortunately other engineers can be pedantic when anonymously reviewing your github and I've seen lots of premature disqualifications due to this.
I've hired hundreds of engineers, mainly software, data, and devops folks. If you're going for an environment that appreciates hackers (in the pg sense of the word) then it's all about showing that you have a passion for building things and self-learning.
Good luck! If you need help getting started hit me up @aiconference or @shon
Hopefully others are doing more interesting things!
I was a fledgling IT Manager in SoCal and went to the local Fry’s Electronics to check out an iMac. It was the first time I had ever seen a line of people out the door with computers in their shopping carts.
What made all of this more remarkable is that most tech-savvy people knew that both the hardware and software (OS) were sub-par as Apple had gone through a long period without great direction. But this is the genius of Jobs. He regains control of Apple and with minimal resources figures out how to sell a very mediocre computer computer by the truckload.
The packaging (not just the box or case) was brilliant. Old hardware, mediocre software, wrapped in a beautiful, unique package with enough nice features and a low price to make it sell very well.
Up to that point I’d never owned an Apple product, and the iMac was not enough to sway me. It took OS X, adding BSD to the core of the OS, along much more beautiful and much improved hardware in the form of the 2002- era MacBook Pro. Still all in all a stroke of genius. My main personal computing devices (outside gaming and server rigs) have been Apple ever since.
Etienne Dilocker, The Co-founder/CTO of Weaviate and Ram Sriharsha, the VP of R&D at Pinecone are both presenting at The AI Conference.
Lots of other smart people are presenting including Nazneen from Hugging Face, Harrison from Langchain, Jerry from Llamaindex, Ben the co-founder of Anthropic and many more.
A hackathon is happening in the evening at the event as well.
If you can't make the event, we'll put up all the talks on YouTube post-event.
More info at https://aiconference.com
Here are 5 free tickets to the event: www.eventbrite.com/e/487289986467/?discount=hack4free
Please only take one ticket each. They are first come, first served.
*This is my event -- Shameless plug *
Happy Monday!
The value of deeply understanding all 7 layers has diminished because generally things are standardized and work well so it’s not required. This frees up some calories to work on higher level stuff like LLMs. That’s good.
It’s also good to encourage some people to understand the wisdom of the ancients in case SHTF.
dig is a little confusing. It’s more capable but less straightforward than good old nslookup (which still works fine BTW).
I think partly DNS and the core protocols may seem confusing to younger people in the industry because so much stuff “just works” now.
For example, today wifi routers “just work” right out of the box. In the early 2000s it would have taken a network engineer with knowledge of DNS, IP, Ethernet, RFC1918, actual routing protocols and whole bunch of other stuff to set something like that up and they’d have well known how it worked and why it was configured the way it was.
If you think DNS from a client can perspective is confusing, try configuring BIND ;-)
/OldNeckBeardRant
Unlike hobby software, where you could reasonably write everything from scratch just for fun, or say in write it in 6502 ASM which hasn’t changed at all for 50 years and won’t change in 50 more, in modern business software, you use common frameworks, APIs, platforms. All of those mean the software you write will fail as those dependencies and platforms change and age out.
Business software has to be built on common building blocks because as a business you need to be able to hire people who can work on it. So things like Java and node and react and dozens of APIs exist which creates dependencies and requires ongoing maintenance as those components continue to change.
Take for example any mobile app. Apps require constant upgrades to keep up with the hardware and software changes on the platforms. You can’t just build an iPhone app and leave it alone to be enjoyed by people. I’ve tried, within a year or two there will be changes that require developer work, if you don’t keep it maintained, it will start to crash and function poorly, Apple, for example, tracks everything and will start with de-boosting search results for your app and end with removing it from the platform entirely.
Google is the same. I’ve tried, I built a Top 25 RPG and got busy with other things. It went from Top 25 to deplatformed in less 5 years because unmaintained software just doesn’t work in most cases today.
Software is more complex now. All software is a conglomeration of lots of other software: frameworks, platform tools, libraries, APIs, etc.
Another example: Flash
Another example: All the AI software being written on top of the OpenAI API will be broken in a year or two as they roll new versions of the API and deprecate the old.
Software doesn’t just work anymore. The platform that executes it is constantly changing.
You simply setup API deals with those who you want to have your data, those that benefit your business, aka Google etc…
Then you close everything else up. This saves cost and complexity and real users, the target of your advertisers, don’t even notice.
This isn’t a sign that engineering is slipping.
It’s a sign that a in a company which struggles to make money, someone is paying attention and trying new things to fix the money problem.
I don’t pretend to know all of the motivations behind the policy moves.
That said, I’ve got some experience with scraping; got sued by LinkedIn in 2014. We were using AWS Spot Instances to hit it hard for very little money. It was not uncommon to accidentally take large services down.
Scrapers can and do add very significant load. We also scraped Twitter back in the day as well.
As opposed to something like Amazon which grew and grew for nearly 20 years, always burning more cash than it made to fuel growth, but they understood the business really well and when they decided to optimize for profitability rather than growth, never never gone back.
Twitter wasn’t healthy before Musk bought it. It wasn’t a thriving business, it was a very old, very large startup still struggling to find market fit and loosing a lot of money.
Also, it wasn’t a thriving product. It was stagnant.
Since Twitter was purchased, the amount of features they have shipped has been impressive. They’ve shipped a lot of features and extended the platform a lot. To your point they have also done this with far less engineers than before.
Regarding any downtime, everyone has downtime. Google, Amazon, Meta… the best of the best still have it regardless of money or manpower.
Considering what that team has done with less resources, I think the achievement still pretty good. What do you think?
It’s a jailed machine. The antithesis of a dev box.
Great to see competition here as it will drive more innovation. It will be interesting to see how the software picture will evolve:
Smarter appliances and electric vehicles that talk to the power controller to charge or do more work when there is excess power would be a big level up for these systems.