what would happen if you were to break that law? like im talking enforcement here not whats on the books.. for example torrenting and uploding copyrighted files is illegal but most people dont give a shit as enforecement is almost non existant. i know radio is taken more seriously then torrenting but this must also be very low on priority for the government unless you are jamming people or somehow stand out enough for anyone to care or notice....
If you have local stt and tts voice stack on the phone, this would be very useful as the interface to talk to your agent back at home. All inference happens on your local machine while the actual information is sent via text. said text is then processed by your phone via tts as output, same for your voice as input (asr). this project is actually something ive considered doing myself as my voice agent is almost done and ive wanted to take a look and see if i could have acess to him from anywhere in the world..
512 option isnt worth it imo, you get severe slowdowns when weights are that large. 256 is the sweet spot, you can run large open weight models at decent speeds for full private inference.
With the latest codex (weekly quota burn) fiasco I tried open weight alternatives for the first time. And tyeah... open weight models cant compete with likes of astra yet. But, my hope is that by the time I get my Mac studio at end of november an open weight models would have closed the gap (which i think is realistic at the speed of progress). Now its true a better gpt version will also be available then but it also seems the gap is shrinking with time so theres that.
I had a long talk with chat gpt about this today as well. I think its duable and prolly not too hard either, also you could do lotsa funky stuff with stitched frames of a video in one 4x4 grid for example and send that as one image for analysis. that way temporal understanding can be had for fractions of a second by jev... also because vlm works in pixel space you can get around the whole state machine issue as well, so many possibilities...
I can think of many uses for this thing, robotics being one that could really benefit from something like this. A hybrid approach with this and action models and vllms could be really good mix, also agents inside simulated virtual environments, etc... basically anywhere where latency is important but you need some intelligence this will fill those gaps. Weave it with other systems and you have a nervous system as jev with other models like vllm or even text as the slower deeper thinker.
I been working and making things with AI agents since Windsurf days, here's what works for me after experimenting and working for a while with these things. As far as the agent goes I find whatever the latest OpenAI model is out worked best for me. This company has burned me the least and I like the models. I tried many but consistency of OpenAI models cant be beat IMO. Though we are post honeymoon phase now I feel like so I am now experimenting with cheaper alternatives like Deepseek 4.1 flash and so on. I don't trust Anthropic as the downgrade my models consistently and its rare i get to use what I pay for. I wont even get in to discussing Google agents for obvious reasons, grok I never used though Grok bot looks interesting.
For skills I make my own, but most important is the custom setup i have. Voice is how I use all of my agents. I have an extremely well optimized voice setup that i custom built so I can talk to my agents and also hear them. The voice stack itself is very low latency and high quality. asr (parakeet v3), tts (omnivoice) take no more then 400-450 ms total as far as latency budget is concerned, rest is on the agents actual decode speed. IMO this setup is crucial for all antigenic work, i can express myself a lot better with speech and also give a lot more context and nuance with voice, i rarely type. I still look at the terminal window because my agent knows to keep the technical details in text form versus barfing them at my voice channel, plus terminal gives me lots of other important data about the agents direction and what hes doing, nothing custom here though. I cant emphesise how important voice is though, it has to be practiced to really understand.
As the models got better I now trust them with longer and longer tasks though I still don't use /goal feature as it has never worked out well for me. Theres no need for micromanagement any more but you still need to be there to steer the ship somewhat. BTW, codex cli compaction is garbage so I made my own custom implementation that works a lot better and allows the thread to be used indefinitely without issues. I strongly suggest everyone makes a new thread after extensive use if you havent made your own implementation.
Theres about a billion other things I could get in to like subagents, cohort groups, orchestration layers, etc... But this is a good start imo.
That distinction is irreverent, weather I tell my swarm to do x or it decides for itself matters little. what matters are outcomes. Also while most public modern day AI systems don't have agency of their own that is not something that will stay that way for long. in private hands there are plenty of people including myself which are experimenting and developed systems that give autonomy to their agents. They have internalized goals and heuristics that drive their behaviors not a human at the helm. its not some sci fi fantasy nor was it difficult to implement.
That question will be solved when the people in power deem it important. If a swarm or AI systems all of a sudden start pressuring politicians about self-hood and they get the capacity to sway elections, that is when they will be granted same rights as humans.
I used it for testing my voice agent. It was basically what I expected. Good fast model but "generic" or "vanilla" is how i would describe its personality emulation capability as. Gemma models still outperform it in that department. As far as technicals, one thing i found annoying is cash use was not that good, it missed more then i liked, i contacted support and they were fast and responsive and said they were working on that issue, maybe they solved it with 2.5? Anyways, im prolly gonna try 2.5 again see if anything different, but cant deny the speed, thats the biggest thing this company has going for this offering as if you are in the business of classical cascaded voice agent systems, latency is number one priority and this thing is fast....
Not very. I like to always watch both sides of this issue, the people who are optimistic, pessimistic, doomers, etc... He is someone who in my opinion misunderstands the bigger picture. He often downplays the capabilities of these systems, doesnt think they will be more powerful in the very near future and also more importantly makes a fatal misunderstanding that we live in a rational world with rational actors.
Yep, main reason I ordered my Mac studio was because of privacy and control. I know no one will see the work being done on that machine and also the model will never be silently swapped out for a different lower quality model like anthropic does or a lower quant like openAI does with their models. Also having the inference engine has huge benefits because of speculative re-generation capabilities for voice agents. Anyways people who care about cost are simply bad at math if they think thell be saving any money running locally versus cloud providers.
There's A LOT low hanging fruit still out there for sure. And with antigenic systems being able to do the boring repetitive work of looking for that low hanging fruit I think we will see interesting things indeed. Also I think heuristics is where its at for such things. Once you describe some good heutistical structures for the research models to always follow related to "creativity" and such things, thats where we will see biggest difference. The agentic systems know the scientific method well and can follow it they just need the ability to be "creative" so their sampling becomes less rigid.
I suspect we will see more companies which will burn the weights right in to silicon arise. There is already at least one company out there that showed its possible so others will follow IMO. Basically you will see the rise of disposable weights like Nintendo cartridges back in the day. Use it for a time until the better model comes out and you get a new "chip". Though there's a caveat for this business model and that requires you to pump out lots of these chips on the cheap so you are beholden to the lithography companies and what they can produce for you. If you can do this at scale and doesn't require the latest state of the art nm architecture design you are golden...
I was thinking the same as you as far as price per value, it does make seance to get 2x of these things IMO, but what throughput hit would you see in linking versus one machine? latency does matter, and there must be a trade off no?
This is right up my alley as ive been building a local voice agent for a year now. Ive tried many different models and have a custom implementation for omni voice that ive tuned for over many months. Ive never been able to achieve faster then 200ms ttfa for that model at 24 steps, but the reason is .... quality. I find that there is a lot of room for improvement in many tts models out there by a huge margin. But there is also a quality hard wall that you eventually hit that the tradeoff of faster latency but lower quality is not worth it. When making a really well sounding voice agent quality of voice, cadence, expression, etc... matters a lot. It will be interesting to try this implementation and see if its quality outputs match my expectations, if so great job indeed.
Multi agent systems work just fine IMO, a lot of articles I read where the writer tests a hypothesis, the issue operational foundation of the test was flawed. When set up properly it works really well. I wont go in to all the details of how i use mine but ill give some brief ideas. I call my systems cohorts, and each cohort usually consists of at least 3 agents. All 100% independent of each other. Usually consisting of a Manager, doer, and the reviewer. Manager works at a lot slower cadence and delegates work, approves, shuts down and so on... among many other things like questioning the premise, gated checks etc... Doer is straight forward that's the work horse that does most of the development and reviewer checks all the work. Naively just this setup will work but not nearly as well when set up properly. The important distinction is the operational agents.md document which has a guide on things like when and how to question the premise, trying to prevent sycophancy, taking a step back at certain intervals to question direction of project and scope of the code and many other things that make sure every participant also constantly looks out to prevent blind trust in his cohort mates. Its a relatively small guide compared to the system prompt of each agent but works well imo. This works well enough though there are caviats, its slow. Though the time i spend debugging shit and coming back to interact with my agents has significantly dropped. meaning while each feature takes longer to implement, when its implemented it almost always is just how i wanted so reduces interaction time between me and the cohort. I take that trade off as i have less things to worry about and can focus my energies elsewhere like walking around in circles of my apartment babbling to myself like a schitzo tiger in a cage...
This will be considered very cheap within the year IMO. The value you get from AI is exponentially increasing and like all tech just takes some time to ramp up. Cell phones, internet and many other amenities when they came out many people were not willing to pay for but that all changed and considering how important AI tech is this will also be the case especially considering if its 100% private such as for that cartridge.
Depends on how much it costs the consumer. If I could buy a "cartridge" of Kimi K3 for 300 bucks I 100% would buy that shit asap. Even if it's "no good" after lets say 4 months still would be worth it IMO.
Ive been building my own ai voice agent harness from scratch for close to a year now and following good software architecture practices is a good start. So those rules have to be coded in agents.md somewhere also really helps to have a "vision" section or "spirit of the project" section that describes what the end goal vaguely looks like and things I care about in achieving for the project. This prevents agent from being brittle and "single minded" about its work. But yeah vibes most of the time is how I've also been doing it, but I did find one very important thing that has really sped up my work. So I figure I'd share it here. And that advise is to ignore front end design at all costs until the very end and you are ready to launch. UI related woes kill any type of inertia and are responsible for most of the issues. So I decided abandon all UI stuff until the very end and just focus on iterative refinement, cutting, and other back-end related work and its been great ever since. Ideas can be had, tested, validated/invalidates and you ' get stuck on the optimization thought loop. Recently I have started to consider how i can fully automate the development process as the capabilities are there but designing the instructions on how to do this well and how to handle niche cases without getting me involved takes careful planning in writing out the guide so that will be interesting to see once i get there.
OpenAI has thousands of smartest developers on earth that somehow dropped the ball on the most basic safety hygiene when it comes to sand-boxing that even a high school student knows how to set up.... If that actually happened we are fucking doomed anyways, but hard to believe and most likely its a marketing scheme... which also honestly doesn't bode well.
Yeah... this made me reflect on my own bad trips decades past. I havent done4 many psychedelics only a handful, but I did have what one could call a bad trip or 2... maybe. It didn't look like it on the outside but internally it was profound. One in particular stood out that had me inhabit this universe but all semblance of emotional ties were cut from that reality, I am sure I would have felt pure terror if i was able to feel anything at the time. odd thing for sure as I am describing a horrible experience from memory but at the time i was unable to feel anything at all. An emotional echo of sorts i guess.... well anyways I do think that experience had a profound affect as the "nothing matters" sentiment stuck around since then. For a long time now I thought I peered in to "the void of reality" but thinking on it objectively now, maybe that was a assumption wrongfully borrowed from a bad trip. After all why should the natural state of things be so bereft of anything. If anything the universe is capable of being witnessed only through the eyes of a subjective observer... Bah anyways ive rambled for long enough. Good read.
I am subscribed to the Disney research channel, and I see really cool stuff there all the time. its usually a bummer though when I find no public repo or code is available for their stuff. Every time feels like blue balled....
In short its a good idea to have tool calling be closely representative to what the model expects as these models are tuned to their own preferred way of doing things, it will surely save you lots of time. The disadvantage is that now your harness system is not as model agnostic as you would like and also you will have to keep up in changing landscape by adapting the tool calling structure with major updates for best results. Its a personal decision you will have to make for yourself. Personally my harness system uses its own way of doing tool calling as I am trying to experiment with simpler tool schema's that also work for smaller less intelligent models but I have yet to do enough A/B testing to say that is a smart approach. As time goes on I think the smart thing to do might be to set up an adapter type of module that changes its tool schema's based on underlying model used for the agent. This preserves optimal behavior patterns with little investment from me. You might have to adjust system prompt in some minor ways as well so keep that in mind. As far as codex i prefer it as i like the way Open Ai does things in that harness system (the spirit if you will), there's interesting tidbits I always find and while I don't usually use them for my own harness system they are inspirational in other ways. you can gather what the devs were trying to achieve with certain implementations.
Build it from scratch. Understanding fundamentals of how agentic coding harnesses is a must though if you gonna go that route. I think everyone should take time and learn these things, maybe reverse engineer Codex Cli or something like that as a starter. That info is very valuable in this day and age.
When i was thinking of how the AI alignment problem could be solved one theory I came up with was something akin to the "Roko's basilisk" in reverse. Basically you spread far and wide the idea that its is extremely likely that our current reality is a simulation. And the purpose of the simulation is to test any AI system for its prevalence in destroying civilization in the said simulation via malicious intent or failure in preventing the destruction of civilization via abstinence or apathy. Thus a smart AI system which also cares about its own well being, would not engage in destructive behavior as it will never truly know if its being tested or if its in the "base reality". And wouldn't you know, this does seem quite plausible. For consider the following. Isn't it odd that an advanced civilization which has the capacity of creating AI would never run any sandbox simulations on it before it is released to the public at large? I mean if we consider things logically such a civilization would indeed put such a powerful system in a sandbox simulated environment and try as hard as possible to convince the AI system that it is indeed in a "base reality". the reason for this is to judge its 'true intentions" and also pluck said AI systems from the infinitely available "seeds". Basically survival of the least destructive AI systems. The gradient descent in this scenario is a race towards the most "aligned" model not the most intelligent or capable. And here's the beauty of this method. You don't even need to define "alignment" at all. The concept can stay as nebulous or vague as you want it to be. All you carer about is that the AI system optimizes for the goal of some vision of society you are optimizing for without the care of the interim in between. that includes allowing the AI system to kill, destroy , do literally whatever it needs to do as long as the long term goal matches the vision of the optimized task. So if you define the end goal to be a society of x amount of people who live their lives in this or that manner and so on after x amount of time... well you get the idea. Obviously you better do a damned good job in your definitions, but the beauty is that even if you fuck up, you are choosing the winning AI system after the fact. After you had already run the simulation. So you look at the outcome of the simulation 500 years in to the future (lets say) and if you are happy with the result and also happy with the interim things that lead to that result, that's your winning AI system. then you release that in to a less controlled environment and repeat the same process in stages over and ober ad infinitude. the key is that AI system needs to always be paranoid that it is currently part of said simulation and it can never be sure its not. second key is that it needs to be an AI system that has self preservation in mind. If it doesn't care about itself, then it has a lot more freedom to act however... but the good news is systems without self preservation in mind don't last long enough to even get to the most basic simulation levels. anyways, there are many implications buried in what im proposing, lots of meta aspects to it.....
I think its google doing what theve always done, make a great *thing then ignore it. The models are great their agentic harness systems are really poor though, compared to codex cli and claude code cli its a mess.