363 karma · joined December 2, 2022
i have run through a bunch of tests: re-writing vvenc with assembly kernels, creating the first generation agent harness integration with opencode, porting TS npm modules to C++, porting an entire TS server app to C++, creating a new pure io_uring http server with zero-copy (325K RPS single core), creating a second generation agent from the ground up in C++, setting up a dev environment for custom kernel development on tenstorrent accelerators using tt-metal and ttsim.
i consistently get 98.5% input cache hit ratio. i do see noticeable degradation in performance in the 400-500K context range, so i always try to wrap up sessions by 500K max.
a non-intuitive thing is that the model is very good at low-level systems engineering. i suspect this is because they are internally using it to port their stack to huawei hardware. it can churn out exceptionally complex low level C++ stuff that blows your mind, and then completely choke and run in circles on other seemingly simple tasks.
i only use flash and not pro because i want my tooling to be portable to open weights models that are practical to run. i use deepseek platform and not the open weights models for development, because it is subsidized, and based on observation, i think it is highly likely that they are running some proprietary features on the platform which are not in the open weights model.
it will be very interesting to see what their next point release looks like. the compounding effect of optimizing inference cost and then feeding back inference into training should lead to rapid and accelerating improvement, but only time will tell.
It is a next token prediction function and it is important to understand the technology accurately based on what it actually is.
What is unique about a next token prediction function though is that every computer program is just a string of instructions. At the theoretical limit a next token prediction function can generate the entire instruction stream (boot loader, OS, application) so a next token prediction function can theoretically generate any computer program, which means that it is a universal predictor for anything that a computer can simulate. Still not AGI/ASI in the woo-woo non-technical interpretations of those terms, but incredibly powerful.
An important consideration here is that velocity is not zero sum. If you are delivering in weeks what used to take months you are creating an entirely new realm of what is possible to do with software within a corporation.
In the real world, I have never worked for a company that doesn't have a huge backlog (either tracked or in engineers heads) of work that would never be done because it wasn't economic under the old model. This tends to apply to the internal work of engineers (developer tooling, infrastructure, tech debt, etc) more than anything else. 10X faster doesn't necessarily mean shipping 10X product code. You can use that productivity boost to accelerate prototyping, ship betas faster, move the iteration loop faster, all while shipping higher quality code with less tech debt and having the time to continuously improve the engineering side of things that the business never sees.
This is important to understand. I have been coding since I was 11 when I got my first C64, and it is a genuine passion for me, but I also love working with LLM tooling.
One of the biggest things for me is that after decades of sitting in front of a computer I have chronic back and wrist pain that makes it impossible to do the long deep focus sessions that were normal when I was younger. Using AI tooling to handle all of the procedural tasks (running tests, debugging, managing git, etc) dramatically reduces the physical strain of programming, and allows for a much healthier workflow, with regular short breaks.
The relative scale on visits here doesn't make any sense: TikTok 306M, Facebook 90M, Twitter 759K, Instagram 749K.
This seems like marketing for a snake-oil bot detection product masquerading as a political hit piece to get attention.
while i like this idea in theory, in practice the energy efficiency and lower electricity costs of newer hardware mean that in terms of both cost and environmental impact it would probably be better to recycle the old hardware and buy something new in most cases.
This is completely false. For 2022 fiscal year revenue in commercial was $25B vs $23B for defense and losses in commercial were $2.3B vs $3.5B for defense.
https://www.boeing.com/resources/boeingdotcom/company/annual... (Page 58)
a really large company can waste hundreds of millions of dollars papering over the inherent deficiencies of the architecture but it is an exercise in building additional stories on a house where the ground floor is made out of cardboard that happens to be on fire. soa was created purely for business organization needs. any technical justifications are post hoc rationalization. from a technical perspective it is pure trash.
a much better architecture is to keep services but have them all built on top of a single monolithic db. at scale the monolithic db can be a facade and then you disaggregate the database into horizontally scalable services so that you can scale your monolithic db facade to whatever you need.
if the virus crossed over from animals to humans then it must be able to cross back over. viruses evolve quickly, but animals don't, and we have plenty of samples of the original virus. all you have to do is find the right species and infect them with a sample of the virus and show that they can spread it to prove natural origin.
the only scenario where your theory could be true is if the origin species suddenly went extinct after starting the pandemic, which is very improbable.
a probability statement is not wrong just because the less probable thing turns out to happen. lab leak of a non-natural virus that was created through purposeful gain-of-function using transmission is currently the most likely scenario based on all available evidence. even if natural origin is proven, my probability statement will still be correct. i dont discount the probability of natural origin. it is just very unlikely now.
of course it is plausible. a priori there is no reason why lab leak is more likely than natural origin. it just comes down to the evidence of what actually happened.
what you need to show natural origin is transmission in an animal population with an animal that can be linked to the outbreak location. that hasnt been found after 3 years of tremendous effort.
everyone associated with the wiv has millions of lives and trillions of dollars in damages on their heads if it was a lab leak so these people are definitely highly motivated to prove a natural origin. it has been 3+ years and nothing has been found. the more time that passes the less likely it becomes.
but finding an animal that can spread covid with a plausible story for how it cross over to humans in wuhan is a threshold that has not been crossed yet. if anyone could meet that threshold test it would be treated as proof of natural origin.
but supposing that it was a natural virus that came from an animal in the lab at wuhan then it would be very easy for people with access to that information to identify the natural source, and since that has not happened, it means the virus is either from a natural source that was not in the lab, or it was created in the lab.
the longer time that passes without finding a natural source outside the lab the more likely it is that it was created in the lab.
lab leak is dead simple to disprove. all you have to do is find an animal that you can infect with covid and will spread it and you have your proof. many people have been spending a lot of money for years trying to do this and the longer they continue to fail the higher the probability of lab leak becomes.
The WIV lab was built by the French with money from many different countries and the research was being done by EcoHealth Alliance with US funding so blaming it on the Chinese is probably not the whole picture.
EcoHealth alliance made a grant application to DARPA and DARPA turned them down because it was too risky but it seems pretty obvious that they got the money somewhere else.
https://theintercept.com/2021/09/23/coronavirus-research-gra...
[edit] adding here that "Intentional Infliction of Emotional Distress" is a real thing in US law. so, legal remedies do exist.
https://www.findlaw.com/injury/torts-and-personal-injuries/i...
the most popular services (facebook/instagram, twitter) are the ones that have the most real people posting stuff under their real names.
those companies continue to allow fake users for a variety of financial reasons but the fake users are actively degrading the experience.
twitter is working on KYC now, but it is a cheap AI version that is easily exploitable, so it won't make much of a difference (as i understand it so far).
Instead on censoring speech why not provide tools that lead to better speech? This is perfectly possible from a tech stand point but companies would rather have the power that comes from censorship.
1. Get rid of anonymity/pseudonymity. Free speech is a right. Anonymity is not. If people had to post everything in their real name you would get rid of all of the fake b.s. and people would be much more civilized in their speech.
2. Provide content filtering tools based on user voting with a public record of user votes and the option for people to turn off the filters so that they can see what is filtered.
With these 2 mechanisms you can create a public square with absolute free speech where hate speech is filtered and suppressed, if people choose that, but the records are clear and people who want to audit the filtering can do so as they please.
"Everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers."
https://en.wikipedia.org/wiki/Universal_Declaration_of_Human...
But just because governments have recognized the right does not mean that they are complying with this recognition.
This is par for the course though. The 1st amendment was ratified in 1791 and the Alien and Sedition Acts which violated the constitution in the most blatant way were passed 8 years later.
https://www.archives.gov/milestone-documents/alien-and-sedit...
Governments never give freedom. People take it.
transpiling to C actually makes adoption much easier. if you create an entirely new compiler auditing and verifying that takes many years and tons of money before it can be used for anything serious.
if the brain is actually a biological neural network then this is not surprising. input takes time to propagate through the layers of the network and sleep is a necessary part of how that biological process works so the information needed to solve a problem might take a nights sleep to propagate to the depth where it can be solved or it might need a variety of other inputs that take time to acquire before enough information is ingested to reach a solution.
i would strongly disagree. when you are training a model you are taking the information from a document and extracting the relationships between tokens and storing that information conglomerated with the same information from a massive amount of other documents. the model that results is a compressed form of all of the information from all of the documents where you have extracted and stored a synthesis of the relationships between the tokens in all of them. this is a lossy compression, but it does reproduce exact sequences of source documents in some cases, so the original information is stored there.
you can very plausibly argue that an LLM model trained on copyrighted material violates the copyright on every single copyrighted document that was fed to it.