It’s not reasonably possible (currently?) to get the same performance from a 7 billion parameter model as a 175 billion parameter model with just an additional 6000 lines of finetuning data.
964 karma · joined September 19, 2017
There are 2 sources of randomness:
1) the random seed during inference
2) the non-determinism of GPu execution (caused due to performance optimizations)
This is one of those things that humans do trivially but computers struggle with.
If you want randomization, ask it the same question multiple times with a different random seed.
Flax+Jax+OpenXLA seems to finally be building some momentum so when a big player launches yet another competitor, the justification for it would be a good thing to see.
What was “not good enough” with Jax? Why did it make sense to put this human time and energy there instead of doubling down on Flax/Jax/OpenXLA? How will this move the needle at all against Nvidia?
I guess the fact that Google is pushing OpenXLA is making the other giants not want to truly lean in?
I don’t want 10 competing “choices”. I want one clear, open, competitor to Cuda that works on all the competing hardware.
https://www.atlantamagazine.com/great-reads/no-accident-insi...
> it meant GM had produced a part that failed to meet not only its own internal minimums for torque but also basic federal safety standards. For all safety-related defects, NHTSA requires auto manufacturers and part suppliers to notify the agency within five business days after learning of a flaw that could lead to an accident. GM had not.
> The documents showed that engineers first noticed the engine stalling problem way back in 2001, four years before Brooke purchased her Cobalt. What’s more, the Cobalt wasn’t the first GM car with a defective switch; that would be the 2003 Saturn Ion. Despite internal probes over the following decade that recommended changes to the key system, no changes were made. Indeed, GM appeared to double down on the bad ignition switch, integrating it into other compact cars, including the Chevy Cobalt, Chevy HHR, Pontiac Solstice, and Pontiac Pursuit.
> As it turned out, DeGiorgio’s 2006 order came four years after an email he wrote to the ignition switch manufacturer, in which he ordered the company, rather than fix the part and incur delays, to “do nothing.” The email was signed “Ray (tired of the switch from hell) DeGiorgio.”
> The report also described the disturbing pattern of disregard for the defect. This could be witnessed in gestures such as the “GM Nod”—in which GM employees affirmed the need for a plan of action, but understood no one actually intended on following through—and the “GM Salute,” described by one employee as the “crossing of the arms and pointing outward towards others, indicating the responsibility belongs to someone else, not me.”
SpaceX will not remain constrained by launch cadence. Also, very Kuoper launch would make money for them funding further Starlink launches.
Space X is the one direct competitor. You never find your competitor.
This misses Amazon’s strength - AWS. Launches are a fraction, granted a non-trivial fraction, but still a fraction, of actually running a space-to-ground network.
Who’s going to have the best latency to your enterprise infra in AWS DCs? Offer you “free egress to satellite links”?
The US military will want multiple providers.
List goes on.
Eg: https://www.intel.com/content/www/us/en/developer/videos/opt...
https://www.intel.com/content/www/us/en/developer/tools/onea...
https://developer.apple.com/metal/tensorflow-plugin/
Large scale opensource is, outside of a few exceptions, built by engineers paid to build it.
Has already happened: https://grahamcluley.com/android-security-fortnite/
From just this week: https://news.ycombinator.com/item?id=37600852
Just because someone built a fantastically functional building doesn't mean we can't criticize their choice of foundation. Case in point: Millennium Tower in SF: https://www.nbcbayarea.com/investigations/series/millennium-...
The license explicitly blocks “Hosting Use” and does not even differentiate between “model as an API” and in-application use.
I hope the community rejects this kind of BS.
This is what Cosmos DB for Mongo DB is (wire format endpoint that talks to Msft’s own DB backend.)
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/in...
The whole system is built around phone numbers. It puts the end of the chain of trust in the hands of the corner store vendor not accepting a fake photocopy of an ID to swap SIMs. It’s ko different than when South Korea mandated ActiveX for browser security for banks.
Let’s not even get started on the whole Unique Identification Authority (UIDAI) joke. India doesn’t have Apple Pay because it’s illegal to have device-local biometrics - you must store it and verify it with the government.
UPI is also an extremely shortsighted system throwing away any semblance of privacy for the population. The whole thing doesn’t have fees partly because sharing every bit of transaction data with 7+ parties in every transaction is how it is funded.
Phone lines have always been part of a functional office. Not sure how you can argue that communications (voice, video, screen share) is not part of a modern, WFH enabled office suite.
300M hires a lot of engineers. Maybe the fact that they are buying instead of building tells you how hard it is to build and how strong a moat Nvidia has.
Musk has publicly said that the true test of their own training hardware efforts is if the ML team switches to it. I.e. it’s a software, not hardware problem.
The web is already FULL of bad medical information. Older folks treat it as gospel already. All this technology does is make both good and bad information more accessible.
More importantly, there is NOTHING anyone can do to stop this. Nothing the US government can do will stop the proliferation of AI models that do everything including giving medical advice (or other things that we know it will screw up)
"Dr. P<hil>" is still on TV.
I take a pragmatic view on these things - assuming appeals to morality and ethics will fail, what do we do to educate people about this?
What is the moat? "We will run your inference" can't be the answer.
Developer time is probably your most expensive resource. Saving $8/mo at the cost of dev time seems… suboptimal.
Every piece of code you write is security-sensitive. IMO It is our professional responsibility to our users that we treat it that way.
For a vast majority of situations, even if you know what you are doing, you are human and therefore will make mistakes.
If the (probably) most heavily invested C++ codebase, Chrome, has memory use-after-free/overflow bugs, there is no way that truly safe C++ can be written at any product-scale.
The set of reasons to start a new project in C/C++ are few and the list is shrinking by the day.
> Starting on March 1, 2023, we are making two changes to our data usage and retention policies:
> OpenAI will not use data submitted by customers via our API to train or improve our models, unless you explicitly decide to share your data with us for this purpose. You can opt-in to share data.
> Any data sent through the API will be retained for abuse and misuse monitoring purposes for a maximum of 30 days, after which it will be deleted (unless otherwise required by law).
> OpenAI will not use data submitted by customers via our API to train or improve our models, unless you explicitly decide to share your data with us for this purpose. You can opt-in to share data.
> Any data sent through the API will be retained for abuse and misuse monitoring purposes for a maximum of 30 days, after which it will be deleted (unless otherwise required by law).
There is a ton of boilerplate around the actual model server that’s just busy work , but if done wrong can be a huge performance suck. Solve that.
Build the proxy that works with the most model servers out there. Do it in a way that once you have mindshare, the model server makers will be find it easy to put up a PR so that they can claim your proxy supports their server.
Don’t take a hard dependency on non-OSS stuff - being able to build an “on-prem” solution (read “deployed into customer’s VPC”) is table stakes for anyone to use your offering for a lot of enterprise use cases.
Edit: another unsolved problem - different models need slightly different prompts to solve the same problem well…
Most production use cases probably need some sort of fall back of small -> medium -> large -> GPT4.
Given the costs + low quota limits, I would be surprised if any significant portion of the market is falling back from one expensive proprietary* API to another.
With the rapid improvements in model servers like llama.cpp and vllm.ai, providing an abstraction layer for “fastest model server of the month” would be useful.