Then this is SD for Apple silicon users. 13B runs on my m1 air at 200-300ms/token using llama.cpp. Outputs feel like original GPT-3, unlike any of the competitors I’ve tried. Granted- non-scientific first impressions.
I understand azure has its own chatgpt api.
IMO, Apple's habit of cheaping out on ram is maddening. Although perhaps this is a sign for me to pick up a m2 pro mini with extra ram.
The Base mac studio with 32 gigs of ram can be a good value from a ram perspective.
Let’s see if the Mac studio is discontinued or updated.
I do hope that it's updated. I could see them reserving the top end chip for the Mac Pro, though. Or maybe make it a dual socket?
In any case, I much prefer the ability to run an Apple computer with a proper cooling setup. I know that cooling pads for laptops exist, but ultimately it's a bit of a janky solution compared to actually adequately cooling the system properly.
The Mac studio is surprisingly quiet and well cooled. I haven’t had to install a fan control utility as of yet.
I thought I’d be trying one of these out and sell it if my setup didn’t work, but it’s the first desktop I’m starting to consider especially if I want to leave a workload running on it instead of a decade of carrying a lot of horsepower around with me.
Instant on for 3 monitors, no need for docking station or USB hubs, everything is plugged in and works, I can run a virtual cam obs setup easily.
The studio is a great upgrade on Mac mini, which might cannibalize the Mac pro sales and why it might not get toasted or refreshed. I’ll F the max studio was a way to put a dent in hackkntoshes, it makes sense.
The M2 mini looks super decent but quickly goes up in price when ram and ssd is setup to match a studio.
For my purposes the ultra wasn’t that much faster than the M1 max on because the software has to be optimized to benefit from it. Maybe some of this ML stuff will be shortly.
Apple's optimization here is nice for the people who own their hardware, but it's totally silly to read through the comments promising the end of CUDA.
I think there might be a different use case for me but to compare with your economics - I picked up a base mac studio (m1 max with 32 gb) for about $1370 USD with 8 months warranty remaining. It’s letting me test my daily setup to see if everything can run on Apple silicon yet without intervention.. if not, I’ll be able to get rid of it at little to no loss, and decide if I want to carry that much computing power in a laptop or not, or head on to other options.
My interest currently is computational power, per watt. Not much comes close to Apple Silicon.
The cost of a loaded pc can quickly outstrip other options with electricity costs included so it needs to be useful in any case.
The integrated speed of the Apple silicon, ram, and ssd is a little astonishing. More than I expected to admit. I don’t know if there’s anything like it on PC. If Apple silicon supported eGPUs it would interest me.
Comparable PCs have described are generally power inefficient. Still, the system you’re laying out is interesting, especially the ram, mind sharing a bill of materials?
I guess my PoV is with a GPU + inexpensive RAM, I can do some things more quickly, and other things are at least possible. But granted, hardware nuts and bolts is a hobby for me too, I've always enjoyed researching and building systems, there is a huge community of people and companies, and I much prefer KDE to MacOS, Android to IOS.
If Apple released a reasonably priced system with lots of RAM, or that could take a GPU, I'd not have as much to stand on, but they don't seem inclined to do that.
Here's BoM, $2170:
$700 for 192GB https://www.corsair.com/us/en/Categories/Products/Memory/VEN...
About $200 for a LGA1700 board with four slots (currently Asus and Gigabyte support 4 X 48GB memory modules on consumer hardware)
$200+ for a CPU
$120 for an 850 watt PSU
$700 for a used 16GB quadro, or a new 12GB A2000
$150 for 2TB NVME storage
$50 for some kinda case (I use an XTIA open frame, it is the opposite of a Macbook)
These are "best prices," You'd have to pay around 10% more if you're not into finding deals.
If it’s 400-800w 24 hours a day, op costs quickly pile up.
It looks really reasonable for everything here. I am interested more in eGPUs for interchangeability but have enough on my plate.
The GP's info is outdated. Apparently they don't realize how quickly LLaMA is being developed on. It's just like the early days of SD.
running 4-bit LLaMa 13B on my MacBook Air M2:
"possible resolutions to the Fermi paradox, the question of where is the evidence for extra-terrestrial civilizations: - "
answer:
possible resolutions to the Fermi paradox, the question of where is the evidence for extra-terrestrial civilizations:
- 1. We are looking in the wrong direction and in the wrong frequency bands for signals from intelligent life in the galaxy.
- 2. We are looking in the right frequency bands, but the technological level of intelligent life in the galaxy is not high enough to be a threat to us.
- 3. We are looking in the right direction and in the right frequency bands, but the probablity of interstellar travel is so small that we are not likely to meet civilizations.
- 4. We are looking in the right direction and in the right frequency bands, and the probability of interstellar travel is not small, but we are unlikely to meet civilizations at our level of technological development.
- 5. We are looking in the right direction and in the right frequency bands, and the probability of interstellar travel is not small, and we are likely to meet civilizations at our level of technological development.
- 6. We are looking in the right direction and in the right frequency bands, and the probability of interstellar travel is not small, and we are likely to meet civilizations at our level of technological development.
I would be inclined to think that option 5 or 6 are the most likely. Given the nature of the universe, I think that we are more likely to encounter civilizations in the earliest stages of their development, than those that are further developed.
I would think that the big question of the next 10 years is whether SETI will be able to detect an ET signal. I think that the chances of being able to do so are less than even.
The reason is that a lot of the technical issues that SETI has been addressing are becoming less of a problem, because the Earth-based radio telescopes are being joined by many smaller radio telescopes that are being sent into space. So, from a technical standpoint, the ability to look for ET signals is becoming easier and easier.
The problem is that any signals that are sent to Earth that come from technologically advanced civilizations are likely to be only a few percent of the power of the ET signals that are coming from sources at our level of technological development. So, for example, SETI@home is looking for