mikeravkine

The Internet is 90% mobile users that can't hover.

Thank you for your efforts on behalf of the GPU poor!

It's getting tougher to use older, cheaper GPUs (Pascal/Maxwell) with modern quantization schemes so anything you can do to keep kernels compatible with SM52 and SM61 would be greatly appreciated.

Ubuntu 18 went EOL on May 31, 2023 so about 8 months ago. What CentOS are you running that's still in support and was affected by this change? I had to upgrade all my systems last year after the EOL as most python packages dropped 3.8 support.

The challenge to Mastodon isn't adoption, its absense is a symptom of a deeper issue: it's a Very Bad Idea to trust random server operators with both your data and it's sovereignty.

You can take your data to another server only if the original server is up. When the admin forgets to pay the bill and ISP nukes the box and the backups admin said they had don't exist and the coadmin turns out to have no access at all, you lose everything. Ask me how I know.

Self hosting in theory should fix this but it doesn't: it's a resource intensive afterthought because the developers only really care about their Big Instance.

I just wanted to say thank you, aider (with the new unified diff format) is the first AI tool that has actually changed the way I work.

The razor doesn't apply to the police.

Hetzner can and will terminate your account and delete all your data on a whim, never telling you why. There is nothing at all you can do to prove your usage was legitimate. I get that they must deal with a lot of abuse but I wouldn't use them for anything mission critical ever.

In terms of economics specifically, in what way have things "gotten better"? Purchasing power has collapsed, the poor sentiment here seems the correct sentiment to me.

There is no difference in download time for 20mb vs 50mb in a datacenter, but the amount of time lost to alpine being "quirky" is incalculable.

If anyone is looking for a more reasonably cost effective solution, Hetzner has 16 vCPU/32GB RAM ARM VMs for $24E/mo that will run 34b Q4 GGUF at around 4 tok/sec. It's not very fast, but it is very cheap.

Where do I get one of these tech writing destroying AIs that will both understand how to use my product and produce clear, concise guides for doing so without burdening the existing team?

This is not about intelligence, it's about agency. Text completion generators don't have agency no matter how big they get.

Hetzner offers incredibly cheap ARM machines in the Falkenstein DC, for 25Eur a month you can snag the top of the line with 16 vCPU and 32GB RAM.

If your usecase fits inside that 32GB (no 70B models, sadly) the price to performance of a GGUF Q4KM is really attractive on this setup.

Pascals := is concise and intuitively obvious, so obviously it never caught on.

1997 era internet would have scared the shit out of you.

Why would you need to compete with such an entity? What does "compete" even mean here? Can't instead everyone get one (or 10) of these to.. assist them? I don't understand why we are jumping directly into dystopian scifi vs what past tool revolutions actually look like.

Receiving a PR is a massive success in my books, it means someone not only found it useful but also improved it! The win-win of open source.

With VS Code remote SSH, there is no "local" you are always on the server so there is also no syncing. They do some tricks to make this seamless and perform and feel as if everything was local.

I always write the fun bits of code myself, it's usually easier then explaining to the model what I want anyway.. but like 80% of any project is scaffolding or plumbing, The Machine can do that for me with 10x speed and precision without taking any enjoyment away from me.

..isn't this just Docker?

It's now called Xitter (with the X pronounced 'sh')

Canadian here, just checked and still no access. It's like some kind of bad joke is being played on us, Google can pound sand.

Hetzner has some weird and very strong opinions on what you can and cannot do on their servers, likely because they are so cheap they attract all sorts of shady customers.

I've had good luck with OneProvider.

OobaBooga supports this kind of load-and-go LORA: https://github.com/oobabooga/text-generation-webui

I've been working on just such a tool [1] to help me digest podcasts and senate hearings.

[1] https://github.com/the-crypt-keeper/tldw

Mismatched parenthesis, specifically using )) instead of ) is a common failure mode I observed in all of my codellama testing.

One caveat here is that whisper.cpp does not offer any CUDA support at all, acceleration is only available for Apple Silicon.

If you have Nvidia hardware the ctranslate2 based faster-whisper is very very fast: https://github.com/guillaumekln/faster-whisper

I have several sets of quant comparisons posted on my HF spaces, the caveat is my prompts are all "English to code": https://huggingface.co/spaces/mike-ravkine/can-ai-code-compa...

The dropdown at the top selects which comparison: Falcon compares GGML, Vicuna compares bits and bytes. I have some more comparisons planned, feel free to open an issue if you'd like to see something specific: https://github.com/the-crypt-keeper/can-ai-code

The model card also has prompt formats for context aware document Q/A and multi-CoT, using those correctly improves performance at such tasks significantly.

I have noticed that on StarLink some sites behind CF go into "prove you are human" loops that are impassable.

What causes such loops? Just a challenge over and over.