easygenes

Nils Berglund posts many very similar simulation videos.. https://youtube.com/@NilsBerglund

I am surprised to not see Jami ( http://jami.net ) mentioned here yet. It is a GNU project and is fully cross platform. I use it to send files and messages between my devices, as easy as Telegram or Whatsapp.

The article suggests not charging your phone to 80% and unplugging, but doesn’t cover the recent iOS setting which allows capping your charge to 80%. That’s what I do and it offers best of both worlds.

Phones over-beautifying faces by default already happened with the iPhone XS, and it wasn't received well. See #beautygate: https://www.imore.com/beautygate

It doesn’t really though. There was hope it would be a dual-layer LCD device that could, but alas we’re stuck with $20k+ Sony monitors for that still.

It’s basically a front end for llama.cpp, so it will only show models with GGUF quantizations.

Having tested this model a lot today, I can say that it indeed in a lot of ways feels much like a 70B model, much to my surprise.

One thing that really struck me: I asked it details about my local suburbs (minor cities in a larger metro). I have tried this on all different Llamas, and only the 70B models don't just hallucinate everything or tell me they don't know anything. In contrast, this model knows quite a bit about my local area. Not as good as Llama2-70B, but not incredibly far off.

XWin is good for AlpacaEval, but was optimized specifically for that. Evaluated more broadly, there are other better choices.

This new release is a finetune of Mistral-7B base model. They claim it achieves 98% of the eval performance of Llama2-70B.

So how does OnlyOffice's Excel replacement compare?

When Llama first released in Feb Yann tweeted that they were committed to releasing all models under GPL, and yet that’s not the license we all got. There’s been a move to more permissive licensing since. So it’s been a bit of a tug of war still.

Let’s not pretend they’re free. Even if you’re heavily optimizing, one of the best local models you can run today is CodeLlama 34B. To run that effectively, you need at least a used RTX 3090: $800 (assuming your computer already has everything else you need to effectively use one). They take 350W by default, but you can power limit them to 200W and preserve most of the LLM performance. If you’re in an average place where residential power is on the order of $0.2/kWh then you’ll pay $1 for every ~25 hours of use in power.

The output will be significantly slower than what you’d get from GPT-3.5, so if your time is worth anything then that will be a huge factor that’s hard to calculate. Just assume everything will take about 3 times longer though.

If you amortize the cost of the 3090 for as long as you’d expect it to be relevant, you can estimate a rough monthly cost of the hardware. I’d give it just over a year at the current pace before you’d probably want a replacement. That’s about $50/mo.

Operation at commodity scale with the still significant lead OAI or Anthropic has on efficient operations has its advantages beyond just SOTA performance.

Yeah, but watt for watt the 3090s will output more tokens, as a single 3090 has more memory bandwidth than an M2 Ultra. That's the main performance constraint for LLMs.

Dramatically oversimplifying of course. There will be niches where one will be the right choice over the other. In a continuous serving context you'd mostly only want to run models which can fully fit in the VRAM of a single 3090, otherwise the crosstalk penalty will apply. 24GB VRAM is enough to run CodeLlama 34B q3_k_m GGUF with 10000 tokens of context though.

For LLM applications, the performance loss when power limiting 3090 to 200w is fairly low and you get peak perf/w.

Depending on the task, the best open models will outperform GPT-3.5, but would be more expensive to run at comparable speed. GPT-4 is in a league of its own.

So comparing the two...

Unique to Restfox

- a nice "beautify json" button

- in addition to response code and time, shows size and exact timestamp of response

- Can drag and drop to reorganize folders and items in sidebar

Unique to RecipeUI

- TypeScript interface definitions

- Generates cURL, Javascript and Python code snippets you can copy

Probably other things, but these are what stood out.

What are some existing data source that are somewhat analogous to this? e.g., Project Euler.

Thanks for that. I've used RunPod GPU cloud to setup vLLM as an Open-AI API compatible endpoint before, but haven't tried any of the serverless options yet.

I'm curious about this. Can you point me to, e.g. some example code for setting up an inference endpoint with a base llama2 model on modal.com?

For what tasks do you consider 70B beyond GPT-3.5 performance? There are some I’m aware of, but they are very much the exception and not the rule, even with the best 70B fine-tunes currently available.

I’m pretty amazed by how good 13B models are since they’ve gotten the orca treatment. This new one released today has the best evaluation performance of all so far and is in some ways comparable or better than the original LLaMA-65b… a bit shocked by that.

https://huggingface.co/Open-Orca/OpenOrca-Platypus2-13B

https://huggingface.co/spaces/Open-Orca/OpenOrca-Platypus2-1...

Ok, the way you wrote it implied the model itself is available to self host.

You mention self hosting the model. Do you have the model weights up on HuggingFace?

Unfortunately, it was trained with a tokenizer which squashes repeated spaces. Basically, this means it can't do code generation or any whitespace-based text formatting.

They had one job, and this wasn't it.

Thanks for the thoughtful response. Glad to see this is something you care about preserving.

Good to see you're careful to only share particular channels.

I have more thoughts on marketing this and also on guidelines for server administrators implementing search indexing. For marketing, most importantly, it could be good to make it clear you're focused on selective sharing only of channels which it would be a public good to make indexable. For administrator guidelines, most importantly, I think there should be several measures to ensure that users are aware of and agree to having their communications in particular channels publicly indexed.

I ran this by GPT-4 for some more context and detail. [1]

I think with measures like this we may be able to realize the good of indexing without going too far to driving away the safety of the walled garden aspect of Discord.

As an aside, for users of existing Discords, I encourage you to learn to use the search features built into Discord. Discord itself indexes servers and the search has good filtering functionality. I suspect if you already know which Discord server has the information you're looking for, you'll have a better experience with the internal search than trying to lean on Google.

If you want to do better than the internal search, perhaps creating a vector store of the channel and setting up an AI chat application in front of it would be a solution.

[1] https://chat.openai.com/share/254632c2-c25b-4299-88c9-2ce49e...

I was an active participant of the 90s web, and in fact a lead editor and forum moderator for a popular turn of the millennium news site, so I understand the frustration you're sharing.

That said, I'd argue it's not the "cozy web" that's out of control, but instead the "dark forest" that has forced the creation of the cozy web. The cozy web is the only bastion of the internet left where there's still some semblance of the pay it forward community aspect of the early web.

Yes, it is at the cost of not being indexed, but it's the only way of having the genuine sorts of conversations and creation with people of shared interests that typified the early web now.

While I see the value here, I don't really think most Discord communities are appropriate to be indexed. It breaks the whole cozy web aspect of it. [1]

[1] https://maggieappleton.com/cozy-web

As someone who has been using GPT-4 and Claude working with Jupyter notebooks the last month, I’ll say that sometimes I am surprised by how well they can understand and segment out a problem. Other times it is obvious I’m asking too much of them, because no amount of clever reframing gets them to a right answer.

A surprise win was as much luck I had talking through an issue and getting correct, functional one-shot time grid and 3D plots including extra complications for the calculation of the axes.

A recent lose was trying to get it to generate a script that could optimize the gravity coefficient in the Reddit algorithm based on target timeframes and historic data. It offered salvageable output, if you pieced together several attempts and reframes, but never pulled it over the finish line.

That’s already a thing, and works rather well: https://github.com/jupyterlab/jupyter-ai

The repo README doesn’t make it obvious how it works, but if you go to the Chat Interface portion of the docs it’s illustrated: https://jupyter-ai.readthedocs.io/en/latest/users/index.html...

Hi, I like this! I'm curious what drove the decision to use the vertical block builder style you chose. I'm partial to node-based editors and have been building things with React Flow recently. LangFlow [1] is a good example, but there's lots of UIs that use a similar interface (e.g. Blender [2] and Unity [3]).

[1] https://github.com/logspace-ai/langflow

[2] https://docs.blender.org/manual/en/3.5/interface/controls/no...

[3] https://unity.com/features/unity-visual-scripting