ndr_

Different approach, which also works with Llama 3.0 8B: let the LLM write the tool invocation of choice in Python syntax, parse the LLM response with the Python AST package. (PoC'y hackathon demo) code here: https://github.com/ndurner/aileen2.

The only thing close are the "logprobs": https://cookbook.openai.com/examples/using_logprobs

However, commenters around here noted that these have likely not been fine-tuned to correlate with accuracy - for plaintext LLM uses. Would be interested in hearing finding for MLLM use-cases!

Prompts in the background:

  const systemPrompt = `
    Convert the following PDF page to markdown. 
    Return only the markdown with no explanation text. 
    Do not exclude any content from the page.
  `;

For each subsequent page: messages.push({ role: "system", content: `Markdown must maintain consistent formatting with the following page: \n\n """${priorPage}"""`, });

Could be handy for general-purpose frontend tools.

There used to be Winpooch Watchguard, based on ClamAV. Stopped using it when it caused Bluescreens. A "Killer" indeed.

Yes, Crowdstrike specifically disabled Linux systems in the past: https://news.ycombinator.com/item?id=41005936

If you're on macOS, check out the MLX community: https://huggingface.co/mlx-community - Ollama not needed, potentially more efficient(?). The Gemma 2 27B model was broken as of a few days ago, but a fix is already committed and underway to the pip repo. Gemma 2 9B works directly.

I have first noticed logprob fluctuations in GPT-4o. Perhaps the same phenomenon is also going on with Turbo. I din‘t recall specifics but it was naming inconsistencies with variable names, meaning: same variable name got a typo somewhere, but the typo was close enough - perhaps a space vs. an underscore or something like that.

Model could be the same, but maybe some in the infra is different.

Python here. And like they said, only noticable in the last few weeks.

Low-value, high-effort work. Currently: create a mockup server in C# from a HAR recording. For this specifically, I use my Amazon Bedrock workbench [0] with Claude 3 Opus. But I also use GPT-4 Turbo and GPT-4 Classic a lot [1] (in that order), because they solve things that Opus cannot. Also, Dall-E 3 (though a Google Colab sheet; for work & play) and Runway AI (for play).

Other tasks I remember: 1) create a webinar transcript and audience-specific summary (using [1]), 2) update documentation with information provided in another E-Mail (also using [1]), 3) create some visualizations in Mermaid JS and D3 (using [0], as a natural language transpiler)

For those without Bedrock, there's also [2].

[0] https://huggingface.co/spaces/ndurner/amz_bedrock_chat , [1] https://huggingface.co/spaces/ndurner/oai_chat , [2] https://huggingface.co/spaces/ndurner/claude_chat

If you prefer an existing webapp over an elisp function, https://huggingface.co/spaces/ndurner/oai_chat. Choose Whisper as the model, upload your 25 MB chunks, hit Send, choose GPT-4 Turbo, ask it to clean up, hit Send. Then, hit the Download button (hidden away on the very bottom).

Helpful fact: Whisper works on 16 KHz sampling behind the scenes, so can make your recording smaller by downsampling to 22 KHz, mono. AAC is supported, and commenters to the web say Whisper is pretty robust so it doesn‘t have do be hi-fi - just so that you can make a split at the beginning of the QA session perhaps, if you can‘t fit it into one 25 MB chunk right away.

ref: https://x.com/stevenheidel/status/1777794809270575135?s=61&t...

This news piece is, first and foremost, about the Model, not the ChatGPT System. (More about the difference between “Model” and “System”: https://ndurner.github.io/antropic-claude-amazon-bedrock). Not sure what their upgrade policy/process for ChatGPT is like, though.

Apparently not. But here: https://huggingface.co/spaces/ndurner/oai_chat

https://huggingface.co/spaces/ndurner/oai_chat (bring your own API key)

Well, there is two Claudes: first, there is claude.ai. Second, there is the Anthropic API. (And there is also AWS Bedrock, but it doesn’t offer Opus). Signing up for the Anthropic API from within the EU works without hassle.

The reason for the geo-block against the EU may also be rooted in the AI act, rather that GDPR. That would explain why the end-user service claude.ai is blocked off, and foundation model access remains available.

To use Claude 3 through Gradio and/or HuggingFace, see https://huggingface.co/spaces/ndurner/claude_chat

With OpenAI, you can first build Question & Answer pairs derived from your documents and use the OpenAI fine-tuning feature to build yourself a custom model. This method is more than just learning behavior in that facts do get recalled. I have written about it here, with a play demo use-case: https://ndurner.github.io/training-own-model-finetuning Note that I have yet to use this in a real world use-case, and I would love to hear feedback.

Other than OpenAI, there is the newly introduced „continued pre-training“ of Amazon Bedrock - but I haven’t tried.

RAG: I think that‘s a fundamentally flawed concept, but RAGfluencers will disagree. ;-)

It‘s generally available in the EU to AWS Bedrock customers. Just in the Frankfurt region, and with a limited context window AFAIK, but it does exist.

Miles Brundage of OpenAI offered a categorization of „AI things“ into Models, Systems, Platforms and Use-Cases: https://www.youtube.com/watch?v=5j4U2UzJWfI&t=5728s

Bard is a System, PaLM 2 would be the model (presumably). The other way round: Bard is not an LLM.

LumaFusion/LumaTouch is one counter-example, and I‘m certain there are others. At the surface, you may describe it as a „video editor“, but I totally agree that it‘s really about „Story Telling“. I use it on iPad, and I am mesmerised about how ideas click into place every time - there must be serious, long earned expertise at work at the makers of this app. „Designed for consuming“? LumaFusion is the most fun (for me as the creator and my tiny audience of family & friends) creator‘s tool on any device - be it programming on/for small or big iron, writing on Windows, photo editing… you name it. Your statement may be true for anything knock-off or underpowered, though. Your sentiment just not generalises to premium.

This article is rooted in the junk science by James Zou et al, published in their paper "How is ChatGPT's behavior changing over time?" (https://arxiv.org/abs/2307.09009). Initially, one of their benchmark scripts stumbled upon backticks added by GPT-4, causing poor scores. Simon Boehm fixed it for them, scores improved significantly (https://twitter.com/Si_Boehm/status/1681801371656536068). Now the authors are back with a revision (without giving credit to Boehm). Still, the benchmarks are silly because they measure what LLMs are historically challenged with: math and cases of "Do not do x" (ARC benchmark). One could argue that this is on purpose. I don't think so: Zou is not part of Stanford HAI, but Stanford Data Science. I think he's just publishing outside his realm of expertise, and popular media are again falling for fake.

So RAG (retrieval augmented generation) is all the rage, but there are problems with it that make it appear almost conceptually flawed - to the point where results are flat-out poor?

There‘s the paper about non-uniform attention („Lost in the Middle: How Language Models Use Long Contexts“) and some other paper mentioned that LLMs may de-focus on irrelevant retrieved content as soon as the ratio between relevant vs. irrelevant content becomes small.

So, what‘s the current best practice to actually embed your content within the model?

Is this update made visible somewhere? The language models offered on my Playground are still the ones from March, same with ChatGPT.

I get this: „ERROR. Quota exceeded for aiplatform.googleapis.com/online_prediction_requests_per_base_model with base model: chat-bison. Please submit a quota increase request.“

Has anyone gotten this fixed?

Check out llama_index at https://github.com/jerryjliu/llama_index. What it does: it creates an index over your data using OpenAI embeddings vectors, using the OpenAI Ada model. When querying, it compiles as much context out of this index as fits into GPT, based on similarity to your prompt. Be cautious however: when I experimented with this, GPT-4 support with it‘s larger context size was not there yet. I have landed https://github.com/hwchase17/langchain/pull/1778, but I never wound up submitting another, yet similar patch (to llama_index? Don‘t remember). Make sure that the GPT-4 context is really fully used, and not some smaller size is assumed. Also, ensure that GPT-4 is used as the LLM in the first place: the defaults used to be the older models.

In multiple entities I have worked for over the past 15-20 years, there was a sentiment that we wanted to support the fine folks behind Qt, thus licensed each and every developer seat, to the tune of 4000€. At some point, they started biting the hand that was feeding them: demanded an enterprise-wide usage audit (so we had to involve our Legal team to fend them off), changing the EULA in such a fast pace we couldn‘t keep up, created this forever-keep-paying rule…. Today, at my new employer, I use the LGPL version, like we always could have. No good deed goes unpunished, and I have taken enough battering.