dberenstein1957

A cool alternative reaching 2.5x speedup is https://github.com/huggingface/flux-fast

Optimisations are widely applied and can reduce inference time, but their impact on quality often remains unclear, so we decided to challenge the status quo and create our own optimised version of FLUX.1[dev] called FLUX-juiced.

One thing we are considering is expanding the dataset with more synthetic examples per example across the taxonomies using the the real-life annotated examples.

I'm David from Giskard, and we work on securing Agents. Today, we are announcing RealHarm: a dataset of real-world problematic interactions with AI agents, drawn from publicly reported incidents. Most of the research on AI harms is focused on theoretical risks or regulatory guidelines. But the real-world failure modes are often different—and much messier. With RealHarm, we collected and annotated hundreds of incidents involving deployed language models, using an evidence-based taxonomy for understanding and addressing the AI risks. We did so by analyzing the cases through the lens of deployers—the companies or teams actually shipping LLMs—and we found some surprising results: Reputational damage was the most common organizational harm.

Misinformation and hallucination were the most frequent hazards

State-of-the-art guardrails have failed to catch many of the incidents. We hope this dataset can help researchers, developers, and product teams better understand, test, and prevent real-world harms. The paper and dataset: https://realharm.giskard.ai/

We'd love feedback, questions, or suggestions—especially if you're deploying LLMs and have real harmful scenarios.

Lovely, for me coreference resolution also is a huge issue. I created this package (https://github.com/Pandora-Intelligence/crosslingual-corefer...) which is currently the only viable solution to do coref resolution in Dutch and some other low-resource language.

Also, if you want to scale, LLMs are going to prove to expensive, so eventually data need to be logged somewhere to create a fine-tuned model that can do sub-tasks and ideally do them better. What do you think?