kaspermarstal

She and one other researcher has manually classified all 7000 papers as per standard protocol. Perhaps for the next article they will measure how this tool agreed with them against them and include it in the protocol if good enough.

I don't know. This paper [1] reports accuracies in the 97-98% range on a similar task with more powerful models. With Gemma 2 2b the accuracy will certainly be lower.

[1] https://www.medrxiv.org/content/10.1101/2024.10.01.24314702v...

I built an Excel Add-In that allows my girlfriend to quickly filter 7000 paper titles and abstracts for a review paper that she is writing [1]. It uses Gemma 2 2b which is a wonderful little model that can run on her laptop CPU. It works surprisingly well for this kind of binary classification task.

The nice thing is that she can copy/paste the titles and abstracts in to two columns and write e.g. "=PROMPT(A1:B1, "If the paper studies diabetic neuropathy and stroke, return 'Include', otherwise return 'Exclude'")" and then drag down the formula across 7000 rows to bulk process the data on her own because it's just Excel. There is a gif on the readme on the Github repo that shows it.

[1] https://github.com/getcellm/cellm

Thanks for sharing! If you want to try it out, it will be published to github.com/kaspermarstal/cellm in the coming weeks. Will reply to this thread when it is live.

What’s your use case?

Yes I am really interested in good use-cases, send me an e-mail at kaspermarstal@gmail.com! It’s called Cellm, implemented with ExcelDna and is fully functional for Anthropic models. Currently preparing the repo for public release and writing some docs.

Thanks!

Omg instantly bookmarked.

How to find "Military science fiction series" that are "Complex and Strategic" and includes "Space battles and tactics", "Action-packed sequences", "Strategic dialogue", "Multiple book series expansions", "Futuristic space environment", "Spaceships and battle fleets"? Use yogurrt. Never found it so easy to narrow down my interests and find new books. Thanks!

I am building an Excel extension that lets me use LLMs in formulas. For example, I can write =PROMPT(A1:E1, "Extract keywords") in a cell to extract keywords from a row and drag the cell to apply the prompt to many other rows. I find this useful when I want to use AI for repetitive tasks that would normally require copy-pasting data in and out of a chat window many times.

I actually built it for my girlfriend who was writing a systematic review paper. She had to compare 7.500 papers against inclusion and exclusion criterias. She obviously did this manually because she cares about scientific integrity, but it sparked the idea to make an AI tool to automate repetitive tasks for people like her who would rather avoid programming. Now I just find it useful myself for a lot of ad-hoc analysis tasks like prompt engineering, rag tuning, and comparing model outputs from anthropic, openai, and google.

Can anyone explain what is gained by training a model? Why not use the foundational LLM for the relevance, evidence, and trajectory processes?

> Don’t ask me about covariance since I honestly don’t know the answer. I recommend reading the section on subtyping and variance and The Rustonomicon. I went through them a few times but still don’t entirely get all the details. Then, we can define each type of object like so:

Haha this is gold. We ALL do this ALL the time.

Very cool! A assume you use Postgres' native full-text search capabilities? Any plans for BM25 or similar? This would make Korvus the end-game for open source rag IMO.

This extension has been developed on top of pgrx and depends on the platforms that pgrx supports. From the pgrx readme:

> Windows is not supported. It could be, but will require a bit of work with cargo-pgrx and figuring out how to compile pgrx's "cshim" static library.

Yes, the PL/Rust code base was a very useful when developing this extension

Good suggestion, thanks

Incredible. RAG will be obsolete in a year or two.

Mhm, interesting development, the paper has been withdrawn.

Can you elaborate? Harder as in writing queries or harder as in making queries performant?

None that I am aware of, and that is why I am very interested in learning about any red flags you might have found that motivated your decision to move off Postgres/Supabase and that others should be aware of.

I’m curious if you investigated the TimescaleDB extension that is built into Supabase for your usecase? And if so, what was the pros and cons?

If they were serious about emissions they should not have filed a patent.

I found that having GPT-4 return a JSON dictionary where keys are line numbers and values are their corresponding lines work the best.

Can you tell who develops Hasura Auth and Hasura Storage? You? Or did you fork it and extend it?

Sure! There is tealengine.com and you can follow github.com/tealmedical to get notified when we put it up. What are you interested in? The foreign data wrapper or the iso?

A postgres foreign data wrapper for PACS so you can send and receive medical images with SQL. DICOM, the medical imaging standard, is from the 80's and you need a bit of domain knowledge to integrate e.g. AI with hospitals. This makes it a lot easier. And with Supabase on top you have a very cool backend for building medical imaging products in my humble opinion. We started out solving the problem of anonymizing medical images and now we want to make it easier for our customers to exchange data with hospitals, both for deployment and for extracting large amounts of data for research.

We are also getting ISO 27001 and we are going to open source the entire bundle of policies and the "information security management system" itself on GitHub, so I am also working on that.

I've been looking for something like this for a long time, so cool! Great work!

Does it work with TimescaleDB?

What typesetting system is this? Is this available somewhere? I love it

Exactly. I was so confused about the bashing of deep learning.

SimpleITK. Hands down the best medical image processing library ever.