txtai

Excellent work! Hope to see a version that has a friendlier commercial license in the future (current version is CC-BY 4.0 NC).

If you want something similar to this that is fully downloadable and open source, check this out - https://huggingface.co/NeuML/txtai-wikipedia

It's an embeddings database of Wikipedia abstracts with page view data integrated to enable filtering pages based on popularity in addition to similarity.

SetFit is a great framework for building a text classifier.

This is a pretty straight forward problem and a good fit for a standard text classifier as well.

Here is an example of fine-tuning a model with txtai: https://colab.research.google.com/github/neuml/txtai/blob/ma...

txtai combines Faiss and SQLite to support similarity search with SQL.

For example: SELECT id, text, date FROM txtai WHERE similar('machine learning') AND date >= '2023-03-30'

GitHub: https://github.com/neuml/txtai

This article is a deep dive on how the index format works: https://neuml.hashnode.dev/anatomy-of-a-txtai-index

txtai combines SQLite and Faiss to enable vector search. It also does a lot more than that.

https://github.com/neuml/txtai

The first comment in that article has details on the new model. Not the original author but per their testing they said they paid $70 to encode 1M records. The embeddings are 1536 dimensions, which require a lot of vector storage. The HF hub has open models for 384 dimensions or 768 dimensions that work well for a lot of use cases.

That's up to you. Many don't want to open an account and pay in order to explore what's possible. There are LLMs available on the HF Hub, such as google/flan-t5-xl.

Starting with HF models and moving to a large model like GPT3 when the task calls for it is a good approach to take with almost all tasks.

If speed and price are concerns, use the FOSS models available on the Hugging Face Hub: https://hf.co/models. Thousands of models, different sizes and tasks. Download locally and fine-tune, if necessary.

For those specifically interested in text embeddings, here is a good analysis: https://medium.com/@nils_reimers/openai-gpt-3-text-embedding...

Thanks. Just want people to know there are quality FOSS alternatives available for encoding text into embeddings.

txtai is an alternative approach to this. It builds a FAISS (also supports HNSW) index alongside a SQLite database. It works with sentence-transformers models. For example, this model https://huggingface.co/sentence-transformers/all-MiniLM-L6-v... is 384 dimensions and works great for semantic search.

https://github.com/neuml/txtai

https://neuml.github.io/txtai/embeddings/query/

This is a really nice article by Dmitry Kan on number of popular vector databases.

https://towardsdatascience.com/milvus-pinecone-vespa-weaviat...

This concept was more for inference on-demand vs training a model. For example, if there was an API call that had a model call as part of it's workflow. Training even a small model still requires serious compute power.

There is also a push to run machine learning models on low-resource devices, which a $4 VPS could be categorized as.

https://www.tensorflow.org/lite

https://huggingface.co/muhtasham/olm-bert-tiny-december-2022

https://neuml.hashnode.dev/train-a-language-model-from-scrat...

There is a GitHub repo: https://github.com/amazon-science/mm-cot

But it doesn't look all that easy to stand up.

InstructGPT which is a "sibling" model to ChatGPT is 1.3B parameters. https://openai.com/blog/instruction-following/

Another thread on HN (https://news.ycombinator.com/item?id=34653075) discusses a model that is less than 1B parameters and outperforms GPT-3.5. https://arxiv.org/abs/2302.00923

These models will get smaller and more efficiently use the parameters available.

Great looking project here. Absolutely need a local/FOSS option. There's been a number of open-source libraries for LLMs lately that simply call into paid/closed models via APIs. Not exactly the spirit of open-source.

There's already great local/FOSS options such as FLAN-T5 (https://huggingface.co/google/flan-t5-base). Would be great to see a local model like that trained specifically for chat.

Outperforms GPT-3.5 by 16% with less than 1B parameters. There is even a 220M parameter version that scores well. Interesting model to watch.

Referenced snippet from the abstract:

With Multimodal-CoT, our model under 1 billion parameters outperforms the previous state-of-the-art LLM (GPT-3.5) by 16% (75.17%->91.68%) on the ScienceQA benchmark and even surpasses human performance.

GitHub repo: https://github.com/neuml/txtai

The goal of this approach is to answer questions with a datasource. The referenced example runs a vector database query to build the context and then runs a prompt to ask the question.

Answer the following question using only the context below. Say 'no answer' when the question can't be answered. Question: {question} Context: {context}

Not sure this prompt works for all scenarios and models. But it can easily be changed and is a starting point.

This example isn't using gpt embeddings but it could. It's using local Hugging Face models (FLAN-T5) with a context provided by a vector store (Faiss).

Very interesting! Looking forward to seeing more embedded AI devices coming out.

More NLP based, but here is an article on an effort to build Transformers micromodels to run on embedded devices. The model in this example is under 1MB. Goal would be to ultimately convert this from ONNX to TFLite.

https://neuml.hashnode.dev/train-a-language-model-from-scrat...

Link to example: https://neuml.hashnode.dev/prompt-driven-search-with-llms

Article also available: https://neuml.hashnode.dev/prompt-driven-search-with-llms

Also available as an article: https://neuml.hashnode.dev/train-a-language-model-from-scrat...

v5.2 adds TextToSpeech and Cross-Encoder pipelines. The performance of the embeddings.batchtransform method was significantly improved, enabling a speed up in building semantic graphs. Default configuration is now available for Embeddings, allowing an Embeddings instance to be created with no arguments like Pipelines.

This is a lightweight, low dependency and fast text-to-speech (TTS) implementation in Python. The models are from ESPnet and exported to ONNX. Text is tokenized using ttstokenizer (https://github.com/neuml/ttstokenizer).

The goal is to use existing high quality TTS models without a heavy install footprint.

Duckdb is on the roadmap. SQLite is the first implementation. The plan is to extend the interface for other database types, including Duckdb.

Sure thing. Examples can be found here: https://neuml.github.io/txtai/examples/

This one re-ranks the output from an Elasticsearch index - https://colab.research.google.com/github/neuml/txtai/blob/ma...

The next major release have more examples using the local BM25 scoring module.

It's all local except for downloading the transformers models for vectorization. TRANSFORMERS_OFFLINE_MODE can be set and models downloaded manually for offline use.