RAGoon is now available on PyPI, and on HF for batched embeddings generation

RAGoon is a set of NLP utilities for multi-model embedding production, high-dimensional vector visualization, and aims to improve language model performance by providing contextually relevant information through search-based querying, web scraping and data augmentation techniques.

Link to GitHub: https://github.com/louisbrulenaudet/ragoon Link to Hugging Face Space: https://huggingface.co/spaces/louisbrulenaudet/ragoon

Here is an example of code to produce Embeddings for a given list of models: ```python from ragoon import EmbeddingsDataLoader from datasets import load_dataset

# Initialize the dataset loader with multiple models loader = EmbeddingsDataLoader( token="hf_token", dataset=load_dataset("louisbrulenaudet/dac6-instruct", split="train"), # If dataset is already loaded. # dataset_name="louisbrulenaudet/dac6-instruct", # If you want to load the dataset from the class. model_configs=[ {"model": "bert-base-uncased", "query_prefix": "Query:"}, {"model": "distilbert-base-uncased", "query_prefix": "Query:"} # Add more model configurations as needed ] )

# Uncomment this line if passing dataset_name instead of dataset. # loader.load_dataset()

# Process the splits with all models loaded loader.process( column="output", preload_models=True )

# To access the processed dataset processed_dataset = loader.get_dataset() ```

In particularly, this tool also provides the functionality to load embeddings from a FAISS index, reduce their dimensionality using PCA and/or t-SNE, and visualize them in an interactive 3D graph.