jarulraj

Really cool project!

I am curious about the benefits and limitations of using Python in this project as opposed to C++. How well is LearnDB able to support low-level concurrency control and storage management?

Highly recommend Lark for Python projects -- it's easy to use :)

Their IDE was super useful for debugging the grammar: https://www.lark-parser.org/ide/

We use Lark for a SQL-like language tailored for using AI models in EvaDB: https://github.com/georgia-tech-db/evadb/blob/master/evadb/p... https://github.com/georgia-tech-db/evadb/

If you like Lark, please consider sponsoring them: https://github.com/sponsors/lark-parser

Interesting observation, @swyx. There seems to be a connection to transitive closure in SQL queries, where the output of the query is fed as the input to the query in the next iteration [1]. We are thinking about how to best support such recursive functions in EvaDB [2].

[1] http://dwhoman.com/blog/sql-transitive-closure.html [2] https://evadb.readthedocs.io/en/stable/source/tutorials/11-s...

Neat idea, @shishirpatil! We are developing EvaDB [1] for shipping simpler, faster, and cost-effective AI apps. Can you share your thoughts on transforming the output of the Gorilla LLM to functions in EvaDB apps -- like this function that uses the HuggingFace API -- https://evadb.readthedocs.io/en/stable/source/tutorials/07-o...?

[1] https://github.com/georgia-tech-db/eva

Interesting observation, @gibsonf1. What kind of hierarchical queries do you have in mind that would be challenging to write in SQL? Would you mind sharing them in natural language?

We also love SQL, @potatoman22 :)

BTW, EVA itself is designed for analyzing sports games -- like touchdowns in a football game. This notebook shows how an action recognition model can be run on an American Sign Language video to identify the correct word [1]. Similarly, a model for identifying touchdowns and players can be used to run interesting queries over a football game.

   SELECT FIRST(id), ASLActionRecognition(SEGMENT(data)) 
                  FROM ASL_ACTIONS 
                  SAMPLE 5
                  GROUP BY '16f';

[1] https://github.com/georgia-tech-db/eva/blob/master/tutorials...

Great question, @ukuina! We will need some time to do a detailed analysis of MindsDB. But here are some key differences a first glance:

1. First-class support for unstructured data: EVA natively supports querying over videos, audio, etc. For instance, it manages a video as a table, mapping each tuple to a video frame.

2. Cascades-style query optimizer: EVA has an AI-centric query optimizer with built-in support for cost-based query optimization, predicate reordering, caching of expensive models, etc.

3. First-class support for vector embeddings: EVA supports similarity search using embeddings stored in a vector database.

Similarities to MindsDB:

1. EVA is implemented in Python and uses sqlalchemy for connecting to a structured database system (PostgreSQL, MySQL, etc.)

2. EVA supports Huggingface and OpenAI pipelines in the form of user-defined functions.

We will love to hear from the developers of MindDB if our high-level analysis is correct :)

Haha. Thanks for sharing this reference! I had no idea about a similar comment [1] in our previous Show HN post :) For others like me, this is ChatGPT's explanation:

    C&C Tiberian Sun, short for Command & Conquer: Tiberian Sun, is a real-time strategy (RTS) video game developed by Westwood Studios and released by Electronic Arts in 1999. It is the sequel to the highly popular game Command & Conquer (1995) and is part of the Command & Conquer series."

    In the real-time strategy game Command & Conquer: Tiberian Sun, the phrase "Welcome back, Commander" is a memorable reference that occurs at the start of each mission when the player resumes playing the game after a break or reloading a saved game.

    The phrase is spoken by the EVA (Electronic Video Agent) computerized voice, which serves as the player's AI assistant throughout the game. The EVA voice welcomes the player with the line "Welcome back, Commander" to indicate that the player has returned to the game and is ready to continue commanding their forces.

[1] https://news.ycombinator.com/reply?id=35768822

Great question! We have a built-in integration test for 1 million images [1] and have benchmarked EVA on large unstructured datasets for our ACM SIGMOD paper [2].

Coming up with an open benchmark for AI-Relational database systems is essential [3]. Please share your thoughts on any ideas for benchmarking :)

[1] https://github.com/georgia-tech-db/eva/blob/e123820c79b902d5...

[2] https://dl.acm.org/doi/abs/10.1145/3514221.3526142

[3] https://dl.acm.org/doi/10.1145/3299869.3324955

Hey friends

We wanted to express our heartfelt gratitude to all of you for your support for the EVA AI-Relational Database System last week [1]. The feedback from the HN community has been truly overwhelming :)

We were planning to make an HN post based on these features we have since added [2], but @jonbaer thankfully beat us to it :)

    - Using ChatGPT/LLM + Whisper (Open AI + Hugging Face) models to ask questions based on videos/tables etc. You can check out the notebook here [3].

        LOAD VIDEO 'russia_ukraine.mp4' INTO VIDEOS;

        CREATE MATERIALIZED VIEW 
            TEXT_SUMMARY(text) AS 
            SELECT SpeechRecognizer(audio) FROM VIDEOS; 

        CREATE UDF SpeechRecognizer 
                TYPE HuggingFace 
                'task' 'automatic-speech-recognition' 
                'model' 'openai/whisper-base';

        SELECT ChatGPT('Is this video summary related to Ukraine-Russia war', text) FROM TEXT_SUMMARY;

    - Joining structured data tables with OCRExtractor model output

        SELECT * FROM MyImages 
            JOIN LATERAL OCRExtractor(data) AS T(a,b,c) 
            JOIN LicensePlateCSV B 
            ON FuzzDistance(T.a, B.label) > 50;

    - Vector index support for similarity search [5]

        CREATE INDEX reddit_sift_object_index
                            ON reddit_object_dataset (SiftFeatureExtractor(Crop(data, bboxes)))
                            USING HNSW

        SELECT name FROM reddit_object_dataset ORDER BY
                                Similarity(
                                SiftFeatureExtractor(Open('{path}')),
                                SiftFeatureExtractor(data)
                                )
                                LIMIT 5

    - Using YOLO models to detect objects [6]

        CREATE UDF IF NOT EXISTS Yolo
                    TYPE  ultralytics
                    'model' 'yolov8m.pt';

        SELECT id, Yolo(data)
                        FROM ObjectDetectionVideos 
                        WHERE id < 20

Your feedback and suggestions have already been instrumental in shaping our roadmap and prioritizing new features! We are actively working on incorporating your ideas and addressing your concerns to enhance the functionality, performance, and user experience of EVA. Please email me (arulraj@gatech.edu) with any questions, ideas, or suggestions on EVA.

[1] https://news.ycombinator.com/item?id=35764355

[2] https://github.com/georgia-tech-db/eva/releases/tag/v0.2.3

[3] https://evadb.readthedocs.io/en/stable/source/tutorials/08-c...

[4] https://github.com/georgia-tech-db/eva/blob/master/tutorials...

[5] https://github.com/georgia-tech-db/eva/blob/a6a6ecc7e5c0d9d4...

[6] https://evadb.readthedocs.io/en/stable/source/tutorials/02-o...

We are in the process of supporting a native `LOAD PDF` command. Meanwhile, you could convert each PDF into a series of images and load them using the `LOAD IMAGE` command. You could then run any text extraction user-defined function (e.g., `textract` [1]) over the loaded documents with additional filters based on your constraints (like PDF author or creation date). As EVA is designed for local usage, you can run it on local private files. We would love to jointly explore how to best support your text extraction pipeline. Please consider opening an issue with more details on your use case.

[1] https://textract.readthedocs.io/en/stable/python_package.htm...

Thanks for the clarifications. Token summation is a cool optimization :)

Query optimizers in SQL database systems typically optimize based on the time to execute the function on a local server. The token summation optimization generalizes time-based optimization of local functions to dollar-based optimization for remote functions.

Execution Time-based optimization: FunctionFoo(input 1) = 2x FunctionFoo(input 2)

Dollar-based optimization: ChatGPT(prompt with 100 tokens) = 2x ChatGPT(prompt with 50 tokens)

We are also exploring dollar-based optimization in EVA, and will check out your openai-to-sqlite tool for ideas [1].

[1] https://datasette.io/tools/openai-to-sqlite

Thanks so much for sharing your thoughts! I also felt that they are pretty similar. But, I am guessing that SQLite (similar to most relational database systems) does not automatically cache the results of functions, do non-trivial cost-based optimization for functions in queries, or reorder function-based predicates based on the estimated cost of running the functions, etc.

Edit: I have shared more details on the function-aware optimization in EVA in this post (in case you are interested) -- https://news.ycombinator.com/item?id=35764355#35773608

Sure, we will try it out and keep you posted :)

Great question! Besides improving usability, the key feature of the EVA database system is the query optimizer that seeks to speed up exploratory queries over a given dataset and save money spent on inference.

Two key optimizations in EVA's AI-centric query optimizer are:

- Caching: EVA automatically caches and reuses previous query results (especially model inference results), eliminating redundant computation and reducing query processing time.

- Predicate Reordering: EVA optimizes the order in which the query predicates are evaluated (e.g., runs the faster, more selective model first), leading to faster queries and lower inference costs.

Consider these two exploratory queries on a dataset of dog images:

  -- Query 1: Find all images of black-colored dogs
  SELECT id, bbox FROM dogs 
  JOIN LATERAL UNNEST(YoloV5(data)) AS Obj(label, bbox, score) 
  WHERE Obj.label = 'dog' 
    AND Color(Crop(data, bbox)) = 'black'; 

  -- Query 2: Find all Great Danes that are black-colored
  SELECT id, bbox FROM dogs 
  JOIN LATERAL UNNEST(YoloV5(data)) AS Obj(label, bbox, score) 
  WHERE Obj.label = 'dog' 
    AND DogBreedClassifier(Crop(data, bbox)) = 'great dane' 
    AND Color(Crop(data, bbox)) = 'black';

By reusing the results of the first query and reordering the predicates based on the available cached inference results, EVA runs the second query 10 times faster!

More generally, EVA's query optimizer factors the dollar cost of running models for a given AI task (like a question-answering LLM). It picks the appropriate model pipeline with the lowest price that satisfies the user's accuracy requirement.

Query optimization with a declarative query language is the crucial difference between EVA and inspiring AI pipeline frameworks like LangChain and TxtAI [1]. We would love to hear the community's thoughts on the pros and cons of these two approaches.

[1] https://github.com/neuml/txtai

Great question! Yes, EVA supports NLP pipelines thanks to our recent integration of Hugging Face pipelines last month. Here is an illustrative text classification application:

  -- Text classification application in EVA
  CREATE TABLE IF NOT EXISTS MyCSV (id INTEGER UNIQUE, comment TEXT(30));

  LOAD CSV 'csv_file_path' INTO MyCSV;

  CREATE UDF HFTextClassifier
  TYPE HuggingFace
  'task' 'text-classification';

  SELECT HFTextClassifier(comment) FROM MyCSV;

EVA supports many other NLP pipelines, including summarization and text2text generation.

[2] is an illustrative notebook that presents an HF-based object segmentation pipeline (not NLP-based though). We would love to jointly explore how to best support your NLP pipeline. Please consider opening an issue with more details on your use case.

[1] https://github.com/georgia-tech-db/eva/blob/4fa52f893e7661d4...

[2] https://evadb.readthedocs.io/en/latest/source/tutorials/07-o...

Hey Simon! Thanks for sharing this. I have long admired your work on Datasette :) We will check out your posts for ideas on coping with prompt injection.

I just came across your recent post on the ChatGPT SQL function in SQLite [1]. We just added a ChatGPT-based UDF in EVA [2]. I would love to hear your thoughts on the difference between these two approaches.

Another coincidence is that EVA uses SQLite for managing structured data by default. Can EVA's SQLite database be an interesting use case for Datasette?

[1] https://simonwillison.net/2023/Apr/29/enriching-data/ [2] https://github.com/georgia-tech-db/eva/pull/655

Thanks for the clarification.

EVA currently does not support remote GPUs. Our ongoing integration [1] of the Ray distributed compute framework [2] into EVA will soon allow us to support remote GPUs. We would love to jointly explore how to best support remote GPUs. Please consider opening an issue with more details on your use case.

[1] https://github.com/georgia-tech-db/eva/blob/master/eva/exper...

[2] https://docs.ray.io/en/latest/cluster/vms/getting-started.ht...

Great questions!

EVA does not currently support a weighted similarity search. We are working on creating a notebook to illustrate similarity queries. But, EVA already supports the queries of this form:

  -- Step 1: Extract objects in Reddit images using the YOLO object detector
  CREATE TABLE reddit_dataset_object (name, data, bboxes) 
  AS SELECT name, data, labels FROM reddit_dataset
  JOIN LATERAL UNNEST(YoloV5(data)) AS Obj(labels, bboxes, scores);

  -- Step 2: Build index over features extracted using SIFT
  CREATE INDEX reddit_sift_object_index
  ON reddit_dataset_object (SiftFeatureExtractor(Crop(data, bboxes)))
  USING HNSW;

  -- Step 3: Retrieve the top 10 most similar images
  SELECT id FROM reddit_sift_object_index
  ORDER BY Similarity(SiftFeatureExtractor(Open(”“input_img_path.jpg”)),
           SiftFeatureExtractor(data))
  LIMIT 10;

https://github.com/georgia-tech-db/eva/blob/bfd424fd5beb3cec...

EVA directly persists the feature vectors in a FAISS index. It does not use a relational database system for this purpose. FAISS supports retrieving the original vector through ID (required for similarity search).

We would love to jointly explore how to support such weighted similarity search queries. Please consider opening an issue with more details on your use case.

Yes, EVA works out of the box on an AWS server with GPU [1].

[1] https://aws.amazon.com/nvidia/

Thanks for the helpful suggestion! Just reordered the examples in the README.

Here are the illustrative queries:

  -- Object detection in a surveillance video
  SELECT id, YoloV5(data)
  FROM ObjectDetectionVideos 
  WHERE id < 20

  -- Emotion analysis in movies
  SELECT id, bbox, EmotionDetector(Crop(data, bbox)) 
  FROM HAPPY JOIN LATERAL  UNNEST(FaceDetector(data)) AS Face(bbox, conf)  
  WHERE id < 15;

Thanks for following up on this.

We designed EVA from scratch for managing unstructured data (e.g., video, audio, images, etc.). EVA leverages relational database systems to manage structured data and widely-used libraries to manage feature embeddings (FAISS library [1]). We aim to leverage decades of experience in relational database systems and reduce risk in production deployment.

[1] https://github.com/facebookresearch/faiss

Thanks for the helpful suggestion! EVA uses an SQL database system for managing structured data using sqlalchemy. It runs on PostgreSQL out of the box. You only need to provide the database connection url in the EVA configuration file.

Thanks for your candid comment. We take it very seriously. EVA is already being used in production by some collaborators and we would love to support more early adopters :) Please let me know if I can DM you to get more feedback.

That’s correct! You can compose multiple models in a single query to set up useful AI pipelines.

Here is an illustrative query that chains together multiple models:

   -- Analyse emotions of faces in a video
   SELECT id, bbox, EmotionDetector(Crop(data, bbox)) 
   FROM MovieVideo JOIN LATERAL UNNEST(FaceDetector(data)) AS Face(bbox, conf)  
   WHERE id < 15;

Honestly, we have not extensively thought about prompt injection attacks -- the equivalent of SQL injection attacks in AI-Relational database systems :)

If you have any thoughts on addressing this, please do share! We will incorporate that in the LLM-based functions in EVA.

Got it! EVA is designed for the local use case. You can define a Python function that wraps around the LLM model and use it anywhere in the query (we refer to such functions as user-defined functions or UDFs).

This notebook illustrates a UDF that wraps around a custom PyTorch vision model: https://evadb.readthedocs.io/en/stable/source/tutorials/04-c...

These functions can be written quickly (~50 lines of Python code). Here is the built-in Resnet50 UDF in EVA: https://github.com/georgia-tech-db/eva/blob/master/eva/udfs/...

This page describes the steps involved in writing a UDF in EVA: https://evadb.readthedocs.io/en/stable/source/reference/udf....

Please open an issue on the Github repo; we will gladly support your use case :)

Great question! We will be adding a ChatGPT-based user-defined function this week (https://github.com/georgia-tech-db/eva/pull/655/).

With LLM-based functions, EVA will support more interesting queries like this:

  SELECT ChatGPT(TextSummarizer(SpeechRecognizer(audio)),
         "Is this video related to the Russia-Ukraine war?")
  FROM VIDEO_CLIPS;

Here, EVA sends the audio of each video clip to a speech recognition model on Hugging Face. It then sends the recognized text to a text summarizer model. EVA executes both models on local GPUs. Lastly, EVA sends the text summary to ChatGPT as a part of the prompt. The ChatGPT UDF is executed remotely.

The critical feature of EVA is that the query optimizer factors the dollar cost of running models for a given AI task (like a question-answering LLM). It picks the appropriate model pipeline with the lowest price that satisfies the user's accuracy requirement.

Thanks! Likewise :)

Thanks for the suggestion! I just added links to the demo applications earlier in the README. All applications are Jupyter notebooks that you can open in Google Colab.

* Examining the emotion palette of actors in a movie: https://evadb.readthedocs.io/en/stable/source/tutorials/03-e...

* Analysing traffic flow at an intersection: https://evadb.readthedocs.io/en/stable/source/tutorials/02-o...

* Classifying images based on their content: https://evadb.readthedocs.io/en/stable/source/tutorials/01-m...

* Recognizing license plates: https://github.com/georgia-tech-db/license-plate-recognition

* Analysing toxicity of social media memes: https://github.com/georgia-tech-db/toxicity-classification

Thanks for the suggestion. I unfortunately don't, but this tool can certainly be integrated with ORMs and in other BI stacks.

Sure :) Any other interesting suggestions?

I must mention that I greatly admire your pragmatic books on improving database performance, and they were a source of inspiration for the development of this tool.

By design, the tool is dialect-agnostic.