iamarunbrahma

Product Hunt: https://www.producthunt.com/products/nook-8

Thanks for sharing!

Hey HN, I'm excited to share Vision Parse, an open-source Python library that uses Vision Language Models to convert PDF documents into perfectly formatted markdown content automatically.

- Converts each page in a PDF document into high-resolution images - Detects texts, tables, links, and images from the high-resolution image using Vision LLMs and parses them in markdown format - Handles multi-page PDF documents effortlessly - And it's easy to get started with this library (just pip install vision-parse, and then a few lines of code to convert a document into markdown formatted content).

Conversion of PDF documents to structured Markdown, optimized for Retrieval Augmented Generation (RAG) and other NLP tasks. Extract text, tables, and images with preserved formatting for enhanced information retrieval and processing.

Get a summarised text for each youtube video using AI Assistant. Ask queries to the AI Assistant regarding information in the youtube video and get instant answers.

The purpose of this project was to showcase how can we fine-tune large models on a free-tier GPU provided by Colab. Hence, any individual can utilize parameter-efficient fine-tuning methods to tune LLMs on domain-specific datasets.

RAG and fine-tuning serves a slightly different purpose. Fine-tuning helps LLM in learning a new task/skill such as question/answering task, summarization task etc, and improving reliability at producing a desired output such as JSON format structure thereby reducing dependency on prompt engineering.

On the other hand, RAG provides you with external domain-specific knowledge, which one can leverage to get latest information.

This is a toy project. Hence, large-scale collection of data and pre-processing them into conversational format, also removing sensitive information is not possible. I was able to curate only 172 rows of data.

I have shared a notebook and explained detailed steps in my blog - https://medium.com/@iamarunbrahma/fine-tuning-of-falcon-7b-l.... If you are interested to replicate the steps on medical-domain therapy chat transcripts, you are definitely welcome. If you face any issues during fine-tuning steps, you can connect with me on my blog. Would love to help out!

I have pre-processed the dataset in a generic conversational format. It's not just raw scraped QnA from Wiki or FAQ's. For few questions which seemed sensitive, I added recommendations to consult a professional mental health care. Transcripts of therapy won't be reliable here because they might contain PII's, unwanted dialogues etc.

Amazing