juliensalinas

Lol, you're welcome ;)

Yep totally. You can forget about training Tranformer-based NLP models from scratch at home, sadly... however you can choose a pre-trained model and fine-tune it with your annotated data.

Serving the models for inference is different, you can definitely perform inference on your local machine for most of the existing NLP models, even Transformer-based models. But GPT-J and GPT-Neo might be exceptions though as they require tons of RAM.

Also, even if you can perform inference at home, you might not have great perfs without NVIDIA GPUs dedicated to AI.

Impressive work. I finally managed to add it to NLPCloud.io with Transformers but it takes 40GB of memory to startup, and then a 16GB GPU is not enough unfortunately, which makes it quite costly to run it. But text generation is really impressive, great job!

Here's the ML stack I have been using for my last project:

- Doing NLP with spaCy (https://spacy.io/) as I consider it to be the most production ready framework for NLP

- Annotating datasets with Prodigy (https://prodi.gy/), a paid tool made by the spaCy team

- Deploying the trained spaCy models onto NLP Cloud (https://nlpcloud.io), a service I helped creating

- Use the models through the NLP Cloud API in production and enrich my Django application out of it

That's really cool to see how accuracy of pre-trained models is improving by simply switching to v3.0

I've been using the v3 nightly version for 2 months and it works like a charm. I'm now training models with v3 and using them in production without any issue.

Great job!