Does the LLM get trained as the database get builded?

As you train a model it should build it's knowledge database at the same time. I guess that's how it work?

When an LLM get trained on next word guessing, can it search the web to improve his results?