You most definitely can, the main difference is that only partial ~2% of the parameters get updated during training. Say you start from a model like llama-70B which already knows english and has some world knowledge based on its pretraining dataset. It might not be ideal for drastic domain shifts, such as adapting a model to learn new languages (which might require a new tokenizer and model embeddings) but still might be possible to some extent.