In academia you would be competing the rest of your career with coworkers who spent their sabbatical teaching in a foreign country or being a principle organizer administrator of a conference or writing a very important paper or book or otherwise adding self directed value to their resume.

So technically you can spend it on the beach, but don't be surprised when your coworker who learned "language or framework of the week" is going to get the choice assignment or promotion you really wanted. Of course if you edit your book or learn a new language while sitting at a beach, or you are the primary organizer of a tech conference that happens to be on a tropical island or other vacation spot ...