> they shouldn't be about work at all

Other tax systems may vary, but in the UK if you have an offsite and it's not overwhelmingly for work (think agendas, sessions, meetings, etc. as proof) - the employee gets taxed on the value as if it was income.

There are ways for the employer to meet the additional tax burden, but far easier to just add in a work component and call it an offsite.