Kudos to you and the team for caring about the culture.

People tend to forget that the company culture - shared interests and activities being a big part of it - plays a huge part in shaping the quality of the product/service. In addition, it helps build a sense of identity and pride among employees, which is especially important for a fast growing company.

Taking some time off to work on a project like this might actually provide huge returns which are not immediately apparent.