Your job is not to be their friend, your job is to make them as effective a team as possible by identifying impediments to them becoming their best selves. If their work is being done, objectively you should know this, then your next task is to find ways to improve the with being done. If there's enough time to socialize then there's enough time for improvement. This is where you find time for projects that might not work, experiments that might improve quality, quantity, or just improve the overall experience for you customers and team members. There are tons of ways to be firm without micro managing.

Here's what I would recommend for you.

* Define the primary work that needs to be done. What is the purpose of your team? How effective are they at this?

* Identify the gaps. Where can you improve? What can you be doing better to delight your customers? What skills or process improvements can you add that will make things better?

* Who can you build to take your spot? You can't be promoted if no one could fill your role. This is hard and you'll get it wrong a few times, that's fine. Just keep working to build up your team

Complaints and gripes are common at first. No one likes change. Eventually it will become normal and your team will adjust. No need to micromanage, no need to complain about showing up on time unless it's crucial, i.e. a call center, just set your expectations and took forward the work starts with you though.