You've pretty much describe where our process has been iterated to.
The story points are less about "how many hours will this take" and more "is this broken down to a manageable level".
Our team is small and we're largely working on legacy systems that nobody at the company has any clear picture on so we do still do things collaboratively in a meeting in order to quickly get to the best information we can and iterate on bits and pieces of the puzzle that may not be common knowledge.
But from there the points are mainly "Small, Medium, Big" (1-2, 3, 5). If anyone throws something out at 8 points the question is immediately whether or not we have sufficient clarity on the work. Why is this 8 points? Is it because we need to rotely reimplement the same functionality across two dozen nearly identical form fields, or is it because we've glossed over some major area of the scope and are hedging based on our _assumptions_ about how it works?
In the latter case, we'll instead see about adjusting the work breakdown and/or throwing in a ticket to actually investigate and document things better before we circle back to the original task.
If we see 13 points we know from experience to run for the hills. That's almost always someone saying "we don't have enough information, but it's going to be big".
Those points feed into two things:
1. A rough estimation of long term timelines. We know average velocity, and we can look big picture at the number of points involved in some sort of higher level initiative and get a feel for "is this a 3 month project or 12 month project?".
2. Trying not to overcommit on each sprint. We know the average we can deliver, and can use them to raise a flag when someone seems to be getting overly optimistic about their output.
Similarly, our daily standup is not focused on being a status meeting. It's to encourage communication. To help kick things off and jog people's memory, we take about 60 seconds to quickly list out tickets that are sitting in a "review" or "deploy" status (if it's nearly done let's get it over the line), "blocked" status (if you're waiting on someone, have you communicated it to them?) as well as any that, based on the points, seem to be taking longer than expected (do you need to ask someone for help or communicate changes in scope?). Then it's just hands up for anyone that needs to say anything, call for any input from the business side, some chat and banter, etc, and we're out. Usually takes about 5 minutes.