There are a lot of "configuration-free" Webpack solutions popping up. The one thing that they all have in common is optional configuration.
Of course being zero-config you loose some flexibility.
The reality is, at this point, Webpack has become a powerful tool with a big ecosystem around it. Configuration is not that complicated. It used to be an absolute mess, but these days, the documentation is really good and there are many sensible defaults. It's worth learning without these wrapper solutions.
I'd argue one of the reasons Next.js is so popular is because you can get started with no configuration. It has optional configuration, but all the webpack config, css, hot reloading, route splitting, etc. all serves as a solid foundation.