If you believe in the Cellular Automaton Interpretation of Quantum Mechanics by the Nobel laureate Gerard 't Hooft (which is somewhat controversial among physicists), building a sufficiently large quantum computer will probably be impossible. His book is available for free:

> https://link.springer.com/book/10.1007%2F978-3-319-41285-6

Simply read section 5.8.

It depends on what you mean by "doubts". In the strict mathematical proof sense, we don't know if quantum computers are more powerful than classical computers at any task. In other words, like many conjectures in complexity theory (like P!=NP), the statement BQP!=P is still a conjecture. But computer scientists have good reasons to think that these conjectures will be true, after many decades of grappling with these problems and failing to prove otherwise. Therefore, in this informal expert-feeling sense of the word, there are not many "doubts".

This isn't even a real-world example. It's yet another circuit on paper.