I really think what you're saying is just something engineers tell themselves to feel better about what they do. I hear it more often from people at FAANG, defense contractors, and other morally ambiguous places than anywhere else.
Also, if you're the guy building a tool that's oppressing someone, you are the guy building the means to oppress someone. There's nothing neutral about that.
But that aside, I do mostly agree. It's nonsense to help produce something that you know will be misused and then absolve yourself of responsibility.
The problem is not that people like this crap, it's that building and selling it is absurdly profitable and people like money. I don't know how you address such a thing other than to have government step in and block it (see GDPR, tracking cookies etc)
Using a gun on another human to defend my family from immediate threat. Moral.
Using a gun on another human to inflict harm on an innocent. Immoral.
Thus "tech is neutral, usage determines morality".
HOWEVER
what if we are in a society where using guns is the normal way to resolve conflict? Where everyone is required to carry a gun at all times and be prepared to use it to defend their family? Is the tech still neutral when it becomes a cornerstone of every interaction?
Not a fanciful example. Think: dueling.
To work for a company is to support their mission (unless you're a corporate spy or saboteur). The morality of your work should be partly decided by how much it improves or worsens the morality of the company's actions. Not "guilt by association," but "guilt by participation."
Personally, I think this logic also applies when the outcome of the technology is obvious. If you devote your life to making a mind control helmet, you can't play the "technology is morally neutral" card.
Philosophical disclaimer: None of this is meant to be black and white rules. There are always murky situations involving trollies and stolen loaves of bread.
Some questions I ask myself
Certain technologies force a certain world view. If that world view is not moral, then the technology is not moral.
If a technology is inherently dehumanizing, how is it moral?
Does the technology have room for forgiveness, repentance, and redemption? If not, how is it moral?
The counter-counter arguments is to move up the stack. "computers are neutral, but a computerized system which does X is immoral. So we can only have computerized systems which do Y". But what if the problem is that computerizing something inherently makes it immoral?
A toy example: When something becomes a metric, it loses its value as a metric ("lines of code", "rankings of universities", ...). Computerizing things makes them standardized metrics.