We could absolutely stop all weapons production in the US, I’d just ask for some time to learn Mandarin first.
The work defence contractors do and will continue to do is a measurable positive in the world. This work generally falls into three categories:
First is projects that make the lives of armed forces personnel easier. There is no moral or ethical hazard with these projects. They simply are making it so that pre-existing work, often non-combat work, is less tedious.
Second is projects that actively protect the lives of armed forces personnel and assets. Why would somebody have any reason to feel anything but pride in their work when they know that the only things it could ever do is save lives.
Third is the arguably morally shaky stuff. Weapons technology would fall under this category. The thing people seem to miss about this category however is what new projects' goals are. Very rarely is it ever to create a more lethal weapon. Almost all development of this kind focuses on either making existing technology cheaper, more precise, or have increased range.
In the first case, you could argue that it makes the weapons more likely to be used which is fair however if a weapon system is in active use already, it is unlikely that a cheaper version would do anything other than bring down operating costs. In the second case, these projects are actively making these weapons safer. Look at the Hellfire R9X as a prime example. This weapon sole purpose is to minimise casualties and minimise damage to the surrounding area. Finally the third and last case I mentioned. Increasing range is important for force projection reasons but the most immediate benefit is that it allows armed forces personnel to be further from danger. These people would be in this fight regardless but now they are able to operate from a safer distance instead.
Weapons tech gets a bad rap for obvious reasons but I see that as mostly having been a relic of a past era. Nowadays there isn't really any reason to make more lethal weapons. We already have those and you can see a steady trend in new military technology towards lower risks, costs, damage to the surroundings, and minimising civilian casualties.
Additionally, in my experience, you generally have a choice whether you work on a project or not. Leadership fully understands what they do as an organisation and get that some people may not be comfortable with doing certain projects. If you can't conscientiously work on a certain type of project, simply say so and leadership will take that into account when planning personnel assignments for projects.
TL;DR Armed conflict and existing weapons will never go away. What we as a society(namely the modern purpose of defence contractors) can do instead is make armed conflict less dangerous to those caught in the crossfire and those who risk life and limb to defend our countries.
Not judging your decision (although I personally prefer not working for defense/defense contractors), just pointing out that "it only makes soldiers safer!" isn't an ironclad rebuttal.
Analogy: Would you consider it unethical to provide tools and services that make the lives of human traffickers easier? To actively protect assets of drug lords distributing contaminated opiates and meth?
Slave- and drug trade are about as likely to go away as armed conflict.
This is in the same way that corrupt charity organisations can defraud the causes they claim to support and corporations can knowingly risk the lives of their workers due to either negligence, mismanagement, or for the sole purpose of saving costs or making more money. This doesn't make charities or corporations unethical solely because they have the potential to be.
Additionally, armed forces can and do maintain their operational fitness by using their immense logistics networks and pools of human labour to do very real quantifiable good in the world. Outside of their use as a deterrent and show of power, the US armed forces spends a large chunk of their time and effort responding to natural disasters, taking part in humanitarian efforts, and building infrastructure.
---
On to the analogy, I would like to reframe both examples a bit.
For the purpose of this discussion, illegal transportation of individuals falls into two categories: Human trafficking and human smuggling. Human trafficking is done without consent of the individuals being transported. Conversely, human smuggling is done with consent. Despite being illegal, human smuggling provides a very important service to individuals. It may be against the law but it can be argued that it can provide a net benefit.
Mind you that it comes with severe downsides and by no means am I trying to minimise those. I'm just trying to keep this from turning into a 20 page essay on the topic. One of those downsides namely is that it increases the volume of people being transported which makes it cheaper and easier for human trafficking rings to operate. Another downside is that a large number of people die during transportation, both those consenting and non-consenting.
Now if you move onto the less clear cut outcomes, human smuggling allows individuals to enter a country without going through the vetting process. This opens a vector for criminals and terrorists to enter a country however at the same time, it provides a means for people who either aren't willing or aren't able to wait untold years to get a visa to enter the country only for it to then not be renewed a few months or years down the road. These may be individuals or families that want to be productive members of society in a country with a higher standard of living or are trying to move to a location where they can be treated for medical issues that they can't be treated for at home. Another common case in human smuggling is individuals using human smuggling to cross countries that have closed their borders to them so that they can get to a country that will accept them as refugees. The number of reasons somebody would illegally enter a country go on quite a bit and the list is full of both morally sound and morally reprehensible reasons.
With that in mind, while I personally wouldn't participate in any type of business connected to the illegal movement of people across national borders, I see no moral or ethical issues with someone who does so for the sole purpose of making the process safer. Despite the fact that it is illegal, the way I see it is that you are reducing the risk that people lose life and limb.
---
Moving on to the other example. My opinion is that most drugs should be legalised and regulated. By moving to a legitimate market and starving these organisations' source of income, we should see drug lords be replaced by corporations. You could argue that the cartels would just find a new market but I see giving them one less black market to deal in as a plus. If this were to happen, in the US those drugs would be required to meet FDA & ATF standards. Sure those that don't would still exist much like improperly distilled/methylated moonshine is still an issue however I could only hope that deaths and injuries due to contaminations would plummet compared to where we are now. Why would you buy contaminated drugs when cheaper, pure, regulated, and mass produced variants exist legally. At this point wouldn't working to improve safety standards and effective distribution in this industry be no more morally or ethically problematic than working for existing alcohol and tobacco corporations?
Now past the hypothetical, I would argue that making things less dangerous would be tangential to your example. Making the drugs themselves less dangerous would be working to increase production standards and decrease contaminations and impurities. With regard to the transportation aspect, drug trafficking is extraordinarily dangerous for those involved. Drug mules expose themselves to immense risk when carrying large doses within their bodies across borders. This is by no means the only way the drugs are transported across the border but if somebody was already involved and wanted to make a difference, making this safer would be one way. Drug mules are often either doing so as a way to support themselves financially, as part of a deal to allow them to be smuggled across the border, or against their will in some cases due to human trafficking or blackmail. In most of those cases, the mule is a more or less unwilling participant. Lowering the risk for these individuals is in my eyes both morally and ethically valid.
---
Ultimately I think all of this comes down to this. If you dedicate yourself to work that reduces the loss of life and limb, generally I find that to be a just cause. Defence contractors do this by developing weapon systems that operate at long range with surgical precision to minimise casualties. Armed forces generally do this by serving as a deterrent to violence and by responding to natural disasters and humanitarian crises. In the same sense, there can be people whose work with drug cartels and organisations facilitating illegal human transportation results in a positive impact on people.
Another way to put it would be that I would prefer a drug cartel making a pure product(of quality meeting FDA standards) that is transported without risking people's lives rather than what we have now. In the same way I would rather we have an organisation facilitating human smuggling and trafficking where nobody died or was permanently injured in transit than one where people died and were injured. Obviously in an ideal world we would prefer that there wasn't a reason for either to exist but like you said, they aren't going away any time soon and just because in these cases the organisation might be unethical it shouldn't make the people trying to improve the situation from the inside unethical.
Note: This post is a bit rushed and quite long winded as I wrote it up while taking a break and I've spent more time on this than I probably should have already. I would like to be able to actually write up a proper concise response with citations to back up my assertions however this should hopefully get enough of my point across.
> These people would be in this fight regardless but now they are able to operate from a safer distance instead.
I sincerely strongly doubt US would be fighting 5 wars with boots-on-the-ground in Afganistan, Pakistan, Somalia, Yemen and Iraq, if it weren't for remotely-operated drones.