In context, his point was that a permanent military industrial complex would be
necessary [0] due to the changing nature of technology and war, but that we must remain vigilant to not allow public policy to be captured by those interests.
People generally disregard the first part when they quote him on the "military industrial complex".
[0] Because the era where the US could remain safely protected by the two oceans was ending due to the development of missile technology, and because the complexity of military equipment had gone beyond what a civilian company could be expected to produce with a few years of factory retooling as in WWII.
If a military was not ready to fight on day 1, if you were starting from a heavily disadvantageous position, you may never have the opportunity to catch up. The weapons technologies on the horizon would allow shipyards and advanced manufacturing facilities to be destroyed by an enemy thousands of miles away. That means you would need to have enough tanks, ships and so forth to put up a fight immediately, rather than waiting a few years, and they would need to be continually modernized. Therefore those industries would need to be kept around permanently, but subservient to the public interest.