Even if you round up to 1,000 the number of ICS officers and dectuple to 10,000 you get a trivial number of returnees to the UK. The British Army in India returning would have had nugatory impact considering the British had just fought WW2.
Leaving India was bad for the British upper classes because there were fewer jobs as officers that would keep a man in the store to which he had become accustomed on salary in the army. The ICS was less important than that in terms of numbers and its peak social impact was as an inspiration for the British civil service. The ICS is the only organisation in British India that might plausibly have had a large impact on the culture of the British civil service and it was too small to have had an impact by numbers alone as you originally posited.