Like you describe, out-sourcing generally provides a net-benefit to society by providing cheaper goods here and jobs in India. But it does not necessary produce a net-benefit to the US, since the trade off is cheaper goods but fewer American jobs.
As a global citizen, I'm not concerned about the latter.
Problem is, other countries aren't necessarily as open as we are to outsourcing and foreign labour. For instance, Western universities have many Asian professors, especially in Math, Science, and Engineering. But South Korean universities very rarely employ foreign professors, at least not full professors.
This goes for many jobs in South Korea -- they simply don't want foreigners to do it, because foreigners aren't Korean. The exception is teaching English Conversation, because the Korean teachers very obviously can't do it well enough themselves. The government is trying to limit South Korea's reliance on foreign ESL teachers though.
This goes for outsourcing too -- the UK government is happy to consider having foreign companies build nuclear power plants or railway lines in the UK, but the South Korean government would rather have everything in South Korea done by South Korean companies, even setting up a company to do it if necessary.
When Westerners open their labour markets to foreigners and foreign companies, what can they do if the foreigners do not reciprocate?
The worldwide median standard of living is a hell of endless hard labor and famine. It would be more egalitarian for outsourcing to continue until everyone is reduced to a level near that, but I'm still concerned about it, partly because nobody would have the luxury of accomplishing anything meaningful.
As a global citizen, I'm not concerned about the latter.