Concerns over labor abuse in Malaysia, long present in the Southeast Asian country, have escalated in the past three months, continuing to spread beyond the palm oil industry to other parts of the economy.

It’s a problem with many implications, first and foremost for the workers themselves, many of whom live in squalid conditions. It also has the potential to threaten the country’s output by stifling direct investment and killing supply contracts.

But a third fallout is in the world of investing, where more and more of the biggest money managers say it’s giving them second thoughts. As labor abuse claims emerged at a supplier to vacuum-cleaner giant Dyson and the glove-maker Supermax Corp., it’s also weighing on the country’s stock market and hurting its people’s pension investments.