The rich and powerful versus the weak and underpaid.

That's how the International Monetary Fund sees Japan's relationship between employers and workers. Unless more is done to address the problems in the country's labor market, Japan as a whole will be the loser, according to the IMF's mission chief for the country, Luc Everaert.

"We are concerned the reduction in the wage bargaining power of labor has gone too far," Everaert said in an interview in Tokyo. "We have introduced too much flexibility in the Japanese labor market in favor of employers."