Prime Minister Shinzo Abe urged South Korea to remove a new statue dedicated to the girls and women hauled into Japan’s military brothels before and during the war, and called on Seoul to stick to the bilateral agreement that was inked in 2015 to settle the thorny issue once and for all.

“The South Korean side should show its sincerity,” the prime minister said on a TV program aired Sunday, referring to the golden statue of a seated young girl installed late last month outside the Japanese Consulate in Busan.

The statue is one of a number in South Korea representing those forced to provide sex for Imperial Japanese troops before and during World War II. Japan euphemistically refers to them as the ianfu, or “comfort women.”

According to media reports, the one in Busan was erected by protesters on the anniversary of the December 2015 bilateral agreement, the text of which only mentions the more famous statue outside the Japanese Embassy in Seoul.

Abe said on an NHK program recorded Friday that South Korea should carry out the agreement whether its leadership changes or not, as this is “a matter of credibility.”

Japanese Ambassador to South Korea Yasumasa Nagamine will return home temporarily Monday in protest of the Busan statue, the Japanese Foreign Ministry said Sunday. He will return to Japan for about a week, which is shorter than the 12-day recall of former Ambassador Masatoshi Muto in 2012, when Tokyo protested then-President Lee Myung-bak’s visit to a pair of disputed rocks in the Sea of Japan, a source close to bilateral relations said. The tiny South Korean-held islets are called Takeshima in Japan and Dokdo in South Korea.

“The issue of statue’s installation is not as serious as the visit” by Lee, the source said.

Chief Cabinet Secretary Yoshihide Suga said last week that the government will recall Nagamine as well as Yasuhiro Morimoto, the consul general in Busan.

South Korea Foreign Minister Yun Byung-se summoned Nagamine on Friday to directly express his regret over the recall, though they also reaffirmed their shared intention to fulfill the comfort women agreement, which is not unanimously backed by the survivors.

The statue issue comes amid political turmoil in South Korea, where the National Assembly impeached President Park Geun-hye last month over a political scandal and had her powers transferred to acting President and Prime Minister Hwang Kyo-ahn.

Under the 2015 pact, South Korea said it would strive to solve the issue of the statue in front of the Japanese Embassy in Seoul “in an appropriate manner.” Japan transferred ¥1 billion ($8.5 million) to a Korean fund designed care for the survivors and their families as part of the agreement.

Addressing other issues on NHK, Abe said he would call on U.S. President-elect Donald Trump when they meet to ease the load on Okinawa Prefecture in hosting the bulk of America’s military bases in Japan.

During his campaign, Trump pledged to make allies contribute more to the cost of hosting U.S. military forces.

Local base protests have climbed since early December when an MV-22 Osprey ditched in waters just off the prefecture’s coast.

Sources familiar with bilateral relations say Abe and Trump are arranging for a meeting to be held in the United States later this month shortly after Trump’s Jan. 20 inauguration.

In November, Abe was the first world leader to meet face-to-face with Trump after his victory over Hillary Clinton.