The $550 billion Japan pledged to invest in the United States in exchange for lower tariffs is "our money," U.S. President Donald Trump said Wednesday, and the U.S. is free to deploy those funds as it sees fit.

He made the comments as Ryosei Akazawa, Japan's chief tariff negotiator, returned to Washington to discuss the handshake agreement reached just weeks ago between the two countries, a deal that is looking increasingly shaky in part over differences about the investment pledge.

“I got a signing bonus from Japan of $550 billion – that’s our money. It’s our money to invest, as we like,” Trump told CNBC in a telephone interview.