The customs-cleared trade surplus shrank 21.7 percent in February from a year earlier to 1.09 trillion yen for the second straight month of decline.
The Finance Ministry on Wednesday blamed the decline on slowing exports and higher oil prices, which inflated the value of imports.
The nation's exports rose 1.7 percent to 4.85 trillion yen, up for the 15th straight month. But in volume terms, exports fell 4.2 percent for the second straight monthly decrease, the ministry said in a preliminary report.
Exports to China, excluding Hong Kong, dropped 2.2 percent to 576.8 billion yen for the first fall in 38 months, it said.
Overall, imports increased 11.3 percent to 3.75 trillion yen, up for the 12th month.
Imports of crude oil expanded 20.4 percent and coal 41.3 percent, while exports of metal-processing machinery rose 33.1 percent.
Private-sector economists are generally not concerned about recent falls in the trade surplus because last year's levels were too high and the world economy is expected to post solid growth in the first half of this year.
"We expect exports to gather steam in the April-June period, giving a slight increase in the trade surplus from year-earlier levels as the U.S. and Chinese economies are projected to remain steady," said Hiromichi Shirakawa, chief economist at UBS Securities Japan Ltd.
"But exports may slow in the July-September quarter as the United States is expected to step up credit-tightening measures in the face of growing inflationary pressure," Shirakawa said.
A Finance Ministry official said he finds it difficult to expect exports to post a double-digit increase in the coming months as recent data showed weakness, especially when a growing number of Japanese companies are shifting their production bases to China and other overseas markets.
According to the report, Japan's trade surplus with the rest of Asia dipped 31.1 percent to 541.7 billion yen.