Japan closed its embassy in Sanaa on Sunday due to a worsening security situation in Yemen, the Foreign Ministry said on Monday.
Ambassador Katsuyoshi Hayashi and four embassy officials have now left the country, the ministry said.
Some of the staff will continue to provide embassy services from Doha, the capital of Qatar, it said.
Although considered a temporary move, it follows the closure of the U.S. Embassy in Yemen last week and then European missions. Yemen's security has worsened in recent months, with Shiite rebels seizing control of Sanaa. The Houthi militant group declared on Feb. 6 that it had seized power.
"The security situation there is extremely severe," Chief Cabinet Secretary Yoshihide Suga told a press briefing in Tokyo.
He added that other countries are taking similar cautionary measures.
Political unrest has escalated in Yemen recently, where attacks on and kidnappings of foreigners are on the rise. The Foreign Ministry says Japanese nationals have a higher risk of being harmed and recommends that Japanese nationals postpone all visits.
Meanwhile, Self-Defense Forces troops are on an assignment in nearby Djibouti, where they are taking part in an antipiracy mission in waters off Somalia. Suga said their operations remain unaffected.