Inoki, a former professional wrestling star, arrived in Beijing on Thursday en route to Tokyo.
The Upper House member from Nippon Ishin no Kai (Japan Restoration Party) could be punished by the Diet because he traveled to North Korea without obtaining permission from fellow lawmakers.
The Upper House Steering Committee withheld approval for Inoki to make the trip, saying the purpose of his visit was unclear.
Lawmakers are supposed to obtain Diet approval before leaving the country during legislative sessions.
Inoki's meeting with Jang Song Thaek took place Wednesday at the Pyongyang Indoor Stadium before basketball games between Japanese students and North Koreans.
The meeting was also attended by Ri Jong Mu, North Korea's minister of physical culture and sports, and Kenshiro Matsunami, a former Japanese lawmaker who arrived in Pyongyang on Tuesday with students from Nippon Sport Science University.
The two countries have no diplomatic relations and the Japanese government has advised its nationals not to travel to North Korea.
During his stay in Pyongyang, the 70-year-old Inoki, whose real first name is Kanji, attended sports events and opened an office for his nonprofit organization focusing on sports exchanges with foreign countries.
Inoki has close ties with North Korea and this is his 27th visit to the country. He last visited Pyongyang in late July, during which he also met with Jang and other senior North Korean officials.
