A meeting between Chinese and ASEAN ministers over the South China Sea has ended in confusion after Malaysia released and then retracted a joint statement that expressed "serious concerns" over developments in the disputed waterway.

The disarray raises fresh questions about unity within the 10-member Association of Southeast Asian Nations over the issue ahead of an international court ruling on a Philippine challenge to China's claims to more than 80 percent of the waterway. ASEAN operates on consensus, which means all members need to agree on a statement before it is released.

ASEAN has avoided citing China by name in prior statements calling for a lowering of tensions over the area.