Real-Time Synchronized Interaction Framework for Emotion-Aware Humanoid Robots

As humanoid robots increasingly introduced into social scene, achieving emotionally synchronized multimodal interaction poses significant challenges. To further grow the integration of humanoid robots into service roles, we present a real-time framework for NAO robots that synchronizes speech prosody with full-body gestures through three innovations: (1) A dual-channel emotion engine where LLM simultaneously generates context-aware text responses and biomechanically feasible motion descriptors, constrained by a structured joint movement library; (2) Dynamic time warping enhanced by duration-aware sequencing to temporally align speech output with kinematic motion keyframes; (3) Closed-loop feasibility verification ensuring gestures adhere to NAO’s physical joint limits through real-time adaptation. Evaluations show 21% higher emotional alignment than rule-based systems, achieved by coordinating vocal pitch (valence-driven) with upper-limb kinematics while maintaining lower-body stability. By enabling seamless sensorimotor coordination, this framework advances the deployment of context-aware social robots in dynamic applications such as personalized healthcare, interactive education, and responsive customer service platforms.