COVER Tech updates real-time voice lip-sync and expression feature

COVER Corporation said on September 24 that its tech blog COVER Tech has updated its feature on expression technology that reads voices in real time to connect voice and facial expression. Written by engineer K.M. of the CTO Office, the piece explains an in-house real-time lip-sync system that has gradually been used in 3D streams and videos since around the end of July 2024. Instead of relying on external libraries that struggled with Japanese and long tones, the team built a lightweight Temporal Convolutional Network that classifies audio into seven mouth-shape groups, including five vowels plus nasal/plosive and other consonants, with only about 20 milliseconds of delay and parallel processing for 10 or more performers. The article compares older footage such as Sakura Miko's 2023 birthday live with newer output like hololive 0th Generation's Ray of Jewelry live to show improved response, especially in singing. It also introduces voice-based expression control used in mekPark, which became animated on June 24, 2026, and in holoAN: a system that estimates emotion categories and laughter from the past 1.5 seconds of audio every 0.5 seconds, driving illustration switching with only a microphone. An applied example was Hinami Yuiga's 2026 birthday stream.