A survey on the evaluation of intent understanding in smart home voice interaction
PAN Yueran
JIAO Limin
YANG Rongxiang
CUI Qingyong
YAO Changsong
XU Yi
Abstract：With the integration of artificial intelligence(AI)and the Internet of Things(IoT),smart home voice interaction is evolving from functional command execution to demand understanding and proactive service.Intent Understanding is the core of this service,and its evaluation system is crucial for measuring voice assistant performance and user experience.Reviews the current status of evaluation for intent understanding in smart home voice interaction,systematically summarizing its technical evolution,datasets,and metric systems.Addressing existing limitations,it identifies three major challenges:difficulty in quantifying intelligence,insufficient coverage of complex interactions,and lack of corpus standardization.Further envisions a human-centered evaluation framework guided by user experience and emotional intelligence.Also puts forward three construction paths:standardizing data and scripts,unifying metrics and reporting protocols,and validating transferability between simulation and real-world scenarios,providing a reference for achieving human-centric intelligence evaluation.
Keywords：Smart homeVoice interactionIntent understandingEvaluation frameworkLarge Language Model(LLM)
Publication Date：2025-12-09
Online Publishing Date：2026-01-06(First online date of this platform, not the publication date of the document)
Pages：7( 17-23 )