Research on hot topic detection based on incremental text clustering algorithm
WEI Yize
GUO Hui
SHI Xiaoxu
Abstract：In order to address the problems of traditional TF-IDF methods not being able to incrementally up-date and having low accuracy when extracting text features and the traditional Single-Pass algorithm has a low clustering accuracy in traditional Single-Pass algorithm clustering this paper reduces the dependency on the corpus when calculating TF-IDF by using an existing corpus to set up IDF table and update it.It improves the accuracy of Single-Pass algorithm in clustering by computing the mean to determine cluster centers.The model is validated using COVID-19 news data obtained from various platforms.The results show that this method allows for incremental updating of traditional TF-IDF keywords extraction,and the improved Single-Pass algorithm can increase the comprehensive evaluation index by 8.64%.Compared to the traditional Single-Pass algorithm,the improved Single-Pass algorithm only needs to compare with a subset of candidate clusters,effectively reducing the number of comparisons and improving the accuracy and efficiency of cluste-ring.
Keywords：Single-Passtext clusteringtext similarityhot topic detectionTF-IDF
Publication Date：2024-02-28
Online Publishing Date：2025-08-15(First online date of this platform, not the publication date of the document)
Pages：7( 76-81,124 )