Research of Searching Strategy in Candidate Link Introducing Topic Link Blocking Factor
ZHOU Xue
LIU Naiwen
Abstract：In crawling process,the urls'weight is need to compute,the crawl queue is filled to meet the crawl conditions.It's the key problem that how to find the most relevant links to the theme and how to avoid"theme drift"problem.Due to anchor text is short,it can't clearly show the page's relevance to the topic which the page linked to.On the basis of Shark- search algorithm intro-ducing the related link weights,the neutron link anchor text is used for calculating blocks'weight.Through contrasted experiments, verified the effectiveness of the improved algorithm is verfied,it can better distinguish the links'relevance score in the same page, improve the precision of the crawler and moderate"theme drift"problem at the same time.
Keywords：page-blockShark-search algorithmlink-structuretopic-relative link block
Publication Date：2018-01-01
Online Publishing Date：2025-08-15(First online date of this platform, not the publication date of the document)
Pages：5( 874-878 )