Search engine feedback for developing reliable whois database reference for restricted search operation

摘要

A system and method monitors and weeds out illegitimate/illegal websites during search engine indexing and domain name registration. The whois database generated during domain name registration is used as a reference database for correlation with a database generated by the search crawler on a search engine server. A whois analyzer from the search engine server extracts a set of URLs into a database called the uncorrelated URL database. The uncorrelated URL database contains those URLs from both the aggregate whois database and reverse index database after removing common URLs. The uncorrelated URLs are contacted and advised by the whois administrator to take necessary action to be listed in the whois database and properly be indexed during search engine crawling. This process ensures that every URL is properly registered and identified on the Internet thus eliminating the success of illegal/unwanted websites.