KEYWORD EXTRACTION FROM UNIFORM RESOURCE LOCATORS (URLS)

摘要

The keyword extraction technique described herein extracts keywords from Uniform Resource Locators (URLs) in web logs. The technique leverages the content and the structure of URLs to extract relevant keywords. First, a URL is divided into multiple components based on its structure. A set of keywords are extracted from each component of the URL independently with the help of a controlled vocabulary. Then a second set of keywords are generated by forming combinations of terms from different segments of the URL. Only those combinations which are present in the controlled vocabulary are retained as keywords. Finally, the keywords are scored with a function which took into account of a wide set of features.