compass.web.website_crawl.COMPASSLinkScorer#

class COMPASSLinkScorer(keyword_points=None)[source]#

Bases: ELMLinkScorer

Custom URL scorer for COMPASS website crawling

Parameters:: keyword_points (dict, optional) – Dictionary mapping keywords to their associated point values for scoring URLs. If None, uses BEST_ZONING_ORDINANCE_KEYWORDS. By default, None.

Methods

score(links)

Score a link based on its title text and URL.

async score(links)#

Score a link based on its title text and URL.

The score is calculated by summing the point values of the keywords found in both the link title text and the URL. The higher the score, the more relevant the link is considered to be for the purpose of retrieving zoning ordinance documents.

Parameters:: links (list of dicts) – A list of dictionaries representing links to be scored. Each dictionary should contain at least the keys “text” and “href”. The “text” key should contain the link title text, and the “href” key should contain the URL.
Returns:: links (list of dicts) – The input list of links with an additional key “score” added to each dictionary. Each “score” key contains the calculated score for that link.