compass.scripts.download.download_known_urls#
- async download_known_urls(jurisdiction, urls, browser_semaphore=None, file_loader_kwargs=None)[source]#
Download documents from known URLs
- Parameters:
jurisdiction (
Jurisdiction
) – Jurisdiction instance representing the jurisdiction corresponding to the documents.urls (iterable of
str
) – Collection of URLs to download documents from.browser_semaphore (
asyncio.Semaphore
, optional) – Semaphore instance that can be used to limit the number of downloads happening concurrently. IfNone
, no limits are applied. By default,None
.file_loader_kwargs (
dict
, optional) – Dictionary of keyword arguments pairs to initializeelm.web.file_loader.AsyncFileLoader
. By default,None
.
- Returns:
out_docs (
list
) – List ofBaseDocument
instances containing documents from the URL’s, or an empty list if something went wrong during the retrieval process.
Notes
Requires
TempFileCachePB
service to be running.