elm.web.search.run.load_docs

async load_docs(urls, browser_semaphore=None, **kwargs)

Load a document for each input URL

Parameters:

• urls (iterable of str) – Iterable of URL’s (as strings) to fetch.

• browser_semaphore (asyncio.Semaphore, optional) – Semaphore instance that can be used to limit the number of playwright browsers open concurrently for document retrieval. If None, no limits are applied. By default, None.

• kwargs – Keyword-argument pairs to initialize elm.web.file_loader.AsyncFileLoader.

Returns:

list – List of non-empty document instances containing information from the URL’s. If a URL could not be fetched (i.e. document instance is empty), it will not be included in the output list.