elm.ords.services.threaded.FileMover
- class FileMover(out_dir, tpe_kwargs=None)[source]
Bases:
StoreFileOnDisk
Service that moves files to an output directory
- Parameters:
out_dir (path-like) – Path to output directory where file should be stored.
tpe_kwargs (dict, optional) – Keyword-value argument pairs to pass to
concurrent.futures.ThreadPoolExecutor
. By default,None
.
Methods
Open thread pool and temp directory
call
(*args, **kwargs)Call the service.
process
(doc)Store file in out directory.
process_using_futures
(fut, *args, **kwargs)Process a call to the service.
Shutdown thread pool and cleanup temp directory
Attributes
Max number of concurrent job submissions.
Always
True
(limiting is handled by asyncio)Service name used to pull the correct queue object.
- MAX_CONCURRENT_JOBS = 10000
Max number of concurrent job submissions.
- acquire_resources()
Open thread pool and temp directory
- async classmethod call(*args, **kwargs)
Call the service.
- Parameters:
*args, **kwargs – Positional and keyword arguments to be passed to the underlying service processing function.
- Returns:
obj – A response object from the underlying service.
- async process(doc)
Store file in out directory.
- Parameters:
doc (elm.web.document.Document) – Document containing meta information about the file. Must have relevant processing keys in the metadata dict, otherwise the file may not be stored in the output directory.
- Returns:
Path | None – Path to output file, or None if no file was stored.
- async process_using_futures(fut, *args, **kwargs)
Process a call to the service.
- Parameters:
fut (asyncio.Future) – A future object that should get the result of the processing operation. If the processing function returns
answer
, this method should callfut.set_result(answer)
.**kwargs – Keyword arguments to be passed to the underlying processing function.
- release_resources()
Shutdown thread pool and cleanup temp directory