elm.ords.services.threaded.StoreFileOnDisk
- class StoreFileOnDisk(out_dir, tpe_kwargs=None)[source]
Bases:
ThreadedService
Abstract service that manages the storage of a file on disk.
Storage can occur due to creation or a move of a file.
- Parameters:
out_dir (path-like) – Path to output directory where file should be stored.
tpe_kwargs (dict, optional) – Keyword-value argument pairs to pass to
concurrent.futures.ThreadPoolExecutor
. By default,None
.
Methods
Open thread pool and temp directory
call
(*args, **kwargs)Call the service.
process
(doc)Store file in out directory.
process_using_futures
(fut, *args, **kwargs)Process a call to the service.
Shutdown thread pool and cleanup temp directory
Attributes
Max number of concurrent job submissions.
Always
True
(limiting is handled by asyncio)Service name used to pull the correct queue object.
- async process(doc)[source]
Store file in out directory.
- Parameters:
doc (elm.web.document.Document) – Document containing meta information about the file. Must have relevant processing keys in the metadata dict, otherwise the file may not be stored in the output directory.
- Returns:
Path | None – Path to output file, or None if no file was stored.
- MAX_CONCURRENT_JOBS = 10000
Max number of concurrent job submissions.
- acquire_resources()
Open thread pool and temp directory
- async classmethod call(*args, **kwargs)
Call the service.
- Parameters:
*args, **kwargs – Positional and keyword arguments to be passed to the underlying service processing function.
- Returns:
obj – A response object from the underlying service.
- async process_using_futures(fut, *args, **kwargs)
Process a call to the service.
- Parameters:
fut (asyncio.Future) – A future object that should get the result of the processing operation. If the processing function returns
answer
, this method should callfut.set_result(answer)
.**kwargs – Keyword arguments to be passed to the underlying processing function.
- release_resources()
Shutdown thread pool and cleanup temp directory