rex.utilities.execution.SpawnProcessPool
- class SpawnProcessPool(*args, loggers=None, **kwargs)[source]
Bases:
ProcessPoolExecutorAn adaptation of concurrent futures ProcessPoolExecutor with spawn processes instead of fork or forkserver.
- Parameters:
loggers (str | list, optional) – logger(s) to initialize on workers, by default None
Methods
map(fn, *iterables[, timeout, chunksize])Returns an iterator equivalent to map(fn, iter).
shutdown([wait, cancel_futures])Clean-up the resources associated with the Executor.
submit(fn, /, *args, **kwargs)Submits a callable to be executed with the given arguments.
- map(fn, *iterables, timeout=None, chunksize=1)
Returns an iterator equivalent to map(fn, iter).
- Args:
- fn: A callable that will take as many arguments as there are
passed iterables.
- timeout: The maximum number of seconds to wait. If None, then there
is no limit on the wait time.
- chunksize: If greater than one, the iterables will be chopped into
chunks of size chunksize and submitted to the process pool. If set to one, the items in the list will be sent one at a time.
- Returns:
An iterator equivalent to: map(func, *iterables) but the calls may be evaluated out-of-order.
- Raises:
- TimeoutError: If the entire result iterator could not be generated
before the given timeout.
Exception: If fn(*args) raises for any values.
- shutdown(wait=True, *, cancel_futures=False)
Clean-up the resources associated with the Executor.
It is safe to call this method several times. Otherwise, no other methods can be called after this one.
- Args:
- wait: If True then shutdown will not return until all running
futures have finished executing and the resources used by the executor have been reclaimed.
- cancel_futures: If True then shutdown will cancel all pending
futures. Futures that are completed or running will not be cancelled.