elm.utilities.retry.async_retry_with_exponential_backoff

async_retry_with_exponential_backoff(base_delay=1, exponential_base=4, jitter=True, max_retries=3, errors=(<class 'openai.RateLimitError'>, <class 'openai.APITimeoutError'>))

Retry an asynchronous function with exponential backoff.

This decorator works out-of-the-box for OpenAI chat completions calls. To configure it for other functions, set the errors input accordingly.

Parameters:

• base_delay (int, optional) – The base delay time, in seconds. This time will be multiplied by the exponential_base (plus any jitter) during each retry iteration. The multiplication applies at the first retry. Therefore, if your base delay is 1 and your exponential_base is 4 (with no jitter), the delay before the first retry will be 1 * 4 = 4 seconds. The subsequent delay will be 4 * 4 = 16 seconds, and so on. By default, 1.

• exponential_base (int, optional) – The multiplication factor applied to the base delay input. See description of delay for an example. By default, 4.

• jitter (bool, optional) – Option to include a random fractional adder (0 - 1) to the exponential_base before multiplying by the delay. This can help ensure each function call is submitted slightly offset from other calls in a batch and therefore help avoid repeated rate limit failures by a batch of submissions arriving simultaneously to a service. By default, True.

• max_retries (int, optional) – Max number of retries before raising an ELMRuntimeError. By default, 3.

• errors (tuple, optional) – The error class(es) to signal a retry. Other errors will be propagated without retrying. By default, (openai.RateLimitError, openai.APITimeoutError).

References

https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/