Skip to main content

AutoscaledPool

Manages a pool of asynchronous resource-intensive tasks that are executed in parallel.

The pool only starts new tasks if there is enough free CPU and memory available. If an exception is thrown in any of the tasks, it is propagated and the pool is stopped.

Index

Methods

Properties

Methods

__init__

  • __init__(*, system_status, concurrency_settings, run_task_function, is_task_ready_function, is_finished_function, task_timeout, autoscale_interval, logging_interval, desired_concurrency_ratio, scale_up_step_ratio, scale_down_step_ratio): None

  • A default constructor.

    Parameters

    • optionalkeyword-onlysystem_status: SystemStatus

      Provides data about system utilization (load).

    • optionalkeyword-onlyconcurrency_settings: ConcurrencySettings | None = None

      Settings of concurrency levels.

    • optionalkeyword-onlyrun_task_function: Callable[[], Awaitable]

      A function that performs an asynchronous resource-intensive task.

    • optionalkeyword-onlyis_task_ready_function: Callable[[], Awaitable[bool]]

      A function that indicates whether run_task_function should be called. This function is called every time there is free capacity for a new task and it should indicate whether it should start a new task or not by resolving to either True or False. Besides its obvious use, it is also useful for task throttling to save resources.

    • optionalkeyword-onlyis_finished_function: Callable[[], Awaitable[bool]]

      A function that is called only when there are no tasks to be processed. If it resolves to True then the pool's run finishes. Being called only when there are no tasks being processed means that as long as is_task_ready_function keeps resolving to True, is_finished_function will never be called. To abort a run, use the abort method.

    • optionalkeyword-onlytask_timeout: timedelta | None = None

      Timeout in which the run_task_function needs to finish.

    • optionalkeyword-onlyautoscale_interval: timedelta = timedelta(seconds=10)

      Defines how often the pool should attempt to adjust the desired concurrency based on the latest system status. Setting it lower than 1 might have a severe impact on performance. We suggest using a value from 5 to 20.

    • optionalkeyword-onlylogging_interval: timedelta = timedelta(minutes=1)

      Specifies a period in which the instance logs its state, in seconds.

    • optionalkeyword-onlydesired_concurrency_ratio: float = 0.9

      Minimum level of desired concurrency to reach before more scaling up is allowed.

    • optionalkeyword-onlyscale_up_step_ratio: float = 0.05

      Defines the fractional amount of desired concurrency to be added with each scaling up.

    • optionalkeyword-onlyscale_down_step_ratio: float = 0.05

      Defines the amount of desired concurrency to be subtracted with each scaling down.

    Returns None

abort

  • async abort(): None

  • Interrupt the autoscaled pool and all the tasks in progress.

    Returns None

pause

  • pause(): None

  • Pause the autoscaled pool so that it does not start new tasks.

    Returns None

resume

  • resume(): None

  • Resume a paused autoscaled pool so that it continues starting new tasks.

    Returns None

run

  • async run(): None

  • Start the autoscaled pool and return when all tasks are completed and is_finished_function returns True.

    If there is an exception in one of the tasks, it will be re-raised.

    Returns None

Properties

current_concurrency

current_concurrency: int

The number of concurrent tasks in progress.

desired_concurrency

desired_concurrency: int

The current desired concurrency, possibly updated by the pool according to system load.

Page Options