Skip to main content

SessionPool

A pool of sessions that are managed, rotated, and persisted based on usage and age.

It ensures effective session management by maintaining a pool of sessions and rotating them based on usage count, expiration time, or custom rules. It provides methods to retrieve sessions, manage their lifecycle, and optionally persist the state to enable recovery.

Index

Methods

Properties

Methods

__aenter__

  • Initialize the pool upon entering the context manager.

    Returns SessionPool

__aexit__

  • async __aexit__(*, exc_type, exc_value, exc_traceback): None

  • Deinitialize the pool upon exiting the context manager.

    Parameters

    • optionalkeyword-onlyexc_type: type[BaseException] | None
    • optionalkeyword-onlyexc_value: BaseException | None
    • optionalkeyword-onlyexc_traceback: TracebackType | None

    Returns None

__init__

  • __init__(*, max_pool_size, create_session_settings, create_session_function, event_manager, persistence_enabled, persist_state_kvs_name, persist_state_key): None

  • A default constructor.

    Parameters

    • optionalkeyword-onlymax_pool_size: int = 1000

      Maximum number of sessions to maintain in the pool. You can add more sessions to the pool by using the add_session method.

    • optionalkeyword-onlycreate_session_settings: dict | None = None

      Settings for creating new session instances. If None, default settings will be used. Do not set it if you are providing a create_session_function.

    • optionalkeyword-onlycreate_session_function: CreateSessionFunctionType | None = None

      A callable to create new session instances. If None, a default session settings will be used. Do not set it if you are providing create_session_settings.

    • optionalkeyword-onlyevent_manager: EventManager | None = None

      The event manager to handle events like persist state.

    • optionalkeyword-onlypersistence_enabled: bool = False

      Flag to enable or disable state persistence of the pool. If it is enabled, make sure to provide an event manager to handle the events.

    • optionalkeyword-onlypersist_state_kvs_name: str = 'default'

      The name of the KeyValueStore used for state persistence.

    • optionalkeyword-onlypersist_state_key: str = 'CRAWLEE_SESSION_POOL_STATE'

      The key under which the session pool's state is stored in the KeyValueStore.

    Returns None

__repr__

  • __repr__(): str

  • Get a string representation.

    Returns str

add_session

  • add_session(*, session): None

  • Add an externally created session to the pool.

    This is intened only for the cases when you want to add a session that was created outside of the pool. Otherwise, the pool will create new sessions automatically.

    Parameters

    • optionalkeyword-onlysession: Session

      The session to add to the pool.

    Returns None

get_session

  • Retrieve a random session from the pool.

    This method first ensures the session pool is at its maximum capacity. If the random session is not usable, retired sessions are removed and a new session is created and returned.

    Returns Session

get_session_by_id

  • async get_session_by_id(*, session_id): Session | None

  • Retrieve a session by ID from the pool.

    This method first ensures the session pool is at its maximum capacity. It then tries to retrieve a specific session by ID. If the session is not found or not usable, None is returned.

    Parameters

    • optionalkeyword-onlysession_id: str

      The ID of the session to retrieve.

    Returns Session | None

get_state

  • get_state(*, as_dict): dict

  • Parameters

    • optionalkeyword-onlyas_dict: Literal[true]

    Returns dict

get_state

get_state

  • Retrieve the current state of the pool either as a model or as a dictionary.

    Parameters

    • optionalkeyword-onlyas_dict: bool = False

    Returns SessionPoolModel | dict

reset_store

  • async reset_store(): None

  • Reset the KVS where the pool state is persisted.

    Returns None

Properties

active

active: bool

Indicate whether the context is active.

retired_session_count

retired_session_count: int

Get the number of sessions that are no longer usable.

session_count

session_count: int

Get the total number of sessions currently maintained in the pool.

usable_session_count

usable_session_count: int

Get the number of sessions that are currently usable.

Page Options