Skip to main content

SessionPool

Session pool is a pool of sessions that are rotated based on the usage count or age.

Index

Methods

Properties

Methods

__aenter__

  • Initialize the pool upon entering the context manager.

    Returns SessionPool

__aexit__

  • async __aexit__(*, exc_type, exc_value, exc_traceback): None

  • Deinitialize the pool upon exiting the context manager.

    Parameters

    • optionalkeyword-onlyexc_type: type[BaseException] | None
    • optionalkeyword-onlyexc_value: BaseException | None
    • optionalkeyword-onlyexc_traceback: TracebackType | None

    Returns None

__init__

  • __init__(*, max_pool_size, create_session_settings, create_session_function, event_manager, persistence_enabled, persist_state_kvs_name, persist_state_key): None

  • A default constructor.

    Parameters

    • optionalkeyword-onlymax_pool_size: int = 1000

      Maximum number of sessions to maintain in the pool. You can add more sessions to the pool by using the add_session method.

    • optionalkeyword-onlycreate_session_settings: dict | None = None

      Settings for creating new session instances. If None, default settings will be used. Do not set it if you are providing a create_session_function.

    • optionalkeyword-onlycreate_session_function: CreateSessionFunctionType | None = None

      A callable to create new session instances. If None, a default session settings will be used. Do not set it if you are providing create_session_settings.

    • optionalkeyword-onlyevent_manager: EventManager | None = None

      The event manager to handle events like persist state.

    • optionalkeyword-onlypersistence_enabled: bool = False

      Flag to enable or disable state persistence of the pool. If it is enabled, make sure to provide an event manager to handle the events.

    • optionalkeyword-onlypersist_state_kvs_name: str = 'default'

      The name of the KeyValueStore used for state persistence.

    • optionalkeyword-onlypersist_state_key: str = 'CRAWLEE_SESSION_POOL_STATE'

      The key under which the session pool's state is stored in the KeyValueStore.

    Returns None

__repr__

  • __repr__(): str

  • Get a string representation.

    Returns str

add_session

  • add_session(*, session): None

  • Add a specific session to the pool.

    This is intened only for the cases when you want to add a session that was created outside of the pool. Otherwise, the pool will create new sessions automatically.

    Parameters

    • optionalkeyword-onlysession: Session

    Returns None

get_session

  • Retrieve a random session from the pool.

    This method first ensures the session pool is at its maximum capacity. If the random session is not usable, retired sessions are removed and a new session is created and returned.

    Returns Session

get_session_by_id

  • async get_session_by_id(*, session_id): Session | None

  • Retrieve a session by ID from the pool.

    This method first ensures the session pool is at its maximum capacity. It then tries to retrieve a specific session by ID. If the session is not found or not usable, None is returned.

    Parameters

    • optionalkeyword-onlysession_id: str

      The ID of the session to retrieve.

    Returns Session | None

get_state

  • get_state(*, as_dict): dict

  • Parameters

    • optionalkeyword-onlyas_dict: Literal[true]

    Returns dict

get_state

get_state

  • Retrieve the current state of the pool either as a model or as a dictionary.

    Parameters

    • optionalkeyword-onlyas_dict: bool = False

    Returns SessionPoolModel | dict

reset_store

  • async reset_store(): None

  • Reset the KVS where the pool state is persisted.

    Returns None

Properties

retired_session_count

retired_session_count: int

Get the number of sessions that are no longer usable.

session_count

session_count: int

Get the total number of sessions currently maintained in the pool.

usable_session_count

usable_session_count: int

Get the number of sessions that are currently usable.

Page Options