ConfigurationOptions
Index
Properties
- availableMemoryRatio
- chromeExecutablePath
- defaultBrowserPath
- defaultDatasetId
- defaultKeyValueStoreId
- defaultRequestQueueId
- disableBrowserSandbox
- eventManager
- headless
- inputKey
- logLevel
- maxUsedCpuRatio
- memoryMbytes
- persistStateIntervalMillis
- persistStorage
- purgeOnStart
- storageClient
- storageClientOptions
- systemInfoIntervalMillis
- xvfb
Properties
optionalavailableMemoryRatio
optionalchromeExecutablePath
Defines a path to Chrome executable.
Alternative to CRAWLEE_CHROME_EXECUTABLE_PATH
environment variable.
optionaldefaultBrowserPath
Defines a path to default browser executable.
Alternative to CRAWLEE_DEFAULT_BROWSER_PATH
environment variable.
optionaldefaultDatasetId
Default dataset id.
Alternative to CRAWLEE_DEFAULT_DATASET_ID
environment variable.
optionaldefaultKeyValueStoreId
Default key-value store id.
Alternative to CRAWLEE_DEFAULT_KEY_VALUE_STORE_ID
environment variable.
optionaldefaultRequestQueueId
Default request queue id.
Alternative to CRAWLEE_DEFAULT_REQUEST_QUEUE_ID
environment variable.
optionaldisableBrowserSandbox
Defines whether to disable browser sandbox by adding --no-sandbox
flag to launchOptions
.
Alternative to CRAWLEE_DISABLE_BROWSER_SANDBOX
environment variable.
optionaleventManager
Defines the Event Manager to be used.
optionalheadless
Defines whether web browsers launched by Crawlee will run in the headless mode.
Alternative to CRAWLEE_HEADLESS
environment variable.
optionalinputKey
Defines the default input key, i.e. the key that is used to get the crawler input value from the default KeyValueStore associated with the current crawler run.
Alternative to CRAWLEE_INPUT_KEY
environment variable.
optionallogLevel
Sets the log level to the given value.
Alternative to CRAWLEE_LOG_LEVEL
environment variable.
optionalmaxUsedCpuRatio
Sets the ratio, defining the maximum CPU usage. When the CPU usage is higher than the provided ratio, the CPU is considered overloaded.
optionalmemoryMbytes
Sets the amount of system memory in megabytes to be used by the AutoscaledPool. By default, the maximum memory is set to one quarter of total system memory.
Alternative to CRAWLEE_MEMORY_MBYTES
environment variable.
optionalpersistStateIntervalMillis
Defines the interval of emitting the persistState
event.
Alternative to CRAWLEE_PERSIST_STATE_INTERVAL_MILLIS
environment variable.
optionalpersistStorage
Defines whether the storage client used should persist the data it stores.
Alternative to CRAWLEE_PERSIST_STORAGE
environment variable.
optionalpurgeOnStart
Defines whether to purge the default storage folders before starting the crawler run.
Alternative to CRAWLEE_PURGE_ON_START
environment variable.
optionalstorageClient
Defines storage client to be used.
optionalstorageClientOptions
Could be used to adjust the storage client behavior e.g. MemoryStorageOptions could be used to adjust the MemoryStorage behavior.
optionalsystemInfoIntervalMillis
Defines the interval of emitting the systemInfo
event.
optionalxvfb
Defines whether to run X virtual framebuffer on the web browsers launched by Crawlee.
Alternative to CRAWLEE_XVFB
environment variable.
Sets the ratio, defining the amount of system memory that could be used by the AutoscaledPool. When the memory usage is more than the provided ratio, the memory is considered overloaded.
Alternative to
CRAWLEE_AVAILABLE_MEMORY_RATIO
environment variable.