Queue
BaseModel
¶
Bases: Protocol
Protocol for pydantic BaseModel to ensure compatibility with context
Source code in python/scouter/queue/_queue.pyi
CustomMetricServerRecord
¶
Source code in python/scouter/queue/_queue.pyi
created_at
property
¶
Return the created at timestamp.
metric
property
¶
Return the metric name.
name
property
¶
Return the name.
space
property
¶
Return the space.
value
property
¶
Return the metric value.
version
property
¶
Return the version.
__init__(space, name, version, metric, value)
¶
Initialize spc drift server record
Parameters:
Name | Type | Description | Default |
---|---|---|---|
space
|
str
|
Model space |
required |
name
|
str
|
Model name |
required |
version
|
str
|
Model version |
required |
metric
|
str
|
Metric name |
required |
value
|
float
|
Metric value |
required |
Source code in python/scouter/queue/_queue.pyi
__str__()
¶
model_dump_json()
¶
Feature
¶
Source code in python/scouter/queue/_queue.pyi
__init__(name, value)
¶
Initialize feature. Will attempt to convert the value to it's corresponding feature type. Current support types are int, float, string.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name
|
str
|
Name of the feature |
required |
value
|
Any
|
Value of the feature. Can be an int, float, or string. |
required |
Example
Source code in python/scouter/queue/_queue.pyi
categorical(name, value)
staticmethod
¶
Create a categorical feature
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name
|
str
|
Name of the feature |
required |
value
|
str
|
Value of the feature |
required |
float(name, value)
staticmethod
¶
Create a float feature
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name
|
str
|
Name of the feature |
required |
value
|
float
|
Value of the feature |
required |
int(name, value)
staticmethod
¶
Create an integer feature
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name
|
str
|
Name of the feature |
required |
value
|
int
|
Value of the feature |
required |
string(name, value)
staticmethod
¶
Create a string feature
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name
|
str
|
Name of the feature |
required |
value
|
str
|
Value of the feature |
required |
Features
¶
Source code in python/scouter/queue/_queue.pyi
entity_type
property
¶
Return the entity type
features
property
¶
Return the list of features
__init__(features)
¶
Initialize a features class
Parameters:
Name | Type | Description | Default |
---|---|---|---|
features
|
List[Feature] | Dict[str, Union[int, float, str]]
|
List of features or a dictionary of key-value pairs. If a list, each item must be an instance of Feature. If a dictionary, each key is the feature name and each value is the feature value. Supported types for values are int, float, and string. |
required |
Example
# Passing a list of features
features = Features(
features=[
Feature.int("feature_1", 1),
Feature.float("feature_2", 2.0),
Feature.string("feature_3", "value"),
]
)
# Passing a dictionary (pydantic model) of features
class MyFeatures(BaseModel):
feature1: int
feature2: float
feature3: str
my_features = MyFeatures(
feature1=1,
feature2=2.0,
feature3="value",
)
features = Features(my_features.model_dump())
Source code in python/scouter/queue/_queue.pyi
KafkaConfig
¶
Source code in python/scouter/queue/_queue.pyi
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 |
|
__init__(username=None, password=None, brokers=None, topic=None, compression_type=None, message_timeout_ms=600000, message_max_bytes=2097164, log_level=LogLevel.Info, config={}, max_retries=3)
¶
Kafka configuration for connecting to and publishing messages to Kafka brokers.
This configuration supports both authenticated (SASL) and unauthenticated connections. When credentials are provided, SASL authentication is automatically enabled with secure defaults.
Authentication Priority (first match wins): 1. Direct parameters (username/password) 2. Environment variables (KAFKA_USERNAME/KAFKA_PASSWORD) 3. Configuration dictionary (sasl.username/sasl.password)
SASL Security Defaults
- security.protocol: "SASL_SSL" (override via KAFKA_SECURITY_PROTOCOL env var)
- sasl.mechanism: "PLAIN" (override via KAFKA_SASL_MECHANISM env var)
Parameters:
Name | Type | Description | Default |
---|---|---|---|
username
|
Optional[str]
|
SASL username for authentication. Fallback: KAFKA_USERNAME environment variable. |
None
|
password
|
Optional[str]
|
SASL password for authentication. Fallback: KAFKA_PASSWORD environment variable. |
None
|
brokers
|
Optional[str]
|
Comma-separated list of Kafka broker addresses (host:port). Fallback: KAFKA_BROKERS environment variable. Default: "localhost:9092" |
None
|
topic
|
Optional[str]
|
Target Kafka topic for message publishing. Fallback: KAFKA_TOPIC environment variable. Default: "scouter_monitoring" |
None
|
compression_type
|
Optional[str]
|
Message compression algorithm. Options: "none", "gzip", "snappy", "lz4", "zstd" Default: "gzip" |
None
|
message_timeout_ms
|
int
|
Maximum time to wait for message delivery (milliseconds). Default: 600000 (10 minutes) |
600000
|
message_max_bytes
|
int
|
Maximum message size in bytes. Default: 2097164 (~2MB) |
2097164
|
log_level
|
LogLevel
|
Logging verbosity for the Kafka producer. Default: LogLevel.Info |
Info
|
config
|
Dict[str, str]
|
Additional Kafka producer configuration parameters. See: https://kafka.apache.org/documentation/#producerconfigs Note: Direct parameters take precedence over config dictionary values. |
{}
|
max_retries
|
int
|
Maximum number of retry attempts for failed message deliveries. Default: 3 |
3
|
Examples:
Basic usage (unauthenticated):
SASL authentication:
config = KafkaConfig(
username="my_user",
password="my_password",
brokers="secure-kafka:9093",
topic="secure_topic"
)
Advanced configuration:
config = KafkaConfig(
brokers="kafka:9092",
compression_type="lz4",
config={
"acks": "all",
"batch.size": "32768",
"linger.ms": "10"
}
)
Source code in python/scouter/queue/_queue.pyi
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 |
|
LLMRecord
¶
LLM record containing context tied to a Large Language Model interaction that is used to evaluate drift in LLM responses.
Examples:
>>> record = LLMRecord(
... context={
... "input": "What is the capital of France?",
... "response": "Paris is the capital of France."
... },
... )
>>> print(record.context["input"])
"What is the capital of France?"
Source code in python/scouter/queue/_queue.pyi
context
property
¶
Get the contextual information.
Returns:
Type | Description |
---|---|
Dict[str, Any]
|
The context data as a Python object (deserialized from JSON). |
Raises:
Type | Description |
---|---|
TypeError
|
If the stored JSON cannot be converted to a Python object. |
entity_type
instance-attribute
¶
Type of entity, always EntityType.LLM for LLMRecord instances.
prompt
instance-attribute
¶
Optional prompt configuration associated with this record.
__init__(context, prompt=None)
¶
Creates a new LLM record to associate with an LLMDriftProfile
.
The record is sent to the Scouter
server via the ScouterQueue
and is
then used to inject context into the evaluation prompts.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
context
|
Context
|
Additional context information as a dictionary or a pydantic BaseModel. During evaluation,
this will be merged with the input and response data and passed to the assigned
evaluation prompts. So if you're evaluation prompts expect additional context via
bound variables (e.g., |
required |
prompt
|
Optional[Prompt | SerializedType]
|
Optional prompt configuration associated with this record. Can be a Potatohead Prompt or a JSON-serializable type. |
None
|
Raises:
Type | Description |
---|---|
TypeError
|
If context is not a dict or a pydantic BaseModel. |
Source code in python/scouter/queue/_queue.pyi
Metric
¶
Source code in python/scouter/queue/_queue.pyi
entity_type
property
¶
Return the entity type
metrics
property
¶
Return the list of metrics
__init__(name, value)
¶
Initialize metric
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name
|
str
|
Name of the metric |
required |
value
|
float | int
|
Value to assign to the metric. Can be an int or float but will be converted to float. |
required |
Metrics
¶
Source code in python/scouter/queue/_queue.pyi
__init__(metrics)
¶
Initialize metrics
Parameters:
Name | Type | Description | Default |
---|---|---|---|
metrics
|
List[Metric] | Dict[str, Union[int, float]]
|
List of metrics or a dictionary of key-value pairs. If a list, each item must be an instance of Metric. If a dictionary, each key is the metric name and each value is the metric value. |
required |
Example
```python
Passing a list of metrics¶
metrics = Metrics( metrics=[ Metric("metric_1", 1.0), Metric("metric_2", 2.5), Metric("metric_3", 3), ] )
Passing a dictionary (pydantic model) of metrics¶
class MyMetrics(BaseModel): metric1: float metric2: int
my_metrics = MyMetrics( metric1=1.0, metric2=2, )
metrics = Metrics(my_metrics.model_dump())
Source code in python/scouter/queue/_queue.pyi
PsiServerRecord
¶
Source code in python/scouter/queue/_queue.pyi
bin_count
property
¶
Return the sample value.
bin_id
property
¶
Return the bin id.
created_at
property
¶
Return the created at timestamp.
feature
property
¶
Return the feature.
name
property
¶
Return the name.
space
property
¶
Return the space.
version
property
¶
Return the version.
__init__(space, name, version, feature, bin_id, bin_count)
¶
Initialize spc drift server record
Parameters:
Name | Type | Description | Default |
---|---|---|---|
space
|
str
|
Model space |
required |
name
|
str
|
Model name |
required |
version
|
str
|
Model version |
required |
feature
|
str
|
Feature name |
required |
bin_id
|
int
|
Bundle ID |
required |
bin_count
|
int
|
Bundle ID |
required |
Source code in python/scouter/queue/_queue.pyi
__str__()
¶
model_dump_json()
¶
Queue
¶
Individual queue associated with a drift profile
Source code in python/scouter/queue/_queue.pyi
insert(entity)
¶
Insert a record into the queue
Parameters:
Name | Type | Description | Default |
---|---|---|---|
entity
|
Union[Features, Metrics, LLMRecord]
|
Entity to insert into the queue. Can be an instance for Features, Metrics, or LLMRecord. |
required |
Example
Source code in python/scouter/queue/_queue.pyi
RabbitMQConfig
¶
Source code in python/scouter/queue/_queue.pyi
__init__(host=None, port=None, username=None, password=None, queue=None, max_retries=3)
¶
RabbitMQ configuration to use with the RabbitMQProducer.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
host
|
Optional[str]
|
RabbitMQ host. If not provided, the value of the RABBITMQ_HOST environment variable is used. |
None
|
port
|
Optional[int]
|
RabbitMQ port. If not provided, the value of the RABBITMQ_PORT environment variable is used. |
None
|
username
|
Optional[str]
|
RabbitMQ username. If not provided, the value of the RABBITMQ_USERNAME environment variable is used. |
None
|
password
|
Optional[str]
|
RabbitMQ password. If not provided, the value of the RABBITMQ_PASSWORD environment variable is used. |
None
|
queue
|
Optional[str]
|
RabbitMQ queue to publish messages to. If not provided, the value of the RABBITMQ_QUEUE environment variable is used. |
None
|
max_retries
|
int
|
Maximum number of retries to attempt when publishing messages. Default is 3. |
3
|
Source code in python/scouter/queue/_queue.pyi
RedisConfig
¶
Source code in python/scouter/queue/_queue.pyi
__init__(address=None, chanel=None)
¶
Redis configuration to use with a Redis producer
Parameters:
Name | Type | Description | Default |
---|---|---|---|
address
|
str
|
Redis address. If not provided, the value of the REDIS_ADDR environment variable is used and defaults to "redis://localhost:6379". |
None
|
channel
|
str
|
Redis channel to publish messages to. If not provided, the value of the REDIS_CHANNEL environment variable is used and defaults to "scouter_monitoring". |
required |
Source code in python/scouter/queue/_queue.pyi
ScouterQueue
¶
Main queue class for Scouter. Publishes drift records to the configured transport
Source code in python/scouter/queue/_queue.pyi
transport_config
property
¶
Return the transport configuration used by the queue
__getitem__(key)
¶
Get the queue for the specified key
Parameters:
Name | Type | Description | Default |
---|---|---|---|
key
|
str
|
Key to get the queue for |
required |
from_path(path, transport_config)
staticmethod
¶
Initializes Scouter queue from one or more drift profile paths
Parameters:
Name | Type | Description | Default |
---|---|---|---|
path
|
Dict[str, Path]
|
Dictionary of drift profile paths. Each key is a user-defined alias for accessing a queue |
required |
transport_config
|
Union[KafkaConfig, RabbitMQConfig, RedisConfig, HTTPConfig]
|
Transport configuration for the queue publisher Can be KafkaConfig, RabbitMQConfig RedisConfig, or HTTPConfig |
required |
Example
queue = ScouterQueue(
path={
"spc": Path("spc_profile.json"),
"psi": Path("psi_profile.json"),
},
transport_config=KafkaConfig(
brokers="localhost:9092",
topic="scouter_topic",
),
)
queue["psi"].insert(
Features(
features=[
Feature("feature_1", 1),
Feature("feature_2", 2.0),
Feature("feature_3", "value"),
]
)
)
Source code in python/scouter/queue/_queue.pyi
ServerRecord
¶
Source code in python/scouter/queue/_queue.pyi
ServerRecords
¶
Source code in python/scouter/queue/_queue.pyi
records
property
¶
Return the drift server records.
__init__(records)
¶
Initialize server records
Parameters:
Name | Type | Description | Default |
---|---|---|---|
records
|
List[ServerRecord]
|
List of server records |
required |
__str__()
¶
SpcServerRecord
¶
Source code in python/scouter/queue/_queue.pyi
created_at
property
¶
Return the created at timestamp.
feature
property
¶
Return the feature.
name
property
¶
Return the name.
space
property
¶
Return the space.
value
property
¶
Return the sample value.
version
property
¶
Return the version.
__init__(space, name, version, feature, value)
¶
Initialize spc drift server record
Parameters:
Name | Type | Description | Default |
---|---|---|---|
space
|
str
|
Model space |
required |
name
|
str
|
Model name |
required |
version
|
str
|
Model version |
required |
feature
|
str
|
Feature name |
required |
value
|
float
|
Feature value |
required |