Policy

Policies enable SWE users to proactively monitor their applications, ML models, and data pipelines and automate the process of detecting issues to scale data and AI operations efficiently and safely. Policies provide users with a flexible way to define types of things they consider as 'issues,' so the platform can automatically start to scan for such issues in the defined scope continuously.

Policies have several main parts that define them, which the user should configure:

Metric

This involves specifying the exact measurement needed for monitoring. A metric is defined by selecting the source of data (both datasets and events are available for selection, and then defining the specific dataset or event). With a SQL-like query builder, users can define the measures and groups that should be scanned. To help users better understand how the configured metric usually works, a simulation chart will be created below the metric to show its past values and illustrate how the condition will apply to it.

Condition

Conditions allow users to define when this policy should check and enforce its constraints, and what constraint needs to be checked against the configured metric. Currently, there are two types of supported conditions:

Fixed

This allows users to define a fixed threshold (e.g., 'Above 3,' 'Below 100'). It is suitable for cases where the monitored metric has well-defined boundaries, and the definition of what is 'unhealthy' is well-known in the use case.

Dynamic

This allows users to configure a dynamic threshold driven by statistical analysis based on mean and standard deviation. This configuration is best suitable for cases where the boundaries are not clear and can change over time, but we want to ensure they don't deviate from historical levels.

Alerting

Alerting allows users to configure where they want to receive notifications when something interesting happens. These settings also define what should be considered interesting - whether it's only when a policy becomes unhealthy or also when it becomes healthy again. The notification settings will leverage the existing configured destinations and can include multiple destinations at once.