Polars¶

Bases: SupervisedLearner, SupervisedIncrementalLearner

Dummy learner that learns nothing.

This learner is useful for testing purposes.

learn(inputs: LazyFrame, outputs: LazyFrame) -> DummyModel

learn_incremental(inputs: LazyFrame, outputs: LazyFrame) -> DummyModel

Bases: Model

Dummy model that predicts zeros.

This model is useful for testing purposes.

Initialize the model.

Parameters:

output_names = output_names

Bases: OfflineEnvironment

A dataset environment.

This environment represents static tabular datasets.

Attributes:

Initialize the dataset environment.

Parameters:

data: LazyFrame

to_incremental(batch_size: int = 1) -> StreamingOfflineEnvironment

from_csv(path: str | PathLike[str], separator: str = ',') -> Self

from_json(path: str | PathLike[str]) -> Self

from_parquet(path: str | PathLike[str]) -> Self

from_yaml(path: str | Path) -> Self

from_uri(uri: str) -> Self

from_rosbag(path: str | PathLike[str], topics: dict[str, list[str]], *, message_paths: Iterable[str | PathLike[str]] | None = None, cache: bool = True, cache_path: str | PathLike[str] | None = None) -> Self

Bases: Exception

Exception raised when an URI scheme is invalid.

Initialize the InvalidUriSchemeError.

Parameters:

Bases: Exception

Exception raised when a file type is not supported.

Initialize the UnsupportedFileTypeError.

Parameters:

Bases: ActiveEnvironment

Dataset prediction environment.

Initialize the dataset prediction environment.

Parameters:

environment: DataFrame = environment

batch_size: int = batch_size

data: LazyFrame | None = None

slice: LazyFrame | None = None

i: int = 0

step() -> None

act(action: DataFrame) -> None

Bases: OfflineEnvironment

Environment that joins multiple offline environments.

Attributes:

Initialize the joined offline environment.

Parameters:

environments: Iterable[OfflineEnvironment] = environments

Bases: IncrementalEnvironment

Streaming offline environment.

This environment streams data from an offline environment in batches.

Initialize the streaming offline environment.

Parameters:

environment: OfflineEnvironment = environment

batch_size: int = batch_size

data: LazyFrame | None = None

slice: LazyFrame | None = None

i: int = 0

sample_count = size

step() -> None

num_steps() -> int | None

Split data into train and test sets.

Initialize the train-test splitter.

Parameters:

ratio = ratio

shuffle = shuffle

split(environment: OfflineEnvironment) -> tuple[DataFrame, DataFrame]

If input is a polars.LazyFrame, collect() it before passing on.

prepare(data: Data) -> Data

Select only specified columns from DataFrame-like objects.

features: Sequence[str] | None = features

prepare(data: Data) -> Data

Bases: Transform

Cast features to a different datatype.

This transform allows to change the datatype of features in a DataFrame. To cast all features to the same datatype, provide a single type as the target_type argument e.g.

transform = Cast(pl.Float64)

By specifying the features keyword argument, only the selected features will be cast e.g.

transform = Cast(pl.Float64, features=["feature_a"])

Lastly, to cast features to different datatypes, provide a dictionary with feature names as keys and target types as values e.g.

transform = Cast(
    {
        "feature_a": pl.Boolean,
        "feature_b": pl.Float64,
    },
)

Initializes the Cast transform.

Parameters:

target_type = target_type

features = features

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Cluster data using a clustering algorithm.

This transform allows to cluster data using a specified clustering algorithm. The resulting cluster label is added as a new feature to the DataFrame.

Initializes the Cluster transform.

Parameters:

clusterer = clusterer

cluster_feature_name = cluster_feature_name

features = features

apply(data: LazyFrame | DataFrame) -> LazyFrame

fit(data: LazyFrame | DataFrame) -> Cluster

Bases: Transform

Calculates the discrete derivative of time series features.

Calculates the discrete derivative of time series features using either forward, backward, or central differences.

Initializes the DiscreteDerivative transform.

Parameters:

features = [features] if isinstance(features, str) else features

method = method

derivative_suffix = derivative_suffix

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Drop features from the data.

Initializes the Drop transform.

features = features

more_features = more_features

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

This wraps the explode method of Polars.

Reference: https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.explode.html#polars.DataFrame.explode If features is None, all columns will be exploded.

Parameters:

features = features

more_features = more_features

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Transform that explodes nested time series data into individual rows.

Each time series is represented as a list of structs, where each struct contains a timestamp and a value. The value is itself a struct holding multiple feature values at that timestamp. This transform expands the list into separate rows, then unnests the nested structs into columns.

features: ColumnNameOrSelector = features

apply(data: LazyFrame) -> LazyFrame

Bases: CollectionExpr

get() -> Expr

Bases: FilterExpr

expr_collection: Iterable[Expr] = (expression() if isinstance(expression, FilterExpr) else _str_to_pl(expression) for expression in expressions)

Bases: Transform

Filter an environment based on one or multiple expressions.

This transform allows to filter an environment based on or multiple boolean expressions. Assuming the input environment is given by

The following transformation can be used to filter the environment so that the result contains only records where x=1:

    Filter("x == 1")

The result dataset after applying the transform will be

To only get records where x=1 and t > 3 the filter expression

Filter(And(["x == 1", "t > 3"]))

can be used.

To filter all records where x=1 and t > 3 or N < 15 use

Filter(And(["x == 1", Or(["t > 3", "N < 15"])]))

Initializes the Filter transform.

Parameters:

predicate: Expr

apply(data: LazyFrame) -> LazyFrame

Bases: Protocol

Expression to be used in a Filter transform.

get() -> Expr

Bases: FilterExpr

expression: Expr = expression() if isinstance(expression, FilterExpr) else _str_to_pl(expression)

get() -> Expr

Bases: CollectionExpr

get() -> Expr

Bases: Transform

Selects the first time value of a time-series feature.

Initializes the First transform.

Parameters:

features = [features] if isinstance(features, str) else features

replace = replace

apply(data: LazyFrame) -> LazyFrame

Bases: Exception

Length of a feature varies over different rows.

Bases: Transform

Flatten all time series in a DataFrame to individual features.

The given DataFrame's time series are converted into individual features, with each time step creating a new feature. This transform will change the order of the columns in the resulting dataset.

For example the dataset

gets flattened into the dataset

Initialize the flatten transform.

Parameters:

features = features

apply(data: LazyFrame) -> LazyFrame

Bases: Exception

Feature is no time series.

Bases: Transform

Apply a custom function to the data of an environment.

Initializes the Lambda transform.

Parameters:

fn = fn

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Selects the last time value of a time-series feature.

Initializes the Last transform.

Parameters:

features = [features] if isinstance(features, str) else features

replace = replace

apply(data: LazyFrame) -> LazyFrame

Bases: Exception

Feature not found in the DataFrame.

This exception is raised when a feature is not found in the DataFrame.

Bases: Transform

Matches the sampling rate of all time series in the DataFrame.

Interpolates the time series to match the sampling rate of the reference time series. The feature_interpolation_map parameter is a dictionary that specifies the interpolation method for each feature. The keys are the feature names, and the values are the interpolation methods. The interpolation method can be 'linear' or 'nearest'. If the feature_interpolation_map parameter is not provided, all features except the reference feature will be interpolated using the 'nearest' method. The fill_strategy parameter specifies the strategy to fill missing values after interpolation. The default value is 'both_ways', which means that missing values will be filled using both forward and backward filling. Other options include 'forward', 'backward', 'min', 'max', 'mean', 'zero', and 'one'.The below example shows the usage of a MatchSamplingRate transform in a run.py file. Assuming the loaded data is represented by the table:

| feature_a                   | feature_b                   | const |
| ---                         | ---                         | ---   |
| list[struct[time,struct[]]] | list[struct[time,struct[]]] | int   |
| --------------------------- | --------------------------- | ----- |
| [{12:26:01.0, {1.2}},       | [{12:26:00.0, {1.0}},       | 1     |
|  {12:26:02.0, {2.4}},       |  {12:26:05.0, {2.0}}]       |       |
|  {12:26:03.0, {3.6}},       |                             |       |
|  {12:26:04.0, {4.8}}]       |                             |       |

The following transform can be used to match the sampling rate of the time series feature_b to the sampling rate of the time series feature_a.

    ...
    environment.load()
    data = environment.get_data()
    transform = MatchSamplingRate(
        reference_feature_name="feature_a",
        feature_interpolation_map={
            "feature_b": "linear",
        },
    )
    transformed_data = transform.transform(data)
    ...

The resulting Dataframe after the transform is:

| feature_a                   | feature_b                   | const |
| ---                         | ---                         | ---   |
| list[struct[time,struct[]]] | list[struct[time,struct[]]] | int   |
| --------------------------- | --------------------------- | ----- |
| [{12:26:00.0, {1.2}},       | [{12:26:00.0, {1.2}},       | 1     |
|  {12:26:01.0, {2.4}},       |  {12:26:01.0, {1.4}},       |       |
|  {12:26:02.0, {3.6}},       |  {12:26:02.0, {1.6}},       |       |
|  {12:26:03.0, {4.8}}]       |  {12:26:03.0, {1.8}}]       |       |

Initialize the transform.

Parameters:

reference_feature_name = reference_feature_name

feature_interpolation_map = feature_interpolation_map

fill_strategy = fill_strategy

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Replaces time-series features with their mean value.

Initializes the Mean transform.

Parameters:

features = [features] if isinstance(features, str) else features

replace = replace

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Replaces time-series features with their median value.

Initializes the Median transform.

Parameters:

features = [features] if isinstance(features, str) else features

replace = replace

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Mode finds the value that appears most often in time-series features.

Initializes the Mode transform.

Parameters:

features = [features] if isinstance(features, str) else features

replace = replace

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Transforms integer features into a set of binary one-cold features.

Transforms integer features into a set of binary one-cold features. The original integer features are dropped and are not part of the resulting data frame.

As an example consider the following data

When the one-cold transformation is applied, the result is as follows

In the default configuration missing categories are ignored. Their respective entries will all be one. If you however want to enforce that each data entry belongs to a certain category, you can set the check_for_missing_categories flag to true when constructing a One-Cold transform. In that case if an unknown value is found which does not belong to any category, a NoMatchingCategoryError is thrown. This however has an impact on the performance and will slow down the transform.

If you want to enable this check, create the transform as follows: python transform = OneCold( feature_categories={ "feature": [0, 1, 2, 5] }, check_for_missing_categories=True )

Initializes the One-Hot transform.

Parameters:

feature_category_mapping: dict[str, dict[str, Any]] = {feature: {f'{feature}_{value}': value for value in values} for feature, values in feature_categories.items()}

check_for_missing_categories: bool = check_for_missing_categories

apply(data: LazyFrame) -> LazyFrame

from_dataframe(data: DataFrame, features: Iterable[str], *, check_for_missing_categories: bool = False) -> Self

Bases: Exception

Bases: Exception

Bases: Transform

Transforms integer features into a set of binary one-hot features.

Transforms integer features into a set of binary one-hot features. The original integer features are dropped and are not part of the resulting data frame.

As an example consider the following data

When the one-hot transformation is applied, the result is as follows

Initializes the One-Hot transform.

Parameters:

feature_category_mapping: dict[str, dict[str, Any]] = {feature: {f'{feature}_{value}': value for value in values} for feature, values in feature_categories.items()}

check_for_missing_categories: bool = check_for_missing_categories

apply(data: LazyFrame) -> LazyFrame

from_dataframe(data: LazyFrame, features: Iterable[str], *, check_for_missing_categories: bool = False) -> Self

Bases: Transform

Pad time-series features to the specified length.

Pad time-series features to the specified end-time by holding their last value for one more sample. This is useful for ensuring that all time-series features cover at least a time interval of the specified length. Time-series that are already longer than the specified will not be modified. The resulting features will not be equidistant in time. To achieve equidistant time-series, consider using the Resample transform after padding.

Initializes the Pad transform.

Parameters:

length = length

features = cast('list[str]', [features] if isinstance(features, str) else list(features)) if features is not None else None

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Rename features in an environment.

Initializes the Rename transform.

Parameters:

mapping = mapping

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Resample time series features to a given sampling rate.

Initializes the Resample transform.

Parameters:

sampling_rate = sampling_rate

interpolation_method = interpolation_method

apply(data: LazyFrame) -> LazyFrame

resample_data(data: dict[str, list[float]], dt: float) -> Series

Bases: Invertible, Transform

Scale features to a fixed range using a linear mapping.

A sample \(x\) is scaled as:

where

• \(m\) is the scaling factor
• \(b\) is the offset.

When instantiating this transform directly, the scaling factor \(m\) and offset \(b\) for each feature are calculated during training from the data. To specify the scaling factor \(m\) and offset \(b\) directly, use the from_limits method.

Attributes:

features: list[str] | None = features

m: dict[str, float] | None = None

b: dict[str, float] | None = None

lower_range: float = lower_range

upper_range: float = upper_range

fit(data: LazyFrame) -> Self

fit_incremental(data: Data) -> Self

apply(data: LazyFrame) -> LazyFrame

inverse() -> Transform

from_limits(feature_limits: dict[str, tuple[float, float]], *, lower_range: float = -1.0, upper_range: float = 1.0) -> Self

Bases: Transform

Selects a subset of features from the data.

Initializes the Select transform.

Parameters:

features = features

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Applies a Butterworth filter to time series features.

Applies a Butterworth lowpass or highpass filter to time series features. For this transform to work, the time series must already have a uniform sampling rate. Use a `Resample' transform to uniformly sample the points of a time series.

Initializes the Filter transform.

Parameters:

features = features

filter_type = filter_type

frequency = filter_frequency

order = order

apply(data: LazyFrame) -> LazyFrame

filter_data(data: dict[str, list[float]]) -> Series

Bases: Transform

Slices a time series at given slice points.

The transform takes two columns as input: 'time_series' and 'slice_points.' The 'time_series' column is sliced at the points specified by the 'time' entry of the 'slice_points' column. The result is a new time series column where each entry contains only the values from the original time series that fall between the specified slice points.

Suppose you have a dataframe with a single line entry as follows:

Applying the slice time series transform results in a multi-line dataframe, where each line corresponds to a slice point:

The transform operates line-wise, meaning that each line in the input dataframe is processed independently. The resulting dataframe will have multiple lines depending on the number of slice points specified in each line.

Initialize the SliceTimeSeries transform.

Parameters:

time_series = time_series

slice_points = slice_points

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Transforms the data with a sliding window.

The sliding window transform transforms the data by creating a sliding window over the row dimension. The data is then transformed by creating a new column for each column in the original data. The new columns are named by appending the index of the row in the sliding window to the original column name. As an example, consider the following data:

If we apply a sliding window with a window size of 3, we get the following

Parameters:

window_size = window_size

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Convert single large time series into a set of smaller sub-series.

Applies a sliding window to each individual time series sample of all or selected time series features while leaving other features unchanged. As a result, the resulting data frame will contain multiple samples for each original sample, where each sample is a sub-series of the original time series. The number of features (columns) will remain the same. For this transform to work, all selected time series features of a sample must have the same time vector. Use a MatchSamplingRate or Resample transform to ensure this is the case.

Initializes the TimeSeriesSlidingWindow transform.

Parameters:

window_size = window_size

features = features

stride = stride

rechunk = rechunk

apply(data: LazyFrame) -> LazyFrame

Bases: Invertible, Transform

Standardize features by removing the mean and scaling to unit variance.

A sample \(x\) is standardized as:

where

• \(\mu\) is the mean of the samples
• \(\sigma\) is the standard deviation of the samples.

Attributes:

mean: dict[str, float] | None = None

std: dict[str, float] | None = None

fit(data: LazyFrame) -> Self

fit_incremental(data: Data) -> Self

apply(data: LazyFrame) -> LazyFrame

inverse() -> Transform

Bases: Transform

Limit time series to a certain time window.

Initializes the TimeWindow transform.

Parameters:

features = features

t_start = time_start

t_end = time_end

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

time_feature: str | dict[str, str] = time_feature

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Decompose struct columns into separate columns for each field.

Example:

data_frame = pl.Series(
    "c",
    [
        {"a": 1, "t": 1},
        {"a": 4, "t": 2},
        {"a": 7, "t": 3},
        {"a": 10, "t": 4},
        {"a": 15, "t": 5},
    ],
).to_frame()

pl.DataFrame(
    {
        "a": [1, 4, 7, 10, 15],
        "t": [1, 2, 3, 4, 5],
    },
)

Initializes the Unnest transform.

Parameters:

features = features

apply(data: LazyFrame) -> LazyFrame

Bases: Transform

Aligns multiple time series features using zero-order-hold.

Initialize the ZeroOrderHoldMatching transform.

Parameters:

features = features

name = name

drop = drop

apply(data: LazyFrame) -> LazyFrame