gradient_boosting_regression — Logtalk 3.100.1

# `gradient_boosting_regression`

Gradient boosting regression supporting continuous and mixed-feature datasets. Builds an additive ensemble of regression trees by repeatedly fitting the current residuals under squared-error loss. Starts from the arithmetic mean of the training targets and then adds a scaled tree prediction at each boosting stage.

The library implements the `regressor_protocol` defined in the `regression_protocols` library and learns an additive ensemble of regression trees by repeatedly fitting the current residuals under squared-error loss.

## API documentation

Open the ../../apis/library_index.html#gradient_boosting_regression link in a web browser.

## Loading

To load this library, load the `loader.lgt` file:

    | ?- logtalk_load(gradient_boosting_regression(loader)).

## Testing

To test this library predicates, load the `tester.lgt` file:

    | ?- logtalk_load(gradient_boosting_regression(tester)).

To run the performance benchmark suite, load the `tester_performance.lgt` file:

    | ?- logtalk_load(gradient_boosting_regression(tester_performance)).

## Features

- Residual Fitting: Fits each regression tree to the residuals of the current additive model under squared-error loss.

- Shrinkage: Supports a fixed learning rate to scale stage contributions and reduce overfitting.

- Continuous and Mixed Features: Supports continuous attributes and categorical attributes encoded by the underlying regression-tree learner.

- Tree Configuration: Exposes the underlying regression-tree depth, minimum-leaf, variance-reduction, and scaling options.

- Diagnostics Metadata: Learned regressors record model name, target, training example count, initial prediction, fitted stage count, and effective options, accessible using the shared regression diagnostics predicates.

- Model Export: Learned regressors can be exported as predicate clauses or written to a file.

- Reference Benchmarks: Includes a dedicated performance suite reporting training time, RMSE, and MAE for representative regression datasets.

## Regressor representation

The learned regressor is represented by default as:

- `gradient_boosting_regressor(InitialPrediction, WeightedTrees, Diagnostics)`

The exported predicate clauses therefore use the shape:

- `Functor(InitialPrediction, WeightedTrees, Diagnostics)`

In this representation, `WeightedTrees` contains `weighted_tree(LearningRate, Tree)` terms and `Diagnostics` stores training metadata including the effective options.

## Diagnostics syntax

The `diagnostics/2` predicate returns a list of metadata terms with the form:

    [
        model(gradient_boosting_regression),
        target(Target),
        training_example_count(TrainingExampleCount),
        options(Options),
        initial_prediction(InitialPrediction),
        stage_count(StageCount)
    ]

Where:

- `model(gradient_boosting_regression)` identifies the learning algorithm that produced the regressor.

- `target(Target)` stores the target attribute name declared by the training dataset.

- `training_example_count(TrainingExampleCount)` stores the number of examples used during training.

- `options(Options)` stores the effective learning options after merging the user options with the library defaults.

- `initial_prediction(InitialPrediction)` stores the constant prediction used to initialize the additive model before fitting any trees.

- `stage_count(StageCount)` stores the number of boosting stages that were actually fitted.

Use the `regression_protocols` `diagnostic/2` and `regressor_options/2` helper predicates when you only need a single metadata term or the effective options.

## Options

The `learn/3` predicate accepts the following options:

- `number_of_estimators/1`: Maximum number of boosting stages to fit. Each stage adds one regression tree to the ensemble. The default is `50`. Training can stop before reaching this limit when the residual sum of squares becomes negligible.

- `learning_rate/1`: Shrinkage factor applied to each stage prediction before it is added to the current model. Smaller values usually require more stages but can improve generalization. The default is `0.1`.

- `maximum_depth/1`: Maximum depth allowed for each regression tree used as a base learner. Lower values produce weaker, simpler trees; higher values allow each stage to model more complex residual structure. The default is `3`.

- `minimum_samples_leaf/1`: Minimum number of training examples required in each leaf of a base learner tree. Increasing this value makes the fitted trees more conservative and can reduce overfitting. The default is `1`.

- `minimum_variance_reduction/1`: Minimum reduction in target variance required to accept a split when fitting a base learner tree. Larger values make tree growth stricter by rejecting weak splits. The default is `0.0`.

- `feature_scaling/1`: Controls continuous-feature scaling in the underlying regression-tree learner. The accepted values are `true` and `false`. The default is `false`.