Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/api_doc/imputation/MeanImputer.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
MeanImputer
===========

.. autoclass:: feature_engine.imputation.MeanImputer
:members:

6 changes: 0 additions & 6 deletions docs/api_doc/imputation/MeanMedianImputer.rst

This file was deleted.

4 changes: 2 additions & 2 deletions docs/api_doc/imputation/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The following table summarises each imputer's functionality:
================================== ===================== ======================= ====================================================================================
Transformer Numerical variables Categorical variables Description
================================== ===================== ======================= ====================================================================================
:class:`MeanMedianImputer()` √ × Replaces missing values with the mean or median
:class:`MeanImputer()` √ × Replaces missing values with the mean or median
:class:`ArbitraryNumberImputer()` √ × Replaces missing values with an arbitrary value
:class:`EndTailImputer()` √ × Replaces missing values with a value at the end of the distribution
:class:`CategoricalImputer()` × √ Replaces missing values with the most frequent category or an arbitrary string
Expand All @@ -27,7 +27,7 @@ Imputers
.. toctree::
:maxdepth: 1

MeanMedianImputer
MeanImputer
ArbitraryNumberImputer
EndTailImputer
CategoricalImputer
Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ Missing Data Imputation: Imputers
Missing data imputation consists in replacing missing values in categorical data and numerical variables with estimates
of those nan values or arbitrary data points. Feature-engine supports the following missing data imputation methods:

- :doc:`api_doc/imputation/MeanMedianImputer`: replaces missing data in numerical variables with the mean or median
- :doc:`api_doc/imputation/MeanImputer`: replaces missing data in numerical variables with the mean or median
- :doc:`api_doc/imputation/ArbitraryNumberImputer`: replaces missing data in numerical variables with an arbitrary number
- :doc:`api_doc/imputation/EndTailImputer`: replaces missing data in numerical variables with numbers at the distribution tails
- :doc:`api_doc/imputation/CategoricalImputer`: replaces missing data with an arbitrary string or with the most frequent category
Expand Down
8 changes: 4 additions & 4 deletions docs/quickstart/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ imputation.
from sklearn.datasets import fetch_openml
from sklearn.model_selection import train_test_split

from feature_engine.imputation import MeanMedianImputer
from feature_engine.imputation import MeanImputer

# Load dataset
X, y = fetch_openml(
Expand All @@ -76,7 +76,7 @@ imputation.
)

# set up the imputer
median_imputer = MeanMedianImputer(
median_imputer = MeanImputer(
imputation_method='median',
variables=['LotFrontage', 'MasVnrArea']
)
Expand Down Expand Up @@ -127,7 +127,7 @@ pickle (.pkl). Here is an example of how to do it:
from feature_engine.discretisation import DecisionTreeDiscretiser
from feature_engine.imputation import (
AddMissingIndicator,
MeanMedianImputer,
MeanImputer,
CategoricalImputer,
)

Expand Down Expand Up @@ -172,7 +172,7 @@ pickle (.pkl). Here is an example of how to do it:
('continuous_var_imputer', AddMissingIndicator(variables=['LotFrontage'])),

# Replace NA with the median in 2 variables
('continuous_var_median_imputer', MeanMedianImputer(
('continuous_var_median_imputer', MeanImputer(
imputation_method='median', variables=['LotFrontage', 'MasVnrArea']
)),

Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
.. _mean_median_imputer:
.. _mean_imputer:

.. currentmodule:: feature_engine.imputation

MeanMedianImputer
=================
MeanImputer
===========

Mean imputation and median imputation consist of replacing missing data in numerical
variables with the variable's mean or median. These simple univariate missing data
Expand Down Expand Up @@ -74,18 +74,18 @@ used with decision tree-based algorithms. When the relationship among the variab
crucial, you might want to consider better ways to estimate the missing data, such as
multiple imputation (aka, multivariate imputation).

MeanMedianImputer
-----------------
MeanImputer
-----------

Feature-engine's :class:`MeanMedianImputer()` replaces missing data with the variable's
Feature-engine's :class:`MeanImputer()` replaces missing data with the variable's
mean or median value, determined over the observed values. Hence, it can only impute
numerical variables. You can pass the
list of variables you want to impute, or alternatively, :class:`MeanMedianImputer()`
list of variables you want to impute, or alternatively, :class:`MeanImputer()`
will automatically impute all numerical variables in the training set.

.. attention::

**New in version 2.0:** When `variables` is `None`, :class:`MeanMedianImputer()` used to
**New in version 2.0:** When `variables` is `None`, :class:`MeanImputer()` used to
raise an error if the dataframe contained no numerical variables. You can now
set the new parameter `return_empty` to `True` to make the transformer return an
empty list of variables and skip the imputation instead, leaving the dataframe
Expand All @@ -97,7 +97,7 @@ will automatically impute all numerical variables in the training set.
Python implementation
---------------------

In this section, we will explore :class:`MeanMedianImputer()`'s functionality. Let's start by
In this section, we will explore :class:`MeanImputer()`'s functionality. Let's start by
importing the required libraries:

.. code:: python
Expand All @@ -107,7 +107,7 @@ importing the required libraries:
from sklearn.datasets import fetch_openml
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import train_test_split
from feature_engine.imputation import MeanMedianImputer
from feature_engine.imputation import MeanImputer
from feature_engine.imputation import AddMissingIndicator


Expand Down Expand Up @@ -156,13 +156,13 @@ either **LotFrontage** or **MasVnrArea**:
1018 Gilbert NaN 76.0


Let's now set up and fit :class:`MeanMedianImputer()` with the strategy set to mean,
Let's now set up and fit :class:`MeanImputer()` with the strategy set to mean,
so we can impute the variables `LotFrontage` and `MasVnrArea`:

.. code:: python

# Set up the imputer
mmi = MeanMedianImputer(
mmi = MeanImputer(
imputation_method='mean',
variables=['LotFrontage', 'MasVnrArea']
)
Expand All @@ -173,10 +173,10 @@ so we can impute the variables `LotFrontage` and `MasVnrArea`:
.. note::

It's worth noting that we have the flexibility to omit the `variables` parameter,
in which case, :class:`MeanMedianImputer()` will automatically find and impute all
in which case, :class:`MeanImputer()` will automatically find and impute all
numeric features.

After fitting :class:`MeanMedianImputer()`, we can check out the statistics
After fitting :class:`MeanImputer()`, we can check out the statistics
(either mean or median; mean in this case) for each of the variables to impute:

.. code:: python
Expand Down Expand Up @@ -217,7 +217,7 @@ Imputing missing values alongside missing indicators
Mean or median imputation are commonly done alongside adding missing indicators.
We can add missing indicators with :class:`AddMissingIndicator()` from feature-engine.

We can chain :class:`AddMissingIndicator()` with :class:`MeanMedianImputer()` using a
We can chain :class:`AddMissingIndicator()` with :class:`MeanImputer()` using a
`scikit-learn pipeline <https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.make_pipeline.html>`_.

For example, let's create an imputation pipeline to add missing indicators and then
Expand All @@ -228,7 +228,7 @@ impute the missing values:
# Create imputation pipeline
imputer = make_pipeline(
AddMissingIndicator(),
MeanMedianImputer()
MeanImputer()
)

# Fit the pipeline
Expand Down
6 changes: 3 additions & 3 deletions docs/user_guide/imputation/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ Feature-engine's imputers' main characteristics
================================== ===================== ======================= ====================================================================================
Transformer Numerical variables Categorical variables Description
================================== ===================== ======================= ====================================================================================
:class:`MeanMedianImputer()` √ × Replaces missing values with the mean or median
:class:`MeanImputer()` √ × Replaces missing values with the mean or median
:class:`ArbitraryNumberImputer()` √ x Replaces missing values with an arbitrary value
:class:`EndTailImputer()` √ × Replaces missing values with a value at the end of the distribution
:class:`CategoricalImputer()` × √ Replaces missing values with the most frequent category or an arbitrary string
Expand Down Expand Up @@ -132,7 +132,7 @@ assume that the missing values are close in value to the majority, that is, to t

Data imputed with the mean or median is commonly used to train linear regression and logistic regression models.

The :class:`MeanMedianImputer()` implements mean-median imputation.
The :class:`MeanImputer()` implements mean-median imputation.

Arbitrary Number Imputation
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -340,7 +340,7 @@ Imputers
.. toctree::
:maxdepth: 1

MeanMedianImputer
MeanImputer
ArbitraryNumberImputer
EndTailImputer
CategoricalImputer
Expand Down
6 changes: 3 additions & 3 deletions docs/user_guide/outliers/Winsoriser.rst
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ using 1.5 of the IQR to find those limits (param `fold`).

.. code:: python

from feature_engine.imputation import MeanMedianImputer
from feature_engine.imputation import MeanImputer
from feature_engine.pipeline import Pipeline
from feature_engine.outliers import Winsoriser

Expand All @@ -218,7 +218,7 @@ using 1.5 of the IQR to find those limits (param `fold`).
)

pipe = Pipeline([
("imputer", MeanMedianImputer()),
("imputer", MeanImputer()),
("outlier", w),
])

Expand Down Expand Up @@ -286,7 +286,7 @@ As an alternative, let's cap the variables at their 10th percentile on the right
)

pipe = Pipeline([
("imputer", MeanMedianImputer()),
("imputer", MeanImputer()),
("outlier", w),
])

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -289,20 +289,20 @@ Imputing rows with nan

If instead of removing the row with nan in the expanding window features, we want to impute those
values, we can do so with any of feature-engine's imputers. Here, we will replace nan with
the median value of the resulting window features, using the `MeanMedianImputer` within
the median value of the resulting window features, using the `MeanImputer` within
a pipeline:


.. code:: python

from feature_engine.imputation import MeanMedianImputer
from feature_engine.imputation import MeanImputer
from feature_engine.pipeline import Pipeline

win_f = ExpandingWindowFeatures(functions=["mean", "std"])

pipe = Pipeline([
("windows", win_f),
("imputer", MeanMedianImputer(imputation_method="median"))
("imputer", MeanImputer(imputation_method="median"))
])

X_tr = pipe.fit_transform(X, y)
Expand Down
4 changes: 2 additions & 2 deletions docs/user_guide/wrappers/Wrapper.rst
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ scikit-learn's PolynomialFeatures. We start with the imports:
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import Pipeline
from feature_engine.datasets import load_titanic
from feature_engine.imputation import CategoricalImputer, MeanMedianImputer
from feature_engine.imputation import CategoricalImputer, MeanImputer
from feature_engine.encoding import OrdinalEncoder
from feature_engine.wrappers import SklearnTransformerWrapper

Expand All @@ -292,7 +292,7 @@ Now, we assemble the pipeline. The last step wraps scikit-learn's PolynomialFeat

pipeline = Pipeline(steps = [
('ci', CategoricalImputer(imputation_method='frequent')),
('mmi', MeanMedianImputer(imputation_method='mean')),
('mmi', MeanImputer(imputation_method='mean')),
('od', OrdinalEncoder(encoding_method='arbitrary')),
('pl', SklearnTransformerWrapper(
PolynomialFeatures(interaction_only = True, include_bias=False),
Expand Down
4 changes: 2 additions & 2 deletions feature_engine/datasets/titanic.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import pandas as pd
from sklearn.pipeline import Pipeline

from feature_engine.imputation import CategoricalImputer, MeanMedianImputer
from feature_engine.imputation import CategoricalImputer, MeanImputer


# TODO: loading the dataset from the internet is not the best, we need to store it
Expand Down Expand Up @@ -95,7 +95,7 @@ def load_titanic(
"categorical_imputer",
CategoricalImputer(imputation_method="missing"),
),
("mean_median_imputer", MeanMedianImputer(imputation_method="mean")),
("mean_imputer", MeanImputer(imputation_method="mean")),
]
)

Expand Down
3 changes: 2 additions & 1 deletion feature_engine/imputation/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,12 @@
from .categorical import CategoricalImputer
from .drop_missing_data import DropMissingData
from .end_tail import EndTailImputer
from .mean_median import MeanMedianImputer
from .mean_median import MeanImputer, MeanMedianImputer
from .missing_indicator import AddMissingIndicator
from .random_sample import RandomSampleImputer

__all__ = [
"MeanImputer",
"MeanMedianImputer",
"ArbitraryNumberImputer",
"CategoricalImputer",
Expand Down
35 changes: 29 additions & 6 deletions feature_engine/imputation/mean_median.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Authors: Soledad Galli <solegalli@protonmail.com>
# License: BSD 3 clause

import warnings
from typing import List, Optional, Union

import pandas as pd
Expand Down Expand Up @@ -43,16 +44,16 @@
transform=_transform_imputers_docstring,
fit_transform=_fit_transform_docstring,
)
class MeanMedianImputer(BaseImputer):
class MeanImputer(BaseImputer):
"""
The MeanMedianImputer() replaces missing data by the mean or median value of the
The MeanImputer() replaces missing data by the mean or median value of the
variable. It works only with numerical variables.

You can pass a list of variables to impute. Alternatively, the
MeanMedianImputer() will automatically select all variables of type numeric in the
MeanImputer() will automatically select all variables of type numeric in the
training set.

More details in the :ref:`User Guide <mean_median_imputer>`.
More details in the :ref:`User Guide <mean_imputer>`.

Parameters
----------
Expand Down Expand Up @@ -87,12 +88,12 @@ class MeanMedianImputer(BaseImputer):

>>> import pandas as pd
>>> import numpy as np
>>> from feature_engine.imputation import MeanMedianImputer
>>> from feature_engine.imputation import MeanImputer
>>> X = pd.DataFrame(dict(
>>> x1 = [np.nan,1,1,0,np.nan],
>>> x2 = ["a", np.nan, "b", np.nan, "a"],
>>> ))
>>> mmi = MeanMedianImputer(imputation_method='median')
>>> mmi = MeanImputer(imputation_method='median')
>>> mmi.fit(X)
>>> mmi.transform(X)
x1 x2
Expand Down Expand Up @@ -151,3 +152,25 @@ def fit(self, X: pd.DataFrame, y: Optional[pd.Series] = None):
self._get_feature_names_in(X)

return self


# TODO: remove in version 2.1.0
class MeanMedianImputer(MeanImputer):
def __init__(
self,
imputation_method: str = "median",
variables: Union[None, int, str, List[Union[str, int]]] = None,
return_empty: bool = False,
) -> None:
warnings.warn(
"MeanMedianImputer was deprecated in favour of MeanImputer in version "
"2.0.0 and will be removed in version 2.1.0. To silence this warning, "
"use MeanImputer instead.",
FutureWarning,
stacklevel=2,
)
super().__init__(
imputation_method=imputation_method,
variables=variables,
return_empty=return_empty,
)
4 changes: 2 additions & 2 deletions tests/check_estimators_with_parametrize_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
CategoricalImputer,
DropMissingData,
EndTailImputer,
MeanMedianImputer,
MeanImputer,
RandomSampleImputer,
)
from feature_engine.outliers import ArbitraryOutlierCapper, OutlierTrimmer, Winsoriser
Expand Down Expand Up @@ -87,7 +87,7 @@ def test_sklearn_compatible_creator(estimator, check):
# imputation
@parametrize_with_checks(
[
MeanMedianImputer(),
MeanImputer(),
ArbitraryNumberImputer(),
CategoricalImputer(fill_value=0, ignore_format=True),
EndTailImputer(),
Expand Down
Loading