diff --git a/README.md b/README.md index 76a9056bc..6e80a58fd 100644 --- a/README.md +++ b/README.md @@ -119,7 +119,6 @@ Please share your story by answering 1 quick question ### Variable Transformation methods * LogTransformer -* LogCpTransformer * ReciprocalTransformer * ArcsinTransformer * PowerTransformer diff --git a/docs/api_doc/transformation/LogCpTransformer.rst b/docs/api_doc/transformation/LogCpTransformer.rst index b5161de6a..b5f050ff0 100644 --- a/docs/api_doc/transformation/LogCpTransformer.rst +++ b/docs/api_doc/transformation/LogCpTransformer.rst @@ -1,3 +1,5 @@ +:orphan: + LogCpTransformer ================ diff --git a/docs/api_doc/transformation/index.rst b/docs/api_doc/transformation/index.rst index a6b273551..1850bdcab 100644 --- a/docs/api_doc/transformation/index.rst +++ b/docs/api_doc/transformation/index.rst @@ -10,7 +10,6 @@ mathematical transformations. :maxdepth: 1 LogTransformer - LogCpTransformer ReciprocalTransformer ArcsinTransformer ArcSinhTransformer diff --git a/docs/index.rst b/docs/index.rst index 6a05f7e9e..6ffbe9747 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -237,7 +237,6 @@ We normally use variance stabilising transformations to make the data meet the a like ANOVA, and machine learning models, like linear regression. Feature-engine supports the following transformations: - :doc:`api_doc/transformation/LogTransformer`: performs logarithmic transformation of numerical variables -- :doc:`api_doc/transformation/LogCpTransformer`: performs logarithmic transformation after adding a constant value - :doc:`api_doc/transformation/ReciprocalTransformer`: performs reciprocal transformation of numerical variables - :doc:`api_doc/transformation/PowerTransformer`: performs power transformation of numerical variables - :doc:`api_doc/transformation/BoxCoxTransformer`: performs Box-Cox transformation of numerical variables diff --git a/docs/user_guide/transformation/LogCpTransformer.rst b/docs/user_guide/transformation/LogCpTransformer.rst index 906e50cd9..89982dbcb 100644 --- a/docs/user_guide/transformation/LogCpTransformer.rst +++ b/docs/user_guide/transformation/LogCpTransformer.rst @@ -1,3 +1,5 @@ +:orphan: + .. _log_cp: .. currentmodule:: feature_engine.transformation @@ -13,6 +15,12 @@ positive values. adding a constant to move distributions towards positive values. For more details about the logarithm transformation, check out the :ref:`LogTransformer()'s user Guide `. +.. warning:: + + :class:`LogCpTransformer()` was deprecated in version 2.0.0 and will be removed in + version 2.1.0. Use ``LogTransformer(C="auto")`` instead, which reproduces + :class:`LogCpTransformer()`'s default behavior exactly. + Defining C ---------- diff --git a/docs/user_guide/transformation/LogTransformer.rst b/docs/user_guide/transformation/LogTransformer.rst index 133b7ba8e..b73d317d8 100644 --- a/docs/user_guide/transformation/LogTransformer.rst +++ b/docs/user_guide/transformation/LogTransformer.rst @@ -28,9 +28,7 @@ LogTransformer .. note:: - Note that the logarithm can only be applied to positive values. Thus, if the variable contains 0 or negative values, this transformer will return an error. - -To transform non-positive variables you can add a constant to shift the data points towards positive values. You can do this by using :class:`LogCpTransformer()`. + The logarithm can only be applied to positive values; if the variable contains 0 or negative values, the transformer will raise an error. To avoid this, add a constant to the variables by using the `C` parameter (more details below). .. attention:: @@ -161,6 +159,64 @@ In the following plots we see histograms showing the variables in their original Following the transformations with scatter plots and residual analysis of the regression models helps understand if the transformations are useful in our regression analysis. +Transforming non-positive variables with C +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:class:`LogTransformer()` can also transform variables that contain zero or +negative values, by adding a constant ``C`` before applying the logarithm, +i.e. ``log(x + C)``. + +Let's load the diabetes dataset that comes with scikit-learn, which contains +variables with negative values: + +.. code:: python + + from sklearn.datasets import load_diabetes + from sklearn.model_selection import train_test_split + from feature_engine.transformation import LogTransformer + + # Load dataset + X, y = load_diabetes(return_X_y=True, as_frame=True) + + # Separate into train and test sets + X_train, X_test, y_train, y_test = train_test_split( + X, y, test_size=0.3, random_state=0) + +Let's set up :class:`LogTransformer()` with ``C="auto"``, so it automatically +finds the constant needed to shift each variable's distribution to positive +values before applying the logarithm: + +.. code:: python + + logt = LogTransformer(variables=["bmi", "s3"], C="auto") + logt.fit(X_train) + +We can inspect the constant values that will be added to each variable in the +``C_`` attribute: + +.. code:: python + + logt.C_ + +Since these variables are not strictly positive, the transformer found the +minimum value needed to make their values positive: + +.. code:: python + + {'bmi': 1.0848862355291056, 's3': 1.102307050517416} + +We can now transform the data: + +.. code:: python + + train_t = logt.transform(X_train) + test_t = logt.transform(X_test) + +You can also pass a fixed constant to all variables (``C=5``), or a dictionary +mapping each variable to its own constant (``C={"bmi": 2, "s3": 3}``), the same +way you would with the deprecated :class:`LogCpTransformer()`. + + Additional resources -------------------- diff --git a/docs/user_guide/transformation/PowerTransformer.rst b/docs/user_guide/transformation/PowerTransformer.rst index 13ac20087..7aef1c6d4 100644 --- a/docs/user_guide/transformation/PowerTransformer.rst +++ b/docs/user_guide/transformation/PowerTransformer.rst @@ -74,7 +74,6 @@ Other transformers Feature-engine also provides the following power transformers: - :class:`LogTransformer` -- :class:`LogCpTransformer` - :class:`ReciprocalTransformer` - :class:`ArcsinTransformer` diff --git a/docs/user_guide/transformation/index.rst b/docs/user_guide/transformation/index.rst index 41a3eedb1..57634feac 100644 --- a/docs/user_guide/transformation/index.rst +++ b/docs/user_guide/transformation/index.rst @@ -24,18 +24,17 @@ See the following illustration: Supported transformations ------------------------- -================================ ================================================ ============================================================================================= ==================== -Transformer Description Suitable for Limitations -================================ ================================================ ============================================================================================= ==================== -:class:`LogTransformer()` Applies natural or decimal logarithm. Positive continuous variables with right skew. Not valid for x<=0 -:class:`LogCpTransformer()` Applies logarithm after adding a constant value. Continuous variables with a right skew. None -:class:`ReciprocalTransformer()` Applies the reciprocal transformation: 1/x. Variables representing ratios or proportions, like tons per acre. Not defined for x=0 -:class:`ArcsinTransformer()` Applies the arcsin square root transformation. Probabilities or proportion variables with values between 0 and 1. 0<= x <= 1 -:class:`ArcSinhTransformer()` Applies the inverse hyperbolic sine function. Similar to log but retaining zero values in a variable. None -:class:`PowerTransformer()` Applies any power transformation x = x**n. Square root is suitable for count variables. Other powers vary. None -:class:`BoxCoxTransformer()` Applies the Box-Cox transformation. Positive continuous variables when the optimal transformation is unknown. Not defined for x<=0 -:class:`YeoJohnsonTransformer()` Applies the Yeo-Johnson transformation. Continuous variables with zero or negative values when the optimal transformation is unknown. None -================================ ================================================ ============================================================================================= ==================== +================================ ===================================================================== ============================================================================================= ====================================================== +Transformer Description Suitable for Limitations +================================ ===================================================================== ============================================================================================= ====================================================== +:class:`LogTransformer()` Applies natural or decimal logarithm, optionally adding a constant C. Positive continuous variables with right skew. Not valid for x<=0 unless C is used to shift the data. +:class:`ReciprocalTransformer()` Applies the reciprocal transformation: 1/x. Variables representing ratios or proportions, like tons per acre. Not defined for x=0 +:class:`ArcsinTransformer()` Applies the arcsin square root transformation. Probabilities or proportion variables with values between 0 and 1. 0<= x <= 1 +:class:`ArcSinhTransformer()` Applies the inverse hyperbolic sine function. Similar to log but retaining zero values in a variable. None +:class:`PowerTransformer()` Applies any power transformation x = x**n. Square root is suitable for count variables. Other powers vary. None +:class:`BoxCoxTransformer()` Applies the Box-Cox transformation. Positive continuous variables when the optimal transformation is unknown. Not defined for x<=0 +:class:`YeoJohnsonTransformer()` Applies the Yeo-Johnson transformation. Continuous variables with zero or negative values when the optimal transformation is unknown. None +================================ ===================================================================== ============================================================================================= ====================================================== .. note:: @@ -49,7 +48,6 @@ Transformers :maxdepth: 1 LogTransformer - LogCpTransformer ReciprocalTransformer ArcsinTransformer ArcSinhTransformer diff --git a/feature_engine/transformation/log.py b/feature_engine/transformation/log.py index 1e34f3308..4e4024ac8 100644 --- a/feature_engine/transformation/log.py +++ b/feature_engine/transformation/log.py @@ -1,6 +1,7 @@ # Authors: Soledad Galli # License: BSD 3 clause +import warnings from typing import Dict, List, Optional, Union import numpy as np @@ -42,13 +43,19 @@ fit_transform=_fit_transform_docstring, inverse_transform=_inverse_transform_docstring, ) -class LogTransformer(BaseNumericalTransformer): +class LogTransformer(BaseNumericalTransformer, FitFromDictMixin): """ The LogTransformer() applies the natural logarithm or the base 10 logarithm to - numerical variables. The natural logarithm is the logarithm in base e. + numerical variables, optionally after adding a constant C, i.e., log(x + C). - The LogTransformer() only works with positive values. If the variable - contains a zero or a negative value the transformer will return an error. + By default, C=0, so LogTransformer() only works with positive values. + If a variable contains a zero or a negative value, the transformer raises + an error. Note that the default value of C will change from 0 to "auto" + in version 2.1.0. + + To transform variables that contain zero or negative values, pass a non-zero + C: either an explicit constant, "auto" to let the transformer determine a + shift per variable, or a dictionary mapping each variable to its own constant. A list of variables can be passed as an argument. Alternatively, the transformer will automatically select and transform all variables of type numeric. @@ -65,10 +72,27 @@ class LogTransformer(BaseNumericalTransformer): Indicates if the natural or base 10 logarithm should be applied. Can take values 'e' or '10'. + C: "auto", int, float or dict, default=0 + The constant C to add to the variable before the logarithm, i.e., log(x + C). + + - If 0 (the default), no constant is added and the variable must be + strictly positive. + - If int or float, then log(x + C). + - If "auto", then C = abs(min(x)) + 1. + - If dict, dictionary mapping the constant C to apply to each variable. + + Note, when C is a dictionary, the parameter `variables` is ignored, because + the variables to transform are taken from the dictionary keys. + Attributes ---------- {variables_} + C_: + The constant C added to each variable. Equal to `C`, unless `C = "auto"`, in + which case it is a dictionary with C = abs(min(variable)) + 1. For strictly + positive variables, C = 0. + {feature_names_in_} {n_features_in_} @@ -109,24 +133,32 @@ def __init__( variables: Union[None, int, str, List[Union[str, int]]] = None, return_empty: bool = False, base: str = "e", + C: Union[int, float, str, Dict[Union[str, int], Union[float, int]]] = 0, ) -> None: if base not in ["e", "10"]: - raise ValueError("base can take only '10' or 'e' as values") + raise ValueError( + f"base can take only '10' or 'e' as values. Got {base} instead." + ) + + if not isinstance(C, (int, float, dict)) and C != "auto": + raise ValueError( + "C can take only 'auto', integers, floats or dictionaries. " + f"Got {C} instead." + ) _check_return_empty_is_bool(return_empty) self.variables = _check_variables_input_value(variables) self.return_empty = return_empty self.base = base + self.C = C def fit(self, X: pd.DataFrame, y: Optional[pd.Series] = None): """ - This transformer does not learn parameters. - - Selects the numerical variables and determines whether the logarithm - can be applied on the selected variables, i.e., it checks that the variables - are positive. + Learn the constant C to add to the variable before the logarithm + transformation, if C="auto". Otherwise, this transformer does not learn + parameters. Parameters ---------- @@ -139,10 +171,28 @@ def fit(self, X: pd.DataFrame, y: Optional[pd.Series] = None): """ # check input dataframe - X = super().fit(X) + if isinstance(self.C, dict): + X = super()._fit_from_dict(X, self.C) + else: + X = super().fit(X) + + self.C_ = self.C - # check contains zero or negative values - if (X[self.variables_] <= 0).any().any(): + # calculate C to add to each variable + if self.C == "auto": + # we add 0 to positive variables + c_dict = {var: 0 for var in self.variables_ if X[var].min() > 0} + + # we add the minimum plus 1 to non-positive variables + non_positive_vars = [ + var for var in self.variables_ if var not in c_dict.keys() + ] + c_dict.update(dict(X[non_positive_vars].min(axis=0).abs() + 1)) + self.C_ = c_dict # type:ignore + + # C=0 is the original LogTransformer contract: no constant is added, + # so fail fast at fit time exactly as before this class supported C. + if self.C_ == 0 and (X[self.variables_] <= 0).any().any(): raise ValueError( "Some variables contain zero or negative values, can't apply log" ) @@ -151,7 +201,7 @@ def fit(self, X: pd.DataFrame, y: Optional[pd.Series] = None): def transform(self, X: pd.DataFrame) -> pd.DataFrame: """ - Transform the variables with the logarithm. + Transform the variables with the logarithm of x plus the constant C. Parameters ---------- @@ -167,19 +217,26 @@ def transform(self, X: pd.DataFrame) -> pd.DataFrame: # check input dataframe and if class was fitted X = self._check_transform_input_and_state(X) - # check contains zero or negative values - if (X[self.variables_] <= 0).any().any(): - raise ValueError( + if self.C_ == 0: + error_msg = ( "Some variables contain zero or negative values, can't apply log" ) + else: + error_msg = ( + "Some variables contain zero or negative values after adding" + + " constant C, can't apply log." + ) + + if (X[self.variables_] + self.C_ <= 0).any().any(): + raise ValueError(error_msg) X[self.variables_] = X[self.variables_].astype(float) # transform if self.base == "e": - X.loc[:, self.variables_] = np.log(X.loc[:, self.variables_]) + X.loc[:, self.variables_] = np.log(X.loc[:, self.variables_] + self.C_) elif self.base == "10": - X.loc[:, self.variables_] = np.log10(X.loc[:, self.variables_]) + X.loc[:, self.variables_] = np.log10(X.loc[:, self.variables_] + self.C_) return X @@ -203,17 +260,17 @@ def inverse_transform(self, X: pd.DataFrame) -> pd.DataFrame: # inverse_transform if self.base == "e": - X.loc[:, self.variables_] = np.exp(X.loc[:, self.variables_]) + X.loc[:, self.variables_] = np.exp(X.loc[:, self.variables_]) - self.C_ elif self.base == "10": - X.loc[:, self.variables_] = np.array(10 ** X.loc[:, self.variables_]) + X.loc[:, self.variables_] = 10 ** X.loc[:, self.variables_] - self.C_ return X def _more_tags(self): tags_dict = _return_tags() # ======= this tests fail because the transformers throw an error - # when the values are 0. Nothing to do with the test itself but - # mostly with the data created and used in the test + # when the values are 0 and C=0 (the default). Nothing to do with the + # test itself but mostly with the data created and used in the test msg = ( "transformers raise errors when data contains zeroes, thus this check fails" ) @@ -231,80 +288,19 @@ def __sklearn_tags__(self): return tags -@Substitution( - return_empty=_return_empty_docstring, - variables_=_variables_attribute_docstring, - feature_names_in_=_feature_names_in_docstring, - n_features_in_=_n_features_in_docstring, - fit_transform=_fit_transform_docstring, - inverse_transform=_inverse_transform_docstring, -) -class LogCpTransformer(BaseNumericalTransformer, FitFromDictMixin): +# TODO: remove in version 2.1.0 +class LogCpTransformer(LogTransformer): """ LogCpTransformer() applies the transformation log(x + C), where x is the - variable to transform and C is a positive constant. It can apply the natural - logarithm or the base 10 logarithm, where the natural logarithm is logarithm in - base e. - - As the logarithm can only be applied to numerical non-negative values, - LogCpTransformer() extends the functionality of LogTransformer, by adding a - constant to shift the distribution of the variables towards positive values. - - Note that if the variable contains a zero or a negative value after adding a - constant C, the transformer will return an error. This can occur if the values of - the variables in the test set are smaller than those seen during `fit()`. - - A list of variables can be passed as an argument. Alternatively, the transformer - will automatically select and transform all variables of type numeric. - - More details in the :ref:`User Guide `. - - Parameters - ---------- - variables: list, default=None - The list of numerical variables to transform. If None, the transformer - will find and select all numerical variables. If C is a dictionary, then this - parameter is ignored and the variables to transform are selected from the - dictionary keys. - - {return_empty} - - base: string, default='e' - Indicates if the natural or base 10 logarithm should be applied. Can take - values 'e' or '10'. - - C: "auto", int or dict, default="auto" - The constant C to add to the variable before the logarithm, i.e., log(x + C). - - - If int, then log(x + C) - - If "auto", then C = abs(min(x)) + 1 - - If dict, dictionary mapping the constant C to apply to each variable. - - Note, when C is a dictionary, the parameter `variables` is ignored. - - Attributes - ---------- - {variables_} + variable to transform and C is a positive constant. - C_: - The constant C to add to each variable. If C = "auto" a dictionary with - C = abs(min(variable)) + 1. For strictly positive variables, C = 0. + .. note:: + `LogCpTransformer` is consolidated into `LogTransformer` and deprecated + in version 2.0.0. It will be removed in version 2.1.0. New code should prefer + ``LogTransformer(C="auto")``, which reproduces `LogCpTransformer`'s + default behavior exactly. - {feature_names_in_} - - {n_features_in_} - - Methods - ------- - fit: - Learn the constant C. - - {fit_transform} - - {inverse_transform} - - transform: - Transform the variables with the logarithm of x plus C. + See :class:`LogTransformer` for the full parameter and attribute reference. Examples -------- @@ -335,124 +331,22 @@ def __init__( base: str = "e", C: Union[int, float, str, Dict[Union[str, int], Union[float, int]]] = "auto", ) -> None: - - if base not in ["e", "10"]: - raise ValueError( - f"base can take only '10' or 'e' as values. Got {base} instead." - ) - - if not isinstance(C, (int, float, dict)) and C != "auto": - raise ValueError( - f"C can take only 'auto', integers or floats. Got {C} instead." - ) - - _check_return_empty_is_bool(return_empty) - - self.variables = _check_variables_input_value(variables) - self.return_empty = return_empty - self.base = base - self.C = C - - def fit(self, X: pd.DataFrame, y: Optional[pd.Series] = None): - """ - Learn the constant C to add to the variable before the logarithm transformation - if C="auto". - - Select the numerical variables or check that the variables entered by the user - are numerical. Then check that the selected variables are positive after - addition of C. - - Parameters - ---------- - X: pandas DataFrame of shape = [n_samples, n_features]. - The training input samples. Can be the entire dataframe, not just the - variables to transform. - - y: pandas Series, default=None - It is not needed in this transformer. You can pass y or None. - """ - - # check input dataframe - if isinstance(self.C, dict): - X = super()._fit_from_dict(X, self.C) - else: - X = super().fit(X) - - self.C_ = self.C - - # calculate C to add to each variable - if self.C == "auto": - # we add 0 to positive variables - c_dict = {var: 0 for var in self.variables_ if X[var].min() > 0} - - # we add the minimum plus 1 to non-positive variables - non_positive_vars = [ - var for var in self.variables_ if var not in c_dict.keys() - ] - c_dict.update(dict(X[non_positive_vars].min(axis=0).abs() + 1)) - self.C_ = c_dict # type:ignore - - return self - - def transform(self, X: pd.DataFrame) -> pd.DataFrame: - """ - Transform the variables with the logarithm of x plus a constant C. - - Parameters - ---------- - X: pandas DataFrame of shape = [n_samples, n_features] - The data to be transformed. - - Returns - ------- - X_new: pandas dataframe - The dataframe with the transformed variables. - """ - - # check input dataframe and if class was fitted - X = self._check_transform_input_and_state(X) - - # check variable is positive after adding c - error_msg = ( - "Some variables contain zero or negative values after adding" - + " constant C, can't apply log." + super().__init__( + variables=variables, return_empty=return_empty, base=base, C=C + ) + warnings.warn( + "LogCpTransformer was deprecated in version 2.0.0 in favour of " + "LogTransformer and will be removed in version 2.1.0. " + 'Use LogTransformer(C="auto") instead.', + FutureWarning, ) - if (X[self.variables_] + self.C_ <= 0).any().any(): - raise ValueError(error_msg) - - X[self.variables_] = X[self.variables_].astype(float) - - # transform - if self.base == "e": - X.loc[:, self.variables_] = np.log(X.loc[:, self.variables_] + self.C_) - else: - X.loc[:, self.variables_] = np.log10(X.loc[:, self.variables_] + self.C_) - - return X - - def inverse_transform(self, X: pd.DataFrame) -> pd.DataFrame: - """ - Convert the data back to the original representation. - - Parameters - ---------- - X: pandas DataFrame of shape = [n_samples, n_features] - The data to be transformed. - - Returns - ------- - X_tr: pandas dataframe - The dataframe with the transformed variables. - """ - - # check input dataframe and if class was fitted - X = self._check_transform_input_and_state(X) - - # inverse transform - if self.base == "e": - X.loc[:, self.variables_] = np.exp(X.loc[:, self.variables_]) - self.C_ - else: - X.loc[:, self.variables_] = 10 ** X.loc[:, self.variables_] - self.C_ + def _more_tags(self): + # LogCpTransformer's default ("auto") always finds a valid shift, so it + # doesn't hit the zero-value errors LogTransformer's C=0 default does. + # Restore the un-xfailed tags rather than inheriting LogTransformer's. + return _return_tags() - return X + def __sklearn_tags__(self): + tags = super().__sklearn_tags__() + return tags diff --git a/tests/test_transformation/test_log_transformer.py b/tests/test_transformation/test_log_transformer.py index d1ccf4a2e..23a74104c 100644 --- a/tests/test_transformation/test_log_transformer.py +++ b/tests/test_transformation/test_log_transformer.py @@ -78,8 +78,11 @@ def test_log_base_10_plus_user_passes_var_list(df_vartypes): def test_error_if_base_value_not_allowed(): - with pytest.raises(ValueError): + with pytest.raises(ValueError) as record: LogTransformer(base="other") + assert str(record.value) == ( + "base can take only '10' or 'e' as values. Got other instead." + ) def test_fit_raises_error_if_na_in_df(df_na): @@ -137,3 +140,127 @@ def test_inverse_e_plus_user_passes_var_list(df_vartypes): assert transformer.n_features_in_ == 5 # test transform output pd.testing.assert_frame_equal(X, df_vartypes) + + +def test_default_C_preserves_original_fail_fast_behavior(): + """LogTransformer()'s default C=0 must raise at fit() time, with the + original exact message, matching pre-merge behavior. See #957.""" + df = pd.DataFrame({"x": [1, 2, 0, 4]}) + tr = LogTransformer() + + assert tr.C == 0 + + with pytest.raises(ValueError) as record: + tr.fit(df) + + assert str(record.value) == ( + "Some variables contain zero or negative values, can't apply log" + ) + + +@pytest.fixture(scope="module") +def df_c(): + df = pd.DataFrame( + { + "vara": [0, 1, 2, 3], + "varb": [5, 5, 6, 7], + "varc": [-2, -1, 0, 4], + "vard": [-3, -2, -1, -5], + "vare": ["a", "b", "c", "d"], + } + ) + return df + + +@pytest.mark.parametrize("c", [1, 0.1, {"var1": 1, "var2": 2}, "auto"]) +def test_c_parameter(c): + tr = LogTransformer(C=c) + assert tr.C == c + + +@pytest.mark.parametrize("c", ["string", [1, 2]]) +def test_c_raises_error(c): + msg = f"C can take only 'auto', integers, floats or dictionaries. Got {c} instead." + with pytest.raises(ValueError) as record: + LogTransformer(C=c) + assert str(record.value) == msg + + +def test_C_when_auto(df_c): + tr = LogTransformer(C="auto") + tr.fit(df_c) + c = {"vara": 1, "varb": 0, "varc": 3, "vard": 6} + assert tr.C_ == c + + +def test_C_when_dict(df_c): + c = {"vara": 1, "varb": 0, "varc": 3, "vard": 6} + tr = LogTransformer(C=c) + tr.fit(df_c) + assert tr.C_ == c + + +def test_C_when_int(df_c): + tr = LogTransformer(C=10) + tr.fit(df_c) + assert tr.C_ == 10 + + +def test_raises_error_when_transformed_data_has_negative_values_with_C(df_c): + tr = LogTransformer(C="auto") + tr.fit(df_c) + dft = df_c.copy() + dft["vara"] = dft["vara"] - 2 + msg = ( + "Some variables contain zero or negative values after adding constant C, " + "can't apply log." + ) + with pytest.raises(ValueError) as record: + tr.transform(dft) + assert str(record.value) == msg + + +def test_log_base_e_with_C(df_c): + dft = LogTransformer(C="auto").fit_transform(df_c) + exp = np.log( + df_c[["vara", "varb", "varc", "vard"]] + + {"vara": 1, "varb": 0, "varc": 3, "vard": 6} + ) + exp["vare"] = df_c["vare"] + pd.testing.assert_frame_equal(dft, exp) + + dft = LogTransformer(C=10).fit_transform(df_c) + exp = np.log(df_c[["vara", "varb", "varc", "vard"]] + 10) + exp["vare"] = df_c["vare"] + pd.testing.assert_frame_equal(dft, exp) + + +def test_log_base_10_with_C(df_c): + dft = LogTransformer(C="auto", base="10").fit_transform(df_c) + exp = np.log10( + df_c[["vara", "varb", "varc", "vard"]] + + {"vara": 1, "varb": 0, "varc": 3, "vard": 6} + ) + exp["vare"] = df_c["vare"] + pd.testing.assert_frame_equal(dft, exp) + + dft = LogTransformer(C=10, base="10").fit_transform(df_c) + exp = np.log10(df_c[["vara", "varb", "varc", "vard"]] + 10) + exp["vare"] = df_c["vare"] + pd.testing.assert_frame_equal(dft, exp) + + +def test_inverse_transform_with_C(df_c): + tr = LogTransformer(C="auto", base="10") + dft = tr.fit_transform(df_c) + orig = tr.inverse_transform(dft) + pd.testing.assert_frame_equal( + orig, df_c, check_dtype=False, check_exact=False, rtol=0.1 + ) + + tr = LogTransformer(C=10, base="e") + dft = tr.fit_transform(df_c) + orig = tr.inverse_transform(dft) + pd.testing.assert_frame_equal( + orig, df_c, check_dtype=False, check_exact=False, rtol=0.1 + ) diff --git a/tests/test_transformation/test_logcp_transformer.py b/tests/test_transformation/test_logcp_transformer.py index 7808c52b4..753cc43bf 100644 --- a/tests/test_transformation/test_logcp_transformer.py +++ b/tests/test_transformation/test_logcp_transformer.py @@ -28,12 +28,23 @@ def test_c_parameter(c): @pytest.mark.parametrize("c", ["string", [1, 2]]) def test_c_raises_error(c): - msg = f"C can take only 'auto', integers or floats. Got {c} instead." + msg = f"C can take only 'auto', integers, floats or dictionaries. Got {c} instead." with pytest.raises(ValueError) as record: LogCpTransformer(C=c) assert str(record.value) == msg +def test_instantiation_raises_future_warning(): + msg = ( + "LogCpTransformer was deprecated in version 2.0.0 in favour of " + "LogTransformer and will be removed in version 2.1.0. " + 'Use LogTransformer(C="auto") instead.' + ) + with pytest.warns(FutureWarning) as record: + LogCpTransformer() + assert str(record[0].message) == msg + + @pytest.fixture(scope="module") def df(): df = pd.DataFrame(