.. include:: _contributors.rst

.. currentmodule:: sklearn

.. _changes_0_20:

Version 0.20 (under development)
================================

This release packs in a mountain of bug fixes, features and enhancements for
the Scikit-learn library, and improvements to the documentation and examples.
Thanks to our many contributors!

Highlights
----------

We have tried to improve our support for common data-science use-cases
including missing values, categorical variables, heterogeneous data, and
features/targets with unusual distributions.

Missing values in features, represented by NaNs, are now accepted in
column-wise preprocessing such as scalers.  Each feature is fitted disregarding
NaNs, and data containing NaNs can be transformed. The new :mod:`impute`
module provides estimators for learning despite missing data.

:class:`~compose.ColumnTransformer` handles the case where different features
or columns of a pandas.DataFrame need different preprocessing.
String or pandas Categorical columns can now be encoded with
:class:`~preprocessing.OneHotEncoder` or
:class:`~preprocessing.OrdinalEncoder`.

:class:`~compose.TransformedTargetRegressor` helps when the regression target
needs to be transformed to be modeled. :class:`~preprocessing.PowerTransformer`
and :class:`~preprocessing.KBinsDiscretizer` join
:class:`~preprocessing.QuantileTransformer` as non-linear transformations.

Beyond this, we have added :term:`sample_weight` support to several estimators
(including :class:`~cluster.KMeans`, :class:`~linear_model.BayesianRidge` and
:class:`~neighbors.KernelDensity`) and improved stopping criteria in others
(including :class:`~neural_network.MLPRegressor`,
:class:`~ensemble.GradientBoostingRegressor` and
:class:`~linear_model.SGDRegressor`).

.. FIXME: remove SGDRegressor if #9043 is not merged for release

This release is also the first to be accompanied by a :ref:`glossary` developed
by `Joel Nothman`_. The glossary is a reference resource to help users and
contributors become familiar with the terminology and conventions used in
Scikit-learn.

Changed models
--------------

The following estimators and functions, when fit with the same data and
parameters, may produce different models from the previous version. This often
occurs due to changes in the modelling logic (bug fixes or enhancements), or in
random sampling procedures.

- :class:`decomposition.IncrementalPCA` in Python 2 (bug fix)
- :class:`isotonic.IsotonicRegression` (bug fix)
- :class:`linear_model.ARDRegression` (bug fix)
- :class:`linear_model.OrthogonalMatchingPursuit` (bug fix)
- :class:`metrics.roc_auc_score` (bug fix)
- :class:`metrics.roc_curve` (bug fix)
- :class:`neural_network.MLPRegressor` (bug fix)
- :class:`neural_network.MLPClassifier` (bug fix)
- :class:`neural_network.BaseMultilayerPerceptron` (bug fix)
- :class:`linear_model.SGDClassifier` (bug fix)
- :class:`linear_model.SGDRegressor` (bug fix)
- :class:`linear_model.PassiveAggressiveClassifier` (bug fix)
- :class:`linear_model.PassiveAggressiveRegressor` (bug fix)
- :class:`linear_model.Perceptron` (bug fix)
- :class:`ensemble.gradient_boosting.GradientBoostingClassifier` (bug fix affecting feature importances)
- The v0.19.0 release notes failed to mention a backwards incompatibility with
  :class:`model_selection.StratifiedKFold` when ``shuffle=True`` due to
  :issue:`7823`.

Details are listed in the changelog below.

(While we are trying to better inform users by providing this information, we
cannot assure that this list is complete.)

Changelog
---------

Support for Python 3.3 has been officially dropped.

New features
............

Classifiers and regressors

- :class:`ensemble.GradientBoostingClassifier` and
  :class:`ensemble.GradientBoostingRegressor` now support early stopping
  via ``n_iter_no_change``, ``validation_fraction`` and ``tol``. :issue:`7071`
  by `Raghav RV`_

- :class:`dummy.DummyRegressor` now has a ``return_std`` option in its
  ``predict`` method. The returned standard deviations will be zeros.

- Added :class:`multioutput.RegressorChain` for multi-target
  regression. :issue:`9257` by :user:`Kumar Ashutosh <thechargedneutron>`.

- Added :class:`naive_bayes.ComplementNB`, which implements the Complement
  Naive Bayes classifier described in Rennie et al. (2003).
  :issue:`8190` by :user:`Michael A. Alcorn <airalcorn2>`.

- :class:`ensemble.BaggingRegressor` and :class:`ensemble.BaggingClassifier` can now
  be fit with missing/non-finite values in X and/or multi-output Y to support
  wrapping pipelines that perform their own imputation.
  :issue:`9707` by :user:`Jimmy Wan <jimmywan>`.

Preprocessing

- Expanded :class:`preprocessing.OneHotEncoder` to allow to encode
  categorical string features as a numeric array using a one-hot (or dummy)
  encoding scheme, and added :class:`preprocessing.OrdinalEncoder` to
  convert to ordinal integers.  Those two classes now handle
  encoding of all feature types (also handles string-valued features) and
  derives the categories based on the unique values in the features instead of
  the maximum value in the features. :issue:`9151` and :issue:`10521` by
  :user:`Vighnesh Birodkar <vighneshbirodkar>` and `Joris Van den Bossche`_.

- Added :class:`preprocessing.KBinsDiscretizer` for turning
  continuous features into categorical or one-hot encoded
  features. :issue:`7668`, :issue:`9647`, :issue:`10195`,
  :issue:`10192`, :issue:`11272` and :issue:`11467`.
  by :user:`Henry Lin <hlin117>`, `Hanmin Qin`_
  and `Tom Dupre la Tour`_.

- Added :class:`compose.ColumnTransformer`, which allows to apply
  different transformers to different columns of arrays or pandas
  DataFrames. :issue:`9012` by `Andreas Müller`_ and `Joris Van den Bossche`_,
  and :issue:`11315` by :user:`Thomas Fan <thomasjpfan>`.

- Added :class:`preprocessing.PowerTransformer`, which implements the Box-Cox
  power transformation, allowing users to map data from any distribution to a
  Gaussian distribution. This is useful as a variance-stabilizing transformation
  in situations where normality and homoscedasticity are desirable.
  :issue:`10210` by :user:`Eric Chang <ericchang00>` and
  :user:`Maniteja Nandana <maniteja123>`.

- Added the :class:`compose.TransformedTargetRegressor` which transforms
  the target y before fitting a regression model. The predictions are mapped
  back to the original space via an inverse transform. :issue:`9041` by
  `Andreas Müller`_ and :user:`Guillaume Lemaitre <glemaitre>`.

- Added :class:`impute.ChainedImputer`, which is a strategy for imputing missing
  values by modeling each feature with missing values as a function of
  other features in a round-robin fashion. :issue:`8478` by
  :user:`Sergey Feldman <sergeyf>`.

- :class:`linear_model.SGDClassifier`, :class:`linear_model.SGDRegressor`,
  :class:`linear_model.PassiveAggressiveClassifier`,
  :class:`linear_model.PassiveAggressiveRegressor` and
  :class:`linear_model.Perceptron` now expose ``early_stopping``,
  ``validation_fraction`` and ``n_iter_no_change`` parameters, to stop
  optimization monitoring the score on a validation set. A new learning rate
  ``"adaptive"`` strategy divides the learning rate by 5 each time
  ``n_iter_no_change`` consecutive epochs fail to improve the model.
  :issue:`9043` by `Tom Dupre la Tour`_.

Model evaluation

- Added the :func:`metrics.davies_bouldin_score` metric for unsupervised
  evaluation of clustering models. :issue:`10827` by :user:`Luis Osa <logc>`.

- Added the :func:`metrics.balanced_accuracy_score` metric and a corresponding
  ``'balanced_accuracy'`` scorer for binary classification.
  :issue:`8066` by :user:`xyguo` and :user:`Aman Dalmia <dalmia>`.

Decomposition, manifold learning and clustering

- :class:`cluster.AgglomerativeClustering` now supports Single Linkage
  clustering via ``linkage='single'``. :issue:`9372` by
  :user:`Leland McInnes <lmcinnes>` and :user:`Steve Astels <sastels>`.

- :class:`cluster.KMeans` and :class:`cluster.MiniBatchKMeans` now support
  sample weights via new parameter ``sample_weight`` in ``fit`` function.
  :issue:`10933` by :user:`Johannes Hansen <jnhansen>`.

- :mod:`dict_learning` functions and models now support positivity constraints.
  This applies to the dictionary and sparse code.
  :issue:`6374` by :user:`John Kirkham <jakirkham>`.

Metrics

- Partial AUC is available via ``max_fpr`` parameter in
  :func:`metrics.roc_auc_score`. :issue:`3273` by
  :user:`Alexander Niederbühl <Alexander-N>`.

- Added ``output_dict`` parameter in :func:`metrics.classification_report`
  to return classification statistics as dictionary.
  :issue:`11160` by :user:`Dan Barkhorn <danielbarkhorn>`.

Misc

- A new configuration parameter, ``working_memory`` was added to control memory
  consumption limits in chunked operations, such as the new
  :func:`metrics.pairwise_distances_chunked`.  See :ref:`working_memory`.
  :issue:`10280` by `Joel Nothman`_ and :user:`Aman Dalmia <dalmia>`.

- An environment variable to use the site joblib instead of the vendored
  one was added (:ref:`environment_variable`). The main API of joblib is now
  exposed in :mod:`sklearn.utils.joblib`. :issue:`11166` by `Gael Varoquaux`_,
  `Joel Nothman`_ and `Olivier Grisel`_.

Enhancements
............

Classifiers and regressors

- In :class:`gaussian_process.GaussianProcessRegressor`, method ``predict``
  is faster when using ``return_std=True`` in particular more when called
  several times in a row. :issue:`9234` by :user:`andrewww <andrewww>`
  and :user:`Minghui Liu <minghui-liu>`.

- Add `named_estimators_` parameter in
  :class:`ensemble.VotingClassifier` to access fitted
  estimators. :issue:`9157` by :user:`Herilalaina Rakotoarison <herilalaina>`.

- Add `var_smoothing` parameter in
  :class:`naive_bayes.GaussianNB` to give a precise control over
  variances calculation. :issue:`9681` by :user:`Dmitry Mottl <Mottl>`.

- Add `n_iter_no_change` parameter in
  :class:`neural_network.BaseMultilayerPerceptron`,
  :class:`neural_network.MLPRegressor`, and
  :class:`neural_network.MLPClassifier` to give control over
  maximum number of epochs to not meet ``tol`` improvement.
  :issue:`9456` by :user:`Nicholas Nadeau <nnadeau>`.

- A parameter ``check_inverse`` was added to
  :class:`preprocessing.FunctionTransformer` to ensure that ``func`` and
  ``inverse_func`` are the inverse of each other.
  :issue:`9399` by :user:`Guillaume Lemaitre <glemaitre>`.

- Add `sample_weight` parameter to the fit method of
  :class:`linear_model.BayesianRidge` for weighted linear regression.
  :issue:`10111` by :user:`Peter St. John <pstjohn>`.

- :class:`dummy.DummyClassifier` and :class:`dummy.DummyRegressor` now
  only require X to be an object with finite length or shape.
  :issue:`9832` by :user:`Vrishank Bhardwaj <vrishank97>`.

- Add `sample_weight` parameter to the fit method of
  :class:`neighbors.KernelDensity` to enables weighting in kernel density
  estimation.
  :issue:`4394` by :user:`Samuel O. Ronsin <samronsin>`.

- :class:`neighbors.RadiusNeighborsRegressor` and
  :class:`neighbors.RadiusNeighborsClassifier` are now
  parallelized according to ``n_jobs`` regardless of ``algorithm``.
  :issue:`8003` by :user:`Joël Billaud <recamshak>`.

- Memory usage improvement for :func:`_class_means` and :func:`_class_cov`
  in :class:`discriminant_analysis`.
  :issue:`10898` by :user:`Nanxin Chen <bobchennan>`.`

- :func:`manifold.t_sne.trustworthiness` accepts metrics other than
  Euclidean. :issue:`9775` by :user:`William de Vazelhes <wdevazelhes>`.

- :mod:`Nearest neighbors <neighbors>` query methods are now more memory
  efficient when ``algorithm='brute'``. :issue:`11136` by `Joel Nothman`_
  and :user:`Aman Dalmia <dalmia>`.

Cluster

- :class:`cluster.KMeans`, :class:`cluster.MiniBatchKMeans` and
  :func:`cluster.k_means` passed with ``algorithm='full'`` now enforces
  row-major ordering, improving runtime.
  :issue:`10471` by :user:`Gaurav Dhingra <gxyd>`.

- :class:`cluster.DBSCAN` now is parallelized according to ``n_jobs``
  regardless of ``algorithm``.
  :issue:`8003` by :user:`Joël Billaud <recamshak>`.

Datasets

- In :func:`datasets.make_blobs`, one can now pass a list to the `n_samples`
  parameter to indicate the number of samples to generate per cluster.
  :issue:`8617` by :user:`Maskani Filali Mohamed <maskani-moh>`
  and :user:`Konstantinos Katrioplas <kkatrio>`.

Preprocessing

- :class:`preprocessing.PolynomialFeatures` now supports sparse input.
  :issue:`10452` by :user:`Aman Dalmia <dalmia>` and `Joel Nothman`_.

- Enable the call to :meth:`get_feature_names` in unfitted
  :class:`feature_extraction.text.CountVectorizer` initialized with a
  vocabulary. :issue:`10908` by :user:`Mohamed Maskani <maskani-moh>`.

- The ``transform`` method of :class:`sklearn.preprocessing.MultiLabelBinarizer`
  now ignores any unknown classes. A warning is raised stating the unknown classes
  classes found which are ignored.
  :issue:`10913` by :user:`Rodrigo Agundez <rragundez>`.

- NaN values are ignored and handled in the following preprocessing methods:
  :class:`preprocessing.MaxAbsScaler`,
  :class:`preprocessing.MinMaxScaler`,
  :class:`preprocessing.RobustScaler`,
  :class:`preprocessing.StandardScaler`,
  :class:`preprocessing.PowerTransformer`,
  :class:`preprocessing.QuantileTransformer` classes and
  :func:`preprocessing.maxabs_scale`,
  :func:`preprocessing.minmax_scale`,
  :func:`preprocessing.robust_scale`,
  :func:`preprocessing.scale`,
  :func:`preprocessing.power_transform`,
  :func:`preprocessing.quantile_transform` functions respectively addressed in
  issues :issue:`11011`, :issue:`11005`, :issue:`11308`, :issue:`11206`,
  :issue:`11306`, and :issue:`10437`.
  By :user:`Lucija Gregov <LucijaGregov>` and
  :user:`Guillaume Lemaitre <glemaitre>`.

- :class:`preprocessing.RobustScaler` and :func:`preprocessing.robust_scale`
  can be fitted using sparse matrices.
  :issue:`11308` by :user:`Guillaume Lemaitre <glemaitre>`.
  
Model evaluation and meta-estimators

- A scorer based on :func:`metrics.brier_score_loss` is also available.
  :issue:`9521` by :user:`Hanmin Qin <qinhanmin2014>`.

- The default of ``iid`` parameter of :class:`model_selection.GridSearchCV`
  and :class:`model_selection.RandomizedSearchCV` will change from ``True`` to
  ``False`` in version 0.22 to correspond to the standard definition of
  cross-validation, and the parameter will be removed in version 0.24
  altogether. This parameter is of greatest practical significance where the
  sizes of different test sets in cross-validation were very unequal, i.e. in
  group-based CV strategies. :issue:`9085` by :user:`Laurent Direr <ldirer>`
  and `Andreas Müller`_.

- The ``predict`` method of :class:`pipeline.Pipeline` now passes keyword
  arguments on to the pipeline's last estimator, enabling the use of parameters
  such as ``return_std`` in a pipeline with caution.
  :issue:`9304` by :user:`Breno Freitas <brenolf>`.

- Add `return_estimator` parameter in :func:`model_selection.cross_validate` to
  return estimators fitted on each split.
  :issue:`9686` by :user:`Aurélien Bellet <bellet>`.

- New ``refit_time_`` attribute will be stored in
  :class:`model_selection.GridSearchCV` and
  :class:`model_selection.RandomizedSearchCV` if ``refit`` is set to ``True``.
  This will allow measuring the complete time it takes to perform
  hyperparameter optimization and refitting the best model on the whole
  dataset. :issue:`11310` by :user:`Matthias Feurer <mfeurer>`.

Decomposition and manifold learning

- Speed improvements for both 'exact' and 'barnes_hut' methods in
  :class:`manifold.TSNE`. :issue:`10593` and :issue:`10610` by
  `Tom Dupre la Tour`_.

- Support sparse input in :meth:`manifold.Isomap.fit`. :issue:`8554` by
  :user:`Leland McInnes <lmcinnes>`.

Metrics

- :func:`metrics.roc_auc_score` now supports binary ``y_true`` other than
  ``{0, 1}`` or ``{-1, 1}``.
  :issue:`9828` by :user:`Hanmin Qin <qinhanmin2014>`.

- :func:`metrics.label_ranking_average_precision_score` now supports vector
  ``sample_weight``.
  :issue:`10845` by :user:`Jose Perez-Parras Toledano <jopepato>`.

- Add ``dense_output`` parameter to :func:`metrics.pairwise.linear_kernel`.
  When False and both inputs are sparse, will return a sparse matrix.
  :issue:`10999` by :user:`Taylor G Smith <tgsmith61591>`.

- :func:`metrics.cluster.silhouette_score` and
  :func:`metrics.cluster.silhouette_samples` are more memory efficient and run
  faster. This avoids some reported freezes and MemoryErrors.
  :issue:`11135` by `Joel Nothman`_.

Linear, kernelized and related models

- Deprecate ``random_state`` parameter in :class:`svm.OneClassSVM` as the
  underlying implementation is not random.
  :issue:`9497` by :user:`Albert Thomas <albertcthomas>`.

Decomposition, manifold learning and clustering

- Deprecate ``precomputed`` parameter in function
  :func:`manifold.t_sne.trustworthiness`. Instead, the new parameter
  ``metric`` should be used with any compatible metric including
  'precomputed', in which case the input matrix ``X`` should be a matrix of
  pairwise distances or squared distances. :issue:`9775` by
  :user:`William de Vazelhes <wdevazelhes>`.

Utils

- Avoid copying the data in :func:`utils.check_array` when the input data is a
  memmap (and ``copy=False``). :issue:`10663` by :user:`Arthur Mensch
  <arthurmensch>` and :user:`Loïc Estève <lesteve>`.

Miscellaneous

- Add ``filename`` attribute to datasets that have a CSV file.
  :issue:`9101` by :user:`alex-33 <alex-33>`
  and :user:`Maskani Filali Mohamed <maskani-moh>`.

Bug fixes
.........

Classifiers and regressors

- Fixed a bug in :class:`isotonic.IsotonicRegression` which incorrectly
  combined weights when fitting a model to data involving points with
  identical X values.
  :issue:`9432` by :user:`Dallas Card <dallascard>`

- Fixed a bug in :class:`neural_network.BaseMultilayerPerceptron`,
  :class:`neural_network.MLPRegressor`, and
  :class:`neural_network.MLPClassifier` with new ``n_iter_no_change``
  parameter now at 10 from previously hardcoded 2.
  :issue:`9456` by :user:`Nicholas Nadeau <nnadeau>`.

- Fixed a bug in :class:`neural_network.MLPRegressor` where fitting
  quit unexpectedly early due to local minima or fluctuations.
  :issue:`9456` by :user:`Nicholas Nadeau <nnadeau>`

- Fixed a bug in :class:`naive_bayes.GaussianNB` which incorrectly raised
  error for prior list which summed to 1.
  :issue:`10005` by :user:`Gaurav Dhingra <gxyd>`.

- Fixed a bug in :class:`linear_model.LogisticRegression` where when using the
  parameter ``multi_class='multinomial'``, the ``predict_proba`` method was
  returning incorrect probabilities in the case of binary outcomes.
  :issue:`9939` by :user:`Roger Westover <rwolst>`.

- Fixed a bug in :class:`linear_model.LogisticRegressionCV` where the
  ``score`` method always computes accuracy, not the metric given by
  the ``scoring`` parameter.
  :issue:`10998` by :user:`Thomas Fan <thomasjpfan>`.

- Fixed a bug in :class:`linear_model.OrthogonalMatchingPursuit` that was
  broken when setting ``normalize=False``.
  :issue:`10071` by `Alexandre Gramfort`_.

- Fixed a bug in :class:`linear_model.ARDRegression` which caused incorrectly
  updated estimates for the standard deviation and the coefficients.
  :issue:`10153` by :user:`Jörg Döpfert <jdoepfert>`.

- Fixed a bug when fitting :class:`ensemble.GradientBoostingClassifier` or
  :class:`ensemble.GradientBoostingRegressor` with ``warm_start=True`` which
  previously raised a segmentation fault due to a non-conversion of CSC matrix
  into CSR format expected by ``decision_function``. Similarly, Fortran-ordered
  arrays are converted to C-ordered arrays in the dense case. :issue:`9991` by
  :user:`Guillaume Lemaitre <glemaitre>`.

- Fixed a bug in :class:`neighbors.NearestNeighbors` where fitting a
  NearestNeighbors model fails when a) the distance metric used is a
  callable and b) the input to the NearestNeighbors model is sparse.
  :issue:`9579` by :user:`Thomas Kober <tttthomasssss>`.

- Fixed a bug in :class:`linear_model.RidgeClassifierCV` where
  the parameter ``store_cv_values`` was not implemented though
  it was documented in ``cv_values`` as a way to set up the storage
  of cross-validation values for different alphas. :issue:`10297` by
  :user:`Mabel Villalba-Jiménez <mabelvj>`.

- Fixed a bug in :class:`naive_bayes.MultinomialNB` which did not accept vector
  valued pseudocounts (alpha).
  :issue:`10346` by :user:`Tobias Madsen <TobiasMadsen>`

- Fixed a bug in :class:`svm.SVC` where when the argument ``kernel`` is
  unicode in Python2, the ``predict_proba`` method was raising an
  unexpected TypeError given dense inputs.
  :issue:`10412` by :user:`Jiongyan Zhang <qmick>`.

- Fixed a bug in :class:`tree.BaseDecisionTree` with `splitter="best"`
  where split threshold could become infinite when values in X were
  near infinite. :issue:`10536` by :user:`Jonathan Ohayon <Johayon>`.

- Fixed a bug in :class:`linear_model.ElasticNet` which caused the input to be
  overridden when using parameter ``copy_X=True`` and ``check_input=False``.
  :issue:`10581` by :user:`Yacine Mazari <ymazari>`.

- Fixed a bug in :class:`sklearn.linear_model.Lasso`
  where the coefficient had wrong shape when ``fit_intercept=False``.
  :issue:`10687` by :user:`Martin Hahn <martin-hahn>`.

- Fixed a bug in :func:`sklearn.linear_model.LogisticRegression` where the
  multi_class='multinomial' with binary output with warm_start = True
  :issue:`10836` by :user:`Aishwarya Srinivasan <aishgrt1>`.

- Fixed a bug in :class:`linear_model.RidgeCV` where using integer ``alphas``
  raised an error. :issue:`10393` by :user:`Mabel Villalba-Jiménez <mabelvj>`.

- Fixed condition triggering gap computation in :class:`linear_model.Lasso`
  and :class:`linear_model.ElasticNet` when working with sparse matrices.
  :issue:`10992` by `Alexandre Gramfort`_.

- Fixed a bug in :class:`linear_model.SGDClassifier`,
  :class:`linear_model.SGDRegressor`,
  :class:`linear_model.PassiveAggressiveClassifier`,
  :class:`linear_model.PassiveAggressiveRegressor` and
  :class:`linear_model.Perceptron`, where the stopping criterion was stopping
  the algorithm before convergence. A parameter `n_iter_no_change` was added
  and set by default to 5. Previous behavior is equivalent to setting the
  parameter to 1. :issue:`9043` by `Tom Dupre la Tour`_.

- Fixed a bug where liblinear and libsvm-based estimators would segfault if
  passed a scipy.sparse matrix with 64-bit indices. They now raise a
  ValueError.
  :issue:`11327` by :user:`Karan Dhingra <kdhingra307>` and `Joel Nothman`_.

- Fixed a bug in :class:`ensemble.gradient_boosting.GradientBoostingRegressor`
  and :class:`ensemble.gradient_boosting.GradientBoostingClassifier` to have
  feature importances summed and then normalized, rather than normalizing on a
  per-tree basis. The previous behavior over-weighted the Gini importance of
  features that appear in later stages. This issue only affected feature
  importances. :issue:`11176` by :user:`Gil Forsyth <gforsyth>`.

Decomposition, manifold learning and clustering

- Fix for uninformative error in :class:`decomposition.IncrementalPCA`:
  now an error is raised if the number of components is larger than the
  chosen batch size. The ``n_components=None`` case was adapted accordingly.
  :issue:`6452`. By :user:`Wally Gauze <wallygauze>`.

- Fixed a bug where the ``partial_fit`` method of
  :class:`decomposition.IncrementalPCA` used integer division instead of float
  division on Python 2 versions. :issue:`9492` by
  :user:`James Bourbeau <jrbourbeau>`.

- Fixed a bug where the ``fit`` method of
  :class:`cluster.AffinityPropagation` stored cluster
  centers as 3d array instead of 2d array in case of non-convergence. For the
  same class, fixed undefined and arbitrary behavior in case of training data
  where all samples had equal similarity.
  :issue:`9612`. By :user:`Jonatan Samoocha <jsamoocha>`.

- In :class:`decomposition.PCA` selecting a n_components parameter greater than
  the number of samples now raises an error.
  Similarly, the ``n_components=None`` case now selects the minimum of
  n_samples and n_features. :issue:`8484`. By :user:`Wally Gauze <wallygauze>`.

- Fixed a bug in :func:`datasets.fetch_kddcup99`, where data were not properly
  shuffled. :issue:`9731` by `Nicolas Goix`_.

- Fixed a bug in :class:`decomposition.PCA` where users will get unexpected error
  with large datasets when ``n_components='mle'`` on Python 3 versions.
  :issue:`9886` by :user:`Hanmin Qin <qinhanmin2014>`.

- Fixed a bug when setting parameters on meta-estimator, involving both a
  wrapped estimator and its parameter. :issue:`9999` by :user:`Marcus Voss
  <marcus-voss>` and `Joel Nothman`_.

- ``k_means`` now gives a warning, if the number of distinct clusters found
  is smaller than ``n_clusters``. This may occur when the number of distinct
  points in the data set is actually smaller than the number of cluster one is
  looking for. :issue:`10059` by :user:`Christian Braune <christianbraune79>`.

- Fixed a bug in :func:`datasets.make_circles`, where no odd number of data
  points could be generated. :issue:`10037`
  by :user:`Christian Braune <christianbraune79>`.

- Fixed a bug in :func:`cluster.spectral_clustering` where the normalization of
  the spectrum was using a division instead of a multiplication. :issue:`8129`
  by :user:`Jan Margeta <jmargeta>`, :user:`Guillaume Lemaitre <glemaitre>`,
  and :user:`Devansh D. <devanshdalal>`.

- Fixed a bug in :class:`mixture.BaseMixture` where the reported `n_iter_` was
  missing an iteration. It affected :class:`mixture.GaussianMixture` and
  :class:`mixture.BayesianGaussianMixture`. :issue:`10740` by :user:`Erich
  Schubert <kno10>` and :user:`Guillaume Lemaitre <glemaitre>`.

- Fixed a bug in :class:`decomposition.SparseCoder` when running OMP sparse
  coding in parallel using readonly memory mapped datastructures. :issue:`5956`
  by :user:`Vighnesh Birodkar <vighneshbirodkar>` and
  :user:`Olivier Grisel <ogrisel>`.

- Fixed a bug in :func:`cluster.k_means_elkan` where the returned `iteration`
  was 1 less than the correct value. Also added the missing `n_iter_` attribute
  in the docstring of :class:`cluster.KMeans`. :issue:`11353` by
  :user:`Jeremie du Boisberranger <jeremiedbb>`.

Metrics

- Fixed a bug in :func:`metrics.precision_recall_fscore_support`
  when truncated `range(n_labels)` is passed as value for `labels`.
  :issue:`10377` by :user:`Gaurav Dhingra <gxyd>`.

- Fixed a bug due to floating point error in :func:`metrics.roc_auc_score` with
  non-integer sample weights. :issue:`9786` by :user:`Hanmin Qin <qinhanmin2014>`.

- Fixed a bug where :func:`metrics.roc_curve` sometimes starts on y-axis instead
  of (0, 0), which is inconsistent with the document and other implementations.
  Note that this will not influence the result from :func:`metrics.roc_auc_score`
  :issue:`10093` by :user:`alexryndin <alexryndin>`
  and :user:`Hanmin Qin <qinhanmin2014>`.

- Fixed a bug to avoid integer overflow. Casted product to 64 bits integer in
  :func:`metrics.mutual_info_score`.
  :issue:`9772` by :user:`Kumar Ashutosh <thechargedneutron>`.

- Fixed a bug in :func:`metrics.fowlkes_mallows_score` to avoid integer
  overflow. Casted return value of `contingency_matrix` to `int64` and computed
  product of square roots rather than square root of product.
  :issue:`9515` by :user:`Alan Liddell <aliddell>` and
  :user:`Manh Dao <manhdao>`.

Neighbors

- Fixed a bug so ``predict`` in :class:`neighbors.RadiusNeighborsRegressor` can
  handle empty neighbor set when using non uniform weights. Also raises a new
  warning when no neighbors are found for samples.  :issue:`9655` by
  :user:`Andreas Bjerre-Nielsen <abjer>`.

Feature Extraction

- Fixed a bug in :func:`feature_extraction.image.extract_patches_2d` which would
  throw an exception if ``max_patches`` was greater than or equal to the number
  of all possible patches rather than simply returning the number of possible
  patches. :issue:`10100` by :user:`Varun Agrawal <varunagrawal>`

- Fixed a bug in :class:`feature_extraction.text.CountVectorizer`,
  :class:`feature_extraction.text.TfidfVectorizer`,
  :class:`feature_extraction.text.HashingVectorizer` to support 64 bit sparse
  array indexing necessary to process large datasets with more than 2·10⁹ tokens
  (words or n-grams). :issue:`9147` by :user:`Claes-Fredrik Mannby <mannby>`
  and `Roman Yurchak`_.

- Fixed bug in :class:`feature_extraction.text.TFIDFVectorizer` which
  was ignoring the parameter ``dtype``. In addition,
  :class:`feature_extraction.text.TFIDFTransformer` will preserve ``dtype``
  for floating and raise a warning if ``dtype`` requested is integer.
  :issue:`10441` by :user:`Mayur Kulkarni <maykulkarni>` and
  :user:`Guillaume Lemaitre <glemaitre>`.

Utils

- :func:`utils.check_array` yield a ``FutureWarning`` indicating
  that arrays of bytes/strings will be interpreted as decimal numbers
  beginning in version 0.22. :issue:`10229` by :user:`Ryan Lee <rtlee9>`

Preprocessing

- Fixed bugs in :class:`preprocessing.LabelEncoder` which would sometimes throw
  errors when ``transform`` or ``inverse_transform`` was called with empty arrays.
  :issue:`10458` by :user:`Mayur Kulkarni <maykulkarni>`.

- Fix ValueError in :class:`preprocessing.LabelEncoder` when using
  ``inverse_transform`` on unseen labels. :issue:`9816` by :user:`Charlie Newey
  <newey01c>`.

- Fix bug in :class:`preprocessing.OneHotEncoder` which discarded the ``dtype``
  when returning a sparse matrix output. :issue:`11042` by :user:`Daniel
  Morales <DanielMorales9>`.

- Fix ``fit`` and ``partial_fit`` in :class:`preprocessing.StandardScaler` in
  the rare case when `with_mean=False` and `with_std=False` which was crashing
  by calling ``fit`` more than once and giving inconsistent results for
  ``mean_`` whether the input was a sparse or a dense matrix. ``mean_`` will be
  set to ``None`` with both sparse and dense inputs. ``n_samples_seen_`` will
  be also reported for both input types.
  :issue:`11235` by :user:`Guillaume Lemaitre <glemaitre>`.

Feature selection

- Fixed computation of ``n_features_to_compute`` for edge case with tied CV
  scores in :class:`feature_selection.RFECV`. :issue:`9222` by `Nick Hoh
  <nickypie>`.

Model evaluation and meta-estimators

- Add improved error message in :func:`model_selection.cross_val_score` when
  multiple metrics are passed in ``scoring`` keyword.
  :issue:`11006` by :user:`Ming Li <minggli>`.

Datasets

- Fixed a bug in :func:`datasets.load_boston` which had a wrong data point.
  :issue:`10801` by :user:`Takeshi Yoshizawa <tarcusx>`.

- Fixed a bug in :func:`datasets.load_iris` which had two wrong data points.
  :issue:`11082` by :user:`Sadhana Srinivasan <rotuna>`
  and :user:`Hanmin Qin <qinhanmin2014>`.

API changes summary
-------------------

Linear, kernelized and related models

- Deprecate ``random_state`` parameter in :class:`svm.OneClassSVM` as the
  underlying implementation is not random.
  :issue:`9497` by :user:`Albert Thomas <albertcthomas>`.

- Deprecate ``positive=True`` option in :class:`linear_model.Lars` as the
  underlying implementation is broken. Use :class:`linear_model.Lasso` instead.
  :issue:`9837` by `Alexandre Gramfort`_.

- ``n_iter_`` may vary from previous releases in
  :class:`linear_model.LogisticRegression` with ``solver='lbfgs'`` and
  :class:`linear_model.HuberRegressor`.  For Scipy <= 1.0.0, the optimizer could
  perform more than the requested maximum number of iterations. Now both
  estimators will report at most ``max_iter`` iterations even if more were
  performed. :issue:`10723` by `Joel Nothman`_.

- The default value of ``gamma`` parameter of :class:`svm.SVC`,
  :class:`~svm.NuSVC`, :class:`~svm.SVR`, :class:`~svm.NuSVR`,
  :class:`~svm.OneClassSVM` will change from ``'auto'`` to ``'scale'`` in
  version 0.22 to account better for unscaled features. :issue:`8361` by
  :user:`Gaurav Dhingra <gxyd>` and :user:`Ting Neo <neokt>`.

- Added convergence warning to :class:`svm.LinearSVC` and
  :class:`linear_model.LogisticRegression` when ``verbose`` is set to 0.
  :issue:`10881` by :user:`Alexandre Sevin <AlexandreSev>`.

Preprocessing

- Deprecate ``n_values`` and ``categorical_features`` parameters and
  ``active_features_``, ``feature_indices_`` and ``n_values_`` attributes
  of :class:`preprocessing.OneHotEncoder`. The ``n_values`` parameter can be
  replaced with the new ``categories`` parameter, and the attributes with the
  new ``categories_`` attribute. Selecting the categorical features with
  the ``categorical_features`` parameter is now better supported using the
  :class:`compose.ColumnTransformer`.
  :issue:`10521` by `Joris Van den Bossche`_.

Decomposition, manifold learning and clustering

- Deprecate ``precomputed`` parameter in function
  :func:`manifold.t_sne.trustworthiness`. Instead, the new parameter
  ``metric`` should be used with any compatible metric including
  'precomputed', in which case the input matrix ``X`` should be a matrix of
  pairwise distances or squared distances. :issue:`9775` by
  :user:`William de Vazelhes <wdevazelhes>`.

- Added function :func:`fit_predict` to :class:`mixture.GaussianMixture` and
  :class:`mixture.GaussianMixture`, which is essentially equivalent to calling
  :func:`fit` and :func:`predict`. :issue:`10336` by
  :user:`Shu Haoran <haoranShu>` and :user:`Andrew Peng <Andrew-peng>`.

Metrics

- Deprecate ``reorder`` parameter in :func:`metrics.auc` as it's no longer required
  for :func:`metrics.roc_auc_score`. Moreover using ``reorder=True`` can hide bugs
  due to floating point error in the input.
  :issue:`9851` by :user:`Hanmin Qin <qinhanmin2014>`.

- The ``batch_size`` parameter to :func:`metrics.pairwise_distances_argmin_min`
  and :func:`metrics.pairwise_distances_argmin` is deprecated to be removed in
  v0.22.  It no longer has any effect, as batch size is determined by global
  ``working_memory`` config. See :ref:`working_memory`. :issue:`10280` by `Joel
  Nothman`_ and :user:`Aman Dalmia <dalmia>`.

Cluster

- Deprecate ``pooling_func`` unused parameter in
  :class:`cluster.AgglomerativeClustering`. :issue:`9875` by :user:`Kumar Ashutosh
  <thechargedneutron>`.

Imputer

- Deprecate :class:`preprocessing.Imputer` and move the corresponding module to
  :class:`impute.SimpleImputer`. :issue:`9726` by :user:`Kumar Ashutosh
  <thechargedneutron>`.

- The ``axis`` parameter that was in :class:`preprocessing.Imputer` is no
  longer present in :class:`impute.SimpleImputer`. The behavior is equivalent
  to ``axis=0`` (impute along columns). Row-wise imputation can be performed
  with FunctionTransformer (e.g., ``FunctionTransformer(lambda X:
  SimpleImputer().fit_transform(X.T).T)``). :issue:`10829` by :user:`Guillaume
  Lemaitre <glemaitre>` and :user:`Gilberto Olimpio <gilbertoolimpio>`.

- The :class:`impute.SimpleImputer` has a new strategy, ``'constant'``, to
  complete missing values with a fixed one, given by the ``fill_value``
  parameter. This strategy supports numeric and non-numeric data, and so does
  the ``'most_frequent'`` strategy now. :issue:`11211` by :user:`Jeremie du
  Boisberranger <jeremiedbb>`.

- The NaN marker for the missing values has been changed between the
  :class:`preprocessing.Imputer` and the :class:`impute.SimpleImputer`.
  ``missing_values='NaN'`` should now be ``missing_values=np.nan``.
  :issue:`11211` by :user:`Jeremie du Boisberranger <jeremiedbb>`.

Outlier Detection models

- More consistent outlier detection API:
  Add a ``score_samples`` method in :class:`svm.OneClassSVM`,
  :class:`ensemble.IsolationForest`, :class:`neighbors.LocalOutlierFactor`,
  :class:`covariance.EllipticEnvelope`. It allows to access raw score
  functions from original papers. A new ``offset_`` parameter allows to link
  ``score_samples`` and ``decision_function`` methods.
  The ``contamination`` parameter of :class:`ensemble.IsolationForest` and
  :class:`neighbors.LocalOutlierFactor` ``decision_function`` methods is used
  to define this ``offset_`` such that outliers (resp. inliers) have negative (resp.
  positive) ``decision_function`` values. By default, ``contamination`` is
  kept unchanged to 0.1 for a deprecation period. In 0.22, it will be set to "auto",
  thus using method-specific score offsets.
  In :class:`covariance.EllipticEnvelope` ``decision_function`` method, the
  ``raw_values`` parameter is deprecated as the shifted Mahalanobis distance
  will be always returned in 0.22. :issue:`9015` by `Nicolas Goix`_.

Covariance

- The :func:`covariance.graph_lasso`, :class:`covariance.GraphLasso` and
  :class:`covariance.GraphLassoCV` have been renamed to
  :func:`covariance.graphical_lasso`, :class:`covariance.GraphicalLasso` and
  :class:`covariance.GraphicalLassoCV` respectively and will be removed in version 0.22.
  :issue:`9993` by :user:`Artiem Krinitsyn <artiemq>`

Misc

- Changed warning type from :class:`UserWarning` to
  :class:`exceptions.ConvergenceWarning` for failing convergence in
  :func:`linear_model.logistic_regression_path`,
  :class:`linear_model.RANSACRegressor`, :func:`linear_model.ridge_regression`,
  :class:`gaussian_process.GaussianProcessRegressor`,
  :class:`gaussian_process.GaussianProcessClassifier`,
  :func:`decomposition.fastica`, :class:`cross_decomposition.PLSCanonical`,
  :class:`cluster.AffinityPropagation`, and :class:`cluster.Birch`.
  :issue:`#10306` by :user:`Jonathan Siebert <jotasi>`.

- Changed ValueError exception raised in :class:`model_selection.ParameterSampler`
  to a UserWarning for case where the class is instantiated with a greater value of
  ``n_iter`` than the total space of parameters in the parameter grid. ``n_iter`` now
  acts as an upper bound on iterations.
  :issue:`#10982` by :user:`Juliet Lawton <julietcl>`

- Invalid input for :class:`model_selection.ParameterGrid` now raises TypeError.
  :issue:`10928` by :user:`Solutus Immensus <solutusimmensus>`

- :func:`utils.check_array` and :func:`utils.check_X_y` now have
  ``accept_large_sparse`` to control whether scipy.sparse matrices with 64-bit
  indices should be rejected.
  :issue:`11327` by :user:`Karan Dhingra <kdhingra307>` and `Joel Nothman`_.

Preprocessing

- In :class:`preprocessing.FunctionTransformer`, the default of ``validate``
  will be from ``True`` to ``False`` in 0.22.
  :issue:`10655` by :user:`Guillaume Lemaitre <glemaitre>`.

Changes to estimator checks
---------------------------

These changes mostly affect library developers.

- Allow tests in :func:`utils.estimator_checks.check_estimator` to test functions
  that accept pairwise data.
  :issue:`9701` by :user:`Kyle Johnson <gkjohns>`

- Allow :func:`~utils.estimator_checks.check_estimator` to check that there is no
  private settings apart from parameters during estimator initialization.
  :issue:`9378` by :user:`Herilalaina Rakotoarison <herilalaina>`

- Add ``check_methods_subset_invariance`` to
  :func:`~utils.estimator_checks.check_estimator`, which checks that
  estimator methods are invariant if applied to a data subset.  :issue:`10420`
  by :user:`Jonathan Ohayon <Johayon>`

- Add tests in :func:`utils.estimator_checks.check_estimator` to check that an
  estimator can handle read-only memmap input data. :issue:`10663` by
  :user:`Arthur Mensch <arthurmensch>` and :user:`Loïc Estève <lesteve>`.

- ``check_sample_weights_pandas_series`` now uses 8 rather than 6 samples
  to accommodate for the default number of clusters in :class:`cluster.KMeans`.
  :issue:`10933` by :user:`Johannes Hansen <jnhansen>`.
