evaluma.methods.improvability
=============================

.. py:module:: evaluma.methods.improvability


Functions
---------

.. autoapisummary::

   evaluma.methods.improvability._compute_error_matrix
   evaluma.methods.improvability.compute_improvability


Module Contents
---------------

.. py:function:: _compute_error_matrix(raw_matrix: pandas.DataFrame, direction_map: dict, optimum_map: dict) -> pandas.DataFrame

   Reconstruct raw error from a dense score matrix and metric metadata.

   Per column, error is ``optimum - score`` for ``max`` metrics and
   ``score - optimum`` for ``min`` metrics, so higher error is always worse.
   Assumes a dense (complete) matrix.

   :param raw_matrix: Model × dataset raw score matrix.
   :param direction_map: Dataset → ``"min"`` / ``"max"`` metric direction.
   :param optimum_map: Dataset → theoretical error optimum (perfect score).

   :returns: Error matrix with the same shape, index, and columns.
   :rtype: pd.DataFrame


.. py:function:: compute_improvability(raw_matrix: pandas.DataFrame, direction_map: dict, optimum_map: dict) -> evaluma.results.ImprovabilityResult

   Compute mean improvability per model in raw error space.

   Reconstructs raw error, takes the per-dataset best error over models, and
   computes per-cell ``(error - best_err) / error * 100`` (percent error
   reduction needed to match the best method). A zero-error cell is already
   optimal, so its improvability is ``0.0`` (avoiding division by zero). Mean
   improvability averages over datasets. Faithful to the TabArena /
   BeyondArena definition.

   :param raw_matrix: Model × dataset raw score matrix (dense).
   :param direction_map: Dataset → ``"min"`` / ``"max"`` metric direction.
   :param optimum_map: Dataset → theoretical error optimum.

   :returns:

             ``.table`` (``model``, ``improvability``; sorted
                 ascending) and ``.per_dataset`` diagnostic long table.
   :rtype: ImprovabilityResult