Skip to content
voiage

voiage.ml_policy_voi.evaluate_decision_focused_model_value

evaluate_decision_focused_model_value

Section titled “evaluate_decision_focused_model_value”
evaluate_decision_focused_model_value([positional or keyword] candidate_predictions: dict[str, np.ndarray] = None, [positional or keyword] actual_outcomes: np.ndarray = None, [positional or keyword] intervention_cost: float = None, [positional or keyword] intervention_payoff: float = None, [positional or keyword] predictive_scores: dict[str, float] | None = None, [positional or keyword] decision_threshold: float = 0.5, [positional or keyword] evaluation_id: str = 'eval_ml_model_01', [positional or keyword] current_production_model_id: str | None = None, [positional or keyword] regret_refresh_threshold: float = 1000.0) -> DecisionFocusedModelValueResult

Evaluate candidate models by downstream economic decision value.

Parameters

Section titled “Parameters”

candidate_predictions : dict[str, np.ndarray] Mapping from model_id to 1D array of predicted probabilities. actual_outcomes : np.ndarray 1D array of true binary outcomes (0 or 1). intervention_cost : float Cost of taking the proactive intervention per unit. intervention_payoff : float Payoff/value saved when intervening on a positive outcome unit. predictive_scores : dict[str, float], optional Standard predictive metric (e.g. AUC-ROC or Brier score) for reference. decision_threshold : float Probability threshold for triggering intervention. Default 0.5. evaluation_id : str Unique evaluation ID. current_production_model_id : str, optional ID of currently deployed baseline model. regret_refresh_threshold : float Dollar threshold of policy regret above which refresh is recommended.

Returns

Section titled “Returns”

DecisionFocusedModelValueResult Decision value, regret, chosen model, and refresh recommendation.

Parameters:

  • candidate_predictions dict[str, np.ndarray]
  • actual_outcomes np.ndarray
  • intervention_cost float
  • intervention_payoff float
  • predictive_scores dict[str, float] | None (default: None)
  • decision_threshold float (default: 0.5)
  • evaluation_id str (default: 'eval_ml_model_01')
  • current_production_model_id str | None (default: None)
  • regret_refresh_threshold float (default: 1000.0)

Returns: DecisionFocusedModelValueResult