Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions docs/_ext/family_contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,18 @@
def parameterization_notes(name, backend):
special = ""
if backend == 'statistical' and name == 'bccg_lss':
special += "Natural columns are ``mu, sigma, nu`` with log/log/identity predictors. Mu is not generally the response mean or exact median. Positive responses require a Jacobian and a truncated-normal normalizer. At nu=0 the law is lognormal. Mean is infinite for -1<=nu<0; variance is NaN there and infinite for -2<=nu<-1. Finite moment quadrature has explicit numerical limits, including negative-power variance below sigma=sqrt(machine epsilon). At least three distinct positive-weight responses are required; this necessary guard does not establish estimability of arbitrary formulas.\n\n"
special += (
"* **Parameters.** Natural columns ``mu, sigma, nu`` use log, log and identity links. Positive responses require a Jacobian and truncated-normal normalizer. At ``nu=0`` the law is lognormal.\n"
"* **Moments.** Compute the mean and median from the response law. The mean is infinite for ``-1 <= nu < 0``. Variance is ``NaN`` there and infinite for ``-2 <= nu < -1``. Finite quadrature has numerical limits, including negative-power variance below ``sigma=sqrt(machine epsilon)``.\n"
"* **Fitting.** Supply at least three distinct positive-weight responses. Formula estimability also depends on the fitted design.\n\n"
)
if backend == "statistical" and name == "beinf_lss":
special += "Four formula keys are ``mu, phi, pi0, pi1``. Natural columns are ``mu, phi, pi0, pi1, pi_interior``; raw columns are logit(mu), log(phi), log(pi0/pi_interior), log(pi1/pi_interior). The last two are coupled category log ratios, not independent probability logits. Components contain exactly four predictors; the interior probability depends on both atom predictors and has no separate term. Full covariance is retained for all five natural errors. R BEINF uses sigma=1/sqrt(1+phi), nu=pi0/pi_interior, tau=pi1/pi_interior. All components and distinct interior observations are required for fitting; neural fitting uses its normalized likelihood. The saved gamlss.dist 6.1.1 variance hook adds rather than subtracts squared mean; NAMpy uses the normalized mixture variance and records this discrepancy.\n\n"
special += (
"* **Predictors.** Formula keys ``mu, phi, pi0, pi1`` map to ``logit(mu)``, ``log(phi)``, ``log(pi0/pi_interior)`` and ``log(pi1/pi_interior)``. The atom predictors are coupled category log ratios.\n"
"* **Natural output.** All five natural-parameter standard errors use the full covariance. The columns are ``mu, phi, pi0, pi1, pi_interior``. The fifth column depends on both atom predictors and has no separate additive term.\n"
"* **Fitting.** All components and distinct interior observations are required. Neural fitting uses its normalized likelihood.\n"
"* **R mapping.** R BEINF uses ``sigma=1/sqrt(1+phi)``, ``nu=pi0/pi_interior`` and ``tau=pi1/pi_interior``. NAMpy uses the normalized mixture variance. The saved gamlss.dist 6.1.1 variance hook adds the squared mean where that variance subtracts it.\n\n"
)
if backend == "statistical" and name == "zaga_lss":
special += "Natural columns are ``mu, sigma, pi0`` with log/log/logit links. Mu is the positive Gamma mean, sigma its coefficient of variation, and pi0 the exact zero probability (gamlss.dist ZAGA calls it nu). The whole-response mean is (1-pi0)*mu. Log probability reports mass at zero and density elsewhere. Finite-logit fitting requires both components and distinct positive observations; exact zero/one atom probabilities are available only for probability queries. Neural ZAGA fitting uses its normalized likelihood.\n\n"
if backend == "statistical" and name == "betabinomial_lss":
Expand Down Expand Up @@ -149,9 +158,17 @@ def mathematical_contract(name, backend):
reference = {"bccg_lss": "BCCG", "student_lss": "TF", "beta_lss": "BE", "nb_lss": "NBI", "lognormal_lss": "LOGNO", "weibull_lss": "WEI", "betabinomial_lss": "BB", "zaga_lss": "ZAGA", "beinf_lss": "BEINF"}[name]
text += f"Distribution reference: `gamlss.dist::{reference} <https://github.com/cran/gamlss.dist/blob/master/man/{reference}.Rd>`_. Fitting uses NAMpy's joint penalized engine, not gamlss RS backfitting.\n\n"
if name == 'bccg_lss':
text += "Fixed smoothing and automatic joint REML with outer_newton, bfgs and efs are validated, including three smoothed predictors. Analytic likelihood derivatives through order four include the truncated-normal normalizer and the zero-power limit. Outer gradients/Hessians are checked with offsets and nonunit/zero weights; selected endpoints are compared to an independent derivative-free REML profile. Other method labels retain mgcv general-family coercion to REML, not separate ML/GCV/UBRE/NCV criteria. The optim route lacks family-defined null deviance scaling; magic and nlm.fd are also unavailable. Neural fitting, alternate links, censoring/truncation and deviance residuals remain unsupported. Generic conditional and approximate smoothing-uncertainty covariance transport are available; simulation/refit centile bands are not. Tight stopping rules may report nonconvergence at a float64 plateau; such fits remain ineligible for family ranking. This is distinct from mgcv's existing censored bcg likelihood.\n\n"
text += (
"* **Fit.** Fixed smoothing and joint REML with ``outer_newton``, ``bfgs`` or ``efs`` are validated, including fits with three smoothed predictors. General-family method aliases resolve to REML. ``optim``, ``magic`` and ``nlm.fd`` are unavailable.\n"
"* **Numerical checks.** Derivatives through order four include the normalizer and zero-power limit. Outer checks cover offsets and nonunit or zero weights. An independent derivative-free REML profile checks selected endpoints.\n"
"* **Inference and limits.** Conditional and approximate smoothing-adjusted covariance are available. Simulation or refit centile bands, neural fitting, alternate links, censoring, truncation and deviance residuals are outside this route. Use the separate censored ``bcg`` likelihood for its own supported scope. Fits that stop at a nonconverged float64 plateau are ineligible for family ranking.\n\n"
)
if name in {"lognormal_lss", "weibull_lss", "betabinomial_lss", "zaga_lss", "beinf_lss"}:
text += "Fixed smoothing and automatic general-family REML with outer_newton, bfgs and efs are validated. Method labels retain mgcv general-family coercion to REML; this is not separate ML/GCV/UBRE/NCV support. The optim route is unavailable because the extension has no family-defined null deviance for its objective scaling; magic and nlm.fd are also unsupported. Native deviance residuals and null deviance are unavailable; shared distribution quantile residuals are available. Censoring, truncation and alternate links are unsupported. Existing transformed-coefficient/observation and general-NCV guards remain in force.\n\n"
text += (
"* **Fit.** Use fixed smoothing or automatic general-family REML with ``outer_newton``, ``bfgs`` or ``efs``. Method labels follow mgcv's coercion to REML. The optim route is unavailable because it requires null-deviance scaling. ``magic`` and ``nlm.fd`` are also unsupported.\n"
"* **Diagnostics.** Use shared distribution quantile residuals. Native deviance residuals and null deviance are unavailable.\n"
"* **Scope.** The fitting route excludes censoring, truncation, alternate links, and guarded transformed-coefficient, transformed-observation and general NCV configurations.\n\n"
)
elif backend == "statistical":
text += "References: `mgcv distribution-family definitions <https://stat.ethz.ch/R-manual/R-devel/library/mgcv/html/family.mgcv.html>`_. NAMpy fitting support is listed below.\n\n"
if name in {"gammals", "shash", "bcg"}:
Expand Down
24 changes: 21 additions & 3 deletions docs/_ext/feature_reference.py
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,12 @@ def generate_smooths(root):
)
text += (
heading("Outputs", "-")
+ "The formula adds a term to the fitted model's additive predictor. Its coefficient block has p_term columns after constraints; this dimension differs from EDF. ``predict(type='terms')`` returns a floating array with n rows and one column per retained term for a scalar predictor. ``lpmatrix`` returns the full floating model design, shape (n, p), including the other terms. The scalar Gaussian examples below produce response predictions of shape (n,). Runtime matrix and penalty contracts are documented separately.\n\n"
+ table(["Operation", "Result"], [
("Formula term", "Adds a term to the link predictor. Its coefficient block has ``p_term`` columns after constraints. EDF is a separate fit statistic."),
("``predict(type='terms')``", "Floating array with ``n`` rows and one column per retained term for a scalar predictor."),
("``lpmatrix``", "Full floating model design of shape ``(n, p)``, including the other terms."),
])
+ "The scalar Gaussian examples produce response predictions of shape ``(n,)``.\n\n"
)
if name in {"te", "ti", "t2"}:
text += "Before identifiability constraints, the tensor has the product of the marginal basis dimensions, not a single k columns. Marginal/term constraints and retained t2 blocks determine the final coefficient count.\n\n"
Expand All @@ -396,7 +401,12 @@ def generate_smooths(root):
)
runtime += (
heading("Outputs", "-")
+ "``fit(X, feature_names)`` accepts a numeric/categorical design with n rows and its d column names, initializes the term, and returns the same term instance. ``transform_new(X_new)`` returns a floating NumPy array of shape (n_new, p_term) in the fitted constraint coordinates. ``penalties`` is a list of floating square arrays, each (p_term, p_term). Multiplying the design by its coefficient vector gives the link contribution of shape (n_new,). Invalid feature names, dimensions and basis-specific domains raise ValueError; unsupported compositions raise NotImplementedError.\n\n"
+ table(["Operation", "Result"], [
("``fit(X, feature_names)``", "Initializes the term from ``n`` design rows and their ``d`` column names. Returns the fitted term."),
("``transform_new(X_new)``", "Floating NumPy design of shape ``(n_new, p_term)`` in fitted constraint coordinates. Multiply by the term coefficients for its link contribution."),
("``penalties``", "List of floating penalty matrices, each of shape ``(p_term, p_term)``."),
])
+ "Invalid inputs raise ``ValueError``. Unsupported compositions raise ``NotImplementedError``.\n\n"
+ module_notes(cls)
)
default_basis = inspect.signature(cls).parameters.get("basis")
Expand Down Expand Up @@ -1592,7 +1602,15 @@ def generate_neural_extensions(root):
)
text += heading("Returns and fitted outputs", "-")
if name == "NeuralEnsemble":
text += "Construction returns an unfitted ensemble; ``fit`` accepts data, explicit validation rows, offsets and weights, returns self and exposes ``estimators_``. Regression predict returns the mean member response, shaped like a member's predictions. Classification predict_proba averages probabilities in ``classes_`` order, and predict returns the most probable class. score returns R² or accuracy, respectively. ``predict_components`` returns ``AdditivePrediction`` containing the mean link-scale contributions. ``predict_component_uncertainty`` returns ``EnsembleAdditivePrediction`` containing mean components and between-member variation; missing selected terms contribute zero for that member. Variation across members measures training/selection variability and is not automatically a calibrated confidence interval.\n\n"
text += table(["Operation", "Result"], [
("``fit``", "Fits the members and exposes them in ``estimators_``. Accepts explicit validation data, offsets and weights, and returns the ensemble."),
("``predict``", "Mean member response for regression or most probable class for classification."),
("``predict_proba``", "Mean member probabilities in ``classes_`` order."),
("``predict_components``", "Mean link-scale contributions as ``AdditivePrediction``."),
("``predict_component_uncertainty``", "Mean contributions and between-member variation as ``EnsembleAdditivePrediction``. Missing selected terms contribute zero for that member."),
("``score``", "R² for regression and accuracy for classification."),
])
text += "Between-member variation describes training or selection variability. Calibrated confidence intervals require a separate inference method.\n\n"
elif name in {"NeuralEstimatorFamily", "estimator_family"}:
text += "Returns a NeuralEstimatorFamily record containing task-specific estimator classes or None for unavailable tasks. This record contains no fitted predictions. Instantiate one of its available classes and call fit to obtain a model.\n\n"
else:
Expand Down
Loading
Loading