kldivergence(): compare probability distributions

kldivergence(p, q) calculates Kullback–Leibler divergence, a measure of how one probability distribution P differs from a reference distribution Q. It is widely used in information theory, Bayesian statistics, and machine learning.

Interpret the result

KL divergence is non-negative and is zero when the distributions match, but it is not symmetric: KL(P‖Q) usually differs from KL(Q‖P). It is therefore a directed discrepancy, not an ordinary distance.

Input requirements

Inputs must be compatible probability distributions: non-negative entries with totals of one. If P assigns positive probability where Q assigns zero, the divergence is infinite or undefined. Normalize counts first with sum() and division.

Related analysis

Use variance() to quantify uncertainty within one distribution, mean() for summary statistics, and log() for the logarithmic core of the formula. Check invalid numeric input with isFinite().