Calculators
Cross Entropy Calculator
Calculate cross entropy in bits and nats between observed and predicted class distributions with automatic normalization in your browser.
Enter matching non-negative counts, weights or probabilities. Each vector is normalized independently to sum to 1. Separate values with spaces, commas or semicolons. Each vector allows 1–10000 values and 200000 characters; nonzero values must be between 1e-100 and 1e100. At least one value per vector must be positive. Coordinates must represent the same categories in the same order. P is the observed distribution; Q is the prediction. A positive P with zero Q gives infinite cross entropy. Calculation runs locally in your browser.
How cross entropy works
Cross entropy is H(P, Q) = −Σpᵢ log₂(qᵢ) in bits, or −Σpᵢ ln(qᵢ) in nats. P represents observed class proportions and Q represents predicted proportions. Each vector is normalized independently, and coordinates must use the same class order. For P = (1, 0) and Q = (1, 3), the result is 2 bits or about 1.38629436112 nats. Terms with zero P contribute zero, including when Q is also zero; positive P paired with zero Q gives infinity. This quantity is directional and may exceed log₂(k). Matching distributions give Shannon entropy, which need not be zero. It measures average logarithmic prediction loss, not a symmetric distance or a binary loss applied independently to each coordinate. Enter 1–10000 non-negative weights per vector, separated by spaces, commas or semicolons, up to 200000 characters. Commas separate values, not thousands. Nonzero weights must be between 1e-100 and 1e100; each vector needs a positive value. Scaling, compensated summation and log1p reduce numerical error. Results use browser floating-point arithmetic and show up to 12 significant digits. All processing runs locally.
Frequently Asked Questions
Everything you need to know about this tool, how it works, and privacy.
What is the cross entropy formula?
H(P, Q) = −Σpᵢ log₂(qᵢ) in bits. Natural logarithms give nats. For P = (1, 0) and Q = (0.25, 0.75), the result is 2 bits.
Do my inputs need to sum to one?
No. Both vectors are normalized independently from non-negative counts or weights. They need equal lengths, matching class order and at least one positive weight each.
How are zero probabilities handled?
A zero observed probability contributes zero. If an observed class has positive probability but its prediction is zero, cross entropy is infinite. No smoothing is applied.
Is cross entropy symmetric?
No. P is the observed distribution and Q is the prediction. Reversing them can change the result.
How does this differ from Shannon entropy?
Shannon entropy uses the same distribution for P and Q. Cross entropy evaluates predictions Q against observations P and equals H(P) + KL(P ∥ Q). It is minimized at Q = P and has no fixed upper bound.