Calculators
Shannon Entropy Calculator
Calculate Shannon entropy in bits and nats from class counts, weights or probabilities with automatic normalization in your browser.
Enter one non-negative count or weight per class, separated by spaces, commas or semicolons. Weights are normalized automatically. Use 1–10000 values and at most 200000 characters. Nonzero weights must be between 1e-100 and 1e100; at least one must be positive. Calculation runs locally in your browser.
How Shannon entropy works
Shannon entropy is H = −Σpᵢ log₂(pᵢ), where each probability is a class weight divided by the total. It measures uncertainty in a categorical distribution. Equal weights 1, 1 give 1 bit; weights 3, 1 give about 0.811278124459 bits. A single positive class gives zero. Zero-weight classes contribute zero. For k entered classes the maximum is log₂(k), attained by equal weights. Entropy in nats is H × ln(2); the effective number of classes is 2^H, the number of equally likely classes with the same entropy. This is distribution entropy, not text compression, password strength, or information gain from a tree split. Enter 1–10000 non-negative counts or weights separated by spaces, commas or semicolons, with at most 200000 characters. Commas are separators, not thousands punctuation. Scientific notation is accepted; nonzero values must be between 1e-100 and 1e100 and at least one must be positive. Scaling avoids overflow; compensated summation and log1p reduce numerical error near pure distributions. Results use browser floating-point arithmetic and display up to 12 significant digits. Everything runs locally.
Frequently Asked Questions
Everything you need to know about this tool, how it works, and privacy.
What is the Shannon entropy formula?
H = −Σpᵢ log₂(pᵢ) in bits. Zero probabilities contribute zero by convention. Equal weights across four classes give 2 bits.
Do inputs need to sum to one?
No. Non-negative counts or weights are automatically normalized; at least one value must be positive.
What do bits, nats and effective classes mean?
Bits use base-2 logarithms; nats use natural logarithms. Effective classes equal 2 raised to the entropy in bits and represent an equally likely distribution with the same uncertainty.
What is the maximum entropy?
For k entered classes it is log₂(k) bits, attained when all k weights are equal. Adding zero-weight classes changes this theoretical maximum but not the measured entropy.
Does this calculate information gain?
No. Information gain requires comparing parent entropy with the weighted entropies of child distributions. This calculator evaluates one distribution at a time.