Probabilities (you may enter up to 10 values)
Event 1
Event 2
Event 3
The Shannon entropy is 0

# Shannon Entropy Calculator

By Julia Żuławińska

Welcome to the Shannon entropy calculator! The entropy of an object or a system is a measure of the randomness within the system. In physics, it's determined by the energy unavailable to do work. In this article, we will explain the form of entropy used in statistics - information entropy. Keep reading to find out how to calculate entropy using the Shannon entropy formula.

Have any other statistical problems, or just interested in the topic? Check out our normal distribution calculator!

## How is Shannon entropy used in information theory?

Shannon entropy, also known as information entropy or the Shannon entropy index, is a measure of the degree of randomness in a set of data.

It is used to calculate the uncertainty that comes with a certain character appearing next in a string of text. The more characters there are, or the more proportional are the frequencies of occurrence, the harder it will be to predict what will come next - resulting in an increased entropy. When the outcome is certain, the entropy is zero.

Apart from information theory, Shannon entropy is used in many fields. Ecology, genetics, and computer sciences are just some of them.

## How to calculate entropy? - entropy formula

Shannon's entropy formula is:

`H(x) = -Σni=1 [P(xi) * logbP(xi)] = Σni=1 [P(xi) * logb(1 / P(xi))]`

Where

• `Σni=1` is a summation operator for probabilities from i to n.
• `P(xi)` is the probability of a single event.

In information theory, entropy has several units. It depends on what the base of the logarithm - `b` - is. Usually, as we're dealing with computers, it's equal to 2 and the unit is known as a bit (also called a shannon). Our Shannon entropy calculator uses this base. When the base equals Euler's number, e, entropy is measured in nats. If it's 10, the unit is a dit, ban or hartley.

Let's use Shannon entropy formula in an example:

1. You have a sequence of numbers: `1035830701`.
2. Each distinct character has a different probability associated with it occurring:
• `p(1) = 2 / 10`.
• `p(0) = 3 / 10`.
• `p(3) = 2 / 10`.
• `p(5) = 1 / 10`.
• `p(8) = 1 / 10`.
• `p(7) = 1 / 10`.
1. Shannon entropy equals:
• `H = p(1) * log2(1/p(1)) + p(0) * log2(1/p(0)) + p(3) * log2(1/p(3)) + p(5) * log2(1/p(5)) + p(8) * log2(1/p(8)) + p(7) * log2(1/p(7))`.
• After inserting the values:
• `H = 0.2 * log2(1/0.2) + 0.3 * log2(1/0.3) + 0.2 * log2(1/0.2) + 0.1 * log2(1/0.1) + 0.1 * log2(1/0.1) + 0.1 * log2(1/0.1)`.
• `H = 2.44644`.

Know you know how to calculate Shannon entropy on your own! Keep reading to find out some facts about entropy!