Imagine a reality in which nothing can be distinguished from anything else. No contrasting colors, no detectable changes, no observation that separates one possibility from another.
Would that reality have zero entropy? Would it contain no information? Would it be equivalent to nothing existing at all?
These questions sound similar, but they ask about different things. Entropy describes uncertainty within a probability model. Distinguishability concerns which differences an observation can reveal. Geometry describes relationships across a space of possibilities. None of these, by itself, settles what exists.
Separating them leads from a simple color experiment to information geometry—and then to a boundary where physics gives way to ontology.
Here, before entropy names a conceptual dependency: defining probabilities over distinct observable outcomes presupposes a way to distinguish those outcomes. It makes no claim that distinguishability appeared earlier in the history of the universe.
Equal Entropy, Different States
Consider two sources that emit colored signals. Source A produces green 80 percent of the time and orange 20 percent of the time. Source B reverses those probabilities.
| Outcome | Source A | Source B |
|---|---|---|
| Green | 80% | 20% |
| Orange | 20% | 80% |
Their Shannon entropies are identical:
H(P) = −Σᵢ pᵢ log₂ pᵢ
For either source, the result is approximately 0.722 bits per signal. Swapping the outcome labels preserves the uncertainty quantified by Shannon entropy. The foundational treatment is Claude Shannon’s A Mathematical Theory of Communication .
Yet the sources are observably different. Watch a long sequence of signals and the dominant color provides evidence about which source is operating. A single green signal does not identify the source with certainty; both can produce it.
That distinction matters. Statistical distinguishability need not mean perfect identification from one observation. It can mean that repeated observations allow competing explanations to be separated with increasing confidence.
Entropy compresses a distribution into one number. That number cannot preserve everything about the distribution it describes.
Measuring a Difference Between Distributions
One way to compare probability distributions is the Kullback–Leibler divergence:
DKL(P ∥ Q) = Σᵢ pᵢ log₂(pᵢ / qᵢ)
For our color sources, it gives:
DKL(A ∥ B) = 0.8 log₂ 4 + 0.2 log₂ ¼ = 1.2 bits.
This calculated value is positive even though their entropies match. KL divergence compares two distributions; entropy characterizes one.
KL divergence is generally asymmetric and does not satisfy the triangle inequality, so it is not an ordinary distance. In this particular example, reversing A and B happens to give the same result. Frank Nielsen’s introduction to information geometry explains statistical divergences and their geometric role.
How Statistical Distinguishability Supports Geometry
Now consider a smooth family of distributions p(x; θ), indexed by parameters θ. Nearby parameter values describe nearby statistical models.
Under standard regularity conditions, the local quadratic expansion of KL divergence yields the Fisher information metric. Its components are:
gij(θ) = Eθ[(∂i ln p)(∂j ln p)].
Using natural logarithms, the expansion takes the form:
DKL(pθ ∥ pθ+dθ) = ½ Σi,j gij(θ) dθi dθj + higher-order terms.
Logarithms in base two introduce a factor of 1/ln 2. For an identifiable regular model, this metric describes how sensitively observable probabilities change as parameters vary. These constructions are developed in Nielsen’s survey .
A landscape of distributions therefore has structure that their individual entropy values cannot specify. Knowing each location’s height would not, by itself, tell you the distances between locations. Likewise, a scalar entropy value does not determine statistical geometry.
This is a construction within statistical information geometry. State spaces can carry other geometric structures, specified independently of this Fisher-metric construction. The example does not derive every possible geometry from probabilities or establish a universal order in which geometry must arise.
Before Probability: Which Differences Are Available?
We can step back from probabilities altogether.
Suppose an observation is represented by a function f. Two underlying states x and y count as indistinguishable under that observation when:
x ∼ y exactly when f(x) = f(y).
This defines equivalence classes: groups of states that the observation treats alike. A detector that records brightness but ignores color might place a green signal and an orange signal in the same class.
Difference and distinguishability are separate concepts. Two states can be distinct in the model while producing the same observation:
x ≠ y does not imply f(x) ≠ f(y).
Model-level difference concerns which states the description treats as distinct; observational distinguishability concerns what a specified observation can separate. Whether the model’s distinctions correspond to ontological differences requires further justification. A limited observation horizon, by itself, establishes no corresponding limit on reality.
The qualification under that observation does real work. Change the detector and the classes may change. For noisy measurements, we can instead group states that produce identical outcome distributions across a specified set of permitted measurements. Merely being difficult to tell apart at some practical threshold need not define an equivalence relation.
At this level, we have a structure of possible distinctions. To calculate Shannon entropy, we must additionally assign probabilities. For the statistical geometry illustrated above, we need a suitable smooth family of probability models.
The existence of several distinguishable possibilities does not yet tell us how likely any of them are.
Two Different Meanings of a Featureless Reality
Return to the imagined reality without visible differences. There are at least two ways to model it.
Only One Distinguishable Possibility
If the model contains one possible outcome with probability one, its Shannon entropy is zero. There is no uncertainty about which outcome will occur.
But zero uncertainty does not mean zero existence. A certain outcome is still an outcome within a model. The entropy calculation does not explain why that model, its state or its underlying structure exists.
Many States, One Observable Outcome
Alternatively, many underlying states might all produce the same observation. The observed outcome then has zero Shannon entropy, while a probability distribution over the hidden states could have positive entropy.
A screen displaying one flat color makes the distinction intuitive. The display might conceal a complex running computation. The appearance alone does not reveal how many internal configurations are possible.
Nor does spatial uniformity establish low thermodynamic entropy. A gas at equilibrium can have uniform macroscopic properties while admitting many microscopic configurations. Uniform appearance, certainty about an observed variable and microscopic multiplicity are separate questions.
Is Difference More Fundamental Than Entropy?
One useful conceptual construction is:
Possible states and permitted observations → distinguishable classes → probabilities over those classes → entropy.
A smooth statistical model can also support information geometry. These are branches of a conceptual construction, rather than stages in a physical process: entropy and the Fisher metric describe different aspects of a probability model. Neither must arise temporally before the other.
Still, it shifts the philosophical question. Before asking how much uncertainty a system contains, we can ask which differences the description allows us to formulate—and which observations could reveal them.
The same caution applies when discussing the geography of knowledge : a measure of quantity does not automatically describe the relationships that make a space navigable.
Where Observation Runs Out
What if a proposed reality could produce no distinguishable consequence under any possible observation?
By that premise, observation could supply no evidence separating its existence from its absence. This establishes an operational limit. It does not establish that the two claims are ontologically identical.
We might regard such an existence claim as empty, meaningful but untestable, or dependent on a broader metaphysical account. The mathematics of entropy cannot choose among those positions.
The question that remains is therefore more basic than how much information reality contains:
What makes a difference possible in the first place?
That opens a further question: can absolute nonexistence coherently be formulated as a state, when a space of possible states already supplies structure? Representing “nothing” as one entry in a model may introduce distinctions that absolute nonexistence was meant to exclude.
But a difficulty in representing nonexistence is not a proof that existence is necessary. Investigating that possibility requires a separate philosophical argument about representation, possibility and reality. The present argument establishes the narrower point: entropy alone cannot describe all differences, and the limits of observation cannot settle what exists beyond them.