Thermodynamics of computation and information distance

Abstract

Intuitively, the minimal information distance between z and y is the length of the shortest program for a universal computer to transform x into y and y into x. This measure will be shown to be, up to a logarithmic additive term, equal to the maximum of the conditional Kol- mogorov complexities E1(z,y) = max{K(y|z), K(z\y)}. Any reasonable distance to measure similarity of pictures should be an effectively approximate, symmetric, positive function of x and y satisfying a reasonable nor-malization condition and obeying the triangle inequality. It turns out that Fy is minimal up to an additive constant among all such distances. Hence it is a universal 'picture distance', which accounts for any effective similarity between pictures. A third information distance, based on the idea that one should aim for dissipationless computations, and hence for reversible ones, is given by the length E2(x,y) = KR{y\x) - KR(z\y) of the shortest reversible program that transforms x into y and y into x on a universal reversible computer. It is shown that also E2 = Ei, up to a logarithmic additive term. It is remarkable that three so differently motivated definitions turn out to define one and the same notion. Another information distance, E$, is obtained by minimizing the total amount of information flowing in and out during a reversible computation in which the program is not retained, in other words the number of extra bits (apart from z) that must be irreversibly supplied at the beginning, plus the number of garbage bits (apart from y) that must be irreversibly erased at the end of the computation to obtain a 'clean' y. This distance is within a logarithmic additive term of the sum of the conditional complexities, E3(z,y) = K(y\z)+K(z\y). Finally, using the physical theory of reversible computation, the simple difference K(x)- K(y) is shown to be an appropriate (universal, antisymmetric, and transitive) measure of the amount of thermodynamic work required to transform string x into string y by the most efficient process.