Bubblin' up

There is a disturbing amount of people willing to compare floating point numbers directly for equality. For most intents and purposes this is harmless, but if one talks about comparing physical quantities it can be a problem. The fact that many people ignore is that real numbers can not be represented accurately in computers. Irrational numbers have infinite number of terms after the decimal points and whatever we do, it is necessary to truncate somewhere to fit a number in the finite memory of our computers.

Another more often acknowledged problem is the dynamic range of physical quantities. While the mass of cars, people or other everyday objects are in the comfortable zone of kilograms up to tonnes, the mass of stars are way beyond these magnitudes, and to make matters worse the mass of particles are on the other far end of the scale. It is obvious that some sort of normalization should be employed to bring all that dynamic range into manageable chunks. Naturally this is done by simply scaling the quantities and is not such a problem. Still it is necessary to be able to represent both large and small real numbers, so here the floating point numbers come in play. The structure of a floating point number is pretty simple – there is a fixed point number between zero and one which is multiplied by an exponent (for computers, conveniently, it is a power of 2):

Where m is the mantissa, and p is the power (sl. exponent).

It is pretty obvious that truncating the fixed point number (the mantissa) and the power of the exponent imposes some constraints on the representation of the numbers:

• The dynamic range of the numbers is finite because there is no way to represent any arbitrary integer for the exponent.
• The precision is finite because one can't represent a fixed point number with infinite number of terms after the decimal dot.

Beside disadvantages this representation has its strong sides, mainly convenient implementation and good dynamic range coupled with appropriate precision. It is noteworthy mentioning that while the relative precision is kept, the absolute valued distance between the representable numbers increases with increase of the exponent.

All that said I will go back to the original problem – how to check if floating point numbers are equal?
It should be obvious by now that the imprecise representation excludes just naïvely comparing them with the equality operator and while I am not proponent of the epsilon-delta formalism in mathematics here it is quite appropriate. The idea is to set an epsilon representing the maximum relative difference between the two numbers and use it as a threshold.

Going dynamic

The Fortran people seem to really like the static libraries, why I couldn't really fathom. The idea behind a shared object (or dynamic library in Windows) is quite simple — flexibility. Their explanation is that static libraries (basically an object file) provides all that you need for linking and doesn't cause the problems loaders do. This is not quite true and there are some problems though:

• If you link against a shared binary in the static library code, the linker has no way of incorporating it, thus creating references exactly like these you have in a dynamic library.
• When you want to change something in a statically linked code, you have to rebuild the whole thing.
• (Especially in Linux/Unix) The code is really shared(!), meaning a lot of applications use the same binary, changes in that binary propagate to all and additionally there is only one copy of each one binary unit.

Now dynamic linking is not without cost:

• A person should ensure binary compatibility of the library as to be able to change the implementation without rebuilding the user applications. This could be quite a bit of work, especially for people who don't have a good set of habits for doing it.
• You should ensure all the dependencies are in a place known to the loader.
• Resolving of symbols may be done by the runtime as it is in Linux/UNIX.
• The library interface and exports should be enforced, as in Linux/UNIX by default all symbols are exported, while in Windows none of them are.

All in all dynamic libraries require a bit more consistent and rigid design, while static libraries could be less fuss. In the long rung though, the advantages of dynamic linking are overwhelming and it is the better way, allowing you to be flexible about your code.

A Gaussian is a Gaussian

Interestingly a sum of Gaussian distributions is again Gaussian distribution. It didn't came as a big surprise to me, but my thesis adviser was truly amazed ... strange. Furthermore it holds true for both uncorrelated, and correlated statistics, and the rule for addition is impressively simple.

So let's say that we have two Gaussian distributions:

Then the sum distribution is simply:

Where for the parameters holds:

In the last formula rho is the correlation coefficient, with value defined as: