See also: Floating point and 902.1 follow-up

I wasted most of my work day trying to figure out why some hardware was not going to the proper frequency. In my case, it looked fine going to 902.0 mHz, but failed on various odd values such as 902.1 mHz, 902.3 mHz, etc. But, I was told, there was an internal “frequency sweep” function that went through all those frequencies and it worked fine.

I finally ended up looking at the difference between what our host system sent (“Go to frequency X”) and what the internal function was doing (“Scan from frequency X to frequency Y”).

Then I saw it.

The frequency was being represented in hertz as a large 32-bit value, such as 902000000 for 902 mHz, or 902100000 for 902.1 mHz. The host program was taking its 902.1 floating point value and converting it to a 32-bit integer by multiplying that by 1,000,000… which resulted it it sending 902099975… which was then fed into some formula and resulted in enough drift due to being slightly off that the end results was also off.

902099975 is not what I expected from “multiply 902.1 by 1,000,000”.

I keep forgetting how bad floating point is. Try this:

#include <stdio.h>
#include <stdlib.h>
int main()
{
   float f = 902.1;
   printf ("float 902.1  = %f\r\n", f);
   double d = 902.1;
   printf ("double 902.1 = %f\r\n", d);
   return EXIT_SUCCESS;
}

It prints:

float 902.1 = 902.099976
double 902.1 = 902.100000

A double precision floating point can correctly represent 902.1, but a single precision float cannot.

The Windows GUI was correctly showing “902.1”, though, probably because it was taking the actual value and rounding it to one decimal place. Thank you, GUI, for hiding the problem.

I guess now I have to go through and change all those floats to doubles so the user gets what the user wants.

Until next time…