Floating point and 902.1 follow-up

Yesterday, I wrote a short bit about how I wasted a work day trying to figure out why we would tell our hardware to go to “902.1 Mhz” but it would not.

The root cause was a limitation in the floating point used in the C program I was working with. A float cannot represent every value, and it turns out values we were using were some of them. I showed a sample like this:

#include <stdio.h>
#include <stdlib.h>
 
int main()
{
   float f = 902.1;
   printf ("float 902.1  = %f\r\n", f);
 
   double d = 902.1;
   printf ("double 902.1 = %f\r\n", d);
 
   return EXIT_SUCCESS;
}

The float representation of 902.1 would display as 902.099976, and this caused a problem when we multiplied it by one million to turn it into hertz and send it to the hardware.

I played WHAC-A-MOLE trying to find all that places in the code that needed to change floats to doubles, and then played more WHAC-A-MOLE to update the user interface to change fields that use floats to use doubles…

Then, I realized we don’t actually care about precision past one decimal point. Here is the workaround I decided to go with, which means I can stop whacking moles.

float MHz;
uint32_t Hertz;
uint32_t Hertz2;
for (MHz=902.0; MHz < 902.9; MHz+=.1)
{
    Hertz = (MHz * 1000000);

    // Round to nearest 10000th of Hertz.
    Hertz2 = (((Hertz + 50000) / 100000)) * 100000;

    printf ("%f * 1000000 = %u -> %u\n", MHz, Hertz, Hertz2);
}

I kept the existing conversion (which would be off by quite a bit after being multiplied by one million) and then did a quick-and-dirty hack to round that Hertz value to the closest decimal point.

Here are the results, showing the MHz float, the converted Hertz, and the new rounded Hertz we actually use:

902.000000 * 1000000 = 902000000 -> 902000000
902.099976 * 1000000 = 902099975 -> 902100000
902.199951 * 1000000 = 902199951 -> 902200000
902.299927 * 1000000 = 902299926 -> 902300000
902.399902 * 1000000 = 902399902 -> 902400000
902.499878 * 1000000 = 902499877 -> 902500000
902.599854 * 1000000 = 902599853 -> 902600000
902.699829 * 1000000 = 902699829 -> 902700000
902.799805 * 1000000 = 902799804 -> 902800000
902.899780 * 1000000 = 902899780 -> 902900000

There. Much nicer. It’s not perfect, but it’s better.

Now I can get back to my real project.

Until next time…

One thought on “Floating point and 902.1 follow-up

  1. Pingback: Floating point and 902.1 | Sub-Etha Software

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.