Category Archives: C Programming

hertz and floats and doubles and integers

This post is an indirect follow-up to an earlier post I made about floating point numbers and the frequency 902.1 Mhz. If you check out that original posting, there was a follow-up to it as well.

My day job is maintaining software that runs solid-state microwave generators. As such, it deals with frequencies such as 902.1 MHz or 902100000 Hz. The Windows-based generator control software initially used 32-bit floating point values to represent frequencies in MHz, such as:

float start_frequency_mhz = 902.0;
float end_frequency_mhz = 928.0;

The user interface would round frequencies to the nearest 100,000 Hz, such as 902.1 MHz or 915.4 MHz. This allowed the limitations of a 32-bit floating point to not be a problem.

Limitations of 32-bit floating point

As discussed in my 902.1 posts referenced at the start of this post, certain values cannot be represented as a 32-bit floating point. 902.1 turns into 902.099976, for example:

float mhz = 902.1;
    
printf ("mhz = %f\n", mhz);

Comparing floating point values as one would compare integers may not work as expected:

float mhz = 902.1;
if (mhz == 902.1)
{
    printf ("mhz IS 902.1\n");
}
else
{
    printf ("mhz is NOT 902.1\n");
}

Running that on a 32-bit compiler will print the message:

mhz is NOT 902.1

But if you change the variable type from float to double (64-bit floating point), that gives it enough precision for this to work:

double mhz = 902.1; // changed from float to double
printf ("mhz = %f\n", mhz);
if (mhz == 902.1)
{
    printf ("mhz IS 902.1\n");
}
else
{
    printf ("mhz is NOT 902.1\n");
}

Running that will print:

mhz IS 902.1

So, yeah. Folks who don’t do much with floating point may find this surprising. I sure did when I first learned it.

Floating point is great if you have it…

Few of the embedded systems I have programmed on were using CPUs with hardware floating point. Instead, any floating point calculations were done using a software library. Bringing in floating point library code made the program grow larger. This is not optimal when dealing with systems with limited program memory. It also made the code slower, which was not optimal on systems with slower processors.

While the Windows program, running on a modern 64-bit CPU, could use floating point easily, the embedded firmware, running on 16-bit CPUs, was written to use integers for smaller code size and faster operation. (No, of course this is not the case on every system. This is just an example you are reading on some random blog on the internet…)

Instead of trying to represent a frequency as MHz in floating point, the firmware would use a 32-bit integer as hz.

float f_mhz = 902.1; // on 64-bit Windows system
uint32_t hz = 902100000; // on 16-bit firmware

When I explain this to someone who is unfamiliar of “the problems with floating point,” I show them quick examples like this. For fun (everybody needs a hobby), tonight I wrote up a longer example that prints out the frequencies of 902 MHz to 928 MHz (matching the L-Band RF frequency used by the microwave systems at my work) in steps of .1 MHz.

The example then converts that frequency value to a float and a double, then converts each back to a 32-bit unsigned integer and prints them all out.

Some comparisons are done (float compared to double, and float converted to 32-bit integer compared to double converted to 32-bit integer) with the output marking values that differ as floating points (“F”) and/or unsigned integers (“U”).

#include <inttypes.h>   // for PRIu32
#include <stdio.h>      // for printf()
#include <stdlib.h>     // for EXIT_SUCCESS
#define MHZ_TO_HZ(f)        ((f) * 1000000UL)
#define HZ_TO_MHZ(f)        ((f) / 1000000.0)
#define FREQUENCY_START_HZ  902000000
#define FREQUENCY_END_HZ    928000000
#define FREQUENCY_STEPS_HZ     100000
int main()
{
    printf ("mhz    float       uint32_t   double      uint32_t   ==?\n"
            "-----  ----------  ---------  ----------  ---------  ---\n"
           );
           
    for (uint32_t hz=FREQUENCY_START_HZ; hz <= FREQUENCY_END_HZ; 
         hz+=FREQUENCY_STEPS_HZ)
    {
        float       mhz_as_float = HZ_TO_MHZ(hz);
        double      mhz_as_double = HZ_TO_MHZ(hz);
        
        uint32_t    mhz_float_as_u32 = MHZ_TO_HZ(mhz_as_float);
        uint32_t    mhz_double_as_u32 = MHZ_TO_HZ(mhz_as_double);
        
        printf ("%.1f  %f  %9" PRIu32 "  %f  %9" PRIu32 "  ",
                HZ_TO_MHZ(hz),
                mhz_as_float, mhz_float_as_u32,
                mhz_as_double, mhz_double_as_u32
               );
    
        // Do floating points match?
        if (mhz_as_float != mhz_as_double)
        {
            printf ("F");
        }
        else
        {
            printf (" ");
        }
    
        // Do converted U32s match?
        if (mhz_float_as_u32 != mhz_double_as_u32)
        {
            printf (" U");
        }
    
        printf ("\n");
    }
    return EXIT_SUCCESS;
}

And here is the output:

mhz    float       uint32_t   double      uint32_t   ==?
----- ---------- --------- ---------- --------- ---
902.0 902.000000 902000000 902.000000 902000000
902.1 902.099976 902099968 902.100000 902100000 F U
902.2 902.200012 902200000 902.200000 902200000 F
902.3 902.299988 902299968 902.300000 902300000 F U
902.4 902.400024 902400000 902.400000 902400000 F
902.5 902.500000 902499968 902.500000 902500000 U
902.6 902.599976 902600000 902.600000 902600000 F
902.7 902.700012 902700032 902.700000 902700000 F U
902.8 902.799988 902800000 902.800000 902800000 F
902.9 902.900024 902900032 902.900000 902900000 F U
903.0 903.000000 903000000 903.000000 903000000
903.1 903.099976 903099968 903.100000 903100000 F U
903.2 903.200012 903200000 903.200000 903200000 F
903.3 903.299988 903299968 903.300000 903300000 F U
903.4 903.400024 903400000 903.400000 903400000 F
903.5 903.500000 903500032 903.500000 903500000 U
903.6 903.599976 903600000 903.600000 903600000 F
903.7 903.700012 903700032 903.700000 903700000 F U
903.8 903.799988 903800000 903.800000 903800000 F
903.9 903.900024 903900032 903.900000 903900000 F U
904.0 904.000000 904000000 904.000000 904000000
904.1 904.099976 904099968 904.100000 904100000 F U
904.2 904.200012 904200000 904.200000 904200000 F
904.3 904.299988 904299968 904.300000 904300000 F U
904.4 904.400024 904400000 904.400000 904400000 F
904.5 904.500000 904499968 904.500000 904500000 U
904.6 904.599976 904600000 904.600000 904600000 F
904.7 904.700012 904700032 904.700000 904700000 F U
904.8 904.799988 904800000 904.800000 904800000 F
904.9 904.900024 904900032 904.900000 904900000 F U
905.0 905.000000 905000000 905.000000 905000000
905.1 905.099976 905099968 905.100000 905100000 F U
905.2 905.200012 905200000 905.200000 905200000 F
905.3 905.299988 905299968 905.300000 905300000 F U
905.4 905.400024 905400000 905.400000 905400000 F
905.5 905.500000 905500032 905.500000 905500000 U
905.6 905.599976 905600000 905.600000 905600000 F
905.7 905.700012 905700032 905.700000 905700000 F U
905.8 905.799988 905800000 905.800000 905800000 F
905.9 905.900024 905900032 905.900000 905900000 F U
906.0 906.000000 906000000 906.000000 906000000
906.1 906.099976 906099968 906.100000 906100000 F U
906.2 906.200012 906200000 906.200000 906200000 F
906.3 906.299988 906299968 906.300000 906300000 F U
906.4 906.400024 906400000 906.400000 906400000 F
906.5 906.500000 906499968 906.500000 906500000 U
906.6 906.599976 906600000 906.600000 906600000 F
906.7 906.700012 906700032 906.700000 906700000 F U
906.8 906.799988 906800000 906.800000 906800000 F
906.9 906.900024 906900032 906.900000 906900000 F U
907.0 907.000000 907000000 907.000000 907000000
907.1 907.099976 907099968 907.100000 907100000 F U
907.2 907.200012 907200000 907.200000 907200000 F
907.3 907.299988 907299968 907.300000 907300000 F U
907.4 907.400024 907400000 907.400000 907400000 F
907.5 907.500000 907500032 907.500000 907500000 U
907.6 907.599976 907600000 907.600000 907600000 F
907.7 907.700012 907700032 907.700000 907700000 F U
907.8 907.799988 907800000 907.800000 907800000 F
907.9 907.900024 907900032 907.900000 907900000 F U
908.0 908.000000 908000000 908.000000 908000000
908.1 908.099976 908099968 908.100000 908100000 F U
908.2 908.200012 908200000 908.200000 908200000 F
908.3 908.299988 908299968 908.300000 908300000 F U
908.4 908.400024 908400000 908.400000 908400000 F
908.5 908.500000 908499968 908.500000 908500000 U
908.6 908.599976 908600000 908.600000 908600000 F
908.7 908.700012 908700032 908.700000 908700000 F U
908.8 908.799988 908800000 908.800000 908800000 F
908.9 908.900024 908900032 908.900000 908900000 F U
909.0 909.000000 909000000 909.000000 909000000
909.1 909.099976 909099968 909.100000 909100000 F U
909.2 909.200012 909200000 909.200000 909200000 F
909.3 909.299988 909299968 909.300000 909300000 F U
909.4 909.400024 909400000 909.400000 909400000 F
909.5 909.500000 909500032 909.500000 909500000 U
909.6 909.599976 909600000 909.600000 909600000 F
909.7 909.700012 909700032 909.700000 909700000 F U
909.8 909.799988 909800000 909.800000 909800000 F
909.9 909.900024 909900032 909.900000 909900000 F U
910.0 910.000000 910000000 910.000000 910000000
910.1 910.099976 910099968 910.100000 910100000 F U
910.2 910.200012 910200000 910.200000 910200000 F
910.3 910.299988 910299968 910.300000 910300000 F U
910.4 910.400024 910400000 910.400000 910400000 F
910.5 910.500000 910499968 910.500000 910500000 U
910.6 910.599976 910600000 910.600000 910600000 F
910.7 910.700012 910700032 910.700000 910700000 F U
910.8 910.799988 910800000 910.800000 910800000 F
910.9 910.900024 910900032 910.900000 910900000 F U
911.0 911.000000 911000000 911.000000 911000000
911.1 911.099976 911099968 911.100000 911100000 F U
911.2 911.200012 911200000 911.200000 911200000 F
911.3 911.299988 911299968 911.300000 911300000 F U
911.4 911.400024 911400000 911.400000 911400000 F
911.5 911.500000 911500032 911.500000 911500000 U
911.6 911.599976 911600000 911.600000 911600000 F
911.7 911.700012 911700032 911.700000 911700000 F U
911.8 911.799988 911800000 911.800000 911800000 F
911.9 911.900024 911900032 911.900000 911900000 F U
912.0 912.000000 912000000 912.000000 912000000
912.1 912.099976 912099968 912.100000 912100000 F U
912.2 912.200012 912200000 912.200000 912200000 F
912.3 912.299988 912299968 912.300000 912300000 F U
912.4 912.400024 912400000 912.400000 912400000 F
912.5 912.500000 912499968 912.500000 912500000 U
912.6 912.599976 912600000 912.600000 912600000 F
912.7 912.700012 912700032 912.700000 912700000 F U
912.8 912.799988 912800000 912.800000 912800000 F
912.9 912.900024 912900032 912.900000 912900000 F U
913.0 913.000000 913000000 913.000000 913000000
913.1 913.099976 913099968 913.100000 913100000 F U
913.2 913.200012 913200000 913.200000 913200000 F
913.3 913.299988 913299968 913.300000 913300000 F U
913.4 913.400024 913400000 913.400000 913400000 F
913.5 913.500000 913500032 913.500000 913500000 U
913.6 913.599976 913600000 913.600000 913600000 F
913.7 913.700012 913700032 913.700000 913700000 F U
913.8 913.799988 913800000 913.800000 913800000 F
913.9 913.900024 913900032 913.900000 913900000 F U
914.0 914.000000 914000000 914.000000 914000000
914.1 914.099976 914099968 914.100000 914100000 F U
914.2 914.200012 914200000 914.200000 914200000 F
914.3 914.299988 914299968 914.300000 914300000 F U
914.4 914.400024 914400000 914.400000 914400000 F
914.5 914.500000 914499968 914.500000 914500000 U
914.6 914.599976 914600000 914.600000 914600000 F
914.7 914.700012 914700032 914.700000 914700000 F U
914.8 914.799988 914800000 914.800000 914800000 F
914.9 914.900024 914900032 914.900000 914900000 F U
915.0 915.000000 915000000 915.000000 915000000
915.1 915.099976 915099968 915.100000 915100000 F U
915.2 915.200012 915200000 915.200000 915200000 F
915.3 915.299988 915299968 915.300000 915300000 F U
915.4 915.400024 915400000 915.400000 915400000 F
915.5 915.500000 915500032 915.500000 915500000 U
915.6 915.599976 915600000 915.600000 915600000 F
915.7 915.700012 915700032 915.700000 915700000 F U
915.8 915.799988 915800000 915.800000 915800000 F
915.9 915.900024 915900032 915.900000 915900000 F U
916.0 916.000000 916000000 916.000000 916000000
916.1 916.099976 916099968 916.100000 916100000 F U
916.2 916.200012 916200000 916.200000 916200000 F
916.3 916.299988 916299968 916.300000 916300000 F U
916.4 916.400024 916400000 916.400000 916400000 F
916.5 916.500000 916499968 916.500000 916500000 U
916.6 916.599976 916600000 916.600000 916600000 F
916.7 916.700012 916700032 916.700000 916700000 F U
916.8 916.799988 916800000 916.800000 916800000 F
916.9 916.900024 916900032 916.900000 916900000 F U
917.0 917.000000 917000000 917.000000 917000000
917.1 917.099976 917099968 917.100000 917100000 F U
917.2 917.200012 917200000 917.200000 917200000 F
917.3 917.299988 917299968 917.300000 917300000 F U
917.4 917.400024 917400000 917.400000 917400000 F
917.5 917.500000 917500032 917.500000 917500000 U
917.6 917.599976 917600000 917.600000 917600000 F
917.7 917.700012 917700032 917.700000 917700000 F U
917.8 917.799988 917800000 917.800000 917800000 F
917.9 917.900024 917900032 917.900000 917900000 F U
918.0 918.000000 918000000 918.000000 918000000
918.1 918.099976 918099968 918.100000 918100000 F U
918.2 918.200012 918200000 918.200000 918200000 F
918.3 918.299988 918299968 918.300000 918300000 F U
918.4 918.400024 918400000 918.400000 918400000 F
918.5 918.500000 918499968 918.500000 918500000 U
918.6 918.599976 918600000 918.600000 918600000 F
918.7 918.700012 918700032 918.700000 918700000 F U
918.8 918.799988 918800000 918.800000 918800000 F
918.9 918.900024 918900032 918.900000 918900000 F U
919.0 919.000000 919000000 919.000000 919000000
919.1 919.099976 919099968 919.100000 919100000 F U
919.2 919.200012 919200000 919.200000 919200000 F
919.3 919.299988 919299968 919.300000 919300000 F U
919.4 919.400024 919400000 919.400000 919400000 F
919.5 919.500000 919500032 919.500000 919500000 U
919.6 919.599976 919600000 919.600000 919600000 F
919.7 919.700012 919700032 919.700000 919700000 F U
919.8 919.799988 919800000 919.800000 919800000 F
919.9 919.900024 919900032 919.900000 919900000 F U
920.0 920.000000 920000000 920.000000 920000000
920.1 920.099976 920099968 920.100000 920100000 F U
920.2 920.200012 920200000 920.200000 920200000 F
920.3 920.299988 920299968 920.300000 920300000 F U
920.4 920.400024 920400000 920.400000 920400000 F
920.5 920.500000 920499968 920.500000 920500000 U
920.6 920.599976 920600000 920.600000 920600000 F
920.7 920.700012 920700032 920.700000 920700000 F U
920.8 920.799988 920800000 920.800000 920800000 F
920.9 920.900024 920900032 920.900000 920900000 F U
921.0 921.000000 921000000 921.000000 921000000
921.1 921.099976 921099968 921.100000 921100000 F U
921.2 921.200012 921200000 921.200000 921200000 F
921.3 921.299988 921299968 921.300000 921300000 F U
921.4 921.400024 921400000 921.400000 921400000 F
921.5 921.500000 921500032 921.500000 921500000 U
921.6 921.599976 921600000 921.600000 921600000 F
921.7 921.700012 921700032 921.700000 921700000 F U
921.8 921.799988 921800000 921.800000 921800000 F
921.9 921.900024 921900032 921.900000 921900000 F U
922.0 922.000000 922000000 922.000000 922000000
922.1 922.099976 922099968 922.100000 922100000 F U
922.2 922.200012 922200000 922.200000 922200000 F
922.3 922.299988 922299968 922.300000 922300000 F U
922.4 922.400024 922400000 922.400000 922400000 F
922.5 922.500000 922499968 922.500000 922500000 U
922.6 922.599976 922600000 922.600000 922600000 F
922.7 922.700012 922700032 922.700000 922700000 F U
922.8 922.799988 922800000 922.800000 922800000 F
922.9 922.900024 922900032 922.900000 922900000 F U
923.0 923.000000 923000000 923.000000 923000000
923.1 923.099976 923099968 923.100000 923100000 F U
923.2 923.200012 923200000 923.200000 923200000 F
923.3 923.299988 923299968 923.300000 923300000 F U
923.4 923.400024 923400000 923.400000 923400000 F
923.5 923.500000 923500032 923.500000 923500000 U
923.6 923.599976 923600000 923.600000 923600000 F
923.7 923.700012 923700032 923.700000 923700000 F U
923.8 923.799988 923800000 923.800000 923800000 F
923.9 923.900024 923900032 923.900000 923900000 F U
924.0 924.000000 924000000 924.000000 924000000
924.1 924.099976 924099968 924.100000 924100000 F U
924.2 924.200012 924200000 924.200000 924200000 F
924.3 924.299988 924299968 924.300000 924300000 F U
924.4 924.400024 924400000 924.400000 924400000 F
924.5 924.500000 924499968 924.500000 924500000 U
924.6 924.599976 924600000 924.600000 924600000 F
924.7 924.700012 924700032 924.700000 924700000 F U
924.8 924.799988 924800000 924.800000 924800000 F
924.9 924.900024 924900032 924.900000 924900000 F U
925.0 925.000000 925000000 925.000000 925000000
925.1 925.099976 925099968 925.100000 925100000 F U
925.2 925.200012 925200000 925.200000 925200000 F
925.3 925.299988 925299968 925.300000 925300000 F U
925.4 925.400024 925400000 925.400000 925400000 F
925.5 925.500000 925500032 925.500000 925500000 U
925.6 925.599976 925600000 925.600000 925600000 F
925.7 925.700012 925700032 925.700000 925700000 F U
925.8 925.799988 925800000 925.800000 925800000 F
925.9 925.900024 925900032 925.900000 925900000 F U
926.0 926.000000 926000000 926.000000 926000000
926.1 926.099976 926099968 926.100000 926100000 F U
926.2 926.200012 926200000 926.200000 926200000 F
926.3 926.299988 926299968 926.300000 926300000 F U
926.4 926.400024 926400000 926.400000 926400000 F
926.5 926.500000 926499968 926.500000 926500000 U
926.6 926.599976 926600000 926.600000 926600000 F
926.7 926.700012 926700032 926.700000 926700000 F U
926.8 926.799988 926800000 926.800000 926800000 F
926.9 926.900024 926900032 926.900000 926900000 F U
927.0 927.000000 927000000 927.000000 927000000
927.1 927.099976 927099968 927.100000 927100000 F U
927.2 927.200012 927200000 927.200000 927200000 F
927.3 927.299988 927299968 927.300000 927300000 F U
927.4 927.400024 927400000 927.400000 927400000 F
927.5 927.500000 927500032 927.500000 927500000 U
927.6 927.599976 927600000 927.600000 927600000 F
927.7 927.700012 927700032 927.700000 927700000 F U
927.8 927.799988 927800000 927.800000 927800000 F
927.9 927.900024 927900032 927.900000 927900000 F U
928.0 928.000000 928000000 928.000000 928000000

In the last column, F means that the float (32-bit) floating point and the double (64-bit) floating point do not match. U means the float (32-bit) converted to 32-bit integer does not match the double (64-bit) converted to integer.

All this to say, as I am beginning to experiment with new routines dealing with frequencies, I am creating all of them to handle those frequencies as Hz using 32-bit unsigned integers. This will allow these routines to also work well on embedded systems that lack floating point hardware, as well as ensure any weird “902.1” issues won’t cause someone else to waste hours of their time dealing with some math that is slightly off. . .

To be continued…

SPI and interrupts and PIC24

How did this ever work: SPI and interrupts

Over the years, I have posted about things that fall into the category of “how did this ever work?” I think I need to make an actual WordPress Category on this blog for these items, and I will do so with this post.

Every embedded programming job I have had came with a bunch of pre-existing code that “worked.” There are always new features to be added, and bugs to be fixed, but overall things were already at an “it works, ship it” state.

And every embedded programming job I have had came with code that “works” but decided to not work later, leading me down a rabbit hole of code exploring to understand what it did and why it was suddenly did not.

This was often followed by finding a problem in the code that made me wonder how it ever worked in the first place.

SPI and interrupts on the PIC24 processor

This post is not specifically about SPI and interrupts on the PIC24 processor. I just happened to run into this particular issue in that environment.

On the seven hardware systems I maintain code for, there are various devices such as ADC, DAC, EEPROM, attenuators, phase shifters, frequency synthesizers, etc. that are hooked up to the CPU using I/O, I2C or SPI bus.

In the code main loop, the firmware reads or writes to these devices as needed, such as sampling RF detectors or making adjustments to power control attenuators.

Recently, we were testing a pulse modulation mode (PWM) where the RF signal is turning on and off. When on, the power is measured (detector read) inside an interrupt that was tied to the PWM signal. This code has been in place since before I joined 6+ years ago and, other than some bugs and enhancements along the way, “it works.”

However, this SPI code ended up being the cause of some odd problems that initially looked like I2C communications were failing. Our Windows-based host program that communicated over I2C would start having communication faults with the boards running firmware.

After a few days of Whac-A-Mole(tm) trying to rework things that could be problematic, I finally learned the root cause of the issue:

SPI and interrupts

The main loop was doing SPI operations (in our case, using the CCS PCD compiler and its API calls such as spi_init, spi_xfer, etc.). One of those SPI operations was for reading RF detectors. When PWM mode was enabled, the an interrupt service routine would be enabled and the main loop RF detectors reads would shut off and we would begin reading the RF detectors inside the interrupt routine as each pulse happened.

When a SPI operation happens, the code may need to reconfigure the SPI hardware between MODE 0 and MODE 1 transfers, for example, or to change the baud rate.

If the main code configured the SPI hardware for a specific mode and baud then began doing some SPI transfers and the pulse occurred, code would jump into the interrupt routine which might have to reconfigure the hardware differently and then read the detectors. Upon completion, execution returned back to the main loop and the SPI hardware could be in the wrong mode to complete that transaction.

Bad things would happen.

But why now?

The original code allowed pulsing at 100 Hz to 1000 Hz. This meant than 100 to 1000 times a second there was a chance that the SPI hardware could get messed up by the interrupt code. Yet, if we saw this happen, it was infrequent enough to be noticed.

At some point, I modified the code to support 10,000 Hz. This meant there was now 10,000 times a second that the problem could happen.

Over the years we had seen some issues at 10,000 Hz, including what we thought was RF interference causing communication problems (and solved through a hardware modification). Since this mode was rarely used, the true depth of the issue was never experienced.

“It works.”

Simple solutions…

A very simple solution was added which appears to have eliminated this issue completely. Some functions where added that could be called from the main loop code around any access to the SPI hardware. Here is a pseudo-code example of what I added:

volatile int g_spi_lock = 1;

void spi_claim (void)
{
interrupts_disable (INT_EXT0); // Disable PWM interrupt.

g_spi_lock = 1; // Flag SPI in use.

interrupts_enable (INT_EXT0); // Re-enable PWM interrupt.
}

void spi_release (void)
{
interrupts_disable (INT_EXT0); // Disable PWM interrupt.

g_spi_lock = 0; // Flag SPI available.

interrupts_enable (INT_EXT0); // Re-enable PWM interrupt.
}

Then, inside the interrupt service routine, that code could simply check that flag and skip reading at that pulse, knowing it would just catch the next one (I said this was the simple fix, not the best fix):

void pwm_isr (void)
{
if (0 == g_spi_lock) // SPI is free.
{
// Do SPI stuff…
}
}

Then all I had to do was add the claim and release around all the non-ISR SPI code…

spi_claim ();

output_high (chip_select_pin);
spi_init (xxx);
msb = spi_xfer (xxx);
lab = spi_xfer (xxx);
output_low (chip_select_pin);

spi_release ();

“And just like that,” the problems all went away. Many of the problems we had been unaware of since some — like doing an EEPROM read — were typically not done while a system was running with RF enabled and pulsing active. Once I learned what the problem was, I could recreate it within seconds just by doing various things that caused SPI activity while in PWM mode. ;-)

That was my quick-and-dirty first, but you may know a better one. If you feel like sharing it in a comment, please do so.

But the question remains…

How did this ever work?

Once the bug was understood, recreating it was simple. Yet, this code has been in use for years and “it worked.”

My next task is going to be reviewing all the other firmware projects and seeing if any of them do any type of SPI or I2C stuff from an interrupt that could mess up similar code happening from the main loop.

And I bet I find some.

In code that “just works” and has been working for years.

Until next time…

Coding standards and talking myself into snake_case…

Obviously, things done “today” can be quite different than how things were done 30 years ago, especially in regards to computers. Is it strange that out of all the places I’ve worked where I wrote code that only one of them had an actual “coding standard” we had to follow? (And that was at a giant mega-corporation.) All the others seemed to have folks who just carried on how things were, for the most part, or were given the freedom to code as they wanted as long as they followed some specific system (such as “Clean Code” at a startup I worked at).

My day job has been undertaking an official coding standard. I started by documenting how things were done in the millions of lines of code we maintain. Over my years here, I have introduced some things from the mega-corporation’s standards into our code base, such as prefixing static variables with “s_” or globals with “g_” or similar.

But why reinvent the wheel? I thought it would make a lot more sense to find an existing standard that applied to embedded C programming and just adopt it. That led me to this:

https://barrgroup.com/embedded-systems/books/embedded-c-coding-standard1

This document clearly states the “why” for any requirement it has, and I have learned a few things. Unlike most standards that are mostly cosmetic (how to name functions or variables, how wide is a tab, etc.), this one focuses on bug detection, bug reduction and making code easier to review.

Which brings me to snake case…

https://en.wikipedia.org/wiki/Snake_case

Most places I have worked use camelCase, where all words are ran together (no spaces) with the first letter in lowercase, then all subsequent words starting with uppercase. itWorksButCanBeHardToRead.

The mega-corp standard I worked under would refer to “camel case starting with an uppercase letter,” which I have since learned is known as PascalCase. ItCanHaveTheSameProblem when the eyes have to un-smush all the letters.

snake_case is using lowercase words separated by underlines. A variant, SCREAMING_SNAKE_CASE uses all uppercase. (I never knew the name of that one, but most places I worked use that for #defines and macros and such.)

…and I expect someone will comment with even more naming conventions than I have encountered.

Do. Or do not. There is no try.

Beyond “how stuff looks,” some suggestions actually matter. Something I only started doing in recent times is when you put the constant on the left side of a comparison like this:

if (42 == answer)
{
    // Ultimate!
}

I saw this for the first time about a decade ago. All the code from a non-USA office we interacted with had conditions written “backwards” like that. I took an instant dislike to it since it did not read naturally. I mean who says “if 225 pounds or more is your weight, you can’t ride this scooter”?

However, all it takes is a good argument and I can flip on a dime. This flip was caused by finding multiple bugs in code where the programmer accidentally forgot one of the equals:

if (answer = 42)
{
// Ultimate?
}

If you use a modern compiler and have compiler warnings enabled, it should catch that unintended variable assignment. But, if you are using a “C-Like” embedded compiler that is missing a lot of modern features, it probably won’t. Or, if even you are running GCC with defaults:

https://onlinegdb.com/pv5Yz24t2

And thus, I broke my habit of making code that “reads more naturally” and started writing comparisons backwards since the compiler will fail if you do “42 = answer”.

This week, I learned they refer to this as Yoda conditions. Of course they do.

https://en.wikipedia.org/wiki/Yoda_conditions

Snakes. Why did it have to be snakes?

This week I read one sentence that may make me get away from camelCase and PascalCase and go retro with my naming convention. It simply had to do with readability.

is_adc_in_error is very easy to read. I’d say much easer than IsAdcInError which looks like some gibberish and requires you to focus on it. If anyone else is going to review the code, being able to “parse” it easier is a benefit.

I am almost convinced, even if I personally dislike it.

Give me some reasons to stick with camelCase or PascalCase, and let’s see if any of them are more than “’cause it looks purtier.” (For what its worth, I’ve seen camelCase for global functions, and CamelCase for static functions.)

Awaiting your responses…

Modern C and initializing an array

I was today years old when I learned something that modern C can do. Thank you, Bing Copilot, for offering this up and learnin’ me somethin’ new.

At my day job I am porting various WIZnet (and hardware TCP/IP ethernet chip) services over to a new circuit board. We are using the W5500 chip, and making use of the provided I/O library from WIZnet:

Wiznet/ioLibrary_Driver: ioLibrary_Driver can be used for the application design of WIZnet TCP/IP chips as W5500, W5300, W5200, W5100, W5100S, W6100, W6300.

For this project, I have gotten basic Ethernet TCP running, then moved on to port their HTTP server code, followed by their SNMP server. While I may have encountered SNMP during my 1990s employment at Microware when I taught an OS-9 networking course, I have not touched it since then so this was “all new” again to me.

Part of configuring SNMP was an array of structures like this:

dataEntryType snmpData[] =
{
    // System MIB
    // SysDescr Entry
    {
        8, {0x2b, 6, 1, 2, 1, 1, 1, 0},
        SNMPDTYPE_OCTET_STRING, 30, {"Description String"},
        NULL, NULL
        },
    // SysObjectID Entry
    {
        8, {0x2b, 6, 1, 2, 1, 1, 2, 0},
        SNMPDTYPE_OBJ_ID, 8, {"\x2b\x06\x01\x02\x01\x01\x02\x00"},
        NULL, NULL
        },

…and so on. You can find this file here:

ioLibrary_Driver/Internet/SNMP/snmp_custom.c at master · Wiznet/ioLibrary_Driver

When I got to the part where I was adding the ability to send an “SNMP trap” (kind of a push notification to a specific IP address), I saw that it would duplicate the values in this array:

void sendTrap_sysObjectID(void) {
    // Create a dataEntryType for sysObjectID
    dataEntryType sysObjectEntry = {
        8, {0x2b, 6, 1, 2, 1, 1, 2, 0},
        SNMPDTYPE_OBJ_ID, 8, {"\x2b\x06\x01\x02\x01\x01\x02\x00"},
        NULL, NULL
    };
    // Send the trap
    snmp_sendTrap( ...params...);
}

Above, the “sysObjectEntry” array is a duplicate of the entry in my definition table.

This would mean keeping two parts of the code in sync, which is a terrible idea since it creates extra places for error, and also doubles the work. The A.I. suggested using the entry in the array, like this:

snmp_sendTrap(
  managerIP,
  agentIP,
  (int8_t*)"public",
  snmpData[1], // SysObjectID or your custom OID
  6,           // enterpriseSpecific
  1,           // specific trap ID
  1,           // one variable binding
  snmpData[10].oid, SNMPDTYPE_INTEGER, (uint8_t[]){1}, 1 // HighTempState = 1
);

Using “snmpData[1]” is more better, since now I can just change the definition table instead of multiple places hard-coding that information.

BUT, how do I know which entry is [1] versus [7]? I’d have to ensure the table entries stayed in order, then I could use a #define as an index, like this example:

typedef struct
{
	int x,y,w,h;
} BoxStruct;
// Declare an array of boxes, the old fashioned way.
BoxStruct box[] = // Initialize four boxes
{
	{ 0, 0, 100, 10},
	{ 10, 10, 90, 10 },
	{ 20, 20, 80, 10 },
	{ 30, 30, 70, 10 }
};
// Declare an array of boxes, the new fangled way.
#define BOX1    0
#define BOX2    1
#define BOX3    2
#define BOX4    3

That would let me get the data for box[BOX1] or box[BOX4]. Easy.

But if this was a long list of entries, like my SNMP was, and later I added something in between, I’d have to update the array as well as make sure this #define table is updated. Again, more place for error and more work.

The first thought was to declare the array as fixed size and initialize it like this:

// Declare an array of boxes, the new fangled way.
#define BOX1    0
#define BOX2    1
#define BOX3    2
#define BOX4    3
#define BOX_MAX 4
BoxStruct box[BOX_MAX];
box[BOX1] = { 1,1,1,1 };
box[BOX2] = { 2,2,2,2 };
box[BOX3] = { 3,3,3,3 };
box[BOX4] = { 4,4,4,4 };

That makes it easy and obvious, BUT you cannot have those initializes with global variables. You can only initialize that way from a function. This is fine if you want to declare your global array, then initialize like…

void initBoxes (void)
{
    box[BOX1] = { 1,1,1,1 };
    box[BOX2] = { 2,2,2,2 };
    box[BOX3] = { 3,3,3,3 };
    box[BOX4] = { 4,4,4,4 };
}

And that is the approach I would probably take on my “C-like” embedded compilers that do not support all of modern C.

However, the A.I. showed me something I had not seen before. Initializing the array like this:

// Initialize by element number, in order.
BoxStruct box[BOX_MAX] =
{
	[BOX1] = { 0, 0, 100, 10},
	[BOX2] = { 10, 10, 90, 10 },
	[BOX3] = { 20, 20, 80, 10 },
	[BOX4] = { 30, 30, 70, 10 }
};

Huh? I did a quick check on the Online GDB compiler, and that was valid. It even lets you initialize out of order:

// Initialize by element number, out of order.
BoxStruct box[BOX_MAX] =
{
	[BOX3] = { 20, 20, 80, 10 },
	[BOX1] = { 0, 0, 100, 10},
	[BOX4] = { 30, 30, 70, 10 },
	[BOX2] = { 10, 10, 90, 10 }
};

By doing it that way, I could “see” the label match the data, regardless of whatever the number the label was set to. And, if I messed up the #defines later (duplicate value or whatever), a “good compiler” with warnings enabled should alert me of that (at least, GCC does).

For my specific use, this is a great solution, and it works in whatever compiler the Microchip MPLAB X IDE is using for PIC24 processors.

Here is my test code:

/******************************************************************************
Welcome to GDB Online.
GDB online is an online compiler and debugger tool for C, C++, Python, Java, PHP, Ruby, Perl,
C#, OCaml, VB, Swift, Pascal, Fortran, Haskell, Objective-C, Assembly, HTML, CSS, JS, SQLite, Prolog.
Code, Compile, Run and Debug online from anywhere in world.
*******************************************************************************/
#include <stdio.h>
/*---------------------------------------------------------------------------*/
// Typedef
/*---------------------------------------------------------------------------*/
typedef struct
{
	int x,y,w,h;
} BoxStruct;
/*---------------------------------------------------------------------------*/
// Globals
/*---------------------------------------------------------------------------*/
// Declare an array of boxes, the old fashioned way.
BoxStruct foo[] = // Initialize four boxes
{
	{ 0, 0, 100, 10},
	{ 10, 10, 90, 10 },
	{ 20, 20, 80, 10 },
	{ 30, 30, 70, 10 }
};
// Declare an array of boxes, the new fangled way.
#define BOX1    0
#define BOX2    1
#define BOX3    2
#define BOX4    3
#define BOX_MAX 4
// Initialize by element number, in order.
BoxStruct foo2[BOX_MAX] =
{
	[BOX1] = { 0, 0, 100, 10},
	[BOX2] = { 10, 10, 90, 10 },
	[BOX3] = { 20, 20, 80, 10 },
	[BOX4] = { 30, 30, 70, 10 }
};
// Initialize by element number, out of order.
BoxStruct foo3[BOX_MAX] =
{
	[BOX3] = { 20, 20, 80, 10 },
	[BOX1] = { 0, 0, 100, 10},
	[BOX4] = { 30, 30, 70, 10 },
	[BOX2] = { 10, 10, 90, 10 }
};
/*---------------------------------------------------------------------------*/
// Prototypes
/*---------------------------------------------------------------------------*/
void ShowBox (BoxStruct box);
/*---------------------------------------------------------------------------*/
// Functions
/*---------------------------------------------------------------------------*/
void ShowBox (BoxStruct box)
{
	printf ("x:%d y:%d w:%d h%d\r\n", box.x, box.y, box.w, box.h);
}
int main()
{
	printf ("---foo---\r\n");
	for (int idx=0; idx<4; idx++)
	{
		ShowBox (foo[idx]);
	}
	printf ("---foo2---\r\n");
	for (int idx=0; idx<4; idx++)
	{
		ShowBox (foo2[idx]);
	}
	printf ("---foo3---\r\n");
	for (int idx=0; idx<4; idx++)
	{
		ShowBox (foo3[idx]);
	}
	return 0;
}
/*---------------------------------------------------------------------------*/
// End of main.c

You can mess with it online here:

https://onlinegdb.com/G73hxWSQuN

I wonder what other stuff has been added to C over the years that I do not know about…

Until next time…

Generating C functions and prototypes using macros – part 2

See Also: part 1 and part 2.

In the previous installment, I discussed how lazy I am and shared my general dislike of doing things manually when they can be automated.

So let’s automate…

If you program in C, you are likely familiar with using #define to set a value you can use elsewhere in the code:

#define NUMBER_OF_GUESSES 10

int answer = RandomNumber (100);
for (int try=1; try<=NUMBER_OF_GUESSES; try++)
{
    printf ("Try #%d of %d - guess a number from 1-100: ",
             try, NUMBER_OF_GUESSES);

    guess = InputNumber ();

    // ... logic continues ...
}

Now instead of updating multiple places in the file for the number of guesses, only the #define has to be changed. The #define has the advantage of not taking extra code or memory space like a variable would, which is important when you are working on embedded systems with 8K of RAM.

You probably have also seen #defines used for marking code to be included or not included in a program:

#define DEBUG_MODE

void Function ()
{
#if defined(DEBUG_MODE)
    printf ("Inside Function()\n");
#endif

    // ...logic continues...
}

WIth “#define DEBUG_MODE” there, the printf() will be included. Remove that #define (or comment it out) and it will not.

But #defines can also become macros with parameters, such as this:

#define SNOOZE(x) SleepMs(x*1000)

If you want to sleep for seconds, you could use a macro that turns into the call to the millisecond sleep with the passed-in value multiplied by 1000:

void Function ()
{
    printf ("Pausing for 5 seconds...\n");

    SNOOZE (5);

    // ...logic continues...
}

The C preprocessor will take that “5” and substitute where the “x” is in the replacement text, becoming:

void Function ()
{
    printf ("Pausing for 5 seconds...\n");

    SleepMs (5*1000);

    // ...logic continues...
}

Now if the code is ported to a system with a different sleep call, the #define can be changed to use whatever is available.

I’d expect anyone who has programmed in C has done one or all of these things.

But wait, there’s more!

But a macro does not have to be a simple number, string or function name. It can be a whole block of code that gets substituted. You can put in many lines, just by adding a “\” at the end of the line to continue parsing the next line after it:

#define DISPLAY_COUNTDOWN(x) \
    for (int idx=x; idx>=0; idx++) \
    { \
        printf ("%d...", idx); \
        sleep (1000); /* 1000ms, 1 second sleep */
    }

void Function ()
{
    printf ("Self-destruct activated...\n");

    DISPLAY_COUNTDOWN (10);

    // ...logic continues ...
}

And that would be processed to replace the “DISPLAY_COUNTDOWN(10)” with the C code in the #define:

void Function ()
{
    printf ("Self-destruct activated...\n");

    for (int idx=x; idx>=0; idx++) {         printf ("%d...", idx);         sleep (1000); /* 1000ms, 1 second sleep */     }

    // ...logic continues ...
}

Yeah, it would look ugly if you could see how the C preprocessor puts it in, but it builds and runs and you never see it (unless you specifically look at preprocessed output files).

But that is probably dumb. You should just make a “DisplayCountdown()” function and have it be more normal.

But wait, there’s less dumb…

But in the case of my panel functions, each one of them had a unique panel name and panel identifier, so using a function for them was not really possible. Each one had to be its own function since the functions contained the name of the panel (“PanelMainInit()”, “PanelMainTerm()”, etc.).

But a #define can do that…

#define GENERATE_PANEL_PROTOTYPES(panel_name) \
    int panel_name##Init (void); \
    int panel_name##GetHandle (void); \
    int panel_name##Display (void); \
    int panel_name##Hide (void); \
    int panel_name##Term (void);

The macro uses “panel_name” as the substitution “variable” passed in, and will place whatever text is there anywhere in the macro where “panel_main” appears. Since I wanted to pass in the filename (without extension) of the panel such as “PanelMain” or “PanelFaults”) and build a function name out of it, I use the ## concatenate feature that will glue the items before and after it together. That macro used like this:

GENERATE_PANEL_PROTOTYPES(PanelMain)

GENERATE_PANEL_PROTOTYPES(PanelFaults)

GENERATE_PANEL_PROTOTYPES(PanelAdmin)

…effectively generates the prototypes like this:

int PanelMainInit (void);
int PanelMainGetHandle (void);
int PanelMainDisplay (void);
int PanelMainHide (void);
int PanelMainTerm (void);

int PanelFaultsInit (void);
int PanelFaultsGetHandle (void);
int PanelFaultsDisplay (void);
int PanelFaultsHide (void);
int PanelFaultsTerm (void);

int PanelAdminInit (void);
int PanelAdminGetHandle (void);
int PanelAdminDisplay (void);
int PanelAdminHide (void);
int PanelAdminTerm (void);

…though it actually looks like one long run-one line for each one if you looked at the pre-processed C output, but the result is the same.

A similar macro could generate the actual functions:

#define STRINGIFY(x) #x
#define TOSTRING(x) STRINGIFY(x)

#define GENERATE_PANEL_FUNCTIONS(panelName, panelResourceID) \
    static int S_##panelName##Handle = 0; /* Zero is not a valid panel handle. */ \
    \
    int panelName##Init (void) \
    { \
        int panelHandle = 0; \
        if (S_##panelName##Handle <= 0) \
        { \
            panelHandle = LoadPanel (0, TOSTRING(panelName)".uir", panelResourceID); \
            if (panelHandle > 0) \
            { \
                S_##panelName##Handle = panelHandle; \
                \
                panelName##UserInit (panelHandle); \
            } \
        } \
        else \
        { \
            panelHandle = S_##panelName##Handle; \
        } \
        return panelHandle; \
    } \
    \
    int panelName##GetHandle (void) \
    { \
         return panelName##Init (); \
    } \
    \
    int panelName##Display (void) \
    { \
        int status = UIEHandleInvalid; \
        int panelHandle = panelName##Init (); \
        if (panelHandle > 0) \
        { \
            status = DisplayPanel (panelHandle); \
        } \
        return status; \
    } \
    \
    int panelName##Hide (void) \
    { \
        int status = UIEHandleInvalid; \
        if (S_##panelName##Handle > 0) \
        { \
            status = HidePanel (S_##panelName##Handle); \
        } \
        return status; \
    } \
    \
    /* Unload the panel, if valid. */ \
    int panelName##Term (void) \
    { \
        int status = UIEHandleInvalid; \
        if (S_##panelName##Handle > 0) \
        { \
            status = DiscardPanel (S_##panelName##Handle); \
            if (status == UIENoError) \
            { \
                S_##panelName##Handle = 0; \
            } \
        } \
        return status; \
    }

That macro would be used like this:

GENERATE_PANEL_FUNCTIONS(PanelMain, PANEL_MAIN)

GENERATE_PANEL_FUNCTIONS(PanelFaults, PANEL_FAULTS)

GENERATE_PANEL_FUNCTIONS(PanelAdmin, PANEL_ADMIN)

…and it would create a fully populated set of functions for those panels.

This allowed me to have a header file that had those macros, such as “PanelMacros.h”, and then have a .c and .h for each panel, or one big file that had them all in it.

// Panels.h
GENERATE_PANEL_PROTOTYPES(PanelMain);
GENERATE_PANEL_PROTOTYPES(PanelFaults);
GENERATE_PANEL_PROTOTYPES(PanelAdmin);

// Panels.c
GENERATE_PANEL_FUNCTIONS(PanelMain, PANEL_MAIN)
GENERATE_PANEL_FUNCTIONS(PanelFaults, PANEL_FAULTS)
GENERATE_PANEL_FUNCTIONS(PanelAdmin, PANEL_ADMIN)

And it worked great! And, if I later decided I wanted to add debugging output or something else, instead of editing one hundred different panel functions I could just modify the macro. For example:

#define GENERATE_PANEL_FUNCTIONS(panelName, panelResourceID) \
    static int S_##panelName##Handle = 0; /* Zero is not a valid panel handle. */ \
    \
    int panelName##Init (void) \
    { \
        int panelHandle = 0; \
        if (S_##panelName##Handle <= 0) \
        { \
            panelHandle = LoadPanel (0, TOSTRING(panelName)".uir", panelResourceID); \
            if (panelHandle > 0) \
            { \
                DebugPrintf ("Panel %s loaded.\n", TOSTRING(panelName)); \
                S_##panelName##Handle = panelHandle; \
                \
                panelName##UserInit (panelHandle); \
            } \
        } \
        else \
        { \
            DebugPrintf ("Panel %s already initialized.\n", TOSTRING(panelName)); \

There are a few things to unpack in this example, such as the use of macros STRINGIFY(x) and TOSTRING(x), but those probably could be their own blog post.

Anyway, if you are lazy, and faced with generating dozens or hundreds of almost identical functions, this macro approach can save a ton of time. The macros I made for my original project, dealing with message functions, are vastly more complex than these, but I figured if I started with those most would run away screaming. (I know I sure would if I had been presented them by a coworker.)

I am sure there will be more to say about this, so perhaps a part 3 will show up.

Until then, I’d love to hear what an experienced C macro programmer has to say about this. I bet there are some better techniques and things I am completely unaware of. I’d love it if you’d share.

Thanks…

Addendum: Since I began writing this post, I have converted about 50 panels at work using a much more complex set of #define macros. They keep evolving as I needed to add support for “parent/child” panels, or extra debugging, or even new functions to check if a panel is displayed at the moment. All I did was update the macro, and the next build could use the new functions. I expect it has already saved me days of typing…

Generating C functions and prototypes using macros – part 1

See Also: part 1 and part 2.

There is always a first time for everything, and my first time doing this was a few years ago with a day job task. I was going to create a Windows DLL (dynamic link library) that would handle hundreds of messages (write/read response) via the I2C protocol.

The message protocol featured a header that contained a Message Type and Command Code. There were several types of messages, but for this blog post let’s just look at two types:

  • DATA – a message sent to do something, with our without payload data. These messages get a response back that is either an ACKnowledgement (it worked) or a NAK (it did not work). For example “set the time to X”.
  • QUERY – a message sent to request information, such as “what time is it?”

The Command Code was a byte, which meant there could be up to 256 (0-255) commands per message type. For example, some DATA messages might be defined like this:

#define DATA_PING        0 // ACK if there, NAK if not
#define DATA_RESET_CPU   1 // Reset CPU
#define DATA_EXPLODE     2 // Cause the penguin on the TV set to explode.

And QUERY messages might be like:

#define QUERY_STATUS      0 // Returns a status response
#define QUERY_VOLTAGE     1 // Returns the voltage
#define QUERY_TEMPERATURE 2 // Returns the temperature

These are, of course, made up examples, but you get the idea.

We had a number of different board types on our system that could receive these messages. Some messages were only applicable to specific boards (like, you couldn’t get temperature from a board that did not have a thermometer circuit or whatever). My idea was to create simple C functions that represented the message sent to the specific board, like this:

resp = AlphaBoardPING ();
resp = BetaBoardPING ();
resp = DeltaBoardPING ();
...
resp = BetaBoardRESET_CPU ();
...
resp = DeltaBoardQUERY_STATUS (...);

…and so on. I thought it would make a super simple way to write code to send messages and get back the status or response payload or whatever.

This is what prompted me to write a post about returning values as full structures in C. That technique was used there, making it super simple to use (and no chance of a wrong pointer crashing things).

I experimented with these concepts on my own time to make sure this idea would work. Some of the things I did ended up on my GitHub page:

  • https://github.com/allenhuffman/GetPutData – routines that let you put bytes or other data types in a buffer. Code similar to this was used to create the message bytes (header, payload, checksum at the end, etc.)
  • https://github.com/allenhuffman/StructureToFromBuffer – routines that would let me define how bytes in a buffer should be copied into a C structure. I was proud of this approach since it let me pass a pointer to a buffer of bytes along with some tables and have it return a fully populated C structure ready to use. This greatly simplified the amount of work needed to use these messages.

But, as I like to point out, I am pretty lazy and hate doing so much typing. The thought of creating hundreds of functions by hand was not how I wanted to spend my time. Instead, I wanted to find a way to automate the creation of these functions. After all, they all followed the same pattern:

  • Header – containing Message Type, Command Code, number of byte in a payload, etc.
  • Payload – any payload bytes
  • Checksum – at the end

The only thing custom would be what Message Type and Command Code to put in, and if there was a payload to send, populating those bytes with the appropriate data.

When a response was received, it would be parsed based on the Message Type and Command Code, and return a C structure matching the response payload.

A program to write a program?

Initially, I thought about making a program or script that would spit out hundreds of functions. But, this not lazy enough. Sure, I could have done this and created hundreds of functions, but what if those functions needed a change later? I’d have to update the program that created the functions and regenerate all the functions all over again.

There has to be a better lazier way.

Macros: a better lazier way

I realized I could make a set of #define macros that could insert proper C code or prototypes. Then, if I ever needed to change something, I only had to change the macro. There would be no regeneration needed, since the next compile would use the updated macro. Magic!

It worked very well, and created hundreds and hundreds of functions without me ever having to type more than the ones in the macro.

It worked so well that I ended up using this approach very recently for another similar task I was far too lazy to do the hard way. I thought I would share that much simpler example in case you are lazy as well.

A much simpler example

At work we use LabWindows/CVI, a Windows C compiler with its own GUI. It has a GUI editor where you create your window with buttons and check boxes and whatever, then you use functions to load the panel, display it, and hide it when done. They look like this:

int panelHandle = LoadPanel (0, "PanelMAIN.uir", PANEL_MAIN);

DisplayPanel (panelHandle);

// Do stuff...

HidePanel (panelHandle);

DiscardPanel (panelHandle);

Then, when you interact with the panel, you have callback functions (if the user clicks the “OK” button, it jumps to a function you might name “UserClickedOKButtonCallback()” or whatever.

If you need to manipulate the panel, such as changing the status of a Control (checkbox, text box, or whatever), you can set Values or Attributes of those Controls.

SetCtrlVal (panelHandle, PANEL_MAIN_OK_BUTTON, 1);
SetCtrlAttribute (panelHandle, PANEL_MAIN_CANCEL_BUTTON, ATTR_DIMMED, 1);

It is a really simple system and one that I, as a non-windows programmer who had never worked with GUIs before, was able to pick up and start using quickly.

Simplicity can get complicated quickly…

One of the issues with this setup is that you had to have the panel handle in order to do something. If a message came in from a board indicating there was a fault, that code might need to toggle on some “RED LED” graphics on a GUI panel to indicate the faulted condition. But, that callback function may not have any of the panel IDs. The designed created a lookup function to work around this:

int mainPanelHandle = LookUpPanelHandle(MAIN_PANEL);
SetCtrlVal (mainPanelHandle, PANEL_MAIN_FAULT_LED, 1);

A function similar to that was in the same C file where all the panels were loaded. Their handles saved in variables, then the LookUp function would go through a huge switch/case with special defines for every panel and return the actual panel handle that matched the define passed in.

It worked great but it was slower since it had to scan through that list every time we wanted to look up a panel. At some point, all the panel handles were just changed to global variables so they could be accessed quickly without any lookup:

SetCtrlVal (g_MainPanelHandle, PANEL_MAIN_FAULT_LED, 1);

This also worked great, but did not work from threads that did not have access to the main GUI context. Since I am not a Windows programmer, and have never used threads on any embedded systems, I do not actually understand the problem (but I hear there are “thread safe” variables that can be used for this purpose).

Self-contained panel functions for the win!

Instead of learning those special “thread safe” techniques, I decided to create a set of self-contained panel functions so you could do things like this:

int mainPanelHandle = PanelMainInit (); // Load/init the main panel.
PanelMainDispay (); // Display the panel.
SetCtrlVal (mainPanelHandle, PANEL_MAIN_FAULT_LEFT, 1);
...
PanelMainHide ();
...
PanelMainTerm (); // Unload main panel and release the memory.

When I needed to access a panel from another routine, I would use a special function that returned the handle:

int panelMainHandle = PanelMainGetHandle ();
SetCtrlVal (mainPanelHandle, PANEL_MAIN_FAULT_LEFT, 1);

I even made these functions automatically Load the panel if needed, meaning a user could just start using a panel and it would be loaded on-demand if was not already loaded. Super nice!

Here is a simple version of what that code looks like:

static int S_panelHandle = 0; // Zero is not a valid panel handle.

int PanelMainInit (void)
{
    int panelHandle = 0;

    if (S_panelHandle <= 0) // Zero is not a valid panel handle. 
    {
        panelHandle = LoadPanel (0, "PanelMAIN.uir", PANEL_MAIN);

        // Only set our static global if this was successful.
        if (panelHandle > 0) // Zero is not a valid panel handle. 
        {
            S_panelHandle = panelHandle; 
        }
    }
    else // S_panelHandle was valid.
    {
        panelHandle = S_panelHandle;
    }	
    
    // Return handle or status in case anyone wants to error check.
    return panelHandle;
}

int PanelMainGetHandle (void)
{
    // Return handle or status in case anyone wants to error check.
    return PanelMainInit ();
}

int PanelMainTerm (void)
{
    int status = UIEHandleInvalid;

    if (S_panelHandle > 0) // Zero is not a valid panel handle.
    {
        status = DiscardPanel (S_panelHandle);
        if (status == UIENoError)
        {
            S_panelHandle = 0; // Zero is not a valid panel handle. 
        }
    }

    // Return status in case anyone wants to error check.
    return status;
}

int PanelMainDisplay (void)
{
    int status = UIEHandleInvalid;
    int panelHandle;
	
    panelHandle = PanelMainInit (); // Init if needed.
	
    if (panelHandle > 0) // Zero is not a valid panel handle.
    {
        status = DisplayPanel (panelHandle);
    }
	
    // Return status in case anyone wants to error check.
    return status;
}

int PanelMainHide (void)
{
    int status = UIEHandleInvalid;

    if (S_panelHandle > 0) // Zero is not a valid panel handle.
    {
        status = HidePanel (S_panelHandle);
    }
	
    // Return status in case anyone wants to error check.
    return status;
}

This greatly simplified dealing with the panels. Now they could “just be used” without worrying about loading, etc. There was no long Look Up table, and no global variables. The only places the panel handles were kept was inside the file where the panel’s functions were.

Nice and simple, and it worked even greater than the first two attempts.

…until you have to make a hundred of these functions…

…and then decide you need to change something and have to make that change in a hundred functions.

My solution was to use #define macros to generate the code and prototypes, then I would only have to change the macro to alter how all the panels works. (Spoiler: This worked even greater than the previous greater.)

In part 2, I will share a simple example of how this works. If you are lazy enough, you might actually find it interesting.

Until then…

C, I can be taught! At least about calloc.

A long, long time ago, I learned about malloc() in C. I could make a buffer like this:

char *buffer = malloc (1024);

…use it, then release it when I was done like this:

free (buffer);

I have discussed malloc here in the past, including a recent post about an error checking malloc I created to find a memory leak.

But today, the Bing CoPilot A.I. suggested I use calloc instead.

And I wasn’t even sure I even remembered this was a thing. And it has been there since before C was standardized…

void* calloc (size_t num, size_t size);

malloc() will return a block of memory and, depending on the operating system and implementation in the compiler, that memory may have old data in it. Hackers were known to write programs that would allocate blocks of memory then inspect it to see what they could find left over from another program previously using it.

Hey, everybody’s gotta have a hobby…

calloc() is a “clear allocation” where it will initialize the memory it returns to zero before returning it to you. It also takes two parameters instead of just one. While malloc() wants to know the number of bytes you wish to reserve, calloc() wants to how know many things (bytes, structures, etc.) you want to reserve, and the size of each thing.

To allocate 1024 bytes using calloc() you would use:

char *buffer = calloc (1, 1024);

…and get one thing of 1024 bytes. Or maybe you prefer reversing that:

char *buffer = calloc (1024, 1);

…so you get 1024 things that are 1 byte each.

Either way, what you get back is memory all set to zeros.

calloc() was suggested by the A.I. because I was allocating a set of structures like this:

// Typedefs
typedef struct
{
    int x;
    int y;
    int color;
} MyStruct;

MyStruct *array = malloc (sizeof(MyStruct) * 42);

The A.I. saw that, and suggested calloc() instead, like this:

MyStruct *array = calloc (42, sizeof(MyStruct));

I do think that looks a bit cleaner and more obvious, if you are familiar with calloc(), and as long as you don’t need the extra speed (setting that memory to 0 should take more time than not doing that), it seems like something to consider.

And maybe that will break me (and other programmers who wrote code before me that I may one day maintain) from doing it manually like…

char *ptr = malloc (1024);
memset (ptr, 0x0, 1024);

I wonder if I will even remember this the next time I need to malloc() something.

I mean calloc() something.

Whatever.

Until then…

C and VLAs (Variable Length Arrays)

When you are old (or “experienced” if you prefer), you begin to realize how much of what you learned is wrong. Even if it was “right” when you learned it. I think of all peers that went through computer courses at colleges back in the late 1980s or 1990s, learning now-obsolete languages and being taught methods and approaches that are today considered wrong.

When I learned C, it was on a pre-ANSI K&R C compiler. I learned it on my Radio Shack Color Computer 3 under the OS-9 operating system, with assistance from a friend of mine who had learned C on his Commodore Amiga.

I had alot of new things to learn in 1995 when I took a job with Microware Systems Corporation (creator of OS-9 and the K&R compiler I had learned on). Their Ultra-C compiler was an ANSI compiler, and it did things quite different.

In that era of the C89/C90 standard, arrays were just arrays and we liked it that way:

int array[42];

if you wanted things to be more flexible, you had to malloc() memory yourself.

int *array = malloc (sizeof(int)*42);

…and remember to stay within your boundaries and clean up/free that memory when you were done with it.

But C99 changed this, somewhat, with the introduction of VLAs (Variable Length Arrays). Now you could declare an array using a variable like this:

int x=42;

int array(x);

Neat. I do not think I have ever used this. One downside is you cannot do this with static variables, since those are created/reserved at compile time. But it is still neat.

But today I learned, you couldn’t rely on VLA is you were using C11. Apparently, they became optional that year. A compiler would define a special define if it did not support them:

__STD_NO_VLA__

But at least for twelve years of the standard, you could rely on them, before not being able to rely on them.

And then C23 happened, which I just learned made VLAs mandatory again.

So, uh, I guess if you have the latest and greatest, you can use them. For now. Until some future change makes them option again. Or removes them. Or whatever.

Still neat.

But I doubt any of the embedded C compilers I use for my day job support them.

A safer memcpy with very limited use cases

Here is a quick one… At my day job, I found lines of code like this:

memcpy(systemDataPtr->serialNumber, resp.serialNumber, 16);

A quick peek at systemDataPtr->serialNumber shows it defined as this:

unsigned char serialNumber[MAX_SERIAL_NUMBER_LENGTH];

…with that constant defined as:

#define MAX_SERIAL_NUMBER_LENGTH        16

So while 16 is correct, the use of hard-coded “magic numbers” (hat tip to a previous manager, Pete S., who introduced me to that term) is probably best to be avoided. Change that #define, and things could go horribly wrong with a memory overrun or massive nuclear explosion or something.

One simple fix is to use the #define in the memcpy:

memcpy(systemDataPtr->serialNumber, resp.serialNumber, MAX_SERIAL_NUMBER_LENGTH);

This, of course, assumes that resp.serialNumber is also 16. Let’s see:

char serialNumber[16];

Ah, magic number! In this case, it comes from a DLL header file that does not share that #define, and the header file for the DLL was made by someone who had never made a Windows DLL before (me) and did not make #defines for these various lengths.

What if the DLL value ever got out-of-sync? Worst case, not all data would be copied (only 16 bytes). That seems fine. But if the DLL value became smaller, like 10, then the memcpy would still copy 16 bytes, copying the 10 from the DLL buffer plus 6 bytes of data in memory after it — buffer overrun?

In this case, since the destination buffer can hold 16 bytes, and we only copy up 16 bytes, the worst case is we could get some unintended data in that buffer.

sizeof() exists for a reason.

One thing I tend to do is use sizeof() instead of hard-coded numbers or the #define, since it will continue to work if the source buffer ever got changed from using the #define:

memcpy(systemDataPtr->serialNumber, resp.serialNumber, sizeof(systemDataPtr->serialNumber));

But this still has the same issue if the source resp.serialNumber became larger.

A safer, and more ridiculous, memcpy

Naturally, I came up with a ridiculous “solution”: A safer memcpy() that is much more of a pain to use because you have to know the size of each buffer and tell it the size of each buffer so it can make sure not to copy something larger than will fit into the destination buffer.

Here is the prototype of memcpy():

void * memcpy ( void * destination, const void * source, size_t num );

It will blindly copy up to “num” bytes from “source” to “destination”. But a ridiculous safer memcpy might look like this:

void * memcpy_safer ( void * destination, size_t sizeOfDestination,
                      const void * source, size_t sizeOfSource,
                      size_t num );

Just think of the extra overhead to add two more parameters for every use! Plus, it is a longer function name so you get to type even more! Wonderful.

Here is a quick implementation:

void * memcpy_safer ( void * destination, size_t sizeOfDestination,
                      const void * source, size_t sizeOfSource,
                      size_t num )
{
    // Use whichever size is the smallest.
    if ((num > sizeOfDestination) || (num > sizeOfSource))
    {
        if (sizeOfDestination < sizeOfSource)
        {
            num = sizeOfDestination;
        }
        else
        {
            num = sizeOfSource;
        }
    }
    
    return memcpy ( destination, source, num);
}

Now that the function knows the size of source and destination, it can do a simple check to make sure to use whichever is smallest for the max number of bytes to copy — regardless (or irregardless, depending on which you prefer) of how many bytes you specified.

If both buffers are larger than the “num” passed in, num is used. But if “num” is larger than either buffer, it is reduced to match the largest of the two buffers.

Note that this does not do any NULL checks, so it is relying on the behavior of memcpy() which would likely be a crash if NULL is passed in. To improve, NULL checks could be added.

I decided to pass this function into Bing CoPilot and see what it “thought” about this:

Limitations & Risks

Silent truncation: If num is too large, the function silently reduces it. This can lead to subtle bugs if the caller expects all bytes to be copied.

No error signaling: There’s no return code or logging to indicate that truncation occurred.

Still not “safe” in all cases: If destination or source are NULL, or if the buffers overlap, memcpy may still cause undefined behavior.

No runtime buffer validation: It assumes the caller provides correct buffer sizes, which may not be guaranteed.

– Bing CoPilot

And, of course, the A.I. offered up these changes:

#include <string.h>
#include <stddef.h>
#include <stdio.h>

void * memcpy_safer ( void * destination, size_t sizeOfDestination,
                      const void * source, size_t sizeOfSource,
                      size_t num )
{
    if (!destination || !source) {
        fprintf(stderr, "Error: NULL pointer passed to memcpy_safer.\n");
        return NULL;
    }

    if (num > sizeOfDestination || num > sizeOfSource) {
        size_t adjusted = (sizeOfDestination < sizeOfSource) ? sizeOfDestination : sizeOfSource;
        fprintf(stderr, "Warning: Truncating copy from %zu to %zu bytes.\n", num, adjusted);
        num = adjusted;
    }

    return memcpy(destination, source, num);
}

That version adds NULL checks, returns a NULL if either buffer passed in was NULL, and adds prints to standard error if a NULL happens or if the value was truncated.

Not bad, predictive language model.

My ridiculous test program

Here is my test program, which I wrote using the Online GDB C compiler:

/******************************************************************************

Welcome to GDB Online.
  GDB online is an online compiler and debugger tool for C, C++, Python, PHP, Ruby, 
  C#, OCaml, VB, Perl, Swift, Prolog, Javascript, Pascal, COBOL, HTML, CSS, JS
  Code, Compile, Run and Debug online from anywhere in world.

*******************************************************************************/
#include <stdint.h> // for uint8_t
#include <stdio.h>  // for printf()
#include <stdlib.h> // for EXIT_SUCCESS
#include <string.h> // for memcpy()

/*---------------------------------------------------------------------------*/
// PROTOTYPES
/*---------------------------------------------------------------------------*/

void * memcpy_safer ( void * destination, size_t sizeOfDestination,
                      const void * source, size_t sizeOfSource,
                      size_t num );

void * memcpy_safer2 ( void * destination, size_t sizeOfDestination,
                       const void * source, size_t sizeOfSource,
                       size_t num );

void initializeBuffer (void *dataPtr, size_t dataSize, uint8_t value);

void dumpBuffer (const char* prefix, void *dataPtr, size_t dataSize);

/*---------------------------------------------------------------------------*/
// MAIN
/*---------------------------------------------------------------------------*/

int main()
{
    uint8_t smallerBuffer[10];
    uint8_t largerBuffer[15];
    
    // Test 1: copy longer buffer into smaller buffer.
    
    printf ("\nInitialized buffers:\n\n");    
    
    // Initialize buffers with something we can identify later.
    initializeBuffer (smallerBuffer, sizeof(smallerBuffer), 0x1);
    dumpBuffer ("smallerBuffer", smallerBuffer, sizeof(smallerBuffer));

    initializeBuffer (largerBuffer, sizeof(largerBuffer), 0x2);
    dumpBuffer ("largerBuffer ", largerBuffer, sizeof(largerBuffer));

    printf ("\nTest 1: Copying largerBuffer into smallerBuffer...\n\n");

    memcpy_safer (smallerBuffer, sizeof(smallerBuffer), largerBuffer, sizeof(largerBuffer), 42);

    dumpBuffer ("smallerBuffer", smallerBuffer, sizeof(smallerBuffer));

    // Test 2: copy smaller buffer into larger buffer.

    printf ("\nInitialized buffers:\n\n");

    // Initialize buffers with something we can identify later.
    initializeBuffer (smallerBuffer, sizeof(smallerBuffer), 0x1);
    dumpBuffer ("smallerBuffer", smallerBuffer, sizeof(smallerBuffer));

    initializeBuffer (largerBuffer, sizeof(largerBuffer), 0x2);
    dumpBuffer ("largerBuffer ", largerBuffer, sizeof(largerBuffer));

    printf ("\nTest 2: Copying smallerBuffer into largerBuffer...\n\n");

    memcpy_safer (largerBuffer, sizeof(largerBuffer), smallerBuffer, sizeof(smallerBuffer), 42);

    dumpBuffer ("largerBuffer ", largerBuffer, sizeof(largerBuffer));

    return EXIT_SUCCESS;
}


/*---------------------------------------------------------------------------*/
// FUNCTIONS
/*---------------------------------------------------------------------------*/

/*---------------------------------------------------------------------------*/
// My ridiculous "safer" memcpy.
/*---------------------------------------------------------------------------*/
void * memcpy_safer ( void * destination, size_t sizeOfDestination,
                      const void * source, size_t sizeOfSource,
                      size_t num )
{
    // Use whichever size is the smallest.
    if ((num > sizeOfDestination) || (num > sizeOfSource))
    {
        if (sizeOfDestination < sizeOfSource)
        {
            num = sizeOfDestination;
        }
        else
        {
            num = sizeOfSource;
        }
    }
    
    return memcpy ( destination, source, num);
}


/*---------------------------------------------------------------------------*/
// Bing CoPilot changes.
/*---------------------------------------------------------------------------*/
void * memcpy_safer2 ( void * destination, size_t sizeOfDestination,
                       const void * source, size_t sizeOfSource,
                       size_t num )
{
    if (!destination || !source) {
        fprintf(stderr, "Error: NULL pointer passed to memcpy_safer.\n");
        return NULL;
    }

    if (num > sizeOfDestination || num > sizeOfSource) {
        size_t adjusted = (sizeOfDestination < sizeOfSource) ? sizeOfDestination : sizeOfSource;
        fprintf(stderr, "Warning: Truncating copy from %zu to %zu bytes.\n", num, adjusted);
        num = adjusted;
    }

    return memcpy(destination, source, num);
}


/*---------------------------------------------------------------------------*/
// Utility function to initialize a buffer to a set value.
/*---------------------------------------------------------------------------*/
void initializeBuffer (void *dataPtr, size_t dataSize, uint8_t value)
{
    if (NULL != dataPtr)
    {
        memset (dataPtr, value, dataSize);
    }
}


/*---------------------------------------------------------------------------*/
// Utility function to dump bytes in a buffer, with an optional prefix.
/*---------------------------------------------------------------------------*/
void dumpBuffer (const char* prefix, void *dataPtr, size_t dataSize)
{
    if (NULL != dataPtr)
    {
        if (NULL != prefix)
        {
            printf ("%s: ", prefix);
        }

        for (size_t idx=0; idx<dataSize; idx++)
        {
            printf ("%02x ", ((uint8_t*)dataPtr)[idx]);
        }
        printf ("\n");
    }
}

// End of memcpy_safer.c

If you want to run it there, you can use this link:

https://onlinegdb.com/Eu7FToIcQ

But of course, I am not using this code. It is ridiculous and requires extra typing.

Besides, I know exactly what I am doing in C and never make any mistakes… Really.

Until next time…

I only understand this a bit…

NOTE: Credit to the content of the article, other than my comments about the content, goes to whatever A.I. engine Microsoft uses for CoPilot…

While trying out a feature of Microsoft’s CoPilot A.I., I decided to ask it a simple question:

What is a good way to count the set bits in a 32-bit integer in the C programming language?

At my day job, we collect a byte from boards that multiplex eight sub-boards. There can be up to four of these multiplexer boards for a total of 32 sub-boards being multiplexed. Bits in those bytes will be set to indicate if a sub-board was found. The original designer has code that retrieves a byte from each multiplexer board then combines them into a 32-bit integer. There is code that counts the set bits to make sure we found as many boards as the system is configured to expect.

To my surprise, CoPilot gave me five different routines. The first was easy to understand, and a bit more clever than what I would have created. It called this the Iterative Bitwise Method with a description of “simple and portable, but not the fastest.”

int countSetBits(int n) {
    int count = 0;
    while (n) {
        count += n & 1;
        n >>= 1;
    }
    return count;
}

Next was something called the Brian Kernighan’s Algorithm, which was “efficient for sparse bit patterns.”

int countSetBits(int n) {
    int count = 0;
    while (n) {
        n &= (n - 1);
        count++;
    }
    return count;
}

Third was a Lookup Table Method for “fast for repeated calls but requires extra memory.”

unsigned char lookupTable[256];

void initializeLookupTable() {
    for (int i = 0; i < 256; i++) {
        lookupTable[i] = countSetBits(i); // Use any method to count bits
    }
}

int countSetBitsUsingLookup(int n) {
    return lookupTable[n & 0xFF] +
           lookupTable[(n >> 8) & 0xFF] +
           lookupTable[(n >> 16) & 0xFF] +
           lookupTable[(n >> 24) & 0xFF];
}

Fourth was something I had never heard of: Compiler Intrinsics. It said “may modern compilers provide built-in functions to count set bits efficiently” and described it as “extremely fast but platform-dependent.”

#include <intrin.h>

int countSetBits(int n) {
    return __popcnt(n);
}

In my world of “barely C” embedded compiles that don’t even support the modern printf formats for printing pointers, I do not expect to have access to this.

And fifth was Parallel Bitwise Operations, described as “fast and portable, but harder to understands.”

And that is the version that made me post this:

int countSetBits(int n) {
    n = n - ((n >> 1) & 0x55555555);
    n = (n & 0x33333333) + ((n >> 2) & 0x33333333);
    n = (n + (n >> 4)) & 0x0F0F0F0F;
    n = n + (n >> 8);
    n = n + (n >> 16);
    return n & 0x3F;
}

Harder to understand is an understatement. My brane hurts just looking at that trying to figure out how it works.

And I thought I’d share that hurt with you.

Have a good C day…