I have had a few glorious weeks of C coding at my day job. It is some of the best C code I have ever written, and I am quite proud of it.

During this project, I learned something new. In the past, I would commonly use #define for values such as:

#define NONE 0
#define WARNING 1
#define ERROR 2

There might be some code somewhere that made us of those values:

void HandleError (int error)
{
   case NONE:
      // All fine.
      break;

   case WARNING:
      // Handle warning.
      break;

   case ERROR:
      // Handle error
      break;

    default:
      // Handle unknown error.
      break;
}

When you use #define like that, the C pre-processor will just do simple substitutions before compiling the code — “NONE” will become “0”. Because of this, those labels mean nothing special to the compiler — they are just numbers.

But, I had learned you could use an enum and ensure a function only allows passing the enum rather than “whatever int”:

typedef enum
{
   NONE = 0,
   WARNING = 1,
   ERROR = 2
} ErrorEnum;

Now the routine can take an “ErrorEnum” as a parameter type.

void HandleError (ErrorEnum error)
{
   case NONE:
      // All fine.
      break;

   case WARNING:
      // Handle warning.
      break;

   case ERROR:
      // Handle error
      break;

    default:
      // Handle unknown error.
      break;
}

This looks nicer, since it is clear what is being passed in (whatever “ErrorEnum” is). It does not necessarily give any extra compiler warnings if you pass in an int instead, since an enum is just an int, by default.

In C, an enum is a user-defined data type that represents a set of named values. By default, the underlying data type of an enum is int, but it can be explicitly specified to be any integral type using a colon : followed by the desired type.

– ChatGPT

Thus, the compiler I am using for this test (Visual Studio, building a C program) does not complain if I do something like this:

int error = 1;
HandleError (error);

However, I just learned of an interesting benefit to using an enum for a switch/case. The compiler can warn you if you don’t have a case for all the items in the enum!

typedef enum {
	NONE,
	WARNING,
	ERROR
} ErrorEnum;

void HandleError(ErrorEnum error)
{
	switch (error)
	{
	case NONE:
		// All fine.
		break;

	case WARNING:
		// Handle warning.
		break;

}

Building that with warnings enabled reports:

enumerator 'ERROR' in switch of enum 'ErrorEnum' is not handled	CTest	

How neat! I actually ran in to this when I had added some extra error types, but a routine that converted them to a text string did not have cases for the new errors.

const char *GetErrorString (ErrorEnum error)
{
   const char *errorStringPtr = "Unknown";

   switch NONE:
      errorStringPtr = "None";
      break;
}

That is a useful warning, and it found a bug/oversight.

Because of this, I will never use #defines for things like this again. Not only does a data type look more obvious in the code, the compiler can give you extra warnings.

Until next time…

In a recent article, I discussed methods to copy variables (or structure elements) in to a packed buffer, as well as parse buffer bytes back in to variables.

Doing it manually was clunky. Using my GetPutData routines made it easier to be clunky.

But today, we’ll make it super simple and far less clunky.

Defining the message bytes

Let’s start with a simple message. It will be represented by a C structure that we’d use in the program. For example, here are the elements that might be part of a “Set Date and Time” message:

typedef struct
{
    uint16_t year;
    uint8_t month;
    uint8_t day;
    uint8_t hour;
    uint8_t minute;
    uint8_t second;
    uint8_t timeZone;
    bool isDST;
} SetDateTimeMessageStruct;

The packed message would be nine bytes and look like this:

[ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 ]
  \  /    |   |   |   |   |   |   |
  year   mon day hr  min sec tz  dst

To automate this process, a lookup table is created that represents the size of each element and the offset to where it is located within the structure.

offsetof()

Inside stddef.h is a macro that is used to calculate where in the structure’s memory a particular element exists. It is a relative offset from wherever the structure variable starts in memory. Here is an example showing offsetof():

#include <stdio.h>
#include <stdlib.h>  // for EXIT_SUCCESS
#include <stdint.h>
#include <stdbool.h>
#include <stddef.h>  // for offsetof()

typedef struct
{
    uint8_t     a;
    uint16_t    b;
    bool        c;
    uint32_t    d;
    float       e;
    double      f;
} MyStruct;

int main()
{
    MyStruct test;
    
    printf ("sizeof(test) = %ld\n", sizeof(test));
    
    printf ("a is %lu bytes at offset %lu\n",
            sizeof(test.a), offsetof(MyStruct, a));
    printf ("b is %lu bytes at offset %lu\n",
            sizeof(test.b), offsetof(MyStruct, b));
    printf ("c is %lu bytes at offset %lu\n",
            sizeof(test.c), offsetof(MyStruct, c));
    printf ("d is %lu bytes at offset %lu\n",
            sizeof(test.d), offsetof(MyStruct, d));
    printf ("e is %lu bytes at offset %lu\n",
            sizeof(test.e), offsetof(MyStruct, e));
    printf ("f is %lu bytes at offset %lu\n",
            sizeof(test.f), offsetof(MyStruct, f));

    return EXIT_SUCCESS;
}

Try it: https://onlinegdb.com/GDvhi9TLq

It produces the following output:

sizeof(test) = 24
a is 1 bytes at offset 0
b is 2 bytes at offset 2
c is 1 bytes at offset 4
d is 4 bytes at offset 8
e is 4 bytes at offset 12
f is 8 bytes at offset 16

This is an example of how elements in a structure may be padded to ensure each one starts on an even-byte in memory. ‘a’ is one byte starting at offset 0, but ‘b’ doesn’t start at 1. As a 16-bit value, it has to skip a byte and start at offset 2. There are more skipped bytes after ‘c’ at offset 4. Since the next value is a 32-bit value, it skips three bytes.

If we just wrote the structure memory out (in this architecture), it would look like this:

[ 0| 1| 2| 3| 4| 5| 6| 7| 8| 9|10|11|12|13|14|15|16|17|18|19|20|21|22|23]
  |     |     |           |           |           |
  a     b--b  c           d--d--d--d  e--e--e--e  f--f--f--f--f--f--f--f

That would waste 4 bytes in that 24 byte message. Plus, if the message were parsed by a system that had different padding/alignment requirements, they wouldn’t be able to read the buffer back in to their structure variable and get the correct data.

Using offsetof() to determine where in a structure an element exists, a lookup table could be created. A C function could be written to use that table to know where to copy the data. Here’s a rough concept:

typedef struct
{
    size_t size;   // size of structure element
    size_t offset; // offset from start of structure to that element
} ElementOffsetStruct;

MyStruct test;

ElementOffsetStruct testTable[] =
{
    { sizeof(test.a), offsetof(MyStruct, a) },
    { sizeof(test.b), offsetof(MyStruct, b) },
    { sizeof(test.c), offsetof(MyStruct, c) },
    { sizeof(test.d), offsetof(MyStruct, d) },
    { sizeof(test.e), offsetof(MyStruct, e) },
    { sizeof(test.f), offsetof(MyStruct, f) },
    { 0, 0 } // end of table
};

for (int idx=0; testTable[idx].size != 0; idx++)
{
    printf ("? is %u bytes at offset %u\n",
            testTable[idx].size, testTable[idx].offset);
}

Knowing where items are in the structure allows the specific element bytes to be copied to a buffer. Something like this:

uint8_t buffer[80];
unsigned int offset;

offset = 0;
for (int idx=0; testTable[idx].size != 0; idx++)
{
    printf ("Copy %lu bytes at structure offset %lu to buffer offset %u\n",
            testTable[idx].size, testTable[idx].offset, offset);

    memcpy (buffer + offset,
        (uint8_t*)&test + testTable[idx].offset,
        testTable[idx].size);
        
    offset = offset + testTable[idx].size; 
}

“And it’s just that easy.” Running the following test program dumps the memory contents of the C structure “test” and then writes them out to a buffer using this code and then dumps the memory contents of the buffer.

#include <stdio.h>
#include <stdlib.h>  // for EXIT_SUCCESS
#include <stdint.h>
#include <stdbool.h>
#include <stddef.h>  // for offsetof()
#include <string.h>  // for memcpy()

typedef struct
{
    uint8_t     a;
    uint16_t    b;
    bool        c;
    uint32_t    d;
    float       e;
    double      f;
} MyStruct;

typedef struct
{
    size_t size;   // size of structure element
    size_t offset; // offset from start of structure to that element
} ElementOffsetStruct;

int main()
{
    MyStruct test = { 0x11, 0x2222, true, 0x33333333, 42.42, 42.42 };
    
    ElementOffsetStruct testTable[] =
    {
        { sizeof(test.a), offsetof(MyStruct, a) },
        { sizeof(test.b), offsetof(MyStruct, b) },
        { sizeof(test.c), offsetof(MyStruct, c) },
        { sizeof(test.d), offsetof(MyStruct, d) },
        { sizeof(test.e), offsetof(MyStruct, e) },
        { sizeof(test.f), offsetof(MyStruct, f) },
        { 0, 0 } // end of table
    };
    
    for (int idx=0; idx<sizeof(test); idx++)
    {
        printf ("%02x ", ((uint8_t*)&test)[idx]);
    }
    printf ("\n");

    uint8_t buffer[80];
    unsigned int offset;
    
    offset = 0;
    for (int idx=0; testTable[idx].size != 0; idx++)
    {
        printf ("Copy %lu bytes at structure offset %lu to buffer offset %u\n",
                testTable[idx].size, testTable[idx].offset, offset);
    
        memcpy (buffer + offset,
            (uint8_t*)&test + testTable[idx].offset,
            testTable[idx].size);
            
        offset = offset + testTable[idx].size; 
    }

    for (int idx=0; idx<offset; idx++)
    {
        printf ("%02x ", buffer[idx]);
    }
    printf ("\n");

    return EXIT_SUCCESS;
}

I get the following output:

11 00 22 22 01 00 00 00 33 33 33 33 14 ae 29 42 f6 28 5c 8f c2 35 45 40 
Copy 1 bytes at structure offset 0 to buffer offset 0
Copy 2 bytes at structure offset 2 to buffer offset 1
Copy 1 bytes at structure offset 4 to buffer offset 3
Copy 4 bytes at structure offset 8 to buffer offset 4
Copy 4 bytes at structure offset 12 to buffer offset 8
Copy 8 bytes at structure offset 16 to buffer offset 12
11 22 22 01 33 33 33 33 14 ae 29 42 f6 28 5c 8f c2 35 45 40 

As I discovered when I figured this out, there are a few gotchas. First, the sample code here requires the structure variable to exist. This is because the macro sizeof() needs to have something to work on to determine the size:

sizeof(test.a)

I asked ChatGPT about this, and it provided me a clever workaround that worked on the structure itself rather than a variable:

sizeof(((MyStruct*)0)->a)

Looking at that, I think I get how it works. First, it is taking a memory location of 0 and casting that to be a pointer to a MyStruct. No such structure exists at memory 0, but you can cast a pointer to anywhere (even 0) as any kind of pointer you want. (Actually using a pointer that point to 0 would be bad, however.)

Then, it gets the offset of the “a” element from the structure we pretend is located at memory location 0. Let’s break that apart a bit:

sizeof(                 )
       (            )->a
        (MyStruct*)0

And this is enough to allow creating a table based on a structure, and not on a variable declared using that structure.

typedef struct
{
    uint8_t u8;
    uint32_t u32;
} SmallStruct;

ElementOffsetStruct smallTable[] =
{
    { sizeof(((SmallStruct*)0)->u8), offsetof(SmallStruct, u8) },
    { sizeof(((SmallStruct*)0)->u32), offsetof(SmallStruct, u32) },
    { 0, 0 }
};

This would allow an easy way to define the structure and a table of offsets for later use. I’d probably make the table structure const since it won’t need to change.

With this framework figured out, a reverse function could be done that would copy bytes from a buffer in to the correct offset inside the structure.

unsigned int offset;
    
offset = 0;
for (int idx=0; testTable[idx].size != 0; idx++)
{
    printf ("Copy %lu bytes at buffer offset %u to structure offset %lu\n",
            testTable[idx].size, offset, testTable[idx].offset);
    
    memcpy ((uint8_t*)&test + testTable[idx].offset,
            buffer + offset,
           testTable[idx].size);
            
    offset = offset + testTable[idx].size; 
}

And now, by creating a structure and a table of size/offsets, any structure can be written to a buffer, or decoded from a buffer.

Too. Much. Typing.

For the final touch, macros can be made to simplify building the table.

#define ENTRY(s, v) { sizeof(((s*)0)->v), offsetof(s, v) }
#define ENTRYEND    {0, 0}

Now, instead of having to type all that gibberish out, the table can be simplified:

ElementOffsetStruct testTable[] =
{
    ENTRY(MyStruct, a),
    ENTRY(MyStruct, b),
    ENTRY(MyStruct, c),
    ENTRY(MyStruct, d),
    ENTRY(MyStruct, e),
    ENTRY(MyStruct, f),
    ENTRYEND
};

And this is just what I have posted to my GitHub:

https://github.com/allenhuffman/StructureToFromBuffer

You can find a short README there that explains the two functions and how to use them. It greatly simplifies the process of creating message buffers or parsing them in to C structures that can easily be used in a program.

If you try it and find it useful, please let me know.

Until then…

I had originally envisioned this post to be another “Old C dog, new C tricks” article, but it really is more of a “how long has it done this? did I just never read the manual?” article.

In C, strcpy() is used to copy a C string (series of character bytes with a 0 byte marking the end of the string) to a new buffer. Some years ago, I wrote a series about the dangers of strcpy() and other functions that look for that 0 byte at the end. I later wrote a follow-up that discussed strcat().

char buffer[10];

strcpy (buffer, "Hello");

Which means something like this can crash your program:

char buffer[10];

strcpy (buffer, "This will cause a buffer overrun.);

strncpy(), on the other hand, takes a third parameter which is the maximum number of characters to copy. Once it reaches that number, it stops copying. If you are copying to a 20-byte buffer, setting this value to 20 will ensure you do not overwrite that 20 byte buffer.

char buffer[20];

strncpy (buffer, "Hello", sizeof(buffer));

But there’s a problem… While strncpy() will blindly copy the source to the destination and then add the terminating 0 byte (without any care or concern to how much room is available at the destination), strncpy() will only add the terminating 0 if the source is smaller than the maximum size value.

This part I was aware of. In my earlier articles, I suggested this as a workaround to ensure the destination is always 0 terminated, even if the source string is as large or larger than the destination buffer:

char buffer[10];

strncpy (buffer, "1234567890", sizeof(buffer)-1); // subtract one
buffer[sizeof(buffer)-1] = '\0'; // 0 terminate

Above, sizeof(buffer) is 10. The copy would copy up to 9 bytes, so it would copy over “123456789”. Then, in the buffer at position 9 (bytes 0 to bytes 9 is ten bytes) it would place the zero.

Problem solved.

But that’s not important to this post.

Instead, there is another behavioral difference I wanted to mention — one that you likely already know. I must have know this and just forgotten. Surely I knew it. It even explains this in the C resource I use when looking up parameters to various C functions:

strncpy() will copy the bytes of the source buffer, and then pad the destination with zeros up until the maximum size value.

#include <stdio.h>
#include <string.h>

#define BUFFER_SIZE 32

void HexDump (const char *label, void *dataPtr, size_t dataSize);

int main()
{
    char buffer1[BUFFER_SIZE] = { 0 };
    char buffer2[BUFFER_SIZE] = { 0 };

    memset (buffer1, 0xff, sizeof(buffer1));
    memset (buffer2, 0xff, sizeof(buffer1));
    
    HexDump ("buffer1", buffer1, sizeof(buffer1));
    HexDump ("buffer2", buffer1, sizeof(buffer1));

    strcpy (buffer1, "Hello");
    strncpy (buffer2, "Hello", sizeof(buffer2));

    HexDump ("buffer1", buffer1, sizeof(buffer1));
    HexDump ("buffer2", buffer2, sizeof(buffer2));
    
    return 0;
}

void HexDump (const char *label, void *dataPtr, size_t dataSize)
{
    printf ("--- %s ---\n", label);
    for (unsigned int idx=0; idx<dataSize; idx++)
    {
        printf ("%02x ", ((unsigned char*)dataPtr)[idx]);
        if (idx % 16 == 15) printf ("\n");
    }
    printf ("\n");
}

Above, this program makes a buffer and sets all the bytes in it to 0xff (just so we can see the results easier).

It then uses strcpy() to copy a string (which will copy up until the 0 byte at the end of the source string), and strncpy() to copy the string to a second butter, with the maximum size specified as the destination buffer size.

Here is the output:

--- buffer1 ---
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 

--- buffer2 ---
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 

--- buffer1 ---
48 65 6c 6c 6f 00 ff ff ff ff ff ff ff ff ff ff 
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 

--- buffer2 ---
48 65 6c 6c 6f 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 

The first “buffer1” and “buffer2” shows the buffers initiated with 0xff’s.

The second “buffer1” shows the result of strcpy() – the five characters of “Hello” copied over, with a 0 byte added.

The second “buffer2” shows the same string copied using strncpy() with the size set to 16, the size of the buffer. You can see it copied the five characters of “Hello” and then filled the rest (up to the size) with 0 bytes.

Was this always the case with strncpy(), or did this get enhanced in later versions of C? I see this is fully documented at places like cplusplus.com:

Copy characters from string

Copies the first num characters of source to destination. If the end of the source C string (which is signaled by a null-character) is found before num characters have been copied, destination is padded with zeros until a total of num characters have been written to it.

No null-character is implicitly appended at the end of destination if source is longer than num. Thus, in this case, destination shall not be considered a null terminated C string (reading it as such would overflow).

– cplusplus.com entry on strncpy()

Why bring this up now?

The only reason I bring this up today is because I saw a new compiler warning I had never seen before recently.

warning: ‘strncpy’ output truncated before terminating nul copying X bytes from a string of the same length [-Wstringop-truncation]

That was the first time I’d ever seen this warning. It is a useful warning, since it informs you that the destination string will NOT be 0 terminated:

char buffer[10];
strncpy (buffer, "1234567890", 10);

And that reminded me of that strncpy() behavior, and made me change how I was using it in my program.

I also saw this variation:

warning: 'strncpy' output truncated copying X bytes from a string of length Y [-Wstringop-truncation]

This warning popped up when I had a source buffer that was longer than the destination. That should be fine, since strncpy() is told how much to copy over, max. I was puzzled why this would even be a warning. I mean, you specifically put the number in there that says “copy up to X bytes.”

I find it odd that the first message (exactly length) lets you know you don’t have a 0 terminated destination buffer, but the second just says “hey, we couldn’t copy as much as you request.”

Anyway, I found it interesting.

Until next time…