C语言 __attribute__((packed)) 对嵌套结构数组的影响？

Effects of __attribute__((packed)) on nested array of structures?

提问by mtalexan

The Problem

问题

I'm working on sending a raw structure over a network to a known program on the other side, but have to worry about the silently introduced memory used for aligning the structures (other issues like endianness are covered). What I'm working with is something like:

我正在通过网络将原始结构发送到另一端的已知程序，但不得不担心用于对齐结构的静默引入的内存（包括字节顺序等其他问题）。我正在使用的是这样的：

typedef struct __attribute__((packed))
{
   uint16_t field1;
   uint16_t field2;
   uint16_t field3;
} packed_1_s;

typedef struct __attribute__((packed))
{
   uint16_t fieldA;
   uint16_t fieldB;
   packed_1_s structArray[10];
} packed_2_s;

typedef struct __attribute__((packed))
{
   uint16_t fieldX;
   packed_2_s fieldY;
   uint8_t arrayZ[20];
} data_s;

I understand that normally the packed_1_s structure could/would have additional space allocated for every instance of the structure to fill it out to the compiler's favored size (dependent on the hardware it's being built for), and that favored size can be anywhere from 2 bytes to 64 bytes (most recently). Normally if I had a single instance of packed_1_s in packed_2_s there would be no problem, but I'm given to understand there's some differences when you try to put elements in an array.

我知道通常packed_1_s结构可以/应该为结构的每个实例分配额外的空间，以将其填充到编译器喜欢的大小（取决于它正在构建的硬件），并且喜欢的大小可以是2个字节到 64 字节（最近）。通常，如果我在packed_2_s 中有一个packed_1_s 的单个实例，那不会有问题，但是我了解到当您尝试将元素放入数组时存在一些差异。

Attempted Solutions

尝试的解决方案

The gcc documentation seems to suggest that by simply including the packed attribute in the packed_2_s definition, the fields, even if they're arrays, will all be as tightly packed as possible and won't add space to the packed_2_s structure to align the elements of the array. The documentation on the align() attribute though suggests that arrays are handled differently than other fields and need the align/packed attribute set on the field directly for it to modify the extra spacing added to match the specified alignment (or lack thereof). I tried setting the packed attribute on both the structArray field, and when that didn't work, did a test by setting the packed attribute on arrayZ in the code above:

gcc 文档似乎表明，通过在packed_2_s 定义中简单地包含packed 属性，字段，即使它们是数组，也将尽可能紧密地打包，并且不会向packed_2_s 结构添加空间以对齐元素的阵列。align() 属性的文档虽然表明数组的处理方式与其他字段不同，并且需要直接在字段上设置 align/packed 属性，以便修改添加的额外间距以匹配指定的对齐（或缺少对齐）。我尝试在 structArray 字段上设置packed属性，当这不起作用时，通过在上面的代码中在arrayZ上设置packed属性来进行测试：

packed_1_s structArray[10] __attribute__((packed));

uint8_t arrayZ[20] __attribute__((packed));

Both attempts gave me a compiler warning that the packed attribute wasn't understood in this context and would be skipped (good thing I build with "-Wall").

两次尝试都给了我一个编译器警告，即在此上下文中无法理解打包属性并将被跳过（我用“-Wall”构建的好东西）。

I hoped a way to get around the issue would be to use attribute align(1), indicating a desired alignment of 1 byte which is comparable to the packed attribute, but documentation says the align() attribute can only increasethe alignment and packed should be used to decreasethe alignment.

我希望解决这个问题的方法是使用属性 align(1)，表示所需的 1 字节对齐方式与打包属性相当，但文档说 align() 属性只能增加对齐方式，而打包应该用于减少对齐。

Considerations

注意事项

From what I can determine from the GCC documentation, it seems like there's 3 major cases of additional memory being inserted.

根据我从 GCC 文档中可以确定的信息，似乎有 3 种主要情况需要插入额外的内存。

1. Structure contents may have additional memory allocated to the structure itself to change the spacing between fields.
   Effectively, the definition of the memory map of the structure contents within the structure may change (though not the order of the elements).
2. Structures may have additional memory allocated to them to fill them out to a more efficient overall size. This is generally intended so other variables coming after a declaration of one of their instances doesn't fall within the same "block" as the structure instance where a "block" is defined by the system/compiler.
3. Arrays, whether within a structure or not, may have additional memory added to them to shift the elements to an efficient alignment.

1. 结构内容可能有额外的内存分配给结构本身，以更改字段之间的间距。
   实际上，结构内结构内容的内存映射的定义可能会改变（尽管不是元素的顺序）。
2. 结构可能有额外的内存分配给它们，以将它们填充到更有效的整体大小。这通常是为了使在声明其实例之一之后出现的其他变量不属于与系统/编译器定义“块”的结构实例相同的“块”。
3. 数组，无论是否在结构内，都可能会添加额外的内存以将元素转换为有效对齐。

As far as I can tell, the packed attribute can be used to affect the structures and block the additional memory added in case 1 and 2 above, but there doesn't seem to be a way to handle case 3 above on my compiler(s).

据我所知，packed 属性可用于影响结构并阻止在上面的情况 1 和 2 中添加的额外内存，但似乎没有办法在我的编译器上处理上面的情况 3 ）。

The Question

问题

Is there any way to guarantee that the data_s structure will have absolutely no additional space added to it or any of its sub-structures so I don't have compiler dependent shifts in the memory map? Am I misunderstanding the cases where the compiler can insert space to intentionally shift the memory map?

有没有办法保证 data_s 结构绝对不会向它或其任何子结构添加额外的空间，所以我在内存映射中没有依赖于编译器的移位？我是否误解了编译器可以插入空间来故意移动内存映射的情况？

EDIT

编辑

I discussed some of the issues with my local guru and it sounds like I have some misunderstanding of case 3 above. The elements in the array don't have space inserted between them, but the additional space to guarantee they align correctly is added to the structure itself. Apparently this suggests things like "sizeof(structureOnlyContaining_uint32_t)" won't always return "4" since additional space may be added to align the uint32_t datatype on the compiler being used. The result is that there are really only 2 cases:

我和我当地的大师讨论了一些问题，听起来我对上面的案例 3 有一些误解。数组中的元素之间没有插入空间，但确保它们正确对齐的额外空间被添加到结构本身。显然，这表明“sizeof(structureOnlyContaining_uint32_t)”之类的东西不会总是返回“4”，因为可能会添加额外的空间来对齐正在使用的编译器上的 uint32_t 数据类型。结果是真的只有两种情况：

1. Larger offsets in between fields in the memory map of the structure.
   The space between the fields can be modified to align each field. This can be changed using the packed or align() attributes.
2. Structure end padding. The size for a structure, as returned by sizeof(), can be modified so arrays of the structures end up correctly aligned for the system. This allows all systems to assume that the start of structures will always be aligned, causing issues if they aren't. This seems unaffected by the pack or align attributes.

1. 结构的内存映射中的字段之间的较大偏移量。
   可以修改字段之间的空间以对齐每个字段。这可以使用packed 或align() 属性进行更改。
2. 结构末端填充。可以修改由 sizeof() 返回的结构的大小，以便结构的数组最终为系统正确对齐。这允许所有系统假设结构的开始总是对齐的，否则会导致问题。这似乎不受 pack 或 align 属性的影响。

Because of the new case 2, elements of an array in a structure don't necessarily obey the packed or align() attributes specified on the structure, though the start of the array and the field immediately following the array do.

由于新的情况 2，结构中数组的元素不一定遵守结构上指定的打包或 align() 属性，尽管数组的开头和数组后面的字段会这样做。

My question is then about how to deal with the structArray in packed_2_s since the size of the array as a whole cannot be guaranteed purely by the packed attribute. Is there some way to guarantee the fixed size of the structArray field as a whole? It should be noted that I can't increase the size of the packed_1_s too much since the data_s struct needs to be kept as small as possible (its replacing Audio/Video data in a streaming scenario).

我的问题是关于如何处理packed_2_s中的structArray，因为整个数组的大小不能完全由packed属性来保证。有什么方法可以保证整个 structArray 字段的大小固定吗？应该注意的是，我不能过多地增加packed_1_s 的大小，因为data_s 结构需要保持尽可能小（它在流场景中替换音频/视频数据）。

回答by Ambroz Bizjak

Note the following points about __attribute__((packed)):

请注意以下几点__attribute__((packed))：

• When packedis used in a structure declaration, it will compress its fields such, such that, sizeof(structure) == sizeof(first_member) + ... + sizeof(last_member).

• Here, an array is just one memberof the struct. Packing the containing structure of an array will not change the array's size. In fact, the size of (any) array is alwayssizeof(element) * number_of_elements.

• Similarly, packing a containing structure of an inner structure will not change the size of the inner structure. The size of a structure is completely determined by its declaration, and is the same no matter where you use.

• Packing a structure will make its required alignment one byte (i.e. it can be placed anywhere in memory).

• Packing will introduce alignment issues when accessing the fields of a packed structure. The compiler will account for that when the the fields are accessed directly, but not when they are accessed via pointers. Of course, this does not apply to fields with required alignment one (such as char's or other packed structures). See my answer to a similar question, which includes a program demonstrating the problem with accessing members via pointers.

• 当packed在一个结构声明时，它会压缩其字段，例如，使得的sizeof（结构）==的sizeof（first_member）+ ... +的sizeof（last_member）。

• 在这里，数组只是结构体的一个成员。打包数组的包含结构不会改变数组的大小。实际上，（任何）数组的大小始终是sizeof(element) * number_of_elements。

• 同样，填充内部结构的包含结构不会改变内部结构的大小。结构体的大小完全由它的声明决定，无论在哪里使用都是一样的。

• 打包一个结构将使其所需的对齐为一个字节（即它可以放置在内存中的任何位置）。

• 访问打包结构的字段时，打包会引入对齐问题。当直接访问字段时，编译器会考虑到这一点，而不是通过指针访问它们时。当然，这不适用于需要对齐的字段（例如字符或其他压缩结构）。请参阅我对类似问题的回答，其中包括一个演示通过指针访问成员的问题的程序。

Finally, to answer the question,

最后，回答这个问题，

Is there any way to guarantee that the data_s structure will have absolutely no additional space added to it or any of its sub-structures so I don't have compiler dependent shifts in the memory map?

有没有办法保证 data_s 结构绝对不会向它或其任何子结构添加额外的空间，所以我在内存映射中没有依赖于编译器的移位？

Yes. Declare the structure as packed, and also all structures that it contains, recursively.

是的。递归地将结构声明为压缩结构，以及它包含的所有结构。

Also note that the packed attribute applies to a structure declaration, and not to a type. There's no such thing as packed version of a structurethat is declared non-packed. When you use a structure somewhere, it (its members) will be packed if and only if the structure itself was declared packed. This is kind of implied by the fact that the size of a structure is completely determined by its declaration.

另请注意，packed 属性适用于结构声明，而不适用于类型。没有被声明为非压缩结构的压缩版本。当您在某处使用结构时，当且仅当结构本身被声明为已打包时，它（其成员）才会被打包。结构的大小完全由其声明决定这一事实暗示了这一点。

UPDATE: For some reason you're still confused about arrays. The solution I provided (declare all structures packed) works with arrays too. For example:

更新：出于某种原因，您仍然对数组感到困惑。我提供的解决方案（声明所有结构打包）也适用于数组。例如：

struct elem_struct {
    uint32_t x;
} __attribute__((packed));
// packed guarantees that sizeof(struct elem_struct) = sizeof(uint32_t) = 4

struct array_struct {
    struct elem_struct arr[10];
} __attribute__((packed));
// packed guarantees that sizeof(struct array_struct) =
// = sizeof(struct elem_struct[10]) = 10 * sizeof(struct elem_struct)
// = 10 * 4 = 40

The two additional points you made about arrays are true - but only when the structures are not packed. Packing forces the fields of the struct to be continuous, and this doescreate alignment issues which, if no packing was used, would be solved by inserting empty space between members and padding the struct (see the point I already raised about alignment).

您对数组提出的另外两点是正确的 - 但仅当结构未打包时。打包强制结构的字段是连续的，这确实会产生对齐问题，如果不使用打包，将通过在成员之间插入空白空间并填充结构来解决（请参阅我已经提出的关于对齐的观点）。