Sunday, 22 April 2012

Struct member alignment errors

Suspicious member variables values? Uninitialized fields? Stack corruption? <Bad ptr>? These symptoms are likely signs of structure’s alignment conflicts. This is when parts of the program are compiled with different alignment settings, and then communicate and use each other as if nothing had happened.

Even if you didn't bother with alignment settings at all, some 3d-party modules included in your project might joyously do it on your behalf.

So, what is structure member alignment all about? Simple example.
class A
{
public:
    void CalculateVars();
private:
    long   m_Var1;
    double m_Var2;
};
While it is easy to see that the size of this class should be 12 (sizeof(long) + sizeof(double)), sizeof(A) would yield 16, because default struct member alignment is 8 with Visual Studio 2008. That is why the compiler adds 4 byte empty memory space after long member variable.  

So far so good. But what if different parts of program would have different member alignment values? This may lead to weird program behaviour. Consider this code:
//A.h
class A
{
public:
    void CalculateVars();
private:
    long   m_Var1;
    double m_Var2;
};
//A.cpp #include "A.h"
void A::CalculateVars() { int size2 = sizeof(*this); m_Var1 = 5; m_Var2 = 4.43; }
//Main.cpp #include "stdafx.h" #pragma pack(4) #include "A.h"
int _tmain(int argc, _TCHAR* argv[]) { int size1 = sizeof(A); A a; a.CalculateVars(); return 0; }

Compiling and running this program would 1) make m_Var2 = 8.6792272474287212e+209 2) produce "Run-Time Check Failure #2 - Stack around the variable 'a' was corrupted" message at exit. This is because inside Main.cpp I maliciously put #pragma that changes alignment to 4 bytes. This way inside Main.cpp class A size is 12 bytes while inside CalculateVars() it is thought to be 16 bytes. As it can be seen, A.cpp knows nothing about the change Main.cpp made to alignment, as it was compiled with the default-from-the-compiler-settings 8 bytes alignment.
This picture shows how the class looks in reality and in CalculateVars()' delusion.
alignment_pic2

It is obvious now  that CalculateVars() writes to m_Var2 at 8 bytes-offset, and gets right into the middle of double field, producing invalid value for it; what's more, as it writes 8 bytes there - the stack would corrupt due to 4 bytes overwrite.

Sometimes it is hard to debug memory-alignment issues, especially if program doesn't tend to fail early. It just works and works, with a few invalid values waiting for an ocassion to become noticed. So it would be nice if compiler would at least throw a warning. Given the code above cl compiler wouldn't warn. However, if I move this nasty #pragma pack in a header, it throws "warning C4103: alignment changed after including header, may be due to missing #pragma pack(pop)". This suggests to me to never put #pragma pack in implementation files, only in headers.

3d-party libraries with different alignment options should normally keep those changes local to themselves, and restore previous memory alignment at the end of their headers (using #pragma pop). For example, when I used stlport library it did just that. It was said that stlport libs where built with 8-byte alignment, and devs wanted to ensure that these libs would be used with identical alignment. Their <vector> header looked something like this:
pragma pack(push,8)
...
# include <stl/_vector.h>
...
pragma pack(pop)

Otherwise the alignment would change for the rest of the included headers, like in the picture below.
alignment_pic1

To conclude, when there is suspicion of structure alignment issue in code, I’d advice to first test structure size with sizeof(*this) in place of problem. Correspondingly, go down the call stack to an object that called the function from disputed class and check its size there. If sizes do not mach then it's likely an alignment issue. Same as in the above code, when sizeof(A) and sizeof(*this) were different

Also, when including someone else's headers keep a close watch on  warning C4103.

Friday, 6 April 2012

Using sox to batch convert .wav files to .raw

I was in need to convert a lot of wav files to raw format (for use within Marmalade SDK). As it is adviced in http://www.immortalcode.com/2010/09/notes-on-running-soxexe-to-prepare-raw.html I decided to use sox command-line utility for audio manipulation. It is easy to convert separate audio file with sox, but to convert all files in folder at once one should write a batch file. I found a batch-example.bat in sox folder and I have modified it in this way:

cd %~dp0
mkdir converted
FOR %%A IN (%*) DO sox %%A -e signed-integer -b 16 "converted/%%~nA.raw" rate 22050 channels 1
pause

Then you drag-n-drop a group of wav files onto this .bat, and voila the subfolder \converted will emerge in the sox folder. This  folder will contain all the converted files, with the same names, but with .raw extensions.