Investigation of application end crash in _CRT_INIT

Never seen?

The application does it's work, but crashes on exit. One clicks on the windows close box or selects "Exit" from menu and - ooops! - the MessageBox with read access vioaltion on the address 0xFFFFFFFC or so.

Well, the crash on exit is mostly not so dangerous as in session progress, but somehow annoying.

Where to search?

Let's investigate it: start WinDbg and attach to the application. Invoke application end and you'll get break on the similar code position:

(500.298): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=008dac60 ebx=30000000 ecx=00000000 edx=04790003 esi=fffffffc edi=00000000eip=3002eaef esp=0012f7e8 ebp=0012f808 iopl=0 nv up ei ng nz na po nccs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010286
MyDLL!_CRT_INIT+0x83:3002eaef 8b0e mov ecx,[esi] ds:0023:fffffffc=????????

What does it mean? Your application uses a DLL MyDLL and crashes trying to unload this DLL.
Why?

Look into disassembly window:

3002ead2 eb3d jmp MyDLL!_CRT_INIT+0xa5 (3002eb11)
3002ead4 85c0 test eax,eax
3002ead6 7539 jnz MyDLL!_CRT_INIT+0xa5 (3002eb11)
3002ead8 a154be0630 mov eax,[HoFa32!__onexitbegin (3006be54)]
3002eadd 85c0 test eax,eax
3002eadf 7430 jz MyDLL!_CRT_INIT+0xa5 (3002eb11)
3002eae1 8b0d50be0630 mov ecx,[HoFa32!__onexitend (3006be50)]
3002eae7 56 push esi
3002eae8 8d71fc lea esi,[ecx-0x4]
3002eaeb 3bf0 cmp esi,eax
3002eaed 7212 jb MyDLL!_CRT_INIT+0x95 (3002eb01)
3002eaef 8b0e mov ecx,[esi] ds:0023:fffffffc=????????
3002eaf1 85c9 test ecx,ecx

What's that? I didn't implement any _CRT_INIT methods in MyDLL? Why does it crash here?

Here is it.

MyDLL is a DLL using MS C-Runtime: the _CRT_INIT comes from C-Runtime stub and will be added automatically to any DLL built with MC C-Runtime stub (refer to Platform SDK for more info).

The function _CRT_INIT is called on each attach to a process or a thread and is repsonsible for store of pointers to methods to be called on DLL start and exit. These methods may be chained, so two internal variables: __onexitend and __onexitbegin.

At DLL exit the C-Runtime stub perfroms sequential calls to registered methods as follows:


/*
* do _onexit/atexit() terminators
* (if there are any)
*
* These terminators MUST be executed in
* reverse order (LIFO)!
*
* NOTE:
* This code assumes that __onexitbegin
* points to the first valid onexit()
* entry and that __onexitend points
* past the last valid entry. If
* __onexitbegin == __onexitend, the
* table is empty and there are no
* routines to call.
*/
if (__onexitbegin) {
while ( --__onexitend >= __onexitbegin )
/*
* if current table entry is not
* NULL, call thru it.
*/
if ( *__onexitend != NULL ) ' <-- here occures the crash!
(**__onexitend)();
/*
* free the block holding onexit table to
* avoid memory leaks. Also zero the ptr
* variable so that it is clearly cleaned up.
*/
_free_crt ( __onexitbegin ) ;
__onexitbegin = NULL ;
}
}

The crash occures due to invalid pointer stored in __onexitend.

How does it come?

The both __onexitend and __onexitbegin resides in memory space of MyDLL. Somewhere in MyDLL code is a memory write operation (for example, varable assignement or array copy or so), that overwrites one or both of these variables with unpredictable values loosing original pointers.
At DLL _CRT_INIT just checks if content of __onexitend is not NULL: having a garbage value due to former overwrite operation causes try to call content of pointed memory fragment as CPU instruction - with read access violation as result.

How to investigate?

Well, if the content is overwritten, it is to late to cure. To find out, what portion of code causes the overwrite action, you need WinDbg and symbols for your application modules - at least of MyDLL, since frequently the local DLL methods are responsible for this boundary violation. So perform following steps:

1. Start WinDbg and open your application as executable.
2. WinDbg halts on start - here you have a chance to set first breakpoint. Set the breakpoint to _CRT_INIT of your DLL:

>bp MyDLL!_CRT_INIT

MyDLL is not loaded yet, so the breakpoint will be set as deferred:

Bp expression 'MyDLL!_CRT_INIT' could not be resolved, adding deferred bp

3. Resume the application and wait until your DLL will be loaded.
4. Some time later your DLL will be loaded and breakpoint hit occures

Breakpoint 0 hit
eax=00000000 ebx=30000000 ecx=000067e8 edx=7c91eb94 esi=00000001 edi=00000000
eip=3002ea6c esp=0012dc18 ebp=0012dc34 iopl=0 nv up ei pl zr na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
MyDLL!_CRT_INIT:
3002ea6c 8b442408 mov eax,[esp+0x8] ss:0023:0012dc20=00000001

5. Now you can set breakpoint on access operation for __onexitend and __onexitbegin:

> ba w4 MyDLL!__onexitend
> ba w4 MyDLL!__onexitbegin

6. You do not need the breakpoint on _CRT_INIT anymore (you used it to set access breakpoints), so you can disable or clear it:

>bd 0

7. Resume application and wait until breakpoint hit:

Breakpoint 1 hit
eax=ffffffff ebx=0000007b ecx=0012dd84 edx=0012dd84 esi=3006be50 edi=3004d84aeip=77c1466e esp=0012db58 ebp=0012dd70 iopl=0 nv up ei pl zr na po nccs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINNT\system32\msvcrt.dll - msvcrt!wscanf+0x23a8:77c1466e e985fbffff
jmp msvcrt!wscanf+0x1f32 (77c141f8)

8. You've got it! This is the code place the content of internal system variables got overwritten. Analyze callstack to find out the calling method where the overwrite occures:

> kp
ChildEBP RetAddr
0012bfbc 77c11ba9 msvcrt!_input+0x97c
0012bff0 3000198a msvcrt!sscanf+0x37
00000000 00000000 MyDLL!process_answer+0x7a

OK, the overwrite happens in the method process_answer in MyDLL. Now you have enough info to navigate to the source code for bugfixing.
Keep memory safe! Enjoy.