The year is 2004.The ring-3 debuggers are used often and often.Since they offer Windows GUI they are more handy instead of the ring-0 debuggers (like SoftIce).In this essay i will talk (write) about the detection of one of the best ring-3 debuggers - OllyDbg.Many have heard of the IsDebbugerPresent and of the fs:[20] detecting tricks, but what about some other new ones? Here I will present you some of my own detecting tricks.I will give you the general explanation so you would be able to use your fantasy to improve it yourself.
Method I: FindWindow
This method is based on the FindWindows function.As all dialogs the OllyDbg's main dialog (window ?) has its caption and Class name.Using this Api we will try to find if the OllyDbg's main window is opened.Microsoft wrote:
The FindWindow function retrieves the handle of the top-level window whose class name and window name match the specified strings. This function does not search child windows
HWND FindWindow(
LPCTSTR lpClassName, // address of class name
LPCTSTR lpWindowName // address of window name
);
Parameters
lpClassName
Points to a null-terminated string that specifies the class name or is an atom that identifies the class-name string. If this parameter is an atom, it must be a global atom created by a previous call to the GlobalAddAtom function. The atom, a 16-bit value, must be placed in the low-order word of lpClassName; the high-order word must be zero.
lpWindowName
Points to a null-terminated string that specifies the window name (the window's title). If this parameter is NULL, all window names match.
Return Value
If the function succeeds, the return value is the handle of the window that has the specified class name and window name.
If the function fails, the return value is NULL. To get extended error information, call GetLastError
This is an interesting method.It is based on 4 apis (CreateToolhelp32Snapshot, Process32First, Process32Next, GetCurrentProcessId) and 1 structure ( PROCESSENTRY32 ).I will tell you what MSDN says first:
[CreateToolhelp32Snapshot]
The CreateToolhelp32Snapshot function takes a snapshot of the specified processes in the system, as well as the heaps, modules, and threads used by these processes.
dwFlags
[in] Portions of the system to include in the snapshot. This parameter can be one of the following values.
TH32CS_INHERIT - Indicates that the snapshot handle is to be inheritable.
TH32CS_SNAPALL - Includes all processes and threads in the system, plus the heaps and modules of the process specified in th32ProcessID.
TH32CS_SNAPHEAPLIST - Includes all heaps of the process specified in th32ProcessID in the snapshot. To enumerate the heaps, see Heap32ListFirst.
TH32CS_SNAPMODULE - Includes all modules of the process specified in th32ProcessID in the snapshot. To enumerate the modules, see Module32First.
TH32CS_SNAPPROCESS - Includes all processes in the system in the snapshot. To enumerate the processes, see Process32First.
TH32CS_SNAPTHREAD - Includes all threads in the system in the snapshot. To enumerate the threads, see Thread32First.
th32ProcessID
[in] Process identifier of the process to be included in the snapshot. This parameter can be zero to indicate the current process. This parameter is used when the TH32CS_SNAPHEAPLIST, TH32CS_SNAPMODULE, or TH32CS_SNAPALL value is specified. Otherwise, it is ignored and all processes are included in the snapshot.
Return Values
If the function succeeds, it returns an open handle to the specified snapshot.
If the function fails, it returns INVALID_HANDLE_VALUE. To get extended error information, call GetLastError.
Remarks
The snapshot taken by this function is examined by the other tool help functions to provide their results. Access to the snapshot is read only. The snapshot handle acts like an object handle and is subject to the same rules regarding which processes and threads it is valid in.
To enumerate the heap or module states for all processes, specify TH32CS_SNAPALL and set th32ProcessID to zero. Then, for each additional process in the snapshot, call CreateToolhelp32Snapshot again, specifying its process identifier and the TH32CS_SNAPHEAPLIST or TH32_SNAPMODULE value.
To destroy the snapshot, use the CloseHandle function.
[Process32First]
Process32First retrieves information about the first process encountered in a system snapshot.
hSnapshot
[in] Handle to the snapshot returned from a previous call to the CreateToolhelp32Snapshot function.
lppe
[in, out] Pointer to a PROCESSENTRY32 structure.
Return Values
Returns TRUE if the first entry of the process list has been copied to the buffer or FALSE otherwise. The ERROR_NO_MORE_FILES error value is returned by the GetLastError function if no processes exist or the snapshot does not contain process information.
Remarks
The calling application must set the dwSize member of PROCESSENTRY32 to the size, in bytes, of the structure. Process32First changes dwSize to the number of bytes written to the structure. This will never be greater than the initial value of dwSize, but it may be smaller. If the value is smaller, do not rely on the values of any members whose offsets are greater than this value.
To retrieve information about other processes recorded in the same snapshot, use the Process32Next function.
[Process32Next]
Process32Next Retrieves information about the next process recorded in a system snapshot.
hSnapshot
[in] Handle to the snapshot returned from a previous call to the CreateToolhelp32Snapshot function.
lppe
[out] Pointer to a PROCESSENTRY32 structure.
Return Values
Returns TRUE if the next entry of the process list has been copied to the buffer or FALSE otherwise. The ERROR_NO_MORE_FILES error value is returned by the GetLastError function if no processes exist or the snapshot does not contain process information.
Remarks
To retrieve information about the first process recorded in a snapshot, use the Process32First function.
[PROCESSENTRY32]
PROCESSENTRY32 describes an entry from a list that enumerates the processes residing in the system address space when a snapshot was taken.
dwSize
Size of the structure, in bytes. Before calling the Process32First function, set this member to sizeof (PROCESSENTRY32). If you do not initialize dwSize, Process32First will fail.
cntUsage
Number of references to the process. A process exists as long as its usage count is nonzero. As soon as its usage count becomes zero, a process terminates.
th32ProcessID
Identifier of the process.
th32DefaultHeapID
Identifier of the default heap for the process. The contents of this member has meaning only to the tool help functions. It is not a handle, nor is it usable by functions other than the ToolHelp functions.
th32ModuleID
Module identifier of the process. The contents of this member has meaning only to the tool help functions. It is not a handle, nor is it usable by functions other than the ToolHelp functions.
cntThreads
Number of execution threads started by the process.
th32ParentProcessID
Identifier of the process that created the process being examined.
pcPriClassBase
Base priority of any threads created by this process
.
dwFlags
Reserved; do not use.
szExeFile
Pointer to a null-terminated string that specifies the name of the executable file for the process.
Windows Me/98/95: The file name includes the path.
[GetCurrentProcessId]
The GetCurrentProcessId function returns the process identifier of the calling process.
DWORD GetCurrentProcessId(VOID)
Parameters
This function has no parameters.
Return Value
The return value is the process identifier of the calling process.
Remarks
Until the process terminates, the process identifier uniquely identifies the process throughout the system.
Here our target is the parent procces of our program.We check if the parent process is OllyDbg.
The plan:
1.) Get the ProcessId of our program using GetCurrentProcessId .
2.) Begin a loop of CreateToolhelp32Snapshot and Process32First/Next and compare every th32ProcessID member of the PROCESSENTRY32 structure with our ProcessId from GetCurrentProcessId till we find it.
3.) Get the value of the th32ParentProcessID member of the PROCESSENTRY32.
4.) Begin a new CreateToolhelp32Snapshot and Process32First/Next loop, but this time compare the th32ProcessID member with the th32ParentProcessID, which we got in 3.) till we find it.
5.) Go to the szExeFile member of the PROCESSENTRY32 and scan the name string for "Ollydbg.exe"
6.) If we find it we will know that our program is running under OllyDbg.
And here is my example:
.586
.model flat, stdcall
option casemap:none
include d:\masm32\INCLUDE\Windows.inc
include d:\masm32\INCLUDE\user32.inc
include d:\masm32\INCLUDE\kernel32.inc
includelib d:\masm32\lib\user32.lib
includelib d:\masm32\lib\kernel32.lib
.data
strCaption db "OllyDbg Detector!",0
strFound db "OllyDbg found!",0
strNotFound db "OllyDbg NOT found!",0
strOllyDbg db "OLLYDBG.EXE",0h
valCurrentPiD dd 0
valParentPiD dd 0
hSnapShot dd 0
;Get the ProcessId of the Current Process
invoke GetCurrentProcessId
mov valCurrentPiD,eax
lea esi,offset proces
assume esi:ptr PROCESSENTRY32
mov [esi].dwSize,sizeof PROCESSENTRY32
;Begin the first Loop and find the current process
;using the valCurrentPiD
invoke Process32First,hSnapShot,addr proces
lea esi,offset proces
assume esi:ptr PROCESSENTRY32
mov ebx,valCurrentPiD
cmp ebx,[esi].th32ProcessID
jne nope1
nope1:
invoke Process32Next,hSnapShot,addr proces
lea esi,offset proces
assume esi:ptr PROCESSENTRY32
mov ebx,valCurrentPiD
cmp ebx,[esi].th32ProcessID
jne nope1
push [esi].th32ParentProcessID
pop valParentPiD
invoke CloseHandle,hSnapShot
; Create the snapshot again
invoke CreateToolhelp32Snapshot, TH32CS_SNAPPROCESS,NULL
mov hSnapShot,eax
mov [esi].dwSize,sizeof PROCESSENTRY32
; Begin the second Loop and find the parent
; process using the valParentPiD
invoke Process32First,hSnapShot,addr proces
lea esi,offset proces
assume esi:ptr PROCESSENTRY32
mov ebx,valParentPiD
cmp ebx,[esi].th32ProcessID
jne nope2
nope2:
invoke Process32Next,hSnapShot,addr proces
lea esi,offset proces
assume esi:ptr PROCESSENTRY32
mov ebx,valParentPiD
cmp ebx,[esi].th32ProcessID
jne nope2
; Extract filename of the Parent Process from the whole string
lea eax, [esi].szExeFile
push eax
invoke lstrlen,eax
sub eax,11
pop ebx
add ebx,eax
; Case Upper the string and compare it with "OLLYDBG.EXE"
invoke CharUpper,ebx
invoke lstrcmp,ebx,addr strOllyDbg
This is a very interesting method of detecting OllyDbg.It is based on Microsoft's SetUnhandledExceptionFilter function:
The SetUnhandledExceptionFilter function lets an application supersede the top-level exception handler that Win32 places at the top of each thread and process.
After calling this function, if an exception occurs in a process that is not being debugged, and the exception makes it to the Win32 unhandled exception filter, that filter will call the exception filter function specified by the lpTopLevelExceptionFilter parameter.
LPTOP_LEVEL_EXCEPTION_FILTER lpTopLevelExceptionFilter // exception filter function
);
Parameters
lpTopLevelExceptionFilter
Supplies the address of a top-level exception filter function that will be called whenever the UnhandledExceptionFilter function gets control, and the process is not being debugged. A value of NULL for this parameter specifies default handling within UnhandledExceptionFilter.
The filter function has syntax congruent to that of UnhandledExceptionFilter: It takes a single parameter of type LPEXCEPTION_POINTERS, and returns a value of type LONG. The filter function should return one of the following values:
Value Meaning
EXCEPTION_EXECUTE_HANDLER
Return from UnhandledExceptionFilter and execute the associated exception handler. This usually results in process termination.
EXCEPTION_CONTINUE_EXECUTION
Return from UnhandledExceptionFilter and continue execution from the point of the exception. Note that the filter function is free to modify the continuation state by modifying the exception information supplied through its LPEXCEPTION_POINTERS parameter.
EXCEPTION_CONTINUE_SEARCH
Proceed with normal execution of UnhandledExceptionFilter . That means obeying the SetErrorMod flags, or invoking the Application Error pop-up message box.
Return Value
The SetUnhandledExceptionFilter function returns the address of the previous exception filter established with the function. A NULL return value means that there is no current top-level exception handler.
Remarks
Issuing SetUnhandledExceptionFilter replaces the existing top-level exception filter for all existing and all future threads in the calling process.
The exception handler specified by lpTopLevelExceptionFilter is executed in the context of the thread that caused the fault. This can affect the exception handler's ability to recover from certain exceptions, such as an invalid stack.
So.The main idea is that we set an exception filter with this api function.After that we cause a small and shy exception.If you have read the blue paragraph you will understand that if the program is debugged the exception will not be passed to the exception handler.We will use exactly this! The debugger will not know what to do and the program will crash! If the program is running standalone it will work normally.
What do i understand under normally? It will enter the Exception Handler, in there it will change the EIP value from the context structure with the address of the next safe command, which will be used to continue the program from there.Before I give you my example I will need to explain you the basic of Exception Handling.When the program enters the Exception Handler the stack looks like this:
At the end of the handler EAX must contain the value which tells Windows what to do next.For us this will be:
EXCEPTION_CONTINUE_EXECUTION equ -1
So here is the plan of the protection:
1.) Install the Exception Handler using the SetUnhandledExceptionFilter Api
2.) Cause exception and go to the Exception Handler only if there is not a debugger.
3.) In the handler - get from stack the pointer to the EXCEPTION_POINTERS structure
4.) Get from the EXCEPTION_POINTERS structure the pointer to the CONTEXT structure
5.) Change the regEip member of the CONTEXT to the safe address, from where the program will continue
6.) Set the EXCEPTION_CONTINUE_EXECUTION flag in EAX
7.) Normalize stack and continue to safe address
And here is the example:
.586
.model flat, stdcall
option casemap:none
include d:\masm32\INCLUDE\Windows.inc
include d:\masm32\INCLUDE\user32.inc
include d:\masm32\INCLUDE\kernel32.inc
includelib d:\masm32\lib\user32.lib
includelib d:\masm32\lib\kernel32.lib
.data
strCaption db "OllyDbg Detector",0
strNotFound db "OllyDbg NOT found!",0
mov ebx,dword ptr [0FFFFFFFFh] ;Exception is here!
safe_address:
invoke ExitProcess,0
end start
Method IV: APi Redirection
One of my own tricks.It is based on the way OllyDbg handles called api functions.Oleh Yuschuk (the author of OllyDbg) has decided to use total api redirection in his debugger, when the debugged program is calling an api function.Here is an example:
So it seems that he manually loads the Import Table and manually fills the IAT, where all api addresses are redirected to an allocated buffer.In this buffer he calls the api functions in a strange way.But there is one more thing, more important for us.He also emulates (?) the GetProcAddress function so it returnes redirected addresses.For example the real address of IsDebuggerPresent in memory is BFF946F6, but the function returns 87FF4110, which is the redirected address.How are we going to use this as OllyDbg detection trick? Well in many ways.I will give you the shortest.
The plan:
1.) Load the kernel32.dll Library, which will return its Image Base (the location where it is loaded in memory)
2.) Call GetProcAddress for function ExitProcess
3.) Compare the return value (address of api function) from GetProcAddress with the Image Base of kernel32.dll
4.) If it is greater than the Image Base we will be sure that the api function is called directly from kernel32.dll.Else we will be sure that there is api redirection.
The example:
.586
.model flat, stdcall
option casemap:none
include d:\masm32\INCLUDE\Windows.inc
include d:\masm32\INCLUDE\user32.inc
include d:\masm32\INCLUDE\kernel32.inc
includelib d:\masm32\lib\user32.lib
includelib d:\masm32\lib\kernel32.lib
.data
strCaption db "OllyDbg Detector",0
strFound db "OllyDbg found!",0
strNotFound db "OllyDbg NOT found!",0
strLibrary db "kernel32.dll",0
strFunction db "ExitProcess",0
Many have heard of the IsDebbugerPresent and of the fs:[20] detecting tricks, but what about some other new ones? Here I will present you the other wasy to detect OllyDbg.
Method I: FindWindow
This method is based on the FindWindows function.As all dialogs the OllyDbg's main dialog
(window ?) has its caption and Class name.Using this Api we will try to find if the OllyDbg's main window is opened.Microsoft wrote:
The FindWindow function retrieves the handle of the top-level window whose class name and window name match the specified strings. This function does not search child windows
LPCTSTR lpClassName, // address of class name 窗口类名的地址。//我理解为是指向字符串的指针
LPCTSTR lpWindowName // address of window name指向一个指定了窗口名(窗口标题)的空结束字符串
Parameters //参数
lpClassName //窗口类名
Points to a null-terminated string that specifies the class name or is an atom that identifies the class-name string. If this parameter is an atom, it must be a global atom created by a previous call to the GlobalAddAtom function. The atom, a 16-bit value, must be placed in the low-order word of lpClassName; the high-order word must be zero.
指向一个指定了类名的空结束字符串,或一个标识类名字符串的成员的指针。如果该参数为一个成员,则它必须为前次调用theGlobafAddAtom函数产生的全局成员。该成员为16位,必须位于IpClassName的低 16位,高位必须为 0。
lpWindowName //窗口名(窗口标题)
Points to a null-terminated string that specifies the window name (the window's title). If this parameter is NULL, all window names match.
指向一个指定了窗口名(窗口标题)的空结束字符串。如果该参数为空,则为所有窗口全匹配。
Return Value
If the function succeeds, the return value is the handle of the window that has the specified class name and window name.
If the function fails, the return value is NULL. To get extended error information, call GetLastError