OllyDbg 2.0
April 17, 2007 - Command search.
Finally I have finished the command search module. Basically, you supply a pattern, like XOR EAX,EAX, and OllyDbg locates all such commands in the memory block. Version 1.xx already featured this, but in a very limited form. For every supplied pattern, old program created the set of code/mask pairs and compared them with the binary code. This approach is simple and quick but features several drawbacks that strongly limited its usefullness. For example, if command is expected to have several prefixes, one must create models for any combination. But the main problem was that code/mask approach was unable to handle memory addresses. x86 addressing model is extremely complex and inhomogeneous, with many exceptions from the regular pattern. Let's take, for example, MOV EAX,[EBX]. There are 16 (sixteen) different binary encodings:
8B03 - the simplest form 8B43 00 - form without SIB with 1-byte zero displacement
8B83 00000000 - form without SIB with 4-byte displacement
8B0423 - form with SIB byte without scaled index

8B0463 - same
8B04A3 - same
8B04E3 - same 8B4423 00 - SIB byte, 1-byte displacement, no index 8B4463 00 - same
8B44A3 00 - same

8B44E3 00 - same

8B8423 00000000 - SIB byte, 4-byte displacement, no index

8B8463 00000000 - same

8B84A3 00000000 - same

8B84E3 00000000 - same

8B041D 00000000 - SIB byte, 4-byte displacement, scale 1, no base

Amazing, no?.. All attempts to reuse the old concept in the new OllyDbg version were in vain, so I was forced to throw it away. New model consists of the opcode, list of prefixes and packed description of operands. Search routine disassembles executable code and compares result with the model. Due to the very fast disassembler, this approach is almost as fast as the old one, but unbelievably flexible!
New search supports more pseudoelements than in the previous version:
R8 - any 8-bit register
R16 - any 16-bit register R32 - any 32-bit register REG - any general register (size is not important, assumed R32 in address) RA,RB - semi-defined 32-bit registers SEG - any segment register FPUREG - any floating-point register MMXREG - any MMX register SSEREG - any SSE register CRREG - any CR register DRREG - any DR register CONST - any constant ANY - any operand or memory address (size is not important)

MOV ANY,ANY, for example, matches any MOV command:

MOV [ANY],ANY - all writes to the memory, dependless on the size:

Note that 16-bit address is included into the list. As you probably know, Windows reserves first 64 K of the process's memory as a trap for the NULL pointers, so flat-mode 16-bit access has no chances, with one important exception. Selector FS points to the thread's data block that keeps thread-dependent information available to the application.16-bit version is 1 byte shorter than its 32-bit countrepart (but may execute longer due to the additional prefix). By the way, first doubleword in TDB is the pointer to the Structured Exception Handling chain that implements try-catch constructs. It's easy to find all SEH chain changes with the single search for MOV [FS:ANY],ANY:

Search for XOR RA,RA finds all commands that zero some register by XORing:

whereas XOR XA,XB - cases where XOR just manipulates bits:

JMP [R32*4+CONST] will find table jumps, LEA RA,[RA*5] - fast multiplications of 32-bit register by 5 (of course, in reality this means [RA*4+RA]), and so on.
Oh, and I'm curious, how useful will you find this feature:

That's all for now, bye!
February 24, 2007 - Progress.
The development of version 2.0 goes steadily forward. In the last three monthes I have written more than 350 K of debugged code. Backup, search, jumps, history, conditional expressions, watches, Assembler - all the stuff necessary for productive work. And - for the first time, 2.0 has paused on the breakpoint!
Yes, this is a big step. This means that the infrastructure is ... well, not yet completed, but is already so stable that it can support complex high-level functions. When I browse through the sources, I'm full of pride that the code is so well-structured, logical and clear. Unfortunately, this was not the case with 1.10. Initial design had several flaws - in 2000, I had no experience and was unable to foresee the requirements of the final version. Every small modification required significant efforts and lengthy testing. So finally I've decided to close the project and rewrite it almost from the scratch.
The first steps of any redesign are very hard psychologically. Maybe you've experienced similar problems - you write loads, heaps, piles of code, but your project is almost dead. All it can is some primitive stuff, like it was in my case - disassembling of several hardcoded binary sequences, dumps of memory blocks at fixed addresses, provisorical code and debugging outputs everywhere, and next to this garbage there is your old version, five years of successfull development, maybe also full of trash inside but at least functional and with shiny storefront...
Anyway, I'm past this stage. OllyDbg 2.0 lives, and it makes plenty of fun again to develop. You've waited for so long - so be patient, please, and sooner or later I'll introduce you my promising younger son :)
November 12, 2006 - Analyser.
Almost two years are gone since the last update of this page. But you don't forget me. The counter has crossed the magic limit of 1,000,000 impressions. So I feel me a bit ashamed and now will try to make up for your patience. Starting from now, every two or three weeks I will inform you here about the actual state of my work.
I'm frequently asked: "What happened to OllyDbg 2.0? Why is it not here?" Well, it is mostly my immanent laziness and, to lower extent, lots of other tasks and projects that have stopped the development of the second version. Nevertheless, it is not dead. In the last month I wrote more than 100 K of code, and now want to show you some highlights of the future version, mainly its new powerful analyser.
Despite highly complex features, like full code prediction, new version is significantly faster than its predecessor. But speed does not influence the quality of recognition. See, for example, how many calls were decoded by old OllyDbg in a large 3-MB application:

and by new:

Impressive, isn't it? Note that list of known functions in v2.0 currently includes only three system DLLs.
New version has strongly improved prediction of registers (especially ESP) and stack contents:

is able to recognize and decode register variables:

functions with variable number of arguments, like formats:

and cases when parameters are copied, rather than pushed, to the stack:

It determines loop variables, i.e. registers or memory items that change by the same amount on each loop iteration:

To help user, it even can rename and change decoding of arguments in some argument-depending cases:

New Analyser features also more reliable distinguishing between code and data. All in one, when OllyDbg will be ready, it will make debugging easier and understandable... I hope.
Part two will come in a couple of weeks. Bye!