Yesterday I tried installing pydbg and pydasm on my notepc. pydasm is popular, famous library to disassemble machine codes (opcodes). pydbg is also popular, famous library to build lightweight, extensible debugger for Windows platform. Actually, pydbg is included in PaiMei, windows platform debugger framework. The journey was hard, full of struggle and traps with python distutils. I left these notes, memos, and traps for future person (including myself) who want installing these excellent reverse engineering tools written in python and c. My notepc environments are: CPU : Intel Pentium M (Centrino) 1.2GHz RAM : 1GB OS : Windows XP Professional SP3 (Japanese) Python: Python 2.5 (install from MSI installer) Install Dir : C:\Python25 Visual Studio : Visual C++ 2008 Express Edision (SP1) Subversion: TortoiseSVN 1.6.x We require Subversion to obtain PaiMei later. #more|| Okay, Let's begin. #outline|| ---- * Installing pydasm step-by-step pydbg requires pydasm. We have to install pydasm before installing pydbg. But pydasm is written by c, and no official binary release for win32 platform. Out environments uses: Python 2.5 Visual C++ 2008 Express Edition SP1 ** getting libdasm pydasm is included "libdasm", simple x86 disassembly library. Author of pydasm is another person to author of libdasm. libdasm was written by jt (nologin.org), pydasm was written by ero (dkbza.org). pydasm was contributed to libdasm. - dkbza - pydasm (just showing libdasm download link, short overview, example codes) -- http://dkbza.org/pydasm.html Now, let's download libdasm, including pydasm from following url: - libdasm : nologin -- code -- http://www.nologin.org/main.pl?action=codeView&codeId=49& When download success, expand tar.gz into your favorite directory. Yesterday, I get libdasm-1.5.tar.gz. After expanding, I got following directory trees: libdasm-1.5\ libdasm.c README.txt ... bin\ pydasm\ ... ** try compile -> errors!! Next, open command prompt, setup your VC++ environment (ex: vcvars32.bat). Confirm "C:\Python25" is added PATH environment or ".py" extension is associated with "C:\Python25\python.exe". In your command prompt, cd to "libdasm-1.5\pydasm", and type: > setup.py build_ext running build_ext error: Python was built with Visual Studio 2003; extensions must be built with a compiler than can generate compatible binaries. Visual Studio 2003 was not found on this system. If you have Cygwin installed, you can try compiling with MingW32, by passing "-c mingw32" to setup.py. Ouch!! Why? My command prompt enviroments are surely prepared for VC++ tools!! > cl Microsoft(R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01 for 80x86 Copyright (C) Microsoft Corporation. All rights reserved. ... > link Microsoft (R) Incremental Linker Version 9.00.30729.01 Copyright (C) Microsoft Corporation. All rights reserved. The answer is, Python's distutils tool (used from setup.py) checks Visual Studio's registry keys and confirm proper version is installed or not. Unfortunately, registry keys installed by VC++2008 Express Edision on my environment are not what distutils expected. ** avoiding distutil's registry key checks Then how to avoid these annoying registry key checks ? The answer is, edit distutil's library source code. Python is open source, and distutils are written in python. (Skips my surveys and researches for distutils and its source code.) As a result, edit following source code: C:\Python25\Lib\distutils\msvccompiler.py In msvccompiler.py: ... class MSVCCompiler (CCompiler) : ... def __init__ (self, verbose=0, dry_run=0, force=0): ... # comment out here!! self.__macros = MacroExpander(self.__version) Registry keys checks are implemented in MacroExpander() and its constructor. And, "self.__macros" is optional, not needed. So removing this line doesn't effect to compiling. After removing unneccesary line, delete old ".pyc" file, msvccompiler.pyc, may be exists in same directory to ".py". ** try compiling again -> build success!! In msvccompiler.py, there's no registry checks when "DISTUTILS_USE_SDK" and "MSSdk" environments are set and "cl.exe" is found in PATH environment. So we set "DISTUTILS_USE_SDK" and "MSSdk" in our command prompt window: > set DISTUTILS_USE_SDK=1 > set MSSdk=1 Every thing is okay for each environment values. msvccompiler.py only checks each key is exists, ignoring values. Okay, let's try compiling: > setup.py build_ext running build_ext building 'pydasm' extension creating build creating build\temp.win32-2.5 creating build\temp.win32-2.5\Release C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe \ /c /nologo /Ox /MD /W3 /GX /DNDEBUG \ -IC:\Python25\include -IC:\Python25\include -IC:\Python25\PC \ /Tc../libdasm.c /Fobuild\temp.win32-2.5\Release\../libdasm.obj C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe \ /c /nologo /Ox /MD /W3 /GX /DNDEBUG \ -IC:\Python25\include -IC:\Python25\include -IC:\Python25\PC \ /Tcpydasm.c /Fobuild\temp.win32-2.5\Release\pydasm.obj creating build\lib.win32-2.5 C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\link.exe \ /DLL /nologo /INCREMENTAL:NO \ /LIBPATH:C:\Python25\libs /LIBPATH:C:\Python25\PCBuild /EXPORT:initpydasm \ build\temp.win32-2.5\Release\../libdasm.obj build\temp.win32-2.5\Release\pydasm.obj \ /OUT:build\lib.win32-2.5\pydasm.pyd \ /IMPLIB:build\temp.win32-2.5\Release\..\pydasm.lib (omitting warnings from build output) When all work correct, we'll get libdasm-1.5\pydasm\build\lib.win32-2.5\pydasm.pyd ** install pydasm and test ... oh-no, "MSVCR90.dll" not foud !? Now invoking install command: > setup.py install running install running build running build_ext running install_lib copying build\lib.win32-2.5\pydasm.pyd -> C:\Python25\Lib\site-packages copying build\lib.win32-2.5\pydasm.pyd.manifest -> C:\Python25\Lib\site-packages running install_egg_info Removing C:\Python25\Lib\site-packages\pydasm-1.5-py2.5.egg-info Writing C:\Python25\Lib\site-packages\pydasm-1.5-py2.5.egg-info Then invoking python and try to import pydasm: > C:\Python25\python.exe Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import pydasm Traceback (most recent call last): File "", line 1, in ImportError: DLL load failed: 指定されたモジュールが見つかりません。 >>> quit() Oh-no, last "ImportError" says "specified module not found". And on my environment, "MSVCR90.dll not found" messagebox was shown. Checking module dependencies: > dumpbin /dependents C:\Python25\python.exe ... python25.dll MSVCR71.dll <-- Python25 depends on MSVCR71.dll KERNEL32.dll ... > dumpbin /dependents build\lib.win32-2.5\pydasm.pyd ... MSVCR90.dll <-- pydasm depends on MSVCR90.dll !! python25.dll KERNEL32.dll On my environment, MSVCR90.dll was installed to Visual Studio installed directory, but not to Windows system directory.If you've already installed VC-Runtime including MSVCR90.dll to your Windows system directory, those ImportError and message box may not be shown, all works correctly. How do that ? There're 2 ways to resolve this error. + install VC-Runtime including MSVCR90.dll to Windows. ++ This is straightforward, but Python uses MSVCR71.dll (compiled by older version VisualStudio). I'm afraid of conflict between runtime dlls. + link VC-Runtime static. ++ This makes portable module, but need to change compile options, module file size become big. I'm afraid of conflits between runtime dlls strongly (may be another trap), so I delve into second way. ** edit msvccompiler.py again, change compile option -> pydasm works correct!! Then how to change compile option ? What option should we choose ? msvccompiler.py setups compiler options in MSVCCompiler class's initialize method. msvccompiler.py: #pre||> ... class MSVCCompiler (CCompiler) : ... def initialize(self): ... self.preprocess_options = None if self.__arch == "Intel": self.compile_options = [ '/nologo', '/Ox', '/MD', '/W3', '/GX' , '/DNDEBUG'] self.compile_options_debug = ['/nologo', '/Od', '/MDd', '/W3', '/GX', '/Z7', '/D_DEBUG'] ||< "/MD" and "/MDd" indicates dynamic linking multithreaded library. For more details about runtime library and compiler options, see following MSDN. - C Run-Time Libraries -- http://msdn.microsoft.com/en-us/library/abx4dbyh.aspx --- "MSDN Library" > "Development Tools and Languages" > "Visual Studio 2010" > "Visual Studio" > "Visual Studio Languages" > "Visual C++" > "Visual C++ Reference" > "Visual C++ Libraries Reference" > "Run-Time Library" > "C Run-Time Libraries" This time we want to link static multithreaded library, So we choose "/MT" and "/MTd". Edit msvccompiler.py: #pre||> ... class MSVCCompiler (CCompiler) : ... def initialize(self): ... self.preprocess_options = None if self.__arch == "Intel": # change '/MD' => '/MT' self.compile_options = [ '/nologo', '/Ox', '/MT', '/W3', '/GX' , '/DNDEBUG'] # change '/MDd' => '/MTd' self.compile_options_debug = ['/nologo', '/Od', '/MTd', '/W3', '/GX', '/Z7', '/D_DEBUG'] ||< Delete old msvccompiler.pyc and pydasm\build directory, and try build again. #pre||> > setup.py build_ext > dumpbin /dependents build\lib.win32-2.5\pydasm.pyd Image has the following dependencies: python25.dll KERNEL32.dll ... > setup.py install ... > C:\Python25\python.exe Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import pydasm >>> buffer = '\x90\x31\xc9\x31\xca\x31\xcb' >>> offset = 0 >>> while offset < len(buffer): ... i = pydasm.get_instruction(buffer[offset:], pydasm.MODE_32) ... print pydasm.get_instruction_string(i, pydasm.FORMAT_INTEL, 0) ... if not i: ... break ... offset += i.length ... nop xor ecx,ecx xor edx,ecx xor ebx,ecx >>> quit() ||< All-right, pydasm seems to work correctly. Next, try to install PaiMei, pydbg. * Installing pydbg step-by-step - PaiMei Official Docs -- http://pedram.redhive.com/PaiMei/docs/ - PaiMei Google Codes Site -- http://code.google.com/p/paimei/ - Other Resources: -- http://thatsbroken.com/?p=26 -- http://maliciousattacker.blogspot.com/2006/12/pydbg-in-vmware.html -- http://www.openrce.org/downloads/details/208/PaiMei -- http://maliciousattacker.blogspot.com/2007/01/setting-up-pydbg.html ** checkout from SVN repositry and install -> "Python26.dll not found" !? 1st, checkout (or export) PaiMei latest source tree from SVN repositry. Use your favorite SVN tools. Yesterday, I checkouted r248 trunk source tree. 2nd, open command prompt window, cd to PaiMei directory, and run setup.py. > setup.py build (...) > setup.py install (...) 3rd, try to import pydbg: #pre||> > C:\Python25\python.exe Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import pydbg Traceback (most recent call last): File "", line 1, in File "pydbg\__init__.py", line 47, in from pydbg import * File "C:\in_vitro\SVNWORK\paimei\pydbg\pydbg.py", line 32, in import pydasm ImportError: DLL load failed: 指定されたモジュールが見つかりません。 ||< And message box was shown, saying "Python26.dll not found". See carefully above python stacktrace. It is very strange that error has occurred at "import pydasm". At previous section, we confirm "import pydasm" work correctly. What's happening? ** remove bundled pydasm.pyd in pydbg -> success!! The Answer is : "pydasm.pyd" is bundled in pydbg module directory, and it was compiled with Python 2.6. #pre||> > dumpbin /dependents C:\Python25\Lib\site-packages\pydbg\pydasm.pyd ^^^^^^^^^^^^^^^^ pydasm.pyd was bundled!! ... Image has the following dependencies: MSVCR90.dll python26.dll KERNEL32.dll ... ||< Then, we simply remove bundled "pydbg\pydasm.pyd". Close command prompt, shutdown all python program, and re-open command prompt (clear loaded dlls). Try-again: #pre||> > C:\Python25\python.exe Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import pydbg >>> ||< All-right, no errors!! Finally, invoke calc.exe, get its PID, and test following example: #pre||> from pydbg import * from pydbg.defines import * def handler_breakpoint (pydbg): if pydbg.first_breakpoint: print "[*] Hit 1st breakpoint!" return DBG_CONTINUE print "[*] Hit breakpoint!" return DBG_CONTINUE dbg = pydbg() dbg.set_callback(EXCEPTION_BREAKPOINT, handler_breakpoint) dbg.attach(XXXXX) # pid of calc.exe recv = dbg.func_resolve("user32", "ShowWindow") dbg.bp_set(recv) dbg.debug_event_loop() ||< Example: #pre||> > C:\Python25\python.exe Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from pydbg import * >>> from pydbg.defines import * >>> def handler_breakpoint (pydbg): ... if pydbg.first_breakpoint: ... print "[*] Hit 1st breakpoint!" ... return DBG_CONTINUE ... print "[*] Hit breakpoint!" ... return DBG_CONTINUE ... >>> dbg = pydbg() >>> dbg.set_callback(EXCEPTION_BREAKPOINT, handler_breakpoint) >>> dbg.attach(5084) >>> recv = dbg.func_resolve("user32", "ShowWindow") >>> dbg.bp_set(recv) >>> dbg.debug_event_loop() [*] Hit 1st breakpoint! [*] Hit breakpoint! ... (minimize, or restore calc window) ... [*] Hit breakpoint! >>> quit() ||< ** note for other modules in PaiMei In this article, we JUST ONLY want to install pydbg in PaiMei. This article ignored to setup other modules in PaiMei, like PIDA, GUI components. If you want to setup these modules, PaiMei framework fullset, see documents in official site or "docs/index.html" in SVN source tree.