Fix IDA 7.5/7.6/7.7SP1 crashing on idapython3.dll in Wine

idafix.md

Description

For some reason IDA executes FreeLibrary() to the plugin immediately after getting its PLUGIN structure's address, so later invocations of the plugin lead to calls to nowhere (that was supposed to be python3.dll). Simply patching the location of the FreeLibrary() call fixes the issue. The location is easy to find: go by cross-references to a place where the call to FreeLibrary is followed by a reference to the string "%s: incompatible plugin version..." and NOP it away.

7.5

ida.dll

+001c1d20  15 9b e6 e3 ff 48 8b 4d  88 48 85 c9 74 0e 90 90  |.....H.M.H..t...|
+001c1d30  90 90 90 90 48 c7 45 88  00 00 00 00 48 8b 4c 24  |....H.E.....H.L$|

ida64.dll

+001cb050  15 83 53 e3 ff 48 8b 4d  88 48 85 c9 74 0e 90 90  |..S..H.M.H..t...|
+001cb060  90 90 90 90 48 c7 45 88  00 00 00 00 48 8b 4c 24  |....H.E.....H.L$|

7.6

ida.dll

+001cb6f0  15 03 4d e3 ff 48 8b 4d  88 48 85 c9 74 0e 90 90  |..M..H.M.H..t...|
+001cb700  90 90 90 90 48 c7 45 88  00 00 00 00 48 8b 4c 24  |....H.E.....H.L$|

ida64.dll

+001d53f0  15 0b b0 e2 ff 48 8b 4d  88 48 85 c9 74 0e 90 90  |.....H.M.H..t...|
+001d5400  90 90 90 90 48 c7 45 88  00 00 00 00 48 8b 4c 24  |....H.E.....H.L$|

7.7SP1

ida.dll Patching not required.

-EAA2A     00 48 85 c9 74 0a ff 15  ea 0d
+EAA2A     00 48 85 c9 eb 0a ff 15  ea 0d

ida64.dll

-EEC80     ff 15 9a 5b 26 00 48 89  7d 00
+EEC80     90 90 90 90 90 90 90 90  90 90

Hey, would really appreciate it if you could provide a more detailed walkthrough on how to patch and set up wine and python3 to work correctly, primarily for 7.7 SP1

First - you should disable idapython:
Rename ~/${IDA_WINE_PREFIX}/drive_c/Program Files/IDA 7.7/plugins/idapython3_64.dll to idapython3_64.dll.disabled
The IDA should start after this rename, but without IdaPython support.
You can do bugfix patch without it using pure IDA features.

Here is how you can:

1. Copy DLL (ida.dll or ida64.dll) to another location or name (IDA may lock its files and prevent you from patching itself on-the-fly).
2. Open IDA 64bit, load your copy of the DLL (ida.dll or ida64.dll) with default settings, reject debug info download.
3. Search for a sequence of bytes "incompatible plugin version" (with quotes).
4. Double-click the only search result and press X to go to xrefs to this string.
5. Select the only xref and press OK.
6. Decompile this function by pressing F5.
7. Dismiss all dialogue windows with Ok button (or set it as you want, but make sure you setting sane rules - the defaults are fine).
   You will see something like this:

    if ( hLibModule )
    {
      FreeLibrary(hLibModule);
      hLibModule = 0i64;
    }
    qfree((void *)lpPathName);
    if ( *v47 != 700 )
      sub_100002D40("%s: incompatible plugin version, skipped\n", v8);

The FreeLibrary call and hLibModule assignment are bogus, and should be patched out.

8. Click on FreeLibrary(hLibModule); line, then click on IDA View A tab and right-click anywhere in disassembly.
9. Select Synchronize with -> Pseudocode A.
   You should see something like this:

.text:00000001000EB627 loc_1000EB627:                          ; CODE XREF: sub_1000EAF20+6F0↑j
.text:00000001000EB627                 mov     rcx, [rbp+140h+hLibModule] ; hLibModule
.text:00000001000EB62B                 test    rcx, rcx
.text:00000001000EB62E                 jz      short loc_1000EB63A
.text:00000001000EB630                 call    cs:FreeLibrary
.text:00000001000EB636                 mov     [rbp+140h+hLibModule], rdi

The jz short loc_1000EB63A should be patched to jmp short loc_1000EB63A
10. Click on jz short loc_1000EB63A line and select Edit->Patch program->Assemble... from window menu.
11. You should now see a dialogue window with an Instruction field, and jz short loc_1000EB63A in this field.
12. Change it to jmp short loc_1000EB63A and press Ok, then press Cancel.
13. Return to Pseudocode A tab and make sure that bogus code block is now gone. Re-decompile it using F5 again if not.
14. When DLL was fixed - save it using Edit->Patch program->Apply patches to input file... from window menu. Select Create backup and press OK.
15. Close IDA saving the database - it may be a reference for yourself after a year or two when a new version will become available.
16. Save backup of original DLL and replace it with your fixed version.
17. Do this again for other DLL, both ida.dll and ida64.dll are buggy and should be patched.
18. Congratulations, you have patched your IDA to fix this bug!

Now you should enable idapython - rename ~/${IDA_WINE_PREFIX}/drive_c/Program Files/IDA 7.7/plugins/idapython3_64.dll.disabled back to idapython3_64.dll - IDA should not crash.

For Python - just install Python for Windows to the same Wine prefix.
You may also need to run ~/${IDA_WINE_PREFIX}/drive_c/Program Files/IDA 7.7/idapyswitch.exe and select your Python.

Also, for Wine - go to Options->Colors, select dark color theme and click OK.
This will fix black interface parts (which became an issue with 7.7SP1 on Wine).

Okay, let me give it a try right now.

Thank you, it worked 😅

Thank for you this. It still works in IDA Pro 9, though the hex address for the jz/jmp has changed and I had to copy the dll in order to patch it

Thank for you this. It still works in IDA Pro 9, though the hex address for the jz/jmp has changed and I had to copy the dll in order to patch it

Wow, I have not expected this bug lasts so long.
So, I upgraded the patching guide (markdown, minor text improvements).