VMSF_DELTA filter in unrar allows arbitrary memory write 

It appears that the VMSF_DELTA memory corruption that was reported to Sophos AV in 2012 (and fixed there) was actually inherited from upstream unrar. For unknown reasons, whoever fixed the bug did not report this to upstream unrar, and the bug seems to have persisted there to this day.

The bug is sometimes called "Sophos Anti-Virus RAR VMSF_DELTA Filter Signedness Error", and referenced in the following links:
   <a href="https://community.sophos.com/kb/en-us/118424#six" title="" class="" rel="nofollow">https://community.sophos.com/kb/en-us/118424#six</a>
   <a href="https://telussecuritylabs.com/threats/show/TSL20121207-01" title="" class="" rel="nofollow">https://telussecuritylabs.com/threats/show/TSL20121207-01</a>

The issue is in the following code:

    case VMSF_DELTA:
      {
        int DataSize=R[4],Channels=R[0],SrcPos=0,Border=DataSize*2;
        if ((uint)DataSize>=VM_MEMSIZE/2)
          break;

        // Bytes from same channels are grouped to continual data blocks,
        // so we need to place them back to their interleaving positions.
        for (int CurChannel=0;CurChannel<Channels;CurChannel++)
        {
          byte PrevByte=0;
          for (int DestPos=DataSize+CurChannel;DestPos<Border;DestPos+=Channels)
            Mem[DestPos]=(PrevByte-=Mem[SrcPos++]);
        }
      }
      break;

An attacker controls R[4] and R[0] arbitrarily. By setting R[0] to be sufficiently
large (perhaps 0x7FFFFFFF), an integer overflow can be caused in DataSize+CurChannel.
The result is a negative value of "DestPos", which allows the attacker to write 
out of bounds when setting Mem[DestPos].

A minimal test example that crashes unrar 5.40 and prior is base64-encoded below:

Base64-encoded RAR file to trigger the VMSF_DELTA issue:

UmFyIRoHAPlOcwAADgAAAAAAAAAAMAh0AAAmAI4AAAAAAAAAAhBBUiEAAAAAHQAGAAAAACBzdGRv
dXQgIVUMzRDNmBGByDAda+AXaSv4KvQr1K/oejL05mXmXmww5tEk8gA9k8nmieyeyeswuOR6cx69
a2Hd6zQwu3aoMDDwMEswADAAMD4P938w+dydoRFwAmwAAAAAvv////+/////+9W3QFgAAQAGAAAA
Ooimhd12AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Given the wide prevalence of the unrar source code in third-party software, quite a few downstream parties will be affected. 

This bug is subject to a 90 day disclosure deadline. After 90 days elapse
or a patch has been made broadly available, the bug report will become
visible to the public.




Found by: thomasdullien