Title: Python 2.7 and 3.2 to 3.5 bytearray pop and remove Buffer Over-read Credit: John Leitch (john@autosectools.com) Url1: http://autosectools.com/Page/Python-bytearray-pop-and-remove-Buffer-Over-read Url2: http://bugs.python.org/issue24467 Resolution: Fixed The Python 2.7 and 3.2 to 3.5 bytearray pop and remove methods suffer from buffer over-reads caused by memmove use under the assumption that PyByteArrayObject ob_size is less than ob_alloc, leading to a single byte over-read. This condition can be triggered by creating a bytearray from a range of length 0x10, then calling pop with a valid index: bytearray(range(0x10)).pop(0) The result is a memmove that reads off the end of src: 0:000> r eax=071aeff0 ebx=00000000 ecx=071aeff1 edx=00000010 esi=06ff80c8 edi=00000010 eip=6234b315 esp=0027fc98 ebp=0027fca0 iopl=0 nv up ei pl nz na po nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 MSVCR90!memmove+0x5: 6234b315 8b750c mov esi,dword ptr [ebp+0Ch] ss:002b:0027fcac=071aeff1 0:000> dV dst = 0x071aeff0 "" src = 0x071aeff1 "???" count = 0x10 0:000> db poi(dst) 071aeff0 00 01 02 03 04 05 06 07-08 09 0a 0b 0c 0d 0e 0f ................ 071af000 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af010 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af020 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af030 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af040 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af050 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af060 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 0:000> db poi(src) 071aeff1 01 02 03 04 05 06 07 08-09 0a 0b 0c 0d 0e 0f ?? ...............? 071af001 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af011 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af021 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af031 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af041 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af051 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 071af061 ?? ?? ?? ?? ?? ?? ?? ??-?? ?? ?? ?? ?? ?? ?? ?? ???????????????? 0:000> g (1968.1a88): Access violation - code c0000005 (first chance) First chance exceptions are reported before any exception handling. This exception may be expected and handled. eax=0c0b0a09 ebx=00000000 ecx=00000004 edx=00000000 esi=071aeff1 edi=071aeff0 eip=6234b468 esp=0027fc98 ebp=0027fca0 iopl=0 nv up ei ng nz ac pe cy cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010297 MSVCR90!UnwindUpVec+0x50: 6234b468 8b448efc mov eax,dword ptr [esi+ecx*4-4] ds:002b:071aeffd=???????? 0:000> k ChildEBP RetAddr 0027fca0 1e0856aa MSVCR90!UnwindUpVec+0x50 [f:\dd\vctools\crt_bld\SELF_X86\crt\src\Intel\MEMCPY.ASM @ 322] 0027fcc0 1e0aafd7 python27!bytearray_pop+0x8a [c:\build27\cpython\objects\bytearrayobject.c @ 2378] 0027fcd8 1e0edd10 python27!PyCFunction_Call+0x47 [c:\build27\cpython\objects\methodobject.c @ 81] 0027fd04 1e0f017a python27!call_function+0x2b0 [c:\build27\cpython\python\ceval.c @ 4033] 0027fd74 1e0f1150 python27!PyEval_EvalFrameEx+0x239a [c:\build27\cpython\python\ceval.c @ 2682] 0027fda8 1e0f11b2 python27!PyEval_EvalCodeEx+0x690 [c:\build27\cpython\python\ceval.c @ 3265] 0027fdd4 1e11707a python27!PyEval_EvalCode+0x22 [c:\build27\cpython\python\ceval.c @ 672] 0027fdec 1e1181c5 python27!run_mod+0x2a [c:\build27\cpython\python\pythonrun.c @ 1371] 0027fe0c 1e118760 python27!PyRun_FileExFlags+0x75 [c:\build27\cpython\python\pythonrun.c @ 1358] 0027fe4c 1e1190d9 python27!PyRun_SimpleFileExFlags+0x190 [c:\build27\cpython\python\pythonrun.c @ 950] 0027fe68 1e038d35 python27!PyRun_AnyFileExFlags+0x59 [c:\build27\cpython\python\pythonrun.c @ 753] 0027fee4 1d001017 python27!Py_Main+0x965 [c:\build27\cpython\modules\main.c @ 643] 0027fef0 1d0011b6 pythonw!WinMain+0x17 [c:\build27\cpython\pc\winmain.c @ 15] 0027ff80 76477c04 pythonw!__tmainCRTStartup+0x140 [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c @ 578] 0027ff94 7799ad1f KERNEL32!BaseThreadInitThunk+0x24 0027ffdc 7799acea ntdll!__RtlUserThreadStart+0x2f 0027ffec 00000000 ntdll!_RtlUserThreadStart+0x1b If the over-read is allowed to succeed, a byte adjacent to the buffer is copied: 0:000> r eax=01d8e978 ebx=00000000 ecx=00000000 edx=0000003a esi=01dc80c8 edi=00000010 eip=1e08569a esp=0027fd0c ebp=01d5aa10 iopl=0 nv up ei pl nz na pe nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 python27!bytearray_pop+0x7a: 1e08569a 8bd7 mov edx,edi 0:000> dt self Local var @ 0x27fd20 Type PyByteArrayObject* 0x01dc80c8 +0x000 ob_refcnt : 0n2 +0x004 ob_type : 0x1e21a6d0 _typeobject +0x008 ob_size : 0n16 +0x00c ob_exports : 0n0 +0x010 ob_alloc : 0n16 +0x014 ob_bytes : 0x01d8e978 "" 0:000> db 0x01d8e978 01d8e978 00 01 02 03 04 05 06 07-08 09 0a 0b 0c 0d 0e 0f ................ 01d8e988 ab ab ab ab ab ab ab ab-00 00 00 00 00 00 00 00 ................ 01d8e998 da 49 7a 0e b6 ac 10 00-b0 af d8 01 78 1c ce 01 .Iz.........x... 01d8e9a8 ee fe ee fe ee fe ee fe-ee fe ee fe ee fe ee fe ................ 01d8e9b8 5f 49 79 88 b7 ac 10 1d-02 00 00 00 f8 8b 22 1e _Iy...........". 01d8e9c8 d6 03 00 00 ff ff ff ff-00 00 00 00 20 54 72 69 ............ Tri 01d8e9d8 65 73 20 74 6f 20 64 65-74 65 72 6d 69 6e 65 20 es to determine 01d8e9e8 74 68 65 20 64 65 66 61-75 6c 74 20 6c 6f 63 61 the default loca 0:000> p eax=01d8e978 ebx=00000000 ecx=00000004 edx=00000000 esi=01dc80c8 edi=00000010 eip=1e0856aa esp=0027fd00 ebp=01d5aa10 iopl=0 nv up ei pl nz na pe nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 python27!bytearray_pop+0x8a: 1e0856aa 4f dec edi 0:000> db 0x01d8e978 01d8e978 01 02 03 04 05 06 07 08-09 0a 0b 0c 0d 0e 0f ab ................ 01d8e988 ab ab ab ab ab ab ab ab-00 00 00 00 00 00 00 00 ................ 01d8e998 da 49 7a 0e b6 ac 10 00-b0 af d8 01 78 1c ce 01 .Iz.........x... 01d8e9a8 ee fe ee fe ee fe ee fe-ee fe ee fe ee fe ee fe ................ 01d8e9b8 5f 49 79 88 b7 ac 10 1d-02 00 00 00 f8 8b 22 1e _Iy...........". 01d8e9c8 d6 03 00 00 ff ff ff ff-00 00 00 00 20 54 72 69 ............ Tri 01d8e9d8 65 73 20 74 6f 20 64 65-74 65 72 6d 69 6e 65 20 es to determine 01d8e9e8 74 68 65 20 64 65 66 61-75 6c 74 20 6c 6f 63 61 the default loca However, a subsequent call to PyByteArray_Resize overwrites the copied byte with a null terminator: 0:000> p eax=00000000 ebx=00000000 ecx=00000004 edx=00000000 esi=01dc80c8 edi=0000000f eip=1e0856c0 esp=0027fd0c ebp=01d5aa10 iopl=0 nv up ei pl zr na pe nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246 python27!bytearray_pop+0xa0: 1e0856c0 0fb6d3 movzx edx,bl 0:000> db 0x01d8e978 01d8e978 01 02 03 04 05 06 07 08-09 0a 0b 0c 0d 0e 0f 00 ................ 01d8e988 ab ab ab ab ab ab ab ab-00 00 00 00 00 00 00 00 ................ 01d8e998 da 49 7a 0e b6 ac 10 00-b0 af d8 01 78 1c ce 01 .Iz.........x... 01d8e9a8 ee fe ee fe ee fe ee fe-ee fe ee fe ee fe ee fe ................ 01d8e9b8 5f 49 79 88 b7 ac 10 1d-02 00 00 00 f8 8b 22 1e _Iy...........". 01d8e9c8 d6 03 00 00 ff ff ff ff-00 00 00 00 20 54 72 69 ............ Tri 01d8e9d8 65 73 20 74 6f 20 64 65-74 65 72 6d 69 6e 65 20 es to determine 01d8e9e8 74 68 65 20 64 65 66 61-75 6c 74 20 6c 6f 63 61 the default loca Because of this, these vulnerabilities should be classified as defense-in-depth. If PyByteArray_Resize could be forced to fail, or its behavior changes at a future date, it may become possible to exploit these issues to read adjacent memory.