CVE Publication: CVE 2016-8636

After a long patching process, CVE 2016-8636 was now fixed and can be publicly disclosed. CVE 2016-8636 is caused by a classic integer-overflow vulnerability, showing that even the linux kernel suffers from this major vulnerability family.


During the end of October 16, I searched for vulnerabilities in the drivers of the kernel, and decided to take a look over the famous RDMA technology. I went to the software implementation of the RDMA protocol over infiniband, also called (Soft RoCE):


Since the main concern was to check the software bounds checking over the incoming read/write requests the flow converged rather quickly to the vulnerable area: mem_check_range.

The Vulnerability

After we identify the matching iov for the current request, there is this following check to ensure the request falls inside the pre-allocated memory region:

int mem_check_range(struct rxe_mem *mem, u64 iova, size_t length)
  switch (mem->type) {
  // EI: no checks on the DMA regions - used only for local write... [22/10/2016]
    return 0;
    // EI [FIX]: should fix this check to be:
    // EI      : ((iova   < mem->iova)   ||
    // EI      :  (length > mem->length) ||
    // EI      :  (iova   > (mem->iova + mem->length - length))) [22/10/2016]
    return ((iova < mem->iova) ||
           ((iova + length) > (mem->iova + mem->length))) ?
           -EFAULT : 0;

    return -EFAULT;

As we can see, the check has a classic integer-overflow in it, since we combine several checked variables together into a single check:

  • iova – address is checked to start in/after the allocated region
  • iova + length – end* is checked to fall in/before the allocated region

Since length is unsigned it is ASSUMED that iova <= iova + length. However, since length was NOT checked prior to this check, it can be a huge unsigned value causing the addition to wrap-around, thus breaking the code’s assumption.

Possible damage

This later enables any such read/write request to work on an undefined memory region dependent on the internal data-structure that holds the chain of iov’s in the kernel’s memory – memory corruption (Write) and leakage (Read).

Proposed fix

As we can see in my code comment, and in the actual patch I committed to the kernel, the fix is to check each variable on it’s own:

  • iova – address is checked to start in/after the allocated region
  • length – length is checked to match (<=) the region’s capacity
  • start – iova is checked to be in the remaining gap:

[mem->iova, mem->iova + mem->length – length]

The key factor is to check the variable itself, without interfering with arithmetic operation that might damage the checked term.


CVE 2016-8636 is a classic example for an integer-overflow in bounds checks. This family of vulnerabilities is responsible for a dominant part of the overall vulnerabilities in C/C++ code, and will be discussed in more details in a future post.

Author: eyalitkin

White hat security researcher. Recently finished my M.s.c at TAU, and now focus on security research, mainly in open sources.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: