While you can prevent the heap from being allocated over the stack, the stack may grow "autonomously" during normal processor instructions where the OS can't intervene.
The classic fix was to put a "guard page", i.e. a segment of memory marked as forbidden, so if stack would grow into there, it'd trigger a page fault and then the process can be stopped.
However, while stack usually grows in small increments, you may allocate some very large data on the stack and not write or read anything in that buffer - so if you do it correctly, the end of the stack will "jump over" that guard page, leaving it in the middle of the stack but not touching it in any way; and the end of the stack will overlap with the heap, enabling all the fun exploits again.
We had to deal with something very similar to this when I was working on REALbasic. When targeting Windows, if the compiler happened to generate a function prolog which would allocate more than one page of stack space for local variables, we had to generate extra instructions which would touch each intermediate page, in order, or the kernel would trap the first access beyond the guard page as out-of-bounds. It was an interesting bit of semi-security.
> The exploits and proofs of concept that we developed in the course of our research are all Local Privilege Escalations: an attacker who has any kind of access to an affected system can exploit the Stack Clash vulnerability and obtain full root privileges.
https://blog.qualys.com/securitylabs/2017/06/19/the-stack-cl...
The vulnerability is in the userland process. However, the kernel can utilize a stop-gap measure that makes exploitation difficult in most scenarios (but this is arbitrary).
However, for that fix to be useful, every single distribution needs to recompile every single userland package and distribute it to every single user - i.e., instead of downloading a security patch, you need to download a new version of every executable on your hdd.
If the distro guys just did for the next version (Atleast the guys like Debian, Fedora etc from whom most other distros are derived) wouldn't most systems be protected?
It would take a while but it wouldn't take 10 years atleast. Mass adoption is impossible but incremental changes are. If critical executables are patched then we have atleast a partially protected system right? (I am not an expert but this sounds a bit more plausible than breaking the entire userland)
The problem is with stack growing into memory previously allocated with mmap, including but not limited to mmap being used to back up large malloc allocations. If this happens, writing to objects in the mmap-ed area can be used to overwrite stack frame of the current function, including its return address, and hence control code execution after returning.
In 2005 it was discovered that when a process fills up all its address space with allocations, Linux will allow an mmap call to allocate memory directly near the top of the stack so that any stack expansion will run into this allocation. This was not fixed.
In 2010, Rafal Wojtczuk developed a working exploit against Xorg using this principle. The problem was finally fixed by enforcing a one page gap at the top of the stack where mmap allocations were forbidden and preventing the stack from growing too close to the nearest mmap allocation. TFA author couldn't resist bitching about some technical issues with this patch which were promptly fixed.
Unfortunately, one page gap is not enough because it can be "jumped over": a sufficiently vulnerable application can be tricked to mmap memory directly adjacent to the gap and then create a large stack allocation (alloca, variable length array or just a fixed-length array larger than 4kB) which covers the whole gap. As long as this allocation isn't used, the kernel has no way of knowing that it exists. If the application then is tricked into calling another function before using this allocation, a new stack frame is created behind the big allocation and the safety gap, in mmap area.