[PATCH] i386 virtual memory layout rework

  Rework the i386 mm layout to allow applications to allocate more virtual
  memory, and larger contiguous chunks.


  - the patch is compatible with existing architectures that either make
    use of HAVE_ARCH_UNMAPPED_AREA or use the default mmap() allocator - there
    is no change in behavior.

  - 64-bit architectures can use the same mechanism to clean up 32-bit
    compatibility layouts: by defining HAVE_ARCH_PICK_MMAP_LAYOUT and
    providing a arch_pick_mmap_layout() function - which can then decide
    between various mmap() layout functions.

  - I also introduced a new personality bit (ADDR_COMPAT_LAYOUT) to signal
    older binaries that dont have PT_GNU_STACK.  x86 uses this to revert back
    to the stock layout.  I also changed x86 to not clear the personality bits
    upon exec(), like x86-64 already does.

  - once every architecture that uses HAVE_ARCH_UNMAPPED_AREA has defined
    its arch_pick_mmap_layout() function, we can get rid of
    HAVE_ARCH_UNMAPPED_AREA altogether, as a final cleanup.

  the new layout generation function (__get_unmapped_area()) got significant
  testing in FC1/2, so i'm pretty confident it's robust.


  Compiles & boots fine on an 'old' and on a 'new' x86 distro as well.

  The two known breakages were:

     http://www.redhatconfig.com/msg/67248.html

     [ 'cyzload' third-party utility broke. ]

     http://www.zipworld.com/au/~akpm/dde.tar.gz

     [ your editor broke :-) ]

  both were caused by application bugs that did:

	int ret = malloc();

	if (ret <= 0)
		failure;

  such bugs are easy to spot if they happen, and if it happens it's possible
  to work it around immediately without having to change the binary, via the
  setarch patch.

  No other application has been found to be affected, and this particular
  change got pretty wide coverage already over RHEL3 and exec-shield, it's in
  use for more than a year.


  The setarch utility can be used to trigger the compatibility layout on
  x86, the following version has been patched to take the `-L' option:

 	http://people.redhat.com/mingo/flexible-mmap/setarch-1.4-2.tar.gz

  "setarch -L i386 <command>" will run the command with the old layout.

From: Hugh Dickins <hugh@veritas.com>

  The problem is in the flexible mmap patch: arch_get_unmapped_area_topdown
  is liable to give your mmap vm_start above TASK_SIZE with vm_end wrapped;
  which is confusing, and ends up as that BUG_ON(mm->map_count).

  The patch below stops that behaviour, but it's not the full solution:
  wilson_mmap_test -s 1000 then simply cannot allocate memory for the large
  mmap, whereas it works fine non-top-down.

  I think it's wrong to interpret a large or rlim_infinite stack rlimit as
  an inviolable request to reserve that much for the stack: it makes much less
  VM available than bottom up, not what was intended.  Perhaps top down should
  go bottom up (instead of belly up) when it fails - but I'd probably better
  leave that to Ingo.

  Or perhaps the default should place stack below text (as WLI suggested and
  ELF intended, with its text defaulting to 0x08048000, small progs sharing
  page table between stack and text and data); with a further personality for
  those needing bigger stack.

From: Ingo Molnar <mingo@elte.hu>

  - fall back to the bottom-up layout if the stack can grow unlimited (if
  the stack ulimit has been set to RLIM_INFINITY)

  - try the bottom-up allocator if the top-down allocator fails - this can
  utilize the hole between the true bottom of the stack and its ulimit, as a
  last-resort effort.

Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

2 files changed, 28 insertions, 0 deletions

diff --git a/include/linux/personality.h b/include/linux/personality.h
index 9b009b6754f6..7efc05ea7efc 100644
--- a/include/linux/personality.h
+++ b/include/linux/personality.h
@@ -30,6 +30,7 @@ extern int abi_fake_utsname;
  */
 enum {
 	MMAP_PAGE_ZERO =	0x0100000,
+	ADDR_COMPAT_LAYOUT =	0x0200000,
 	READ_IMPLIES_EXEC =	0x0400000,
 	ADDR_LIMIT_32BIT =	0x0800000,
 	SHORT_INODE =		0x1000000,
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 32527b2edd4b..bb126a63a3a6 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -189,10 +189,26 @@ extern int sysctl_max_map_count;
 
 #include <linux/aio.h>
 
+extern unsigned long
+arch_get_unmapped_area(struct file *, unsigned long, unsigned long,
+		       unsigned long, unsigned long);
+extern unsigned long
+arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr,
+			  unsigned long len, unsigned long pgoff,
+			  unsigned long flags);
+extern void arch_unmap_area(struct vm_area_struct *area);
+extern void arch_unmap_area_topdown(struct vm_area_struct *area);
+
+
 struct mm_struct {
 	struct vm_area_struct * mmap;		/* list of VMAs */
 	struct rb_root mm_rb;
 	struct vm_area_struct * mmap_cache;	/* last find_vma result */
+	unsigned long (*get_unmapped_area) (struct file *filp,
+				unsigned long addr, unsigned long len,
+				unsigned long pgoff, unsigned long flags);
+	void (*unmap_area) (struct vm_area_struct *area);
+	unsigned long mmap_base;		/* base of mmap area */
 	unsigned long free_area_cache;		/* first hole */
 	pgd_t * pgd;
 	atomic_t mm_users;			/* How many users with user space? */
@@ -999,6 +1015,17 @@ static inline void set_task_cpu(struct task_struct *p, unsigned int cpu)
 
 #endif /* CONFIG_SMP */
 
+#ifdef HAVE_ARCH_PICK_MMAP_LAYOUT
+extern void arch_pick_mmap_layout(struct mm_struct *mm);
+#else
+static inline void arch_pick_mmap_layout(struct mm_struct *mm)
+{
+	mm->mmap_base = TASK_UNMAPPED_BASE;
+	mm->get_unmapped_area = arch_get_unmapped_area;
+	mm->unmap_area = arch_unmap_area;
+}
+#endif
+
 #endif /* __KERNEL__ */
 
 #endif