[PATCH] x86-64 merge

Lots of changes that have accumulated over the last weeks.

This makes it compile and boot again, Lots of bug fixes, including
security fixes.  Several speedups.

Only changes x86-64 specific files.

 - Use private copy of siginfo.h (for si_band)
 - Align 32bit vsyscall coredump (from Roland McGrath)
 - First steps towards 64bit vsyscall coredump (not working yet)
 - Use in kernel trampoline for signals
 - Merge APIC pm update from Pavel/Mikael
 - Security fix for ioperm (from i386)
 - Reenable vsyscall dumping for 32bit coredumps
 - Fix bugs in 32bit coredump that could lead to oopses.
 - Fix 64bit vsyscalls
 - Revert change in pci-gart.c: pci_alloc_consistent must use an
   0xffffffff mask hardcoded.
 - Fix bug in noexec= option handling
 - Export fake_node
 - Cleanups from Pavel
 - Disable 32bit vsyscall coredump again.  Still has some problems.
 - Implement new noexec= and noexec32= options to give a wide choice
   of support for non executable mappings for 32bit and 64bit processes.
   The default is now to honor PROT_EXEC, but mark stack and heap
   PROT_EXEC.
 - 32bit emulation changes from Pavel: use compat_* types.
 - (2.4) Use physical address for GART register.
 - Convert debugreg array to individual members and clean up ptrace
   access.  This saves 16 byte per task.
 - (2.4) Use new streamlined context switch code.  This avoids a
   pipeline stall and pushes the register saving to C code.
 - Save flags register in context switch
 - Clean up SMP early bootup.  Remove some unnecessary code.
 - (2.4) Process numa= option early
 - (2.4) Merge 2.4 clear_page, copy_*_user, copy_page, memcpy, memset.
   These are much faster.  clear/copy_page don't force the new page out
   of memory now which should speed up user processes.  Also full
   workaround for errata #91.
 - Some cleanup in pageattr.c code.
 - Fix warning in i387.h
 - Fix wrong PAGE_KERNEL_LARGE define.  This fixes a security hole and
   makes AGP work again.
 - Fix wrong segment exception handler to not crash.
 - Fix incorrect swapgs handling in bad iret exception handling
 - Clean up some boot printks
 - Micro optimize exception handling preamble.
 - New reboot handling.  Supports warm reboot and BIOS reboot vector
   reboot now.
 - (2.4) Use MTRRs by default in vesafb
 - Fix bug in put_dirty_page: use correct page permissions for the stack
 - Fix type of si_band in asm-generic/siginfo.h to match POSIX/glibc
   (needs checking with other architecture maintainers)
 - (2.4) Define ARCH_HAS_NMI_WATCHDOG
 - Minor cleanup in calling.h
 - IOMMU tuning: only flush the GART TLB when the IOMMU aperture area
   allocation wraps.  Also don't clear entries until needed.  This
   should increase IO performance for IOMMU devices greatly.  Still a
   bit experimental, handle with care.
 - Unmap the IOMMU aperture from kernel mapping to prevent unwanted CPU
   prefetches.
 - Make IOMMU_LEAK_TRACE depend on IOMMU_DEBUG
 - Fix minor bug in pci_alloc_consistent - always check against the dma
   mask of the device, not 0xffffffff.
 - Remove streamining mapping delayed flush in IOMMU: not needed anymore
   and didn't work correctly in 2.5 anyways.
 - Fix the bad pte warnings caused by the SMP/APIC bootup.
 - Forward port 2.4 fix: ioperm was changing the wrong io ports in some
   cases.
 - Minor cleanups
 - Some cleanups in pageattr.c (still buggy)
 - Fix some bugs in the AGP driver.
 - Forward port from 2.4: mask all reserved bits in debug register in
   ptrace.  Previously gdb could crash the kernel by passing invalid
   values.
 - Security fix: make sure FPU is in a defined state after an
   FXSAVE/FXRSTOR exception occurred.
 - Eats keys on panic (works around a buggy KVM)
 - Make user.h user includeable.
 - Disable sign compare warnings for gcc 3.3-hammer
 - Use DSO for 32bit vsyscalls and dump it in core dumps.  Add dwarf2
   information for the vsyscalls.
   Thanks to Richard Henderson for helping me with the nasty parts of
   it.  I had to do some changes over his patch and it's currently only
   lightly tested.  Handle with care.  This only affects 32bit programs
   that use a glibc 3.2 with sysenter support.
 - Security fixes for the 32bit ioctl handlers.  Also some simplications
   and speedups.
 - gcc 3.3-hammer compile fixes for inline assembly
 - Remove acpi.c file corpse.
 - Lots of warning fixes
 - Disable some Dprintks to make the bootup quieter again
 - Clean up ptrace a bit (together with warning fixes)
 - Merge with i386 (handle ACPI dynamic irq entries properly)
 - Disable change_page_attr in pci-gart for now.  Strictly that's
   incorrect, need to do more testing for the root cause of the current
   IOMMU problems.
 - Update defconfig
 - Disable first prefetch in copy_user that is likely to trigger Opteron
   Errata #91
 - More irqreturn_t fixes
 - Add pte_user and fix the vsyscall ptrace hack in generic code.
   It's still partly broken
 - Port verbose MCE handler from 2.4

21 files changed, 179 insertions, 83 deletions

diff --git a/include/asm-x86_64/apic.h b/include/asm-x86_64/apic.h
index bce8e2252907..a07746b1dc07 100644
--- a/include/asm-x86_64/apic.h
+++ b/include/asm-x86_64/apic.h
@@ -9,7 +9,7 @@
 
 #ifdef CONFIG_X86_LOCAL_APIC
 
-#define APIC_DEBUG 1
+#define APIC_DEBUG 0
 
 #if APIC_DEBUG
 #define Dprintk(x...) printk(x)
diff --git a/include/asm-x86_64/calling.h b/include/asm-x86_64/calling.h
index 8bbf1971fd7b..a3a04b806744 100644
--- a/include/asm-x86_64/calling.h
+++ b/include/asm-x86_64/calling.h
@@ -84,8 +84,9 @@
 	movq \offset+72(%rsp),%rax
 	.endm
 			
+#define REST_SKIP 6*8			
 	.macro SAVE_REST
-	subq $6*8,%rsp
+	subq $REST_SKIP,%rsp
 	movq %rbx,5*8(%rsp) 
 	movq %rbp,4*8(%rsp) 
 	movq %r12,3*8(%rsp) 
@@ -94,7 +95,6 @@
 	movq %r15,(%rsp) 
 	.endm		
 
-#define REST_SKIP 6*8
 	.macro RESTORE_REST
 	movq (%rsp),%r15
 	movq 1*8(%rsp),%r14
diff --git a/include/asm-x86_64/compat.h b/include/asm-x86_64/compat.h
index b73ae6b2a85f..63064e779369 100644
--- a/include/asm-x86_64/compat.h
+++ b/include/asm-x86_64/compat.h
@@ -1,9 +1,11 @@
 #ifndef _ASM_X86_64_COMPAT_H
 #define _ASM_X86_64_COMPAT_H
+
 /*
  * Architecture specific compatibility types
  */
 #include <linux/types.h>
+#include <linux/sched.h>
 
 #define COMPAT_USER_HZ	100
 
diff --git a/include/asm-x86_64/debugreg.h b/include/asm-x86_64/debugreg.h
index 2c4fe65e69ac..bd1aab1d8c4a 100644
--- a/include/asm-x86_64/debugreg.h
+++ b/include/asm-x86_64/debugreg.h
@@ -58,7 +58,7 @@
    We can slow the instruction pipeline for instructions coming via the
    gdt or the ldt if we want to.  I am not sure why this is an advantage */
 
-#define DR_CONTROL_RESERVED (0xFFFFFFFFFC00) /* Reserved by Intel */
+#define DR_CONTROL_RESERVED (0xFFFFFFFF0000FC00UL) /* Reserved */
 #define DR_LOCAL_SLOWDOWN (0x100)   /* Local slow the pipeline */
 #define DR_GLOBAL_SLOWDOWN (0x200)  /* Global slow the pipeline */
 
diff --git a/include/asm-x86_64/e820.h b/include/asm-x86_64/e820.h
index 9b447dd69663..ad24c50d4967 100644
--- a/include/asm-x86_64/e820.h
+++ b/include/asm-x86_64/e820.h
@@ -50,7 +50,7 @@ extern void contig_e820_setup(void);
 extern unsigned long e820_end_of_ram(void);
 extern void e820_reserve_resources(void);
 extern void e820_print_map(char *who);
-extern int e820_mapped(unsigned long start, unsigned long end, int type);
+extern int e820_mapped(unsigned long start, unsigned long end, unsigned type);
 
 extern void e820_bootmem_free(pg_data_t *pgdat, unsigned long start,unsigned long end);
 
diff --git a/include/asm-x86_64/floppy.h b/include/asm-x86_64/floppy.h
index 738395d6d31c..f68f972f3685 100644
--- a/include/asm-x86_64/floppy.h
+++ b/include/asm-x86_64/floppy.h
@@ -64,7 +64,7 @@ static irqreturn_t floppy_hardint(int irq, void *dev_id, struct pt_regs * regs)
 #endif
 	if(!doing_pdma) {
 		floppy_interrupt(irq, dev_id, regs);
-		return;
+		return IRQ_HANDLED;
 	}
 
 #ifdef TRACE_FLPY_INT
@@ -96,7 +96,7 @@ static irqreturn_t floppy_hardint(int irq, void *dev_id, struct pt_regs * regs)
 	calls++;
 #endif
 	if(st == 0x20)
-		return;
+		return IRQ_HANDLED;
 	if(!(st & 0x20)) {
 		virtual_dma_residue += virtual_dma_count;
 		virtual_dma_count=0;
diff --git a/include/asm-x86_64/i387.h b/include/asm-x86_64/i387.h
index e5bc31f19001..b9ecf53519a2 100644
--- a/include/asm-x86_64/i387.h
+++ b/include/asm-x86_64/i387.h
@@ -17,6 +17,7 @@
 #include <asm/sigcontext.h>
 #include <asm/user.h>
 #include <asm/thread_info.h>
+#include <asm/uaccess.h>
 
 extern void fpu_init(void);
 extern void init_fpu(struct task_struct *child);
@@ -91,6 +92,8 @@ static inline int restore_fpu_checking(struct i387_fxsave_struct *fx)
 		     ".previous"
 		     : [err] "=r" (err)
 		     : [fx] "r" (fx), "0" (0)); 
+	if (unlikely(err))
+		init_fpu(current);
 	return err;
 } 
 
@@ -109,6 +112,8 @@ static inline int save_i387_checking(struct i387_fxsave_struct *fx)
 		     ".previous"
 		     : [err] "=r" (err)
 		     : [fx] "r" (fx), "0" (0)); 
+	if (unlikely(err))
+		__clear_user(fx, sizeof(struct i387_fxsave_struct));
 	return err;
 } 
 
diff --git a/include/asm-x86_64/ia32.h b/include/asm-x86_64/ia32.h
index a102c0ed785d..d558fc04fcdc 100644
--- a/include/asm-x86_64/ia32.h
+++ b/include/asm-x86_64/ia32.h
@@ -133,6 +133,25 @@ typedef struct siginfo32 {
 	} _sifields;
 } siginfo_t32;
 
+struct sigframe32
+{
+        u32 pretcode;
+        int sig;
+        struct sigcontext_ia32 sc;
+        struct _fpstate_ia32 fpstate;
+        unsigned int extramask[_COMPAT_NSIG_WORDS-1];
+};
+
+struct rt_sigframe32
+{
+        u32 pretcode;
+        int sig;
+        u32 pinfo;
+        u32 puc;
+        struct siginfo32 info;
+        struct ucontext_ia32 uc;
+        struct _fpstate_ia32 fpstate;
+};
 
 struct ustat32 {
 	__u32	f_tfree;
diff --git a/include/asm-x86_64/io.h b/include/asm-x86_64/io.h
index e6614c5ebb1f..7d80bcbc798e 100644
--- a/include/asm-x86_64/io.h
+++ b/include/asm-x86_64/io.h
@@ -120,6 +120,7 @@ __OUTS(l)
   #define __io_virt(x) ((void *)(x))
 #endif
 
+#ifndef __i386__
 /*
  * Change virtual addresses to physical addresses and vv.
  * These are pretty trivial
@@ -133,6 +134,7 @@ extern inline void * phys_to_virt(unsigned long address)
 {
 	return __va(address);
 }
+#endif
 
 /*
  * Change "struct page" to physical address.
@@ -259,6 +261,7 @@ out:
 	return retval;
 }
 
+#ifndef __i386__
 /**
  *	isa_check_signature		-	find BIOS signatures
  *	@io_addr: mmio address to check 
@@ -288,6 +291,7 @@ static inline int isa_check_signature(unsigned long io_addr,
 out:
 	return retval;
 }
+#endif
 
 /* Nothing to do */
 
diff --git a/include/asm-x86_64/irq.h b/include/asm-x86_64/irq.h
index 16c5cea93ab7..7415660ffb69 100644
--- a/include/asm-x86_64/irq.h
+++ b/include/asm-x86_64/irq.h
@@ -32,4 +32,8 @@ extern void disable_irq(unsigned int);
 extern void disable_irq_nosync(unsigned int);
 extern void enable_irq(unsigned int);
 
+#ifdef CONFIG_X86_LOCAL_APIC
+#define ARCH_HAS_NMI_WATCHDOG		/* See include/linux/nmi.h */
+#endif
+
 #endif /* _ASM_IRQ_H */
diff --git a/include/asm-x86_64/page.h b/include/asm-x86_64/page.h
index a0399da03099..e4198d618b26 100644
--- a/include/asm-x86_64/page.h
+++ b/include/asm-x86_64/page.h
@@ -58,9 +58,13 @@ typedef struct { unsigned long pgprot; } pgprot_t;
 #define __pte(x) ((pte_t) { (x) } )
 #define __pmd(x) ((pmd_t) { (x) } )
 #define __pgd(x) ((pgd_t) { (x) } )
-#define __level4(x) ((level4_t) { (x) } )
+#define __pml4(x) ((pml4_t) { (x) } )
 #define __pgprot(x)	((pgprot_t) { (x) } )
 
+extern unsigned long vm_stack_flags, vm_stack_flags32;
+extern unsigned long vm_data_default_flags, vm_data_default_flags32;
+extern unsigned long vm_force_exec32;
+
 #endif /* !__ASSEMBLY__ */
 
 /* to align the pointer to the (next) page boundary */
@@ -119,9 +123,19 @@ extern __inline__ int get_order(unsigned long size)
 #define virt_addr_valid(kaddr)	pfn_valid(__pa(kaddr) >> PAGE_SHIFT)
 #define pfn_to_kaddr(pfn)      __va((pfn) << PAGE_SHIFT)
 
-#define VM_DATA_DEFAULT_FLAGS  (VM_READ | VM_WRITE | VM_EXEC | \
+#define __VM_DATA_DEFAULT_FLAGS	(VM_READ | VM_WRITE | VM_EXEC | \
+				 VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
+#define __VM_STACK_FLAGS 	(VM_GROWSDOWN | VM_READ | VM_WRITE | VM_EXEC | \
                                 VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
 
+#define VM_DATA_DEFAULT_FLAGS \
+	(test_thread_flag(TIF_IA32) ? vm_data_default_flags32 : \
+	  vm_data_default_flags) 
+
+#define VM_STACK_DEFAULT_FLAGS \
+	(test_thread_flag(TIF_IA32) ? vm_stack_flags32 : vm_stack_flags) 
+	
+
 #endif /* __KERNEL__ */
 
 #endif /* _X86_64_PAGE_H */
diff --git a/include/asm-x86_64/pci.h b/include/asm-x86_64/pci.h
index d94c514a06ab..4fd91a20ab84 100644
--- a/include/asm-x86_64/pci.h
+++ b/include/asm-x86_64/pci.h
@@ -76,8 +76,8 @@ extern void pci_free_consistent(struct pci_dev *hwdev, size_t size,
  * Once the device is given the dma address, the device owns this memory
  * until either pci_unmap_single or pci_dma_sync_single is performed.
  */
-extern dma_addr_t __pci_map_single(struct pci_dev *hwdev, void *ptr,
-			  size_t size, int direction, int flush);
+extern dma_addr_t pci_map_single(struct pci_dev *hwdev, void *ptr, size_t size, 
+				 int direction);
 
 
 void pci_unmap_single(struct pci_dev *hwdev, dma_addr_t addr,
@@ -126,8 +126,8 @@ static inline void pci_dma_sync_sg(struct pci_dev *hwdev,
 
 
 #else
-static inline dma_addr_t __pci_map_single(struct pci_dev *hwdev, void *ptr,
-					size_t size, int direction, int flush)
+static inline dma_addr_t pci_map_single(struct pci_dev *hwdev, void *ptr,
+					size_t size, int direction)
 {
 	dma_addr_t addr; 
 
@@ -214,12 +214,6 @@ extern int pci_map_sg(struct pci_dev *hwdev, struct scatterlist *sg,
 extern void pci_unmap_sg(struct pci_dev *hwdev, struct scatterlist *sg,
 			 int nents, int direction);
 
-static inline dma_addr_t pci_map_single(struct pci_dev *hwdev, void *ptr,
-			  size_t size, int direction)
-{
-	return __pci_map_single(hwdev,ptr,size,direction,1); 
-}
-
 #define pci_unmap_page pci_unmap_single
 
 /* Return whether the given PCI device DMA address mask can
diff --git a/include/asm-x86_64/pda.h b/include/asm-x86_64/pda.h
index da8b4f570d24..dbb63c9047d7 100644
--- a/include/asm-x86_64/pda.h
+++ b/include/asm-x86_64/pda.h
@@ -46,11 +46,11 @@ extern void __bad_pda_field(void);
 #define pda_to_op(op,field,val) do { \
        switch (sizeof_field(struct x8664_pda, field)) { 		\
 case 2: \
-asm volatile(op "w %0,%%gs:%c1"::"r" (val),"i"(pda_offset(field)):"memory"); break; \
+asm volatile(op "w %0,%%gs:%P1"::"r" (val),"i"(pda_offset(field)):"memory"); break; \
 case 4: \
-asm volatile(op "l %0,%%gs:%c1"::"r" (val),"i"(pda_offset(field)):"memory"); break; \
+asm volatile(op "l %0,%%gs:%P1"::"r" (val),"i"(pda_offset(field)):"memory"); break; \
 case 8: \
-asm volatile(op "q %0,%%gs:%c1"::"r" (val),"i"(pda_offset(field)):"memory"); break; \
+asm volatile(op "q %0,%%gs:%P1"::"r" (val),"i"(pda_offset(field)):"memory"); break; \
        default: __bad_pda_field(); 					\
        } \
        } while (0)
@@ -63,11 +63,11 @@ asm volatile(op "q %0,%%gs:%c1"::"r" (val),"i"(pda_offset(field)):"memory"); bre
        typedef typeof_field(struct x8664_pda, field) T__; T__ ret__; \
        switch (sizeof_field(struct x8664_pda, field)) { 		\
 case 2: \
-asm volatile(op "w %%gs:%c1,%0":"=r" (ret__):"i"(pda_offset(field)):"memory"); break;\
+asm volatile(op "w %%gs:%P1,%0":"=r" (ret__):"i"(pda_offset(field)):"memory"); break;\
 case 4: \
-asm volatile(op "l %%gs:%c1,%0":"=r" (ret__):"i"(pda_offset(field)):"memory"); break;\
+asm volatile(op "l %%gs:%P1,%0":"=r" (ret__):"i"(pda_offset(field)):"memory"); break;\
 case 8: \
-asm volatile(op "q %%gs:%c1,%0":"=r" (ret__):"i"(pda_offset(field)):"memory"); break;\
+asm volatile(op "q %%gs:%P1,%0":"=r" (ret__):"i"(pda_offset(field)):"memory"); break;\
        default: __bad_pda_field(); 					\
        } \
        ret__; })
diff --git a/include/asm-x86_64/pgtable.h b/include/asm-x86_64/pgtable.h
index 7e6a4b577bae..dc105425b961 100644
--- a/include/asm-x86_64/pgtable.h
+++ b/include/asm-x86_64/pgtable.h
@@ -28,6 +28,7 @@ extern unsigned long __supported_pte_mask;
 #define swapper_pg_dir NULL
 
 extern void paging_init(void);
+extern void clear_kernel_mapping(unsigned long addr, unsigned long size);
 
 extern unsigned long pgkern_mask;
 
@@ -165,11 +166,11 @@ static inline void set_pml4(pml4_t *dst, pml4_t val)
 #define PAGE_NONE	__pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED)
 #define PAGE_SHARED	__pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | _PAGE_ACCESSED | _PAGE_NX)
 #define PAGE_SHARED_EXEC __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | _PAGE_ACCESSED)
-#define PAGE_COPY    __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED | _PAGE_NX)
+#define PAGE_COPY_NOEXEC __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED | _PAGE_NX)
+#define PAGE_COPY PAGE_COPY_NOEXEC
 #define PAGE_COPY_EXEC __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED)
 #define PAGE_READONLY	__pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED | _PAGE_NX)
 #define PAGE_READONLY_EXEC __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED)
-
 #define __PAGE_KERNEL \
 	(_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_NX)
 #define __PAGE_KERNEL_EXECUTABLE \
@@ -181,7 +182,7 @@ static inline void set_pml4(pml4_t *dst, pml4_t val)
 #define __PAGE_KERNEL_VSYSCALL \
 	(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED)
 #define __PAGE_KERNEL_LARGE \
-	(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED | _PAGE_PSE | _PAGE_NX)
+	(__PAGE_KERNEL | _PAGE_PSE)
 
 #define MAKE_GLOBAL(x) __pgprot((x) | _PAGE_GLOBAL)
 
@@ -241,6 +242,7 @@ static inline pte_t pfn_pte(unsigned long page_nr, pgprot_t pgprot)
  * The following only work if pte_present() is true.
  * Undefined behaviour if not..
  */
+static inline int pte_user(pte_t pte)		{ return pte_val(pte) & _PAGE_USER; }
 extern inline int pte_read(pte_t pte)		{ return pte_val(pte) & _PAGE_USER; }
 extern inline int pte_exec(pte_t pte)		{ return pte_val(pte) & _PAGE_USER; }
 extern inline int pte_dirty(pte_t pte)		{ return pte_val(pte) & _PAGE_DIRTY; }
diff --git a/include/asm-x86_64/processor.h b/include/asm-x86_64/processor.h
index 7021042a4444..c1fbae233c35 100644
--- a/include/asm-x86_64/processor.h
+++ b/include/asm-x86_64/processor.h
@@ -218,14 +218,18 @@ struct tss_struct {
 
 struct thread_struct {
 	unsigned long	rsp0;
-	unsigned long	rip;
 	unsigned long	rsp;
 	unsigned long 	userrsp;	/* Copy from PDA */ 
 	unsigned long	fs;
 	unsigned long	gs;
 	unsigned short	es, ds, fsindex, gsindex;	
 /* Hardware debugging registers */
-	unsigned long	debugreg[8];  /* %%db0-7 debug registers */
+	unsigned long	debugreg0;  
+	unsigned long	debugreg1;  
+	unsigned long	debugreg2;  
+	unsigned long	debugreg3;  
+	unsigned long	debugreg6;  
+	unsigned long	debugreg7;  
 /* fault info */
 	unsigned long	cr2, trap_no, error_code;
 /* floating point info */
@@ -304,17 +308,30 @@ extern inline void sync_core(void)
 
 #define cpu_has_fpu 1
 
-#if 0
-/* disabled for now to work around opteron errata #91. Also gcc 3.2
-   doesn't like this in some cases. */
+/* Some early Opteron versions incorrectly fault on prefetch (errata #91). 
+   If this happens just jump back. */
 #define ARCH_HAS_PREFETCH
-#define prefetch(x) __builtin_prefetch((x),0,1)
-#endif
+static inline void prefetch(void *x) 
+{ 
+	asm volatile("2: prefetchnta %0\n1:\t" 
+		    ".section __ex_table,\"a\"\n\t"
+		    "  .align 8\n\t"
+		    "  .quad  2b,1b\n\t"
+		    ".previous" :: "m" (*(unsigned long *)x));
+} 
 
 #define ARCH_HAS_PREFETCHW
+static inline void prefetchw(void *x) 
+{ 
+	asm volatile("2: prefetchw %0\n1:\t" 
+		    ".section __ex_table,\"a\"\n\t"
+		    "  .align 8\n\t"
+		    "  .quad  2b,1b\n\t"
+		    ".previous" :: "m" (*(unsigned long *)x));
+} 
+
 #define ARCH_HAS_SPINLOCK_PREFETCH
 
-#define prefetchw(x) __builtin_prefetch((x),1,1)
 #define spin_lock_prefetch(x)  prefetchw(x)
 #define cpu_relax()   rep_nop()
 
diff --git a/include/asm-x86_64/segment.h b/include/asm-x86_64/segment.h
index d8a494f1cd69..848141b81158 100644
--- a/include/asm-x86_64/segment.h
+++ b/include/asm-x86_64/segment.h
@@ -18,6 +18,7 @@
 #define __USER_CS     0x33   /* 6*8+3 */ 
 #define __USER32_DS	__USER_DS 
 #define __KERNEL16_CS	(GDT_ENTRY_KERNELCS16 * 8)
+#define __KERNEL_COMPAT32_CS   0x8
 
 #define GDT_ENTRY_TLS 1
 #define GDT_ENTRY_TSS 8	/* needs two entries */
diff --git a/include/asm-x86_64/suspend.h b/include/asm-x86_64/suspend.h
index cc193889eefa..6ac4f0315185 100644
--- a/include/asm-x86_64/suspend.h
+++ b/include/asm-x86_64/suspend.h
@@ -54,13 +54,6 @@ extern unsigned long saved_ebx;
 extern unsigned long saved_esi;
 extern unsigned long saved_edi;
 
-static inline void acpi_save_register_state(unsigned long return_point)
-{
-	/* FIXME: This is probably no longer correct: we need to save all caller-saved registers */
-}
-
-#define acpi_restore_register_state()  do {} while (0)
-
 /* routines for saving/restoring kernel state */
 extern int acpi_save_state_mem(void);
 extern int acpi_save_state_disk(void);
diff --git a/include/asm-x86_64/system.h b/include/asm-x86_64/system.h
index a392e47b5fa7..e9efe11d03e7 100644
--- a/include/asm-x86_64/system.h
+++ b/include/asm-x86_64/system.h
@@ -16,9 +16,14 @@
 #define __STR(x) #x
 #define STR(x) __STR(x)
 
-#define __PUSH(x) "pushq %%" __STR(x) "\n\t"
-#define __POP(x)  "popq  %%" __STR(x) "\n\t"
+#define __SAVE(reg,offset) "movq %%" #reg ",(14-" #offset ")*8(%%rsp)\n\t"
+#define __RESTORE(reg,offset) "movq (14-" #offset ")*8(%%rsp),%%" #reg "\n\t"
 
+#ifdef CONFIG_X86_REMOTE_DEBUG
+
+/* full frame for the debug stub */
+/* Should be replaced with a dwarf2 cie/fde description, then gdb could
+   figure it out all by itself. */
 struct save_context_frame { 
 	unsigned long rbp; 
 	unsigned long rbx;
@@ -34,46 +39,64 @@ struct save_context_frame {
 	unsigned long r12;
 	unsigned long rdi;
 	unsigned long rsi;
+	unsigned long flags;
 }; 
 
-/* frame pointer must be last for get_wchan */
-/* It would be more efficient to let the compiler clobber most of these registers.
-   Clobbering all is not possible because that lets reload freak out. Even just 
-   clobbering six generates wrong code with gcc 3.1 for me so do it this way for now.
-   rbp needs to be always explicitly saved because gcc cannot clobber the
-   frame pointer and the scheduler is compiled with frame pointers. -AK */
 #define SAVE_CONTEXT \
-	__PUSH(rsi) __PUSH(rdi) \
-    __PUSH(r12) __PUSH(r13) __PUSH(r14) __PUSH(r15)  \
-	__PUSH(rdx) __PUSH(rcx) __PUSH(r8) __PUSH(r9) __PUSH(r10) __PUSH(r11)  \
-	__PUSH(rbx) __PUSH(rbp) 
+	"pushfq\n\t"							\
+	"subq $14*8,%%rsp\n\t" 						\
+	__SAVE(rbx, 12) __SAVE(rdi,  1)					\
+	__SAVE(rdx,  6) __SAVE(rcx,  7)					\
+	__SAVE(r8,   8) __SAVE(r9,   9)					\
+	__SAVE(r12,  2) __SAVE(r13,  3)					\
+	__SAVE(r14,  4) __SAVE(r15,  5)					\
+	__SAVE(r10, 10) __SAVE(r11, 11)					\
+	__SAVE(rsi, 0)  __SAVE(rbp, 13) 				\
+
+
 #define RESTORE_CONTEXT \
-	__POP(rbp) __POP(rbx) \
-	__POP(r11) __POP(r10) __POP(r9) __POP(r8) __POP(rcx) __POP(rdx) \
-	__POP(r15) __POP(r14) __POP(r13) __POP(r12) \
-	__POP(rdi) __POP(rsi)
+	__RESTORE(rbx, 12) __RESTORE(rdi,  1) 					\
+	__RESTORE(rdx,  6) __RESTORE(rcx,  7)					\
+	__RESTORE(r12,  2) __RESTORE(r13,  3)					\
+	__RESTORE(r14,  4) __RESTORE(r15,  5)					\
+	__RESTORE(r10, 10) __RESTORE(r11, 11)					\
+	__RESTORE(r8,   8) __RESTORE(r9,   9)					\
+	__RESTORE(rbp, 13) __RESTORE(rsi, 0) 		   		        \
+	"addq $14*8,%%rsp\n\t" 							\
+	"popfq\n\t"
+
+#define __EXTRA_CLOBBER 
 
-/* RED-PEN: pipeline stall on ret because it is not predicted */
-/* RED-PEN: the register saving could be optimized */
+#else
 /* frame pointer must be last for get_wchan */
+#define SAVE_CONTEXT    "pushfq ; pushq %%rbp ; movq %%rsi,%%rbp\n\t"
+#define RESTORE_CONTEXT "movq %%rbp,%%rsi ; popq %%rbp ; popfq\n\t" 
+
+#define __EXTRA_CLOBBER  \
+	,"rcx","rbx","rdx","r8","r9","r10","r11","r12","r13","r14","r15"
+#endif
 
 #define switch_to(prev,next,last) \
 	asm volatile(SAVE_CONTEXT						    \
-		     "movq %%rsp,%[prevrsp]\n\t"				    \
-		     "movq %[nextrsp],%%rsp\n\t"				    \
-		     "movq $thread_return,%[prevrip]\n\t"			   \
-		     "pushq %[nextrip]\n\t"					    \
-		     "jmp __switch_to\n\t"		\
+		     "movq %%rsp,%P[threadrsp](%[prev])\n\t" /* save RSP */	  \
+		     "movq %P[threadrsp](%[next]),%%rsp\n\t" /* restore RSP */	  \
+		     "call __switch_to\n\t"					  \
 		     ".globl thread_return\n"					\
 		     "thread_return:\n\t"					    \
+		     "movq %%gs:%P[pda_pcurrent],%%rsi\n\t"			  \
+		     "movq %P[thread_info](%%rsi),%%r8\n\t"			  \
+		     "btr  %[tif_fork],%P[ti_flags](%%r8)\n\t"			  \
+		     "movq %%rax,%%rdi\n\t" 					  \
+		     "jc   ret_from_fork\n\t"					  \
 		     RESTORE_CONTEXT						    \
-		     :[prevrsp] "=m" (prev->thread.rsp), 			    \
-		      [prevrip] "=m" (prev->thread.rip),		    	    \
-		      "=a" (last)						    \
-		     :[nextrsp] "m" (next->thread.rsp), 			    \
-		      [nextrip] "m" (next->thread.rip),				    \
-		      [next] "S" (next), [prev] "D" (prev)  			    \
-	             :"memory")
+		     : "=a" (last)					  	  \
+		     : [next] "S" (next), [prev] "D" (prev),			  \
+		       [threadrsp] "i" (offsetof(struct task_struct, thread.rsp)), \
+		       [ti_flags] "i" (offsetof(struct thread_info, flags)),\
+		       [tif_fork] "i" (TIF_FORK),			  \
+		       [thread_info] "i" (offsetof(struct task_struct, thread_info)), \
+		       [pda_pcurrent] "i" (offsetof(struct x8664_pda, pcurrent))   \
+		     : "memory", "cc" __EXTRA_CLOBBER)
     
 extern void load_gs_index(unsigned); 
 
@@ -88,14 +111,14 @@ extern void load_gs_index(unsigned);
 		"2:\n"				\
 		".section .fixup,\"ax\"\n"	\
 		"3:\t"				\
-		"pushq $0 ; popq %% " #seg "\n\t"	\
+		"movl %1,%%" #seg "\n\t" 	\
 		"jmp 2b\n"			\
 		".previous\n"			\
 		".section __ex_table,\"a\"\n\t"	\
 		".align 8\n\t"			\
 		".quad 1b,3b\n"			\
 		".previous"			\
-		: :"r" (value))
+		: :"r" (value), "r" (0))
 
 #define set_debug(value,register) \
                 __asm__("movq %0,%%db" #register  \
@@ -298,4 +321,7 @@ static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
 void disable_hlt(void);
 void enable_hlt(void);
 
+#define HAVE_EAT_KEY
+void eat_key(void);
+
 #endif
diff --git a/include/asm-x86_64/thread_info.h b/include/asm-x86_64/thread_info.h
index 1bdf0dd9914c..a4bbd2239afa 100644
--- a/include/asm-x86_64/thread_info.h
+++ b/include/asm-x86_64/thread_info.h
@@ -68,7 +68,7 @@ static inline struct thread_info *current_thread_info(void)
 static inline struct thread_info *stack_thread_info(void)
 {
 	struct thread_info *ti;
-	__asm__("andq %%rsp,%0; ":"=r" (ti) : "0" (~8191UL));
+	__asm__("andq %%rsp,%0; ":"=r" (ti) : "0" (~(THREAD_SIZE - 1)));
 	return ti;
 }
 
@@ -104,6 +104,7 @@ static inline struct thread_info *stack_thread_info(void)
 #define TIF_IRET		5	/* force IRET */
 #define TIF_POLLING_NRFLAG	16	/* true if poll_idle() is polling TIF_NEED_RESCHED */
 #define TIF_IA32		17	/* 32bit process */ 
+#define TIF_FORK		18	/* ret_from_fork */
 
 #define _TIF_SYSCALL_TRACE	(1<<TIF_SYSCALL_TRACE)
 #define _TIF_NOTIFY_RESUME	(1<<TIF_NOTIFY_RESUME)
@@ -113,6 +114,7 @@ static inline struct thread_info *stack_thread_info(void)
 #define _TIF_IRET		(1<<TIF_IRET)
 #define _TIF_POLLING_NRFLAG	(1<<TIF_POLLING_NRFLAG)
 #define _TIF_IA32		(1<<TIF_IA32)
+#define _TIF_FORK		(1<<TIF_FORK)
 
 #define _TIF_WORK_MASK		0x0000FFFE	/* work to do on interrupt/exception return */
 #define _TIF_ALLWORK_MASK	0x0000FFFF	/* work to do on any return to u-space */
diff --git a/include/asm-x86_64/user.h b/include/asm-x86_64/user.h
index 4d263733a864..12785c649ac5 100644
--- a/include/asm-x86_64/user.h
+++ b/include/asm-x86_64/user.h
@@ -52,13 +52,13 @@ struct user_i387_struct {
 	unsigned short	swd;
 	unsigned short	twd; /* Note this is not the same as the 32bit/x87/FSAVE twd */
 	unsigned short	fop;
-	u64	rip;
-	u64	rdp;
-	u32	mxcsr;
-	u32	mxcsr_mask;
-	u32	st_space[32];	/* 8*16 bytes for each FP-reg = 128 bytes */
-	u32	xmm_space[64];	/* 16*16 bytes for each XMM-reg = 256 bytes */
-	u32	padding[24];
+	__u64	rip;
+	__u64	rdp;
+	__u32	mxcsr;
+	__u32	mxcsr_mask;
+	__u32	st_space[32];	/* 8*16 bytes for each FP-reg = 128 bytes */
+	__u32	xmm_space[64];	/* 16*16 bytes for each XMM-reg = 256 bytes */
+	__u32	padding[24];
 };
 
 /*
diff --git a/include/asm-x86_64/vsyscall32.h b/include/asm-x86_64/vsyscall32.h
new file mode 100644
index 000000000000..36aa57397b95
--- /dev/null
+++ b/include/asm-x86_64/vsyscall32.h
@@ -0,0 +1,13 @@
+#ifndef _ASM_VSYSCALL32_H
+#define _ASM_VSYSCALL32_H 1
+
+/* Values need to match arch/x86_64/ia32/vsyscall.lds */
+
+#define VSYSCALL32_BASE 0xffffe000UL
+#define VSYSCALL32_EHDR ((const struct elf32_hdr *) VSYSCALL32_BASE)
+
+#define VSYSCALL32_VSYSCALL ((void *)VSYSCALL32_BASE + 0x400) 
+#define VSYSCALL32_SIGRETURN ((void *)VSYSCALL32_BASE + 0x500) 
+#define VSYSCALL32_RTSIGRETURN ((void *)VSYSCALL32_BASE + 0x500) 
+
+#endif