kevin/linux
1
Fork 0
Code
a530b79586
linux/arch/x86/kernel
History
Tejun Heo a530b79586 percpu: teach large page allocator about NUMA 
Large page first chunk allocator is primarily used for NUMA machines;
however, its NUMA handling is extremely simplistic.  Regardless of
their proximity, each cpu is put into separate large page just to
return most of the allocated space back wasting large amount of
vmalloc space and increasing cache footprint.

This patch teachs NUMA details to large page allocator.  Given
processor proximity information, pcpu_lpage_build_unit_map() will find
fitting cpu -> unit mapping in which cpus in LOCAL_DISTANCE share the
same large page and not too much virtual address space is wasted.

This greatly reduces the unit and thus chunk size and wastes much less
address space for the first chunk.  For example, on 4/4 NUMA machine,
the original code occupied 16MB of virtual space for the first chunk
while the new code only uses 4MB - one 2MB page for each node.

[ Impact: much better space efficiency on NUMA machines ]

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Jan Beulich <JBeulich@novell.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Miller <davem@davemloft.net>
2009-07-04 08:11:00 +09:00
..
acpi Merge branches 'acerhdf', 'acpi-pci-bind', 'bjorn-pci-root', 'bugzilla-12904', 'bugzilla-13121', 'bugzilla-13396', 'bugzilla-13533', 'bugzilla-13612', 'c3_lock', 'hid-cleanups', 'misc-2.6.31', 'pdc-leak-fix', 'pnpacpi', 'power_nocheck', 'thinkpad_acpi', 'video' and 'wmi' into release 2009-06-24 01:19:50 -04:00
apic Intel-IOMMU, intr-remap: source-id checking 2009-06-23 22:09:17 +01:00
cpu Merge branch 'master' into for-next 2009-07-04 07:13:18 +09:00
.gitignore
alternative.c x86: expand irq-off region in text_poke() 2009-03-10 16:24:23 +01:00
amd_iommu_init.c amd-iommu: resume cleanup 2009-06-16 10:19:16 +02:00
amd_iommu.c amd-iommu: flush domain tlb when attaching a new device 2009-06-15 15:42:00 +02:00
aperture_64.c
apm_32.c PM core: rename suspend and resume functions 2009-06-12 21:32:31 +02:00
asm-offsets_32.c lguest: optimize by coding restore_flags and irq_enable in assembler. 2009-06-12 22:27:03 +09:30
asm-offsets_64.c x86, boot: make kernel_alignment adjustable; new bzImage fields 2009-05-11 17:44:39 -07:00
asm-offsets.c
audit_64.c
bios_uv.c x86, UV: system table in bios accessed after unmap 2009-04-03 19:25:57 +02:00
bootflag.c
check.c x86: fix 64k corruption-check 2009-03-15 07:03:15 +01:00
cpuid.c Driver Core: x86: add nodename for cpuid and msr drivers. 2009-06-15 21:30:25 -07:00
crash_dump_32.c
crash_dump_64.c
crash.c x86: disable IOMMUs on kernel crash 2009-06-15 15:20:40 +02:00
doublefault_32.c
ds_selftest.c x86, ds: fix buffer alignment in debug store selftest 2009-04-24 10:18:51 +02:00
ds_selftest.h x86, bts: cleanups 2009-03-13 11:57:22 +01:00
ds.c x86, ds: support Core i7 2009-04-07 13:36:36 +02:00
dumpstack_32.c
dumpstack_64.c
dumpstack.c x86: Add sysctl to allow panic on IOCK NMI error 2009-06-25 22:06:11 +02:00
dumpstack.h x86: avoid multiple declaration of kstack_depth_to_print 2009-04-14 11:45:33 +02:00
e820.c x86: add boundary check for 32bit res before expand e820 resource to alignment 2009-07-02 12:11:12 -07:00
early_printk.c x86: properly __init-annotate recent early_printk additions 2009-03-13 02:37:18 +01:00
early-quirks.c x86 early quirks: eliminate unused function 2009-04-08 14:16:32 +02:00
efi_32.c
efi_64.c Merge branch 'core/percpu' into percpu-cpumask-x86-for-linus-2 2009-03-27 17:28:43 +01:00
efi_stub_32.S Merge branch 'x86/asm' into x86/mm 2009-02-25 08:27:46 +01:00
efi_stub_64.S x86: efi_stub_32,64 - add missing ENDPROCs 2009-02-24 18:08:40 +01:00
efi.c x86: correct the conversion of EFI memory types 2009-06-16 17:47:32 -07:00
entry_32.S Merge branch 'tracing-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-06-20 10:56:46 -07:00
entry_64.S function-graph: add stack frame test 2009-06-18 18:40:18 -04:00
ftrace.c function-graph: add stack frame test 2009-06-18 18:40:18 -04:00
geode_32.c
head32.c x86-32: use brk segment for allocating initial kernel pagetable 2009-03-14 17:23:47 -07:00
head64.c x86: add brk allocation for very, very early allocations 2009-03-14 15:37:14 -07:00
head_32.S x86: de-assembler-ize asm/desc.h 2009-06-17 21:35:10 -07:00
head_64.S x86: de-assembler-ize asm/desc.h 2009-06-17 21:35:10 -07:00
head.c
hpet.c x86: hpet: Mark per cpu interrupts IRQF_TIMER to prevent resume failure 2009-06-14 18:24:29 +02:00
i386_ksyms_32.c
i387.c x86, math-emu: fix init_fpu for task != current 2009-03-04 20:33:16 +01:00
i8237.c
i8253.c time: move PIT_TICK_RATE to linux/timex.h 2009-06-16 19:47:27 -07:00
i8259.c x86: refactor x86_quirks support 2009-02-23 00:08:11 +01:00
init_task.c mm: consolidate init_mm definition 2009-06-16 19:47:28 -07:00
io_delay.c x86: io_delay.c cleanup 2009-03-21 16:57:04 +05:30
ioport.c x86-32: use non-lazy io bitmap context switching 2009-03-02 12:07:48 +01:00
irq_32.c Merge branch 'tj-percpu' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/misc into core/percpu 2009-02-24 21:52:45 +01:00
irq_64.c
irq.c Merge branch 'linus' into x86/mce3 2009-06-11 23:31:52 +02:00
irqinit.c Merge branch 'linus' into x86/mce3 2009-06-11 23:31:52 +02:00
k8.c
kdebugfs.c x86: kdebugfs.c cleanup 2009-03-21 16:55:45 +05:30
kgdb.c Merge branch 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-06-10 15:49:10 -07:00
kprobes.c Merge branch 'tracing/core-v2' into tracing-for-linus 2009-04-02 00:49:02 +02:00
kvm.c Merge branch 'kvm-updates/2.6.31' of git://git.kernel.org/pub/scm/virt/kvm/kvm 2009-06-11 10:03:30 -07:00
kvmclock.c clocksource: pass clocksource to read() callback 2009-04-21 13:41:47 -07:00
ldt.c
machine_kexec_32.c x86, kexec: fix crashdump panic with CONFIG_KEXEC_JUMP 2009-05-07 22:01:05 -07:00
machine_kexec_64.c x86, kexec: fix crashdump panic with CONFIG_KEXEC_JUMP 2009-05-07 22:01:05 -07:00
Makefile gcov: enable GCOV_PROFILE_ALL for x86_64 2009-06-18 13:03:58 -07:00
mca_32.c x86: refactor x86_quirks support 2009-02-23 00:08:11 +01:00
mfgpt_32.c cpumask: remove references to struct irqaction's mask field. 2009-03-30 22:05:14 +10:30
microcode_amd.c x86, microcode: Simplify vfree() use 2009-06-07 16:35:11 +02:00
microcode_core.c Driver Core: misc: add nodename support for misc devices. 2009-06-15 21:30:25 -07:00
microcode_intel.c x86: microcode: use smp_call_function_single instead of set_cpus_allowed, cleanup of synchronization logic 2009-05-12 10:36:44 +02:00
mmconf-fam10h_64.c x86: move various CPU initialization objects into .cpuinit.rodata 2009-03-12 13:13:07 +01:00
module.c module: cleanup FIXME comments about trimming exception table entries. 2009-06-12 21:47:05 +09:30
mpparse.c x86, irq: don't call mp_config_acpi_gsi() if update_mptable is not enabled 2009-05-18 09:33:29 +02:00
msr.c Driver Core: x86: add nodename for cpuid and msr drivers. 2009-06-15 21:30:25 -07:00
olpc.c
paravirt_patch_32.c
paravirt_patch_64.c
paravirt-spinlocks.c
paravirt.c Merge branch 'x86-xen-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-06-10 16:16:27 -07:00
pci-calgary_64.c x86: calgary: remove IOMMU_DEBUG 2009-04-14 13:03:36 +02:00
pci-dma.c intel-iommu: Make iommu=pt work on i386 too 2009-07-01 18:56:16 +01:00
pci-gart_64.c x86: enable GART-IOMMU only after setting up protection methods 2009-06-06 09:42:09 +02:00
pci-nommu.c dma-mapping: replace all DMA_32BIT_MASK macro with DMA_BIT_MASK(32) 2009-04-07 08:31:11 -07:00
pci-swiotlb.c Merge git://git.infradead.org/~dwmw2/iommu-2.6.31 2009-06-22 21:38:22 -07:00
pcspeaker.c
pmtimer_64.c
probe_roms_32.c
process_32.c Merge branch 'tracing-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-06-10 19:53:40 -07:00
process_64.c Merge branch 'tracing-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-06-10 19:53:40 -07:00
process.c kmemcheck: add mm functions 2009-06-15 12:40:03 +02:00
ptrace.c x86, bts, mm: clean up buffer allocation 2009-04-24 10:18:52 +02:00
pvclock.c
quirks.c Merge branch 'linus' into x86/cpu 2009-06-07 12:22:15 +02:00
reboot_fixups_32.c
reboot.c x86: Add quirk for reboot stalls on a Dell Optiplex 360 2009-06-07 15:51:20 +02:00
relocate_kernel_32.S x86, kexec: fix kexec x86 coding style 2009-03-10 18:13:25 -07:00
relocate_kernel_64.S x86, kexec: x86_64: add kexec jump support for x86_64 2009-03-10 18:13:25 -07:00
rtc.c x86: rtc.c cleanup 2009-03-21 16:56:37 +05:30
scx200_32.c
setup_percpu.c percpu: teach large page allocator about NUMA 2009-07-04 08:11:00 +09:00
setup.c x86: Move init_gbpages() to setup_arch() 2009-06-23 10:33:32 +02:00
signal.c Merge branch 'linus' into x86/mce3 2009-06-11 23:31:52 +02:00
smp.c Merge branch 'linus' into x86/mce3 2009-06-11 23:31:52 +02:00
smpboot.c x86: make zap_low_mapping could be used early 2009-06-12 13:50:24 +03:00
stacktrace.c x86: add save_stack_trace_bp() for tracing from a specific stack frame 2009-06-12 23:01:05 +02:00
step.c
sys_i386_32.c
sys_x86_64.c
syscall_64.c
syscall_table_32.S Merge branch 'core/signal' into perfcounters/core 2009-04-30 21:16:49 +02:00
tce_64.c
test_nx.c
test_rodata.c
time_32.c x86: refactor x86_quirks support 2009-02-23 00:08:11 +01:00
time_64.c cpumask: remove references to struct irqaction's mask field. 2009-03-30 22:05:14 +10:30
tlb_uv.c x86: Fix uv bau sending buffer initialization 2009-06-24 17:33:58 +02:00
tls.c
tls.h
topology.c x86: topology.c cleanup 2009-03-21 16:55:24 +05:30
trampoline_32.S
trampoline_64.S
trampoline.c
traps.c x86: Add sysctl to allow panic on IOCK NMI error 2009-06-25 22:06:11 +02:00
tsc_sync.c x86: clean up arch/x86/kernel/tsc_sync.c a bit 2009-05-07 09:32:10 +02:00
tsc.c Merge branch 'sched-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-06-20 10:57:40 -07:00
uv_irq.c
uv_sysfs.c x86: prevent /sys/firmware/sgi_uv from being created on non-uv systems 2009-04-08 14:58:10 +02:00
uv_time.c uv_time: add parameter to uv_read_rtc() 2009-04-22 17:41:25 +02:00
verify_cpu_64.S
visws_quirks.c x86: convert obsolete irq_desc_t typedef to struct irq_desc 2009-03-11 09:49:01 +01:00
vm86_32.c Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-06-10 16:15:14 -07:00
vmi_32.c Merge commit 'origin/master' into for-linus/xen/master 2009-04-07 13:34:16 -07:00
vmiclock_32.c clocksource: pass clocksource to read() callback 2009-04-21 13:41:47 -07:00
vmlinux.lds.S x86: Provide _sdata in the vmlinux.lds.S file 2009-06-12 09:21:33 +02:00
vsmp_64.c Revert "x86: don't compile vsmp_64 for 32bit" 2009-03-25 21:34:28 +01:00
vsyscall_64.c x86: move rdtsc_barrier() into the TSC vread method 2009-05-28 14:15:54 +02:00
x8664_ksyms_64.c
xsave.c x86-64: fix FPU corruption with signals and preemption 2009-04-20 14:33:00 -07:00