kevin/linux
1
Fork 0
Code
90151c35b1
linux/arch/x86/kernel
History
Stephane Eranian 90151c35b1 perf_events: Fix event scheduling issues introduced by transactional API 
The transactional API patch between the generic and model-specific
code introduced several important bugs with event scheduling, at
least on X86. If you had pinned events, e.g., watchdog,  and were
over-committing the PMU, you would get bogus counts. The bug was
showing up on Intel CPU because events would move around more
often that on AMD. But the problem also existed on AMD, though
harder to expose.

The issues were:

 - group_sched_in() was missing a cancel_txn() in the error path

 - cpuc->n_added was not properly maintained, leading to missing
   actions in hw_perf_enable(), i.e., n_running being 0. You cannot
   update n_added until you know the transaction has succeeded. In
   case of failed transaction n_added was not adjusted back.

 - in case of failed transactions, event_sched_out() was called
   and eventually invoked x86_disable_event() to touch the HW reg.
   But with transactions, on X86, event_sched_in() does not touch
   HW registers, it simply collects events into a list. Thus, you
   could end up calling x86_disable_event() on a counter which
   did not correspond to the current event when idx != -1.

The patch modifies the generic and X86 code to avoid all those problems.

First, we keep track of the number of events added last. In case the
transaction fails, we substract them from n_added. This approach is
necessary (as opposed to delaying updates to n_added) because not all
event updates use the transaction API, e.g., single events.

Second, we encapsulate the event_sched_in() and event_sched_out() in
group_sched_in() inside the transaction. That makes the operations
symmetrical and you can also detect that you are inside a transaction
and skip the HW reg access by checking cpuc->group_flag.

With this patch, you can now overcommit the PMU even with pinned
system-wide events present and still get valid counts.

Signed-off-by: Stephane Eranian <eranian@google.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1274796225.5882.1389.camel@twins>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-05-31 08:46:10 +02:00
..
acpi Merge branch 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6 2010-05-28 14:42:18 -07:00
apic x86, apic: ack all pending irqs when crashed/on kexec 2010-05-24 13:31:35 -07:00
cpu perf_events: Fix event scheduling issues introduced by transactional API 2010-05-31 08:46:10 +02:00
.gitignore
alternative.c Merge branch 'x86/asm' into x86/atomic 2010-04-29 16:53:17 -07:00
amd_iommu_init.c x86/amd-iommu: Add amd_iommu=off command line option 2010-05-11 17:12:33 +02:00
amd_iommu.c Merge branch 'iommu/largepages' into amd-iommu/2.6.35 2010-05-11 17:40:57 +02:00
apb_timer.c x86, mrst: Conditionally register cpu hotplug notifier for apbt 2010-04-20 14:38:28 -07:00
aperture_64.c x86/gart: Disable GART explicitly before initialization 2010-04-07 14:36:30 +02:00
apm_32.c i8253: Convert i8253_lock to raw_spinlock 2010-03-02 10:28:38 +01:00
asm-offsets_32.c lguest: optimize by coding restore_flags and irq_enable in assembler. 2009-06-12 22:27:03 +09:30
asm-offsets_64.c tracing: Define NR_syscalls for x86_64 2009-08-26 21:29:58 +02:00
asm-offsets.c
audit_64.c
bios_uv.c Merge branch 'x86-uv-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-02-28 11:00:55 -08:00
bootflag.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
check.c
cpuid.c x86: convert cpu notifier to return encapsulate errno value 2010-05-27 09:12:48 -07:00
crash_dump_32.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
crash_dump_64.c
crash.c Revert "x86: disable IOMMUs on kernel crash" 2010-04-07 11:51:17 +02:00
doublefault_32.c x86: Use get_desc_base() 2009-07-19 18:27:51 +02:00
dumpstack_32.c perf: Drop useless check for ignored frame 2010-01-13 10:09:08 +01:00
dumpstack_64.c perf/x86-64: Use frame pointer to walk on irq and process stacks 2010-03-10 14:26:40 +01:00
dumpstack.c x86, perf, bts, mm: Delete the never used BTS-ptrace code 2010-03-26 11:33:55 +01:00
dumpstack.h perf: Fix unsafe frame rewinding with hot regs fetching 2010-04-08 19:03:28 +02:00
e820.c x86: Make e820_remove_range to handle all covered case 2010-03-31 17:40:57 -07:00
early_printk.c earlyprintk,vga,kdb: Fix \b and \r for earlyprintk=vga with kdb 2010-05-20 21:04:31 -05:00
early-quirks.c
efi_32.c
efi_64.c x86: Make 64-bit efi_ioremap use ioremap on MMIO regions 2009-08-03 13:34:25 -07:00
efi_stub_32.S
efi_stub_64.S
efi.c x86: Remove trailing spaces in messages 2010-02-07 17:47:51 +01:00
entry_32.S x86-32: Rework cache flush denied handler 2010-05-03 13:39:26 -07:00
entry_64.S Merge branch 'x86-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-12-16 12:02:37 -08:00
ftrace.c Merge branch 'tracing/core' of git://git.kernel.org/pub/scm/linux/kernel/git/frederic/random-tracing into tracing/core 2010-02-27 10:06:10 +01:00
head32.c x86: Make sure free_init_pages() frees pages on page boundary 2010-03-29 18:55:33 +02:00
head64.c x86: Make sure free_init_pages() frees pages on page boundary 2010-03-29 18:55:33 +02:00
head_32.S Merge branch 'master' into percpu 2010-01-05 09:17:33 +09:00
head_64.S tree-wide: Assorted spelling fixes 2010-02-09 11:13:56 +01:00
head.c
hpet.c x86, hpet: Restrict read back to affected ATI chipsets 2010-04-28 18:14:29 -07:00
hw_breakpoint.c hw-breakpoints: Change/Enforce some breakpoints policies 2010-05-01 04:32:10 +02:00
i386_ksyms_32.c x86: Don't generate cmpxchg8b_emu if CONFIG_X86_CMPXCHG64=y 2009-10-01 08:42:24 +02:00
i387.c x86: Introduce 'struct fpu' and related API 2010-05-10 10:48:55 -07:00
i8237.c
i8253.c i8253: Convert i8253_lock to raw_spinlock 2010-03-02 10:28:38 +01:00
i8259.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
init_task.c Use new __init_task_data macro in arch init_task.c files. 2009-09-21 06:27:08 +02:00
io_delay.c
ioport.c x86-64, paravirt: Call set_iopl_mask() on 64 bits 2009-12-09 16:54:08 -08:00
irq_32.c x86: Unify fixup_irqs() for 32-bit and 64-bit kernels 2009-11-02 15:56:34 +01:00
irq_64.c x86: Unify fixup_irqs() for 32-bit and 64-bit kernels 2009-11-02 15:56:34 +01:00
irq.c genirq: Convert irq_desc.lock to raw_spinlock 2009-12-14 23:55:33 +01:00
irqinit.c x86: Merge simd_math_error() into math_error() 2010-05-03 13:39:29 -07:00
k8.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
kdebugfs.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
kgdb.c x86,kgdb: Implement early hardware breakpoint debugging 2010-05-20 21:04:30 -05:00
kprobes.c Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-05-18 08:19:03 -07:00
kvm.c KVM guest: do not batch pte updates from interrupt context 2009-09-10 18:10:50 +03:00
kvmclock.c x86, paravirt: don't compute pvclock adjustments if we trust the tsc 2010-05-19 11:41:05 +03:00
ldt.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
machine_kexec_32.c Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-12-08 13:27:33 -08:00
machine_kexec_64.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
Makefile x86, perf, bts, mm: Delete the never used BTS-ptrace code 2010-03-26 11:33:55 +01:00
mca_32.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
microcode_amd.c Revert "x86: ucode-amd: Load ucode-patches once ..." 2010-01-23 06:21:59 +01:00
microcode_core.c driver core: add devname module aliases to allow module on-demand auto-loading 2010-05-25 15:08:26 -07:00
microcode_intel.c x86: Improve Intel microcode loader performance 2010-03-11 13:49:06 +01:00
mmconf-fam10h_64.c x86: Move range related operation to one file 2010-02-10 17:47:17 -08:00
module.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
mpparse.c x86, ioapic: In mpparse use mp_register_ioapic 2010-05-04 13:34:59 -07:00
mrst.c x86, mrst: add nop functions to x86_init mpparse functions 2010-05-16 22:47:41 -07:00
msr.c x86: convert cpu notifier to return encapsulate errno value 2010-05-27 09:12:48 -07:00
olpc.c x86, olpc: Use pci subarch init for OLPC 2010-02-25 19:26:23 -08:00
paravirt_patch_32.c
paravirt_patch_64.c
paravirt-spinlocks.c locking: Convert __raw_spin* functions to arch_spin* 2009-12-14 23:55:32 +01:00
paravirt.c x86, paravirt: Remove kmap_atomic_pte paravirt op. 2010-02-27 14:41:35 -08:00
pci-calgary_64.c tree-wide: Assorted spelling fixes 2010-02-09 11:13:56 +01:00
pci-dma.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
pci-gart_64.c Merge branch 'iommu/fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/linux-2.6-iommu into x86/urgent 2010-04-13 13:24:54 +02:00
pci-nommu.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
pci-swiotlb.c x86: remove unnecessary sync_single_range_* in swiotlb_dma_ops 2010-05-27 09:12:52 -07:00
pcspeaker.c
pmtimer_64.c
probe_roms_32.c
process_32.c Merge branch 'x86-fpu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-05-18 08:58:16 -07:00
process_64.c Merge branch 'x86-fpu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-05-18 08:58:16 -07:00
process.c Merge branch 'x86-fpu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-05-18 08:58:16 -07:00
ptrace.c hw-breakpoints: Tag ptrace breakpoint as exclude_kernel 2010-05-01 04:32:07 +02:00
pvclock.c x86, paravirt: don't compute pvclock adjustments if we trust the tsc 2010-05-19 11:41:05 +03:00
quirks.c x86, hpet: Add reference to chipset erratum documentation for disable-hpet-msi-quirk 2010-05-17 10:04:43 -07:00
reboot_fixups_32.c cs5535: move the DIVIL MSR definition into linux/cs5535.h 2009-12-15 08:53:28 -08:00
reboot.c x86: Add iMac9,1 to pci_reboot_dmi_table 2010-02-17 08:08:21 +01:00
relocate_kernel_32.S
relocate_kernel_64.S
rtc.c Merge branch 'x86-platform-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-09-18 14:05:47 -07:00
scx200_32.c
setup_percpu.c numa: x86_64: use generic percpu var numa_node_id() implementation 2010-05-27 09:12:57 -07:00
setup.c x86, setup: Phoenix BIOS fixup is needed on Dell Inspiron Mini 1012 2010-05-24 13:33:14 -07:00
sfi.c x86, ioapic: Teach mp_register_ioapic to compute a global gsi_end 2010-05-04 13:34:56 -07:00
signal.c x86: Merge sys_sigaltstack 2009-12-09 16:28:59 -08:00
smp.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
smpboot.c x86: "nosmp" command line option should force the system into UP mode 2010-05-24 13:31:52 -07:00
stacktrace.c perf events, x86/stacktrace: Make stack walking optional 2009-12-17 09:56:19 +01:00
step.c x86, ptrace: Fix block-step 2010-03-26 11:33:57 +01:00
sys_i386_32.c Add generic sys_olduname() 2010-03-12 15:52:32 -08:00
sys_x86_64.c improve sys_newuname() for compat architectures 2010-03-12 15:52:32 -08:00
syscall_64.c
syscall_table_32.S Add generic sys_old_mmap() 2010-03-12 15:52:32 -08:00
tboot.c Merge branch 'kvm-updates/2.6.35' of git://git.kernel.org/pub/scm/virt/kvm/kvm 2010-05-21 17:16:21 -07:00
tce_64.c
test_nx.c
test_rodata.c
time.c x86: Convert i8259_lock to raw_spinlock 2010-02-16 18:21:32 +01:00
tlb_uv.c Merge branch 'x86-uv-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-05-18 09:46:35 -07:00
tls.c
tls.h
topology.c
trampoline_32.S x86: cpuinit-annotate SMP boot trampolines properly 2009-09-20 20:23:37 +02:00
trampoline_64.S x86: Fix Suspend to RAM freeze on Acer Aspire 1511Lmi laptop 2009-10-12 18:06:48 +02:00
trampoline.c x86: Use find_e820() instead of hard coded trampoline address 2009-12-11 09:28:22 +01:00
traps.c x86, kgdb: early trap init for early debug 2010-05-20 21:04:29 -05:00
tsc_sync.c locking: Convert __raw_spin* functions to arch_spin* 2009-12-14 23:55:32 +01:00
tsc.c Merge branch 'for-next' into for-linus 2010-03-08 16:55:37 +01:00
uv_irq.c Merge branch 'x86-uv-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-05-18 09:46:35 -07:00
uv_sysfs.c x86: Remove trailing spaces in messages 2010-02-07 17:47:51 +01:00
uv_time.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
verify_cpu_64.S
visws_quirks.c Merge remote branch 'origin/x86/apic' into x86/mrst 2010-02-22 16:25:18 -08:00
vm86_32.c x86, 32-bit: Convert sys_vm86 & sys_vm86old 2009-12-09 16:29:23 -08:00
vmi_32.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
vmiclock_32.c Merge branch 'for-next' into for-linus 2010-03-08 16:55:37 +01:00
vmlinux.lds.S x86: Make smp_locks end with page alignment 2010-03-29 18:42:30 +02:00
vsmp_64.c
vsyscall_64.c x86: Raise vsyscall priority on hotplug notifier chain 2010-03-01 12:35:40 -03:00
x86_init.c Merge branch 'x86-mrst-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2010-03-07 15:59:39 -08:00
x8664_ksyms_64.c x86-64: Don't export init_level4_pgt 2010-04-28 17:25:47 -07:00
xsave.c x86: Introduce 'struct fpu' and related API 2010-05-10 10:48:55 -07:00