kevin/linux
1
Fork 0
Code
adc218676e
linux/fs/nfs
History
Mike Snitzer 867da60d46 nfs: avoid i_lock contention in nfs_clear_invalid_mapping 
Multi-threaded buffered reads to the same file exposed significant
inode spinlock contention in nfs_clear_invalid_mapping().

Eliminate this spinlock contention by checking flags without locking,
instead using smp_rmb and smp_load_acquire accordingly, but then take
spinlock and double-check these inode flags.

Also refactor nfs_set_cache_invalid() slightly to use
smp_store_release() to pair with nfs_clear_invalid_mapping()'s
smp_load_acquire().

While this fix is beneficial for all multi-threaded buffered reads
issued by an NFS client, this issue was identified in the context of
surprisingly low LOCALIO performance with 4K multi-threaded buffered
read IO.  This fix dramatically speeds up LOCALIO performance:

before: read: IOPS=1583k, BW=6182MiB/s (6482MB/s)(121GiB/20002msec)
after:  read: IOPS=3046k, BW=11.6GiB/s (12.5GB/s)(232GiB/20001msec)

Fixes: 17dfeb9113 ("NFS: Fix races in nfs_revalidate_mapping")
Signed-off-by: Mike Snitzer <snitzer@kernel.org>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Anna Schumaker <anna.schumaker@oracle.com>
2024-11-04 10:24:19 -05:00
..
blocklayout nfs/blocklayout: add support for NVMe 2024-07-12 11:35:50 -04:00
filelayout nfs: pass struct nfsd_file to nfs_init_pgio and nfs_init_commit 2024-09-23 15:03:30 -04:00
flexfilelayout pnfs/flexfiles: enable localio support 2024-09-23 15:03:30 -04:00
cache_lib.c
cache_lib.h NFS client updates for Linux 4.15 2017-11-17 14:18:00 -08:00
callback_proc.c pNFS: Add a flag argument to pnfs_destroy_layouts_byclid() 2024-07-08 13:47:26 -04:00
callback_xdr.c SUNRPC: Fix integer overflow in decode_rc_list() 2024-10-03 16:19:13 -04:00
callback.c sunrpc: allow svc threads to fail initialisation cleanly 2024-09-20 19:31:03 -04:00
callback.h NFSv4: Add CB_GETATTR support for delegated attributes 2024-07-08 13:47:25 -04:00
client.c nfs_common: fix localio to cope with racing nfs_local_probe() 2024-11-04 10:24:19 -05:00
delegation.c NFS: remove revoked delegation from server's delegation list 2024-10-09 15:39:22 -04:00
delegation.h NFSv4: Delegreturn must set m/atime when they are delegated 2024-07-08 13:47:25 -04:00
dir.c nfs: Annotate struct nfs_cache_array with __counted_by() 2024-09-23 15:03:12 -04:00
direct.c nfs: drop the incorrect assertion in nfs_swap_rw() 2024-06-24 20:52:11 -07:00
dns_resolve.c
dns_resolve.h NFS: Avoid memcpy() run-time warning for struct sockaddr overflows 2022-10-27 15:52:10 -04:00
export.c
file.c fs: Convert aops->write_begin to take a folio 2024-08-07 11:33:21 +02:00
fs_context.c nfs: add 'noalignwrite' option for lock-less 'lost writes' prevention 2024-09-23 15:03:13 -04:00
fscache.c netfs: Speed up buffered reading 2024-09-12 12:20:41 +02:00
fscache.h netfs: Speed up buffered reading 2024-09-12 12:20:41 +02:00
getroot.c nfs: fix the comment of nfs_get_root 2024-09-23 15:03:13 -04:00
inode.c nfs: avoid i_lock contention in nfs_clear_invalid_mapping 2024-11-04 10:24:19 -05:00
internal.h nfs/localio: use dedicated workqueues for filesystem read and write 2024-09-23 15:03:30 -04:00
io.c
iostat.h mm: add comments for allocation helpers explaining why they are macros 2024-07-12 15:52:20 -07:00
Kconfig nfs_common: factor out nfs_errtbl and nfs_stat_to_errno 2024-09-23 15:03:29 -04:00
localio.c nfs_common: fix localio to cope with racing nfs_local_probe() 2024-11-04 10:24:19 -05:00
Makefile nfs: add LOCALIO support 2024-09-23 15:03:30 -04:00
mount_clnt.c NFS: remove unused struct 'mnt_fhstatus' 2024-07-08 13:47:24 -04:00
namespace.c
netns.h
nfs2super.c fs: nfs: add missing MODULE_DESCRIPTION() macros 2024-07-08 13:47:24 -04:00
nfs2xdr.c nfs_common: factor out nfs_errtbl and nfs_stat_to_errno 2024-09-23 15:03:29 -04:00
nfs3_fs.h fs: drop unused posix acl handlers 2023-03-06 09:57:12 +01:00
nfs3acl.c
nfs3client.c
nfs3proc.c Return the delegation when deleting sillyrenamed files 2024-07-08 13:47:26 -04:00
nfs3super.c fs: nfs: add missing MODULE_DESCRIPTION() macros 2024-07-08 13:47:24 -04:00
nfs3xdr.c nfs_common: factor out nfs_errtbl and nfs_stat_to_errno 2024-09-23 15:03:29 -04:00
nfs4_fs.h nfs: simplify and guarantee owner uniqueness. 2024-09-23 15:03:12 -04:00
nfs4client.c NFSv4.1 another fix for EXCHGID4_FLAG_USE_PNFS_DS for DS server 2024-07-08 13:47:27 -04:00
nfs4file.c
nfs4getroot.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nfs4idmap.c
nfs4idmap.h
nfs4namespace.c
nfs4proc.c NFS: Fix attribute delegation behaviour on exclusive create 2024-11-04 10:24:19 -05:00
nfs4renewd.c treewide: remove editor modelines and cruft 2021-05-07 00:26:34 -07:00
nfs4session.c
nfs4session.h
nfs4state.c NFSv4: Prevent NULL-pointer dereference in nfs42_complete_copies() 2024-10-03 16:19:13 -04:00
nfs4super.c fs: nfs: add missing MODULE_DESCRIPTION() macros 2024-07-08 13:47:24 -04:00
nfs4sysctl.c
nfs4trace.c nfs/blocklayout: SCSI layout trace points for reservation key reg/unreg 2024-07-08 13:47:27 -04:00
nfs4trace.h nfs/blocklayout: SCSI layout trace points for reservation key reg/unreg 2024-07-08 13:47:27 -04:00
nfs4xdr.c NFS Client Updates for Linux 6.12 2024-09-24 15:44:18 -07:00
nfs42.h
nfs42proc.c NFSv4: Prevent NULL-pointer dereference in nfs42_complete_copies() 2024-10-03 16:19:13 -04:00
nfs42xattr.c mm, slab: remove last vestiges of SLAB_MEM_SPREAD 2024-03-12 20:32:19 -07:00
nfs42xdr.c NFSv4.2: Rework scratch handling for READ_PLUS (again) 2023-08-23 15:58:47 -04:00
nfs.h
nfsroot.c NFS: Fix an off by one in root_nfs_cat() 2024-03-09 09:14:51 -05:00
nfstrace.c
nfstrace.h nfs: add LOCALIO support 2024-09-23 15:03:30 -04:00
pagelist.c nfs: enable localio for non-pNFS IO 2024-09-23 15:03:30 -04:00
pnfs_dev.c nfs: remove the unused max_deviceinfo_size field from struct pnfs_layoutdriver_type 2024-07-10 13:23:17 -04:00
pnfs_nfs.c nfs: pass struct nfsd_file to nfs_init_pgio and nfs_init_commit 2024-09-23 15:03:30 -04:00
pnfs.c NFSv4: Fix clearing of layout segments in layoutreturn 2024-08-22 17:01:10 -04:00
pnfs.h nfs: remove the unused max_deviceinfo_size field from struct pnfs_layoutdriver_type 2024-07-10 13:23:17 -04:00
proc.c Return the delegation when deleting sillyrenamed files 2024-07-08 13:47:26 -04:00
read.c nfs: Remove unnecessary NULL check before kfree() 2024-09-23 15:03:12 -04:00
super.c NFSv3: only use NFS timeout for MOUNT when protocols are compatible 2024-11-04 10:24:18 -05:00
symlink.c nfs: Remove calls to folio_set_error 2024-05-31 12:31:41 +02:00
sysctl.c fs: Remove the now superfluous sentinel elements from ctl_table array 2023-12-28 04:57:57 -08:00
sysfs.c NFS: Fix sysfs server name memory leak 2023-08-19 10:26:29 -04:00
sysfs.h NFS: Add sysfs links to sunrpc clients for nfs_clients 2023-06-19 15:04:13 -04:00
unlink.c Return the delegation when deleting sillyrenamed files 2024-07-08 13:47:26 -04:00
write.c nfs: enable localio for non-pNFS IO 2024-09-23 15:03:30 -04:00