kevin/linux
1
Fork 0
Code
4a5df37964
linux/fs/nfs
History
Mike Snitzer 867da60d46 nfs: avoid i_lock contention in nfs_clear_invalid_mapping 
Multi-threaded buffered reads to the same file exposed significant
inode spinlock contention in nfs_clear_invalid_mapping().

Eliminate this spinlock contention by checking flags without locking,
instead using smp_rmb and smp_load_acquire accordingly, but then take
spinlock and double-check these inode flags.

Also refactor nfs_set_cache_invalid() slightly to use
smp_store_release() to pair with nfs_clear_invalid_mapping()'s
smp_load_acquire().

While this fix is beneficial for all multi-threaded buffered reads
issued by an NFS client, this issue was identified in the context of
surprisingly low LOCALIO performance with 4K multi-threaded buffered
read IO.  This fix dramatically speeds up LOCALIO performance:

before: read: IOPS=1583k, BW=6182MiB/s (6482MB/s)(121GiB/20002msec)
after:  read: IOPS=3046k, BW=11.6GiB/s (12.5GB/s)(232GiB/20001msec)

Fixes: 17dfeb9113 ("NFS: Fix races in nfs_revalidate_mapping")
Signed-off-by: Mike Snitzer <snitzer@kernel.org>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Anna Schumaker <anna.schumaker@oracle.com>
2024-11-04 10:24:19 -05:00
..
blocklayout nfs/blocklayout: add support for NVMe 2024-07-12 11:35:50 -04:00
filelayout nfs: pass struct nfsd_file to nfs_init_pgio and nfs_init_commit 2024-09-23 15:03:30 -04:00
flexfilelayout pnfs/flexfiles: enable localio support 2024-09-23 15:03:30 -04:00
cache_lib.c
cache_lib.h
callback_proc.c pNFS: Add a flag argument to pnfs_destroy_layouts_byclid() 2024-07-08 13:47:26 -04:00
callback_xdr.c SUNRPC: Fix integer overflow in decode_rc_list() 2024-10-03 16:19:13 -04:00
callback.c sunrpc: allow svc threads to fail initialisation cleanly 2024-09-20 19:31:03 -04:00
callback.h NFSv4: Add CB_GETATTR support for delegated attributes 2024-07-08 13:47:25 -04:00
client.c nfs_common: fix localio to cope with racing nfs_local_probe() 2024-11-04 10:24:19 -05:00
delegation.c NFS: remove revoked delegation from server's delegation list 2024-10-09 15:39:22 -04:00
delegation.h NFSv4: Delegreturn must set m/atime when they are delegated 2024-07-08 13:47:25 -04:00
dir.c nfs: Annotate struct nfs_cache_array with __counted_by() 2024-09-23 15:03:12 -04:00
direct.c nfs: drop the incorrect assertion in nfs_swap_rw() 2024-06-24 20:52:11 -07:00
dns_resolve.c
dns_resolve.h
export.c
file.c fs: Convert aops->write_begin to take a folio 2024-08-07 11:33:21 +02:00
fs_context.c nfs: add 'noalignwrite' option for lock-less 'lost writes' prevention 2024-09-23 15:03:13 -04:00
fscache.c netfs: Speed up buffered reading 2024-09-12 12:20:41 +02:00
fscache.h netfs: Speed up buffered reading 2024-09-12 12:20:41 +02:00
getroot.c nfs: fix the comment of nfs_get_root 2024-09-23 15:03:13 -04:00
inode.c nfs: avoid i_lock contention in nfs_clear_invalid_mapping 2024-11-04 10:24:19 -05:00
internal.h nfs/localio: use dedicated workqueues for filesystem read and write 2024-09-23 15:03:30 -04:00
io.c
iostat.h mm: add comments for allocation helpers explaining why they are macros 2024-07-12 15:52:20 -07:00
Kconfig nfs_common: factor out nfs_errtbl and nfs_stat_to_errno 2024-09-23 15:03:29 -04:00
localio.c nfs_common: fix localio to cope with racing nfs_local_probe() 2024-11-04 10:24:19 -05:00
Makefile nfs: add LOCALIO support 2024-09-23 15:03:30 -04:00
mount_clnt.c NFS: remove unused struct 'mnt_fhstatus' 2024-07-08 13:47:24 -04:00
namespace.c
netns.h nfs: make the rpc_stat per net namespace 2024-03-09 09:14:51 -05:00
nfs2super.c fs: nfs: add missing MODULE_DESCRIPTION() macros 2024-07-08 13:47:24 -04:00
nfs2xdr.c nfs_common: factor out nfs_errtbl and nfs_stat_to_errno 2024-09-23 15:03:29 -04:00
nfs3_fs.h
nfs3acl.c
nfs3client.c NFS: enable nconnect for RDMA 2024-03-09 09:14:51 -05:00
nfs3proc.c Return the delegation when deleting sillyrenamed files 2024-07-08 13:47:26 -04:00
nfs3super.c fs: nfs: add missing MODULE_DESCRIPTION() macros 2024-07-08 13:47:24 -04:00
nfs3xdr.c nfs_common: factor out nfs_errtbl and nfs_stat_to_errno 2024-09-23 15:03:29 -04:00
nfs4_fs.h nfs: simplify and guarantee owner uniqueness. 2024-09-23 15:03:12 -04:00
nfs4client.c NFSv4.1 another fix for EXCHGID4_FLAG_USE_PNFS_DS for DS server 2024-07-08 13:47:27 -04:00
nfs4file.c filelock: split leases out of struct file_lock 2024-02-05 13:11:44 +01:00
nfs4getroot.c
nfs4idmap.c
nfs4idmap.h
nfs4namespace.c
nfs4proc.c NFS: Fix attribute delegation behaviour on exclusive create 2024-11-04 10:24:19 -05:00
nfs4renewd.c
nfs4session.c
nfs4session.h
nfs4state.c NFSv4: Prevent NULL-pointer dereference in nfs42_complete_copies() 2024-10-03 16:19:13 -04:00
nfs4super.c fs: nfs: add missing MODULE_DESCRIPTION() macros 2024-07-08 13:47:24 -04:00
nfs4sysctl.c fs: Remove the now superfluous sentinel elements from ctl_table array 2023-12-28 04:57:57 -08:00
nfs4trace.c nfs/blocklayout: SCSI layout trace points for reservation key reg/unreg 2024-07-08 13:47:27 -04:00
nfs4trace.h nfs/blocklayout: SCSI layout trace points for reservation key reg/unreg 2024-07-08 13:47:27 -04:00
nfs4xdr.c NFS Client Updates for Linux 6.12 2024-09-24 15:44:18 -07:00
nfs42.h NFSv4.2: fix listxattr maximum XDR buffer size 2024-02-28 16:18:18 -05:00
nfs42proc.c NFSv4: Prevent NULL-pointer dereference in nfs42_complete_copies() 2024-10-03 16:19:13 -04:00
nfs42xattr.c mm, slab: remove last vestiges of SLAB_MEM_SPREAD 2024-03-12 20:32:19 -07:00
nfs42xdr.c
nfs.h
nfsroot.c NFS: Fix an off by one in root_nfs_cat() 2024-03-09 09:14:51 -05:00
nfstrace.c
nfstrace.h nfs: add LOCALIO support 2024-09-23 15:03:30 -04:00
pagelist.c nfs: enable localio for non-pNFS IO 2024-09-23 15:03:30 -04:00
pnfs_dev.c nfs: remove the unused max_deviceinfo_size field from struct pnfs_layoutdriver_type 2024-07-10 13:23:17 -04:00
pnfs_nfs.c nfs: pass struct nfsd_file to nfs_init_pgio and nfs_init_commit 2024-09-23 15:03:30 -04:00
pnfs.c NFSv4: Fix clearing of layout segments in layoutreturn 2024-08-22 17:01:10 -04:00
pnfs.h nfs: remove the unused max_deviceinfo_size field from struct pnfs_layoutdriver_type 2024-07-10 13:23:17 -04:00
proc.c Return the delegation when deleting sillyrenamed files 2024-07-08 13:47:26 -04:00
read.c nfs: Remove unnecessary NULL check before kfree() 2024-09-23 15:03:12 -04:00
super.c NFSv3: only use NFS timeout for MOUNT when protocols are compatible 2024-11-04 10:24:18 -05:00
symlink.c nfs: Remove calls to folio_set_error 2024-05-31 12:31:41 +02:00
sysctl.c fs: Remove the now superfluous sentinel elements from ctl_table array 2023-12-28 04:57:57 -08:00
sysfs.c
sysfs.h
unlink.c Return the delegation when deleting sillyrenamed files 2024-07-08 13:47:26 -04:00
write.c nfs: enable localio for non-pNFS IO 2024-09-23 15:03:30 -04:00