1
linux/net
Jesper Dangaard Brouer be5dba25b4 net: page_pool: use alloc_pages_bulk in refill code path
There are cases where the page_pool need to refill with pages from the
page allocator.  Some workloads cause the page_pool to release pages
instead of recycling these pages.

For these workload it can improve performance to bulk alloc pages from the
page-allocator to refill the alloc cache.

For XDP-redirect workload with 100G mlx5 driver (that use page_pool)
redirecting xdp_frame packets into a veth, that does XDP_PASS to create an
SKB from the xdp_frame, which then cannot return the page to the
page_pool.

Performance results under GitHub xdp-project[1]:
 [1] https://github.com/xdp-project/xdp-project/blob/master/areas/mem/page_pool06_alloc_pages_bulk.org

Mel: The patch "net: page_pool: convert to use alloc_pages_bulk_array
variant" was squashed with this patch. From the test page, the array
variant was superior with one of the test results as follows.

	Kernel		XDP stats       CPU     pps           Delta
	Baseline	XDP-RX CPU      total   3,771,046       n/a
	List		XDP-RX CPU      total   3,940,242    +4.49%
	Array		XDP-RX CPU      total   4,249,224   +12.68%

Link: https://lkml.kernel.org/r/20210325114228.27719-10-mgorman@techsingularity.net
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Reviewed-by: Alexander Lobakin <alobakin@pm.me>
Cc: Alexander Duyck <alexander.duyck@gmail.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Chuck Lever <chuck.lever@oracle.com>
Cc: David Miller <davem@davemloft.net>
Cc: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2021-04-30 11:20:43 -07:00
..
6lowpan
9p
802
8021q Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2021-04-26 12:00:00 -07:00
appletalk
atm
ax25
batman-adv
bluetooth Networking changes for 5.13. 2021-04-29 11:57:23 -07:00
bpf
bpfilter
bridge net: bridge: mcast: fix broken length + header check for MRDv6 Adv. 2021-04-27 14:02:06 -07:00
caif
can
ceph
core net: page_pool: use alloc_pages_bulk in refill code path 2021-04-30 11:20:43 -07:00
dcb
dccp
decnet
dns_resolver
dsa net: selftest: fix build issue if INET is disabled 2021-04-28 14:06:45 -07:00
ethernet
ethtool
hsr
ieee802154
ife
ipv4 icmp: standardize naming of RFC 8335 PROBE constants 2021-04-28 13:42:23 -07:00
ipv6 net: bridge: mcast: fix broken length + header check for MRDv6 Adv. 2021-04-27 14:02:06 -07:00
iucv
kcm
key
l2tp net: fix a concurrency bug in l2tp_tunnel_register() 2021-04-27 14:23:13 -07:00
l3mdev
lapb
llc llc2: Remove redundant assignment to rc 2021-04-27 14:16:14 -07:00
mac80211 Networking changes for 5.13. 2021-04-29 11:57:23 -07:00
mac802154
mpls mpls: Remove redundant assignment to err 2021-04-27 14:17:00 -07:00
mptcp Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2021-04-26 12:00:00 -07:00
ncsi
netfilter netfilter: nft_socket: fix build with CONFIG_SOCK_CGROUP_DATA=n 2021-04-27 22:34:05 +02:00
netlabel Networking changes for 5.13. 2021-04-29 11:57:23 -07:00
netlink
netrom net: netrom: nr_in: Remove redundant assignment to ns 2021-04-28 13:59:08 -07:00
nfc Networking changes for 5.13. 2021-04-29 11:57:23 -07:00
nsh
openvswitch Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2021-04-26 12:00:00 -07:00
packet
phonet
psample
qrtr Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2021-04-26 12:00:00 -07:00
rds Networking changes for 5.13. 2021-04-29 11:57:23 -07:00
rfkill
rose
rxrpc Networking changes for 5.13. 2021-04-29 11:57:23 -07:00
sched net/sched: act_ct: Remove redundant ct get and check 2021-04-28 13:51:18 -07:00
sctp
smc net/smc: Remove redundant assignment to rc 2021-04-27 14:17:50 -07:00
strparser
sunrpc SUNRPC: refresh rq_pages using a bulk page allocator 2021-04-30 11:20:43 -07:00
switchdev
tipc Networking changes for 5.13. 2021-04-29 11:57:23 -07:00
tls net/tls: Remove redundant initialization of record 2021-04-27 14:15:19 -07:00
unix
vmw_vsock Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2021-04-26 12:00:00 -07:00
wireless Networking changes for 5.13. 2021-04-29 11:57:23 -07:00
x25
xdp
xfrm
compat.c
devres.c
Kconfig net: selftest: fix build issue if INET is disabled 2021-04-28 14:06:45 -07:00
Makefile
socket.c
sysctl_net.c