diff options
| author | Jason Xing <kernelxing@tencent.com> | 2025-10-31 18:33:28 +0800 |
|---|---|---|
| committer | Paolo Abeni <pabeni@redhat.com> | 2025-11-11 10:21:08 +0100 |
| commit | 8da7bea7db692e786165b71729fb68b7ff65ee56 (patch) | |
| tree | 86f436936e3aa1c281a7e80ce046e00bb97f86a2 /net/core | |
| parent | b981e100c19dcd91ce8cca8562c3cdabd4fcf28c (diff) | |
xsk: add indirect call for xsk_destruct_skb
Since Eric proposed an idea about adding indirect call wrappers for
UDP and managed to see a huge improvement[1], the same situation can
also be applied in xsk scenario.
This patch adds an indirect call for xsk and helps current copy mode
improve the performance by around 1% stably which was observed with
IXGBE at 10Gb/sec loaded. If the throughput grows, the positive effect
will be magnified. I applied this patch on top of batch xmit series[2],
and was able to see <5% improvement from our internal application
which is a little bit unstable though.
Use INDIRECT wrappers to keep xsk_destruct_skb static as it used to
be when the mitigation config is off.
Be aware of the freeing path that can be very hot since the frequency
can reach around 2,000,000 times per second with the xdpsock test.
[1]: https://lore.kernel.org/netdev/20251006193103.2684156-2-edumazet@google.com/
[2]: https://lore.kernel.org/all/20251021131209.41491-1-kerneljasonxing@gmail.com/
Suggested-by: Alexander Lobakin <aleksander.lobakin@intel.com>
Signed-off-by: Jason Xing <kernelxing@tencent.com>
Reviewed-by: Alexander Lobakin <aleksander.lobakin@intel.com>
Link: https://patch.msgid.link/20251031103328.95468-1-kerneljasonxing@gmail.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Diffstat (limited to 'net/core')
| -rw-r--r-- | net/core/skbuff.c | 8 |
1 files changed, 5 insertions, 3 deletions
diff --git a/net/core/skbuff.c b/net/core/skbuff.c index d95658b738d1..4f4d7ab7057f 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -81,6 +81,7 @@ #include <net/page_pool/helpers.h> #include <net/psp/types.h> #include <net/dropreason.h> +#include <net/xdp_sock.h> #include <linux/uaccess.h> #include <trace/events/skb.h> @@ -1140,12 +1141,13 @@ void skb_release_head_state(struct sk_buff *skb) if (skb->destructor) { DEBUG_NET_WARN_ON_ONCE(in_hardirq()); #ifdef CONFIG_INET - INDIRECT_CALL_3(skb->destructor, + INDIRECT_CALL_4(skb->destructor, tcp_wfree, __sock_wfree, sock_wfree, + xsk_destruct_skb, skb); #else - INDIRECT_CALL_1(skb->destructor, - sock_wfree, + INDIRECT_CALL_2(skb->destructor, + sock_wfree, xsk_destruct_skb, skb); #endif |