diff options
| author | Chuck Lever <chuck.lever@oracle.com> | 2016-09-13 10:52:50 -0400 |
|---|---|---|
| committer | Ben Hutchings <ben@decadent.org.uk> | 2017-02-23 03:54:03 +0000 |
| commit | cd2abef604580d2864af4428da2f9c79c888fa17 (patch) | |
| tree | 6a18a1a9273f35387a53d3168dbe0e3b09b2c6d6 /include/linux/sunrpc | |
| parent | 4f00b3d78eaaf05193567df939b19aae3c708976 (diff) | |
svcrdma: Tail iovec leaves an orphaned DMA mapping
commit cace564f8b6260e806f5e28d7f192fd0e0c603ed upstream.
The ctxt's count field is overloaded to mean the number of pages in
the ctxt->page array and the number of SGEs in the ctxt->sge array.
Typically these two numbers are the same.
However, when an inline RPC reply is constructed from an xdr_buf
with a tail iovec, the head and tail often occupy the same page,
but each are DMA mapped independently. In that case, ->count equals
the number of pages, but it does not equal the number of SGEs.
There's one more SGE, for the tail iovec. Hence there is one more
DMA mapping than there are pages in the ctxt->page array.
This isn't a real problem until the server's iommu is enabled. Then
each RPC reply that has content in that iovec orphans a DMA mapping
that consists of real resources.
krb5i and krb5p always populate that tail iovec. After a couple
million sent krb5i/p RPC replies, the NFS server starts behaving
erratically. Reboot is needed to clear the problem.
Fixes: 9d11b51ce7c1 ("svcrdma: Fix send_reply() scatter/gather set-up")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: J. Bruce Fields <bfields@redhat.com>
[bwh: Backported to 3.16:
- Adjust context
- Drop changes to svc_rdma_bc_sendto()
- s/xprt->sc_pd->local_dma_lkey/xprt->sc_dma_lkey/
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Diffstat (limited to 'include/linux/sunrpc')
| -rw-r--r-- | include/linux/sunrpc/svc_rdma.h | 9 |
1 files changed, 9 insertions, 0 deletions
diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 5cf99a016368..45a69411dafc 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -83,6 +83,7 @@ struct svc_rdma_op_ctxt { unsigned long flags; enum dma_data_direction direction; int count; + unsigned int mapped_sges; struct ib_sge sge[RPCSVC_MAXPAGES]; struct page *pages[RPCSVC_MAXPAGES]; }; @@ -178,6 +179,14 @@ struct svcxprt_rdma { #define RPCRDMA_MAX_REQUESTS 16 #define RPCRDMA_MAX_REQ_SIZE 4096 +/* Track DMA maps for this transport and context */ +static inline void svc_rdma_count_mappings(struct svcxprt_rdma *rdma, + struct svc_rdma_op_ctxt *ctxt) +{ + ctxt->mapped_sges++; + atomic_inc(&rdma->sc_dma_used); +} + /* svc_rdma_marshal.c */ extern void svc_rdma_rcl_chunk_counts(struct rpcrdma_read_chunk *, int *, int *); |