IB/mlx5: Use direct mkey destroy command upon UMR unreg failure

Use a direct firmware command to destroy the mkey in case the unreg UMR
operation has failed.

This prevents a case that a mkey will leak out from the cache post a
failure to be destroyed by a UMR WR.

In case the MR cache limit didn't reach a call to add another entry to the
cache instead of the destroyed one is issued.

In addition, replaced a warn message to WARN_ON() as this flow is fatal
and can't happen unless some bug around.

Link: https://lore.kernel.org/r/20190723065733.4899-4-leon@kernel.org
Cc: <stable@vger.kernel.org> # 4.10
Fixes: 49780d42dfc9 ("IB/mlx5: Expose MR cache for mlx5_ib")
Signed-off-by: Yishai Hadas <yishaih@mellanox.com>
Reviewed-by: Artemy Kovalyov <artemyko@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Reviewed-by: Jason Gunthorpe <jgg@mellanox.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>

diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 266edaf..b83361a 100644 (file)
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -545,13 +545,16 @@ void mlx5_mr_cache_free(struct mlx5_ib_dev *dev, struct mlx5_ib_mr *mr)
                return;
 
        c = order2idx(dev, mr->order);
-       if (c < 0 || c >= MAX_MR_CACHE_ENTRIES) {
-               mlx5_ib_warn(dev, "order %d, cache index %d\n", mr->order, c);
-               return;
-       }
+       WARN_ON(c < 0 || c >= MAX_MR_CACHE_ENTRIES);
 
-       if (unreg_umr(dev, mr))
+       if (unreg_umr(dev, mr)) {
+               mr->allocated_from_cache = false;
+               destroy_mkey(dev, mr);
+               ent = &cache->ent[c];
+               if (ent->cur < ent->limit)
+                       queue_work(cache->wq, &ent->work);
                return;
+       }
 
        ent = &cache->ent[c];
        spin_lock_irq(&ent->lock);