drm/amdgpu: fix a potential deadlock in amdgpu_bo_create_restricted()

When ttm_bo_init() fails, the reservation mutex should be unlocked.

In debug build, the kernel reported "possible recursive locking
detected" in this codepath. For debugging purposes, I also added
a "WARN_ON(ww_mutex_is_locked())" when ttm_bo_init() fails and the
mutex was locked as expected.

This should fix (random) GPU hangs. The easy way to reproduce the
issue is to change the "Super Sampling" option from 1.0 to 2.0 in
Hitman. It will create a huge buffer, evict a bunch of buffers
(around ~5k) and deadlock.

This regression has been introduced pretty recently.

v2: only release the mutex if resv is NULL

Fixes: 12a852219583 ("drm/amdgpu: improve AMDGPU_GEM_CREATE_VRAM_CLEARED handling (v2)")
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
index da3a53a..993c003 100644 (file)
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
@@ -405,8 +405,11 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,
                        &bo->placement, page_align, !kernel, NULL,
                        acc_size, sg, resv ? resv : &bo->tbo.ttm_resv,
                        &amdgpu_ttm_bo_destroy);
-       if (unlikely(r != 0))
+       if (unlikely(r != 0)) {
+               if (!resv)
+                       ww_mutex_unlock(&bo->tbo.resv->lock);
                return r;
+       }
 
        bo->tbo.priority = ilog2(bo->tbo.num_pages);
        if (kernel)