OSDN Git Service

locking/qspinlock: Elide back-to-back RELEASE operations with smp_wmb()
authorWill Deacon <will.deacon@arm.com>
Thu, 26 Apr 2018 10:34:25 +0000 (11:34 +0100)
committerIngo Molnar <mingo@kernel.org>
Fri, 27 Apr 2018 07:48:52 +0000 (09:48 +0200)
commit9d4646d14d51d62b967a12452c30ea7edf8dd8fa
tree9867a52f6c68760063540e8344ebd340bd66eeba
parent626e5fbc14358901ddaa90ce510e0fbeab310432
locking/qspinlock: Elide back-to-back RELEASE operations with smp_wmb()

The qspinlock slowpath must ensure that the MCS node is fully initialised
before it can be reached by another other CPU. This is currently enforced
by using a RELEASE operation when updating the tail and also when linking
the node into the waitqueue, since the control dependency off xchg_tail
is insufficient to enforce sufficient ordering, see:

  95bcade33a8a ("locking/qspinlock: Ensure node is initialised before updating prev->next")

Back-to-back RELEASE operations may be expensive on some architectures,
particularly those that implement them using fences under the hood. We
can replace the two RELEASE operations with a single smp_wmb() fence and
use RELAXED operations for the subsequent publishing of the node.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Waiman Long <longman@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: boqun.feng@gmail.com
Cc: linux-arm-kernel@lists.infradead.org
Cc: paulmck@linux.vnet.ibm.com
Link: http://lkml.kernel.org/r/1524738868-31318-12-git-send-email-will.deacon@arm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
kernel/locking/qspinlock.c