Fix sign-correcting of masked stencil reference.

The stencil test compares the unsigned masked reference value against
the unsigned masked stencil value. But because x86 vector instructions
only support signed comparison, we have to add 0x80 to both sides.
However, this offset was incorrectly added before the masking.

Bug b/64683344

Change-Id: I49748e25d8d99e1c2b3c87d3dbe74a9dba75fb1c
Reviewed-on: https://swiftshader-review.googlesource.com/12028
Tested-by: Nicolas Capens <nicolascapens@google.com>
Reviewed-by: Alexis Hétu <sugoi@google.com>
Reviewed-by: Nicolas Capens <nicolascapens@google.com>

diff --git a/src/Renderer/PixelProcessor.hpp b/src/Renderer/PixelProcessor.hpp
index dd54b72..65f6b92 100644 (file)
--- a/src/Renderer/PixelProcessor.hpp
+++ b/src/Renderer/PixelProcessor.hpp
@@ -149,7 +149,7 @@ namespace sw
                                writeMaskQ = replicate(writeMask);
                                invWriteMaskQ = ~writeMaskQ;
                                referenceMaskedQ = referenceQ & testMaskQ;
-                               referenceMaskedSignedQ = replicate((reference + 0x80) & 0xFF & testMask);
+                               referenceMaskedSignedQ = replicate(((reference & testMask) + 0x80) & 0xFF);
                        }
 
                        static int64_t replicate(int b)