OSDN Git Service

[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics
authorJay Foad <jay.foad@amd.com>
Tue, 7 Jan 2020 15:43:46 +0000 (15:43 +0000)
committerJay Foad <jay.foad@amd.com>
Fri, 17 Jul 2020 10:02:13 +0000 (11:02 +0100)
commit62fd7f767c4151b4ec3f4f703e73a27c52f5315c
tree0ce61d564f5597b8679078dc70ba1f398ffc99ed
parentf3fab392f57421a5bdabfb7e40820257d8f637b2
[MachineScheduler] Fix the TopDepth/BotHeightReduce latency heuristics

tryLatency compares two sched candidates. For the top zone it prefers
the one with lesser depth, but only if that depth is greater than the
total latency of the instructions we've already scheduled -- otherwise
its latency would be hidden and there would be no stall.

Unfortunately it only tests the depth of one of the candidates. This can
lead to situations where the TopDepthReduce heuristic does not kick in,
but a lower priority heuristic chooses the other candidate, whose depth
*is* greater than the already scheduled latency, which causes a stall.

The fix is to apply the heuristic if the depth of *either* candidate is
greater than the already scheduled latency.

All this also applies to the BotHeightReduce heuristic in the bottom
zone.

Differential Revision: https://reviews.llvm.org/D72392
114 files changed:
llvm/lib/CodeGen/MachineScheduler.cpp
llvm/test/CodeGen/AArch64/arm64-zero-cycle-zeroing.ll
llvm/test/CodeGen/AMDGPU/GlobalISel/cvt_f32_ubyte.ll
llvm/test/CodeGen/AMDGPU/GlobalISel/divergent-control-flow.ll
llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.atomic.inc.ll
llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.div.fmas.ll
llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.ubfe.ll
llvm/test/CodeGen/AMDGPU/GlobalISel/load-constant.96.ll
llvm/test/CodeGen/AMDGPU/GlobalISel/localizer.ll
llvm/test/CodeGen/AMDGPU/GlobalISel/zextload.ll
llvm/test/CodeGen/AMDGPU/add.v2i16.ll
llvm/test/CodeGen/AMDGPU/amdgpu-codegenprepare-idiv.ll
llvm/test/CodeGen/AMDGPU/bitreverse.ll
llvm/test/CodeGen/AMDGPU/bswap.ll
llvm/test/CodeGen/AMDGPU/chain-hi-to-lo.ll
llvm/test/CodeGen/AMDGPU/copy-illegal-type.ll
llvm/test/CodeGen/AMDGPU/ctlz.ll
llvm/test/CodeGen/AMDGPU/cttz_zero_undef.ll
llvm/test/CodeGen/AMDGPU/cvt_f32_ubyte.ll
llvm/test/CodeGen/AMDGPU/fcmp.f16.ll
llvm/test/CodeGen/AMDGPU/fneg-combines.ll
llvm/test/CodeGen/AMDGPU/idiv-licm.ll
llvm/test/CodeGen/AMDGPU/idot2.ll
llvm/test/CodeGen/AMDGPU/idot4s.ll
llvm/test/CodeGen/AMDGPU/idot4u.ll
llvm/test/CodeGen/AMDGPU/idot8s.ll
llvm/test/CodeGen/AMDGPU/idot8u.ll
llvm/test/CodeGen/AMDGPU/insert_vector_elt.v2i16.ll
llvm/test/CodeGen/AMDGPU/llvm.amdgcn.buffer.atomic.ll
llvm/test/CodeGen/AMDGPU/llvm.amdgcn.struct.buffer.atomic.ll
llvm/test/CodeGen/AMDGPU/llvm.amdgcn.ubfe.ll
llvm/test/CodeGen/AMDGPU/llvm.cos.f16.ll
llvm/test/CodeGen/AMDGPU/llvm.fma.f16.ll
llvm/test/CodeGen/AMDGPU/llvm.maxnum.f16.ll
llvm/test/CodeGen/AMDGPU/llvm.minnum.f16.ll
llvm/test/CodeGen/AMDGPU/llvm.round.f64.ll
llvm/test/CodeGen/AMDGPU/llvm.sin.f16.ll
llvm/test/CodeGen/AMDGPU/lshl64-to-32.ll
llvm/test/CodeGen/AMDGPU/lshr.v2i16.ll
llvm/test/CodeGen/AMDGPU/madak.ll
llvm/test/CodeGen/AMDGPU/max.i16.ll
llvm/test/CodeGen/AMDGPU/memory_clause.ll
llvm/test/CodeGen/AMDGPU/mul24-pass-ordering.ll
llvm/test/CodeGen/AMDGPU/promote-constOffset-to-imm.ll
llvm/test/CodeGen/AMDGPU/saddo.ll
llvm/test/CodeGen/AMDGPU/salu-to-valu.ll
llvm/test/CodeGen/AMDGPU/sched-assert-dead-def-subreg-use-other-subreg.mir
llvm/test/CodeGen/AMDGPU/sched-assert-onlydbg-value-empty-region.mir
llvm/test/CodeGen/AMDGPU/sdiv.ll
llvm/test/CodeGen/AMDGPU/sdiv64.ll
llvm/test/CodeGen/AMDGPU/sdwa-peephole.ll
llvm/test/CodeGen/AMDGPU/select.f16.ll
llvm/test/CodeGen/AMDGPU/shift-i64-opts.ll
llvm/test/CodeGen/AMDGPU/shl.ll
llvm/test/CodeGen/AMDGPU/shl.v2i16.ll
llvm/test/CodeGen/AMDGPU/shrink-add-sub-constant.ll
llvm/test/CodeGen/AMDGPU/sign_extend.ll
llvm/test/CodeGen/AMDGPU/sub.v2i16.ll
llvm/test/CodeGen/AMDGPU/trunc-combine.ll
llvm/test/CodeGen/AMDGPU/udiv64.ll
llvm/test/CodeGen/AMDGPU/urem64.ll
llvm/test/CodeGen/AMDGPU/v_madak_f16.ll
llvm/test/CodeGen/AMDGPU/vector-extract-insert.ll
llvm/test/CodeGen/AMDGPU/vector_shuffle.packed.ll
llvm/test/CodeGen/AMDGPU/widen-smrd-loads.ll
llvm/test/CodeGen/PowerPC/PR33671.ll
llvm/test/CodeGen/PowerPC/botheightreduce.mir [new file with mode: 0644]
llvm/test/CodeGen/PowerPC/canonical-merge-shuffles.ll
llvm/test/CodeGen/PowerPC/dform-adjust.ll
llvm/test/CodeGen/PowerPC/extract-and-store.ll
llvm/test/CodeGen/PowerPC/f128-aggregates.ll
llvm/test/CodeGen/PowerPC/f128-conv.ll
llvm/test/CodeGen/PowerPC/f128-passByValue.ll
llvm/test/CodeGen/PowerPC/float-load-store-pair.ll
llvm/test/CodeGen/PowerPC/handle-f16-storage-type.ll
llvm/test/CodeGen/PowerPC/load-shuffle-and-shuffle-store.ll
llvm/test/CodeGen/PowerPC/loop-instr-form-prepare.ll
llvm/test/CodeGen/PowerPC/machine-pre.ll
llvm/test/CodeGen/PowerPC/ppcf128-constrained-fp-intrinsics.ll
llvm/test/CodeGen/PowerPC/pr45432.ll
llvm/test/CodeGen/PowerPC/pr45448.ll
llvm/test/CodeGen/PowerPC/pr45628.ll
llvm/test/CodeGen/PowerPC/pre-inc-disable.ll
llvm/test/CodeGen/PowerPC/recipest.ll
llvm/test/CodeGen/PowerPC/rematerializable-instruction-machine-licm.ll
llvm/test/CodeGen/PowerPC/remove-redundant-load-imm.ll
llvm/test/CodeGen/PowerPC/scalar_vector_test_4.ll
llvm/test/CodeGen/PowerPC/sched-addi.ll
llvm/test/CodeGen/PowerPC/sms-cpy-1.ll
llvm/test/CodeGen/PowerPC/sms-grp-order.ll
llvm/test/CodeGen/PowerPC/sms-phi-3.ll
llvm/test/CodeGen/PowerPC/sms-simple.ll
llvm/test/CodeGen/PowerPC/srem-vector-lkk.ll
llvm/test/CodeGen/PowerPC/stack-clash-dynamic-alloca.ll
llvm/test/CodeGen/PowerPC/topdepthreduce-postra.mir [new file with mode: 0644]
llvm/test/CodeGen/PowerPC/uint-to-fp-v4i32.ll
llvm/test/CodeGen/PowerPC/unaligned-addressing-mode.ll
llvm/test/CodeGen/PowerPC/urem-vector-lkk.ll
llvm/test/CodeGen/PowerPC/vavg.ll
llvm/test/CodeGen/PowerPC/vec-bswap.ll
llvm/test/CodeGen/PowerPC/vec_conv_fp32_to_i16_elts.ll
llvm/test/CodeGen/PowerPC/vec_conv_fp32_to_i8_elts.ll
llvm/test/CodeGen/PowerPC/vec_conv_fp64_to_i16_elts.ll
llvm/test/CodeGen/PowerPC/vec_conv_fp64_to_i32_elts.ll
llvm/test/CodeGen/PowerPC/vec_conv_fp64_to_i8_elts.ll
llvm/test/CodeGen/PowerPC/vec_conv_i16_to_fp32_elts.ll
llvm/test/CodeGen/PowerPC/vec_conv_i16_to_fp64_elts.ll
llvm/test/CodeGen/PowerPC/vec_conv_i32_to_fp64_elts.ll
llvm/test/CodeGen/PowerPC/vec_conv_i64_to_fp32_elts.ll
llvm/test/CodeGen/PowerPC/vec_conv_i8_to_fp32_elts.ll
llvm/test/CodeGen/PowerPC/vec_conv_i8_to_fp64_elts.ll
llvm/test/CodeGen/PowerPC/vector-constrained-fp-intrinsics.ll
llvm/test/CodeGen/X86/testb-je-fusion.ll
llvm/test/CodeGen/X86/topdepthreduce-postra.mir [new file with mode: 0644]