OSDN Git Service

[ThinLTO] Emit individual index files for distributed backends
authorTeresa Johnson <tejohnson@google.com>
Thu, 5 May 2016 13:44:56 +0000 (13:44 +0000)
committerTeresa Johnson <tejohnson@google.com>
Thu, 5 May 2016 13:44:56 +0000 (13:44 +0000)
commit0c4a898ea550699d1b2f4fe3767251c8f9a48d52
tree8847ba53b1677fdd2d56b032415a6b49a0678094
parentd11ea2f4ad864507e5ba977f200bca61254e94be
[ThinLTO] Emit individual index files for distributed backends

Summary:
When launching ThinLTO backends in a distributed build (currently
supported in gold via the thinlto-index-only plugin option), emit
an individual index file for each backend process as described here:
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html

The individual index file encodes the summary and module information
required for implementing the importing/exporting decisions made
for a given module in the thin link step.
This is in place of the current mechanism that uses the combined index
to make importing decisions in each back end independently. It is an
enabler for doing global summary based optimizations in the thin link
step (which will be recorded in the individual index files), and reduces
the size of the index that must be sent to each backend process, and
the amount of work to scan it in the backends.

Rather than create entirely new ModuleSummaryIndex structures (and all
the included unique_ptrs) for each backend index file, a map is created
to record all of the GUID and summary pointers needed for a particular
index file. The IndexBitcodeWriter walks this map instead of the full
index (hiding the details of managing the appropriate summary iteration
in a new iterator subclass). This is more efficient than walking the
entire combined index and filtering out just the needed summaries during
each backend bitcode index write.

Depends on D19481.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19556

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268627 91177308-0d34-0410-b5e6-96231b3b80d8
include/llvm/Bitcode/ReaderWriter.h
include/llvm/LTO/ThinLTOCodeGenerator.h
include/llvm/Transforms/IPO/FunctionImport.h
lib/Bitcode/Writer/BitcodeWriter.cpp
lib/LTO/ThinLTOCodeGenerator.cpp
lib/Transforms/IPO/FunctionImport.cpp
test/ThinLTO/X86/Inputs/distributed_indexes.ll [new file with mode: 0644]
test/ThinLTO/X86/distributed_indexes.ll [new file with mode: 0644]
test/tools/gold/X86/thinlto.ll
tools/gold/gold-plugin.cpp
tools/llvm-lto/llvm-lto.cpp