From 1d89880b681082c16eec5639678e38300f24a722 Mon Sep 17 00:00:00 2001 From: Keith Marshall Date: Fri, 31 Dec 2021 19:45:39 +0000 Subject: [PATCH] Publish MinGW linker's library search path HOWTO. * libpath.html: New file; adapted from original Drupal content, this describes the GCC/ld library search mechanism, and offers advice on installation of third party libraries, to take best advantage of this search mechanism. --- libpath.html | 729 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 729 insertions(+) create mode 100644 libpath.html diff --git a/libpath.html b/libpath.html new file mode 100644 index 0000000..5117c19 --- /dev/null +++ b/libpath.html @@ -0,0 +1,729 @@ + + + +
+

Introduction

+

This Mini‑HOWTO further develops the theme introduced by the +“Include File Search Path HOWTO”, +progressing to a complementary consideration of where the MinGW implementation +of GCC, (or more accurately, of the linker, ld, which is actually +a component of the GNU binutils package), searches for object +code libraries. +In particular, it address the question: “How do I ensure that the +MinGW linker will find my object libraries? ” +

+

In the case of header files, +we observed that the GCC Manual +makes a definitive statement, regarding the search path for header files, +(although it became significantly less definitive, +in the documentation for GCC‑7 and later ), +yet we also noted that this statement is not applicable to the MinGW +implementation of GCC. In the case of libraries, +the GCC Manual +makes a much less definitive statement, +which is paradoxically more applicable to the MinGW implementation, +regarding the default search path: +

+
+-llibrary
+-l library +

Search the library named library when linking. +(The second alternative +with the library as a separate argument is only for POSIX compliance and +is not recommended.) +

+

It makes a difference where in the command you write this option; the +linker searches and processes libraries and object files in the order they +are specified. +Thus, +‘foo.o -lz bar.o’ +searches library ‘z’ after file foo.o +but before bar.o. +If bar.o refers to functions in ‘z’, +those functions may +not be loaded. +

+

The linker searches a standard list of directories for the library, which is +actually a file named liblibrary.a. The linker then uses this file as if it +had been specified precisely by name. +

+

The directories searched include several standard system directories plus +any that you specify with -L. +

+
+

Note: +the second paragraph of the above extract is not strictly relevant to +the subject of this Mini‑HOWTO; I’ve chosen not to elide it, +because it answers another FAQ: “Why do I get undefined reference +errors, when the symbols in question are definitely present in the libraries +I’ve specified with -l options? ” — +the usual answer is that the -l options have been placed +too early in the command line argument sequence. +The remainder, and in particular the final paragraph, are pertinent, +vaguely indicating that libraries will be searched for in +several standard system directories; +these will be identified more explicitly in this Mini‑HOWTO. +

+
+ +
+

Consulting an Alternative Reference Source
+The GNU Binary File Utilities Manual’s Perspective

+

We have observed that the +GCC Manual +makes no definitive statement, regarding the default directories +to be searched for libraries. +Since the actual library search is performed by the linker, +ld, which is a component of the GNU Binary File Utilities, +(more commonly known as binutils), we might hope to find a +more definitive statement in the +applicable section of the +Binutils Manual. +Unfortunately, while obviously a useful reference for +the GNU Binary File Utilities in general, +this manual appears to be equally noncommittal; +the only references to be found, to library search paths, +are in the +Command Line Options section: +

+
+-Lsearchdir +
--library-path=searchdir + +

Add path searchdir to the list of paths +that ld will search for archive +libraries and ld control scripts. +You may use this option any number of times. +The directories are searched in the order in which they are specified +on the command line. +Directories specified on the command line are searched +before the default directories. +All -L options apply to all -l options, +regardless of the order in which the options appear. +

+

If searchdir begins with =, +then the = will be replaced by the sysroot prefix, +a path specified when the linker is configured. +

+

The default set of paths searched (without being specified with +‘-L’) depends +on which emulation mode ld is using, +and in some cases also on how it was configured. +See Environment. +

+

The paths can also be specified in a link script with +the SEARCH_DIR command. +Directories specified this way are searched at the point in which the linker +script appears in the command line. +

+

and in the Linker Scripts / File Commands section: +

+
+SEARCH_DIR(path) +

The SEARCH_DIR command adds path to the list of paths +where ld looks for archive libraries. +Using SEARCH_DIR(path) is exactly like using +‘-L path’ +on the command line (see Command Line Options). +If both are used, then the linker will search both paths. +Paths specified using the command line option +are searched first. +

+

while the Environment reference, appearing within the Command Line Options reference, apparently has nothing to say, +which is related to this subject. +

+
+ +
+

Determining MinGW’s Default Library Search Path

+

So, if the applicable manuals don’t give us any definitive +indication of which directories will be searched for libraries, how +can we identify where the MinGW tools will search? +

+

The ld manual tells us that the directories searched will be those specified on the command line, +using -L options, +followed by those specified in the effective linker script, +using the SEARCH_DIR script command. +We may identify the search directories specified within +the default linker script, +by running the command: +

+
+$ ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012
+
+

which, with a standard MinGW installation, +might produce output similar to: +

+
+$ ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012
+SEARCH_DIR("/mingw/mingw32/lib")
+SEARCH_DIR("/mingw/lib")
+SEARCH_DIR("/usr/local/lib")
+SEARCH_DIR("/lib")
+SEARCH_DIR("/usr/lib")
+
+

The first thing we might notice about these +is that they are all specified in the style of POSIX paths; +however, since the MinGW tools are all native MS‑Windows applications, +the linker will be unable to resolve these paths, +in any manner other than as absolute with respect to +the current drive, +at the time when the linker is invoked. +Thus, even if these directories do exist, +(which is unlikely in a standard MS‑Windows installation), +they may represent different locations, +depending on whichever drive the user has made current, +at the time when the linker is invoked. +(For users of MSYS, +they will definitely not be resolved +relative to the root of the MSYS virtual file system, +even if that is what the user might like). +

+

Now, we have seen that none of its pre‑configured search paths +seem to be particularly useful as default search paths, +for use with the MinGW linker. +That may be considered to be a configuration error, +(which a future binutils package release may address); +nevertheless, it does not appear to present a significant obstacle +to operation in practice, +since suitable search paths may always be specified +using -L options. Of course, +any such options which we are required +to specify on the command line, +can hardly be described as default search paths; +however, +if we adopt the recommended practice of always +invoking ld via an appropriate GCC front‑end, +then the GCC driver has an opportunity to supply such options, +in a manner which may be so described, and in fact, +this is precisely what happens. +

+

Thus, we have established that the linker’s +default search paths have little value, in a standard MinGW installation, +but that GCC itself furnishes the effective defaults, +by supplying appropriate -L options. +To discover what these default -L options are, +we might use the command: +

+
+$ gcc -print-search-dirs
+
+

If we run this, +we will see a sprawling mess of nigh incomprehensible output; +some judicious filtering, to extract only the library search paths, +normalize them to canonical forms, and present them one per line, +can reduce it to a form which is more readily interpreted, +(as in this example, with an old MinGW GCC‑3.4.5 system, +running under MSYS): +

+
+$ gcc -print-search-dirs | \
+> sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,;s,;,;  ,g' | \
+> tr \; \\012
+libraries:
+c:/mingw/lib/gcc/mingw32/3.4.5/
+c:/mingw/lib/gcc/
+/mingw/lib/gcc/mingw32/3.4.5/
+/usr/lib/gcc/mingw32/3.4.5/
+c:/mingw/mingw32/lib/mingw32/3.4.5/
+c:/mingw/mingw32/lib/
+/mingw/mingw32/lib/mingw32/3.4.5/
+/mingw/mingw32/lib/
+/mingw/lib/mingw32/3.4.5/
+/mingw/lib/
+c:/mingw/lib/mingw32/3.4.5/
+c:/mingw/lib/
+/mingw/lib/mingw32/3.4.5/
+/mingw/lib/
+/lib/mingw32/3.4.5/
+/lib/
+/usr/lib/mingw32/3.4.5/
+/usr/lib/
+
+

Once again, +we may observe that this list includes a number of POSIX style paths, +which most likely are not relevant for a MinGW installation. +If we further filter, to remove the POSIX paths, +and keep only the MS‑Windows style paths, +this list reduces to: +

+
+$ gcc -print-search-dirs | \
+> sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,;s,;,;  ,g' | \
+> tr \; \\012 | \
+> grep -v '^ */'
+libraries:
+c:/mingw/lib/gcc/mingw32/3.4.5/
+c:/mingw/lib/gcc/
+c:/mingw/mingw32/lib/mingw32/3.4.5/
+c:/mingw/mingw32/lib/
+c:/mingw/lib/mingw32/3.4.5/
+c:/mingw/lib/
+
+

From this, +we might deduce that GCC will pass each of these paths to the linker, +using an appropriate -L option; to verify that deduction, +we might use GCC’s -### option, +in a command such as: +

+
+gcc -### -o foo foo.c
+
+

to reveal exactly what command GCC will pass to the linker; (again, the output is not formatted for easy reading, but some judicious filtering will extract just the -L options, and present them in a form which can be more easily read by humans): +

+
+$ touch foo.c
+$ gcc -### -o foo foo.c 2>&1 | \
+> tr -s \\040 \\012 | \
+> sed '/^"-L/!d;s,,,;s,"$,/,;:1;s,/[^/.][^/]*/\.\./,/,;t 1' | \
+> sed 's,^,  ,;1h;1s,.*,libraries:,;1G'
+libraries:
+c:/mingw/lib/gcc/mingw32/3.4.5/
+c:/mingw/lib/gcc/
+c:/mingw/mingw32/lib/
+c:/mingw/lib/
+
+

Oops! What happened there? In these examples, +GCC’s ‑print‑search‑dirs +option tells us that there are six MS‑Windows paths +it will search, but only four of those are passed on +to the linker, as -L options. +Why are all six not passed on? +

+

Well, +recall that we arbitrarily chose to ignore +the POSIX style paths in GCC’s built‑in list, +when we formulated the list of MS‑Windows paths we expected +to be searched, but why would GCC itself make such an arbitrary choice? +In reality, it doesn’t. +Let’s re‑examine that full list of built‑in paths, +but this time we will filter it on an analytical basis, +selecting only those paths which represent +actual physical directories +on our host machine: +

+
+$ ( echo libraries:
+>   drive=`pwd -W | sed 's,:.*,,'`
+>   for dir in `gcc -print-search-dirs \
+>     | sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,' \
+>     | tr \; '\012' \
+>     | sed "s,^/,$drive:/,"`
+>     do test -d $dir && echo "  $dir"
+>     done
+> )
+libraries:
+c:/mingw/lib/gcc/mingw32/3.4.5/
+c:/mingw/lib/gcc/
+c:/mingw/mingw32/lib/
+c:/mingw/lib/
+
+

which, we observe, +now exactly matches the list of -L options +we see passed to the linker, +when we invoke GCC with the -### option. +Thus, we may refine our previous deduction, +to conclude that GCC will examine its built‑in +list of pre‑configured library search directories, +passing on to the linker, in the form of -L options, +those which represent actual physical directories on the host machine; +therefore, this becomes the list of default library search +directories for MinGW. +

+
+ +
+

Installation and Use of Supplementary Libraries with MinGW

+

When a project uses only the standard core libraries, +either as provided by the MS‑Windows operating system itself, +or those provided as integral components of the MinGW distribution, +the user need take no action to ensure that those libraries +will be searched at link time; +the compiler driver will take care of it automatically. +Even in cases where the project relies on optional libraries +supplied as standard operating system components, +and for which MinGW provides import libraries, +the user need do no more than specify an appropriate +-lNAME option to the compiler driver, +and the library will be linked, as required. +In either of these cases, +the necessary libraries are located within the default search path, +and no particular action is required of the user, +to specify where that might be. +

+

Conversely, +when a project supplements the standard libraries, +with one or more which it creates exclusively for its own use, +then those libraries will not be found +in the default search path, +so their location must be explicitly identified to the compiler driver, +to ensure that they can be found at link time. +In such cases, +in which the libraries are not installed for use by other projects, +the most common, and indeed the expected practice, +is for the project maintainer to specify the library locations explicitly, +by addition of appropriate -L options to the linking commands +within the project’s build system infrastructure. +When this is done correctly, +there is again no action required of the user, +to specify the location of such supplementary libraries. +

+

The situation becomes much more interesting, +when a project provides one or more supplementary libraries +for use by other projects. +In such cases, +there is an expectation that the libraries will be installed +in some central location, +whence they will become readily accessible +when client projects are linked, +without a need for any particular intervention by +the user building the client project, +to specify that location. +

+

On typical POSIX systems, +and in particular on those conforming to the GNU model, +it is common for the standard system libraries, +including those provided as standard components of the compiler suite, +to be installed in /lib or /usr/lib, +while supplementary libraries may be installed in +/usr/local/lib. +In fact, as we have already observed, +these three locations are already included, +by means of SEARCH_DIR("...") commands +in the default linker scripts, +as defaults to be searched by the MinGW linker; +unfortunately, they are specified exactly as such, +in the original POSIX format, +which cannot be readily interpreted in the file system context +in which the MinGW linker must operate! +This is a limitation of the MinGW linker, +which the user installing supplementary libraries must work around, +possibly by adopting one of the following strategies:– +

+
    +
  1. Install in a non‑default location, +and require every user of the installed libraries to explicitly +add the requisite -L options to the linking commands, +for every subsequent project which requires any supplementary library. +This is inconvenient for subsequent use, +and is generally not recommended. +
    2. +
  2. Install in a location +which is already considered to be a default by the compiler driver. +This often recommended option is the simplest possible, +for both the user installing the libraries, +and for those using them. +If using MSYS as the command line environment +for building and installing the libraries, +and the libraries themselves are delivered as GNU standard source packages, +such installation may be readily accomplished by the command sequence: +
    +$ ../path/to/configure --prefix=`cd /mingw ; pwd -W` ... && make && make install
+
    +

    (Note: +here we explicitly force the use of MS‑Windows native +path format for the prefix; this is normally necessary, +if the libraries being built hard code +the prefix within object files). +

    +

    It may be noted that, +in this example, we appear to have arbitrarily chosen the +top level lib sub‑directory, +within MinGW’s own installation directory, +as the preferred location for installation of supplementary libraries; +in reality, of the pre‑existing default search directories, +this is the only sane choice, +since the others are much more specifically associated with +the component libraries provided by the compiler suite itself, +and it is undesirable to pollute them with “foreign” libraries. +Indeed, many users also consider it undesirable to pollute the top level +MinGW library directory with such “foreign” libraries; +such users may prefer the following alternative installation strategy. +

    3. +
  3. Install in a non‑default location, +just as in the first option, but also customize the linker scripts, +or the compiler driver specs, +to implicitly add the requisite -L option, +such that this location becomes an additional default search path. +Available techniques for achieving such customization will be discussed, +in the following section. +
    4. +
+
+ +
+

Customizing MinGW’s Default Library Search Path

+

So, you’ve decided you would prefer to segregate +your add‑on libraries from the core MinGW libraries? +The first issue to address, +is where you would like to install them. +

+

Here, you have two basic options: +

    +
  1. Establish a set up, +analogous to the common /usr/local hierarchy +of typical Unix systems, with all additional libraries, +and their associated header files, collected into this one location. +
    2. +
  2. Provide a separate directory for each individual library package, +and customize the GCC search paths to consider all of them, +as appropriate, when looking for libraries and header files. +
    3. +
+

+

It should be fairly obvious that +the first of these will require less GCC customization than the second. +In fact, the principles are the same in both cases; +however, whereas the first requires the addition of only one include path, +(-I option), and one library path, (-L option), +the second will require one of each per library package installed, +which may be detrimental to GCC performance. +Thus, we will describe only the first option in detail; +users preferring the second may extend the principles described, +as necessary, to accommodate their preference. +

+

Having chosen to install all add‑on libraries in a common location, +the choice of that location is fairly arbitrary. +MSYS users may choose to simply use the conceptual /usr/local +of their existing MSYS installation, in which case, +they must ensure that that directory physically exists, +(it may not in a bare, standard MSYS installation), +and they must identify its true native Windows path, +for use in the GCC customization, as shown in the following example, +for a standard MSYS installation in its +default installation directory: +

+
+$ mkdir -p /usr/local
+$ ( cd /usr/local && pwd -W )
+c:/msys/1.0/local
+
+

Caution: +Although this choice of directory may seem attractive, +particularly because it matches the default prefix assumed by +the majority of GNU packages, +it may not represent the best choice +for all MSYS users. +This is particularly true for those users +who may wish to develop MSYS applications. +The special MSYS build of GCC +used to compile such applications reserves /usr/local +for its own purposes; +in this case, +it must not be polluted by headers and libraries +intended for use with the MinGW build of GCC. +

+

Alternatively, +masochists who prefer to persist with cmd.exe, +or MSYS users who prefer to keep MinGW related add‑ons more +closely associated with their MinGW installations, +might choose to create an installation directory +locally within their MinGW tree: +

+
+C:\> mkdir c:\mingw\local
+
+

or adjacent to it: +

+
+C:\> mkdir c:\mingw-local
+
+

Caution: +Whichever of these options you choose, +please do not choose to create any directory +with white space appearing anywhere in +its absolute path name. +If you adopt this asinine practice, +do not be surprised if it turns around and bites. +When it does, please do not come here to complain; +you have been warned! +

+

Once the tree into which supplementary libraries, +and their headers, are to be installed has been selected, +and created, the next step is to install the requisite +library packages into this tree. +In the case of standard GNU library packages, +and assuming you have chosen C:\MinGW\local as your +supplementary installation path, +(MSYS users may refer to it simply as /mingw/local), +this may be accomplished from the MSYS shell, by running: +

+
+$ ../path/to/configure --prefix=/mingw/local ... && make && make install
+
+

(or, if you suspect that the library may hard code +its installation path within itself): +

+
+$ ../path/to/configure --prefix=`cd /mingw/local ; pwd -W` ... && make && make install
+
+

Note: +If the library package is not a standard GNU package, +you will need to adapt its installation procedure to achieve the effect +of the above. +Details of how this may be achieved are package specific, +and are beyond the scope of this HOWTO. +

+

Finally, +having installed supplementary libraries and their headers +into the supplementary directory tree, +they must be made available to the GCC compilers. +Of course, +this may always be achieved by adding +appropriate -I ..., +and -L ... specifications to the commands, +when the compilers are invoked, +but that hardly seems consistent with the level of convenience +we are striving to achieve! +Obviously, the desired objective can best be met if it can be arranged +for the compilers to search in the supplementary tree by default; +this may be achieved by adding the path name for the supplementary +include file tree to the compilers’ default search paths, +as described in the +“Include Path HOWTO”, +accompanied by either of: +

+
    +
  1. Adding the path name for the supplementary library directory to +the semicolon separated list of library search directories specified +by the LIBRARY_PATH environment variable, +(which must be exported, +if it is assigned from within the MSYS shell): +
    +$ export LIBRARY_PATH
+$ LIBRARY_PATH="C:/mingw/local/lib;$LIBRARY_PATH"
+
    +

    Note: +The GCC manual states that this technique may be used +only +when GCC is configured as a native compiler; +thus, this method of specifying the supplementary library path +may not be used when your MinGW GCC is configured as a cross‑compiler. +Indeed, it would be illogical to use this technique with a cross‑compiler, +as it would lead to confusion between those libraries which are specific +to the native GCC compiler on the build platform, +and those which are specific to the cross‑compiler. +

    +
    2. +
  2. Follow the method, +as described in the +“GCC Customization HOWTO”, +to add a supplementary path to the default library search path. +

    Note: +This method is applicable equally to MinGW GCC +configured as a native compiler or configured as +a cross‑compiler; +the additional search path applies only to +the specific instance of the compiler +with which the modified specs file is associated. +

    3. +
+ +
+ + -- 2.11.0