Discussion:
[Mingw-users] libtool hard coded paths?
Roger Pack
2011-12-01 20:45:44 UTC
Permalink
Hello.
I installed mingw today, and noticed in my file:
G:\installs\mingw-raw\bin\libtool

some hard coded paths, like:

compiler_lib_search_dirs="c:/mingw/bin

Is this expected?
Thanks.
-roger-
JonY
2011-12-01 21:51:52 UTC
Permalink
Post by Roger Pack
Hello.
G:\installs\mingw-raw\bin\libtool
compiler_lib_search_dirs="c:/mingw/bin
Is this expected?
Thanks.
-roger-
Yes.
Charles Wilson
2011-12-01 22:01:57 UTC
Permalink
Post by Roger Pack
G:\installs\mingw-raw\bin\libtool
compiler_lib_search_dirs="c:/mingw/bin
Is this expected?
Yes; the installed copy of libtool always has hardcoded paths like that.
What's worse is that it ALSO has hardcoded references to the *compiler*
that was used to create it -- in this case, gcc-4.4.x I think.

This usually isn't an issue, because typical libtool usage is to include
it in your own package's configure/build system, so that a "new" one
with appropriate paths is generated when you configure/build your package.

The only package I know of that attempts to use the installed "libtool"
from $PATH is ncurses...which is not ideal, but is at least better than
hand-rolling the commands to build sharedlibs on 27 different platforms.

--
Chuck
Ralph Engels
2011-12-02 06:49:41 UTC
Permalink
+ subversion and apache which use there own rollout of libtool in case
jlibtool fails to build on the specified platform.

But yes better than having a gazillion link commands in the makefiles
even if libtool sometimes makes me want to crawl up the walls :).

Ralph
Earnie
2011-12-02 13:25:44 UTC
Permalink
This post might be inappropriate. Click to display it.
Ralph Engels
2011-12-02 13:44:05 UTC
Permalink
My bad then i just assumed after reading several sources that libtool
made it easier handling, if i could get rid of it easily i wouldnt
complain though :)

i experimented a bit with jlibtool from apache and made it an
executable but as someone pointed out most sources come with there own
version of libtool and dont use the system one for anything other than
picking up special m4 defs if you libtoolize the source and in that
case jlibtool does not get picked up as i dont have any m4 defs to
tell the source i want to use that.

Ralph
Earnie
2011-12-02 14:02:25 UTC
Permalink
Post by Ralph Engels
My bad then i just assumed after reading several sources that
libtool made it easier handling, if i could get rid of it easily i
wouldnt complain though :)
i experimented a bit with jlibtool from apache and made it an
executable but as someone pointed out most sources come with there
own version of libtool and dont use the system one for anything
other than picking up special m4 defs if you libtoolize the source
and in that case jlibtool does not get picked up as i dont have any
m4 defs to tell the source i want to use that.
Package distribution should not assume the user has libtoolize and the
configure script builds the libtool script in the build directory from
the distributed files which are given by libtoolize itself. It isn't
a choice of the casual user to make. The package maintainer
distributes the version that works for his package by using
libtoolize. I assume the j in your jlibtool is for "java" but why
would GNU libtool want to circumvent the use of what it delivers and
allow the user to pick something else that most likely won't work with
the package?
--
Earnie
-- https://sites.google.com/site/earnieboyd/
Ralph Engels
2011-12-02 14:14:14 UTC
Permalink
The j in the libtool just carried over when building it but no its not
java its standard C. apache as far as i can understand use it because
its heaps faster than the script version and "allmost" as portable as
the script version.
allmost being a few systems failing to build it in that case it
defaults to the script version.
Ralph Engels
2011-12-02 14:02:00 UTC
Permalink
Sometimes the deeper meaning escapes me (enlish not my native language
etc.) by a gazillion link commands i ment stuff like libtool normally
handles like hiding ld <very long complicated link command> or gcc or
what ever the tag uses as link command.
Sorry about that.

Ralph
Charles Wilson
2011-12-02 18:06:34 UTC
Permalink
Post by Earnie
Post by Ralph Engels
But yes better than having a gazillion link commands in the makefiles
even if libtool sometimes makes me want to crawl up the walls :).
Huh!? This makes no sense to me. A properly coded Makefile.in doesn't
require a gazillion link commands. It isn't the purpose of libtool to
remove the responsibility of building a proper Makefile. The biggest
purpose of libtool is to help control which libraries are needed for the
build and where to actually find the library.
I disagree. That's ONE purpose. The other purpose is to replace this:

ifeq ($(findstring CYGWIN,$(uname)),CYGWIN)
PLATFORM:=cygwin
else
ifeq ($(findstring MINGW,$(uname)),MINGW)
PLATFORM:=mingw
else
PLATFORM:=linux
endif
endif

ifeq ($(PLATFORM), cygwin)
SHRTARGET=cygfoo-2.dll
$(SHRTARGET): $(BASE_OBJS)
$(CC) -shared -o $@ $< $(LDFLAGS) $(LIBS) \
-Wl,--enable-auto-image-base \
-Wl,--out-implib=libfoo.dll.a
else
ifeq ($(PLATFORM), mingw)
SHRTARGET=libfoo-2.dll
$(SHRTARGET): $(BASE_OBJS) getopt_long.o getopt.o mkdtemp.o
$(CC) -shared -o $@ $< $(LDFLAGS) $(LIBS) -lws2_32 \
-Wl,--enable-auto-image-base \
-Wl,--out-implib=libfoo.dll.a
else
SHRTARGET=libfoo.so.2.0.0
$(SHRTARGET): $(BASE_OBJS)
$(CC) -shared -o $@ $< $(LDFLAGS) $(LIB) -lnet
ln -s $(SHRTARGET) libfoo.so.2
ln -s $(SHRTARGET) libfoo.so
endif
endif


SHRLIB = $(LIBTARGET)

all: $(SHRLIB)


In your libtool-less Makefile, with this, in your Makefile.am:

libfoo_la: $(LIB_OBJS)

(Note that you also need ugly stuff like the above for your install
rule, etc. -- and it just gets uglier and uglier as you add support for
HP/UX vs. Solaris vs. OSX vs. Irix vs...

--
Chuck
Earnie
2011-12-02 12:40:20 UTC
Permalink
Post by Roger Pack
Hello.
G:\installs\mingw-raw\bin\libtool
compiler_lib_search_dirs="c:/mingw/bin
Is this expected?
Another reason to dislike libtool. Surely this should become
compiler_lib_default_search_dirs to allow other directories to be
specified. Maybe it does already, don't know that much about libtool.
What I would expect for MinGW is that this becomes `cd /mingw && pwd -W`
to make it directory portable. Of course that assumes the user has
/mingw mapped in /etc/fstab which hopefully once mingw-get is able to do
post-install scripts can happen automagically.

One work-around is to get the junction.exe tool from sysinternals and
issue ``junction c:\mingw g:\installs\mingw-raw''. The other, get the
libtool source and build the package yourself. Make sure to use a
--prefix=`cd /mingw && pwd -W` during the configure process.
--
Earnie
-- https://sites.google.com/site/earnieboyd/
Charles Wilson
2011-12-02 18:25:36 UTC
Permalink
Post by Earnie
Post by Roger Pack
Hello.
G:\installs\mingw-raw\bin\libtool
compiler_lib_search_dirs="c:/mingw/bin
Is this expected?
Another reason to dislike libtool.
Everybody hates libtool, until they get involved in making it better.
Then two things happen: (1) they hate it more (2) they realize that all
that ugliness is there for a reason.

It really is necessary. Sad, but true.

One of my long-term goals for libtool on cygwin|msys|mingw is to teach
libtool to simply use the language driver -- $(CC), $(CXX), $(F77), etc
-- to link, rather than it's current (more universal) approach which is to:

1) detect at libtool-creation-time what "stuff" the language driver adds
to the link command -- pre-deps, post-deps, additional flags, extra
libs, runtime objects, etc -- and "save" those.

2) Use the linker ($LD) directly to link -- adding back in all the
"extra stuff" that was discovered in step one.

BTW -- it's step #1 that leads to the hardcoded compiler (version)
dependent paths getting embedded in the "built" libtool.

All that extra work is necessary for a number of arcane reasons on
various platforms, so the same procedure is used on ALL platforms. But,
if the language driver is from the GNU Compiler Collection, and the
linker is the GNU linker, then theoretically you can skip all of that --
and just use the language driver as $(LD) and not worry about
pre/post-deps, libs, extra objs, etc.

It's a big job to convince libtool to work this way, and I haven't had
the tuits to tackle it yet. Maybe after $git-port. It would benefit
*all* GCC+GnuLD platforms, not just cygming.
Post by Earnie
Surely this should become
compiler_lib_default_search_dirs to allow other directories to be
specified. Maybe it does already, don't know that much about libtool.
What I would expect for MinGW is that this becomes `cd /mingw&& pwd -W`
to make it directory portable.
It really can't do the latter, for two reasons: (1) multiple passes of
evaluation. At some point it's GOING to turn into what it represents, on
the builder's machine, and not the shell exec command. (2) This breaks
libtool's ability to cross-compile.
Post by Earnie
One work-around is to get the junction.exe tool from sysinternals and
issue ``junction c:\mingw g:\installs\mingw-raw''. The other, get the
libtool source and build the package yourself. Make sure to use a
--prefix=`cd /mingw&& pwd -W` during the configure process.
NO!

The installed (mingw32) libtool itself is built using --prefix=/mingw;
just like the other (mingw32) autotools, it is perl/m4/shell-based and
thus lives in a hybrid world. mingw32-libtool itself is an *msys* tool,
because it uses /bin/sh -- but is USED to create mingw32 "stuff" (unlike
msys-libtool, which is also msys-based but is used to create msys
"stuff" and thus isn't "hybrid" at all).

Now, if you have a $foo source package that has been libtoolized, then
during $foo's configure it will create a local libtool script. Since
you probably configured $foo with --prefix=`cd /mingw && pwd -W`, then
THAT local libtool script will ALSO have internal references to
$(whatever `cd /mingw && pwd -W` evaluates to locally). That's fine,
and it works -- because you're using it locally, and if you've used the
magic prefix expression above then obviously you're building with
$build=mingw for $host=mingw -- that is, native mingw.

--
Chuck
Ralph Engels
2011-12-02 18:43:14 UTC
Permalink
Not to mention other pitfalls especially if used with mingw
and you where so unlucky as to have the file magic executable on path.
Then you run into stuff like cannot link to static library -lws2_32 Oo
us poor windows users know for a fact that ws2_32 is a dll but file
magic seems to be unable to comprehend libraries which dont have a
dll.a extention as dynamic libraries no matter whats in the library.
Atleast over the years i found ways to overcome most of these nasties,
and today i use a config.site in mingw/etc to tell libtool whats right
even if it thinks im wrong. Another way is doing say echo
"lt_cv_deplibs_check_method='pass_all'" >>config.cache and then
configure with --cache-file=config.cache this fixes cannot link to
static library error above. But yes as wilson said it requires some
knowladge of the m4 macros to work around other pitfalls this way.

Ralph
Charles Wilson
2011-12-02 22:09:13 UTC
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Not to mention other pitfalls especially if used with mingw
and you where so unlucky as to have the file magic executable on path.
Then you run into stuff like cannot link to static library -lws2_32 Oo
us poor windows users know for a fact that ws2_32 is a dll but file
magic seems to be unable to comprehend libraries which dont have a
dll.a extention as dynamic libraries no matter whats in the library.
Something else is wrong with your installation, then -- OR you are
building a package which has a VERY old version of libtool.m4/ltmain.sh
and you aren't re-libtoolizing.

For MANY years, the cygwin and mingw "code path" in libtool.m4/ltmain.sh
-- and thus, in libtool when created using modern versions of those
files -- jumps thru a TON of hoops to work around this problem. I even
submitted a patch (which was accepted) to binutils to add capabilities
to dlltool.exe to help with this. Take a look at libtool-2.4's
func_win32_libid() implementation...
Atleast over the years i found ways to overcome most of these nasties,
and today i use a config.site in mingw/etc to tell libtool whats right
even if it thinks im wrong. Another way is doing say echo
"lt_cv_deplibs_check_method='pass_all'">>config.cache and then
This is a different issue. If you try to build a dll, and you try link
against a static library (and I mean a REAL static library, not simply
an import lib with a ".a" name rather than .dll.a) then yes, libtool
will refuse. *As It Should*.

There are TONS of problems that can happen if you attempt to build a dll
that links against a regular static library, unless that "static
library" is actually just a convenient "holder" for a bunch of objects
INTENDED to be incorporated into a DLL. If your
lt_cv_deplibs_check_methods override hasn't caused very subtle bugs in
your code, leading to a lot of hard-to-track-down erroroneous behaviors,
then I'd be very suprised -- and you are very lucky.

And THAT's why *modern* libtool uses LOTS of different techniques to
determine whether a file is an import lib or a true static library --
SPECIFICALLY so that folks like you won't be tempted to override
lt_cv_deplibs_check_methods when "libtool refuses to create a dll when I
link against so-called 'static' library libkernel.a that is really an
import library."

FYI:

$ func_win32_libid /mingw/lib/libws2_32.a
x86 archive import

See attached; all functions taken from /mingw/bin/libtool (2.4-1).

--
Chuck
Ralph Engels
2011-12-03 07:49:58 UTC
Permalink
I use libtool-2.4.1 :) but it still fails if file magic is in $PATH im
not sure how the file magic tool reads dynamic libraries but i found
that if i copy the dll it complains about to mingw/bin and rename it
to say libws2_32.dll then it works. Maybe a bug with the win32 file
magic tool i use for some projects ? (its rather old :/).

I allways try without pass_all in lt_cv_deplibs_check_method first but
i newer had any adverse effects (besides my own stupidity) from using it.

Strangely enough tor lilquist uses it all over the place in opensuses
mingw32 crosscompiler so i guessed the impact was atleast minimal.

the contents of the config.site i use ->

lt_cv_cc_dll_switch=${lt_cv_cc_dll_switch=-mdll}
lt_cv_compiler_c_o=${lt_cv_compiler_c_o=yes}
lt_cv_compiler_o_lo=${lt_cv_compiler_o_lo=yes}
lt_cv_deplibs_check_method=${lt_cv_deplibs_check_method='pass_all'}
lt_cv_file_magic_cmd=${lt_cv_file_magic_cmd='$OBJDUMP -f'}
lt_cv_file_magic_test_file=${lt_cv_file_magic_test_file=}
test "${lt_cv_global_symbol_to_c_name_address+set}" = set ||
lt_cv_global_symbol_to_c_name_address='sed -n -e '\''s/^: \([^ ]*\) $/
{\"\1\", (lt_ptr) 0},/p'\'' -e '\''s/^[BCDEGRST] \([^ ]*\) \([^
]*\)$/ {"\2", (lt_ptr) \&\2},/p'\'''
lt_cv_global_symbol_to_cdecl=${lt_cv_global_symbol_to_cdecl='sed -n -e
'\''s/^. .* \(.*\)$/extern char \1;/p'\'''}
lt_cv_ld_reload_flag=${lt_cv_ld_reload_flag=-r}
lt_cv_need_dllmain=${lt_cv_need_dllmain=no}
lt_cv_prog_cc_can_build_shared=${lt_cv_prog_cc_can_build_shared=yes}
lt_cv_prog_cc_no_builtin=${lt_cv_prog_cc_no_builtin=}
lt_cv_prog_cc_pic=${lt_cv_prog_cc_pic=' -DDLL_EXPORT'}
lt_cv_prog_cc_pic_works=${lt_cv_prog_cc_pic_works=yes}
lt_cv_prog_cc_shlib=${lt_cv_prog_cc_shlib=}
lt_cv_prog_cc_static=${lt_cv_prog_cc_static=-static}
lt_cv_prog_cc_static_works=${lt_cv_prog_cc_static_works=yes}
lt_cv_prog_cc_wl=${lt_cv_prog_cc_wl=-Wl,}
lt_cv_prog_gnu_ld=${lt_cv_prog_gnu_ld=yes}
lt_cv_sys_global_symbol_pipe=${lt_cv_sys_global_symbol_pipe='sed -n -e
'\''s/^.*[ ]\([ABCDGIRSTW][ABCDGIRSTW]*\)[ ][
]*\([_A-Za-z][_A-Za-z0-9]*\)$/\1 \2 \2/p'\'''}
lt_cv_sys_path_separator=${lt_cv_sys_path_separator=:}
Earnie
2011-12-03 14:37:19 UTC
Permalink
Maybe a bug with the win32 file magic tool i use for some projects
? (its rather old :/).
Use the MSYS provided tool. Otherwise we cannot help.
--
Earnie
-- https://sites.google.com/site/earnieboyd/
Earnie
2011-12-03 15:23:56 UTC
Permalink
Post by Charles Wilson
Post by Earnie
Post by Roger Pack
Hello.
G:\installs\mingw-raw\bin\libtool
compiler_lib_search_dirs="c:/mingw/bin
Is this expected?
Another reason to dislike libtool.
Everybody hates libtool, until they get involved in making it better.
Then two things happen: (1) they hate it more (2) they realize that all
that ugliness is there for a reason.
It does a fantastic job, it's a love/hate relationship.
Post by Charles Wilson
It really is necessary. Sad, but true.
Acknowledged and I agree.
Post by Charles Wilson
One of my long-term goals for libtool on cygwin|msys|mingw is to teach
libtool to simply use the language driver -- $(CC), $(CXX), $(F77), etc
1) detect at libtool-creation-time what "stuff" the language driver adds
to the link command -- pre-deps, post-deps, additional flags, extra
libs, runtime objects, etc -- and "save" those.
2) Use the linker ($LD) directly to link -- adding back in all the
"extra stuff" that was discovered in step one.
I've begun to see a combination of --tag=CC --mode=link in packages for
the link step which does already what you want to teach it, does it not?
Post by Charles Wilson
BTW -- it's step #1 that leads to the hardcoded compiler (version)
dependent paths getting embedded in the "built" libtool.
Yes, I understand that. Perhaps instead of a learn as it is configured
libtool should deliver a database of known options for each version and
only if the configured step discovers a version not in the database
would it need to add hard coded versions.
Post by Charles Wilson
All that extra work is necessary for a number of arcane reasons on
various platforms, so the same procedure is used on ALL platforms. But,
if the language driver is from the GNU Compiler Collection, and the
linker is the GNU linker, then theoretically you can skip all of that --
and just use the language driver as $(LD) and not worry about
pre/post-deps, libs, extra objs, etc.
Acknowledged and agree.
Post by Charles Wilson
It's a big job to convince libtool to work this way, and I haven't had
the tuits to tackle it yet. Maybe after $git-port. It would benefit
*all* GCC+GnuLD platforms, not just cygming.
I really appreciate the effort you put into Open Source.
Post by Charles Wilson
Post by Earnie
Surely this should become
compiler_lib_default_search_dirs to allow other directories to be
specified. Maybe it does already, don't know that much about libtool.
What I would expect for MinGW is that this becomes `cd /mingw&& pwd -W`
to make it directory portable.
It really can't do the latter, for two reasons: (1) multiple passes of
evaluation. At some point it's GOING to turn into what it represents, on
the builder's machine, and not the shell exec command. (2) This breaks
libtool's ability to cross-compile.
I need to think about this longer.
Post by Charles Wilson
Post by Earnie
One work-around is to get the junction.exe tool from sysinternals and
issue ``junction c:\mingw g:\installs\mingw-raw''. The other, get the
libtool source and build the package yourself. Make sure to use a
--prefix=`cd /mingw&& pwd -W` during the configure process.
NO!
The installed (mingw32) libtool itself is built using --prefix=/mingw;
just like the other (mingw32) autotools, it is perl/m4/shell-based and
thus lives in a hybrid world. mingw32-libtool itself is an *msys* tool,
because it uses /bin/sh -- but is USED to create mingw32 "stuff" (unlike
msys-libtool, which is also msys-based but is used to create msys
"stuff" and thus isn't "hybrid" at all).
Oh, yea, I forgot. I even tried the --prefix=c:/mingw once and it just
breaks.
Post by Charles Wilson
Now, if you have a $foo source package that has been libtoolized, then
during $foo's configure it will create a local libtool script. Since
you probably configured $foo with --prefix=`cd /mingw && pwd -W`, then
THAT local libtool script will ALSO have internal references to
$(whatever `cd /mingw && pwd -W` evaluates to locally). That's fine,
and it works -- because you're using it locally, and if you've used the
magic prefix expression above then obviously you're building with
$build=mingw for $host=mingw -- that is, native mingw.
Ack.
--
Earnie
-- https://sites.google.com/site/earnieboyd/
Charles Wilson
2011-12-03 17:12:53 UTC
Permalink
Post by Earnie
Post by Charles Wilson
2) Use the linker ($LD) directly to link -- adding back in all the
"extra stuff" that was discovered in step one.
I've begun to see a combination of --tag=CC --mode=link in packages for
the link step which does already what you want to teach it, does it not?
No, it doesn't. The --tag=CC just informs libtool which set of
prediscovered add-ons it needs to include when invoking $LD.
Post by Earnie
Post by Charles Wilson
BTW -- it's step #1 that leads to the hardcoded compiler (version)
dependent paths getting embedded in the "built" libtool.
Yes, I understand that. Perhaps instead of a learn as it is configured
libtool should deliver a database of known options for each version and
only if the configured step discovers a version not in the database
would it need to add hard coded versions.
The problem is, that the "discovered options" vary depending on the
LDFLAGS in uses when libtool was configured. (e.g. -fmud-flap, or
-fopen-mp -- or even --static-libgcc / --static-libstdc++). And it
varies sometimes with even .micro changes in the version of GCC.

It's also why people sometimes run into problems if they configure
without setting those LDFLAGS on the configure command line, and then
try to set LDFLAGS when invoking make. This is especially prevalent
with --static-libgcc, I've noticed.
Post by Earnie
Post by Charles Wilson
It really can't do the latter, for two reasons: (1) multiple passes of
evaluation. At some point it's GOING to turn into what it represents, on
the builder's machine, and not the shell exec command. (2) This breaks
libtool's ability to cross-compile.
I need to think about this longer.
I'm starting to say that myself a lot, too. Either I'm getting old, or
all the easy problems have been solved...
Post by Earnie
Post by Charles Wilson
The installed (mingw32) libtool itself is built using --prefix=/mingw;
just like the other (mingw32) autotools, it is perl/m4/shell-based and
thus lives in a hybrid world. mingw32-libtool itself is an *msys* tool,
because it uses /bin/sh -- but is USED to create mingw32 "stuff" (unlike
msys-libtool, which is also msys-based but is used to create msys
"stuff" and thus isn't "hybrid" at all).
Oh, yea, I forgot. I even tried the --prefix=c:/mingw once and it just
breaks.
:-(

--
Chuck

Loading...