Hi there, I recently started to port my engine from Windows to Linux using Eclipse CDT. However, I have a problem linking/finding the shared objects at runtime. I know I could install the *.so files to /usr/lib but to do that I need root rights and that's something I'd like to avoid. In CDT I can enter post-build commands like "install <path_to_so_file> /usr/lib" but as stated I need root rights, so the call fails. What I'd like to have is: - shared object files in the directory of the executable or a subdirectory (e.g. "lib") - build time linking (avoid dlopen() and the like for core libraries) For a first test I have the following setup: workspace/FileSystem/Debug/libFileSystem.so workspace/Test/Debug/Test (binary) That results in libFileSystem.so no being found at runtime. The setup I'd prefer (binary should be able to find the shared object in relative path "./lib/": workspace/Test/Debug/lib/libFileSystem.so workspace/Test/Debug/Test (binary) Can I do this? How? Thanks in advance. Thomas

Not that I'd encourage either solution, but you should probably look into setting either LD_LIBRARY_PATH (set inside a script pre-execution, say) or an RPATH.

Thanks, I will look into this.

Why solution would you suggest?

Installing into /usr/lib or similar is unacceptable for obvious reasons.

So we use an rpath, and it works very well.

Quote:
From the wiki article:
The primary disadvantage of using RPATH is that it overrides the LD_LIBRARY_PATH settings which makes things like running a precompiled binary out of a user's home directory or some other non-default location difficult or impossible. Use of RPATH also makes it difficult, if not impossible, to upgrade libraries without forcing a reinstallation of all the software dependent on (even the older versions of) the libraries (see RPATH considered harmful).

I'd like to be independent from the installation directory, so an rpath doesn't seem to be that good a solution.

With LD_LIBRARY_PATH it seems to be possible to set the path at startup and restore the previous path at tear down.
However, I think simultaneously running applications could cause problems.

What do you think?

Quote: Original post by Lord_Evil
I'd like to be independent from the installation directory, so an rpath doesn't seem to be that good a solution.

The wiki article is incorrect. If you supply '$ORIGIN' as an rpath (through the -R option when linking, ie. -R'$ORIGIN'), it will search the directory where the executable resides in for libraries. This is done at runtime, so it doesn't matter where the user installs your software.

Quote:
Use of RPATH also makes it difficult, if not impossible, to upgrade libraries without forcing a reinstallation of all the software dependent on (even the older versions of) the libraries (see RPATH considered harmful).

This is an advantage, and fully intended behaviour. You wouldn't want anybody except yourself to modify and/or replace the libraries of your engine.

I do it like so:

1) Build application and libraries with no mutual knowledge of the other existing outside of shared header files.
2) Manually place libraries into an easily locatable directory (i.e. data/lib/linux-amd64)
3) At runtime:
a) Get the current directory (i.e. /home/niteice/oe)
b) Concatenate the expected directory (i.e. data/lib/linux-amd64) to the current directory (you now have /home/niteice/oe/data/lib/linux-amd64)
c) Concatenate the library you want (i.e. renderer.so) to the path (you now have /home/niteice/oe/data/lib/linux-amd64/renderer.so)
4) Open the library and get pointers to the functions you want

Quite easy to implement and platform-transparent to boot.

Thank you for your advice guys, I'll look into that.

Yann, your suggestion seems to be what I want. Would the following be possible: -R'$ORIGIN/lib' for searching the libraries in the lib directory?

Niteice, that's what I'd do for plugins but not for the core functionality of my engine. I don't want to get the pointers to functions and class members one by one for use within my engine.

Basically I have mulitple libraries that contain the several subsystems of the engine, like render system, sound system, geometry 'builders' (e.g. a terrain library), file system, physics system etc. Each provides a defined interface and thus might be replaced, e.g. the physics system using either PhysX, ODE or another library. The application will "plug" the different subsystems into the engine as needed (e.g. if you don't need sound, you don't link the sound library into your application).

Additionally I'd like to provide real plugin capabilities, e.g. for adding file codecs, non-scripted ai functionality etc.

So I think I'll use rpaths for the main systems and really dynamic linking (dopen(), ...) for the plugin framework.

niteice's suggestion is the way to go for entirely dynamic linking, ie. plugins, as you mentioned. However, this doesn't work very well with C++ class linking, and it doesn't work with implicit linking.

Quote:
Would the following be possible: -R'$ORIGIN/lib' for searching the libraries in the lib directory?

It should, but I don't know if it does in practice. I'm not really sure who is responsible for parsing the $ORIGIN macro. I would guess the implicit runtime linker, but I'm not sure. I'm also not certain how sophisticated that parsing is.

The problem with rpath is that its implementation is very hackish. The whole concept is very alien to the common Linux open source folks, because it promotes proprietary libraries that are locked to a certain application. For us, this is exactly what we're looking for. Bur for most FOSS people, this is against their ideology. That's why this rpath mechanism is often considered evil (don't worry, it's not) and doesn't get the attention it should from the maintainers of the linking subsystem.

Now, you could simply supply the current directory in the rpath: '.' or './libs'. However, this will add the directory the user is currently in when he launches your app, and not the directory where the app is located. It works as long as the user always launches your app from its own directory (eg. "./mygame") but not if he is somewhere else (eg. "./coolgame/mygame"). It will also introduce security issues, because it can allow an attacker to perform a type of DLL-injection on system libraries, possibly using your software to gain root privileges. Not good. As I said, the whole thing wasn't very well thought out.

The $ORIGIN feature was hacked in to address this issue, against the will of a number of FOSS fanatics. It works, but it's considered a second class citizen.

Oh, and if you are porting a C++ engine from Windows to Linux and intend to support C++ plugins, then you might also want to look into fvisibility. This feature is extremely important. Again, it's something that's not widely publicized in the Linux community (due to, again, resistance from some fanatics), but developing/porting a good plugin system without it is next to impossible. I almost went insane back when I tried it, until I finally found this feature...

Thanks for the explanation Yann. I'll have a look at that.