SDL3 SDL_LoadObject and dlopen

All,

Before I end up wasting more fruitless weeks going down another path to oblivion I would like to ask about the interaction (on linux) between SDL_LoadObject() and dlopen(). For a bit of a primer you can look at this file, scrolling down to line 256. You will see a lot of commented out code trying lt_dlopen() which was supposed to work, but even the maintainers on the mailing list for it could not figure out how to make it work.

Task
Build a complex library into a local directory tree without having to hack RPATH relentlessly (not allowed on secure platforms) and build demo programs against it that need to use both the libraries and the plugins. Oh Yes! The plugins need the libraries as well.

Situation

Qt has historically had a qt.conf file that never worked. The reason it never worked was the fact dlopen() reads the environment as part of the startup code before ever getting to your code. This means, even if you force using the entry in qt.conf (or any other place) to find the plugins you are still hosed because dlopen() will look for the supporting libraries in only the system defined areas. You either hack LD_LIBRARY_PATH environment variable prior to running or you hack the system search path adding a .conf at a place ldconfig will find it telling the system search to also look at your local install directories.

Not cool.

Last I looked the official Qt method was deploy_qt (or something like that) which dumps all of the libraries and plugins in with the executable so dlopen() can find them.

Very not cool.

Bit of Background

LsCs is a port of CopperSpice which was a re-development of Qt 4.x (last actual Open-Source version of Qt). I’m finishing up adding Cups 3.x API support as well as distro packaging before migrating this to LsX which will be a re-development using SDL3 as the core IO. That will get rid of thousands of lines of decades old.

As such, I am debating about eliminating plugins.

What I need to know is: Does SDL_LoadObject() perpetuate the bugs of the past? When you load a plugin from the plugin directory that needs libraries found in a non-standard/non-system directory tree, does it also fail?

lt_dlopen() had a method of injecting your own search paths at runtime. There was supposed to be a flag/option for it to itself, and thus the new search paths, for all dependencies. The maintainers don’t remember how to make that work.

A quick look at the SDL_LoadObject documentation makes it obvious I have to find the plugin on my own. Not a problem.

What about the supporting libraries?

How is SDL_LoadObject() designed to handle them?

I did not scour the documentation, but I did not see any method of

  • setting a search path for supporting libraries the plugin depends on
  • getting a list of supporting libraries the plugin needs prior to some form of broken load happening
  • can it load a C++ library? The function call won’t work due to name mangling, but the loading of the library should still work, correct?

I would really like to not perpetuate the long history of busted application frameworks that cannot be developed and tested via local non-system (non-system includes /usr/local tree).

Thanks,

SDL_LoadObject() doesn’t do anything special, it’s just a simple wrapper around dlopen(), so any issues you have with that you’ll also have with SDL_LoadObject().

1 Like

Edit: This is what I would say to someone just getting started with plugins.
Sorry it does not fit your actual question.

Here’s a small tutorial on dlopen. SDL’s version is very similar. The rest of this post assumes you read the tutorial. I prefer using SDL’s dll/so API as it adds cross-platform compatibility to the mix.

I haven’t had any issues with plugins failing upon linking to other libraries, so I can’t answer that part, but here’s my experience so far:

C++ now has built-in filesystem search API, this makes it much more simple to search a folder for all plugins.

Personally, I like to β€œinstall” plugins using SDL’s drag and drop API to get the path, you have the option to copy/move the file to a plugin folder that makes sense to the project.
Once the plugin is loaded and confirmed, save that path to a pluginList.txt file to be β€œdetected” and loaded the next time the program starts.
Both dlopen and SDL_LoadObject fail by returning NULL on failure, if this occurs then you have the option to remove it from the list of maintained plugins, or otherwise ignore it.

Plugins pose a security risk. It is good practice to create a menu that allows the user to β€œenable” and β€œdisable” plugins, and default to disabling newly added plugins until the user decides it is time to enable them. β†’ This allows the user a chance to keep a copy of a plugin in the plugin folder and time to research it without giving the plugin immediate full access to the program.

One of the key issues is how do you know which functions are contained within the plugin file?
I use a C++ base class with a strong API; init, handleInput, update, draw, setPos, move, etc. Outline the functions that you believe a plugin is likely to need at each stage of the program’s life and call them in those stages. If the plugin does not provide the class’s method, then the base class should basically do nothing and return from said function.
Anyhow, the idea is that you provide a strong API through which your own program expects to interact with all plugins.
If the plugins already exist and you are trying to build a new program to use them, then you instead need to know and understand what API they were built for.

@Sward There was no need to delete your post. It’s true you were looking at the wrong end of the heifer, but that was my fault. I made a comment based on the documentation. You should restore your post because someone else is going to have that exact problem and not know how to ask the question.

At the root of this problem is the fact dlopen is a massive security risk.

lt_dlopen() which is part of GNU LibTool was meant to fix this. Problem is, they poorly documented it and even the maintainers on the mailing list cannot tell me how to make it work. So, after I finish adding Cups 3.x API support to LsCs and convert the project to XMake, I will once again dive into libtool. Actually going to have to read through all the source this time.

Consider a local build and install of an application framework.

$ tree
.
└── LsCs
β”œβ”€β”€ cmake
β”‚   β”œβ”€β”€ LsCsBinaryTargets.cmake
β”‚   β”œβ”€β”€ LsCsBinaryTargets-debug.cmake
β”‚   β”œβ”€β”€ LsCsConfig.cmake
β”‚   β”œβ”€β”€ LsCsConfigVersion.cmake
β”‚   β”œβ”€β”€ LsCsLibraryTargets.cmake
β”‚   β”œβ”€β”€ LsCsLibraryTargets-debug.cmake
β”‚   └── LsCsMacros.cmake
β”œβ”€β”€ libLsCsCore.so β†’ libLsCsCore.so.0
β”œβ”€β”€ libLsCsCore.so.0 β†’ libLsCsCore.so.0.3.2
β”œβ”€β”€ libLsCsCore.so.0.3.2
β”œβ”€β”€ libLsCsGui.so β†’ libLsCsGui.so.0
β”œβ”€β”€ libLsCsGui.so.0 β†’ libLsCsGui.so.0.3.2
β”œβ”€β”€ libLsCsGui.so.0.3.2
β”œβ”€β”€ libLsCsMultimedia.so β†’ libLsCsMultimedia.so.0
β”œβ”€β”€ libLsCsMultimedia.so.0 β†’ libLsCsMultimedia.so.0.3.2
β”œβ”€β”€ libLsCsMultimedia.so.0.3.2
β”œβ”€β”€ libLsCsNetwork.so β†’ libLsCsNetwork.so.0
β”œβ”€β”€ libLsCsNetwork.so.0 β†’ libLsCsNetwork.so.0.3.2
β”œβ”€β”€ libLsCsNetwork.so.0.3.2
β”œβ”€β”€ libLsCsOpenGL.so β†’ libLsCsOpenGL.so.0
β”œβ”€β”€ libLsCsOpenGL.so.0 β†’ libLsCsOpenGL.so.0.3.2
β”œβ”€β”€ libLsCsOpenGL.so.0.3.2
β”œβ”€β”€ libLsCsSql.so β†’ libLsCsSql.so.0
β”œβ”€β”€ libLsCsSql.so.0 β†’ libLsCsSql.so.0.3.2
β”œβ”€β”€ libLsCsSql.so.0.3.2
β”œβ”€β”€ libLsCsSvg.so β†’ libLsCsSvg.so.0
β”œβ”€β”€ libLsCsSvg.so.0 β†’ libLsCsSvg.so.0.3.2
β”œβ”€β”€ libLsCsSvg.so.0.3.2
β”œβ”€β”€ libLsCsXcbSupport.so β†’ libLsCsXcbSupport.so.0
β”œβ”€β”€ libLsCsXcbSupport.so.0 β†’ libLsCsXcbSupport.so.0.3.2
β”œβ”€β”€ libLsCsXcbSupport.so.0.3.2
β”œβ”€β”€ libLsCsXmlPatterns.so β†’ libLsCsXmlPatterns.so.0
β”œβ”€β”€ libLsCsXmlPatterns.so.0 β†’ libLsCsXmlPatterns.so.0.3.2
β”œβ”€β”€ libLsCsXmlPatterns.so.0.3.2
β”œβ”€β”€ libLsCsXml.so β†’ libLsCsXml.so.0
β”œβ”€β”€ libLsCsXml.so.0 β†’ libLsCsXml.so.0.3.2
β”œβ”€β”€ libLsCsXml.so.0.3.2
└── plugins
β”œβ”€β”€ imageformats
β”‚   └── LsCsImageFormatsSvg.so
β”œβ”€β”€ mediaservices
β”‚   β”œβ”€β”€ LsCsMultimedia_gst_audiodecoder.so
β”‚   β”œβ”€β”€ LsCsMultimedia_gst_camerabin.so
β”‚   └── LsCsMultimedia_gst_mediaplayer.so
β”œβ”€β”€ platforms
β”‚   └── LsCsGuiXcb.so
β”œβ”€β”€ playlistformats
β”‚   └── LsCsMultimedia_m3u.so
β”œβ”€β”€ sqldrivers
β”‚   β”œβ”€β”€ LsCsSqlMySql.so
β”‚   β”œβ”€β”€ LsCsSqlOdbc.so
β”‚   └── LsCsSqlPsql.so
└── xcbglintegrations
└── LsCsGuiXcb_Glx.so

10 directories, 47 files
roland@mxz2g4:~/cups-stuff/LsCs_local_release/lib

We will center this conversation around the fact some example program has begun starting up. It uses this framework so it has to load libLsCsCore and since the core has built-in support for SQLite3 to save/store settings it also loads libLsCsSql.

Eventually SDL3 will replace all of the image and platform, Xcb stuff which is how I got onto this research.

We will focus on PostgreSQL support for this conversation so we don’t veer off into the weeds with the other things.

$ ldd sqldrivers/LsCsSqlPsql.so
linux-vdso.so.1 (0x00007ffc449fc000)
libLsCsSql.so.0 => /home/roland/cups-stuff/LsCs_local_release/lib/LsCs/libLsCsSql.so.0 (0x00007f1c3ce00000)
libpq.so.5 => /lib/x86_64-linux-gnu/libpq.so.5 (0x00007f1c3d56f000)
libLsCsCore.so.0 => /home/roland/cups-stuff/LsCs_local_release/lib/LsCs/libLsCsCore.so.0 (0x00007f1c3b600000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f1c3b200000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1c3d54f000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1c3cc1e000)
libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f1c3b4a1000)
libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007f1c3d4a4000)
libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007f1c3ac00000)
libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f1c3d451000)
libldap-2.5.so.0 => /lib/x86_64-linux-gnu/libldap-2.5.so.0 (0x00007f1c3d1a1000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f1c3d430000)
libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f1c3b0c8000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1c3ab20000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1c3d5dd000)
libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f1c3aa46000)
libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f1c3d174000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f1c3d428000)
libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f1c3d41a000)
liblber-2.5.so.0 => /lib/x86_64-linux-gnu/liblber-2.5.so.0 (0x00007f1c3d164000)
libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f1c3d147000)
libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f1c3a800000)
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007f1c3a766000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f1c3d411000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f1c3d136000)
libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f1c3a632000)
libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f1c3d105000)
libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f1c3a47c000)
libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f1c3b48c000)
libnettle.so.8 => /lib/x86_64-linux-gnu/libnettle.so.8 (0x00007f1c3b43e000)
libhogweed.so.6 => /lib/x86_64-linux-gnu/libhogweed.so.6 (0x00007f1c3a433000)
libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f1c3a3b2000)
libffi.so.8 => /lib/x86_64-linux-gnu/libffi.so.8 (0x00007f1c3b432000)
roland@mxz2g4:~/cups-stuff/LsCs_local_release/lib/LsCs/plugins
$


You will note this requires the previously mentioned core and sql libraries. You will also note that ldd managed to find them. Why? Because for a local build I have to hack RPATH, something that isn’t allowed in secured environments.

ldopen does not search that which it previously loaded when loading a plugin.

Off top of head without looking it up, ldopen looks

  1. Same location as the just opened library file
  2. RPATH/RunPath baked into the just opened library file
  3. The LD_LIBRARY_PATH environment variable (which can trash many other things) and is only parsed as part of the standard library startup so no application level changes to it will be honored. Windows parses PATH each and everytime.
  4. Finally the system library paths/links in the cache populated by ldconfig which has some hard coded things, then all the .conf files from /etc/ld.so.conf.

What happens here, especially when one has malicious intent, is the main module loads the official libraries and libraries of the same name providing the same classes and functions but doing malicious things get loaded for the plugin.

This is basically not auditable. Given the constant push of updates from distros and product vendors, you just cannot reliably create a script to parse out RPATH and RUNPATH of everything into a file you compare for changes. There will always be changes because dates and sizes change even with legit updates.

Enter libtool.

The concept is you create a .conf text file stored at a known place with your application. Humans and scripts can audit this.

$ tree share
share
β”œβ”€β”€ doc
β”‚   └── LsCs
β”‚       └── license
β”‚           β”œβ”€β”€ LGPL_EXCEPTION.txt
β”‚           β”œβ”€β”€ LICENSE.FDL
β”‚           └── LICENSE.LGPL
β”œβ”€β”€ LsCs
β”‚   └── lscs.conf


$ cat share/LsCs/lscs.conf
[Paths]
Prefix = /home/roland/cups-stuff/LsCs_local_release
Headers = include/LsCs
Libraries = lib/LsCs
Binaries = bin
Plugins = lib/LsCs/plugins
Data = share/LsCs
Translations = share/LsCs/translations


This text file is completely auditable via scripts that search for diffs.

lt_dlopen() first checks values set by one or more of these

============================

Function: int lt_dladdsearchdir (const char *search_dir)

Append the search directory search_dir to the current user-defined library search path. Return 0 on success.

Function: int lt_dlinsertsearchdir (const char *before, const char *search_dir)

Insert the search directory search_dir into the user-defined library search path, immediately before the element starting at address before. If before is β€˜NULL’, then search_dir is appending as if lt_dladdsearchdir had been called. Return 0 on success.

Function: int lt_dlsetsearchpath (const char *search_path)

Replace the current user-defined library search path with search_path, which must be a list of absolute directories separated by LT_PATHSEP_CHAR. Return 0 on success.

Function: const char * lt_dlgetsearchpath (void)

Return the current user-defined library search path.

Function: int lt_dlforeachfile (const char *search_path, int (*func) (const char *filename, void * data), void * data)

In some applications you may not want to load individual modules with known names, but rather find all of the modules in a set of directories and load them all during initialisation. With this function you can have libltdl scan the LT_PATHSEP_CHAR-delimited directory list in search_path for candidates, and pass them, along with data to your own callback function, func. If search_path is β€˜NULL’, then search all of the standard locations that lt_dlopen would examine. This function will continue to make calls to func for each file that it discovers in search_path until one of these calls returns non-zero, or until the files are exhausted. β€˜lt_dlforeachfile’ returns the value returned by the last call made to func.

For example you could define func to build an ordered argv-like vector of files using data to hold the address of the start of the vector.

============================

You use your β€œprefix” and other values to create delimited list of paths for lt_dlopen() to search.

Utopia is broken.

By default, just like the SDL3 library, lt_dlopen() opens your plugin, then uses ordinary dlopen() to find the dependencies.

There is vague documentation about certain flags that can be set so lt_dlopen() uses itself to open dependencies which would then be opened using the list it used to find the plugin in the first place.

If SDL3 is β€œjust wrapping” dlopen, it has the same security hole as other packages.

https://www.vulmon.com/searchpage?q=rpath+linux&sortby=byrelevance&scoretype=cvssv3