Dynamic Loading of Shared Objects at Runtime
I’ve been doing a lot of tutorials about dynamic stuff. It’s because we trees rustle whenever the wind blows.
Anyway, this post comes as an addition to @_py’s Linux Internals - The Art Of Symbol Resolution.
This whole thing started when I decided that the bot in our channel (#0x00sec at irc.0x00sec.org), pinetree, needed to be able to dynamically load and unload event-driven modules at runtime.
Building a complex piece of software is very difficult. The typical, rudimentary way to do it is that every time you add a feature (or module) you recompile the entire program. This is just a waste of time. Instead, you can support dynamic modules that get loaded at runtime. This way, if I need to make a new command for pinetree or if I need to revise an existing one, I can simply unload the command from the running instance of pinetree, edit the code for the command, recompile just that code, and then reload it into the bot.
Like most things in C++, you need a few C functions to implement this behaviour.
The functions we’ll be using today are:
void *dlopen(const char *filename, int flags);int dlclose(void *handle);- and,
void *dlsym(void *handle, const char *symbol);
All of these are defined in dlfcn.h, which comes standard. These functions are only available on Linux. If the viewers want to see this done on Windoze, then pestering @dtm might be a good idea!
Details on the dlfcn API.
The dlopen function takes two parameters:
const char *filename is the path to the shared object file to load.
Note: A shared object file (usually denoted with the extension *.so) is a compiled binary with no main function. Instead, it is to be loaded by one or more other programs. The most common shared object file is… libc. That’s right! Your C program (unless static) calls upon a shared object file to get all of its standard library functionality. In C++, libc++ is the corresponding standard library.
int flags is not too important for now because most of the defaults are fine for us. I will explicitly use only one flag: RTLD_NOW which performs symbol resolution for all symbols within the shared object file right when it is first loaded. You may recall @_py’s article on lazy symbol resolution, Linux Internals - Dynamic Linking Wizardry. The flag that resolves symbols lazily, or as-needed, is RTLD_LAZY which is a default flag that would be used if RTLD_NOW were not specified. I use RTLD_NOW because I want the program to crash immediately if a symbol is unresolvable (it helps for debugging).
The return value of dlopen is handle (a void*) to the newly-loaded shared library. If the file couldn’t be loaded or couldn’t be found, dlopen returns NULL (nullptr in C++).
dlclose takes one parameter, a void *handle to the shared library. This is the same void* that was returned from dlopen.
Any non-zero value indicates that dlclose failed; a return of 0 means that dlclose executed successfully. Note: It’s very important to be aware of return values since many functions return the same values (e.g., 0, 1, -1) yet the values carry very difference meanings.
dlsym is the program that accesses symbols from the shared library. It returns any/all requested symbols simply as void*'s. Some casting must be done, then, to get your desired results.
The first parameter void *handle is just the handle you get from dlopen.
const char *symbol is the string representation of the symbol you want. Note: A symbol can be a function, struct, variable, or anything, really.
If the symbol is found, you get a pointer to it (that, again, you may have to cast); otherwise, the function returns NULL or nullptr.
Symbols in C++
Symbols in C++ can only be described as a giant pain. Name mangling makes it difficult to resolve symbols at runtime because there is no standard convention for how symbols are mangled (i.e., it is compiler-specific).
For example, a typical C function like do_something has the symbol do_something when compiled with a C compiler. With a C++ compiler, that function could look like AFDa_do_somethingFAK_x.
I think it’s important to briefly mention the purpose of name mangling, however: name mangling allows for function overriding (did someone say “class inheritance?”) and other really convenient things about C++ that wouldn’t work in C.
Of course, if you want your C++ compiler to not name-mangle a certain function or variable, you can use extern "C" to direct the compiler to compile the code as C. Here is an example:
// do_something.cpp
extern C int do_something(void) {
return 0;
}
int main() {
return do_something();
}
If we compile the program with g++ and then run nm (a program that shows us the symbols within the binary), we can see that do_something() has this symbol:
00000000004004d6 T do_something
Let’s use g++ again but we will remove the extern "C" part:
00000000004004d6 T _Z12do_somethingv
See the difference?
The Problem with extern "C"
extern "C" will not work with things that are specific to C++ and have no corresponding C behaviour. For example, you can’t extern "C" a class because there are no classes in C!
The solution to this is to use helper functions to instantiate classes like so:
extern "C" SomeObject* factory(void) {
return new SomeObject;
}
A helper function that instantiates a class this way is often called a factory.
The Second Problem
So you have some class declared in some file and a factory to instantiate it. You want to compile that code as a shared library so that some main program can utilize the functionality.
The solution is Polymorphism, a programming concept that allows to to define some base functionality that all subclasses can implement differently. Because this topic is so large, I won’t go into it here.
We will define a base class with some basic functionality. In this tutorial, we will have a base class Widget that is purely abstract, meaning that descending classes must implement all of Widget's virtual functions to even compile.
// widget.hpp
#ifndef __WIDGET_H
#define __WIDGET_H
// ignore this line for now...
extern int unique_signal; // for use in proving symbol stuff.
#include <string>
// pure virtual class!
class Widget {
public:
virtual std::string message(void) = 0; // pure virtual func
};
#endif
All descending classes will inherit from Widget and implement std::string message(void). When we load the dynamic libraries, we will treat all of the instances as Widget*'s (this is the polymorphism at work).
The Main Program
// main.cpp
#include <dlfcn.h>
#include <iostream>
#include <fstream>
#include <stdexcept>
#include <string>
#include <vector>
#include "./widget.hpp"
typedef void* dynamic_lib_handle;
static dynamic_lib_handle load_lib(const std::string& path);
static Widget* instantiate(const dynamic_lib_handle handle);
static void close_lib(dynamic_lib_handle handle);
struct dynamic_lib {
dynamic_lib_handle handle;
std::string path;
dynamic_lib(std::string p) : path(p), handle(nullptr) {}
~dynamic_lib() {
if (handle != nullptr)
close_lib(handle);
}
};
int unique_signal = 42;
int main(int argc, char **argv) {
if (argc < 2)
return 1;
std::vector<dynamic_lib> libs;
try {
std::cout << "Opening: " << argv[1] << std::endl;
std::ifstream fs(argv[1]);
std::string tmp;
// read from the file.
while(std::getline(fs, tmp))
libs.push_back(dynamic_lib(tmp));
} catch (std::exception& e) {
std::cerr << e.what() << std::endl;
return 2;
}
// load up all the libs
for (auto& l : libs) {
l.handle = load_lib(l.path);
}
std::vector<Widget*> widgets;
// instantiate!
for (auto& l : libs)
widgets.push_back( instantiate(l.handle) );
// call each widget's message() func.
for (Widget* w : widgets) {
if (w == nullptr) continue;
std::cout << w->message() << std::endl;
delete w;
}
}
static dynamic_lib_handle load_lib(const std::string& path) {
std::cout << "Trying to open: " << path << std::endl;
return dlopen(path.data() , RTLD_NOW); // get a handle to the lib, may be nullptr.
// RTLD_NOW ensures that all the symbols are resolved immediately. This means that
// if a symbol cannot be found, the program will crash now instead of later.
}
// ...
static void close_lib(dynamic_lib_handle handle) {
dlclose(handle);
}
I hope the above code is fairly straight forward to you guys. Feel free to comment questions about it, though. Study it carefully first.
The really important part of code is the static Widget* instantiate(const dynamic_lib_handle handle) function. This function does the symbol resolution of the factory function, does the appropriate casting, and then returns an instance of the widget subclass defined in the shared library pointed to by the handle:
static Widget* instantiate(const dynamic_lib_handle handle) {
if (handle == nullptr) return nullptr;
void *maker = dlsym(handle , "factory");
if (maker == nullptr) return nullptr;
typedef Widget* (*fptr)();
fptr func = reinterpret_cast<fptr>(reinterpret_cast<void*>(maker));
return func();
}
The casting part was a pain to do because C++ is very picking about casting a void* to a function pointer. I will post the StackOverflow link that helped me if I find it again.
Adding Some Widgets
Now, let’s add some widgets:
// test-widget1.cpp
#include <string>
#include "./widget.hpp"
class TestWidget1 : public Widget {
public:
std::string message(void) {
return "Hello. I'm Test Widget1\nOh and the unique_signal is: " + std::to_string(unique_signal);
}
};
extern "C" Widget* factory(void) {
return static_cast<Widget*>(new TestWidget1);
}
// test-widget2.cpp
#include <string>
#include "./widget.hpp"
class TestWidget2 : public Widget {
public:
std::string message(void) {
return "Hello. I'm Test Widget2";
}
};
extern "C" Widget* factory(void) {
return static_cast<Widget*>(new TestWidget2);
}
The main difference between the two widgets above is that TestWidget1 uses unique_signal in its message function. This symbol is declared as extern in widget.hpp and is defined in main.cpp. TestWidget1 will crash the program if it cannot access the value of unique_signal. This means that the widgets need access to symbols in the main program. By tinkering with soem compiler flags (specifically -rdynamic) we can make this happen.
Compiling
Let’s compile the widgets first (after all, order doesn’t matter and that’s sort of the point of this whole thing…).
$ g++ --std=c++17 -fPIC -rdynamic -shared -o ./test-widget1.so ./test-widget1.cpp
$ g++ --std=c++17 -fPIC -rdynamic -shared -o ./test-widget2.so ./test-widget2.cpp
-shared tells the compiler that we intentionally didn’t put a main() in the source for these binaries. -rdynamic and -fPIC have to do with symbol access.
Compiling the main program isn’t too different:
$ g++ --std=c++17 -rdynamic -o main.out main.cpp -ldl
-rdynamic is what lets the shared libraries access the symbols in our main program. -ldl links our main program with the dlfcn code. Experienced C/C++ people know that libc and/or libc++ are automatically linked by the compiler as if llibc(++) was passed to gcc or g++. However, -ldl must be specified.
Running
The main program expects one argument, a path to a file listing out which libraries to load. I have called this file libs.txt and it looks like this:
./test-widget1.so
./test-widget2.so
$ ./main.out ./libs.txt
Opening: ./libs.txt
Trying to open: ./test-widget1.so
Trying to open: ./test-widget2.so
Hello. I'm Test Widget1
Oh and the unique_signal is: 42
Hello. I'm Test Widget2
Looks like it all worked!
Conclusion
All of the source code used here is up on the 0x00sec GitLab here.
I hope that this was straightforward and informative.