Full fbembed integration into .NET applications Posted on 21 May 2012 at 04:12

Or as I would call it: going full retard.

In my last article, I provided static builds for the fbembed.dll library, even including the ADO.NET driver. As I am a fan of no-setup, single-EXE deployment, this was a good step forward, because it eliminated the dependencies to the ICU and MSVCRT libraries. However, there is a problem: as ILMerge doesn't work for WPF applications anymore, I use the Costura library to merge assemblies into one executable. It basically works by intercepting the exception when an assembly cannot be resolved, and instead loads the assembly from a resource stream and thus from memory. This is favorable, because without temporary file extraction, no problems will arise if multiple instances, even with multiple versions of the assembly, are active.

Although the second of the provided DLLs is a managed assembly, it cannot be loaded from memory, because native code has to be mapped into process space, and thus physically reside on the file system (there is an exception, but I doubt it'll play well with DllImport). This was a major problem, because without extraction of the library, I could not load the native code in it.

But there is one guaranteed place for executable code which is always available: the executable itself. There is no mayor difference between an EXE and a DLL. Both are PEs (portable executable), both can export symbols, and both can dynamically and statically link to code. So the only way to embed the Firebird engine into a .NET application was to directly embed the native code into the application itself.

First try: C++/CLI wrapper

My first idea was to write a FirebirdSql.Data.Client.Native.IFbClient implementation, which, instead of doing lots of interop calls, being a C++/CLI class, that directly calls into the statically linked fbembed.lib. I produced a little test case, which worked, so I got into implementing the wrapper class. However, turns out, C++/CLI projects cannot be statically linked against the MSVCRT libraries (/MT switch is incompatible with /CLR), which was the whole point. It's also a mayor PITA to write all the interop code yourself. So this proved a dead end. Every API change would also mean changing the wrapper code.

Second try: linking everything together

As the ADO.NET driver allows arbitrary names for the client library, my idea was to link my application against the fbembed.lib with a .def file declaring all the exported symbols. This has a two-fold advantage, because the .def file will warn you about missing symbols, and on the other hand, force all the relevant code into the result.

It's ugly, and I will describe its ugliness below, but the first test failed. As all the linking didn't yield any .pdb files, debugging was impossible, so it really was a guessing game. In the end, I came to two conclusions:

  • If you create an executable which is linked against MSVCRT, the runtime library will do some initialization, like initializing global and static variables, preparing the heap, etc., which is a transparent process.
  • If you create a DLL, the moment you call LoadLibrary, the runtime library will basically do the same initialization, much more transparent.

In either case, the modules will fail if the initialization has not been done. Because we are treating a DLL like an EXE, the function that does the initialization is called DllMain. This is called once per load/unload, and each time a thread attaches. But DllMain doesn't initialize the runtime library. The symbol that does that is _DllMainCRTStartup. So the whole process is the following.

  1. Implement a "jump start" for the MSVCRT code you will later link in
  2. Compile your .NET/WinForms/WPF project as usual
  3. Disassemble the assembly, remove the assembly manifest part from the IL code, and recompile as a .netmodule
  4. Link all necessary libraries, including Windows libraries, fbembed, all its dependencies, the static MSVCRT (libcmt.lib), your native .RES file and .NET resources together, with a .def file not only containing the fbclient/fbembed symbols, but also _DllMainCRTStartup

You need to use the Visual Studio 2008 tools, because 2010 will only yield 4.0 .NET executables. In my case, I want 3.5, which uses the 2.0 runtime. The jump start code looks like this:

public enum ReasonForCall : uint { DLL_PROCESS_ATTACH = 1, DLL_PROCESS_DETACH = 0, DLL_THREAD_ATTACH = 2, DLL_THREAD_DETACH = 3 } [DllImport("kernel32.dll", CharSet = CharSet.Auto)] private static extern IntPtr GetModuleHandle(IntPtr module); [DllImport("YourAssembly.exe")] private static extern uint _DllMainCRTStartup(IntPtr hModule, ReasonForCall ulReasonForCall, IntPtr lpReserved); [...] _DllMainCRTStartup(GetModuleHandle(IntPtr.Zero), ReasonForCall.DLL_PROCESS_ATTACH, IntPtr.Zero);

You have to specify your connection string to include the not-so-common client library:

string connectionString = "ServerType=1;User=SYSDBA;Password=masterkey;Dialect=3;Charset=UTF8;Database=DATA.FDB;client library=YourAssembly.exe";

The ADO.NET driver will load/create databases and work as usual. Later on, I will create a small executable that does the necessary steps, because currently manual modification of the IL code is necessary. It's also important to know which libraries are required and how to link them. For now, it's possible, a fully functional WPF application which creates/opens a Firebird database and executes queries against it, without external dependencies besides the .NET Framework itself.