Task Dialogs, Activation Contexts, and SEHExceptions

Recently I’ve been involved with creating a general purpose library (to be used at my company) that includes, among other things, a nice API for creating and manipulating Task Dialogs from .NET-land. As with everything unmanaged, implementing it not only yields the benefit of being able to use it, but you tend to learn some interesting things in the process.

Unable to find an entry point named ‘TaskDialogIndirect’…

It won’t take long until you run into this lovely issue. Shouldn’t be much of a surprise to anyone familiar with such matters, but, in case you didn’t know, the TaskDialog API is only available on version 6.0 and up of the Common Controls library (on post-XP of course…I think there’s also a v6 on XP, but it is different version nonetheless). If you’re running Windows Vista or 7, then you have this library installed by default, however it is the older version 5.8 of the Common Controls library that will be loaded by default.

We have multiple versions of comctl32.dll because of that libraries participation in the side-by-side assembly system, which was Microsoft’s answer to DLL Hell and somewhat of a precursor to .NET’s Global Assembly Cache. If we intend on having our .NET application load the correct version of comctl32.dll, then we’re going to have to participate in that system, which then requires us to provide a manifest dictating the version we need.

Providing a manifest is simple if your end-product is a .NET executable: you simply add an application manifest which will isolate the application and load the appropriate DLL versions at run-time; however, it is not as straight forward if you’re writing a .NET library, since simply embedding a manifest into the DLL exerts no influence at run-time on an executable referencing it. Specifically, the type of manifest we are talking about here is an application manifest (as opposed to an assembly manifest). Visual Studio offers support in the C# project properties designer for embedding application manifests into application projects, but not libraries.

Activation Contexts

Requiring that all applications referencing your library also include their own correctly-made application manifest if they want a specific subset of features provided by your library to work is an exceptionally unrealistic requirement. If we cannot automatically affect the run-time so that the proper unmanaged DLL’s are targeted, then we are providing functionality that is not guaranteed to work. As such, such functionality cannot exist in any proper library, and would have to be removed. Luckily, we do have the power to influence the run-time from the confines of a .NET DLL, and the way we do it is by making use of activation contexts.

By activating an activation context, we are essentially telling Windows to redirect the running application to a particular DLL version according to a manifest of our choice. In fact, activation contexts are the very engines that power the side-by-side assembly system. When an application needs to make a call to a library or create a process or other resource, Windows checks that application for the presence of an application manifest, using the information found in that manifest in order to populate an activation context to use to guide that call to the correct destination.

Normally, activation contexts are managed entirely by the Windows system; however, in our case, we’re going to be rudely intruding into that system so that we can perform some actions not taken by the Windows system. Specifically, prior to our P/Invoke call to TaskDialogIndirect, we’re going to create and activate an activation context that will redirect that call to the proper version of comctl32.dll. There is official precedence for this activity: the Windows Forms library does exactly what I’ve just described when you make a call to Application.EnableVisualStyles.

Microsoft provides some documentation on how we can do that from a managed environment in the form of a KB article. I’m not going to provide a complete walkthrough on the process here, as the KB article covers most of it, but I do want to address one of the limitations of the approach offered by that article. In particular, I’m referring to how the approach offered by the KB article requires that an actual manifest file be present on disk. Relying on an external non-binary file for something that shouldn’t be configurable by an end-user anyway is clunky and not desirable.

Luckily, we can do better than that. Instead of using a physical manifest file as a source for the activation context, we can create an activation context using a resource embedded in our DLL instead. And we can do all of this simply by configuring the ACTCTX structure populated during the process differently.

Activation Context Creation Using a PE Image Source

The application manifest can typically be found in the .rsrc section of a PE image, where it exists as a resource like any other. With Visual Studio, you can add an application manifest to your project (let’s call ours Library.manifest), and then enable the embedding of that manifest into project’ s output through an option located in the project properties designer. However, no such option exists for DLL projects, but this doesn’t matter since we can get Visual Studio to do what we want anyways. Open up your *.csproj file with the XML editor, and add the following to the first <PropertyGroup> in the file:

<ApplicationManifest>Library.manifest</ApplicationManifest>

This will result in your manifest file being embedded into the DLL compiled from your project. You will see that the manifest is embedded, not as a .NET resource, but as a true native resource of the RT_MANIFEST type. The resource ID of the manifest should be 2. This is the standard resource ID for all manifests found in DLLs. In fact, whenever a native DLL containing an embedded manifest resource with an ID of 2 is dynamically loaded at run-time, the operating system loader automatically creates an activation context using that manifest. It does this so it can then proceed to load all dependencies of that DLL without issue.

This obviously is not going to happen for our DLL, however, since this our DLL is a managed DLL, of course, and it is loaded a bit differently. Regardless of this, we still need our manifest embedded in our DLL so that it can be sourced appropriately by the activation context we are going to be creating.

Following this, we need to change some of the code you may have picked up from that KB article. Specifically, we need to populate the ACTCTX structure that gets provided to CreateActCtx a bit differently.

1. The KB article sets the lpAssemblyDirectory to the directory containing the current assembly. Although the KB article is throwing terms like “security hole” at us in the nearby code comments, we’re actually going to remove this assignment, and leave lpAssemblyDirectory unset. I don’t believe this is documented anywhere, but in actuality, I believe the Activation Context API ignores lpAssemblyDirectory when loading the manifest in the way we are going to be doing it.
2. Next, the KB article has us setting the dwFlags field to ACTCTX_FLAG_ASSEMBLY_DIRECTORY_VALID. We actually want to set it to ACTCTX_FLAG_RESOURCE_NAME_VALID instead (which is 8, by the way).
3. The provided example sets the lpSource field to the path of the physical manifest file. Since we don’t have one of those, and because our manifest file is embedded in our DLL, we actually want to set lpSource to the path to our DLL file.
4. Finally, we need to tell Windows what the resource ID of our manifest is, and we provide that information by setting the lpResourceName member using MAKEINTRESOURCE.

The last step in the steps listed above requires us to set lpResourceName using a value that we derive from MAKEINTRESOURCE. While that’s not a very tall order when we’re developing in C++, how do we do this from a managed, C# environment? The simplest way is to actually change the return type for this field in our ACTCTX structure definition.

Looking at the KB article, the sample ACTCTX structure they provide looks like the following:

private struct ACTCTX
{
   public int       cbSize;
   public uint      dwFlags;
   public string    lpSource;
   public ushort    wProcessorArchitecture;
   public ushort    wLangId;
   public string    lpAssemblyDirectory;
   public string    lpResourceName;
   public string    lpApplicationName;
}

Change the return type of lpResourceName to an IntPtr (!!); yes, an IntPtr, so that we have the following:

   public IntPtr    lpResourceName;

And then, remembering that the ID for our manifest resource is 2, we can then populate our structure like so (substituting “ClassName” for the name of the class where this is being done of course):

ACTCTX context = new ACTCTX
                     {
                         cbSize = Marshal.SizeOf(typeof (ACTCTX)),
                         dwFlags = ACTCTX_FLAG_RESOURCE_NAME_VALID,
                         lpSource = typeof(ClassName).Assembly.Location,
                         lpResourceName = (IntPtr) 0x2
                     };

Providing this structure to CreateActCtx will then create a new activation context based on our embedded manifest. Well almost. Did you happen to create your application manifest using Visual Studio’s application manifest file template?

Fix Your Application Manifest

I found CreateActCtx to be extremely touchy when it comes to the actual form of the application manifest itself, and that the manifest generated by Visual Studio was utterly incompatible with it. Attempted to create a new activation context using a manifest like that would result in CreateActCtx returning an error code.

The manifest generated by Visual Studio contains a ton of XML namespace attributes which may or may not make CreateActCtx puke. I say “may or may not” because I’m not sure if it was these namespaces or another part of the stock manifest content that caused it to fail. But, that doesn’t really matter. Here’s a cleaned up manifest file that is guaranteed to work for you:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
  <noInherit/>
  <description>
    Your manifest
  </description>
  <assemblyIdentity version="0.0.0.1"
                    name="Your.Library"
                    type="win32"
                    processorArchitecture="*"
                    />
  <dependency optional="yes">
    <dependentAssembly>
      <assemblyIdentity
       type="win32"
       name="Microsoft.Windows.Common-Controls"
       version="6.0.1.0"
       publicKeyToken="6595b64144ccf1df"
       language="*"
       processorArchitecture="*"/>
    </dependentAssembly>
  </dependency>
</assembly>

You can add other sections, such as a <compatibility> section if you’d like, and they should work fine. Also, the <description> element is not required for this to work (hell, I’m not even sure if that’s a standard manifest element…all I know is that I’ve seen it in a number of in-use manifests originating from Microsoft). After you do all of this, it should start to work with CreateActCtx.

But hey…this is where the fun begins. You shouldn’t get the false impression from all of this that you can now go willy-nilly and be able to act foolishly without suffering consequences. I heavily tested my code and came across a few “edge” cases that you need to be very careful about, as you can easily cause issues when acting within one of these self-made activation contexts.

Who Doesn’t Love SEHExceptions?!

You know that when you’re getting structured exception handling errors in managed code, that you’re doing something very special. Now, before I go on, let me state that I tested my use of activation contexts very heavily and found them to be very stable. However, a factor allowing me to come to that conclusion is the fact that the code executing within the activation context is very stable. If your code is anything less than that, or if it is operating in an extreme environment, you probably want to exercise caution.

Through my testing, I did identify an issue that I do not altogether understand, due to the fact that the problem was occurring deep in unmanaged land, and I couldn’t come across much material relevant to my issue.

The problem I encountered was an interesting one. As we know, we’re dealing with Task Dialogs here. While the Task Dialog is open, the thread that opened it will be blocked until it is closed. Well, not entirely. While blocked, however, the same thread is going to handle any callbacks fired by the Task Dialog. Because we require an activation context to open the Task Dialog, the call to open the Task Dialog is done within a using block for the disposable class which handles the activation context creation and activation. When we hit the finally block of that using block, it’s going to make a call to DeactiveActCtx.

I found that if I threw an exception while handling a callback from the Task Dialog, that an SEHException would get thrown by DeactiveActCtx during the disposal of the class that created the activation context. The activation context would essentially become impossible to deactivate, indicating perhaps that somehow the activation context stack was corrupt. The error code for the SEHException was 0×80004005, or: External component has thrown an exception. Throwing an exception within the activation context, but not during the handling of a callback, would cause no problems when deactivating the context.

So…if anyone else has this issue, then I would advise to make sure the dialog is closed first before throwing the exception. My Task Dialog class has a Closed event, so I would simply schedule the exception to be thrown in an event handler for that, and then proceed to close the dialog from the callback. The context would deactivate with no issue, and the exception could then get thrown and thrown in the developer’s face.