Introduction

When mixing multiple languages in the same project, there are often some subtle issues that can crop up from their interaction. One of those tricky cases is exception handling. Even handling exceptions across shared libraries implemented in a single language, like C++, can be a problem if different compilers were used to compile the libraries. Since the D Programming Language has exceptions built in, and they are always on, some special care should be taken when interacting with other languages. In this article, I'm going to explain a specific scenario I encountered using GLFW 3 in D and the solution I came up with. I think this solution can work in many situations in which D is used to create an executable interacting with C libraries. If it's the other way around, where a C executable is using D libraries, this isn't going to be as broadly applicable.

About GLFW

GLFW 3 is a cross-platform library that can be used to create OpenGL applications. It abstracts away window and context creation and system event processing. The latest version has been pared down quite a bit to provide little more than that. It's a small library and does its job well. Events in GLFW are processed via a number of callback functions. For example, the following C code shows how to handle notification of window close events. #include // Assume this is opened elsewhere. FILE *_log; static void onWindowClose( GLFWwindow* win ) { fputs( "The window is closing!", _log ); // Tell the main loop to exit setExitFlag(); } void initEventHandlersForThisWindow( GLFWwindow* win ) { // Pass the callback to glfw glfwSetWindowCloseCallback( win, onWindowClose ); } All system events are handled in this manner. In C, there's very little to get into trouble with here. There is a warning in the documentation not to call glfwDestroyWindow from a callback, but other than that pretty much anything goes. When doing this in D, the restriction on destroying windows still holds, but that's not all.

The Problem

I've seen code from people using D who misunderstood the meaning of D's extern( C ) linkage attribute. They believed that this restricted them to only using C libraries and/or C constructs inside the function. This is not the case. Any D library calls can be made and any valid D code can be used in these functions. All the attribute does is tell the compiler that the function uses the __cdecl calling convention. I bring this up because one of the primary use cases of implementing a function in D with extern( C ) linkage is for callbacks to pass to C libraries. To demonstrate, I'll rewrite the above C example in D. import std.stdio; // Imaginary module that defines an App class import mygame.foo.app; // Assume this is opened elsewhere. File _log; // private at module scope makes the function local to the module, // just as a static function in C. This is going to be passed to // glfw as a callback, so it must be declared as extern( C ). private extern( C ) void onWindowClose( GLFWwindow* win ) { // Calling a D function _log.writeln( "The window is closing!" ); // Tell the app to exit. myApp.stop(); } void initEventHandlersForThisWindow( GLFWwindow* win ) { glfwSetWindowCloseCallback( win, &onWindowClose ); } This bit of code will work fine, probably very near 100% of the time. But taking a gander at the documentation for std.stdio.File.writeln will reveal the following.

Throws: Exception if the file is not opened. ErrnoException on an error writing to the file.

Now imagine that there is a write error in the callback and an ErrnoException is thrown. What's going to happen? GLFW events are processed by calling glfwPollEvents. Then, as events are internally processed, the appropriate callbacks are called by GLFW to handle them. So the sequence goes something like this: DFunc -> glfwPollevents -> onWindowClose -> return to glfwPollEvents -> return to DFunc. Now, imagine for a moment that all GLFW is written in D so that every function of that sequence is a D function. If log.writeln throws an ErrnoException, the sequence is going to look like this: DFunc -> glfwPollEvents -> onWindowClose -> propagate exception to glfwPollEvents -> propagate exception to DFunc. That would be great, but it isn't reality. Here's what the sequence looks like when compiling a 32-bit app with DMD on Windows: DFunc -> glfwPollEvents ->onWindowClose -> {} -> return to glfwPollEvents -> return to DFunc. The {} indicates that the exception is never propagated beyond the callback. Once onWindowClose returns, execution returns to a part of the binary that was written in C and was not compiled with any instructions to handle exception propagation. So the exception is essentially dropped and the program will, with luck, continue as if it were never thrown at all. On Linux, what happens depends on which compiler you use. If it's DMD, the app will just abort. If it's GDC, the exception will propagate up the callstack, but there's likely a risk of leaving the C side of things in an invalid state. Relying on compiler behavior only works if it can be guaranteed the code will be compiled with a specific compiler and platform. Otherwise, that's just not a reasonable approach. If the exceptions are going to be dealt with, they ought to be handled in a way that is consistent across compilers and platforms. The result of not handling an exception can be unpredictable. Sometimes, it's harmless. In this particular example, it could be that nothing is ever written to the same log object again, or maybe the next call to _log.writeln succeeds, or maybe it fails again but happens in D code where the exception can be propagated. In my own tests using callbacks that do nothing but throw exceptions, no harm is ever done. But it's not a perfect world. Sometimes exceptions are thrown at a certain point in a function call, or as a result of a certain failure, that causes the application to be in an invalid state. This can, sooner or later, cause crashes, unexpected behavior, and hard to find bugs. For a program to be more robust, exceptions thrown in D from C callbacks ought to be handled somehow.

A Solution

I'm convinced that there's a genetic mutation we programmers have that leads us all to believe we can be disciplined about our code. I know I've suffered from it. I've used C for years and never felt the need to choose C++ for any of my projects. I know all of the pitfalls of C strings and C arrays. I can manage them effectively! I don't need any std::string or std::vector nonsense! I've got some good C code lying around that mitigates most of those problems most of the time. Yet despite (or maybe because of) all of my confidence in my ability to properly manage the risks of C, I've still had bugs that I wouldn't have had if I'd just gone and used C++ in the first place. Coding discipline is a skill that takes time to learn and is never perfectly maintained. It's irrational to believe otherwise. We all lapse and make mistakes. Any solution to this problem that relies on discipline is a nonsolution. And that goes doubly so for a library that's going to be put out there for other people to use. In this particular case, that of GLFW event handling, one way around this is to use the callbacks solely to queue up custom event objects, and then process the queue once glfwPollEvents returns. That's a workable solution, but it's not what I settled on. I have an aversion to implementing anything that I don't absolutely need. It just doesn't feel clean. Besides which, it's a solution that's useful only for a subset of cases. Other cases that don't translate to the event-handling paradigm would require a different approach. Another solution is to is to wrap any external calls made by the callback in a try...catch block and then save the exception somewhere. Then, when the original call into C returns, the exception can be rethrown from D. Here's what that might look like. // In D, class references are automatically initialized to null. private Throwable _rethrow; private extern( C ) void onWindowClose( GLFWwindow* win ) { try { _log.writeln( "The window is closing!" ); myApp.stop(); } catch( Throwable t ) { // Save the exception so it can be rethrown below. _rethrow = t; } } void pumpEvents() { glfwPollEvents(); // The C function has returned, so it is safe to rethrow the exception now. if( _rethrow !is null ) { auto t = _rethrow; _rethrow = null; throw t; } } Notice that I'm using Throwable here. D has two exception types in the standard library, which both derive from Throwable: Exception and Error. The former is intended for recoverable errors, while the latter should be thrown to indicate an unrecoverable error in the program. However, the language does not prevent them being caught. Andrei Alexandrescu's book "The D Programming Language" has this to say about catching Throwable.

The first rule of Throwable is that you do not catch Throwable. If you do decide to catch it, you can't count on struct destructors being called and finally clauses being executed.

Another issue is that any Error caught might be of the type AssertError. This sort of error really should be propagated all the way up the call stack. It's acceptable to catch Throwable here, since I'm rethrowing it. By making _rethrow a Throwable, I'm ensuring that I won't miss any exceptions, regardless of type. With one caveat. This implementation is fine if only one callback has been set. But if others have been set, glfwPollEvents can call any number of them on any given execution. If more than one exception is thrown, the newest will always overwrite its immediate predecessor. This means that, potentially, an Exception might overwrite an Error that really shouldn't be lost. In practice, this is unlikely to be a problem. If I'm implementing this for my own personal use, I know whether or not I care about handling multiple exceptions and can modify the code at any time if I find that I need to later on. But for something I want to distribute to others, this solution is not enough. Like exceptions in Java, exceptions in D have built-in support for what's known as 'exception chaining.' Throwable exposes a public field called next. This can be set in a constructor when the exception is thrown. void foo() { try { ... } catch( Throwable t ) { throw new MyException( "Something went horribly wrong!", t ); } } void bar() { try { foo(); } catch( MyException me ) { // Do something with MyException me.doSomething(); // Throw the original throw me.next; } } Exception chaining is something that isn't needed often, but it can be useful in a number of cases. The problem I'm describing in this article is one of them. Given the information so far, a first pass modification of onWindowClose might look something like this. try { ... } catch( Throwable t ) { // Chain the previously saved exception to t. t.next = _rethrow; // Save t. _rethrow = t; } This appears to do the job, making sure that no exceptions caught here are lost. However, there's still a problem. If t has an existing exception chain, then setting t.next will cause any previous exceptions connected to it to be lost. To make the problem clear, what I want to do here is to save the caught exception, t and anything chained to it, along with any previously saved exceptions and their chains. This way, all of that information is available if needed once the C callback returns. That appears to call for more than one next field. Additionally, it would probably be a good idea to distinguish between saved Errors and Exceptions and handle them appropriately so that there's no need to rely on programmer discipline to do so elsewhere. Finally, it would be nice for user code to be able to tell the difference between exceptions thrown from the C callbacks and exceptions thrown from normal D code. This can be a useful aid in debugging. Given the above contraints, a simple solution is a custom exception class. Here is the one I've implemented for my own code. class CallbackThrowable : Throwable { // This is for the previously saved CallbackThrowable, if any. CallbackThrowable nextCT; // The file and line params aren't strictly necessary, but they are good // for debugging. this( Throwable payload, CallbackThrowable t, string file = __FILE__, size_t line = __LINE__ ) { // Call the superclass constructor that takes file and line info, // and make the wrapped exception part of this exception's chain. super( "An exception was thrown from a C callback.", file, line, payload ); // Store a reference to the previously saved CallbackThrowable nextCT = t; } // This method aids in propagating non-Exception throwables up the callstack. void throwWrappedError() { // Test if the wrapped Throwable is an instance of Exception and throw it // if it isn't. This will cause Errors and any other non-Exception Throwable // to be rethrown. if( cast( Exception )next is null ) { throw next; } } } Generally, it is frowned upon to subclass Throwable directly. However, this subclass is not a typical Exception or Error and is intended to be handled in a special way. Furthermore, it wraps both types, so doesn't really match either. Conceptually, I think it's the right choice. This is intended to be used to wrap any exceptions thrown in the callbacks. I assign the caught exception to the next member, via the superclass constructor, so that it and its entire chain are saved. Then I assign the previously saved CallbackThrowable to the custom nextCT member. If nothing was previously saved, that's okay, since D will have initialized _rethrow to null. Finally, I save the new CallbackThrowable to the _rethrow variable. The modified callback example below demonstrates how this is used. private CallbackThrowable _rethrow; private extern( C ) void onWindowClose( GLFWwindow* win ) { try { _log.writeln( "The window is closing!" ); myApp.stop(); } catch( Throwable t ) { // Save a new CallbackThrowable that wraps t and chains _rethrow. _rethrow = new CallbackThrowable( t, _rethrow ); } } void pumpEvents() { glfwPollEvents(); if( _rethrow !is null ) { // Loop through each CallbackThrowable in the chain and rethrow the first // non-Exception throwable encountered. for( auto ct = _rethrow; ct !is null; ct = ct.nextCT ) { ct.throwWrappedError(); } // No Errors were caught, so all we have are Exceptions. // Throw the saved CallbackThrowable. auto t = _rethrow; _rethrow = null; throw t; } } Now, code further up the call stack can look for instances of CallbackThrowable and not miss a single Exception that was thrown in the callbacks. Errors will still be thrown independently, propagating up the callstack as intended. This still isn't quite as perfect as one would like. If multiple Errors were thrown, then all but one would be lost. If that's important to handle, it's not difficult to do so. One potential solution would be to log each error in the loop above and rethrow the last one logged, rather than calling throwWrappedError. Another would be to implement CallbackError and CallbackException subclasses of CallbackThrowable. Errors can be chained to the former, Exceptions to the latter. Then the loop can be eliminated. I'll leave that as an exercise for the reader.

Conclusion

In Java, I see exceptions used everywhere, but that's primarily because there's often no choice (I still cringe thinking about my first experience with JDBC). Checked exceptions can be extremely annoying to deal with. Conversely, in C++ where there are no checked exceptions, I've found that they are less common (in the source I've seen, which certainly is in a narrow domain -- I wouldn't be surprised to find them used extensively in some fields outside of game development). In D, like C++, there are no checked exceptions. However, their usage in D tends to be pervasive, but in a way that is more subtle than in Java. That is, you don't see try...catch blocks all over a D codebase as you do a Java codebase. Often, in user code it's enough just to use try...catch only in special cases, such as file IO when you don't want to abort the app just because a file failed to load, and let the rest propagate on up the call stack. The runtime will generate a stack trace on exit. But they can be thrown from anywhere any time, particularly from the standard library. So in D, whether a codebase is littered with exceptions or not, they should always be on the mind. As such, cross-boundary exception handling is one potential source of problems that needs to be considered when using D and C in the same codebase. The solution presented here is only one of several possible. I find it not only simple, but flexible enough to cover a number of use cases.

Changelog

August 7, 2013: Clarified the difference between Errors and Exceptions. Updated the CallbackThrowable solution to handle Errors differently. Added new information about how exception propagation interacts with C when using different compilers.