return_by_smart_ptr policy for Boost Python

As part of my ongoing work to create some minimal Python bindings for Nebula3 I've implemented a new return value policy for Boost Python that's tailored for Nebula3 singletons. There were two existing potential candidates but neither quite fit the job. One was the manage_new_object policy, which will delete the C++ object embedded in a Python object when the Python object is destroyed, this leads to problems like this:

>>> from pynebula3.Core import CoreServer
# C++ singleton object is created and embedded in a new Python object
>>> coreServer = CoreServer.Create()
>>> print coreServer
<pynebula3.Core.CoreServer object at 0x00AB1340>
# existing C++ singleton object is embedded in a new Python object
>>> coreServer2 = CoreServer.Instance()
>>> print coreServer2
<pynebula3.Core.CoreServer object at 0x00AB1378>
>>> print coreServer2.AppName
Nebula3
# the C++ singleton object embedded in the Python object is destroyed
>>> coreServer = None
# remaining Python object now contains a dangling pointer to the C++ singleton object
>>> print coreServer2
<pynebula3.Core.CoreServer object at 0x00AB1378>
>>> print coreServer2.AppName # crash!

The other candidate was the reference_existing_object policy, and this is the one that is usually suggested for use with singletons. Under this policy the C++ object embedded in a Python object is not deleted when the Python object is destroyed, you have to delete the C++ object explicitly. The problem is that you should never delete a reference counted Nebula object explicitly, you should use AddRef()/Release() to adjust the reference count, and it will be automatically deleted when the count hits zero. So you'll have to do something like this:

>>> from pynebula3.Core import CoreServer
>>> coreServer = CoreServer.Create()
>>> coreServer.AddRef()
>>> print coreServer.AppName
Nebula3
>>> coreServer.Release()
>>> coreServer = None

Two lines to create an object and two lines to destroy it? No thanks! And so I set out to create my own policy. Essentially what I wanted is the behavior of the manage_new_object policy where the C++ object is deleted when the Python object is destroyed, but instead of embedding the C++ object itself I wanted it to embed a Nebula smart pointer to the C++ object. In fact manage_new_policy already embeds smart pointers instead of plain pointers, but they're not Nebula smart pointers. So all I had to do was take the manage_new_object policy and make it use Nebula smart pointers. Here is the new policy:

// return_by_smart_ptr.h

#include <boost/python/detail/prefix.hpp>
#include <boost/python/detail/indirect_traits.hpp>
#include <boost/mpl/if.hpp>
#include <boost/python/to_python_indirect.hpp>
#include <boost/type_traits/composite_traits.hpp>
#include "pynebula3/foundation/core/pointee.h"

namespace boost {
namespace python {
namespace detail {

// attempting to instantiate this type will result in a compiler error,
// if that happens it means you're trying to use return_by_smart_pointer
// on a function/method that doesn't return a pointer!
template <class R>
struct return_by_smart_ptr_requires_a_pointer_return_type
# if defined(__GNUC__) && __GNUC__ >= 3 || defined(__EDG__)
    {}
# endif
    ;

// this is where all the work is done, first the plain pointer is
// converted to a smart pointer, and then the smart pointer is embedded
// in a Python object
struct make_owning_smart_ptr_holder
{
    template <class T>
    static PyObject* execute(T* p)
    {
        typedef Ptr<T> smart_pointer;
        typedef objects::pointer_holder<smart_pointer, T> holder_t;

        smart_pointer ptr(const_cast<T*>(p));
        return objects::make_ptr_instance<T, holder_t>::execute(ptr);
    }
};

} // namespace detail

struct return_by_smart_ptr
{
    template <class T>
    struct apply
    {
        typedef typename mpl::if_c<
            boost::is_pointer<T>::value,
            to_python_indirect<T, detail::make_owning_smart_ptr_holder>,
            detail::return_by_smart_ptr_requires_a_pointer_return_type<T>
        >::type type;
    };
};

}} // namespace boost::python

Using the Core::CoreServer again as an example, I can bind it like this:

namespace bp = boost::python;

bp::class_<Core::CoreServer, Ptr<Core::CoreServer>, boost::noncopyable>("CoreServer", bp::no_init)
    .def("Create", &Core::CoreServer::Create, bp::return_value_policy<bp::return_by_smart_ptr>())
    .staticmethod("Create")
    .def("HasInstance", &Core::CoreServer::HasInstance)
    .staticmethod("HasInstance")
    .def("Instance", &Core::CoreServer::Instance, bp::return_value_policy<bp::return_by_smart_ptr>())
    .staticmethod("Instance")
    .add_property("AppName",
        bp::make_function(&Core::CoreServer::GetAppName, bp::return_value_policy<bp::return_by_value>()),
        bp::make_function(&Core::CoreServer::SetAppName)
    ))
    // etc.
    ;

And then use it in Python:

>>> from pynebula3.Core import CoreServer
# singleton instance is created and stored in a new Ptr<CoreServer> instance
# which is in turn embedded in a new Python object
>>> coreServer = CoreServer.Create()
>>> print coreServer
<pynebula3.Core.CoreServer object at 0x00AB1340>
# existing singleton instance is stored in a new Ptr<CoreServer> instance
# which is embedded in a new Python object
>>> coreServer2 = CoreServer.Instance()
>>> print coreServer2
<pynebula3.Core.CoreServer object at 0x00AB1378>
>>> print coreServer2.AppName
Nebula3
# the Ptr<CoreServer> instance embedded in the Python object is deleted,
# the singleton instance itself stays alive because another Ptr<CoreServer>
# instance still references it
>>> coreServer = None
>>> print coreServer2
<pynebula3.Core.CoreServer object at 0x00AB1378>
>>> print coreServer2.AppName
Nebula3
# the remaining Ptr<CoreServer> instance embedded in the Python object is deleted,
# since no other Ptr<CoreServer> instances reference the singleton instance it
# too is deleted
>>> coreServer2 = None

And that's all there is to it, though it remains to be seen how well it all works in practice.