Basic compile-time reflection

21 Jun 2019 - Simon Coenen - Reading time: 11 mins - Comments

When working on my game engine project, I always get distracted by new interesting things or thoughts I want to look into and one of them was reflection. Almost all commercial game engine have some kind of reflection that makes GUI editors and visual scripting possible. Unlike those engines, I didn’t look into full all-the-way reflection because I didn’t want to bloat my code with that and maintain it. I was mostly interested in very simple type reflection so before you start frowning when looking at the code, this is not meant to be a complete reflection system at all! This is quite basic but I found it to be an interesting use of compile-time expressions.

All the code can be found on GitHub. The relevant files to look at are:

What it does

I mainly started working on this out of curiosity and to see how far I could take it without having to modify loads of files everywhere. I can definitely take it much further but in its current state I can retrieve type information and create instances of classes using string hashes and object factories. Getting type information is compile-time without any runtime cost.

I currently use it for getting game object components, object instantiation, shader variable addressing and general dynamic casting.

Just these functions are already quite powerful especially for (de)serialization.

class Foo : public Object
{
    FLUX_OBJECT(Foo, Object)
};
RegisterFactory<Foo>(); // Register type to factory
Object* pObj = NewObject("Foo"); // Create instance using string
Object* pObj2 = NewObject(Foo::GetTypeStatic()); // Create instance using string hash
Foo* pFoo = DynamicCast<Foo>(pObj); // Free dynamic type casting
pFoo->IsTypeOf<Object>(); // == true
pFoo->GetTypeName(); // == "Foo"
pFoo->IsAbstract(); // == false

Compile-time string hashing

At the base of all this lies compile-time string hashing. This is done to get a unique (ideally) hash for every type name so that we can create a lookup table. By making use of constexpr, we can have expressions get evaluated at compile time. Combined with constexpr, we can create a constexpr FNV-1a stringhash function.

constexpr size_t val_const{ 0xcbf29ce484222325 };
constexpr size_t prime_const{ 0x100000001b3 };

constexpr size_t Hash(const char* const str, const size_t value) noexcept
{
    return (str[0] == '\0') ? value : Hash(&str[1], (value ^ size_t(str[0])) * prime_const);
}

Like the C++ documentation says, constexpr doesn’t necessarily force the code to be executed at compile time, it is just a suggestion.

From cppreference.com:

The constexpr specifier declares that it is possible to evaluate the value of the function or variable at compile time. Such variables and functions can then be used where only compile time constant expressions are allowed.

So we can’t be sure when the function we wrote is going to be executed at compile time. A way to actually enforce this is to make the value in which the return value will get stored also constexpr. That way, if the function can not be evaluated at compile-time, the compiler will output an error.

//constexpr function
constexpr StringHash Hash = StringHash::Hash("Foo");

//ERROR: function call must have a constant value
constexpr StringHash Hash = StringHash::NonConstexprHash("Foo");

A cool little feature in Visual Studio with constexpr values is that it shows the resulting value when you hover over it. However, I’ve noticed that it sometimes shows the wrong value.

The “Object” class

Each class that wants to “participate” is this reflection system will need to inherit from a base class Object and insert a macro in its class body. Much like UObject in Unreal Engine, it provides the implementation with a load of functionality like reflection. In this case, relevant to reflection, it has the following functions:

virtual StringHash GetType() const { return GetTypeInfoStatic()->GetType(); }
virtual const std::string& GetTypeName() const { return GetTypeInfoStatic()->GetTypeName(); }
virtual const TypeInfo* GetTypeInfo() const { return GetTypeInfoStatic(); }
static StringHash GetTypeStatic() { return GetTypeInfoStatic()->GetType(); }
static const std::string& GetTypeNameStatic() { return GetTypeInfoStatic()->GetTypeName(); }
static const TypeInfo* GetTypeInfoStatic() { static const TypeInfo typeInfoStatic("Object", nullptr); return &typeInfoStatic; }
bool IsTypeOf(StringHash type) const;

As you can see, there are two versions of each function: a virtual method and a static method. The virtual method gets overridden by each class that inherits from Object and the static method gets hidden also by the class that inherits from Object. This might seem like a chore to do for each class by this can easily be wrapped in a small macro, just like Unreal Engine has UOBJECT_BODY().

#define FLUX_OBJECT_BASE(typeName, baseTypeName, isAbstract) \
    private: \
        static constexpr TypeInfo TYPE_INFO = TypeInfo(#typeName, baseTypeName::GetTypeInfoStatic(), &typeName::CreateInstanceStatic, isAbstract); \
    public: \
        using ClassName = typeName; \
        using BaseClass = baseTypeName; \
        virtual StringHash GetType() const override { return GetTypeInfoStatic()->GetType(); } \
        virtual const char* GetTypeName() const override { return GetTypeInfoStatic()->GetTypeName(); } \
        virtual const TypeInfo* GetTypeInfo() const override { return GetTypeInfoStatic(); } \
        static constexpr StringHash GetTypeStatic() { return GetTypeInfoStatic()->GetType(); } \
        static constexpr const char* GetTypeNameStatic() { return GetTypeInfoStatic()->GetTypeName(); } \
        static constexpr const TypeInfo* GetTypeInfoStatic() { return &TYPE_INFO; } \
    private: \

This provides each class with the tools to get its type id and name either statically or with a reference to an instance. As you can see, the static functions can be used at compile-time.

Unreal Engine obviously has much more magic here. On top of that, UE4 has UnrealHeaderTool which will parse the file and actually generate extra code (which are the classname.generated.h files).

TypeInfo

One thing you might have seen in the code above, is the class TypeInfo. This class serves as the metadata for a class that describes the relation between the class and its parent and will provide us with the ability to create a simple DynamicCast function. There is exactly one TypeInfo for each class as it gets statically initialized in GetTypeInfoStatic(), it contains the type hash, name and a reference to the TypeInfo of the parent’s class.

Knowing what the parent class of an object is, allows us to create a simple DynamicCast by walking up the class hierarchy until we either hit the TypeInfo we’re looking for (success) or the parent TypeInfo is nullptr (fail).

bool TypeInfo::IsTypeOf(const StringHash& type) const
{
    const TypeInfo* pInfo = this;
    //Keep walking up until we find a match or hit a null parent (== Object)
    while (pInfo != nullptr)
    {
        if (type == pInfo->m_Type)
            return true;
        pInfo = pInfo->m_pBaseTypeInfo;
    }
    return false;
}

template<typename T>
T* DynamicCast(Object* pObject)
{
    if(pObject->IsTypeOf(T::GetTypeStatic()))
        return static_cast<T*>(pObject);
    return nullptr;
}

Object factories

One of the main reasons for a reflection system is making serialization and deserialization possible. Say, I want to serialize and then deserialize an object, the only way to do that without reflection, is having a huge if/else block checking what object it needs to create because in C++, you can’t just do something like object = new "Foo".

This is pretty awful and not really maintainable.

Object* pObj = nullptr;
const std::string typeName = deserializer.ReadString();
if(typeName == "Foo")
    pObj = new Foo();
else if(typeName == "Bar")
    pObj = new Bar();
else if(...)
    ...
else
    //Not handled??? PANIC!!

This is where object factories come in. Each object type has its own factory and get stored in a simple container to be able to associate the type hash with a factory. In my implementation, I just store a function pointer in each TypeInfo that creates an instance of that class. This is also nicely wrapped in that macro I’ve mentioned before:

static Object* CreateInstanceStatic(Context* pContext) { return static_cast<Object*>(new typeName(pContext)); }
static constexpr TypeInfo TYPE_INFO = TypeInfo(#typeName, baseTypeName::GetTypeInfoStatic(), &typeName::CreateInstanceStatic, isAbstract); \

With a full serialization system, each class would have either its own Serialize and Deserialize function so it can be constructed properly or there would be some other system that would be able to “inject” the data into the created objects.

An extra detail here to note is that I’ve made a difference between a regular class and an abstract class. This way we can get runtime errors when we’re trying to instantiate an instance of an abstract class.

Other uses of compile-time StringHash

Initially, I’ve worked on the compile time string hashing just for the purpose of basic reflection and “messing around”. Later, I realized there are lot more powerful things you can do with them. For example with shader variables, Before, I was addressing them using strings. It’s easy to make typo’s when using strings and using them makes renaming identifiers hard because there is no validation and this can cause some really annoying bugs. Using compile time hashes, I could improve the performance and prevent making typo’s by putting common shader variables names in an enum. This provides some kind of “safety net” and IntelliSense has some nice auto-complete now :smile:. An enum enforces compile-time constants so you can’t put strings in it. However, since our hashes are compile time, we can wrap the strings with StringHash as follows:

enum ShaderConstant
{
#define DEFINE_SHADER_PARAMETER(variableName, name) variableName = StringHash(name);
    DEFINE_SHADER_PARAMETER(cElapsedTime, "cElapsedTime");
    DEFINE_SHADER_PARAMETER(cDeltaTime, "cDeltaTime");
    DEFINE_SHADER_PARAMETER(cLights, "cLights");
#undef DEFINE_SHADER_PARAMETER
};

//This...
SetShaderVariable("cElapsedTime", deltaTime);
//Becomes this
SetShaderVariable(ShaderConstant::cElapsedTime, deltaTime);