Research on Persistence

The past week I have been doing preliminary research on persistence in the context of game development in preparation to developing a standard system to handle this.

I have scoured the web and found quite a few sources on this topic, though most of them have not been very useful. Most questions and responses in the Unity community focus on the smaller problems like serialization. There is very little deeper discussion on the design of systems that should handle this. The same goes for most game development forums. It is commonly said that every game has different needs and thus persistence should be custom-built for every game. I am not satisfied with that, because I feel that most of them deal with fundamentally the same problems, just in different situations. In my opinion, a lot of this could be generalized.

The problem of persistence has been largely solved in enterprise software. There are advanced databases and other data stores available, with abstraction layers to ease development. Application functionality is divided into domain models for persistence and other parts for the rest of the functionality. This makes persistence a trivial problem for most software engineers.

This is usually not the case in game development. In my experience a lot of games are developed more in an ad-hock fashion than most  applications. For reasons of performance and ease of use, the data we want to save is often embedded in the scene in ways that make it difficult to get it out if persistence was an afterthought. Any Unity developer who has ever tried to save and restore entire game object hierarchies at run-time will know the difficulties I am talking about. Whether these designs are wise is another matter I will not go into here. In addition to this, there is the fact that different games use different storage methods. Some might use a save file on a local hard disk while others will save state to a web service or database. All these circumstances make persistence for games a touchy subject.

I took another close look at all the resources I had gathered and noted that the entire problem of persistence in game design can be summarized in a few sub-problems:

1. Extraction: Separating the data you want to persist from the rest of the game
2. Serialization: Encoding the data into a sequential format suitable for storage and back
3. Storage: Actually storing and retrieving the serialized data
4. Reconstruction: Reconstructing the scene or game state from loaded data

In a lot of situations, these problems might overlap or even blur as to be almost indistinguishable from each other. At all times every one of these problems is present in some way. Most resources I have found focused on serialization and storage. The more difficult problems are actually extraction and reconstruction, but often overlooked, which often causes problems in serialization.

For each of these problems I have found a number of ways to deal with the problem. I will choose the most promising solutions I have found and integrate these into my first prototype.

Internship at Little Chicken

Soon after the jam I started on my last semester of my studies and my planned internship at Little Chicken Game Company in Amsterdam. Among other work, I will be doing research on and developing a standard, flexible, and easy to use system to handle persistence in games, in the Unity environment in particular.

Goals of this project:

1. Standard system: Take care of all the little details and make the game developer’s life easier when dealing with persistence in games.
2. Generic: Relevant and usable in as wide variety of use cases and types of games as practically possible, though there will always be exceptions.
3. Simplicity: Adding extra types of persistent game elements should be intuitive and involve minimal effort, and minimal intrusion into the rest of a project’s code base.
4. Flexibility: Easily adapted to situations requiring different forms of serialization or data storage.

The past weeks I have mostly been getting acquainted with the environment, setting up my workstation, and writing my project initiation document. I have also poked around at the existing code base to get familiar with the current project my solution will be integrated with.

I plan to do some preliminary research on my topic of persistence in games, after which I will develop a number of prototypes. For each prototype I will evaluate its strong points and shortcomings. This way I can iterate towards the final solution.

Global Game Jam

A lot has been going on these past few months for me. I am sad to say this blog has been neglected a bit, though that will change.

In Januari I participated in the Global Game Jam, in location Hilversum. Me and my team made a very nice and simple game which was received very well by the audience and which won third prize in the popularity contest and first prize by the jury. Me and my team are now looking at options to develop this further. Jamming is a great experience and I recommend it to every game developer.

Download