Ways To Save Files

January 26, 2021

Saving the player’s progress is a small, yet crucial part of game dev that is often taken seriously too late into a project’s development. Your save system should be well thought out as early as possible as it can shape the way your game works in subtle ways, and can speed up testing throughout production.

Getting the Data

How do you gather all of the data that you want to save? There’s a few ways to go about this.

The most obvious is to have the saving system overviewing everything else, so it will pick only the data it needs by direct reference. This is fine for a smaller game, but you will run into trouble when the data it needs is all over the code base. Besides, the save system shouldn’t have access to every other system in the game; it’s violating every good design rule in the book. Another downside is that if you ever want to add or remove data from the file’s layout, the old file will be incompatible. This can be rectified with a file version and lots of if statements, but it will get very tedious if you’re making lots of changes.

An alternative solution is serialization. Serialization is when binary data in memory is converted in a way that it can be saved to a file. This will involve the save system passing a serializer to the rest of the game which can then serialize itself. That way the save system doesn’t need to know what needs to be saved or how it should be saved; the individual systems can take care of themselves! On the down side, this will require the individual systems to deserialize themselves too and, once again, making any changes to the file’s layout will render it unreadable, but this way it will be up to the subsystems to version themselves.

Another solution is using a generic hierarchical key-value pair structure. Think of JSON or XML where any primitive types can be stored. The data is not expected to be in any particular layout, so versioning is not so much of an issue (especially if you supply default values when fetching from the file). So saving could look like this:

1

file.SetInt("playerLevel", level);

And reading could then be

1

int level = file.GetInt("playerLevel", 1);

In this case, 1 is the default value if playerLevel doesn’t currently exist.

Choosing Files

If you’re not rolling your own file type, don’t hesitate to use an existing generic data file solution. The main ones are:

JSON (lean and simple rules to understand, lots of tools available)
BSON (binary version of JSON, quite a bit more compressed)
YAML (more readable for humans than json and can have more complex rules)
XML (a bit verbose but lets you contain multiline data between tags)
CSV (only really good when data has consistent number of columns, but is super slim and easy to parse)

Compression

The size of save files vary greatly. Some games only need a few bytes to know what stage the player was up to. Others need gigabytes to remember the state of every object the player has ever come across. In the latter case, you might want to look into compressing data. Not only does compression save in storage space, but it can also be faster to load up (since fetching data from hard drives can be insanely slow).

If your save files are in plain text (such as JSON, YAML and XML), putting your file through any basic encryption algorithm (LZF4, GZIP, LZMA) is going to yield great results. Compression algorithms on binary files are most likely going to reduce the size, but may not be as impressive.

If your save files are really bloated, often the best way to compress the data is to look at the data you’re trying to save and use the right tool for the job.

Saving an image? Don’t save it as text and don’t save it as a list of raw bytes. Compress it to a JPEG (lossy) or PNG (lossless) file, or even in your GPU format (DDS, PVRTC, ETC2, etc) for even faster loading.

Saving a huge series of jumbled bytes? Don’t compress it at all. Compression may not reduce the size and will only slow down loading.

Saving float/double values to a plain text file? Round the numbers to a specific decimal place. Often the numbers will blow out to something ridiculous like 1.00000003 when it could just be 1, or -547.2374563 can be -547.24. It all comes down to how much precision you need.

If you’re writing your own binary file, you can do other smaller optimisations:

Convert quaternion rotations (4 numbers) to euler rotations (3 numbers).
For transform matrices, save the position rotation and scale (9 numbers) instead of the full matrix (16 numbers)
Convert angles to ints (4 bytes to 1 byte)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


byte CompressAngle(float angle)
{
	angle %= 360;
	if (angle < 0)
		angle += 360;
	return (byte)(255 * angle / 360);
}

float DecompressAngle(byte compressedAngle)
{
	return (float)compressedAngle * 360 / 255;
}

If position coordinates can only exist within a space, consider using normalized coordinates within a bounding box (12 bytes to 6 bytes)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


CompressedPosition CompressBoundedPosition(Vector3 position, Bounds bounds)
{
	const int ushortMax = 65535;
	return new CompressedPosition()
	{
		x = (ushort)(Mathf.Clamp01(position.x - bounds.min.x) / (bounds.max.x - bounds.min.x)) * ushortMax),
		y = (ushort)(Mathf.Clamp01(position.y - bounds.min.y) / (bounds.max.y - bounds.min.y)) * ushortMax),
		z = (ushort)(Mathf.Clamp01(position.z - bounds.min.z) / (bounds.max.z - bounds.min.z)) * ushortMax)
	};
}

Vector3 DecompressBoundedPosition(CompressedPosition position, Bounds bounds)
{
	const int ushortMax = 65535;
	return new Vector3()
	{
		x = position.x = bounds.min.x + ((float)position.x / ushortMax) * (bounds.max.x - bounds.min.x),
		y = position.x = bounds.min.y + ((float)position.y / ushortMax) * (bounds.max.y - bounds.min.y),
		z = position.x = bounds.min.z + ((float)position.z / ushortMax) * (bounds.max.z - bounds.min.z)
	};
}

If you have a list of boolean values, combine them all into a bitset (1 byte to 1/8 bytes)
When saving colors, save as bytes instead of floats (12 bytes to 4 bytes), don’t save the alpha if you don’t need to (4 bytes to 3 bytes), and compress to R5G6B5 where you can (3 bytes to 2 bytes).

Encryption

If you’re making an offline or single player game, forget it. You don’t need encryption. Seriously. If the file is on the player’s local hard drive, they can always find a way to unlock it’s contents. If the player wants to change the contents of their save file then just let them. It may ruin their experience of the game, but it won’t affect other players so it’s fine. Some games even provide tools to let the players modify their save files, allowing them full access to items, abilities, stats and more.

So what about online multiplayer games? Should it be encrypted? If you have a single player component to your game, save that part of the game locally and read the previous paragraph again. For any multiplayer components, it needs to be saved to the cloud. Whether the data is encrypted or not is irrelevant because it will be on the developer’s server and not accessible by the player. Whenever the player needs access to multiplayer data, it should be done through online calls. Do not save any multiplayer data locally!

Archiving

Sometimes one file for saving isn’t enough. You may find that you need a JSON file, PNGs, binary data and an audio file. Sure, you could stick all of this in a horribly-optimised JSON file, but then you will have slow load times, excess memory usage, and a bulky save file. This is a perfect use case for archiving.

Archiving is simply sticking a bunch of files into a single file. Think of a zip file. You can grab any file type and compress it down to a single file. Zip is great at this, yet it’s not perfect for our needs. Zip requires a large chunk of the file to be decompressed in order to get anything out of it. We want to leave the compression up to the individual files and just pull out what we need as fast as possible.

The basic idea is simple. You have a header which is a table of contents, and then you have the data. The header contains a list of the files (the name of the file, the start position in the file, and the file size). With this, you can jump to exactly where the file is and read all of the data you need without touching anything else.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


0x00000000 Header:
	file1.txt @ 0x00004000
	file2.wav @ 0x00007000
	file3.png @ 0x00012000
0x00004000 file1.txt:
	...
0x00007000 file2.wav:
	...
0x00012000 file3.png:
	...

This is super fast, allows you to keep per-file compression, and elegantly keep all data together in a single file.

Reducing Load Times

If you have large save files, load times can be brutal, but there are a few things you can do to mitigate it.

The first thing to look into is reducing the data that you’re saving. Do you need to store the position of every cloud, or can you just save a seed or time and generate it from that?

Do you need to load everything at once? If you are saving map data, why not just load up the data for the current map and leave the rest until you actually need it. This means that your save file may need to be split up into separate files. Do not use zip or some other compression archiving system for this as all of the files will need to be decompressed just to pull out a single file. Instead, compress each file individually and archive them all in the same file.

Archiving all of your files will also decrease load times as it will be jumping around less on the hard drive. Load the archive table of contents into memory and then seek to where you need when you need it.