Category: Personal

Posted 2017-09-12Technology / Personal32 minutes read (About 4872 words)

Introduction

One of my brothers gifted Terraria to me on Steam. Needless to say, instead of doing the things that I ought to be doing; I have been instead playing it far more frequently. At least until my world save became corrupted.

While I was playing, Terraria suddenly crashed. I figure it was in the middle of a save/backup because when I tried to load the game back up Terraria informed me that my save was corrupted (I am not sure what caused it either). I shrugged it off thinking “No big deal, I will just use the backup file.” As it turned out, the backup was not as recent as I would have liked.

I had just gotten through a particularly nasty section of sand and had no desire to repeat it. Logically, the next thing to try was to open the world save in TEdit to see if there was anything salvageable. To my surprise, I opened the world save with TEdit and was informed that TEdit could try to recover it. Cool! After it loaded the map, nothing seemed out of place, so I saved it and went on with my business of exploring.

It was not until I had an inventory full of items that I realized what TEdit had not been able to recover - chest data. All the chests in my player home were now empty. Curious, I checked other chests using TEdit. All of the chests I checked were empty too! I would later find out that about 100 of the chests were now empty.

That would not do, that was about a third of the chests in my world. Half the fun of exploring and finding a chest is the goodies you get inside it. The only option I could think of at the time was to create a new world, maybe even using the same world seed to get the same map. However, I was not sure (still am not sure either) whether chest data is randomly generated, which could mean that even using the same world seed would not result in the same items. To top it off, it would have been easier to go through the section of sand that I was trying to avoid. So the only real option was to see if I could repair the world save to get the chest items back.

Fortunately, I had been backing my save files up to Google Drive. Without these backups, there would have been no way to restore the chest data. That does not mean it was a breeze though, there were a few hiccups along the way; most of them involving the sheer amount of data included in the world save. In the end, I was able to restore my game save to a state that I consider to be close enough to where it was that I do not miss anything. Probably, I am still missing a few items, but nothing I have noticed so I do not feel like I lost anything.

Researching the Terraria Source Code

The first step of the process required parsing world save data. I had to find the chest data in the good backup save to get the list of items that I was missing. Luckily, the Terraria client is fairly easy to decompile (NOTE: The decompiled source code will not run without modifications). There could be legal implications for decompiling the Terraria client - I do not know, I did not read the End User License Agreement. Instead, I used a repository that someone else had posted from dotPeek. Using the decompiled source code I could replicate how the game client reads the world save data and compare the chests from two of my world saves.

The code I was looking for is located in Terraria/IO/WorldFile.cs and begins in loadWorld. loadWorld does some date checking for special events, checks if the file exists, and then reads some data to determine how to parse the rest of the data. Depending on this value, the code is directed to either WorldFile.LoadWorld_Version1 or WorldFile.LoadWorld_Version2, since I know that my world file is fairly recent I immediately continued to WorldFile.LoadWorld_Version2.

`WorldFile.LoadWorld_Version2`

public static int LoadWorld_Version2(BinaryReader reader)
{
  reader.BaseStream.Position = 0L;
  bool[] importance;
  int[] positions;
  if (!WorldFile
        .LoadFileFormatHeader(reader, out importance, out positions)
    || reader.BaseStream.Position != (long) positions[0])
    return 5;
  WorldFile.LoadHeader(reader);
  if (reader.BaseStream.Position != (long) positions[1])
    return 5;
  WorldFile.LoadWorldTiles(reader, importance);
  if (reader.BaseStream.Position != (long) positions[2])
    return 5;
  WorldFile.LoadChests(reader);
  if (reader.BaseStream.Position != (long) positions[3])
    return 5;
  WorldFile.LoadSigns(reader);
  if (reader.BaseStream.Position != (long) positions[4])
    return 5;
  WorldFile.LoadNPCs(reader);
  if (reader.BaseStream.Position != (long) positions[5])
    return 5;
  if (WorldFile.versionNumber >= 116)
  {
    if (WorldFile.versionNumber < 122)
    {
      WorldFile.LoadDummies(reader);
      if (reader.BaseStream.Position != (long) positions[6])
        return 5;
    }
    else
    {
      WorldFile.LoadTileEntities(reader);
      if (reader.BaseStream.Position != (long) positions[6])
        return 5;
    }
  }
  if (WorldFile.versionNumber >= 170)
  {
    WorldFile.LoadWeightedPressurePlates(reader);
    if (reader.BaseStream.Position != (long) positions[7])
      return 5;
  }
  if (WorldFile.versionNumber >= 189)
  {
    WorldFile.LoadTownManager(reader);
    if (reader.BaseStream.Position != (long) positions[8])
      return 5;
  }
  return WorldFile.LoadFooter(reader);
}

The WorldFile.LoadWorld_Version2 function provided me with a layout of the different sections in a world save. The data appeared to be broken out into the following sections which are read sequentially from the world save:

File Format Header
Header
World Tiles
Chests
Signs
NPCs
Tile Entities
Weighted Pressure Plates
Town Manager
Footer

Wow! That is more data than I thought there would be. It appears as though after every section a check is done to verify that the current position in the data file matches a position that is read from the WorldFile.LoadFileFormatHeader. I should be able to use the position at the corresponding index to jump directly to the Chest data section.

`WorldFile.LoadFileFormatHeader`

private static bool LoadFileFormatHeader(
    BinaryReader reader,
    out bool[] importance,
    out int[] positions)
{
  importance = (bool[]) null;
  positions = (int[]) null;
  if ((WorldFile.versionNumber = reader.ReadInt32()) >= 135)
  {
    try
    {
      Main.WorldFileMetadata =
        FileMetadata.Read(reader, FileType.World);
    }
    catch (FileFormatException ex)
    {
      Console.WriteLine(
        Language.GetTextValue("Error.UnableToLoadWorld"));
      Console.WriteLine((object) ex);
      return false;
    }
  }
  else
    Main.WorldFileMetadata =
        FileMetadata.FromCurrentSettings(FileType.World);
  short num1 = reader.ReadInt16();
  positions = new int[(int) num1];
  for (int index = 0; index < (int) num1; ++index)
    positions[index] = reader.ReadInt32();
  short num2 = reader.ReadInt16();
  importance = new bool[(int) num2];
  byte num3 = 0;
  byte num4 = 128;
  for (int index = 0; index < (int) num2; ++index)
  {
    if ((int) num4 == 128)
    {
      num3 = reader.ReadByte();
      num4 = (byte) 1;
    }
    else
      num4 <<= 1;
    if (((int) num3 & (int) num4) == (int) num4)
      importance[index] = true;
  }
  return true;
}

The WorldFile.LoadFileFormatHeader is responsible for reading the WorldFileMetadata, the sections, and an array of items known as ‘importance’. Each section position is represented as a single integer value. I am familiar with this kind of data storage technique since HTTP packets do something similar:

Using the section list from the WorldFile.LoadWorld_Version2 and the position data from the WorldFile.LoadFileFormatHeader, I could read the Chest data immediately by jumping to that position in the save file. The next question was to determine how the Chest data was stored.

`WorldFile.LoadChests`

private static void LoadChests(BinaryReader reader)
{
  int num1 = (int) reader.ReadInt16();
  int num2 = (int) reader.ReadInt16();
  int num3;
  int num4;
  if (num2 < 40)
  {
    num3 = num2;
    num4 = 0;
  }
  else
  {
    num3 = 40;
    num4 = num2 - 40;
  }
  int index1;
  for (index1 = 0; index1 < num1; ++index1)
  {
    Chest chest = new Chest(false);
    chest.x = reader.ReadInt32();
    chest.y = reader.ReadInt32();
    chest.name = reader.ReadString();
    for (int index2 = 0; index2 < num3; ++index2)
    {
      short num5 = reader.ReadInt16();
      Item obj = new Item();
      if ((int) num5 > 0)
      {
        obj.netDefaults(reader.ReadInt32());
        obj.stack = (int) num5;
        obj.Prefix((int) reader.ReadByte());
      }
      else if ((int) num5 < 0)
      {
        obj.netDefaults(reader.ReadInt32());
        obj.Prefix((int) reader.ReadByte());
        obj.stack = 1;
      }
      chest.item[index2] = obj;
    }
    for (int index2 = 0; index2 < num4; ++index2)
    {
      if ((int) reader.ReadInt16() > 0)
      {
        reader.ReadInt32();
        int num5 = (int) reader.ReadByte();
      }
    }
    Main.chest[index1] = chest;
  }
  List<Point16> point16List = new List<Point16>();
  for (int index2 = 0; index2 < index1; ++index2)
  {
    if (Main.chest[index2] != null)
    {
      Point16 point16 =
        new Point16(
            Main.chest[index2].x, Main.chest[index2].y);
      if (point16List.Contains(point16))
        Main.chest[index2] = (Chest) null;
      else
        point16List.Add(point16);
    }
  }
  for (; index1 < 1000; ++index1)
    Main.chest[index1] = (Chest) null;
  if (WorldFile.versionNumber >= 115)
    return;
  WorldFile.FixDresserChests();
}

Not quite as straight-forward to decipher, but still doable:

num1 represents the number of chests stored in the world.
num2 represents the number of items stored in the chest.
num3 represents the items the chest is holding.
num4 represents the overflow of items if the chest had more items than the maximum.

Each chest then has the following properties:

x represents the x-coordinate of the chest in the world.
y represents the y-coordinate of the chest in the world.
name represents the name of the chest in the world.

I found it odd that the Chest data did not have an Id property that could be used to identify specific chests. Though the x and y properties are probably sufficient in most cases, it just meant that I would have to be more careful about identifying chests.

Depending on the next value a chest can have between 0 and 40 items that have the following properties:

id represents the item id
stack represents the quantity of that item in the single item slot.
prefix represents a prefix value that affects the stats on the item.

The code deviates a little from the normal layout with items. Instead of having an Int16 represent the number of items in a chest, each slot is read. If the slot is empty a zero will be placed in that Item data location and the code must account for that. This is caused by items in chests being located anywhere within the 40 slots. By storing the data this way, the game can reduce the size of the world save file.

After reviewing WorldFile.LoadChests, I had enough information to parse the chest data.

Applying the Research

In an effort NOT to confuse myself, I tried to break the sections out a little further into methods. The code was written using LINQPad to rapidly develop the prototype for reading the data. Main is the entry point for the ‘script’. The final code can be found in Appendix A.

void Main()
{
    World GoodWorld = GetWorld(@"201708310826.wld");
}

World GetWorld(string worldPath)
{
    using (BinaryReader worldReader =
        new BinaryReader(File.OpenRead(worldPath)))
    {
        return GetWorld(worldReader);
    }
}

GetWorld takes a string file path where the world file is located. This would allow me to add a single line to get the data from BadWorld once I was satisfied that the current implementation worked as intended. Once the file was opened for reading, the data needed to be read:

World GetWorld(BinaryReader worldReader)
{
    World world = new World();

    GetWorldMetaData(worldReader, world);
    GetWorldSections(worldReader, world);

    worldReader.BaseStream.Position = world.SectionSize[2];

    GetChestData(worldReader, world);

    return world;
}

The File MetaData is read to get the BinaryReader to the correct position. This could be done with math - 2x Int32 (8 bytes), 2x Int64 (16 bytes) would yield 24 bytes. However, to maintain readability instead of jumping to an arbitrary location, reading the MetaData seemed more appropriate. Especially since there is an unknown value and the structure of this location in the file could be updated or changed between versions. An exception to this ‘rule’ is made once the section data has been read since it is more obvious why and where the jump is occurring:

void GetWorldMetaData(BinaryReader worldReader, World world)
{
    world.Version = worldReader.ReadInt32();
    world.TypeCheck = worldReader.ReadInt64();
    world.Revision = worldReader.ReadInt32();
    // Unknown
    world.UnknownMetaData = worldReader.ReadInt64();
}

Once the MetaData has been read, the Sections can be read:

void GetWorldSections(BinaryReader worldReader, World world)
{
    world.SectionCount = worldReader.ReadInt16();

    for(int section = 0; section < world.SectionCount; section++)
    {
        world.SectionSize[section] = worldReader.ReadInt32();
    }
}

After jumping to the corresponding location in the BinaryReader, the Chest data is read:

void GetChestData(BinaryReader worldReader, World world)
{
    world.TotalChests = worldReader.ReadInt16();
    world.MaxItems = worldReader.ReadInt16();

    int itemsPerChest = world.MaxItems;
    int overflowItems = 0;

    if (world.MaxItems > 40)
    {
        itemsPerChest = 40;
        overflowItems = world.MaxItems - 40;
    }

    for (int i = 0; i < world.TotalChests; i++)
    {
        world.ChestCollection[i] =
            GetChestData(worldReader, itemsPerChest, overflowItems);
    }
}

Chest GetChestData(
    BinaryReader worldReader,
    int itemsPerChest,
    int overflowItems)
{
    Chest chest = new Chest(itemsPerChest)
        {
            X = worldReader.ReadInt32(),
            Y = worldReader.ReadInt32(),
            Name = worldReader.ReadString(),
        };

    for (int i = 0; i < itemsPerChest; i++)
    {
        chest.ItemCollection[i] = GetItemData(worldReader);
    }

    for (int i = 0; i < overflowItems; i++)
    {
        GetItemData(worldReader);
    }

    return chest;
}

Each chest can contain one or more items up to the max item. For each chest, the items are parsed in a separate method:

Item GetItemData(BinaryReader worldReader)
{
    short stackSize = worldReader.ReadInt16();

    if (stackSize <= 0)
    {
        return null;
    }

    return new Item()
        {
            StackSize = stackSize,
            Id = worldReader.ReadInt32(),
            Prefix = worldReader.ReadByte(),
        };
}

This yields the chest data in the world. Add a line to parse the BadWorld and verify that there are discrepancies between the two:

void Main()
{
    World BadWorld = GetWorld(@"201709010835.wld");
    Console.WriteLine(
        "{0} Empty Chests in Bad World out of {1}",
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .Count(),
        BadWorld.ChestCollection.Length);

    MergeEmptyChests(
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .ToArray(),
        GoodWorld.ChestCollection);
}

Which yields the following output:

13 Empty Chests in Good World out of 302 145 Empty Chests in Bad World out of 302

That seems to match what I was seeing. Although I was a little surprised there were empty chests in the good world. It turned out that the empty chests in the good world came from the Hell layer and one in my player home that I was not utilizing.

The next step is to try to compare the chests in to the two worlds to see if they can be matched. If they can, a repair is possible.

Comparing Chest Data

Unfortunately, there is no way to prevent having to iterate two lists in order to compare. However, the size of the lists can be reduced by retrieving the empty chests from the bad world. It can be reduced even further by excluding chests contained within player housing. I recommend doing this because these chests have been touched by player(s) which I was more comfortable doing by hand with TEdit. It probably would not have made any difference though:

void Main()
{
    World BadWorld = GetWorld(@"201709010835.wld");
    Console.WriteLine(
        "{0} Empty Chests in Bad World out of {1}",
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .Count(),
        BadWorld.ChestCollection.Length);

    MergeEmptyChests(
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .ToArray(),
        GoodWorld.ChestCollection);

    MergeEmptyChests(
        GoodWorld.ChestCollection,
        BadWorld.ChestCollection
            .Where(chest =>
                !(chest.X > 3100
                    && chest.X < 3125
                    && chest.Y > 335
                    && chest.Y < 340)
                && chest.ItemCollection.All(
                    item => item == null))
            .ToArray());

    Console.WriteLine();
    Console.WriteLine("After Merge:");
    Console.WriteLine(
        "{0} Empty Chests in Bad World out of {1}",
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .Count(),
        BadWorld.ChestCollection.Length);
}

Then each Chest from the bad world is identified and the items it should contain are added from the matching Chest in the good world.

void MergeEmptyChests(
    Chest[] sourceChests,
    Chest[] destinationChests)
{
    foreach (Chest destinationChest in destinationChests)
    {
        foreach (Chest sourceChest in sourceChests)
        {
            if (destinationChest.X != sourceChest.X
                && destinationChest.Y != sourceChest.Y)
            {
                continue;
            }

            int sourceChestItemLength =
                sourceChest.ItemCollection.Length;

            for (int i = 0; i < sourceChestItemLength; i++)
            {
                destinationChest.ItemCollection[i] =
                    sourceChest.ItemCollection[i];
            }
        }
    }
}

The only way to identify chests in the world is based on their location. Name possibly could be used - except named chests likely mean that the player has placed these chests intentionally. As previously stated, any chests that I had placed I wanted to do manually as I needed to compare with the chests I had placed items in after I noticed the save file corruption.

After running the application I get the following output:

13 Empty Chests in Good World out of 302 145 Empty Chests in Bad World out of 302
After Merge: 13 Empty Chests in Bad World out of 302

Perfect, I was able to restore all of the wild chests back to their generated state. The last step in the process is saving the changes.

Saving Chest Data

Everything up to this point had been relatively easy, so it was only a matter of time before I came across an issue. Unfortunately, the issue did not appear until I got to the final step of saving the merged Chest data.

The Issue

Remember how Chest data is read? Or more specifically, how the item data is read?

Chest GetChestData(
    BinaryReader worldReader,
    int itemsPerChest,
    int overflowItems)
{
    Chest chest = new Chest(itemsPerChest)
        {
            X = worldReader.ReadInt32(),
            Y = worldReader.ReadInt32(),
            Name = worldReader.ReadString(),
        };

    for (int i = 0; i < itemsPerChest; i++)
    {
        chest.ItemCollection[i] = GetItemData(worldReader);
    }

    for (int i = 0; i < overflowItems; i++)
    {
        GetItemData(worldReader);
    }

    return chest;
}

Item GetItemData(BinaryReader worldReader)
{
    short stackSize = worldReader.ReadInt16();

    if (stackSize <= 0)
    {
        return null;
    }

    return new Item()
        {
            StackSize = stackSize,
            Id = worldReader.ReadInt32(),
            Prefix = worldReader.ReadByte(),
        };
}

Do you see the issue?

The issue is that the Chest data has grown after the merge since items have been added to a Chest that require data representations for a stack size, an id, and (potentially?) a prefix. This made my plan to just overwrite the data in the Chest data section impossible because it overwrites the data in the next section and consequentially causes the section locations to be incorrect. I was able to come up with only two ways to solve this:

Parse Entire WorldFile
Partition WorldFile

Parse Entire WorldFile

The first solution was to add support for parsing the entire WorldFile save into the object that is being manipulated. This would prevent any data from getting overwritten but would still require the section data to be updated. Either through mathematically calculating the size of each section given the size of each representation of the data in those sections, or by updating the section positions after each section was written.

While this is probably a more robust approach in the long run, it would require a lot of effort on my part to build all of the representations for the data in each section and the corresponding code to read it.

Partition WorldFile

The second solution would segment the file into ~5 partitions. Three partitions for the data that is not changing, and two partitions for the data that needs updated (section locations and Chest data). In this way the data is accounted for without actually defining a representation for each piece of the data like the previous solution would have had to. Keep in mind that this solution is only worthwhile because the data changes that were made made were local to the Chest data section. If this were not the case, 2n+1 (where n is the number of sections) would need to be updated:

Partition Name	Partition Type
MetaData	Original
Section Location	Modified
2x Sections	Original
Chest Data	Modified
Remaining Sections	Original

Solution Implementation

I chose the second solution as it seemed like the easier of the two for what I was trying to accomplish. If this wasn’t a one-off project, I would suggest the first solution. To implement the second solution, all of the data from the file needs to be read. Fortunately, the data that I am not interested in can be read into a byte array. Then during the save process, the original data can be preserved by writing the contents of these byte arrays. The GetWorld function then gets updated to look like this:

World GetWorld(BinaryReader worldReader)
{
    World world = new World();

    GetWorldMetaData(worldReader, world);
    GetWorldSections(worldReader, world);

    int bytesToRead = world.SectionSize[2]
        - (int)worldReader.BaseStream.Position;

    world.SkippedSectionsBeforeChestData =
        worldReader.ReadBytes(bytesToRead);

    GetChestData(worldReader, world);

    bytesToRead = (int)worldReader.BaseStream.Length
        - (int)worldReader.BaseStream.Position;

    world.SkippedSectionsAfterChestData =
        worldReader.ReadBytes(bytesToRead);

    return world;
}

Now that I had all of the file data in a mechanism that prevents data loss, I could write the function to save the world file and call it from Main after the Merge.

void Main()
{
    World GoodWorld = GetWorld(@"201708310826.wld");
    Console.WriteLine(
        "{0} Empty Chests in Good World out of {1}",
        GoodWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .Count(),
        GoodWorld.ChestCollection.Length);

    World BadWorld = GetWorld(@"201709010835.wld");
    Console.WriteLine(
        "{0} Empty Chests in Bad World out of {1}",
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .Count(),
        BadWorld.ChestCollection.Length);

    MergeEmptyChests(
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .ToArray(),
        GoodWorld.ChestCollection);

    Console.WriteLine();
    Console.WriteLine("After Merge:");
    Console.WriteLine(
        "{0} Empty Chests in Bad World out of {1}",
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .Count(),
        BadWorld.ChestCollection.Length);

    SaveWorld(BadWorld, @"201709010835-modified.wld");
}

void SaveWorld(World world, string worldFileSave)
{
    using (BinaryWriter binaryWriter = new BinaryWriter(
        File.Create(worldFileSave)))
    {
        SaveWorld(binaryWriter, world);
    }
}

Just like before, this is just a utility method around the SaveWorld method that actually does all the work. The worldFileSave property is the location to save the data, I recommend making it different than the original file to prevent further data loss!

void SaveWorld(BinaryWriter worldWriter, World world)
{
    WriteWorldMetaData(worldWriter, world);

    long sectionPosition = worldWriter.BaseStream.Position;

    WriteWorldSections(worldWriter, world);

    worldWriter.Write(world.SkippedSectionsBeforeChestData);

    WriteChestData(worldWriter, world);

    long newChestDataSectionPosition =
        worldWriter.BaseStream.Position;

    worldWriter.Write(world.SkippedSectionsAfterChestData);

    int sectionSizeAdjustment =
        (int)newChestDataSectionPosition - world.SectionSize[3];
    for(int i = 3; i < world.SectionCount - 1; i++)
    {
        world.SectionSize[i] += sectionSizeAdjustment;
    }

    worldWriter.BaseStream.Position = sectionPosition;
    WriteWorldSections(worldWriter, world);
}

This method is the exact opposite of the GetWorld method with a few extra modifications. The sectionPosition is used at the end to update the section offsets after all the data has been written since the size of the data section will be known at that point. The newChestDataSectionPosition is the local variable used to update the current section data to the new updated values. The writer then moves back to the section offset location stored in sectionPosition and writes the new section data. Each write method defined in this method is the inverse of the corresponding get method. The BinaryWriter will try to pad some of the values which is why the data is converted to the proper type before writing. This could probably be corrected by changing the underlying value in the corresponding object to the exact value needed:

void WriteWorldMetaData(BinaryWriter worldWriter, World world)
{
    worldWriter.Write((Int32)world.Version);
    worldWriter.Write((Int64)world.TypeCheck);
    worldWriter.Write((Int32)world.Revision);
    worldWriter.Write((Int64)world.UnknownMetaData);
}

No scary code here.

void WriteWorldSections(BinaryWriter worldWriter, World world)
{
    worldWriter.Write((Int16)world.SectionCount);

    for(int section = 0; section < world.SectionCount; section++)
    {
        worldWriter.Write((Int32)world.SectionSize[section]);
    }
}

A little more complex, but still nothing to be concerned with.

void WriteChestData(BinaryWriter worldWriter, World world)
{
    worldWriter.Write((Int16)world.TotalChests);
    worldWriter.Write((Int16)world.MaxItems);

    for (int i = 0; i < world.TotalChests; i++)
    {
        WriteChestData(worldWriter, world.ChestCollection[i]);
    }
}

Similar to the WriteWorldSections but with an extra value written.

void WriteChestData(BinaryWriter worldWriter, Chest chest)
{
    worldWriter.Write((Int32)chest.X);
    worldWriter.Write((Int32)chest.Y);
    worldWriter.Write(chest.Name ?? string.Empty);

    for (int i = 0; i < chest.ItemCollection.Length; i++)
    {
        WriteItemData(worldWriter, chest.ItemCollection[i]);
    }
}

The complexity is starting to build. The empty string value may not be necessary, but I did not want to change a NullReferenceException.

void WriteItemData(BinaryWriter worldWriter, Item item)
{
    if (item == null)
    {
        worldWriter.Write((Int16)0);
        return;
    }

    worldWriter.Write((Int16)item.StackSize);

    if (item.StackSize == 0)
    {
        return;
    }

    worldWriter.Write((Int32)item.Id);
    worldWriter.Write((byte)item.Prefix);
}

The value of 0 needs to be explicitly cast down to Int16 as it is by default an Int32. This was causing my file to be larger than it needed to be at first and prevented it from being loaded.

Results

Once the file has been written to disk, it can be loaded in TEdit or Terraria to see if it can be parsed.

Alternate Solutions

Here are a couple alternative approaches to the one outlined above.

TEdit

It would have been faster to reference the TEdit executable and reuse the code they have written to parse the world file data for each instance of the save. This would have been like the first solution except I would be relying on TEdit’s implementation of it instead of rolling my own. The only code that would have needed to be written is MergeEmptyChests.

Terraria

Similar to the previous solution, it may be possible to reference the Terraria executable and reuse the code to load worlds. This would have made the first solution redundant and in hindsight is probably what I should have done in the first place. The save functionality probably does not have a Validate method check like TEdit does though.

Summary

In the end, was it worth it?

Probably not. Honestly, it would have been faster to go through the sand pit again than to research and code a solution like this. Granted, at the time I had no idea I had lost anything. At least I was able to recover from my mistake and learn something in the process.

Moral of the story? Back up your data.

Appendix A

void Main()
{
    World GoodWorld = GetWorld(@"201708310826.wld");
    Console.WriteLine(
        "{0} Empty Chests in Good World out of {1}",
        GoodWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .Count(),
        GoodWorld.ChestCollection.Length);

    World BadWorld = GetWorld(@"201709010835.wld");
    Console.WriteLine(
        "{0} Empty Chests in Bad World out of {1}",
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .Count(),
        BadWorld.ChestCollection.Length);

    MergeEmptyChests(
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .ToArray(),
        GoodWorld.ChestCollection);

    Console.WriteLine();
    Console.WriteLine("After Merge:");
    Console.WriteLine(
        "{0} Empty Chests in Bad World out of {1}",
        BadWorld.ChestCollection
            .Where(chest =>
                chest.ItemCollection
                    .All(item =>
                        item == null))
            .Count(),
        BadWorld.ChestCollection.Length);

    SaveWorld(BadWorld, @"201709010835-modified.wld");
}

World GetWorld(string worldPath)
{
    using (BinaryReader worldReader = new BinaryReader(
        File.OpenRead(worldPath)))
    {
        return GetWorld(worldReader);
    }
}

World GetWorld(BinaryReader worldReader)
{
    World world = new World();

    GetWorldMetaData(worldReader, world);
    GetWorldSections(worldReader, world);

    worldReader.BaseStream.Position = world.SectionSize[2];

    GetChestData(worldReader, world);

    return world;
}

void GetWorldMetaData(BinaryReader worldReader, World world)
{
    world.Version = worldReader.ReadInt32();
    world.TypeCheck = worldReader.ReadInt64();
    world.Revision = worldReader.ReadInt32();
    // Unknown
    world.UnknownMetaData = worldReader.ReadInt64();
}

void GetWorldSections(BinaryReader worldReader, World world)
{
    world.SectionCount = worldReader.ReadInt16();

    for(int section = 0; section < world.SectionCount; section++)
    {
        world.SectionSize[section] = worldReader.ReadInt32();
    }
}

void GetChestData(BinaryReader worldReader, World world)
{
    world.TotalChests = worldReader.ReadInt16();
    world.MaxItems = worldReader.ReadInt16();

    int itemsPerChest = world.MaxItems;
    int overflowItems = 0;

    if (world.MaxItems > 40)
    {
        itemsPerChest = 40;
        overflowItems = world.MaxItems - 40;
    }

    for (int i = 0; i < world.TotalChests; i++)
    {
        world.ChestCollection[i] =
            GetChestData(
                worldReader,
                itemsPerChest,
                overflowItems);
    }
}

Chest GetChestData(
    BinaryReader worldReader,
    int itemsPerChest,
    int overflowItems)
{
    Chest chest = new Chest(itemsPerChest)
        {
            X = worldReader.ReadInt32(),
            Y = worldReader.ReadInt32(),
            Name = worldReader.ReadString(),
        };

    for (int i = 0; i < itemsPerChest; i++)
    {
        chest.ItemCollection[i] = GetItemData(worldReader);
    }

    for (int i = 0; i < overflowItems; i++)
    {
        GetItemData(worldReader);
    }

    return chest;
}

Item GetItemData(BinaryReader worldReader)
{
    short stackSize = worldReader.ReadInt16();

    if (stackSize <= 0)
    {
        return null;
    }

    return new Item()
        {
            StackSize = stackSize,
            Id = worldReader.ReadInt32(),
            Prefix = worldReader.ReadByte(),
        };
}

void MergeEmptyChests(
    Chest[] destinationChests,
    Chest[] sourceChests)
{
    foreach (Chest destinationChest in destinationChests)
    {
        foreach (Chest sourceChest in sourceChests)
        {
            if (destinationChest.X != sourceChest.X
                && destinationChest.Y != sourceChest.Y)
            {
                continue;
            }

            int numberOfItems =
                sourceChest.ItemCollection.Length;
            for (int i = 0; i < numberOfItems; i++)
            {
                destinationChest.ItemCollection[i] =
                    sourceChest.ItemCollection[i];
            }
        }
    }
}

void SaveWorld(World world, string worldFileSave)
{
    using (BinaryWriter binaryWriter = new BinaryWriter(
        File.Create(worldFileSave)))
    {
        SaveWorld(binaryWriter, world);
    }
}

void SaveWorld(BinaryWriter worldWriter, World world)
{
    WriteWorldMetaData(worldWriter, world);

    long sectionPosition = worldWriter.BaseStream.Position;

    WriteWorldSections(worldWriter, world);

    worldWriter.Write(world.SkippedSectionsBeforeChestData);

    WriteChestData(worldWriter, world);

    long newChestDataSectionPosition =
        worldWriter.BaseStream.Position;

    worldWriter.Write(world.SkippedSectionsAfterChestData);

    int sectionSizeAdjustment =
        (int)newChestDataSectionPosition - world.SectionSize[3];

    for(int i = 3; i < world.SectionCount - 1; i++)
    {
        world.SectionSize[i] += sectionSizeAdjustment;
    }

    worldWriter.BaseStream.Position = sectionPosition;

    WriteWorldSections(worldWriter, world);
}

void WriteWorldMetaData(BinaryWriter worldWriter, World world)
{
    worldWriter.Write((Int32)world.Version);
    worldWriter.Write((Int64)world.TypeCheck);
    worldWriter.Write((Int32)world.Revision);
    worldWriter.Write((Int64)world.UnknownMetaData);
}

void WriteWorldSections(BinaryWriter worldWriter, World world)
{
    worldWriter.Write((Int16)world.SectionCount);

    for(int section = 0; section < world.SectionCount; section++)
    {
        worldWriter.Write((Int32)world.SectionSize[section]);
    }
}

void WriteChestData(BinaryWriter worldWriter, World world)
{
    worldWriter.Write((Int16)world.TotalChests);
    worldWriter.Write((Int16)world.MaxItems);

    for (int i = 0; i < world.TotalChests; i++)
    {
        WriteChestData(worldWriter, world.ChestCollection[i]);
    }
}

void WriteChestData(BinaryWriter worldWriter, Chest chest)
{
    worldWriter.Write((Int32)chest.X);
    worldWriter.Write((Int32)chest.Y);
    worldWriter.Write(chest.Name ?? string.Empty);

    for (int i = 0; i < chest.ItemCollection.Length; i++)
    {
        WriteItemData(worldWriter, chest.ItemCollection[i]);
    }
}

void WriteItemData(BinaryWriter worldWriter, Item item)
{
    if (item == null)
    {
        worldWriter.Write((Int16)0);
        return;
    }

    worldWriter.Write((Int16)item.StackSize);

    if (item.StackSize == 0)
    {
        return;
    }

    worldWriter.Write((Int32)item.Id);
    worldWriter.Write((byte)item.Prefix);
}

class World
{
    private short sectionCount = 0;

    private short totalChests = 0;

    public int Version { get; set; }

    public long TypeCheck { get; set; }

    public int Revision { get; set; }

    public long UnknownMetaData { get; set; }

    public short SectionCount
    {
        get
        {
            return this.sectionCount;
        }

        set
        {
            this.sectionCount = value;
            this.SectionSize = new int[this.sectionCount];
        }
    }

    public int[] SectionSize { get; private set; }

    public short TotalChests
    {
        get
        {
            return this.totalChests;
        }

        set
        {
            this.totalChests = value;
            this.ChestCollection = new Chest[this.totalChests];
        }
    }

    public Chest[] ChestCollection { get; private set; }

    public short MaxItems { get; set; }
}

class Chest
{
    public Chest(int numberOfItemsPerChest)
    {
        this.ItemCollection = new Item[numberOfItemsPerChest];
    }

    public int X { get; set; }

    public int Y { get; set; }

    public string Name { get; set; }

    public Item[] ItemCollection { get; private set; }
}

class Item
{
    public short StackSize { get; set; }

    public int Id { get; set; }

    public byte Prefix { get; set; }
}

Blog

Books

Pluralsight Courses

Programming Languages

Projects

Software Setup

Tech Setup

Work

Lacking

Introduction

Baseline

eBay

Forbes

CNN

Ads-Blocker

Setup

Verification

eBay Verification

Forbes Verification

CNN Verification

Ads-Blocker Verification

Conclusion

Introduction

Researching the Terraria Source Code

WorldFile.LoadWorld_Version2

WorldFile.LoadFileFormatHeader

WorldFile.LoadChests

Applying the Research

Comparing Chest Data

Saving Chest Data

The Issue

Parse Entire WorldFile

Partition WorldFile

Solution Implementation

Results

Alternate Solutions

TEdit

Terraria

Summary

Appendix A

Introduction

Three Reasons Why I Chose NodeJS

Getting A List of Frameworks

Six Reasons Why I Chose Hexo

1. Hexo Knows What It Is and What It Is Not

2. Hexo Keeps It Super Simple

3. Hexo Is Still Actively Maintained

4. Extensible

5. Cross-Platform Support and Baked In Package Manager

6. Pluralsight Author Endorsed

Conclusion

Recents

Archives

Categories

Tags

`WorldFile.LoadWorld_Version2`

`WorldFile.LoadFileFormatHeader`

`WorldFile.LoadChests`