One of my brothers gifted Terraria to me on Steam. Needless to say, instead of doing the things that I ought to be doing; I have been instead playing it far more frequently. At least until my world save became corrupted.
While I was playing, Terraria suddenly crashed. I figure it was in the middle of a save/backup because when I tried to load the game back up Terraria informed me that my save was corrupted (I am not sure what caused it either). I shrugged it off thinking “No big deal, I will just use the backup file.” As it turned out, the backup was not as recent as I would have liked.
I had just gotten through a particularly nasty section of sand and had no desire to repeat it. Logically, the next thing to try was to open the world save in TEdit to see if there was anything salvageable. To my surprise, I opened the world save with TEdit and was informed that TEdit could try to recover it. Cool! After it loaded the map, nothing seemed out of place, so I saved it and went on with my business of exploring.
It was not until I had an inventory full of items that I realized what TEdit had not been able to recover - chest data. All the chests in my player home were now empty. Curious, I checked other chests using TEdit. All of the chests I checked were empty too! I would later find out that about 100 of the chests were now empty.
That would not do, that was about a third of the chests in my world. Half the fun of exploring and finding a chest is the goodies you get inside it. The only option I could think of at the time was to create a new world, maybe even using the same world seed to get the same map. However, I was not sure (still am not sure either) whether chest data is randomly generated, which could mean that even using the same world seed would not result in the same items. To top it off, it would have been easier to go through the section of sand that I was trying to avoid. So the only real option was to see if I could repair the world save to get the chest items back.
Fortunately, I had been backing my save files up to Google Drive. Without these backups, there would have been no way to restore the chest data. That does not mean it was a breeze though, there were a few hiccups along the way; most of them involving the sheer amount of data included in the world save. In the end, I was able to restore my game save to a state that I consider to be close enough to where it was that I do not miss anything. Probably, I am still missing a few items, but nothing I have noticed so I do not feel like I lost anything.
Researching the Terraria Source Code
The first step of the process required parsing world save data. I had to find the chest data in the good backup save to get the list of items that I was missing. Luckily, the Terraria client is fairly easy to decompile (NOTE: The decompiled source code will not run without modifications). There could be legal implications for decompiling the Terraria client - I do not know, I did not read the End User License Agreement. Instead, I used a repository that someone else had posted from dotPeek. Using the decompiled source code I could replicate how the game client reads the world save data and compare the chests from two of my world saves.
The code I was looking for is located in Terraria/IO/WorldFile.cs and begins in loadWorld. loadWorld does some date checking for special events, checks if the file exists, and then reads some data to determine how to parse the rest of the data. Depending on this value, the code is directed to either WorldFile.LoadWorld_Version1 or WorldFile.LoadWorld_Version2, since I know that my world file is fairly recent I immediately continued to WorldFile.LoadWorld_Version2.
publicstaticintLoadWorld_Version2(BinaryReader reader) { reader.BaseStream.Position = 0L; bool[] importance; int[] positions; if (!WorldFile .LoadFileFormatHeader(reader, out importance, out positions) || reader.BaseStream.Position != (long) positions[0]) return5; WorldFile.LoadHeader(reader); if (reader.BaseStream.Position != (long) positions[1]) return5; WorldFile.LoadWorldTiles(reader, importance); if (reader.BaseStream.Position != (long) positions[2]) return5; WorldFile.LoadChests(reader); if (reader.BaseStream.Position != (long) positions[3]) return5; WorldFile.LoadSigns(reader); if (reader.BaseStream.Position != (long) positions[4]) return5; WorldFile.LoadNPCs(reader); if (reader.BaseStream.Position != (long) positions[5]) return5; if (WorldFile.versionNumber >= 116) { if (WorldFile.versionNumber < 122) { WorldFile.LoadDummies(reader); if (reader.BaseStream.Position != (long) positions[6]) return5; } else { WorldFile.LoadTileEntities(reader); if (reader.BaseStream.Position != (long) positions[6]) return5; } } if (WorldFile.versionNumber >= 170) { WorldFile.LoadWeightedPressurePlates(reader); if (reader.BaseStream.Position != (long) positions[7]) return5; } if (WorldFile.versionNumber >= 189) { WorldFile.LoadTownManager(reader); if (reader.BaseStream.Position != (long) positions[8]) return5; } return WorldFile.LoadFooter(reader); }
The WorldFile.LoadWorld_Version2 function provided me with a layout of the different sections in a world save. The data appeared to be broken out into the following sections which are read sequentially from the world save:
File Format Header
Header
World Tiles
Chests
Signs
NPCs
Tile Entities
Weighted Pressure Plates
Town Manager
Footer
Wow! That is more data than I thought there would be. It appears as though after every section a check is done to verify that the current position in the data file matches a position that is read from the WorldFile.LoadFileFormatHeader. I should be able to use the position at the corresponding index to jump directly to the Chest data section.
The WorldFile.LoadFileFormatHeader is responsible for reading the WorldFileMetadata, the sections, and an array of items known as ‘importance’. Each section position is represented as a single integer value. I am familiar with this kind of data storage technique since HTTP packets do something similar:
Using the section list from the WorldFile.LoadWorld_Version2 and the position data from the WorldFile.LoadFileFormatHeader, I could read the Chest data immediately by jumping to that position in the save file. The next question was to determine how the Chest data was stored.
privatestaticvoidLoadChests(BinaryReader reader) { int num1 = (int) reader.ReadInt16(); int num2 = (int) reader.ReadInt16(); int num3; int num4; if (num2 < 40) { num3 = num2; num4 = 0; } else { num3 = 40; num4 = num2 - 40; } int index1; for (index1 = 0; index1 < num1; ++index1) { Chest chest = new Chest(false); chest.x = reader.ReadInt32(); chest.y = reader.ReadInt32(); chest.name = reader.ReadString(); for (int index2 = 0; index2 < num3; ++index2) { short num5 = reader.ReadInt16(); Item obj = new Item(); if ((int) num5 > 0) { obj.netDefaults(reader.ReadInt32()); obj.stack = (int) num5; obj.Prefix((int) reader.ReadByte()); } elseif ((int) num5 < 0) { obj.netDefaults(reader.ReadInt32()); obj.Prefix((int) reader.ReadByte()); obj.stack = 1; } chest.item[index2] = obj; } for (int index2 = 0; index2 < num4; ++index2) { if ((int) reader.ReadInt16() > 0) { reader.ReadInt32(); int num5 = (int) reader.ReadByte(); } } Main.chest[index1] = chest; } List<Point16> point16List = new List<Point16>(); for (int index2 = 0; index2 < index1; ++index2) { if (Main.chest[index2] != null) { Point16 point16 = new Point16( Main.chest[index2].x, Main.chest[index2].y); if (point16List.Contains(point16)) Main.chest[index2] = (Chest) null; else point16List.Add(point16); } } for (; index1 < 1000; ++index1) Main.chest[index1] = (Chest) null; if (WorldFile.versionNumber >= 115) return; WorldFile.FixDresserChests(); }
Not quite as straight-forward to decipher, but still doable:
num1 represents the number of chests stored in the world.
num2 represents the number of items stored in the chest.
num3 represents the items the chest is holding.
num4 represents the overflow of items if the chest had more items than the maximum.
Each chest then has the following properties:
x represents the x-coordinate of the chest in the world.
y represents the y-coordinate of the chest in the world.
name represents the name of the chest in the world.
I found it odd that the Chest data did not have an Id property that could be used to identify specific chests. Though the x and y properties are probably sufficient in most cases, it just meant that I would have to be more careful about identifying chests.
Depending on the next value a chest can have between 0 and 40 items that have the following properties:
id represents the item id
stack represents the quantity of that item in the single item slot.
prefix represents a prefix value that affects the stats on the item.
The code deviates a little from the normal layout with items. Instead of having an Int16 represent the number of items in a chest, each slot is read. If the slot is empty a zero will be placed in that Item data location and the code must account for that. This is caused by items in chests being located anywhere within the 40 slots. By storing the data this way, the game can reduce the size of the world save file.
After reviewing WorldFile.LoadChests, I had enough information to parse the chest data.
Applying the Research
In an effort NOT to confuse myself, I tried to break the sections out a little further into methods. The code was written using LINQPad to rapidly develop the prototype for reading the data. Main is the entry point for the ‘script’. The final code can be found in Appendix A.
1 2 3 4 5 6 7 8 9 10 11 12 13
voidMain() { World GoodWorld = GetWorld(@"201708310826.wld"); }
World GetWorld(string worldPath) { using (BinaryReader worldReader = new BinaryReader(File.OpenRead(worldPath))) { return GetWorld(worldReader); } }
GetWorld takes a string file path where the world file is located. This would allow me to add a single line to get the data from BadWorld once I was satisfied that the current implementation worked as intended. Once the file was opened for reading, the data needed to be read:
1 2 3 4 5 6 7 8 9 10 11 12 13
World GetWorld(BinaryReader worldReader) { World world = new World();
The File MetaData is read to get the BinaryReader to the correct position. This could be done with math - 2x Int32 (8 bytes), 2x Int64 (16 bytes) would yield 24 bytes. However, to maintain readability instead of jumping to an arbitrary location, reading the MetaData seemed more appropriate. Especially since there is an unknown value and the structure of this location in the file could be updated or changed between versions. An exception to this ‘rule’ is made once the section data has been read since it is more obvious why and where the jump is occurring:
for (int i = 0; i < world.TotalChests; i++) { world.ChestCollection[i] = GetChestData(worldReader, itemsPerChest, overflowItems); } }
Chest GetChestData( BinaryReader worldReader, int itemsPerChest, int overflowItems) { Chest chest = new Chest(itemsPerChest) { X = worldReader.ReadInt32(), Y = worldReader.ReadInt32(), Name = worldReader.ReadString(), };
for (int i = 0; i < itemsPerChest; i++) { chest.ItemCollection[i] = GetItemData(worldReader); }
for (int i = 0; i < overflowItems; i++) { GetItemData(worldReader); }
return chest; }
Each chest can contain one or more items up to the max item. For each chest, the items are parsed in a separate method:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Item GetItemData(BinaryReader worldReader) { short stackSize = worldReader.ReadInt16();
voidMain() { World BadWorld = GetWorld(@"201709010835.wld"); Console.WriteLine( "{0} Empty Chests in Bad World out of {1}", BadWorld.ChestCollection .Where(chest => chest.ItemCollection .All(item => item == null)) .Count(), BadWorld.ChestCollection.Length);
13 Empty Chests in Good World out of 302 145 Empty Chests in Bad World out of 302
That seems to match what I was seeing. Although I was a little surprised there were empty chests in the good world. It turned out that the empty chests in the good world came from the Hell layer and one in my player home that I was not utilizing.
The next step is to try to compare the chests in to the two worlds to see if they can be matched. If they can, a repair is possible.
Comparing Chest Data
Unfortunately, there is no way to prevent having to iterate two lists in order to compare. However, the size of the lists can be reduced by retrieving the empty chests from the bad world. It can be reduced even further by excluding chests contained within player housing. I recommend doing this because these chests have been touched by player(s) which I was more comfortable doing by hand with TEdit. It probably would not have made any difference though:
voidMain() { World BadWorld = GetWorld(@"201709010835.wld"); Console.WriteLine( "{0} Empty Chests in Bad World out of {1}", BadWorld.ChestCollection .Where(chest => chest.ItemCollection .All(item => item == null)) .Count(), BadWorld.ChestCollection.Length);
Console.WriteLine(); Console.WriteLine("After Merge:"); Console.WriteLine( "{0} Empty Chests in Bad World out of {1}", BadWorld.ChestCollection .Where(chest => chest.ItemCollection .All(item => item == null)) .Count(), BadWorld.ChestCollection.Length); }
Then each Chest from the bad world is identified and the items it should contain are added from the matching Chest in the good world.
voidMergeEmptyChests( Chest[] sourceChests, Chest[] destinationChests) { foreach (Chest destinationChest in destinationChests) { foreach (Chest sourceChest in sourceChests) { if (destinationChest.X != sourceChest.X && destinationChest.Y != sourceChest.Y) { continue; }
int sourceChestItemLength = sourceChest.ItemCollection.Length;
for (int i = 0; i < sourceChestItemLength; i++) { destinationChest.ItemCollection[i] = sourceChest.ItemCollection[i]; } } } }
The only way to identify chests in the world is based on their location. Name possibly could be used - except named chests likely mean that the player has placed these chests intentionally. As previously stated, any chests that I had placed I wanted to do manually as I needed to compare with the chests I had placed items in after I noticed the save file corruption.
After running the application I get the following output:
13 Empty Chests in Good World out of 302 145 Empty Chests in Bad World out of 302
After Merge: 13 Empty Chests in Bad World out of 302
Perfect, I was able to restore all of the wild chests back to their generated state. The last step in the process is saving the changes.
Saving Chest Data
Everything up to this point had been relatively easy, so it was only a matter of time before I came across an issue. Unfortunately, the issue did not appear until I got to the final step of saving the merged Chest data.
The Issue
Remember how Chest data is read? Or more specifically, how the item data is read?
Chest GetChestData( BinaryReader worldReader, int itemsPerChest, int overflowItems) { Chest chest = new Chest(itemsPerChest) { X = worldReader.ReadInt32(), Y = worldReader.ReadInt32(), Name = worldReader.ReadString(), };
for (int i = 0; i < itemsPerChest; i++) { chest.ItemCollection[i] = GetItemData(worldReader); }
for (int i = 0; i < overflowItems; i++) { GetItemData(worldReader); }
return chest; }
Item GetItemData(BinaryReader worldReader) { short stackSize = worldReader.ReadInt16();
The issue is that the Chest data has grown after the merge since items have been added to a Chest that require data representations for a stack size, an id, and (potentially?) a prefix. This made my plan to just overwrite the data in the Chest data section impossible because it overwrites the data in the next section and consequentially causes the section locations to be incorrect. I was able to come up with only two ways to solve this:
The first solution was to add support for parsing the entire WorldFile save into the object that is being manipulated. This would prevent any data from getting overwritten but would still require the section data to be updated. Either through mathematically calculating the size of each section given the size of each representation of the data in those sections, or by updating the section positions after each section was written.
While this is probably a more robust approach in the long run, it would require a lot of effort on my part to build all of the representations for the data in each section and the corresponding code to read it.
Partition WorldFile
The second solution would segment the file into ~5 partitions. Three partitions for the data that is not changing, and two partitions for the data that needs updated (section locations and Chest data). In this way the data is accounted for without actually defining a representation for each piece of the data like the previous solution would have had to. Keep in mind that this solution is only worthwhile because the data changes that were made made were local to the Chest data section. If this were not the case, 2n+1 (where n is the number of sections) would need to be updated:
Partition Name
Partition Type
MetaData
Original
Section Location
Modified
2x Sections
Original
Chest Data
Modified
Remaining Sections
Original
Solution Implementation
I chose the second solution as it seemed like the easier of the two for what I was trying to accomplish. If this wasn’t a one-off project, I would suggest the first solution. To implement the second solution, all of the data from the file needs to be read. Fortunately, the data that I am not interested in can be read into a byte array. Then during the save process, the original data can be preserved by writing the contents of these byte arrays. The GetWorld function then gets updated to look like this:
Now that I had all of the file data in a mechanism that prevents data loss, I could write the function to save the world file and call it from Main after the Merge.
voidMain() { World GoodWorld = GetWorld(@"201708310826.wld"); Console.WriteLine( "{0} Empty Chests in Good World out of {1}", GoodWorld.ChestCollection .Where(chest => chest.ItemCollection .All(item => item == null)) .Count(), GoodWorld.ChestCollection.Length);
World BadWorld = GetWorld(@"201709010835.wld"); Console.WriteLine( "{0} Empty Chests in Bad World out of {1}", BadWorld.ChestCollection .Where(chest => chest.ItemCollection .All(item => item == null)) .Count(), BadWorld.ChestCollection.Length);
Console.WriteLine(); Console.WriteLine("After Merge:"); Console.WriteLine( "{0} Empty Chests in Bad World out of {1}", BadWorld.ChestCollection .Where(chest => chest.ItemCollection .All(item => item == null)) .Count(), BadWorld.ChestCollection.Length);
voidSaveWorld(World world, string worldFileSave) { using (BinaryWriter binaryWriter = new BinaryWriter( File.Create(worldFileSave))) { SaveWorld(binaryWriter, world); } }
Just like before, this is just a utility method around the SaveWorld method that actually does all the work. The worldFileSave property is the location to save the data, I recommend making it different than the original file to prevent further data loss!
This method is the exact opposite of the GetWorld method with a few extra modifications. The sectionPosition is used at the end to update the section offsets after all the data has been written since the size of the data section will be known at that point. The newChestDataSectionPosition is the local variable used to update the current section data to the new updated values. The writer then moves back to the section offset location stored in sectionPosition and writes the new section data. Each write method defined in this method is the inverse of the corresponding get method. The BinaryWriter will try to pad some of the values which is why the data is converted to the proper type before writing. This could probably be corrected by changing the underlying value in the corresponding object to the exact value needed:
1 2 3 4 5 6 7
voidWriteWorldMetaData(BinaryWriter worldWriter, World world) { worldWriter.Write((Int32)world.Version); worldWriter.Write((Int64)world.TypeCheck); worldWriter.Write((Int32)world.Revision); worldWriter.Write((Int64)world.UnknownMetaData); }
No scary code here.
1 2 3 4 5 6 7 8 9
voidWriteWorldSections(BinaryWriter worldWriter, World world) { worldWriter.Write((Int16)world.SectionCount);
The value of 0 needs to be explicitly cast down to Int16 as it is by default an Int32. This was causing my file to be larger than it needed to be at first and prevented it from being loaded.
Results
Once the file has been written to disk, it can be loaded in TEdit or Terraria to see if it can be parsed.
Alternate Solutions
Here are a couple alternative approaches to the one outlined above.
TEdit
It would have been faster to reference the TEdit executable and reuse the code they have written to parse the world file data for each instance of the save. This would have been like the first solution except I would be relying on TEdit’s implementation of it instead of rolling my own. The only code that would have needed to be written is MergeEmptyChests.
Terraria
Similar to the previous solution, it may be possible to reference the Terraria executable and reuse the code to load worlds. This would have made the first solution redundant and in hindsight is probably what I should have done in the first place. The save functionality probably does not have a Validate method check like TEdit does though.
Summary
In the end, was it worth it?
Probably not. Honestly, it would have been faster to go through the sand pit again than to research and code a solution like this. Granted, at the time I had no idea I had lost anything. At least I was able to recover from my mistake and learn something in the process.
voidMain() { World GoodWorld = GetWorld(@"201708310826.wld"); Console.WriteLine( "{0} Empty Chests in Good World out of {1}", GoodWorld.ChestCollection .Where(chest => chest.ItemCollection .All(item => item == null)) .Count(), GoodWorld.ChestCollection.Length);
World BadWorld = GetWorld(@"201709010835.wld"); Console.WriteLine( "{0} Empty Chests in Bad World out of {1}", BadWorld.ChestCollection .Where(chest => chest.ItemCollection .All(item => item == null)) .Count(), BadWorld.ChestCollection.Length);
Console.WriteLine(); Console.WriteLine("After Merge:"); Console.WriteLine( "{0} Empty Chests in Bad World out of {1}", BadWorld.ChestCollection .Where(chest => chest.ItemCollection .All(item => item == null)) .Count(), BadWorld.ChestCollection.Length);
for (int i = 0; i < world.TotalChests; i++) { world.ChestCollection[i] = GetChestData( worldReader, itemsPerChest, overflowItems); } }
Chest GetChestData( BinaryReader worldReader, int itemsPerChest, int overflowItems) { Chest chest = new Chest(itemsPerChest) { X = worldReader.ReadInt32(), Y = worldReader.ReadInt32(), Name = worldReader.ReadString(), };
for (int i = 0; i < itemsPerChest; i++) { chest.ItemCollection[i] = GetItemData(worldReader); }
for (int i = 0; i < overflowItems; i++) { GetItemData(worldReader); }
return chest; }
Item GetItemData(BinaryReader worldReader) { short stackSize = worldReader.ReadInt16();
