Can DFHACK saved stockpile settings be safely used in different worlds?

One of the features that the DFHACK mod adds to Dwarf Fortress is the ability to save stockpile settings and load them into a different stockpile, which is much faster than repeating the workflow of selecting the item types that you want in the stockpile one at a time. It even allows you to load stockpile settings saved from a different world.

But some of the item types listed in the stockpile settings menu are procedurally generated - for example forgotten beast meats and musical instruments and their components - and the number of types of these in one world won't necessarily be the same as the number of types in a different world.

It's not difficult to imagine how an implementation of saved stockpile settings could cause problems when importing stockpile settings from one world into another world - if it's just a save of a block of opaque binary data, the correct size for that data won't be the same between worlds with different numbers of procedurally generated item types, and loading stockpile settings from another world would cause a buffer overrun if it's too big or leave uninitialized memory if it's too small.

So, is it safe to load stockpile settings into different world from the one in which they were saved?

Solution 1:

This shouldn't happen for instrument parts

Instrument parts are sorted under Furniture -> Tools in the menu. You can't specify the specific parts for a stockpile, only generic categories of material under the Tools subcategory, e.g. Furniture -> Tools -> Stone. Instrument parts are not explicitly stated. In other words, the stockpile menu is actually fixed, even though some of the things in the game are procedurally generated.

For creatures (refuse) specifically this could happen:

If the stockpile settings are of different length

The one instance where things could go haywire is when writing the data back into the game. DFhack could indeed corrupt internal memory if it simply wrote the binary data back without understanding what is in the binary data (treating it as a black box). Fortunately this is not the case.

Just looking at the DFhack code, the stockpileSerializer has some good defensive coding practices going on that should protect you from writing past the boundaries of arrays or vectors, so segmentation faults and buffer overflows should not occur in any circumstances, and (bugs notwithstanding) you should be able to import stockpile settings across worlds without corrupting your game state. If you do manage to corrupt your game state you could get into contact with the DFhack team: they're likely to consider it a bug.

If for some reason you attempt to write a stockpile setting array into the game that is longer than the size of the array (which could potentially happen if you try and mix versions or worlds), then DFhack will print an error message to the console stating something similar to:

error item_type index too large!   idx[224] max_size[223]

Which means it spotted an attempt to write into index 224 of a 223-member vector. It will ignore the value and continue importing in the next (sub)category.

Conversely, if you attempt to write an array that is too short, nothing really happens. The final setting(s) are simply not loaded and left as whatever they currently are.

Note that when you do configure these 'procedural' settings, or any setting in the menu past it, that DFhack will mess up when trying to restore. To see why this happens, let's consider an example.

Example

Consider a hypothetical category with three fixed members and one to five procedural members. Say you set your stockpile in world A to be (here zeroes mean 'do not accept' and ones mean 'accept'):

fixed_1   1
proc_1    0
proc_2    1
fixed_2   1
fixed_3   1

And now load into world B, which has 3 procedural members.

fixed_1   1
proc_1    0
proc_2    1
proc_3    1
fixed_2   1
fixed_3   ?

The value ? will be whatever happened to be configured before you loaded in your stockpile settings.

Which theoretically if initialized from nothing could be a random value (whatever happened to be in memory), or some sane default (e.g. always on or always off). But, Stockpile loading is always applied to an existing stockpile, meaning there should be no potential for uninitialized memory to come into play. The missing value(s) will simply be skipped.

Advice for working around this behavour?

If you don't mind being a tiny bit spoiled when starting afresh, you can set aside some time to generate a very large world, and then make a copy of the world to play in. You can have a separate fortress save in each copy, and they're going to have identical procedural data.

This would allow you to play in a "fresh" environment each time. A large world has so much stuff in it that you could theoretically start multiple forts in it before most of your encountered procedural content like megabeasts and werecreatures start being repeats. This would allow you to save your stockpile settings across a reasonable amount of saves.