Forum Announcement, Click Here to Read More From EA_Cade.
Tdesc file structure as compared to tuning files? (and some thoughts on error checking)
Triplis
Posts: 3,048 Member
So as a little side project for attempting to make it easier to spot errors in mods in need of fixing when updates happen, I was looking at the idea of using the TDESC file structure compared to the format of code in a tuning file and compare the two to see if anything is awry.
I got a little tripped up in the details though and figured I'd ask around if anyone knows more about the TDESC file structure, or some perspective I hadn't thought of that might lead to a solution.
So far, I worked out that for tuning files:
I = Instance ("instance_type")
L = Tunable List
U = Tunable Tuple
V = Tunable Variant
E = Tunable Enum
T = Tunable Type
And that 't' and 'n' in the tuning files are short for 'type' and 'name' in the TDESC files.
But it seems to get increasingly complicated from there, rife with exceptions to the rule. Where some branches of commands will follow a structure identical (or near) identical to its sister tree in the TDESC file, some others most definitely don't. For example, for some branches you can't just translate it like:
To something like:
And looking for a branching setup like that in the Interaction.tdesc file as it just wouldn't exist in that format. If I remember correctly for this specific case, the TunableVariant part at the top is instead decided on from a series of different types and is, by default, none.
In theory, exceptions like these could be hard-coded for, but it's hard to tell just how many of them there are and to what extent they would result in false positives when testing for tuning code with errors. I remember trying to resolve the one as in the example above and running into more confusing exceptions to the rule, or I misunderstood what I had thought was a rule that wasn't, or something like that.
However, I know folks like Zerbu have created tools that can generate certain code. I'm not sure if those tools are based off of the tdesc files or copies of tuning files, but presumably the TDESC files could be interpreted from somehow.
The other approach I had considered was testing bits of code (ex: the VFX segment above) against the most up-to-date tuning files and simply seeing if it matches any of the existing code. This would probably work for a lot of things, but runs the risk of false positives if someone has utilized a combination of tuning possibilities that simply hasn't been used in the actual game. It also risks being performance-heavy to analyze, as there's no way of knowing for sure how many files would need to be searched through and compared against. In the case of an error, you'd have to assume it analyzes at least every single game file of a certain type (ex: Interaction) until it finds that there is no code like the sample checked for. Best case scenario for performance would be that it locates an existing example quickly and returns with no error.
So I'm at a bit of a standstill at the moment, in terms of tackling the problem. I think interpretation of the TDESC files and comparing that to tuning files is a realistic possibility. It's just proven to be more convoluted that I thought it would be and I'm not sure if that's because of how the TDESC files are structured, or if I'm just not understanding them well enough.
I got a little tripped up in the details though and figured I'd ask around if anyone knows more about the TDESC file structure, or some perspective I hadn't thought of that might lead to a solution.
So far, I worked out that for tuning files:
I = Instance ("instance_type")
L = Tunable List
U = Tunable Tuple
V = Tunable Variant
E = Tunable Enum
T = Tunable Type
And that 't' and 'n' in the tuning files are short for 'type' and 'name' in the TDESC files.
But it seems to get increasingly complicated from there, rife with exceptions to the rule. Where some branches of commands will follow a structure identical (or near) identical to its sister tree in the TDESC file, some others most definitely don't. For example, for some branches you can't just translate it like:
<V t="vfx">
<U n="vfx">
<V t="from_participant" n="participant">
<U n="from_participant">
<E n="participant">Actor</E>
</U>
</V>
</U>
</V>
To something like:
<TunableVariant type="vfx"
<TunableTuple name="vfx"
<TunableVariant type="from_participant" name="participant"
<TunableTuple name="from_participant"
<TunableEnum name="participant"
And looking for a branching setup like that in the Interaction.tdesc file as it just wouldn't exist in that format. If I remember correctly for this specific case, the TunableVariant part at the top is instead decided on from a series of different types and is, by default, none.
In theory, exceptions like these could be hard-coded for, but it's hard to tell just how many of them there are and to what extent they would result in false positives when testing for tuning code with errors. I remember trying to resolve the one as in the example above and running into more confusing exceptions to the rule, or I misunderstood what I had thought was a rule that wasn't, or something like that.
However, I know folks like Zerbu have created tools that can generate certain code. I'm not sure if those tools are based off of the tdesc files or copies of tuning files, but presumably the TDESC files could be interpreted from somehow.
The other approach I had considered was testing bits of code (ex: the VFX segment above) against the most up-to-date tuning files and simply seeing if it matches any of the existing code. This would probably work for a lot of things, but runs the risk of false positives if someone has utilized a combination of tuning possibilities that simply hasn't been used in the actual game. It also risks being performance-heavy to analyze, as there's no way of knowing for sure how many files would need to be searched through and compared against. In the case of an error, you'd have to assume it analyzes at least every single game file of a certain type (ex: Interaction) until it finds that there is no code like the sample checked for. Best case scenario for performance would be that it locates an existing example quickly and returns with no error.
So I'm at a bit of a standstill at the moment, in terms of tackling the problem. I think interpretation of the TDESC files and comparing that to tuning files is a realistic possibility. It's just proven to be more convoluted that I thought it would be and I'm not sure if that's because of how the TDESC files are structured, or if I'm just not understanding them well enough.
Mods moved from MTS, now hosted at: https://triplis.github.io
Post edited by EA_Cade on
1
Comments
The second program, which I run after I've updated all my mods, is the simplest; it goes through all the extracted tuning files in every mod I have, creates a clean version of the tuning in memory (by stripping all comments, ordering all attributes, and pretty-printing it to a string), compares it with a similarly cleaned version of the corresponding vanilla tuning, and creates a diff file for every such tuning inside a folder automatically created from the mod name.
The first program, which I run as soon as a new patch drops and I've extracted the vanilla tunings, does something similar to the second one, but instead of storing the diff files it compares the newly calculated difference with the one stored by the other program, and if they don't match (or if there is no previous difference stored) it opens WinMerge with the vanilla tuning in the left and my modded one in need of an update on the right (using the command line options /s /wl makes WinMerge open all files in tabs and makes the the left side file in the comparison as read-only).
Both programs also create a list of all my tunings for which they couldn't find a corresponding vanilla tuning, so I can quickly fix that issue.
This method has some advantages. Using it I usually don't even need the TDESC file to update my tunings, and it catches tuning changes that don't result in a change to the TDESC files. Also, by clearly marking my changes with comments and making sure to not introduce any unneeded change to the tuning file, finding what changed and updating each tuning file becomes very easy.
Part of the reason I was looking at more flexible was because in some of my mods I have interactions for which there is no exact match. For example, there are some interaction files in my Sorcerer mod that are an amalgamation of tuning possibilities with similarities to some maxis tunings, but not necessarily close enough to where diffing would spot everything (or I wouldn't know what to compare it to get even a close match).
I can see how diffing would be a breeze for overrides and tunings that are mostly identical in what they contain (ex: traits). I'm not sure how effective it'd be for things like custom interactions.
One thing that can be quite helpful is to run the Sims Log Enabler with the option set in the script to log everything starting at game load. It will produce a Tuning.log file chock full of all the errors in tuning files that do not throw exceptions, but are logged. For example:
[ERROR] [cjiang] Incorrectly matched tuning types found in tuning for put_down_tuning in Instance: 18211831855231043257 (graycurse:pasta_object), Types.OBJECT. Expected 'L', got 'T' [ERROR] [cjiang] ATTRS 3: 🐸🐸🐸🐸_items([('n', 'reference')])That's not entirely automated, of course - you have to run the game and see what errors get tossed in there, but it is very helpful in tracking down what isn't tuned properly but isn't actually throwing an exception. (Or in the above example case, what mods you should probably get around to downloading the latest version of...)Thanks for the advice. Really useful tool.