Hi everyone,
I want to give you a heads up that in an upcoming release, we will be upgrading the version of Python used by the game. We currently use Python 3.3.5 and will be moving to
Python 3.7.0. Once this change is released, all compiled script mods will completely stop working. Note that in this new version of Python, the .pyo extension is replaced by .pyc, see
https://www.python.org/dev/peps/pep-0488/
We know this will have impact on your script mod authoring pipelines so wanted to provide notice in advance so you can be ready for the change when it arrives. To assist, I’ve attached the Python scripts from the currently released version of the game (v1.46.18) recompiled under Python 3.7.0.
If all goes as planned, this change should arrive in November.
Note that there will not be any fundamental changes to tuning data, other than the typical types of changes we make to tuning for patches.
Thanks!
SGMS
Comments
As Turbo says, THANKS FOR THE WARNING and for including those recompiled scripts so we can try and figure out how we're going to reverse engineer those in advance.
Any chance we get a more concrete idea of the time frame as you know more about when this bomb will drop?
Anyone know if i can still use the bat if i change it to this after upgrading phyton to 3.7.0?:
Is it possible that there is some documentation that we can get that explains all of the files and functions? Is there anything that can help us now @SimGuruModSquad?
@SimGuruModSquad The problem will be getting the EA scripts we refer to for new mods. So far the best decompiler we've been able to work with for Python 3.7 is uncompyle6 and that just chokes and dies on any seriously important script files. Many critical files that would be a required reference for a script modder are simply no longer accessible (unless someone finds a better uncompiler, but they are mostly unmaintained since Python 3.5 bytecode changes).
game_object.py, object_tests.py, sim_info.py and many other critical files we'd need to reference are simply cannot be decompiled at this point. Unless something changes, this may well spend the end of script modding for me personally and likely others - I for one have no desire to continue to support mods when I can't look at script changes that break those mods. And certainly any new scripting would be unthinkable for me.
Unless somehow EA can reverse their viewpoint on releasing the core scripts to modders for reference, or we can find a reliable method to reverse engineer those files - something which EA has been aware of us doing and quietly approved of since the game came out.
When Sims 4 dropped it toted itself as being modder friendly, and in many ways it is. But other ways it's frustratingly restrictive for the most arbitrary reasons.
If we're able to get the tuning source, why not the python source as well? Without it, this is practically a move to end script mods, whether that's the actual reason or not.
Fun must be always -- Tomas Hertl (San Jose Sharks hockey player)
Otherwise, compiling to a new version and all, we can make work. Those of us still around, anyway.
Hadn't checked email since this morning -- I'm a Patreon though I had to step back to first tier due to some financial issues (hope to be back up to second tier in the next couple of months).
Fun must be always -- Tomas Hertl (San Jose Sharks hockey player)
Small correction, the file sgms posted here is still compiled code. It remains to be seen if they'll provide uncompiled code but here's hoping. 🤞
But no really, I have hopes that the super awesome modding community can work their way even out of this one...
I hope.
Nice to know. I haven't unzipped it or done anything with it yet. Planned to do that tomorrow. Thanks!
IMO, "potential fallout" (from the players perspective) would best be discussed in other threads besides this one.
Let's leave this one for the Coding discussions.
You could join us on Deaderpool's Discord, too. Hot Topic in #creators_workshop there, for sure.
https://deaderpool-mccc.com/
https://www.patreon.com/egwarhammer
https://twitter.com/eg4mccc
https://eg4mccc.tumblr.com/
If you still aren't willing to give us the uncompiled script sources, could you reconsider giving us the documentation?
I am assuming that you generate documentation for internal use, and I can't see how it would be worse than giving us the TDESC's.
Although I haven't delved much into script modding myself, my guess is that having access to the documentation is far more useful to us than having access to decompiled code.
@Scumbumbo
Remind me if I'm wrong, but weren't we unable to decompile a lot of files in the beginning, but then someone went in and fixed a decompiler?
We probably could look into why uncompyle6 chokes on the files and maybe fix the problem there.
As our community uses these tools so much it would be nice if we gave back some, if we can that is.
To everyone else:
I'd suggest that anyone that Isn't going to be working on decompiling the scripts keep from posting here, not because your opinions aren't valid, but because if you derail the conversation it's less likely that we will come to a good solution. Consider making a new thread instead.
Gender Less - Gender neutral pronouns and titles
Hidden Highlight - Removes the white outline
Unlocked CAS Parts - Unlocks hidden CAS items
If not decompiled code, the best they can do is give us an outline of the code (by wiping the actual code) with the developer notes. But something tells me that there might not be any useful (for us) notes in there. I might be wrong about it but the programming staff hasn't changed much for the last 4 years which means it's a very minimal documentation. Looks like a lot of sections of the sources is dealt with by individual programmers since they include their names there, so they don't need that much documentation in the first place.
And we can generate an outline like that ourselves, but it won't be anywhere near as helpful as the actual source.
The decompiler for Python 3.3 was actually designed for 3.2 but then edited for someone somewhere to work with 3.3. I don't think it was the Sims community that got it to work, it was the case of using the decomplier for the right version since they are named the same. And if I'm wrong, just by looking at uncompyle6, we are dealing with something a lot more complex and editing will be challenging.
Well, as Scumbumbo pointed out, early days yet, EA putting out the feelers, and thus may well tune their final approach to fully consider the script modders who have put so much hard work in over these last four years.
I imagine healthy game sales figures do depend on it. Don't bite the hands that feed you, nor the ones holding up your chair. So...
The complex fixes were, as Turbo mentions above, done by someone outside TS4 modding community.
Andrew and I have both tried some similar hacks in the same vein without any really notable results.
I'm going to run everything through decompilation tomorrow and summarize exactly what results we end up with and provide those to SGMS. Andrew and I were just chatting with them on Discord, I won't post the gist of the convo here as I don't want to put words in EA's mouth obviously. All I can say here is that releasing sources is unlikely (as we expected), but they seem committed to helping us work out issues with decompilation.
At a guess, I'd say roughly 75% of files seem to uncompile properly using uncompyle6, but those that fail are quite important ones as a general rule. Some that uncompile have some invalid syntax in them, particularly those with complex nested expressions. My tests I run will involve recompiling the uncompiled sources to check their validity.
Feel free to PM me on DP's Discord channel if you want some more unofficial info, I imagine SGMS will be posting here as well once anything is official.
@SimGuruModSquad - thanks a lot for chatting with us, it is good to know definitively that you will assist in-so-far as you are able to.
Although I really shouldn't be working on the sims, I have fallen down the rabbit hole again. I might be interested in looking into trying to fix the issues in uncompyle6. Although python is not my language of choice I do have knowledge about compilers and such.
Gender Less - Gender neutral pronouns and titles
Hidden Highlight - Removes the white outline
Unlocked CAS Parts - Unlocks hidden CAS items
1,261 (73%) files uncompiled and tested 100% accurate.
296 (17%) files uncompoiled but could not be recompiled due to syntax errors.
169 (10%) files threw exceptions during the uncompile run, of those 10 were due to pressing Ctrl C as those files would get stuck in an infinite loop. The remaining 159, at least all the ones I saw, threw the assert failure due to not being able to properly parse the kwargs ending up with a negative positional arg count.
I can send you a copy of the results, which would include the failed uncompiled files and the log that was generated with all exceptions and messages about failed verifications, just let me know how you want me to get that to you.
Meanwhile, Fogity and Turbo have been trying some stuff with pycdc and I'll likely try again to get a working copy of that compiled so I can look into that.
Thanks,
Diane Pinkus