Shell command to change "Prefer tabs when opening documents"

You don't need to keep changing the Global Preferences: you can set the preferences per app with:

defaults write -app LiveCode AppleWindowTabbingMode -string manual

Other options to try include never, always, and fullscreen.

The app should not be running when you edit the plist.