How to decrypt an encrypted Apple iTunes iPhone backup?
I've been asked by a number of unfortunate iPhone users to help them restore data from their iTunes backups. This is easy when they are unencrypted, but not when they are encrypted, whether or not the password is known.
As such, I'm trying to figure out the encryption scheme used on mddata and mdinfo files when encrypted. I have no problems reading these files otherwise, and have built some robust C# libraries for doing so. (If you're able to help, I don't care which language you use. It's the principle I'm after here!)
The Apple "iPhone OS Enterprise Deployment Guide" states that "Device backups can be stored in encrypted format by selecting the Encrypt iPhone Backup option in the device summary pane of iTunes. Files are encrypted using AES128 with a 256-bit key. The key is stored securely in the iPhone keychain."
That's a pretty good clue, and there's some good info here on Stackoverflow on iPhone AES/Rijndael interoperability suggesting a keysize of 128 and CBC mode may be used.
Aside from any other obfuscation, a key and initialisation vector (IV)/salt are required.
One might assume that the key is a manipulation of the "backup password" that users are prompted to enter by iTunes and passed to "AppleMobileBackup.exe", padded in a fashion dictated by CBC. However, given the reference to the iPhone keychain, I wonder whether the "backup password" might not be used as a password on an X509 certificate or symmetric private key, and that the certificate or private key itself might be used as the key. (AES and the iTunes encrypt/decrypt process is symmetric.)
The IV is another matter, and it could be a few things. Perhaps it's one of the keys hard-coded into iTunes, or into the devices themselves.
Although Apple's comment above suggests the key is present on the device's keychain, I think this isn't that important. One can restore an encrypted backup to a different device, which suggests all information relevant to the decryption is present in the backup and iTunes configuration, and that anything solely on the device is irrelevant and replacable in this context. So where might be the key be?
I've listed paths below from a Windows machine but it's much of a muchness whichever OS we use.
The "\appdata\Roaming\Apple Computer\iTunes\itunesprefs.xml" contains a PList with a "Keychain" dict entry in it. The "\programdata\apple\Lockdown\09037027da8f4bdefdea97d706703ca034c88bab.plist" contains a PList with "DeviceCertificate", "HostCertificate", and "RootCertificate", all of which appear to be valid X509 certs. The same file also appears to contain asymmetric keys "RootPrivateKey" and "HostPrivateKey" (my reading suggests these might be PKCS #7-enveloped). Also, within each backup there are "AuthSignature" and "AuthData" values in the Manifest.plist file, although these appear to be rotated as each file gets incrementally backed up, suggested they're not that useful as a key, unless something really quite involved is being done.
There's a lot of misleading stuff out there suggesting getting data from encrypted backups is easy. It's not, and to my knowledge it hasn't been done. Bypassing or disabling the backup encryption is another matter entirely, and is not what I'm looking to do.
This isn't about hacking apart the iPhone or anything like that. All I'm after here is a means to extract data (photos, contacts, etc.) from encrypted iTunes backups as I can unencrypted ones. I've tried all sorts of permutations with the information I've put down above but got nowhere. I'd appreciate any thoughts or techniques I might have missed.
Solution 1:
Security researchers Jean-Baptiste Bédrune and Jean Sigwald presented how to do this at Hack-in-the-box Amsterdam 2011.
Since then, Apple has released an iOS Security Whitepaper with more details about keys and algorithms, and Charlie Miller et al. have released the iOS Hacker’s Handbook, which covers some of the same ground in a how-to fashion. When iOS 10 first came out there were changes to the backup format which Apple did not publicize at first, but various people reverse-engineered the format changes.
Encrypted backups are great
The great thing about encrypted iPhone backups is that they contain things like WiFi passwords that aren’t in regular unencrypted backups. As discussed in the iOS Security Whitepaper, encrypted backups are considered more “secure,” so Apple considers it ok to include more sensitive information in them.
An important warning: obviously, decrypting your iOS device’s backup
removes its encryption. To protect your privacy and security, you should
only run these scripts on a machine with full-disk encryption. While it
is possible for a security expert to write software that protects keys in
memory, e.g. by using functions like VirtualLock() and
SecureZeroMemory() among many other things, these
Python scripts will store your encryption keys and passwords in strings to
be garbage-collected by Python. This means your secret keys and passwords
will live in RAM for a while, from whence they will leak into your swap
file and onto your disk, where an adversary can recover them. This
completely defeats the point of having an encrypted backup.
How to decrypt backups: in theory
The iOS Security Whitepaper explains the fundamental concepts of per-file keys, protection classes, protection class keys, and keybags better than I can. If you’re not already familiar with these, take a few minutes to read the relevant parts.
Now you know that every file in iOS is encrypted with its own random per-file encryption key, belongs to a protection class, and the per-file encryption keys are stored in the filesystem metadata, wrapped in the protection class key.
To decrypt:
-
Decode the keybag stored in the
BackupKeyBagentry ofManifest.plist. A high-level overview of this structure is given in the whitepaper. The iPhone Wiki describes the binary format: a 4-byte string type field, a 4-byte big-endian length field, and then the value itself.The important values are the PBKDF2
ITERations andSALT, the double protection saltDPSLand iteration countDPIC, and then for each protectionCLS, theWPKYwrapped key. -
Using the backup password derive a 32-byte key using the correct PBKDF2 salt and number of iterations. First use a SHA256 round with
DPSLandDPIC, then a SHA1 round withITERandSALT.Unwrap each wrapped key according to RFC 3394.
-
Decrypt the manifest database by pulling the 4-byte protection class and longer key from the
ManifestKeyinManifest.plist, and unwrapping it. You now have a SQLite database with all file metadata. -
For each file of interest, get the class-encrypted per-file encryption key and protection class code by looking in the
Files.filedatabase column for a binary plist containingEncryptionKeyandProtectionClassentries. Strip the initial four-byte length tag fromEncryptionKeybefore using.Then, derive the final decryption key by unwrapping it with the class key that was unwrapped with the backup password. Then decrypt the file using AES in CBC mode with a zero IV.
How to decrypt backups: in practice
First you’ll need some library dependencies. If you’re on a mac using a homebrew-installed Python 2.7 or 3.7, you can install the dependencies with:
CFLAGS="-I$(brew --prefix)/opt/openssl/include" \
LDFLAGS="-L$(brew --prefix)/opt/openssl/lib" \
pip install biplist fastpbkdf2 pycrypto
In runnable source code form, here is how to decrypt a single preferences file from an encrypted iPhone backup:
#!/usr/bin/env python3.7
# coding: UTF-8
from __future__ import print_function
from __future__ import division
import argparse
import getpass
import os.path
import pprint
import random
import shutil
import sqlite3
import string
import struct
import tempfile
from binascii import hexlify
import Crypto.Cipher.AES # https://www.dlitz.net/software/pycrypto/
import biplist
import fastpbkdf2
from biplist import InvalidPlistException
def main():
## Parse options
parser = argparse.ArgumentParser()
parser.add_argument('--backup-directory', dest='backup_directory',
default='testdata/encrypted')
parser.add_argument('--password-pipe', dest='password_pipe',
help="""\
Keeps password from being visible in system process list.
Typical use: --password-pipe=<(echo -n foo)
""")
parser.add_argument('--no-anonymize-output', dest='anonymize',
action='store_false')
args = parser.parse_args()
global ANONYMIZE_OUTPUT
ANONYMIZE_OUTPUT = args.anonymize
if ANONYMIZE_OUTPUT:
print('Warning: All output keys are FAKE to protect your privacy')
manifest_file = os.path.join(args.backup_directory, 'Manifest.plist')
with open(manifest_file, 'rb') as infile:
manifest_plist = biplist.readPlist(infile)
keybag = Keybag(manifest_plist['BackupKeyBag'])
# the actual keys are unknown, but the wrapped keys are known
keybag.printClassKeys()
if args.password_pipe:
password = readpipe(args.password_pipe)
if password.endswith(b'\n'):
password = password[:-1]
else:
password = getpass.getpass('Backup password: ').encode('utf-8')
## Unlock keybag with password
if not keybag.unlockWithPasscode(password):
raise Exception('Could not unlock keybag; bad password?')
# now the keys are known too
keybag.printClassKeys()
## Decrypt metadata DB
manifest_key = manifest_plist['ManifestKey'][4:]
with open(os.path.join(args.backup_directory, 'Manifest.db'), 'rb') as db:
encrypted_db = db.read()
manifest_class = struct.unpack('<l', manifest_plist['ManifestKey'][:4])[0]
key = keybag.unwrapKeyForClass(manifest_class, manifest_key)
decrypted_data = AESdecryptCBC(encrypted_db, key)
temp_dir = tempfile.mkdtemp()
try:
# Does anyone know how to get Python’s SQLite module to open some
# bytes in memory as a database?
db_filename = os.path.join(temp_dir, 'db.sqlite3')
with open(db_filename, 'wb') as db_file:
db_file.write(decrypted_data)
conn = sqlite3.connect(db_filename)
conn.row_factory = sqlite3.Row
c = conn.cursor()
# c.execute("select * from Files limit 1");
# r = c.fetchone()
c.execute("""
SELECT fileID, domain, relativePath, file
FROM Files
WHERE relativePath LIKE 'Media/PhotoData/MISC/DCIM_APPLE.plist'
ORDER BY domain, relativePath""")
results = c.fetchall()
finally:
shutil.rmtree(temp_dir)
for item in results:
fileID, domain, relativePath, file_bplist = item
plist = biplist.readPlistFromString(file_bplist)
file_data = plist['$objects'][plist['$top']['root'].integer]
size = file_data['Size']
protection_class = file_data['ProtectionClass']
encryption_key = plist['$objects'][
file_data['EncryptionKey'].integer]['NS.data'][4:]
backup_filename = os.path.join(args.backup_directory,
fileID[:2], fileID)
with open(backup_filename, 'rb') as infile:
data = infile.read()
key = keybag.unwrapKeyForClass(protection_class, encryption_key)
# truncate to actual length, as encryption may introduce padding
decrypted_data = AESdecryptCBC(data, key)[:size]
print('== decrypted data:')
print(wrap(decrypted_data))
print()
print('== pretty-printed plist')
pprint.pprint(biplist.readPlistFromString(decrypted_data))
##
# this section is mostly copied from parts of iphone-dataprotection
# http://code.google.com/p/iphone-dataprotection/
CLASSKEY_TAGS = [b"CLAS",b"WRAP",b"WPKY", b"KTYP", b"PBKY"] #UUID
KEYBAG_TYPES = ["System", "Backup", "Escrow", "OTA (icloud)"]
KEY_TYPES = ["AES", "Curve25519"]
PROTECTION_CLASSES={
1:"NSFileProtectionComplete",
2:"NSFileProtectionCompleteUnlessOpen",
3:"NSFileProtectionCompleteUntilFirstUserAuthentication",
4:"NSFileProtectionNone",
5:"NSFileProtectionRecovery?",
6: "kSecAttrAccessibleWhenUnlocked",
7: "kSecAttrAccessibleAfterFirstUnlock",
8: "kSecAttrAccessibleAlways",
9: "kSecAttrAccessibleWhenUnlockedThisDeviceOnly",
10: "kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly",
11: "kSecAttrAccessibleAlwaysThisDeviceOnly"
}
WRAP_DEVICE = 1
WRAP_PASSCODE = 2
class Keybag(object):
def __init__(self, data):
self.type = None
self.uuid = None
self.wrap = None
self.deviceKey = None
self.attrs = {}
self.classKeys = {}
self.KeyBagKeys = None #DATASIGN blob
self.parseBinaryBlob(data)
def parseBinaryBlob(self, data):
currentClassKey = None
for tag, data in loopTLVBlocks(data):
if len(data) == 4:
data = struct.unpack(">L", data)[0]
if tag == b"TYPE":
self.type = data
if self.type > 3:
print("FAIL: keybag type > 3 : %d" % self.type)
elif tag == b"UUID" and self.uuid is None:
self.uuid = data
elif tag == b"WRAP" and self.wrap is None:
self.wrap = data
elif tag == b"UUID":
if currentClassKey:
self.classKeys[currentClassKey[b"CLAS"]] = currentClassKey
currentClassKey = {b"UUID": data}
elif tag in CLASSKEY_TAGS:
currentClassKey[tag] = data
else:
self.attrs[tag] = data
if currentClassKey:
self.classKeys[currentClassKey[b"CLAS"]] = currentClassKey
def unlockWithPasscode(self, passcode):
passcode1 = fastpbkdf2.pbkdf2_hmac('sha256', passcode,
self.attrs[b"DPSL"],
self.attrs[b"DPIC"], 32)
passcode_key = fastpbkdf2.pbkdf2_hmac('sha1', passcode1,
self.attrs[b"SALT"],
self.attrs[b"ITER"], 32)
print('== Passcode key')
print(anonymize(hexlify(passcode_key)))
for classkey in self.classKeys.values():
if b"WPKY" not in classkey:
continue
k = classkey[b"WPKY"]
if classkey[b"WRAP"] & WRAP_PASSCODE:
k = AESUnwrap(passcode_key, classkey[b"WPKY"])
if not k:
return False
classkey[b"KEY"] = k
return True
def unwrapKeyForClass(self, protection_class, persistent_key):
ck = self.classKeys[protection_class][b"KEY"]
if len(persistent_key) != 0x28:
raise Exception("Invalid key length")
return AESUnwrap(ck, persistent_key)
def printClassKeys(self):
print("== Keybag")
print("Keybag type: %s keybag (%d)" % (KEYBAG_TYPES[self.type], self.type))
print("Keybag version: %d" % self.attrs[b"VERS"])
print("Keybag UUID: %s" % anonymize(hexlify(self.uuid)))
print("-"*209)
print("".join(["Class".ljust(53),
"WRAP".ljust(5),
"Type".ljust(11),
"Key".ljust(65),
"WPKY".ljust(65),
"Public key"]))
print("-"*208)
for k, ck in self.classKeys.items():
if k == 6:print("")
print("".join(
[PROTECTION_CLASSES.get(k).ljust(53),
str(ck.get(b"WRAP","")).ljust(5),
KEY_TYPES[ck.get(b"KTYP",0)].ljust(11),
anonymize(hexlify(ck.get(b"KEY", b""))).ljust(65),
anonymize(hexlify(ck.get(b"WPKY", b""))).ljust(65),
]))
print()
def loopTLVBlocks(blob):
i = 0
while i + 8 <= len(blob):
tag = blob[i:i+4]
length = struct.unpack(">L",blob[i+4:i+8])[0]
data = blob[i+8:i+8+length]
yield (tag,data)
i += 8 + length
def unpack64bit(s):
return struct.unpack(">Q",s)[0]
def pack64bit(s):
return struct.pack(">Q",s)
def AESUnwrap(kek, wrapped):
C = []
for i in range(len(wrapped)//8):
C.append(unpack64bit(wrapped[i*8:i*8+8]))
n = len(C) - 1
R = [0] * (n+1)
A = C[0]
for i in range(1,n+1):
R[i] = C[i]
for j in reversed(range(0,6)):
for i in reversed(range(1,n+1)):
todec = pack64bit(A ^ (n*j+i))
todec += pack64bit(R[i])
B = Crypto.Cipher.AES.new(kek).decrypt(todec)
A = unpack64bit(B[:8])
R[i] = unpack64bit(B[8:])
if A != 0xa6a6a6a6a6a6a6a6:
return None
res = b"".join(map(pack64bit, R[1:]))
return res
ZEROIV = "\x00"*16
def AESdecryptCBC(data, key, iv=ZEROIV, padding=False):
if len(data) % 16:
print("AESdecryptCBC: data length not /16, truncating")
data = data[0:(len(data)/16) * 16]
data = Crypto.Cipher.AES.new(key, Crypto.Cipher.AES.MODE_CBC, iv).decrypt(data)
if padding:
return removePadding(16, data)
return data
##
# here are some utility functions, one making sure I don’t leak my
# secret keys when posting the output on Stack Exchange
anon_random = random.Random(0)
memo = {}
def anonymize(s):
if type(s) == str:
s = s.encode('utf-8')
global anon_random, memo
if ANONYMIZE_OUTPUT:
if s in memo:
return memo[s]
possible_alphabets = [
string.digits,
string.digits + 'abcdef',
string.ascii_letters,
"".join(chr(x) for x in range(0, 256)),
]
for a in possible_alphabets:
if all((chr(c) if type(c) == int else c) in a for c in s):
alphabet = a
break
ret = "".join([anon_random.choice(alphabet) for i in range(len(s))])
memo[s] = ret
return ret
else:
return s
def wrap(s, width=78):
"Return a width-wrapped repr(s)-like string without breaking on \’s"
s = repr(s)
quote = s[0]
s = s[1:-1]
ret = []
while len(s):
i = s.rfind('\\', 0, width)
if i <= width - 4: # "\x??" is four characters
i = width
ret.append(s[:i])
s = s[i:]
return '\n'.join("%s%s%s" % (quote, line ,quote) for line in ret)
def readpipe(path):
if stat.S_ISFIFO(os.stat(path).st_mode):
with open(path, 'rb') as pipe:
return pipe.read()
else:
raise Exception("Not a pipe: {!r}".format(path))
if __name__ == '__main__':
main()
Which then prints this output:
Warning: All output keys are FAKE to protect your privacy
== Keybag
Keybag type: Backup keybag (1)
Keybag version: 3
Keybag UUID: dc6486c479e84c94efce4bea7169ef7d
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Class WRAP Type Key WPKY Public key
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
NSFileProtectionComplete 2 AES 4c80b6da07d35d393fc7158e18b8d8f9979694329a71ceedee86b4cde9f97afec197ad3b13c5d12b
NSFileProtectionCompleteUnlessOpen 2 AES 09e8a0a9965f00f213ce06143a52801f35bde2af0ad54972769845d480b5043f545fa9b66a0353a6
NSFileProtectionCompleteUntilFirstUserAuthentication 2 AES e966b6a0742878ce747cec3fa1bf6a53b0d811ad4f1d6147cd28a5d400a8ffe0bbabea5839025cb5
NSFileProtectionNone 2 AES 902f46847302816561e7df57b64beea6fa11b0068779a65f4c651dbe7a1630f323682ff26ae7e577
NSFileProtectionRecovery? 3 AES a3935fed024cd9bc11d0300d522af8e89accfbe389d7c69dca02841df46c0a24d0067dba2f696072
kSecAttrAccessibleWhenUnlocked 2 AES 09a1856c7e97a51a9c2ecedac8c3c7c7c10e7efa931decb64169ee61cb07a0efb115050fd1e33af1
kSecAttrAccessibleAfterFirstUnlock 2 AES 0509d215f2f574efa2f192efc53c460201168b26a175f066b5347fc48bc76c637e27a730b904ca82
kSecAttrAccessibleAlways 2 AES b7ac3c4f1e04896144ce90c4583e26489a86a6cc45a2b692a5767b5a04b0907e081daba009fdbb3c
kSecAttrAccessibleWhenUnlockedThisDeviceOnly 3 AES 417526e67b82e7c6c633f9063120a299b84e57a8ffee97b34020a2caf6e751ec5750053833ab4d45
kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly 3 AES b0e17b0cf7111c6e716cd0272de5684834798431c1b34bab8d1a1b5aba3d38a3a42c859026f81ccc
kSecAttrAccessibleAlwaysThisDeviceOnly 3 AES 9b3bdc59ae1d85703aa7f75d49bdc600bf57ba4a458b20a003a10f6e36525fb6648ba70e6602d8b2
== Passcode key
ee34f5bb635830d698074b1e3e268059c590973b0f1138f1954a2a4e1069e612
== Keybag
Keybag type: Backup keybag (1)
Keybag version: 3
Keybag UUID: dc6486c479e84c94efce4bea7169ef7d
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Class WRAP Type Key WPKY Public key
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
NSFileProtectionComplete 2 AES 64e8fc94a7b670b0a9c4a385ff395fe9ba5ee5b0d9f5a5c9f0202ef7fdcb386f 4c80b6da07d35d393fc7158e18b8d8f9979694329a71ceedee86b4cde9f97afec197ad3b13c5d12b
NSFileProtectionCompleteUnlessOpen 2 AES 22a218c9c446fbf88f3ccdc2ae95f869c308faaa7b3e4fe17b78cbf2eeaf4ec9 09e8a0a9965f00f213ce06143a52801f35bde2af0ad54972769845d480b5043f545fa9b66a0353a6
NSFileProtectionCompleteUntilFirstUserAuthentication 2 AES 1004c6ca6e07d2b507809503180edf5efc4a9640227ac0d08baf5918d34b44ef e966b6a0742878ce747cec3fa1bf6a53b0d811ad4f1d6147cd28a5d400a8ffe0bbabea5839025cb5
NSFileProtectionNone 2 AES 2e809a0cd1a73725a788d5d1657d8fd150b0e360460cb5d105eca9c60c365152 902f46847302816561e7df57b64beea6fa11b0068779a65f4c651dbe7a1630f323682ff26ae7e577
NSFileProtectionRecovery? 3 AES 9a078d710dcd4a1d5f70ea4062822ea3e9f7ea034233e7e290e06cf0d80c19ca a3935fed024cd9bc11d0300d522af8e89accfbe389d7c69dca02841df46c0a24d0067dba2f696072
kSecAttrAccessibleWhenUnlocked 2 AES 606e5328816af66736a69dfe5097305cf1e0b06d6eb92569f48e5acac3f294a4 09a1856c7e97a51a9c2ecedac8c3c7c7c10e7efa931decb64169ee61cb07a0efb115050fd1e33af1
kSecAttrAccessibleAfterFirstUnlock 2 AES 6a4b5292661bac882338d5ebb51fd6de585befb4ef5f8ffda209be8ba3af1b96 0509d215f2f574efa2f192efc53c460201168b26a175f066b5347fc48bc76c637e27a730b904ca82
kSecAttrAccessibleAlways 2 AES c0ed717947ce8d1de2dde893b6026e9ee1958771d7a7282dd2116f84312c2dd2 b7ac3c4f1e04896144ce90c4583e26489a86a6cc45a2b692a5767b5a04b0907e081daba009fdbb3c
kSecAttrAccessibleWhenUnlockedThisDeviceOnly 3 AES 80d8c7be8d5103d437f8519356c3eb7e562c687a5e656cfd747532f71668ff99 417526e67b82e7c6c633f9063120a299b84e57a8ffee97b34020a2caf6e751ec5750053833ab4d45
kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly 3 AES a875a15e3ff901351c5306019e3b30ed123e6c66c949bdaa91fb4b9a69a3811e b0e17b0cf7111c6e716cd0272de5684834798431c1b34bab8d1a1b5aba3d38a3a42c859026f81ccc
kSecAttrAccessibleAlwaysThisDeviceOnly 3 AES 1e7756695d337e0b06c764734a9ef8148af20dcc7a636ccfea8b2eb96a9e9373 9b3bdc59ae1d85703aa7f75d49bdc600bf57ba4a458b20a003a10f6e36525fb6648ba70e6602d8b2
== decrypted data:
'<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE plist PUBLIC "-//Apple//DTD '
'PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">\n<plist versi'
'on="1.0">\n<dict>\n\t<key>DCIMLastDirectoryNumber</key>\n\t<integer>100</integ'
'er>\n\t<key>DCIMLastFileNumber</key>\n\t<integer>3</integer>\n</dict>\n</plist'
'>\n'
== pretty-printed plist
{'DCIMLastDirectoryNumber': 100, 'DCIMLastFileNumber': 3}
Extra credit
The iphone-dataprotection code posted by Bédrune and Sigwald can decrypt the keychain from a backup, including fun things like saved wifi and website passwords:
$ python iphone-dataprotection/python_scripts/keychain_tool.py ...
--------------------------------------------------------------------------------------
| Passwords |
--------------------------------------------------------------------------------------
|Service |Account |Data |Access group |Protection class|
--------------------------------------------------------------------------------------
|AirPort |Ed’s Coffee Shop |<3FrenchRoast |apple |AfterFirstUnlock|
...
That code no longer works on backups from phones using the latest iOS, but there are some golang ports that have been kept up to date allowing access to the keychain.
Solution 2:
Sorry, but it might even be more complicated, involving pbkdf2, or even a variation of it. Listen to the WWDC 2010 session #209, which mainly talks about the security measures in iOS 4, but also mentions briefly the separate encryption of backups and how they're related.
You can be pretty sure that without knowing the password, there's no way you can decrypt it, even by brute force.
Let's just assume you want to try to enable people who KNOW the password to get to the data of their backups.
I fear there's no way around looking at the actual code in iTunes in order to figure out which algos are employed.
Back in the Newton days, I had to decrypt data from a program and was able to call its decryption function directly (knowing the password, of course) without the need to even undersand its algorithm. It's not that easy anymore, unfortunately.
I'm sure there are skilled people around who could reverse engineer that iTunes code - you just have to get them interested.
In theory, Apple's algos should be designed in a way that makes the data still safe (i.e. practically unbreakable by brute force methods) to any attacker knowing the exact encryption method. And in WWDC session 209 they went pretty deep into details about what they do to accomplish this. Maybe you can actually get answers directly from Apple's security team if you tell them your good intentions. After all, even they should know that security by obfuscation is not really efficient. Try their security mailing list. Even if they do not repond, maybe someone else silently on the list will respond with some help.
Good luck!
Solution 3:
Haven't tried it, but Elcomsoft released a product they claim is capable of decrypting backups, for forensics purposes. Maybe not as cool as engineering a solution yourself, but it might be faster.
http://www.elcomsoft.com/eppb.html