How can I safely eliminate duplicate apps from my iPhone?

I have just had my iPhone replaced under warranty, and when restoring it from backup much of my preexisting free space vanished.

What I've found, after some shell hacking, is that I appear to have duplicate installs of several apps, some of which are quite large. Here is an alphabetically sorted list of these apps:

100 Rogues  50048   2B0EBC40-39DC-4EB9-AC13-21521314D202
100 Rogues  50048   8712B20E-CE00-4725-B4E8-5561C99C68E5
170,000+ Recipes - BigOven  8280    8A325E2D-2C79-40F3-A8D2-DA139824A982
170,000+ Recipes - BigOven  8200    ED9641E4-1258-4BA7-B4D4-5E4571E2E905
1Password Pro   15256   D83770FC-3021-47DD-A950-8BF2C8BB2526
1Password Pro   15104   84C1767D-F429-4E00-8D2B-74FF084E082D
2 Across    1600    07D94823-6768-4AC6-809A-4585AE21752A
2 Across    1600    52411FB8-F553-4E37-AA23-74884FAE176B
360 Live    8032    5CCDAE0B-1174-4A66-9C39-99F9EB45A3C0
360 Live    8032    C5D806B8-BB02-4CCF-96E8-F1615AD8B261
Angry Birds 43568   65C6EC54-A390-4C3A-A108-1E4BD15CE923
Angry Birds 43568   B289BCB4-FE4E-4AD9-879E-F257ED20E713
Angry Birds Seasons 32968   4B8A5E2C-4554-4EDF-A89E-6E3AE4AC1280
ArithmeGrid Full    1040    6FE96AF0-5261-4A9F-A643-CA6C18498D9E
ArithmeGrid Full    1040    F513D748-F5CA-482F-9586-E6C48383F06F
AroundMe    6512    BA1802E7-2C58-443D-AF2B-5699584A6EBD
AroundMe    6504    F97E85F1-9F2A-403E-AD6A-D14F6885E3AE
Ash 89808   1DAECBA5-80A5-4461-81B4-799B438F1402
Ash 89808   969F7768-69CF-496F-8138-9F6C75D43B1B
Battleheart 239520  236C27CF-6222-47C0-9BE7-EBE0C737A5C8
Battleheart 239520  816A8155-8C75-4F7B-8AC5-CA2CDA29F938
Beneath a Steel Sky: Remastered 167136  D4CC69B4-3DB6-4A8D-B2E5-18A82DA601B8
Boxcar  23536   CA71F1C9-1ED4-413E-BB36-2A5B26A367F3
Boxcar  23256   89C3C612-C543-4369-8FA0-5D325C63BA67
Camera+​  45336   015B6087-7514-4249-81AB-57002F8B4A64
Camera+​  45336   C800AF53-6759-4183-8A2D-248062781C11
Canadian YellowPages.ca Business and People Finder  3376    D5AE87DE-37CB-4BA9-B2F3-6070D898B5C6
Carcassonne 146608  CAC1C704-DE9F-4051-89DB-7645747D32D9
Carcassonne 146608  D4BB7015-FB72-43C6-B14E-E55DBFF29B7B
CBC Radio   5432    5F9A4FEB-46CF-48ED-A02B-F486C6517FF4
CBC Radio   5432    716B93AA-0642-4B9B-9A65-5CFDD42965F0
Civilization Revolution 98544   2FD14644-14CC-449D-9976-A4AFD5233AD7
Civilization Revolution 98544   C8A3CD0E-D2E2-41EA-946C-254D545F7CD2
ClearCam    271608  86FB6803-D5B6-4BDC-BC53-65FFCBF43D05
ClearCam    271608  96BEFE40-CB4D-45EE-9509-4A8E61628EA6
Cocktails+  21432   79E914F0-8998-4B4F-A51E-9326B33593AB
Cocktails+  21432   CA29F2B6-2F5D-4731-A7AA-012AADDEA1BA
Cogs    42856   8648A128-F9EB-4D15-B2FE-D819120DF4E9
Cogs    42856   A44BE8FA-069F-40DE-B4B9-E97F84957CD3
CompendiumHelper    312 81A5C784-62E3-4A5C-8F51-4A09848A6D83
CompendiumHelper    312 F5CDBA2C-0AB4-42B5-AF66-99B2C4EEE6C4
Consume 15984   685355AF-7CF6-47B1-9046-8E640CA24CEF
Consume 15984   E26E2146-D6EA-4B74-9A34-0603F4211371
Controllr ~ Plex remote 11928   545E4696-9407-47E7-8D2D-6BD88B09AB21
Controllr ~ Plex remote 11928   DB608992-3798-476B-BE78-98858B15D664
Convertbot — The Amazing Unit Converter   10352   822F6C91-056D-4257-8A11-D779D7422177
Convertbot — The Amazing Unit Converter   10352   EA24DE01-51F1-4081-91C1-70829EC9DDE0
Convore 6808    80F540C1-00AB-4DDD-AD72-72A4F22B2548
Convore 6808    F666C5C6-9128-4712-AE1F-912A55AEEDC7
Cook’s Illustrated    3544    E3FDD73D-FE51-4B64-9BAB-5582C9EFB390
Cook’s Illustrated    3544    EF6B0310-447C-4647-90BE-B9659469E058
Cookmate    4584    6A2ED1F4-F57C-45E7-B566-77B2D5CDC52D
Cookmate    4584    9038D2F1-06AD-44A2-B4FE-7C2879E587DC
Craigsphone - craigslist for iphone 4872    573D30E7-D547-4271-9EE3-AC0E3D5FA270
Craigsphone - craigslist for iphone 4872    73ED9FAE-DD16-4B37-8830-03DC8E7D3D91
Crush the Castle    15440   5C602CCA-F90F-4DD9-89E9-F0759F9BFF4A
Crush the Castle    15440   72AAAC2C-10B1-47E2-9065-2EE00207631B
Dice Bag    2008    8DC4EEE6-E136-4C1E-B3C3-180CF2557BBA
Dice Bag    2008    A153B606-ADD2-4643-BC81-9CB7D762BB29
Dictionary.com - Dictionary & Thesaurus - Free  142160  3407CC86-AEF8-441D-B347-35DC899B0BF9
Dictionary.com - Dictionary & Thesaurus - Free  142160  C50C4E97-209E-4577-BB78-FE7022110710
DMs Tracker 1304    149328B4-488E-4A23-AE7A-EBD4168DA4BD
DMs Tracker 1304    EB1B4658-428A-43CF-A639-E5C6A05F98BD
DMTools 1440    90DCF117-7B41-4EFA-A4E4-2A8FBF7EE7A8
DMTools 1440    DBFE5309-EBAE-45FB-A102-0E65241CA176
Dominion Kingdom Deck   2288    8EC09A3F-7ED5-4B64-91FE-7E2ECB419E9E
Dominion Kingdom Deck   2288    D8510380-3DBC-413E-AEE2-1DC06635614F
DOOM II RPG 63664   10993BCB-2DDD-4E83-B0DD-8C337ED26B38
DOOM II RPG 63664   98918726-D91C-4649-B4A8-7B8318921C22
Dragon Dictation    8032    4A069A76-01D3-4C89-80DC-F122507DFCA5
Dragon Dictation    8032    A01A60AA-B046-4266-84C8-BA85533B9D26
Dropbox 9464    13651CBD-1BB9-4A1F-B030-64D17055B08C
Dropbox 9288    E8DB5D21-4B30-4F1E-A359-4BCDD3AA2E33
Drync Wine Free 9416    183BBDED-A958-4523-95E1-549BDC1A92CA
Drync Wine Free 9416    D3442A6F-034C-4079-86A0-CDB1C1AC3671
eBay Mobile 15048   0F0DB0FA-ED53-4B7F-BB82-30F53950C38E
eBay Mobile 14840   82B35251-B297-4151-BEEF-290004743BE5
Edge    22224   375CB640-EFD8-44ED-95AE-7C2231F33665
Edge    22224   741E820F-0420-4178-9422-F34EBA8DDFEF
Epicurious Recipes & Shopping List  20928   05F11D7E-6526-489D-87CC-31C155C33EF1
Epicurious Recipes & Shopping List  20928   BE6D213A-2987-420E-B977-17C748D8C6F6
Episodes (formerly Episodes Lite)   6584    3A63EDFF-7982-432A-B329-41B35B4D18C4
Episodes (formerly Episodes Lite)   6568    D99F1C18-7C3D-477B-B84E-82B88D23A5CF
EPL Mobile  5264    91950828-48FB-46D8-9CBC-B36A4C3C3FF9
EPL Mobile  5264    B7F4A08F-59A8-40F9-BB9E-4E5FA182D803
eReader 10656   9927E6BE-776D-44C2-A45F-C9A03F24E3A7
eReader 10656   E8A46E21-C5E1-4F3D-AE74-4B6533413B98
Evernote    19328   0583A443-A16F-497D-9097-6028A6EB93DE
Evernote    19304   11236E21-D574-40F2-B221-CA5F5FCB6AC1
Flickr  22968   540969EB-651F-4F21-80A1-C30FF5E2E045
Flickr  22968   F0BC0045-72EC-467E-BDF6-A54C4E8221EC
Flight Control  18128   20C63388-6D8B-4E47-B8C5-2FA131A74850
Flight Control  18128   3032291F-F2BC-47AB-99AE-D72AA3342393
Food Network Canada Recipes, Grocery Lists & More   4096    60165498-47AD-4CD8-8C78-C4B896A4B382
Food Network Canada Recipes, Grocery Lists & More   4096    A55FF6E9-FFCE-4208-A458-7F0F39016870
Frenzic 10760   2D4664DE-A12D-49C8-9728-C8B7FDCCB670
Frenzic 10760   451AD935-9338-49B9-9614-8EB0797F93D7
Galcon Labs 14520   3E137685-632B-47D5-8462-35B69AF108B9
Galcon Labs 14520   418372B9-1C96-4658-A258-DA2B05C0B0FC
Gas Cubby FREE • Fuel Economy (MPG) & Vehicle Maintenance 5096    9B2FC5DD-3DD2-4596-A869-D188E19F06E4
Gas Cubby FREE • Fuel Economy (MPG) & Vehicle Maintenance 5096    E06E1706-0930-4D0A-BAC6-62F80D0793EC
GateGuru - featuring Airport Maps   33672   4E7986C9-A036-41F3-880C-D18B5038D982
GateGuru - featuring Airport Maps   33672   85714638-1A17-468E-9226-680BB7543443
GitHubby    6440    1819F7BD-76B4-42EC-AB7B-34D1907389D9
GitHubby    6424    44E4BC5A-434C-4FB8-8582-054CB3486C04
GMToolKit - RPG Helper  1864    0A011895-9CD6-4236-9F2B-6185E8A67601
GMToolKit - RPG Helper  1864    AAA2C17F-4E02-44FB-ADB5-E7D693F87851
GoodReader for iPhone   47024   06527C76-E1DF-4787-8AEE-19418FEBB57E
GoodReader for iPhone   47024   68B9EF26-C744-4707-B135-7AC941F82433
Goodreads   15896   448B7485-F28D-4722-BB47-C66439077005
Goodreads   15896   E030D52C-6EA7-4BFB-9528-EB44AD9C96F0
Google Earth    17816   898F55C7-2821-4C1C-BF61-04DDAF909F7C
Google Earth    17816   A0D01DBB-91FA-43CE-8215-6B7EE7C075BC
Google Mobile App   17280   57F00F23-D112-4B4D-A494-A6698EE74D7D
Google Mobile App   17160   E5A48717-3E2F-4B9B-983B-4805631486CF
Gravity Wars Free   576 941985C5-AA29-41E3-93A7-E7A267E6D27E
Gravity Wars Free   576 CFCEA74B-55E0-41B9-80EB-2B809CF4DEC2
Groceries   26352   090BFCE4-6B70-4D12-A523-7A1537913CD0
Groceries   26352   77F8485C-69B6-430B-ADD2-B0BAADCCE377
Grocery iQ  17576   A5F13AED-3BC3-4350-9292-6C3BCF0C5ED9
Grocery iQ  17576   E6BADEFB-D35B-436B-BE03-DB583801484D
Groove  10512   14A7F8B5-E42E-42FD-B2E5-A5EDE4BBC44F
Groove  10512   5EAA513C-A9AD-450B-B3BD-358185BA027C
Hand of Greed - dodge the blade!    32720   94ED2DA3-FF81-4128-8F13-880F9FFA4CC0
Hand of Greed - dodge the blade!    32720   FA9C2EF4-4E15-444A-9A3A-D447E514D227
Helsing's Fire  30152   121D7B08-9B22-4427-B238-59789379459C
Helsing's Fire  30152   E7EBA7B0-16AE-4932-8DD9-3A45D6F148E2
iBlast Moki 28464   52BFCD6A-A1E5-44FE-BD1B-58959548D1FE
iBlast Moki 28464   F824C865-E058-4B36-989B-EAE373F447D5
iBooks  23032   BE918F06-2814-40E6-BC00-5217EAF9ABF8
iBooks  21944   F582CD4C-839B-4E01-A66D-1EC6C75EE93C
iControlBits    2920    560A8219-02DD-487E-AEEA-DD3D55A1FBC7
iControlBits    2920    AA3238DC-2D99-4BED-AF33-D519ADC7928C
iDOS    29760   97C25E24-A7EF-4EBD-A897-3755E9DA768C
iDOS    29760   C119F2D8-2662-444C-8BF5-6AB07F49CCAF
Infinity Blade  872448  913BC358-63B1-4E03-B2FC-8D29EC994A76
Instagram   10648   BC112509-E107-4F8D-A889-EB85E869EEB7
Instagram   10256   674BCC8C-A54D-4422-AC78-732012ABC9D3
Instapaper  13792   DC5C90E5-80E7-4042-AFB9-2C7F756EF98E
Instapaper  13784   276769A6-B12A-4D6D-AFDE-59F52D6F84F6
Intelligentsia Coffee   55040   45D8F43C-F01D-4EBB-AAA6-ACBF5E6DF1F5
Intelligentsia Coffee   55040   EBBE9BA1-E083-40C9-9756-974E541D248F
iScores ScoreKeeper 592 8C0E577E-BA3E-4C71-AD9F-C5001B2DA2FC
iScores ScoreKeeper 592 E78F90FC-6C60-4D0F-AED8-EC46E724120E
iSlash  29256   010376E7-2B5B-4B4A-8AF6-DA94BA34A32C
iSlash  29256   E30B7438-9A8B-4EEC-81A3-4872C4EED5E5
iSSH - SSH / VNC Console    18080   1C248060-156E-41A5-A840-FD47F60EE6F1
iSSH - SSH / VNC Console    18080   E898BCE2-0626-4FB4-A8CF-7D678E452D7F
iTrade - Stock Market Simulator 4656    28B58EAB-F472-486D-8658-395823665925
iTrade - Stock Market Simulator 4656    85D79769-B43F-4EC3-B6E2-7B0CE14F6B83
Kijiji Free Local Classifieds   5176    A0A4082D-1560-45FF-9416-1DDFC90B1EBA
Kijiji Free Local Classifieds   5176    FEB85CB7-4890-4BAC-94ED-0ED20CDBAE30
Kindle  12072   10A38764-8DE7-4F3E-9F5A-104E9060363C
Kindle  12072   83648C0F-37DB-4604-9A2B-F97FE83A86DA
Labyrinth   24320   6E1CE06D-50B1-4872-BA77-425AD83E8630
Labyrinth   24320   AB1CAA69-FC4A-4390-9531-345BCB128610
LandFormer  7832    F8C81123-FE01-4F76-988B-4A11A2C51BEC
LandFormer  7832    FBF5AB58-2A19-4228-ABE7-3DB59AE8765F
LinkedIn    16936   3741B017-8E4C-4281-9B90-E6B39826C92F
LinkedIn    16936   808A95A9-745E-43B7-B96B-91ADF334F2BC
Local Books 1072    DC12DB09-CEBA-48D1-9820-25FAD56B9D3C
Local Books 1072    F2F2B434-EDB6-4975-90A7-2A473943EECC
Lux Touch   9360    2ED79730-DAF6-47AE-9DAC-517C6C7C06BA
Lux Touch   9360    4CADA890-8467-449B-B093-8500482B47FA
Meebo   14648   152D0B8D-E15B-4CC7-9E8B-4E718FC49D07
Meebo   14648   228F0FDB-C1D8-4942-A3D2-172F57048720
Meetup  6968    22556746-1926-47CA-BE22-2D498DAC8DAF
Mirror's Edgeâ„¢    110512  32FE8B6F-ADAA-488B-9818-2458AEE4FE63
Monkey Island 2 Special Edition: LeChuck's Revenge  502536  C5CD8CE8-7191-495C-9B50-F5B4CEE9C2BC
Monkey Island 2 Special Edition: LeChuck's Revenge  502536  C96859AE-B05E-42DA-A5A4-EC690642AC65
MTG life v2 456 631BC56E-8E8E-4267-8C77-C8F971A8A1AB
MTG life v2 456 7D132FE7-F1F4-4E2E-99FA-4A99223FF22E
Music Sleep Timer   648 114511F6-66B7-445F-BD6F-AD174560A1A3
Music Sleep Timer   648 C3387ACB-9FFB-4888-B6FB-FBE767EF96AC
myStarbucks Canada  53336   E7FFDB9B-A62E-455A-AD04-20F66F233DB6
NetNewsWire 16920   8214B10A-6EBA-4FE9-8251-C1F29C1D60FD
NetNewsWire 16920   9CF0A6BD-BC45-4DB4-BC1A-5DF3A6532DA5
OpenTable   3416    2D3B0542-2AF3-4AFC-B467-E7FD05322185
OpenTable   3416    88AA0F78-A75E-4815-B195-EEFF7017C5BE
ORBITAL 8216    62BFD091-309E-44EC-9A7C-BBDA7F794C8D
ORBITAL 8216    B08598E2-D58F-4E38-A347-CAA57125BE3A
Osmos   37688   1D513E64-FBAA-45E2-95C1-335C897AE995
Osmos   37688   CBA10BC6-EA07-48DD-9C9E-0628E85850D5
Pageonce Pro – Money & Bills  18760   2E53CD6F-0CC4-4C23-B644-14AD009447C0
Parcel  4048    2C378284-C8D0-492B-A0ED-9C6BC9EAAE8F
Parcel  3960    8B23552D-481D-4A83-B8BF-4A7A92D22E16
PCalc Lite Calculator   15512   2CB96138-2335-4041-AABB-020B69487607
PCalc Lite Calculator   15512   B11B95B1-F3D3-40FB-BB60-1EA6C0F86688
Photo Factory   10752   0C02C74F-B516-44C4-9B43-16F51FBB6764
Photo Factory   10752   B0414704-F6D3-4DDD-850F-545C635CAA69
PhotoWizard Lite    17800   81DDC79B-4ED5-46FC-BEE6-0B86C147259B
Point Inside Maps for Airports & Malls  4056    B99D5F4C-9B5D-4E8E-8F03-C1D24C1A4D9A
Qik Live    8544    10CE86E8-78BC-4624-89C6-1C6BEAFA50A9
Qik Live    8544    14C33F7E-F080-441E-9118-49DE77517070
QuickCam    2600    E079A408-62D1-40F3-AE18-5F801114B32D
RedLaser    4008    49E2B7D2-17A6-42C4-B2B6-3D563C03C993
RedLaser    4008    BCA7AB2E-C3E8-4D6A-8560-25BB0B250C7C
Reiner Knizia's Labyrinth   57792   559F589B-B6E4-49C3-B6C3-CE697A477DDA
Reiner Knizia's Labyrinth   57784   CD94AB93-8162-4BFA-904B-2209F5A4C9C8
Remote  20056   CA5B30F2-7309-4B7A-9A38-7502F6B22E2C
Remote  20056   EE33B399-7580-4BA0-9563-CAC52F223B0B
Rogers My Account   2160    72A6E277-3281-4F12-92B8-B0AEEC9268C6
Rogers My Account   2160    82A8C3C7-4B50-4320-8B38-2112E1816DFC
Rolando 37208   3059A3AC-3BE1-4E88-96C0-27F13A809A37
Route 411   1848    1CA89BCD-A40E-4B8E-A9AC-EE511EDB2650
Route 411   1848    3DD980E2-2993-4F7C-8AB3-EAF307B3CB03
RunKeeper Pro   15160   99AAE813-B51C-4EB0-A3BA-2B3F8CD44144
ScanLife    2024    29FF04CA-2417-428C-9D16-AF6EE84E8314
ScanLife    2024    E5C206EF-ECFB-400B-9212-E80592E1B81A
Shazam  10064   BEE325AF-52C7-498D-A90E-D6FED17DFFF9
Shazam  9128    0223970E-DA31-4C23-A091-C4AB1FB3842E
Showtimes - Local Movie Times & Tickets 5296    94596A89-040E-4746-94E9-67780BBF049F
Showtimes - Local Movie Times & Tickets 5288    2110A831-7444-4F79-8B6A-B3728A2BBE1F
Skype   25664   E90C54A6-C980-4981-B974-767E4C482D6B
SkySafari   244720  29E6AD95-4C15-4DC5-8A5A-C32A77BD3A69
Sleep Cycle alarm clock 9360    0563438A-AFE0-4BFE-BDDB-6891A8B28597
Sleep Cycle alarm clock 9296    AD40BB57-DEEE-4363-8A69-9F1F5599C339
Solomon's Boneyard  23664   920C1346-4F83-43EB-9464-DC72DE8A08C8
Sonar Ruler 11296   27FEC6FB-F590-4485-A958-F4036FF9F6C3
Stanza  266808  C903306C-131E-4B9F-86D4-5DC10A1A93C9
Stanza  177744  37030F82-EB56-4F02-9192-069B2A49D1B7
Strategery  9480    EDDFB064-EE24-4AF4-BE40-F12BB16C4133
Sway    15144   38815C72-3646-44AC-A2D4-9B93A2A3E5E2
Tangram Puzzle Pro  15848   10334041-F715-4798-AD7F-71CD21343F1F
TD  2152    66A965A3-13A3-4454-9FF0-C738CB6A0F4A
TD  2152    AE1A97BF-56FC-4997-AA39-C796BCECCC86
TimeTracker 1016    418C632B-B14B-4E8E-8FE5-5D25E3CCA976
TimeTracker 1016    75EF1359-2600-4C71-8222-28A763531A49
Tomes   9944    77768C1D-F2D0-45CE-8997-F7C3C9EA0DC5
Topple  24256   6FC3660E-D47E-448E-A57E-D4F70DBA64F8
TouchPad    12512   69B94087-B39B-4277-AD7D-46A0EAD89584
Trainyard   11768   E695028C-E514-4962-8D42-1AD018220E7B
Trainyard Express   10160   B3940AA5-7CC7-4F56-9F03-DA37BAEF3936
Trapster speed trap alerts (now with Caravan and Patrol)    9288    E8838E2F-F49D-478D-9AA0-4B702CAE986C
TripIt - Travel Organizer   10368   3BE579CB-27BE-40F4-8836-1D436F51F843
Tumblr  3656    AC7DC979-E269-42CA-863C-58B73E2A487D
Twitter 24552   E30BA165-716F-4257-8FE6-E2CCE9A1FA5A
Twitter 14640   50B49D57-6215-4FBE-8985-989BAD5DADC5
Undercroft  30800   34A8FC0A-15A3-41EF-BBBA-441DAF53639D
Urbanspoon  4840    060C7229-C634-491E-94BF-9728DC1D0AAD
VLC Media Player    10552   55CCFC88-54DD-480D-B0A6-99D33D2A35D3
WeatherEye  2528    C17CA73D-9B6C-4DF6-9652-0FD731F5A630
What's On TV? From Zap2it   20888   0B748EE1-42D8-4616-B911-045CDAE5EDD7
WolframAlpha    8296    0750BA4F-909F-4734-941F-1490D0B1EE61
WordPress   7816    754BD487-E232-400B-8CF1-45042CCBA4FD
Yum 1720    64B77483-1165-4954-849A-549D701B05E1
Yum 1720    77F30AC7-8F15-4310-8241-7FB16F97F69A

Here's the command I used to dig up this list (run in /var/mobile/Applications):

for f in *; do 
    if [ -f $f/iTunesMetadata.plist ]; then 
        echo -e -n "$(plutil -key itemName $f/iTunesMetadata.plist)\t"; 
    else 
        echo -e -n "No App\t"; 
    fi; 
    /usr/libexec/cydia/du -s $f; 
done

Some of these apps are not installed anymore -- Infinity Blade, for example -- and others are clearly installed twice. This is a problem, obviously, because space is precious on these machines.

How can I determine which apps iTunes believes to be installed, so that I can safely remove the unused duplicates?


Solution 1:

The hard easiest way would be to wipe your phone, and re-download all the apps again. You won't pay twice: iTunes is clever enough to know what you already downloaded.

But at this stage, I think you'll be better off. It also gives you the confidence that you've done something from scratch. And I think the time it takes will outweigh the time to clear out the list of duplicates.