How can I safely eliminate duplicate apps from my iPhone?
I have just had my iPhone replaced under warranty, and when restoring it from backup much of my preexisting free space vanished.
What I've found, after some shell hacking, is that I appear to have duplicate installs of several apps, some of which are quite large. Here is an alphabetically sorted list of these apps:
100 Rogues 50048 2B0EBC40-39DC-4EB9-AC13-21521314D202
100 Rogues 50048 8712B20E-CE00-4725-B4E8-5561C99C68E5
170,000+ Recipes - BigOven 8280 8A325E2D-2C79-40F3-A8D2-DA139824A982
170,000+ Recipes - BigOven 8200 ED9641E4-1258-4BA7-B4D4-5E4571E2E905
1Password Pro 15256 D83770FC-3021-47DD-A950-8BF2C8BB2526
1Password Pro 15104 84C1767D-F429-4E00-8D2B-74FF084E082D
2 Across 1600 07D94823-6768-4AC6-809A-4585AE21752A
2 Across 1600 52411FB8-F553-4E37-AA23-74884FAE176B
360 Live 8032 5CCDAE0B-1174-4A66-9C39-99F9EB45A3C0
360 Live 8032 C5D806B8-BB02-4CCF-96E8-F1615AD8B261
Angry Birds 43568 65C6EC54-A390-4C3A-A108-1E4BD15CE923
Angry Birds 43568 B289BCB4-FE4E-4AD9-879E-F257ED20E713
Angry Birds Seasons 32968 4B8A5E2C-4554-4EDF-A89E-6E3AE4AC1280
ArithmeGrid Full 1040 6FE96AF0-5261-4A9F-A643-CA6C18498D9E
ArithmeGrid Full 1040 F513D748-F5CA-482F-9586-E6C48383F06F
AroundMe 6512 BA1802E7-2C58-443D-AF2B-5699584A6EBD
AroundMe 6504 F97E85F1-9F2A-403E-AD6A-D14F6885E3AE
Ash 89808 1DAECBA5-80A5-4461-81B4-799B438F1402
Ash 89808 969F7768-69CF-496F-8138-9F6C75D43B1B
Battleheart 239520 236C27CF-6222-47C0-9BE7-EBE0C737A5C8
Battleheart 239520 816A8155-8C75-4F7B-8AC5-CA2CDA29F938
Beneath a Steel Sky: Remastered 167136 D4CC69B4-3DB6-4A8D-B2E5-18A82DA601B8
Boxcar 23536 CA71F1C9-1ED4-413E-BB36-2A5B26A367F3
Boxcar 23256 89C3C612-C543-4369-8FA0-5D325C63BA67
Camera+​ 45336 015B6087-7514-4249-81AB-57002F8B4A64
Camera+​ 45336 C800AF53-6759-4183-8A2D-248062781C11
Canadian YellowPages.ca Business and People Finder 3376 D5AE87DE-37CB-4BA9-B2F3-6070D898B5C6
Carcassonne 146608 CAC1C704-DE9F-4051-89DB-7645747D32D9
Carcassonne 146608 D4BB7015-FB72-43C6-B14E-E55DBFF29B7B
CBC Radio 5432 5F9A4FEB-46CF-48ED-A02B-F486C6517FF4
CBC Radio 5432 716B93AA-0642-4B9B-9A65-5CFDD42965F0
Civilization Revolution 98544 2FD14644-14CC-449D-9976-A4AFD5233AD7
Civilization Revolution 98544 C8A3CD0E-D2E2-41EA-946C-254D545F7CD2
ClearCam 271608 86FB6803-D5B6-4BDC-BC53-65FFCBF43D05
ClearCam 271608 96BEFE40-CB4D-45EE-9509-4A8E61628EA6
Cocktails+ 21432 79E914F0-8998-4B4F-A51E-9326B33593AB
Cocktails+ 21432 CA29F2B6-2F5D-4731-A7AA-012AADDEA1BA
Cogs 42856 8648A128-F9EB-4D15-B2FE-D819120DF4E9
Cogs 42856 A44BE8FA-069F-40DE-B4B9-E97F84957CD3
CompendiumHelper 312 81A5C784-62E3-4A5C-8F51-4A09848A6D83
CompendiumHelper 312 F5CDBA2C-0AB4-42B5-AF66-99B2C4EEE6C4
Consume 15984 685355AF-7CF6-47B1-9046-8E640CA24CEF
Consume 15984 E26E2146-D6EA-4B74-9A34-0603F4211371
Controllr ~ Plex remote 11928 545E4696-9407-47E7-8D2D-6BD88B09AB21
Controllr ~ Plex remote 11928 DB608992-3798-476B-BE78-98858B15D664
Convertbot — The Amazing Unit Converter 10352 822F6C91-056D-4257-8A11-D779D7422177
Convertbot — The Amazing Unit Converter 10352 EA24DE01-51F1-4081-91C1-70829EC9DDE0
Convore 6808 80F540C1-00AB-4DDD-AD72-72A4F22B2548
Convore 6808 F666C5C6-9128-4712-AE1F-912A55AEEDC7
Cook’s Illustrated 3544 E3FDD73D-FE51-4B64-9BAB-5582C9EFB390
Cook’s Illustrated 3544 EF6B0310-447C-4647-90BE-B9659469E058
Cookmate 4584 6A2ED1F4-F57C-45E7-B566-77B2D5CDC52D
Cookmate 4584 9038D2F1-06AD-44A2-B4FE-7C2879E587DC
Craigsphone - craigslist for iphone 4872 573D30E7-D547-4271-9EE3-AC0E3D5FA270
Craigsphone - craigslist for iphone 4872 73ED9FAE-DD16-4B37-8830-03DC8E7D3D91
Crush the Castle 15440 5C602CCA-F90F-4DD9-89E9-F0759F9BFF4A
Crush the Castle 15440 72AAAC2C-10B1-47E2-9065-2EE00207631B
Dice Bag 2008 8DC4EEE6-E136-4C1E-B3C3-180CF2557BBA
Dice Bag 2008 A153B606-ADD2-4643-BC81-9CB7D762BB29
Dictionary.com - Dictionary & Thesaurus - Free 142160 3407CC86-AEF8-441D-B347-35DC899B0BF9
Dictionary.com - Dictionary & Thesaurus - Free 142160 C50C4E97-209E-4577-BB78-FE7022110710
DMs Tracker 1304 149328B4-488E-4A23-AE7A-EBD4168DA4BD
DMs Tracker 1304 EB1B4658-428A-43CF-A639-E5C6A05F98BD
DMTools 1440 90DCF117-7B41-4EFA-A4E4-2A8FBF7EE7A8
DMTools 1440 DBFE5309-EBAE-45FB-A102-0E65241CA176
Dominion Kingdom Deck 2288 8EC09A3F-7ED5-4B64-91FE-7E2ECB419E9E
Dominion Kingdom Deck 2288 D8510380-3DBC-413E-AEE2-1DC06635614F
DOOM II RPG 63664 10993BCB-2DDD-4E83-B0DD-8C337ED26B38
DOOM II RPG 63664 98918726-D91C-4649-B4A8-7B8318921C22
Dragon Dictation 8032 4A069A76-01D3-4C89-80DC-F122507DFCA5
Dragon Dictation 8032 A01A60AA-B046-4266-84C8-BA85533B9D26
Dropbox 9464 13651CBD-1BB9-4A1F-B030-64D17055B08C
Dropbox 9288 E8DB5D21-4B30-4F1E-A359-4BCDD3AA2E33
Drync Wine Free 9416 183BBDED-A958-4523-95E1-549BDC1A92CA
Drync Wine Free 9416 D3442A6F-034C-4079-86A0-CDB1C1AC3671
eBay Mobile 15048 0F0DB0FA-ED53-4B7F-BB82-30F53950C38E
eBay Mobile 14840 82B35251-B297-4151-BEEF-290004743BE5
Edge 22224 375CB640-EFD8-44ED-95AE-7C2231F33665
Edge 22224 741E820F-0420-4178-9422-F34EBA8DDFEF
Epicurious Recipes & Shopping List 20928 05F11D7E-6526-489D-87CC-31C155C33EF1
Epicurious Recipes & Shopping List 20928 BE6D213A-2987-420E-B977-17C748D8C6F6
Episodes (formerly Episodes Lite) 6584 3A63EDFF-7982-432A-B329-41B35B4D18C4
Episodes (formerly Episodes Lite) 6568 D99F1C18-7C3D-477B-B84E-82B88D23A5CF
EPL Mobile 5264 91950828-48FB-46D8-9CBC-B36A4C3C3FF9
EPL Mobile 5264 B7F4A08F-59A8-40F9-BB9E-4E5FA182D803
eReader 10656 9927E6BE-776D-44C2-A45F-C9A03F24E3A7
eReader 10656 E8A46E21-C5E1-4F3D-AE74-4B6533413B98
Evernote 19328 0583A443-A16F-497D-9097-6028A6EB93DE
Evernote 19304 11236E21-D574-40F2-B221-CA5F5FCB6AC1
Flickr 22968 540969EB-651F-4F21-80A1-C30FF5E2E045
Flickr 22968 F0BC0045-72EC-467E-BDF6-A54C4E8221EC
Flight Control 18128 20C63388-6D8B-4E47-B8C5-2FA131A74850
Flight Control 18128 3032291F-F2BC-47AB-99AE-D72AA3342393
Food Network Canada Recipes, Grocery Lists & More 4096 60165498-47AD-4CD8-8C78-C4B896A4B382
Food Network Canada Recipes, Grocery Lists & More 4096 A55FF6E9-FFCE-4208-A458-7F0F39016870
Frenzic 10760 2D4664DE-A12D-49C8-9728-C8B7FDCCB670
Frenzic 10760 451AD935-9338-49B9-9614-8EB0797F93D7
Galcon Labs 14520 3E137685-632B-47D5-8462-35B69AF108B9
Galcon Labs 14520 418372B9-1C96-4658-A258-DA2B05C0B0FC
Gas Cubby FREE • Fuel Economy (MPG) & Vehicle Maintenance 5096 9B2FC5DD-3DD2-4596-A869-D188E19F06E4
Gas Cubby FREE • Fuel Economy (MPG) & Vehicle Maintenance 5096 E06E1706-0930-4D0A-BAC6-62F80D0793EC
GateGuru - featuring Airport Maps 33672 4E7986C9-A036-41F3-880C-D18B5038D982
GateGuru - featuring Airport Maps 33672 85714638-1A17-468E-9226-680BB7543443
GitHubby 6440 1819F7BD-76B4-42EC-AB7B-34D1907389D9
GitHubby 6424 44E4BC5A-434C-4FB8-8582-054CB3486C04
GMToolKit - RPG Helper 1864 0A011895-9CD6-4236-9F2B-6185E8A67601
GMToolKit - RPG Helper 1864 AAA2C17F-4E02-44FB-ADB5-E7D693F87851
GoodReader for iPhone 47024 06527C76-E1DF-4787-8AEE-19418FEBB57E
GoodReader for iPhone 47024 68B9EF26-C744-4707-B135-7AC941F82433
Goodreads 15896 448B7485-F28D-4722-BB47-C66439077005
Goodreads 15896 E030D52C-6EA7-4BFB-9528-EB44AD9C96F0
Google Earth 17816 898F55C7-2821-4C1C-BF61-04DDAF909F7C
Google Earth 17816 A0D01DBB-91FA-43CE-8215-6B7EE7C075BC
Google Mobile App 17280 57F00F23-D112-4B4D-A494-A6698EE74D7D
Google Mobile App 17160 E5A48717-3E2F-4B9B-983B-4805631486CF
Gravity Wars Free 576 941985C5-AA29-41E3-93A7-E7A267E6D27E
Gravity Wars Free 576 CFCEA74B-55E0-41B9-80EB-2B809CF4DEC2
Groceries 26352 090BFCE4-6B70-4D12-A523-7A1537913CD0
Groceries 26352 77F8485C-69B6-430B-ADD2-B0BAADCCE377
Grocery iQ 17576 A5F13AED-3BC3-4350-9292-6C3BCF0C5ED9
Grocery iQ 17576 E6BADEFB-D35B-436B-BE03-DB583801484D
Groove 10512 14A7F8B5-E42E-42FD-B2E5-A5EDE4BBC44F
Groove 10512 5EAA513C-A9AD-450B-B3BD-358185BA027C
Hand of Greed - dodge the blade! 32720 94ED2DA3-FF81-4128-8F13-880F9FFA4CC0
Hand of Greed - dodge the blade! 32720 FA9C2EF4-4E15-444A-9A3A-D447E514D227
Helsing's Fire 30152 121D7B08-9B22-4427-B238-59789379459C
Helsing's Fire 30152 E7EBA7B0-16AE-4932-8DD9-3A45D6F148E2
iBlast Moki 28464 52BFCD6A-A1E5-44FE-BD1B-58959548D1FE
iBlast Moki 28464 F824C865-E058-4B36-989B-EAE373F447D5
iBooks 23032 BE918F06-2814-40E6-BC00-5217EAF9ABF8
iBooks 21944 F582CD4C-839B-4E01-A66D-1EC6C75EE93C
iControlBits 2920 560A8219-02DD-487E-AEEA-DD3D55A1FBC7
iControlBits 2920 AA3238DC-2D99-4BED-AF33-D519ADC7928C
iDOS 29760 97C25E24-A7EF-4EBD-A897-3755E9DA768C
iDOS 29760 C119F2D8-2662-444C-8BF5-6AB07F49CCAF
Infinity Blade 872448 913BC358-63B1-4E03-B2FC-8D29EC994A76
Instagram 10648 BC112509-E107-4F8D-A889-EB85E869EEB7
Instagram 10256 674BCC8C-A54D-4422-AC78-732012ABC9D3
Instapaper 13792 DC5C90E5-80E7-4042-AFB9-2C7F756EF98E
Instapaper 13784 276769A6-B12A-4D6D-AFDE-59F52D6F84F6
Intelligentsia Coffee 55040 45D8F43C-F01D-4EBB-AAA6-ACBF5E6DF1F5
Intelligentsia Coffee 55040 EBBE9BA1-E083-40C9-9756-974E541D248F
iScores ScoreKeeper 592 8C0E577E-BA3E-4C71-AD9F-C5001B2DA2FC
iScores ScoreKeeper 592 E78F90FC-6C60-4D0F-AED8-EC46E724120E
iSlash 29256 010376E7-2B5B-4B4A-8AF6-DA94BA34A32C
iSlash 29256 E30B7438-9A8B-4EEC-81A3-4872C4EED5E5
iSSH - SSH / VNC Console 18080 1C248060-156E-41A5-A840-FD47F60EE6F1
iSSH - SSH / VNC Console 18080 E898BCE2-0626-4FB4-A8CF-7D678E452D7F
iTrade - Stock Market Simulator 4656 28B58EAB-F472-486D-8658-395823665925
iTrade - Stock Market Simulator 4656 85D79769-B43F-4EC3-B6E2-7B0CE14F6B83
Kijiji Free Local Classifieds 5176 A0A4082D-1560-45FF-9416-1DDFC90B1EBA
Kijiji Free Local Classifieds 5176 FEB85CB7-4890-4BAC-94ED-0ED20CDBAE30
Kindle 12072 10A38764-8DE7-4F3E-9F5A-104E9060363C
Kindle 12072 83648C0F-37DB-4604-9A2B-F97FE83A86DA
Labyrinth 24320 6E1CE06D-50B1-4872-BA77-425AD83E8630
Labyrinth 24320 AB1CAA69-FC4A-4390-9531-345BCB128610
LandFormer 7832 F8C81123-FE01-4F76-988B-4A11A2C51BEC
LandFormer 7832 FBF5AB58-2A19-4228-ABE7-3DB59AE8765F
LinkedIn 16936 3741B017-8E4C-4281-9B90-E6B39826C92F
LinkedIn 16936 808A95A9-745E-43B7-B96B-91ADF334F2BC
Local Books 1072 DC12DB09-CEBA-48D1-9820-25FAD56B9D3C
Local Books 1072 F2F2B434-EDB6-4975-90A7-2A473943EECC
Lux Touch 9360 2ED79730-DAF6-47AE-9DAC-517C6C7C06BA
Lux Touch 9360 4CADA890-8467-449B-B093-8500482B47FA
Meebo 14648 152D0B8D-E15B-4CC7-9E8B-4E718FC49D07
Meebo 14648 228F0FDB-C1D8-4942-A3D2-172F57048720
Meetup 6968 22556746-1926-47CA-BE22-2D498DAC8DAF
Mirror's Edgeâ„¢ 110512 32FE8B6F-ADAA-488B-9818-2458AEE4FE63
Monkey Island 2 Special Edition: LeChuck's Revenge 502536 C5CD8CE8-7191-495C-9B50-F5B4CEE9C2BC
Monkey Island 2 Special Edition: LeChuck's Revenge 502536 C96859AE-B05E-42DA-A5A4-EC690642AC65
MTG life v2 456 631BC56E-8E8E-4267-8C77-C8F971A8A1AB
MTG life v2 456 7D132FE7-F1F4-4E2E-99FA-4A99223FF22E
Music Sleep Timer 648 114511F6-66B7-445F-BD6F-AD174560A1A3
Music Sleep Timer 648 C3387ACB-9FFB-4888-B6FB-FBE767EF96AC
myStarbucks Canada 53336 E7FFDB9B-A62E-455A-AD04-20F66F233DB6
NetNewsWire 16920 8214B10A-6EBA-4FE9-8251-C1F29C1D60FD
NetNewsWire 16920 9CF0A6BD-BC45-4DB4-BC1A-5DF3A6532DA5
OpenTable 3416 2D3B0542-2AF3-4AFC-B467-E7FD05322185
OpenTable 3416 88AA0F78-A75E-4815-B195-EEFF7017C5BE
ORBITAL 8216 62BFD091-309E-44EC-9A7C-BBDA7F794C8D
ORBITAL 8216 B08598E2-D58F-4E38-A347-CAA57125BE3A
Osmos 37688 1D513E64-FBAA-45E2-95C1-335C897AE995
Osmos 37688 CBA10BC6-EA07-48DD-9C9E-0628E85850D5
Pageonce Pro – Money & Bills 18760 2E53CD6F-0CC4-4C23-B644-14AD009447C0
Parcel 4048 2C378284-C8D0-492B-A0ED-9C6BC9EAAE8F
Parcel 3960 8B23552D-481D-4A83-B8BF-4A7A92D22E16
PCalc Lite Calculator 15512 2CB96138-2335-4041-AABB-020B69487607
PCalc Lite Calculator 15512 B11B95B1-F3D3-40FB-BB60-1EA6C0F86688
Photo Factory 10752 0C02C74F-B516-44C4-9B43-16F51FBB6764
Photo Factory 10752 B0414704-F6D3-4DDD-850F-545C635CAA69
PhotoWizard Lite 17800 81DDC79B-4ED5-46FC-BEE6-0B86C147259B
Point Inside Maps for Airports & Malls 4056 B99D5F4C-9B5D-4E8E-8F03-C1D24C1A4D9A
Qik Live 8544 10CE86E8-78BC-4624-89C6-1C6BEAFA50A9
Qik Live 8544 14C33F7E-F080-441E-9118-49DE77517070
QuickCam 2600 E079A408-62D1-40F3-AE18-5F801114B32D
RedLaser 4008 49E2B7D2-17A6-42C4-B2B6-3D563C03C993
RedLaser 4008 BCA7AB2E-C3E8-4D6A-8560-25BB0B250C7C
Reiner Knizia's Labyrinth 57792 559F589B-B6E4-49C3-B6C3-CE697A477DDA
Reiner Knizia's Labyrinth 57784 CD94AB93-8162-4BFA-904B-2209F5A4C9C8
Remote 20056 CA5B30F2-7309-4B7A-9A38-7502F6B22E2C
Remote 20056 EE33B399-7580-4BA0-9563-CAC52F223B0B
Rogers My Account 2160 72A6E277-3281-4F12-92B8-B0AEEC9268C6
Rogers My Account 2160 82A8C3C7-4B50-4320-8B38-2112E1816DFC
Rolando 37208 3059A3AC-3BE1-4E88-96C0-27F13A809A37
Route 411 1848 1CA89BCD-A40E-4B8E-A9AC-EE511EDB2650
Route 411 1848 3DD980E2-2993-4F7C-8AB3-EAF307B3CB03
RunKeeper Pro 15160 99AAE813-B51C-4EB0-A3BA-2B3F8CD44144
ScanLife 2024 29FF04CA-2417-428C-9D16-AF6EE84E8314
ScanLife 2024 E5C206EF-ECFB-400B-9212-E80592E1B81A
Shazam 10064 BEE325AF-52C7-498D-A90E-D6FED17DFFF9
Shazam 9128 0223970E-DA31-4C23-A091-C4AB1FB3842E
Showtimes - Local Movie Times & Tickets 5296 94596A89-040E-4746-94E9-67780BBF049F
Showtimes - Local Movie Times & Tickets 5288 2110A831-7444-4F79-8B6A-B3728A2BBE1F
Skype 25664 E90C54A6-C980-4981-B974-767E4C482D6B
SkySafari 244720 29E6AD95-4C15-4DC5-8A5A-C32A77BD3A69
Sleep Cycle alarm clock 9360 0563438A-AFE0-4BFE-BDDB-6891A8B28597
Sleep Cycle alarm clock 9296 AD40BB57-DEEE-4363-8A69-9F1F5599C339
Solomon's Boneyard 23664 920C1346-4F83-43EB-9464-DC72DE8A08C8
Sonar Ruler 11296 27FEC6FB-F590-4485-A958-F4036FF9F6C3
Stanza 266808 C903306C-131E-4B9F-86D4-5DC10A1A93C9
Stanza 177744 37030F82-EB56-4F02-9192-069B2A49D1B7
Strategery 9480 EDDFB064-EE24-4AF4-BE40-F12BB16C4133
Sway 15144 38815C72-3646-44AC-A2D4-9B93A2A3E5E2
Tangram Puzzle Pro 15848 10334041-F715-4798-AD7F-71CD21343F1F
TD 2152 66A965A3-13A3-4454-9FF0-C738CB6A0F4A
TD 2152 AE1A97BF-56FC-4997-AA39-C796BCECCC86
TimeTracker 1016 418C632B-B14B-4E8E-8FE5-5D25E3CCA976
TimeTracker 1016 75EF1359-2600-4C71-8222-28A763531A49
Tomes 9944 77768C1D-F2D0-45CE-8997-F7C3C9EA0DC5
Topple 24256 6FC3660E-D47E-448E-A57E-D4F70DBA64F8
TouchPad 12512 69B94087-B39B-4277-AD7D-46A0EAD89584
Trainyard 11768 E695028C-E514-4962-8D42-1AD018220E7B
Trainyard Express 10160 B3940AA5-7CC7-4F56-9F03-DA37BAEF3936
Trapster speed trap alerts (now with Caravan and Patrol) 9288 E8838E2F-F49D-478D-9AA0-4B702CAE986C
TripIt - Travel Organizer 10368 3BE579CB-27BE-40F4-8836-1D436F51F843
Tumblr 3656 AC7DC979-E269-42CA-863C-58B73E2A487D
Twitter 24552 E30BA165-716F-4257-8FE6-E2CCE9A1FA5A
Twitter 14640 50B49D57-6215-4FBE-8985-989BAD5DADC5
Undercroft 30800 34A8FC0A-15A3-41EF-BBBA-441DAF53639D
Urbanspoon 4840 060C7229-C634-491E-94BF-9728DC1D0AAD
VLC Media Player 10552 55CCFC88-54DD-480D-B0A6-99D33D2A35D3
WeatherEye 2528 C17CA73D-9B6C-4DF6-9652-0FD731F5A630
What's On TV? From Zap2it 20888 0B748EE1-42D8-4616-B911-045CDAE5EDD7
WolframAlpha 8296 0750BA4F-909F-4734-941F-1490D0B1EE61
WordPress 7816 754BD487-E232-400B-8CF1-45042CCBA4FD
Yum 1720 64B77483-1165-4954-849A-549D701B05E1
Yum 1720 77F30AC7-8F15-4310-8241-7FB16F97F69A
Here's the command I used to dig up this list (run in /var/mobile/Applications):
for f in *; do
if [ -f $f/iTunesMetadata.plist ]; then
echo -e -n "$(plutil -key itemName $f/iTunesMetadata.plist)\t";
else
echo -e -n "No App\t";
fi;
/usr/libexec/cydia/du -s $f;
done
Some of these apps are not installed anymore -- Infinity Blade, for example -- and others are clearly installed twice. This is a problem, obviously, because space is precious on these machines.
How can I determine which apps iTunes believes to be installed, so that I can safely remove the unused duplicates?
Solution 1:
The hard easiest way would be to wipe your phone, and re-download all the apps again. You won't pay twice: iTunes is clever enough to know what you already downloaded.
But at this stage, I think you'll be better off. It also gives you the confidence that you've done something from scratch. And I think the time it takes will outweigh the time to clear out the list of duplicates.