What is the most efficient, native way to image a Windows partition?

Solution 1:

What is the most efficient, native way to image a Windows partition?

For the majority of users, capturing an image of a Windows partition via DISM (Win XP ≤ 7: ImageX) is generally the best and most efficient method, while also not causing the configuration issues all too common with third-party tools.

• Windows XP ≥ 10 has always natively supported imaging of partitions and filesystems:
  • Imaging the System partition is slightly different than other partitions, as it can only be imaged from WinPE/WinRE
  • While ancient, Microsoft's Windows Imaging File Format whitepaper explains the WIM format
    • WIMs (Windows IMage) can capture an entire partition or individual folders/files
    • ESDs (Electronic Software Distribution) can only capture a System partition and must use /Compress:Recovery (algorithm is ~33% more efficient than /Compress:Max)
      • Windows ≥ 10: Can only be used for PBR [Push-Button Reset] exported images
        Windows ≤ 8.1: Only a bootable Windows install can be captured as an ESD
        
        
• All WinPE/WinRE WIMs have DISM included within them (Win XP ≤ 7: ImageX):
  • WinPE: Windows Preinstallation Environment
    (Windows Setup boot media: SHIFT+F10 to access terminal)
  • WinRE: Windows Recovery Environment
    (WinRE is a WinPE image containing extra WinPE Optional Components vital to recovery)
    
    

Annotation:

• Many have taken issue with the "image" nomenclature, with "image" [per Microsoft] being the correct terminology
  • It's not up to an individual to change a developer's nomenclature and if I was to arbitrarily and individually change the nomenclature, it would only sow more confusion when referencing Microsoft Docs [Windows' man pages]
• While I can't definitively point to any specific Windows whitepaper, Windows' "image" nomenclature likely comes from how Windows is referred to from a servicing standpoint, which is as an "image", and is why DISM has the /Online and /Image parameters:
  • Online image servicing deals with a %SystemDrive% while booted to it
  • Offline image servicing deals with a non-booted to %SystemDrive%
  • Image management deals with the topic of this answer
    
    

Imaging:

(Powershell cmdlet mapping)

Specify exclusions or exceptions by creating a WimScript.ini config file, with /ScratchDir being required in WinPE since it only has 32MB of scratch [temp] space by default:

1. Either Capture or Append an image:
   • Capture Image:
     

     # Windows ≥8: DISM
       Dism /Capture-Image /ImageFile:"Z:\Base.wim" /CaptureDir:"C:" /Name:"Windows Backup" /Description:"Base Image 2020.08.29 @ 11:30" /Compress:Max /CheckIntegrity /Verify /ScratchDir:"Z:"
     
     # Windows XP ≤ 7: ImageX
       ImageX /Capture "C:" "Z:\Base.esd" "Windows Backup" "Base Image 2020.08.29 @ 11:30" /Compress:Recovery /Check /Verify /ScratchDir:"Z:\"
     

     • Change /Compress:Max to /Compress:Fast if not saving captured image to an SSD
     • For managing size constraints, images can be split into multiple read-only .swm files via /Split-Image
   • Append Image:

     # Windows ≥8: DISM
       Dism /Append-Image /ImageFile:"Z:\Base.wim" /CaptureDir:"C:" /Name:"Windows Backup" /Description:"Base Image 2020.08.29 @ 11:30" /CheckIntegrity /Verify /ScratchDir:"Z:"
     
     # Windows XP ≤ 7: ImageX
       ImageX /Append "C:" "Z:\Base.esd" "Windows Backup" "Base Image 2020.08.29 @ 11:30" /Compress:Recovery /Check /Verify /ScratchDir:"Z:\"
     

     • Compression is locked to the value set when the base image was captured
     • Individual indexes can be deleted via /Delete-Image or exported to their own image via /Export-Image
       
       
2. Apply Image:

   # Windows ≥8: DISM
     Dism /Apply-Image /ImageFile:"Z:\Base.wim" /Index:1 /ApplyDir:"C:" /CheckIntegrity /Verify /ScratchDir:"Z:"
   
   # Windows XP ≤ 7: ImageX
     ImageX /Apply "Z:\Base.wim" 1 "C:" /Check /Verify /ScratchDir:"Z:\"
   

   • Prior to applying, get Image Info, ensuring correct index [image] is being applied:

     Dism /Get-ImageInfo /ImageFile:"Z:\Base.wim"
     

   • If applying an OS image, the following must be run prior to exiting WinPE/WinRE:
     • BIOS:

       BootRec /FixMBR && BootRec /FixBoot && BootRec /RebuildBCD
       

     • UEFI:

       ::# With existing bootable EFI partition:
           BootRec /FixMBR && BootRec /RebuildBCD
       
       
       ::# Without existing bootable EFI partition:
           ::# Create EFI directories and enter:
               MkDir "Y:\EFI\Microsoft\Boot"
               Cd /d "Y:\EFI\Microsoft\Boot"
       
           ::# Create EFI boot structure:
               BootRec /Fixboot
       
               ::# If Access Denied error occurs (C: is applied image):
                   BcdBoot C:\Windows /s C: /f UEFI
       
           ::# Resolve any other boot issues:
               BootRec /FixMBR && BootRec /RebuildBCD
       

Accessing data within a WIM or ESD:

1. Read-only:

   1. Mount Image: (as /ReadOnly)

      # Windows ≥8: DISM
        Dism /Mount-Image /ImageFile:"Z:\Base.wim" /Index:2 /MountDir:"C:\Mount" /Optimize /CheckIntegrity /ReadOnly
      
      # Windows XP ≤ 7: ImageX
        ImageX /Mount "Z:\Base.wim" 2 "C:\Mount" /Check
      

      • In lieu of this, I prefer opening the .wim/.esd within the 7zip GUI
   2. Unmount Image: (/discard changes)

      # Windows ≥8: DISM
        Dism /Unmount-Image /MountDir:"C:\Mount" /CheckIntegrity /Discard
      
      # Windows XP ≤ 7: ImageX
        ImageX /Unmount "C:\Mount"
      

      
      

2. Make changes, or add data, to an image [index]:

   1. Mount Image:

      # Windows ≥8: DISM
        Dism /Mount-Image /ImageFile:"Z:\Base.wim" /Index:2 /MountDir:"C:\Mount" /Optimize /CheckIntegrity
      
      # Windows XP ≤ 7: ImageX
        ImageX /MountRW "Z:\Base.wim" 2 "C:\Mount" /Check
      

   2. Unmount Image: (/Commit changes)

      # Windows ≥8: DISM
        Dism /Unmount-Image /MountDir:"C:\Mount" /CheckIntegrity /Commit
      
      # Windows XP ≤ 7: ImageX
        ImageX /Unmount "C:\Mount" /Commit
      

      • If using DISM, to save changes as a new appended image, add /Append
        
        

Why is the native method generally the best method for most users?

• No additional imaging tools or boot media is required since support is natively built-in to Windows
• Its impossible for WIMs/ESDs to become corrupted, provided /CheckIntegrity (ImageX: /Check) & /Verify are always used
• Capturing and applying a .wim/.esd is not the best solution for all imaging [cloning] use cases, but is for most:
  • Capturing a .wim/.esd requires a storage medium to house the captured image (partition not being imaged, USB drive, network share, etc.), serving the dual purpose of also being an actual backup base image
  • Additional backups of the same partition can be appended to the base image with minimal [relative] file size increase, and while data from multiple partitions can be appended to the same base image, the smart compression feature benefits WIMs provide would be lost
    
    
• WIMs/ESDs are smart compression image formats and therefore storage efficient:
  • Only changed files are added to a .wim/.esd when a new image [index] is appended to it; newly appended images utilize the same copy of unchanged files already contained within the image from the previous image(s) (hash verified), allowing for an image to remain small in relation to the data within
    
    Appended image example:
    (note Base.wim size compared to each contained index and sum of all data therein):

    PS $  Ls -File
    
          Directory: Z:\WIM
    
            Mode                LastWriteTime            Length  Name
            ----                -------------            ------  ----
            -a----        2018.12.24 03:34:13   95,019,530,773B  Base.wim
            -a----        2016.06.14 22:32:36              568B  Dism.cmd
            -a----        2016.05.17 05:36:10               97B  WimScript.ini
    
    
    PS $  Dism /Get-ImageInfo /ImageFile:"Base.wim"
    
          Deployment Image Servicing and Management tool
          Version: 10.0.19041.329
    
            Details for image : Base.wim
    
            Index : 1
              Name : Alienware 18: Windows 10
              Description : v1803: Base (Drivers Only)
              Size : 22,710,283,446 bytes
    
            Index : 2
              Name : Alienware 18: Windows 10
              Description : v1803: Software Installed (No Customizations)
              Size : 45,591,850,754 bytes
    
            Index : 3
              Name : Alienware 18: Windows 10
              Description : v1803: Software Installed (Customized)
              Size : 94,958,267,312 bytes
    
            Index : 4
              Name : Alienware 18: Windows 10
              Description : v1803: Software Group 1 Installed (Customized)
              Size : 101,588,267,910 bytes
    
            Index : 5
              Name : Alienware 18: Windows 10
              Description : v1803: Software Group 2 Installed (Customized)
              Size : 101,905,314,237 bytes
    
            Index : 6
              Name : Alienware 18: Windows 10
              Description : v1809: Updated Applications
              Size : 114,959,954,040 bytes
    

    PS $  Dism /Get-ImageInfo /ImageFile:"Base.wim" /Index:1
    
          Deployment Image Servicing and Management tool
          Version: 10.0.19041.329
    
            Details for image : Base.wim
    
              Index : 1
              Name : Alienware 18: Windows 10
              Description : v1803: Base (Drivers Only)
              Size : 22,710,283,446 bytes
              WIM Bootable : No
              Architecture : x64
              Hal : acpiapic
              Version : 10.0.17134
              ServicePack Build : 1
              ServicePack Level : 1
              Edition : Professional
              Installation : Client
              ProductType : WinNT
              ProductSuite : Terminal Server
              System Root : WINDOWS
              Directories : 24288
              Files : 112665
              Created : 2018.05.05 - 13:56:47
              Modified : 2018.05.05 - 13:56:47
              Languages : en-US (Default)
    
    
    PS $  Dism /Get-ImageInfo /ImageFile:"Base.wim" /Index:2
    
          Deployment Image Servicing and Management tool
          Version: 10.0.19041.329
    
            Details for image : Base.wim
    
              Index : 2
              Name : Alienware 18: Windows 10
              Description : v1803: Software Installed (No Customizations)
              Size : 45,591,850,754 bytes
              WIM Bootable : No
              Architecture : x64
              Hal : acpiapic
              Version : 10.0.17134
              ServicePack Build : 1
              ServicePack Level : 1
              Edition : Professional
              Installation : Client
              ProductType : WinNT
              ProductSuite : Terminal Server
              System Root : WINDOWS
              Directories : 45803
              Files : 203058
              Created : 2018.05.06 - 01:55:47
              Modified : 2018.05.06 - 01:55:48
              Languages : en-US (Default)
    
    
    PS $  Dism /Get-ImageInfo /ImageFile:"Base.wim" /Index:3
    
          Deployment Image Servicing and Management tool
          Version: 10.0.19041.329
    
            Details for image : Base.wim
    
              Index : 3
              Name : Alienware 18: Windows 10
              Description : v1803: Software Installed (Customized)
              Size : 94,958,267,312 bytes
              WIM Bootable : No
              Architecture : x64
              Hal : acpiapic
              Version : 10.0.17134
              ServicePack Build : 1
              ServicePack Level : 81
              Edition : Professional
              Installation : Client
              ProductType : WinNT
              ProductSuite : Terminal Server
              System Root : WINDOWS
              Directories : 62409
              Files : 350446
              Created : 2018.06.01 - 19:09:51
              Modified : 2018.06.19 - 21:26:18
              Languages : en-US (Default)
    
    
    PS $  Dism /Get-ImageInfo /ImageFile:"Base.wim" /Index:4
    
          Deployment Image Servicing and Management tool
          Version: 10.0.19041.329
    
            Details for image : Base.wim
    
              Index : 4
              Name : Alienware 18: Windows 10
              Description : v1803: Software Group 1 Installed (Customized)
              Size : 101,588,267,910 bytes
              WIM Bootable : No
              Architecture : x64
              Hal : acpiapic
              Version : 10.0.17134
              ServicePack Build : 1
              ServicePack Level : 81
              Edition : Professional
              Installation : Client
              ProductType : WinNT
              ProductSuite : Terminal Server
              System Root : WINDOWS
              Directories : 61908
              Files : 346074
              Created : 2018.06.08 - 21:54:02
              Modified : 2018.06.19 - 21:26:18
              Languages : en-US (Default)
    
    
    PS $  Dism /Get-ImageInfo /ImageFile:"Base.wim" /Index:5
    
          Deployment Image Servicing and Management tool
          Version: 10.0.19041.329
    
            Details for image : Base.wim
    
              Index : 5
              Name : Alienware 18: Windows 10
              Description : v1803: Software Group 2 Installed (Customized)
              Size : 101,905,314,237 bytes
              WIM Bootable : No
              Architecture : x64
              Hal : acpiapic
              Version : 10.0.17134
              ServicePack Build : 1
              ServicePack Level : 81
              Edition : Professional
              Installation : Client
              ProductType : WinNT
              ProductSuite : Terminal Server
              System Root : WINDOWS
              Directories : 76113
              Files : 423408
              Created : 2018.06.09 - 20:38:36
              Modified : 2018.06.19 - 21:26:18
              Languages : en-US (Default)
    
    
    PS $  Dism /Get-ImageInfo /ImageFile:"Base.wim" /Index:6
    
          Deployment Image Servicing and Management tool
          Version: 10.0.19041.329
    
            Details for image : Base.wim
    
              Index : 6
              Name : Alienware 18: Windows 10
              Description : v1809: Updated Applications
              Size : 114,959,954,040 bytes
              WIM Bootable : No
              Architecture : x64
              Hal : acpiapic
              Version : 10.0.17763
              ServicePack Build : 195
              ServicePack Level : 0
              Edition : Professional
              Installation : Client
              ProductType : WinNT
              ProductSuite : Terminal Server
              System Root : WINDOWS
              Directories : 87659
              Files : 452028
              Created : 2018.12.24 - 04:27:13
              Modified : 2018.12.24 - 04:27:15
              Languages : en-US (Default)
    

How does the native method differ from conventional cloning?

The vast majority of Windows users have no need for partition-level or disk-level images:

• A conventional partition-level or disk-level image (contains offset, alignment, block size, etc.):
  • Lacks native Windows support and therefore requires non-standard boot media (boot media that's not WinPE/WinRE) and third-party programs
  • Often lacks compression by default, one of the main advantages of the WIM/ESD smart compression image format
  • Locks the user to that specific partition or drive layout (offset, alignment, block size, etc.)
  • Often has no data verification and is therefore subject to data corruption
    
    
• DISM/ImageX creates a filesystem image, not a partition partition-level or disk-level image:
  (Win ≥ XP uses NTFS as the default filesystem)
  • When pointed at the root of a partition [C:\], DISM/ImageX captures an image of all data on that partition, but not the structure of the partition/drive itself (offset, alignment, block size, etc.), bypassing the inconvenience a conventional partition/drive image creates, as only filesystem data is contained within a .wim/.esd, allowing it to be applied to any partition, regardless of size difference or whether there is existing data on the partition.
    
    

What are the pros and cons of native versus 3rd party?

Third-party tools will almost always fall into one of two categories, Linux-based or Windows-based via DISM/ImageX/Powershell, with many resulting in configuration issues, and the latter sometimes encompassing developers who use proprietary image file formats and custom boot environments (many of which are Linux-based).

• There's a minute number of posts on StackExchange (or Spiceworks) regarding imaging issues arising from using Windows' native DISM (Win XP ≤ 7: ImageX), however thousands of questions, answers, and comments exist for issues arising from third-party imaging tools:
  • Windows cloning issue (2,751 results)
    Windows cloning problem (3,838 results)
  • DISM:
    • Windows Dism /Capture-Image issue (60 results)
      Windows Dism /Capture-Image problem (44 results)
    • Windows Dism /Append-Image issue (20 results)
      Windows Dism /Append-Image problem (12 results)
    • Windows Dism /Apply-Image issue (85 results)
      Windows Dism /Apply-Image problem (93 results)
  • ImageX:
    • Windows ImageX /Capture issue (19 results)
      Windows ImageX /Capture problem (20 results)
    • Windows ImageX /Append issue (10 results)
      Windows ImageX /Append problem (5 results)
    • Windows ImageX /Apply issue (15 results)
      Windows ImageX /Apply problem (12 results)
      
      
• I have the perspective it's unacceptable for a Windows user to receive advice to use Linux tools to image Windows, as that's inefficient, forcing the user to rely on not only a non-native boot environment unsupported by Windows, but also on an image format unsupported by Windows, both of which over-complicate imaging.
  
  Ever come across advice telling a BSD or Linux user to boot to Windows or use Wine to back up their data? For example, ntfsclone (part of ntfs-3g) is a popular Linux utility, with the following from it's man page:

  Windows Cloning
  
  If you want to copy, move or restore a system or boot partition to another computer, to a different disk, partition... or to a different disk sector offset, then you will need to take extra care.
  
  Usually, Windows will not be able to boot, unless you copy, move or restore NTFS to the same partition which starts at the same sector on the same type of disk having the same BIOS legacy cylinder setting as the original partition and disk had.
  
  The ntfsclone utility guarantees to make an exact copy of NTFS, but it won't deal with booting issues. This is by design: ntfsclone is a filesystem, not system utility; its aim is only NTFS cloning, not Windows cloning. Hereby ntfsclone can be used as a very fast and reliable build block for Windows cloning but [it] itself it's not enough.

WIMs/ESDs don't have these issues since they only contain filesystem information (files and directories), not partition/drive level data, allowing them to be applied to any partition, regardless of size difference or whether there is existing data.

Native Pros:

• WIMs/ESDs are natively supported by all Windows editions ≥ XP
  • WIMs/ESDs are versatile and can be captured, applied, or modified when booted to WinPE (Windows install media), WinRE (Windows Recovery), or the OS
  • WIMs/ESDs do not require additional tools or boot media since all required tools are built-in to all three environments
• WIMs/ESDs are smart compression image formats, able to contain multiple images, backups or otherwise, within a relatively small single image file
• Its impossible for WIMs/ESDs to become corrupted, provided /CheckIntegrity (ImageX: /Check) & /Verify are always used
• WIMs/ESDs can be deployed remotely via PXE, even to a machine without an OS installed
  
  

Native Cons:

• Requires a storage medium for the captured image (another partition, USB drive, network share, etc.)
  • If saving the image to a mechanical HDD, compression takes longer, so if wanting to use /Compress:Max or /Compress:Recovery, it's more efficient to use /Compress:Fast, exporting the image later using Max or Recovery
• Capturing, Appending, Applying, or Exporting an image is resource-intensive
  • Even though /CheckIntegrity (ImageX: /Check) and Verify do extend the image processing time, they should always be used
    
    

How do I configure system partitions on a new drive for applying an image?

1. Use DiskPart: (select the OS drive the image is being applied to)
   Assumes no data on drive is being preserved, as clean wipes the drive's partition table

   DiskPart
   
   Lis Dis
   Sel Dis #
   Clean
   

   UEFI:

   Convert Gpt
   

   
   
2. Create WinRE partition: (must have 320MB free (WinRE.wim is ~300MB in size)
   • BIOS:

     Cre Par Pri Offset=1024 Size=665 Id=27
     Format Quick Fs=NTFS Label=WinRE
     

   • UEFI:

     Cre Par Pri Offset=1024 Size=665 Id=de94bba4-06d1-4d40-a16a-bfd50179d6ac
     Format Quick Fs=NTFS Label=WinRE
     Gpt Attributes=0x8000000000000001
     

   
   
3. Create boot partition:
   • BIOS:

     Cre Par Pri Size=100
     Format Quick Fs=NTFS Label=Boot
     Active
     

   • UEFI:

     Cre Par EFI Size=100
     Format Quick Fs=FAT32 Label=EFI
     Assign Letter=Y
     Cre Par Msr Size=16
     

   
   
4. Create System partition:
   • Rest of the drive as the System partition: (if C: can't be assigned: change 4 & 5 to another letter)
     BIOS:

     Cre Par Pri
     Format Quick Fs=NTFS Label=System
     Exit
     

     UEFI:

     Cre Par Pri Id=ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
     Format Quick Fs=NTFS Label=System
     Assign Letter=C
     Exit
     

   • Additional partitions after the [200GB] System partition:
     If storing User Data directories on a partition other than C:\ (recommended), max size required is ~300GB (multiply size wanted by 1024: 200*1024=204800)
     BIOS:

     Cre Par Pri Size=204800
     Format Quick Fs=NTFS Label=System
     Exit
     

     UEFI:

     Cre Par Pri Size=204800 Id=ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
     Format Quick Fs=NTFS Label=System
     Assign Letter=C
     Exit
     

   
   
5. Resolve any boot issues: (Once system image has been applied)
   BIOS:

   BootRec /FixMBR && BootRec /FixBoot && BootRec /RebuildBCD
   

   UEFI:

   ::# With existing bootable EFI partition:
       BootRec /FixMBR && BootRec /RebuildBCD
   
   
   ::# Without existing bootable EFI partition:
      ::# Create EFI directories and enter:
          MkDir "Y:\EFI\Microsoft\Boot"
          Cd /d "Y:\EFI\Microsoft\Boot"
   
      ::# Create EFI boot structure:
          BootRec /Fixboot
   
          ::# If Access Denied error occurs (C: is applied image):
              BcdBoot C:\Windows /s C: /f UEFI
   
      ::# Resolve any other boot issues:
          BootRec /FixMBR && BootRec /RebuildBCD
   

   
   
6. Remove EFI mountpoint (if applicable) and Reboot
   UEFI:

   DiskPart
   
   Sel Vol Y
   Remove
   Exit
   

Solution 2:

What is the most efficient, native way to image a Windows partition?

There isn't one any more since Windows Backup is being phased out (probably because this was a bad product to start with).

Only DISM is left, but it only does file backup, not partition image backup. Its new Full Flash Update (FFU) images takes a sector-by-sector image of the entire disk, which unfortunately also includes unused sectors, so not at all efficient.

Why is the native method generally the best method for most users?

It isn't for Windows, as above. Microsoft has left the field in favor of third-party products.

How does the native method differ from conventional cloning?

DISM does not do cloning at all.

What are the pros and cons of native versus third-party tools?

The pros of third-party tools is that they work well and efficiently. Most are also free to use.

Example products are AOMEI Backupper, Clonezilla, Macrium Reflect, EaseUS ToDo BackUp. YMMV.

Historical note: DISM was conceived by Microsoft decades ago in an ancient version of Windows (Vista), using the Windows Imaging Format (WIM), which is a file-based disk image format, used mostly for software distribution. For backup, Microsoft has created Windows Backup, of which a limited version is still available in Windows 10 as "Back up and Restore (Windows 7)", but without its problematic image backup feature. The use of DISM as a backup utility is very strongly not recommended.