Convert text files recursively to UTF-8 in PowerShell

Try this one.

foreach($i in Get-ChildItem -Recurse) {
    if ($i.PSIsContainer) {
        continue
    }

    $dest = $i.Fullname.Replace($PWD, "some_folder")
    if (!(Test-Path $(Split-Path $dest -Parent))) {
        New-Item $(Split-Path $dest -Parent) -type Directory
    }

    get-content $i | out-file -encoding utf8 -filepath $dest
}

It grabs the full path of the file and replaces the current directory with the one you want. For example, you run this command in the directory C:\1\ ($PWD = C:\1\). If it finds the file C:\1\2\file.txt, it'll give you a $dest of some_folder\2\file.txt.

The first if block is there so you don't try to convert a directory.

The directories have to be created if they don't already exist - I originally forgot that.

If you want UTF8 without BOM, replace the get-content $i | out-file -encoding utf8 -filepath $dest line with the following (source):

$filecontents = Get-Content $i
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding($False)
[System.IO.File]::WriteAllLines($i, $filecontents, $Utf8NoBomEncoding)

Note that this may not be very performant for larger files, since it reads entire files into memory before writing them again. If efficiency is needed, it is possible to read line by line, or even a specific number of bytes at a time. However, I'd rather just write a quick program in C# by that point (since you'd be using .NET fuctions in PS anyway).

Allows for files and folders
File extension agnostic
Overwrites original file if destination equals the path
Encoding as a parameter

Usage: &"TextEncoding.ps1" -path "c:\windows\temps\folder1" -encoding "UTF8"

Here is the script I created:

[CmdletBinding()]
param(  
    [Parameter(Mandatory=$true)]
    [string]$path,
    [Parameter(Mandatory=$false)]
    [string]$dest = $path,
    [Parameter(Mandatory=$true)]
    [string]$encoding
)

function Set-Encoding(){

    #ensure it is a valid path
    if(-not(Test-Path -Path $path)){

        throw "File or directory not found at {0}" -f $path
    }

    #if the path is a file, else a directory
    if(Test-Path $path -PathType Leaf){

        #if the provided path equals the destination
        if($path -eq $dest){

            #get file extension
            $ext = [System.IO.Path]::GetExtension($path)

            #create destination
            $dest = $path.Replace([System.IO.Path]::GetFileName($path), ("temp_encoded{0}" -f $ext))

            #output to file with encoding
            Get-Content $path | Out-File -FilePath $dest -Encoding $encoding -Force

            #copy item to original path to overwrite (note move-item loses encoding)
            Copy-Item -Path $dest -Destination $path -Force -PassThru | ForEach-Object { Write-Output -inputobject ("{0} encoded {1}" -f $encoding, $_) }

            #remove the extra file
            Remove-Item $dest   

        }else{

            #output to file with encoding
            Get-Content $path | Out-File -FilePath $dest -Encoding $encoding -Force     

        }

    }else{

        #get all the files recursively
        foreach($i in Get-ChildItem -Path $path -Recurse) {


            if ($i.PSIsContainer) {
                continue
            }

            #get file extension
            $ext = [System.IO.Path]::GetExtension($i)

            #create destination
            $dest = "$path\temp_encoded{0}" -f $ext

            #output to file with encoding
            Get-Content $i.FullName | Out-File -FilePath $dest -Encoding $encoding -Force

            #copy item to original path to overwrite (note move-item loses encoding)
            Copy-Item -Path $dest -Destination $i.FullName -Force -PassThru | ForEach-Object { Write-Output -inputobject ("{0} encoded {1}" -f $encoding, $_) }

            #remove the extra file
            Remove-Item $dest

        }

    }

}

Set-Encoding

Convert text files recursively to UTF-8 in PowerShell

Related

Recent Posts