Analyze colors of an Image

Solution 1:

When performing sequential operations on a Bitmap's color data, the Bitmap.LockBits method can provide a huge increase in performace, since the Bitmap data needs to be loaded in memory just once, as opposed to sequential GetPixel/SetPixel calls: each call will load a partial section of the Bitmap data in memory and then discard it, to repeat the process when these methods are called again.

If a single call to GetPixel/SetPixel is needed instead, these methods may have a performace advantage over Bitmap.LockBits(). But, in this case, performace is not a factor, in practice.

How Bitmap.LockBits() works:

This is the function call:

public BitmapData LockBits (Rectangle rect, ImageLockMode flags, PixelFormat format);
// VB.Net
Public LockBits (rect As Rectangle, flags As ImageLockMode, format As PixelFormat) As BitmapData

• rect As Rectangle: This parameter specifies the section of the Bitmap data we're interested in; this section's bytes will be loaded in memory. It can be the whole size of the Bitmap or a smaller section of it.

• flags As ImageLockMode: Specifies the type of lock to perform. The access to the memory to can be limited to Read or Write, or concurrent Read/Write operations are allowed.
  It can be also used to specify - setting ImageLockMode.UserInputBuffer - that the BitmapData object is provided by the calling code.
  The BitmapData object defines some of the Bitmap properties (Width and Height of the Bitmap, width of the scan line (the Stride: number of bytes that compose a single line of pixels, represented by the Bitmap.Width multiplied by the number of bytes per pixel, rounded to a 4-bytes boundary. See the note about the Stride).
  The BitmapData.Scan0 property is the Pointer (IntPtr) to the initial memory location where the Bitmap data is stored.
  This property allows to specify the memory location where a pre-existing Bitmap data buffer is already stored. It becomes useful when Bitmap data is exchanged between processes using Pointers.
  Note that the MSDN documentation about ImageLockMode.UserInputBuffer is confusing (if not wrong).

• format As PixelFormat: the format used to describe the Color of a single Pixel. It translates, in practice, in the number of bytes used to represent a Color.
  When PixelFormat = Format24bppRgb, each Color is represented by 3 bytes (RGB values). With PixelFormat.Format32bppArgb, each Color is represented by 4 bytes (RGB values + Alpha).
  Indexed formats, as Format8bppIndexed, specify that each byte value is the index to a Palette entry. The Palette is part of the Bitmap information, except when the pixel format is PixelFormat.Indexed: in this case, each value is an entry in the System color table.
  The default PixelFormat of a new Bitmap object, if not specified, is PixelFormat.Format32bppArgb, or PixelFormat.Canonical.

Important notes about the Stride:

As mentioned before, the Stride (also called scan-line) represents the number of bytes that compose a single line of pixels. Because of harware alignment requirements, it's always rounded up to a 4-bytes boundary (an integer number multiple of 4).

Stride =  [Bitmap Width] * [bytes per Color]
Stride += (Stride Mod 4) * [bytes per Color]

This is one of the reasons why we always work with Bitmaps created with PixelFormat.Format32bppArgb: the Bitmap's Stride is always already aligned to the required boundary.

What if the Bitmap's format is instead PixelFormat.Format24bppRgb (3 bytes per Color)?

If the Bitmap's Width multiplied by the Bytes per Pixels is not a multiple of 4, the Stride will be padded with 0s to fill the gap.

A Bitmap of size (100 x 100) will have no padding in both 32 bit and 24 bit formats:

100 * 3 = 300 : 300 Mod 4 = 0 : Stride = 300
100 * 4 = 400 : 400 Mod 4 = 0 : Stride = 400

It will be different for a Bitmap of size (99 x 100):

99 * 3 = 297 : 297 Mod 4 = 1 : Stride = 297 + ((297 Mod 4) * 3) = 300
99 * 4 = 396 : 396 Mod 4 = 0 : Stride = 396

The Stride of a 24 bit Bitmap is padded adding 3 bytes (set to 0) to fill the boundary.

It's not a problem when we inspect/modify internal values accessing single Pixels by their coordinates, similar to how SetPixel/GetPixel operate: the position of a Pixel will always be found correctly.

Suppose we need to inspect/change a Pixel at position (98, 70) in a Bitmap of size (99 x 100).
Considering only the bytes per pixel. The pixel position inside the Buffer is:

[Bitmap] = new Bitmap(99, 100, PixelFormat = Format24bppRgb)

[Bytes x pixel] = Image.GetPixelFormatSize([Bitmap].PixelFormat) / 8
[Pixel] = new Point(98, 70)
[Pixel Position] = ([Pixel].Y * [BitmapData.Stride]) + ([Pixel].X * [Bytes x pixel])
[Color] = Color.FromArgb([Pixel Position] + 2, [Pixel Position] + 1, [Pixel Position])

Multiplying the Pixel's vertical position by the width of the scan line, the position inside the buffer will always be correct: the padded size is included in the calculation.
The Pixel Color at the next position, (0, 71), will return the expected results:

It will be different when reading color bytes sequentially.
The first scan line will return valid results up to the last Pixel (the last 3 bytes): the next 3 bytes will return the value of the bytes used to round the Stride, all set to 0.

This might also not be a problem. For example, applying a filter, each sequence of bytes that represent a pixel is read and modifyed using the values of the filter's matrix: we would just modify a sequence of 3 bytes that won't be considered when the Bitmap is rendered.

But it does matter if we are searching for specific sequences of pixels: reading a non-existent pixel Color may compromise the result and/or unbalance an algorithm.
The same when performing statistical analisys on a Bitmap's colors.

Of course, we could add a check in the loop: if [Position] Mod [BitmapData].Width = 0 : continue.
But this adds a new calculation to each iteration.

Operations in practice

The simple solution (the more common one) is to create a new Bitmap with a format of PixelFormat.Format32bppArgb, so the Stride will be always correctly aligned:

Imports System.Drawing
Imports System.Drawing.Imaging
Imports System.Runtime.InteropServices

Private Function CopyTo32BitArgb(image As Image) As Bitmap
    Dim imageCopy As New Bitmap(image.Width, image.Height, PixelFormat.Format32bppArgb)
    imageCopy.SetResolution(image.HorizontalResolution, image.VerticalResolution)

    For Each propItem As PropertyItem In image.PropertyItems
        imageCopy.SetPropertyItem(propItem)
    Next

    Using g As Graphics = Graphics.FromImage(imageCopy)
        g.DrawImage(image,
            New Rectangle(0, 0, imageCopy.Width, imageCopy.Height),
            New Rectangle(0, 0, image.Width, image.Height),
            GraphicsUnit.Pixel)
        g.Flush()
    End Using
    Return imageCopy
End Function

This generates a byte-compatible Bitmap with the same DPI definition; the Image.PropertyItems are also copied from the source image.

To test it, let's apply a sepia tone filter to an Image, using a copy of it to perform all the modifications needed to the Bitmap data:

Public Function BitmapFilterSepia(source As Image) As Bitmap
    Dim imageCopy As Bitmap = CopyTo32BitArgb(source)
    Dim imageData As BitmapData = imageCopy.LockBits(New Rectangle(0, 0, source.Width, source.Height),
        ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb)

    Dim buffer As Byte() = New Byte(Math.Abs(imageData.Stride) * imageCopy.Height - 1) {}
    Marshal.Copy(imageData.Scan0, buffer, 0, buffer.Length)

    Dim bytesPerPixel = Image.GetPixelFormatSize(source.PixelFormat) \ 8;
    Dim red As Single = 0, green As Single = 0, blue As Single = 0

    Dim pos As Integer = 0
    While pos < buffer.Length
        Dim color As Color = Color.FromArgb(BitConverter.ToInt32(buffer, pos))
        ' Dim h = color.GetHue()
        ' Dim s = color.GetSaturation()
        ' Dim l = color.GetBrightness()

        red = buffer(pos) * 0.189F + buffer(pos + 1) * 0.769F + buffer(pos + 2) * 0.393F
        green = buffer(pos) * 0.168F + buffer(pos + 1) * 0.686F + buffer(pos + 2) * 0.349F
        blue = buffer(pos) * 0.131F + buffer(pos + 1) * 0.534F + buffer(pos + 2) * 0.272F

        buffer(pos + 2) = CType(Math.Min(Byte.MaxValue, red), Byte)
        buffer(pos + 1) = CType(Math.Min(Byte.MaxValue, green), Byte)
        buffer(pos) = CType(Math.Min(Byte.MaxValue, blue), Byte)
        pos += bytesPerPixel
    End While

    Marshal.Copy(buffer, 0, imageData.Scan0, buffer.Length)
    imageCopy.UnlockBits(imageData)
    imageData = Nothing
    Return imageCopy
End Function

Bitmap.LockBits is not always necessarily the best choice available.
The same procedure to apply a filter could also be performed quite easily using the ColorMatrix class, which allows to apply a 5x5 matrix transformation to a Bitmap, using just a simple array of float (Single) values.

For example, let's apply a Grayscale filter using the ColorMatrix class and a well-known 5x5 Matrix:

Public Function BitmapMatrixFilterGreyscale(source As Image) As Bitmap
    ' A copy of the original is not needed but maybe desirable anyway 
    ' Dim imageCopy As Bitmap = CopyTo32BitArgb(source)
    Dim filteredImage = New Bitmap(source.Width, source.Height, source.PixelFormat)
    filteredImage.SetResolution(source.HorizontalResolution, source.VerticalResolution)

    Dim grayscaleMatrix As New ColorMatrix(New Single()() {
        New Single() {0.2126F, 0.2126F, 0.2126F, 0, 0},
        New Single() {0.7152F, 0.7152F, 0.7152F, 0, 0},
        New Single() {0.0722F, 0.0722F, 0.0722F, 0, 0},
        New Single() {0, 0, 0, 1, 0},
        New Single() {0, 0, 0, 0, 1}
   })

    Using g As Graphics = Graphics.FromImage(filteredImage), attributes = New ImageAttributes()
        attributes.SetColorMatrix(grayscaleMatrix)
        g.DrawImage(source, New Rectangle(0, 0, source.Width, source.Height),
                    0, 0, source.Width, source.Height, GraphicsUnit.Pixel, attributes)
    End Using
    Return filteredImage
End Function