Best way to store images so that they are same across formats

We are designing an algorithm that builds a Merkle tree from a list of Perceptual hashes. the hashes are generated for every frame that we capture from a video. The incentive behind this is that we are able to identify hashes even if the video format has changed.

To verify this, we had two images : Video.mp4 and Video.avi. We extracted frames at 30 fps, and ran pHash over these images. To test our functionality, it is imperative that both the images at every instance (from .mp4 and from .avi) stay the same. However there are still some differences in those two images.

Including code for reference:

1. Extract frames from video:

def extract_frames(file_path, write_to_path, fps=30):
  cap = cv2.VideoCapture(file_path)

  count = 0
  os.mkdir(f'{write_to_path}/frames')
  while cap.isOpened():

    ret, frame = cap.read()
    if ret:

        grayed_image = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        cv2.imwrite(f"{write_to_path}/frames/frame{count}.bmp", grayed_image)
        count += fps # i.e. at 30 fps, this advances one second
        cap.set(1, count)
    else:
        cap.release()
        break
  print(f"Frame Extraction complete. Extracted {count // fps} frames.")
  return count

2. Test if two images are similar

def check_images(path_1, path_2):
    img_1 = cv2.imread(path_1, 0)
    img_2 = cv2.imread(path_2, 0)

    if img_1.shape == img_2.shape:
      difference = cv2.subtract(img_1, img_2)
      print(difference)
      result = not np.any(difference)
      return result
    
    print("Unequal shapes, ", img_1.shape, img_2.shape)
    return False

3. The Perceptual hash function

def generate_p_hashes(count, frame_path, fps=30):
  count_two = 0
  hashes = []
  # fileToWrite = open('/content/hash.txt', 'a')

  while count_two != count:
    temp_hash = imagehash.phash(Image.open(f"{frame_path}/frames/frame{count_two}.bmp"))
    count_two += fps
    str_temp_hash = str(temp_hash)
    hashes.append(str_temp_hash)
  
  print(f"PHash generation complete. Generated {count_two // fps} hashes")
  return hashes

Imagehash is a Python package available at : https://github.com/JohannesBuchner/imagehash

4. The images:

a. Frame captured from .avi file:

b. Frame captured from .mp4 file:

Here's what I've tried:

1. Convert image to grayscale so color channels are excluded.
2. Try all different image formats (JPEG, PNG with compression 0, TIFF, BMP)

Sample output:

What is the best way to store these images, so that irrespective of the video source that I am extracting from, the image will stay the same ?

Lossy-compressed files or video streams of different technologies will never give you exactly the same content from the same original source, this is not possible. With a high compression ratio, the images can be quite different.

If the goal is to authenticate, watermark or detect copies, you need to use features that are robust to lossy compression/decompression.