Give a custom item_frame item a special ID I can later use to target with /data

Solution 1:

Your current data:

{
  EntityTag: {
    Invisible: 1b,
    CustomName: {
      text: "InvisibleFrame"
    }
  }
}

The correct data:

{
  EntityTag: {
    Invisible: 1b,
    CustomName: '{"text": "InvisibleFrame"}'
  }
}

The JSON text component is a String tag inside of NBT containing JSON. You therefore need to surround it in apostrophes.

In addition, you should never target an entity by its name. Give it a tag instead:

{
  EntityTag: {
    Invisible: 1b,
    CustomName: '{"text": "InvisibleFrame"}',
    Tags: ["invis"]
  }
}

@e[type=minecraft:item_frame,tag=invis]