x86_64 ASM - maximum bytes for an instruction?

The x86 instruction set (16, 32 or 64 bit, all variants/modes) guarantees / requires that instructions are at most 15 bytes. Anything beyond that will give an "invalid opcode". You can't achieve that without using redundant prefixes (e.g. multiple 0x66 or 0x67 prefixes, for example).

The only instruction that actually takes 64-bits as a data item is the load constant to register (Intel syntax: mov reg, 12345678ABCDEF00h, at&t syntax: movabs $12345678ABCDEF00, %reg) - so if you wanted to jump more than 31 bits forward/backward, it would be a move of the target location into a register, and then call/jump to the register. Using 32-bit immediates and displacements (in relative jumps and addressing modes) saves four bytes on many instructions in 64-bit mode.

Questions is, what’s the longest possible instruction in the x86 instruction set?

Answer: you can form a valid x86 instruction with an infinite number of bytes!

That’s right, you could fill up an entire 64K ROM image with a single valid instruction. To be more specific, there is no limit to the length of 8086 instructions. Cool! Unfortunately, modern day i386 variants throw a general protection fault when attempting to decode instructions longer than 15 bytes.

So what does an infinitely-long-but-valid 8086 instruction look like? Kinda boring, actually. You could only form an infinitely long instruction by using redundant prefixes in front on the opcodes. Instruction prefixes are bytes pre-pended to the beginning of an instruction that can modify the default address size, data size, or segment registers used by an instruction.

For example, you can take the innocuous looking instruction:

89 E5              mov %sp,%bp

And turn it into a really long instruction:

66 66 66 66 … 66 66 89 E5                mov %sp,%bp

Now that’s just evil.

https://web.archive.org/web/20131109063453/https://www.onlinedisassembler.com/blog/?p=23

Another long instruction without repeating prefixes

In some cases it is possible to encode valid instructions that exceed the traditional 15-byte length limit. For example:

  ; 16-bit mode
  F2 F0 36 66 67 81 84 24 disp32 imm32 =  xaquire lock add [ss:esp*1+disp32],imm32
  F3 F0 36 66 67 81 84 24 disp32 imm32 = xrelease lock add [ss:esp*1+disp32],imm32

  ; 16-bit mode
  36 67 8F EA 78 12 84 24 disp32 imm32 = lwpins eax,[ss:esp*1+disp32],imm32
  36 67 8F EA 78 12 8C 24 disp32 imm32 = lwpval eax,[ss:esp*1+disp32],imm32
  36 67 8F EA 78 10 84 24 disp32 imm32 =  bextr eax,[ss:esp*1+disp32],imm32

  ; 64-bit mode
  64 67 8F EA F8 12 84 18 disp32 imm32 = lwpins rax,[fs:eax+ebx+disp32],imm32
  64 67 8F EA F8 12 8C 18 disp32 imm32 = lwpval rax,[fs:eax+ebx+disp32],imm32
  64 67 8F EA F8 10 84 18 disp32 imm32 =  bextr rax,[fs:eax+ebx+disp32],imm32

http://www.sandpile.org/x86/opc_enc.htm

From the Intel® 64 and IA-32 Architectures Software Developer’s Manual:

2.3.11 AVX Instruction Length

The maximum length of an Intel 64 and IA-32 instruction remains 15 bytes.

You can construct instructions that would encode to more than 15 bytes, but such instructions would be illegal and would probably not execute.