Call an absolute pointer in x86 machine code

What's the "correct" way to call an absolute pointer in x86 machine code? Is there a good way to do it in a single instruction?

What I want to do:

I'm trying to build a kind of simplified mini-JIT (still) based on "subroutine threading". It's basically the shortest possible step up from a bytecode interpreter: each opcode is implemented as a separate function, so each basic block of bytecodes can be "JITted" into a fresh procedure of its own that looks something like this:

{prologue}
call {opcode procedure 1}
call {opcode procedure 2}
call {opcode procedure 3}
...etc
{epilogue}

So the idea is that the actual machine code for every block can just be pasted out of a template (extending the middle part as necessary), and the only bit that needs to be "dynamically" handled is copying the function pointers for each opcode into the right places as part of each call instruction.

The problem I'm having is understanding what to use for the call ... part of the template. x86 doesn't seem to be set up with this kind of usage in mind, and favours relative and indirect calls.

It looks like I can use either FF 15 EFBEADDE or 2E FF 15 EFBEADDE to call the function hypothetically at DEADBEEF (basically discovered these by putting stuff into an assembler and disassembler and seeing what produced valid results, not by understanding what they do), but I don't understand the stuff about segments and privileges and associated information well enough to see the difference, or how these will behave differently from a more frequently-seen call instruction. The Intel architecture manual also suggests that these are only valid in 32-bit mode, and "invalid" in 64-bit mode.

Can someone explain these opcodes and how, or if, I would use them or others for this purpose?

(There's also the obvious answer of using an indirect call through a register, but that seems like the "wrong" approach - assuming a direct call instruction actually exists.)

Solution 1:

Everything here applies to jmp to absolute addresses, too, and the syntax for specifying the target is the same. The question asks about JITing, but I also included NASM and AT&T syntax to broaden the scope.

See also Handling calls to far away intrinsic functions in a JIT for ways to allocate "nearby" memory so you can use rel32 to call ahead-of-time compiled functions from your JITed code.

x86 doesn't have an encoding for a normal (near) call or jmp to an absolute address encoded in the instruction There are no absolute direct call/jmp encodings, except jmp far which you don't want. See Intel's insn set ref manual entry for call. (See also the x86 tag wiki for other links to docs and guides.) Most computer architectures use relative encodings for normal jumps like x86, BTW.

The best option (if you can make position-dependent code that knows its own address) is to use the normal call rel32, the E8 rel32 direct near call encoding, where the rel32 field is target - end_of_call_insn (2's complement binary integer).

See How does $ work in NASM, exactly? for an example of manually encoding a call instruction; doing it while JITing should be just as easy.

In AT&T syntax: call 0x1234567
In NASM syntax: call 0x1234567
Also works to a named symbol with an absolute address (e.g. created with equ or .set). There's no equivalent for MASM, it apparently only accepts a label as a destination so people sometimes use inefficient workarounds to workaround that toolchain (and/or object file format relocation type) limitation.

These assemble and link just fine in position-dependent code (not a shared lib or a PIE executable). But not in x86-64 OS X where the text section is mapped above 4GiB so it couldn't reach a low address with a rel32.

Allocate your JIT buffer in range of the absolute addresses you want to call. e.g. with mmap(MAP_32BIT) on Linux to allocate memory in the low 2GB where +-2GB can reach any other address in that region, or provide a non-NULL hint address somewhere near where your jump target is. (Don't use MAP_FIXED, though; probably best to let the kernel just pick a different address if your hint overlapped with any existing mappings.)

(Linux non-PIE executables are mapped in the low 2GB of virtual address space, so they can use [disp32 + reg] array indexing with sign-extended 32-bit absolute addresses, or put static addresses in registers with mov eax, imm32 for zero-extended absolutes. Thus low 2GB, not low 4GB. But PIE executables are becoming the norm, so don't assume that static addresses in your main executable are in the low 32 unless you make sure to build+link with -no-pie -fno-pie. And other OSes like OS X always put executables above 4GB.)

If you can't make call rel32 usable

But if you need to make position-independent code that doesn't know its own absolute address, or if the address you need to call is more than +-2GiB away from the caller (possible in 64-bit, but it's better to place code close enough), you should use a register-indirect call

; use any register you like as a scratch
mov   eax, 0xdeadbeef               ; 5 byte  mov r32, imm32
     ; or mov rax, 0x7fffdeadbeef   ; for addresses that don't fit in 32 bits
call  rax                           ; 2 byte  FF D0

Or AT&T syntax

mov   $0xdeadbeef, %eax
# movabs $0x7fffdeadbeef, %rax      # mov r64, imm64
call  *%rax

Obviously you can use any register, like r10 or r11 which are call-clobbered but not used for arg-passing in x86-64 System V. AL = number of XMM args to a variadic function, so you require a fixed value in AL=0 before a call to a variadic function in the x86-64 System V calling convention.

If you really need to avoid modifying any registers, maybe keep the absolute address as a constant in memory and use a memory-indirect call with a RIP-relative addressing mode, like

NASM call [rel function_pointer] ; If you can't clobber any reg
AT&T call *function_pointer(%rip)

Note that indirect calls / jumps make your code potentially vulnerable to Spectre attacks, especially if you're JITing as part of a sandbox for untrusted code within the same process. (In that case kernel patches alone won't protect you).

You may want a "retpoline" instead of a normal indirect branch to mitigate Spectre at the cost of performance.

Indirect jumps will also have slightly worse branch-misprediction penalties than direct (call rel32). The destination of a normal direct call insn is known as soon as it's decoded, earlier in the pipeline as soon as its detected that there's a branch there at all.

Indirect branches generally predict well on modern x86 hardware, and are commonly used for calls to dynamic libraries / DLLs. It's not terrible, but call rel32 is definitely better.

Even direct call needs some branch prediction to avoid pipeline bubbles entirely, though. (Prediction is needed before decode, e.g. given that we just fetched this block, which block should the fetch stage fetch next. A sequence of jmp next_instruction slows down when you run out of branch-predictor entries). mov + indirect call reg is also worse even with perfect branch prediction because it's larger code-size and more uops, but that's a pretty minimal effect. If an extra mov is an issue, inlining the code instead of calling it is a good idea, if possible.

Fun fact: call 0xdeadbeef will assemble but not link into a 64-bit static executable on Linux, unless you use a linker script to put the .text section / text segment closer to that address. The .text section normally starts at 0x400080 in a static executable (or a non-PIE dynamic executable), i.e. in the low 2GiB of virtual address space, where all static code / data lives in the default code model. But 0xdeadbeef is in the high half of the low 32 bits (i.e. in the low 4G but not the low 2G), so it can be represented as a zero-extended 32-bit integer but not sign-extended 32-bit. And 0x00000000deadbeef - 0x0000000000400080 doesn't fit in a signed 32-bit integer that will correctly extend to 64 bits. (The part of address space you can reach with negative rel32 that wraps around from a low address is the top 2GiB of the 64-bit address space; normally the top half of address space is reserved for use by the kernel.)

It does assemble ok with yasm -felf64 -gdwarf2 foo.asm, and objdump -drwC -Mintel shows:

foo.o:     file format elf64-x86-64
Disassembly of section .text:
0000000000000000 <.text>:
    0:   e8 00 00 00 00       call   0x5   1: R_X86_64_PC32        *ABS*+0xdeadbeeb

But when ld tries to actually link it into a static executable where .text starts at 0000000000400080, ld -o foo foo.o says foo.o:/tmp//foo.asm:1:(.text+0x1): relocation truncated to fit: R_X86_64_PC32 against '*ABS*'.

In 32-bit code call 0xdeadbeef assembles and links just fine, because a rel32 can reach anywhere from anywhere. The relative displacement doesn't have to be sign-extended to 64-bits, it's just 32-bit binary addition which can wrap around or not.

Direct far call encodings (slow, don't use)

You might notice in the manual entries for call and jmp that there are encodings with absolute target addresses encoded right into the instruction. But those only exist for "far" call/jmp that also set CS to a new code segment selector, which is slow (see Agner Fog's guides).

CALL ptr16:32 ("Call far, absolute, address given in operand") has a 6-byte segment:offset encoded right into the instruction, rather than loading it as data from a location given by a normal addressing mode. So it's a direct call to an absolute address.

Far call also pushes CS:EIP as the return address instead of just EIP, so it's not even compatible with normal (near) call that only pushes EIP. That's not an issue for jmp ptr16:32, just the slowness and figuring out what to put for the segment part.

Changing CS is generally only useful for changing from 32 to 64-bit mode or vice versa. Usually only kernels would do this, although you can do this in user-space under most normal OSes that keep 32 and 64-bit segment descriptors in the GDT. That would be more of a silly computer trick than something useful, though. (64-bit kernels return to 32-bit userspace with iret or maybe with sysexit. Most OSes would only use a far jmp once during bootup to switch to a 64-bit code segment in kernel mode.)

Mainstream OSes use a flat memory model where you never need to change cs, and it's not standardized what cs value will be used for user-space processes. Even if you wanted to use a far jmp, you'd have to figure out what value to put in the segment selector part. (Easy while JITing: just read the current cs with mov eax, cs. But hard to be portable for ahead-of-time compilation.)

call ptr16:64 doesn't exist, the far direct encodings only exist for 16 and 32-bit code. In 64-bit mode you can only far-call with a 10-byte m16:64 memory operand, like call far [rdi]. Or push segment:offset on the stack and use retf.

Solution 2:

You can't do it with only one instruction. A decent way of doing it is with MOV + CALL:

0000000002347490: 48b83412000000000000  mov rax, 0x1234
000000000234749a: 48ffd0                call rax

If the address of the procedure to call changes, change the eight bytes starting at offset 2. If the address of the code calling 0x1234 changes, you don't have to do anything because the addressing is absolute.