Segmentation fault when using DB (define byte) inside a function

I'm trying to define a byte in Assembly language inside my .text section. I know data should go to the .data section but I was wondering why it gives me a segmentation fault when I do it. If I define the byte inside .data, it doesn't give me any errors, unlike .text. I am using a Linux machine running Mint 19.1 and using NASM + LD to compile and link the executable.

This runs without segmentation faults:

global _start
section .data
db 0x41
section .text
_start:
    mov rax, 60    ; Exit(0) syscall
    xor rdi, rdi
    syscall

This gives me a segfault:

global _start
section .text
_start:
    db 0x41
    mov rax, 60     ; Exit(0) syscall
    xor rdi, rdi
    syscall

I'm using the following script to compile and link it:

nasm -felf64 main.s -o main.o
ld main.o -o main

I expect the program to work without any segmentation faults, but it doesn't when I use DB inside .text. I suspect that .text is readonly and that may be the reason of this problem, am I correct? Can someone explain to me why my second code example segfaults?

If you tell the assembler to assemble arbitrary bytes somewhere, it will. db is a pseudo-instruction that emits bytes, so mov eax, 60 and db 0xb8, 0x3c, 0, 0, 0 are exactly equivalent as far as NASM is concerned. Either one will emit those 5 bytes into the output at the current position.

If you don't want your data decoded as (part of) instructions, don't put it where it will be reached by execution.

Since you're using NASM¹, it optimizes mov rax,60 into mov eax,60, so the instruction doesn't have the REX prefix you'd expect from the source.

Your manually-encoded REX prefix for mov changes it into a mov to R8D instead of EAX:
41 b8 3c 00 00 00 mov r8d,0x3c

(I checked with objdump -drwC -Mintel instead of looking up which bit is which in the REX prefix. I only remember that REX.W is 0x48. But 0x41 is a REX.B prefix in x86-64).

So instead of making a sys_exit system call, your code runs syscall with EAX=0, which is __NR_read. (The Linux kernel zeros all the registers other than RSP before process startup, and in a statically-linked executable, _start is the true entry point with no dynamic linker code running first. So RAX is still zero).

$ strace ./rex 
execve("./rex", ["./rex"], 0x7fffbbadad60 /* 54 vars */) = 0
read(0, NULL, 0)                        = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
+++ killed by SIGSEGV (core dumped) +++

And then execution falls through into whatever is after syscall, which in this case is 00 00 bytes that decode as add [rax], al, and thus segfault. You would have seen this if you'd run your code inside GDB.

Footnote 1: If you'd used YASM which doesn't optimize to 32-bit operand size:

Intel's manuals say that it's illegal to have 2 REX prefixes on one instruction. I expected an illegal-instruction fault (#UD machine exception -> kernel delivers SIGILL), but my Skylake CPU ignores the first REX prefix and decodes it as mov rax, sign_extended_imm32.

Single-stepping, it's treated as one long instructions, so I guess Skylake chooses to handle it like other cases of multiple prefixes, where only the last one of a type has an effect. (But remember this is not future-proof, other x86 CPUs could handle it differently.)

Related / same bug in other situations:

• Assembly (x86): <label> db 'string',0 does not get executed unless there's a jump instruction in a BIOS MBR boot sector
• Unknown opcode skipped: 66, not 8086 instruction - not supported yet in EMU8086