Running 32 bit assembly code on a 64 bit Linux & 64 bit Processor : Explain the anomaly

Remember that everything by default on a 64-bit OS tends to assume 64-bit. You need to make sure that you are (a) using the 32-bit versions of your #includes where appropriate (b) linking with 32-bit libraries and (c) building a 32-bit executable. It would probably help if you showed the contents of your makefile if you have one, or else the commands that you are using to build this example.

FWIW I changed your code slightly (_start -> main):

#include <asm/unistd.h>
#include <syscall.h>
#define STDOUT 1

    .data
hellostr:
    .ascii "hello wolrd\n" ;
helloend:

    .text
    .globl main

main:
    movl $(SYS_write) , %eax  //ssize_t write(int fd, const void *buf, size_t count);
    movl $(STDOUT) , %ebx
    movl $hellostr , %ecx
    movl $(helloend-hellostr) , %edx
    int $0x80

    movl $(SYS_exit), %eax //void _exit(int status);
    xorl %ebx, %ebx
    int $0x80

    ret

and built it like this:

$ gcc -Wall test.S -m32 -o test

verfied that we have a 32-bit executable:

$ file test
test: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.4, dynamically linked (uses shared libs), not stripped

and it appears to run OK:

$ ./test
hello wolrd

As noted by Paul, if you want to build 32-bit binaries on a 64-bit system, you need to use the -m32 flag, which may not be available by default on your installation (some 64-bit Linux distros don't include 32-bit compiler/linker/lib support by default).

On the other hand, you could instead build your code as 64-bit, in which case you need to use the 64-bit calling conventions. In that case, the system call number goes in %rax, and the arguments go in %rdi, %rsi, and %rdx

Edit

Best place I've found for this is www.x86-64.org, specifically abi.pdf

64-bit CPUs can run 32-bit code, but they have to use a special mode to do it. Those instructions are all valid in 64-bit mode, so nothing stopped you from building a 64-bit executable.

Your code builds and runs correctly with gcc -m32 -nostdlib hello.S. That's because -m32 defines __i386, so /usr/include/asm/unistd.h includes <asm/unistd_32.h>, which has the right constants for the int $0x80 ABI.

See also Assembling 32-bit binaries on a 64-bit system (GNU toolchain) for more about _start vs. main with/without libc and static vs. dynamic executables.

$ file a.out 
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, BuildID[sha1]=973fd6a0b7fa15b2d95420c7a96e454641c31b24, not stripped

$ strace ./a.out  > /dev/null
execve("./a.out", ["./a.out"], 0x7ffd43582110 /* 64 vars */) = 0
strace: [ Process PID=2773 runs in 32 bit mode. ]
write(1, "hello wolrd\n", 12)           = 12
exit(0)                                 = ?
+++ exited with 0 +++

Technically, if you'd used the right call numbers, your code would happen to work from 64-bit mode as well: What happens if you use the 32-bit int 0x80 Linux ABI in 64-bit code? But int 0x80 is not recommended in 64-bit code. (Actually, it's never recommended. For efficiency, 32-bit code should call through the kernel's exported VDSO page so it can use sysenter for fast system calls on CPUs that support it).

But that doesn't answer my my questions. What exactly is happening in this case?

Good question.

On Linux, int $0x80 with eax=1 is sys_exit(ebx), regardless of what mode the calling process was in. The 32-bit ABI is available in 64-bit mode (unless your kernel was compiled without i386 ABI support), but don't use it. Your exit status is from movl $(STDOUT), %ebx.

(BTW, there's a STDOUT_FILENO macro defined in unistd.h, but you can't #include <unistd.h> from a .S because it also contains C prototypes which aren't valid asm syntax.)

Notice that __NR_exit from unistd_32.h and __NR_write from unistd_64.h are both 1, so your first int $0x80 exits your process. You're using the wrong system call numbers for the ABI you're invoking.

strace is decoding it incorrectly, as if you'd invoked syscall (because that's the ABI a 64-bit process is expected to use). What are the calling conventions for UNIX & Linux system calls on x86-64

eax=1 / syscall means write(rd=edi, buf=rsi, len=rdx), and this is how strace is incorrectly decoding your int $0x80.

rdi and rsi are 0 (aka NULL) on entry to _start, and your code sets rdx=12 with movl $(helloend-hellostr) , %edx.

Linux initializes registers to zero in a fresh process after execve. (The ABI says undefined, Linux chooses zero to avoid info leaks). In your statically-linked executable, _start is the first user-space code that runs. (In a dynamic executable, the dynamic linker runs before _start, and does leave garbage in registers).

See also the x86 tag wiki for more asm links.