Linux kernel ARM exception stack init

I am using Linux kernel 3.0.35 on Freescale i.MX6 (ARM Cortex-A9). After running into a kernel OOPS I tried to understand the exception stack initialization. Here is what I have uncovered so far.

In cpu_init() in arch/arm/kernel/setup.c, I see the exception stack getting initialized:

struct stack {
    u32 irq[3];
    u32 abt[3];
    u32 und[3];
} ____cacheline_aligned;

static struct stack stacks[NR_CPUS];

void cpu_init(void)
{
    struct stack *stk = &stacks[cpu];

    ...<snip>

    /*
     * setup stacks for re-entrant exception handlers
     */
    __asm__ (
     "msr   cpsr_c, %1\n\t"
    "add    r14, %0, %2\n\t"
    "mov    sp, r14\n\t"
    "msr    cpsr_c, %3\n\t"
    "add    r14, %0, %4\n\t"
    "mov    sp, r14\n\t"
    "msr    cpsr_c, %5\n\t"
    "add    r14, %0, %6\n\t"
    "mov    sp, r14\n\t"
    "msr    cpsr_c, %7"
        :
        : "r" (stk),
          PLC (PSR_F_BIT | PSR_I_BIT | IRQ_MODE),
          "I" (offsetof(struct stack, irq[0])),
          PLC (PSR_F_BIT | PSR_I_BIT | ABT_MODE),
          "I" (offsetof(struct stack, abt[0])),
          PLC (PSR_F_BIT | PSR_I_BIT | UND_MODE),
          "I" (offsetof(struct stack, und[0])),
          PLC (PSR_F_BIT | PSR_I_BIT | SVC_MODE)
        : "r14");

I see that each stack has room for only three words. That is how the macro vector_stub in arch/arm/kernel/entry-armv.S uses it. It saves R0, LR (parent PC) and SPSR (parent CPSR) into those three words. Then it jumps to __irq_svc. That starts with a macro svc_entry which creates a stack frame

    .macro  svc_entry, stack_hole=0
 UNWIND(.fnstart        )
 UNWIND(.save {r0 - pc}     )
    sub sp, sp, #(S_FRAME_SIZE + \stack_hole - 4)

That is also how I see the disassembled code from KGDB:

Dump of assembler code for function __irq_svc:
   0xc01402c0 <+0>:  44 d0 4d e2    sub sp, sp, #68 ; 0x44
   0xc01402c4 <+4>:  04 00 1d e3    tst sp, #4
   0xc01402c8 <+8>:  04 d0 4d 02    subeq   sp, sp, #4
   0xc01402cc <+12>:     fe 1f 8d e8    stm sp, {r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12}
   0xc01402d0 <+16>:     0e 00 90 e8    ldm r0, {r1, r2, r3}
   0xc01402d4 <+20>:     30 50 8d e2    add r5, sp, #48 ; 0x30
   0xc01402d8 <+24>:     00 40 e0 e3    mvn r4, #0
   0xc01402dc <+28>:     44 00 8d e2    add r0, sp, #68 ; 0x44
   0xc01402e0 <+32>:     04 00 80 02    addeq   r0, r0, #4
   0xc01402e4 <+36>:     04 10 2d e5    push    {r1}        ; (str r1, [sp, #-4]!)
   0xc01402e8 <+40>:     0e 10 a0 e1    mov r1, lr
   0xc01402ec <+44>:     1f 00 85 e8    stm r5, {r0, r1, r2, r3, r4}
   0xc01402f0 <+48>:     ad 96 a0 e1    lsr r9, sp, #13
   0xc01402f4 <+52>:     89 96 a0 e1    lsl r9, r9, #13
   0xc01402f8 <+56>:     04 80 99 e5    ldr r8, [r9, #4]
   0xc01402fc <+60>:     01 70 88 e2    add r7, r8, #1
   0xc0140300 <+64>:     04 70 89 e5    str r7, [r9, #4]
   0xc0140304 <+68>:     54 50 9f e5    ldr r5, [pc, #84]   ; 0xc0140360
   0xc0140308 <+72>:     00 50 95 e5    ldr r5, [r5]
   0xc014030c <+76>:     0c 60 95 e5    ldr r6, [r5, #12]
   0xc0140310 <+80>:     4c e0 9f e5    ldr lr, [pc, #76]   ; 0xc0140364
   0xc0140314 <+84>:     07 0b c6 e3    bic r0, r6, #7168   ; 0x1c00
   0xc0140318 <+88>:     1d 00 50 e3    cmp r0, #29
   0xc014031c <+92>:     00 00 50 31    cmpcc   r0, r0
   0xc0140320 <+96>:     0e 00 50 11    cmpne   r0, lr
   0xc0140324 <+100>:    00 00 50 21    cmpcs   r0, r0
   0xc0140328 <+104>:    0d 10 a0 11    movne   r1, sp
   0xc014032c <+108>:    28 e0 4f 12    subne   lr, pc, #40 ; 0x28
   0xc0140330 <+112>:    32 eb ff 1a    bne 0xc013b000 <asm_do_IRQ>
   0xc0140334 <+116>:    04 80 89 e5    str r8, [r9, #4]
   0xc0140338 <+120>:    00 00 99 e5    ldr r0, [r9]
   0xc014033c <+124>:    00 00 38 e3    teq r8, #0
   0xc0140340 <+128>:    00 00 a0 13    movne   r0, #0
   0xc0140344 <+132>:    02 00 10 e3    tst r0, #2
   0xc0140348 <+136>:    06 00 00 1b    blne    0xc0140368 <svc_preempt>
   0xc014034c <+140>:    40 40 9d e5    ldr r4, [sp, #64]   ; 0x40
   0xc0140350 <+144>:    04 f0 6f e1    msr SPSR_fsxc, r4
   0xc0140354 <+148>:    1f f0 7f f5    clrex
   0xc0140358 <+152>:    ff ff dd e8    ldm sp, {r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, sp, lr, pc}^
End of assembler dump.

During an exception, SP is the banked R13. If I am following correctly, there is no room for this frame on that stack. That means I must have missed something. Is there some other place where the exception stacks are initialized?

Solution 1:

tl;dr - We switch modes to supervisor and use that stack.

You are missing the key point of where control is handed to the CPU via the vector table and the mode is switched. See: entry-armv.S and __vectors_start. The vector stubs is the code where control is initially sent after the branch in the main vector table. The vector_stub macro saves three items; a corrected lr, r0 and the spsr of the excepted mode (as you noted).

The point you miss is, after this all exceptions switch to SVC_MODE and as such use the current tasks stack, which also has the thread_info structure. mode switching is a tough concept to get in ARM system level assembler. Registers that were previously set are now completely different. Pay attention to msr and cps type instructions. Things can change completely after them; I have been confused by this dozens of times.

The spsr is used as an index into a vector_stub table, which will normally jump to either __irq_svc or __irq_usr. Just scroll down to look at the bottom of the entry-arm.S which you already found.

Related: Physical address of ARM-Linux vector table