Where does RET return to? [duplicate]

Solution 1:

CALL subroutine address is equivalent to
PUSH next instruction address + JMP subroutine address.

At the same time, PUSH address is nearly equivalent to
SUB xSP, pointer size + MOV [xSP], address.

SUB xSP, pointer size can be replaced by PUSH.

RET is nearly equivalent to
JMP [xSP] followed by ADD xSP, pointer address at the location where JMP leads.

And ADD xSP, pointer address can be replaced by POP.

So, you can see what kind of basic freedom the compiler has. Oh, btw, it can optimize your code such that your function is entirely inlined and there's neither a call to it, nor a return from it.

While somewhat perverse, it's not impossible to devise much weirder control transfers using instructions and techniques highly specific to the platform (CPU and OS).

You can use IRET instead of CALL and RET for control transfer, provided you put the appropriate stuff on the stack for the instruction.

You can use Windows Structured Exception Handling in a way that an instruction that causes a CPU exception (e.g. division by 0, page fault, etc) diverts execution to your exception handler and from there control can be transferred either back to that same instruction or to the next or to the next exception handler or to any location. And most of x86 instructions can cause CPU exceptions.

I'm sure there are other unusual ways for control transfer to, from and within subroutines/functions.

It's not uncommon to see code something like this either:

...
CALL A
A: JMP B
db "some data", 0
B: CALL C ; effectively call C with a pointer to "some data" as a parameter.
...

C:
; extracts the location of "some data" from the stack and uses it.
...
RET

Here, the first call isn't to a subroutine, it's just a way to put on the stack the address of the data stuck in the middle of the code.

This is probably what a programmer would write, not a compiler. But I may be wrong.

What I'm trying to say with all this is that you shouldn't expect to have CALL and RET as the only ways to enter and leave subroutines and you shouldn't expect them to be used for that purpose only and balance each other.

Solution 2:

A "well behaved" C program could be translated by a compiler to a program that does not follow this pattern. For example for obfuscation reasons the code could use a push / ret combination instead of a jmp.