Why are some words combined into a single word while others stay as two words?

Language is always changing, and most often in the direction of simplification. You can even see the evolution happening before your own eyes. "All ready" became "already"; "all right" is in the process, through usage and repetition, of becoming "alright" (if not in fact "a'ight"). It is already accepted as an informal alternative to "all right" and I predict that it will supplant the two-word version altogether (!) except in the most formal writing (e.g., academic papers) within the lifetimes of many of us.

Pure convention. Unfortunately, there isn't a logical reason why some of those are written as a single word, and some aren't. It's essentially a matter of tradition. Consider especially the case of "no one", which is very clearly a single phonological word with a single word stress, but which has never been accepted as a compound.