Nesting <p> won't work while nesting <div> will?
Because a paragraph is a paragraph .. and that's how HTML is defined (and HTML is not XML).
Any <p> (or other block-level element) will implicitly close any open <p>.
Per 9.3.1 Paragraphs: the P element of the HTML 4.01 specification:
The P element represents a paragraph. It cannot contain block-level elements (including P itself).
Note that this is how the HTML is parsed and that even a <div> would have implicitly closed the paragraph!
However, a <span> with display:block; would not have closed the <p> as a <span> is not a block-level element.
That is, the CSS is irrelevant at this stage of the HTML processing and the CSS is irrelevant to the DOM/parser when determining if an element is a block-level element or not. Consider the case when CSS is applied dynamically or through a not-yet-loaded-stylesheet: the applied CSS does not alter the DOM.
While the HTML5 (working-draft) specification does not include the language above in the HTML4 specification, it does go on to define a paragraph as a container for phasing content and further has a section on paragraphs.
The accepted answer to List of HTML5 elements that can be nested inside P element? says that <p> elements cannot nest in HTML5. The key phrase from the documentation is: "Runs of phrasing content [which does not include <p> elements] form paragraphs". Furthermore, HTML5, trying to be backwards-compatible in many aspects, has a rationale on "Restrictions on content models and on attribute values":
Certain elements are parsed in somewhat eccentric ways (typically for historical reasons), and their content model restrictions are intended to avoid exposing the author to these issues.
This behavior is referenced from a HTML5 WG wiki entry on flow content:
HTML5's restrictions on nesting of p elements and on what p elements may contain, are due to, quote: “peculiarities of the parser” that causes p to be auto-closed ..
From the HTML 4.01 specification section 9.3.1
The P element represents a paragraph. It cannot contain block-level elements (including P itself).