Why does .json() return a promise?

Why does response.json return a promise?

Because you receive the response as soon as all headers have arrived. Calling .json() gets you another promise for the body of the http response that is yet to be loaded. See also Why is the response object from JavaScript fetch API a promise?.

Why do I get the value if I return the promise from the then handler?

Because that's how promises work. The ability to return promises from the callback and get them adopted is their most relevant feature, it makes them chainable without nesting.

You can use

fetch(url).then(response => 
    response.json().then(data => ({
        data: data,
        status: response.status
    })
).then(res => {
    console.log(res.status, res.data.title)
}));

or any other of the approaches to access previous promise results in a .then() chain to get the response status after having awaited the json body.

This difference is due to the behavior of Promises more than fetch() specifically.

When a .then() callback returns an additional Promise, the next .then() callback in the chain is essentially bound to that Promise, receiving its resolve or reject fulfillment and value.

The 2nd snippet could also have been written as:

iterator.then(response =>
    response.json().then(post => document.write(post.title))
);

In both this form and yours, the value of post is provided by the Promise returned from response.json().

When you return a plain Object, though, .then() considers that a successful result and resolves itself immediately, similar to:

iterator.then(response =>
    Promise.resolve({
      data: response.json(),
      status: response.status
    })
    .then(post => document.write(post.data))
);

post in this case is simply the Object you created, which holds a Promise in its data property. The wait for that promise to be fulfilled is still incomplete.

Also, what helped me understand this particular scenario that you described is the Promise API documentation, specifically where it explains how the promised returned by the then method will be resolved differently depending on what the handler fn returns:

if the handler function:

• returns a value, the promise returned by then gets resolved with the returned value as its value;
• throws an error, the promise returned by then gets rejected with the thrown error as its value;
• returns an already resolved promise, the promise returned by then gets resolved with that promise's value as its value;
• returns an already rejected promise, the promise returned by then gets rejected with that promise's value as its value.
• returns another pending promise object, the resolution/rejection of the promise returned by then will be subsequent to the resolution/rejection of the promise returned by the handler. Also, the value of the promise returned by then will be the same as the value of the promise returned by the handler.