Why is only index.html available from an S3 bucket served by CloudFront, and nothing else?

I am trying to expose a static site hosted in S3 though CloudFront.

The S3 bucket (testyop1) is set to host static websites (not in public mode) and its bucket policy is

{
    "Version": "2008-10-17",
    "Id": "PolicyForCloudFrontPrivateContent",
    "Statement": [
        {
            "Sid": "1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E...."
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::testyop1/*"
        }
    ]
}

The website hosting configuration states that index.html is the index document.

On the CloudFront side, the Origin Domain Name is set to testyop1.s3-website-eu-west-1.amazonaws.com without any Origin Path. The name has been taken from the S3 web hosting configuration (without the http://).

I can access https://xxxxx.cloudfront.net, as well as https://xxxxx.cloudfront.net/index.html (where xxxxx is the name generated by CloudFront)

All other calls to https://xxxxx.cloudfront.net fail with

403 Forbidden

Code: AccessDenied
Message: Access Denied
RequestId: F...D
HostId: i...V4X7l4=

despite .html files being present (both in the root directory and in a subdirectory).

Does that error message come from S3? (or CloudFront?)

What specific authorizations beyond the Bucket Policy above should be set?

You can't combine an Origin Access Identity with the website hosting feature. OAI is for REST only. Your objects have to be publicly accessible in order to use the website endpoint behind CloudFront, because website endpoints don't support authentication.