AWS Boto3 Failure to Upload File Object

I am facing an issue where I am trying to test my authentication to upload files into an S3 bucket. Currently I am getting an access denied issue, but I want to ensure that my configuration is working as expected. The way I understand this code to be working is that I am using the profile configuration 'PROFILE_CONFIG_1'. This means I am capturing the access_keys and secret keys with this configuration.

I am then simply taking my local file and attempting to place in the S3 container. My doubt is that I am able to use the S3 browser to actually place the file in the path as expected, but the code is telling me differently.

def upload_to_aws():
    session = boto3.Session(profile_name='PROFILE_CONFIG_1')
    dev_s3_client = session.client('s3')    

    local_file = 'test.txt'

    bucket = 'myBucketName'

    filename = 'path/to/dir/'

    with open(local_file, 'rb') as f:
        dev_s3_client.upload_fileobj(f, bucket, filename) # Fails with error (AccessDenied) when calling the PutObject operation

UPDATE: I have attempted to add a file with the AWS CLI and everything worked as expected. Only now am I seeing an Access Denied issue in the python code. I also simplified by code to something like this, and still see the error message.

session = boto3.Session(profile_name='PROFILE_CONFIG_1')
s3_client = session.client('s3')
s3_client.upload_file('Path_To_File\\test.txt', 'myBucketName','test.txt')

Update 2 for John:

s3 = boto3.client('s3')
with open("test_file.txt", "rb") as f:
    s3.upload_fileobj(f, 'take-uat-ics', 'destination_path/test_file.txt')

Solution 1:

There are two related problems in your posted code with your target S3 object keys.

The 1st is the use of path/to/dir/ as the target S3 object key. S3 copy doesn't work like regular Windows/Mac/Linux file copy where you would be able to copy a file to a folder using something equivalent to cp file.txt /usr/mary/ that results in /usr/mary/file.txt. That doesn't work with S3 APIs. The target must be the S3 object's full key.

The 2nd problem is the use of leading forward slash in /destination_path/test_file.txt. S3 keys don't require, or allow, leading forward slash as an indicator of the root of the S3 bucket.

Valid S3 object keys neither start nor end in forward slash. Examples of valid S3 object keys are:

dogs/small/poodle.png
destination_path/test_file.txt