How can I fetch all items from a DynamoDB table without specifying the primary key?
Amazon DynamoDB provides the Scan operation for this purpose, which returns one or more items and its attributes by performing a full scan of a table. Please be aware of the following two constraints:
-
Depending on your table size, you may need to use pagination to retrieve the entire result set:
Note
If the total number of scanned items exceeds the 1MB limit, the scan stops and results are returned to the user with a LastEvaluatedKey to continue the scan in a subsequent operation. The results also include the number of items exceeding the limit. A scan can result in no table data meeting the filter criteria.The result set is eventually consistent.
-
The Scan operation is potentially costly regarding both performance and consumed capacity units (i.e. price), see section Scan and Query Performance in Query and Scan in Amazon DynamoDB:
[...] Also, as a table grows, the scan operation slows. The scan operation examines every item for the requested values, and can use up the provisioned throughput for a large table in a single operation. For quicker response times, design your tables in a way that can use the Query, Get, or BatchGetItem APIs, instead. Or, design your application to use scan operations in a way that minimizes the impact on your table's request rate. For more information, see Provisioned Throughput Guidelines in Amazon DynamoDB. [emphasis mine]
You can find more details about this operation and some example snippets in Scanning Tables Using the AWS SDK for PHP Low-Level API for Amazon DynamoDB, with the most simple example illustrating the operation being:
$dynamodb = new AmazonDynamoDB();
$scan_response = $dynamodb->scan(array(
'TableName' => 'ProductCatalog'
));
foreach ($scan_response->body->Items as $item)
{
echo "<p><strong>Item Number:</strong>"
. (string) $item->Id->{AmazonDynamoDB::TYPE_NUMBER};
echo "<br><strong>Item Name: </strong>"
. (string) $item->Title->{AmazonDynamoDB::TYPE_STRING} ."</p>";
}
Hi you can download using boto3. In python
import boto3
from boto3.dynamodb.conditions import Key, Attr
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Table')
response = table.scan()
items = response['Items']
while 'LastEvaluatedKey' in response:
print(response['LastEvaluatedKey'])
response = table.scan(ExclusiveStartKey=response['LastEvaluatedKey'])
items.extend(response['Items'])
I figured you are using PHP but not mentioned (edited). I found this question by searching internet and since I got solution working , for those who use nodejs here is a simple solution using scan :
var dynamoClient = new AWS.DynamoDB.DocumentClient();
var params = {
TableName: config.dynamoClient.tableName, // give it your table name
Select: "ALL_ATTRIBUTES"
};
dynamoClient.scan(params, function(err, data) {
if (err) {
console.error("Unable to read item. Error JSON:", JSON.stringify(err, null, 2));
} else {
console.log("GetItem succeeded:", JSON.stringify(data, null, 2));
}
});
I assume same code can be translated to PHP too using different AWS SDK
I fetch all items from dynamodb with the following query. It works fine. i create these function generic in zend framework and access these functions over the project.
public function getQuerydata($tablename, $filterKey, $filterValue){
return $this->getQuerydataWithOp($tablename, $filterKey, $filterValue, 'EQ');
}
public function getQuerydataWithOp($tablename, $filterKey, $filterValue, $compOperator){
$result = $this->getClientdb()->query(array(
'TableName' => $tablename,
'IndexName' => $filterKey,
'Select' => 'ALL_ATTRIBUTES',
'KeyConditions' => array(
$filterKey => array(
'AttributeValueList' => array(
array('S' => $filterValue)
),
'ComparisonOperator' => $compOperator
)
)
));
return $result['Items'];
}
//Below i Access these functions and get data.
$accountsimg = $this->getQuerydataWithPrimary('accounts', 'accountID',$msgdata[0]['accountID']['S']);