Posted by & filed under Amazon Web Services, node.js, Open-source. 13,110 views

Another itch I needed to scratch. I needed the basic ability to export data from (an Amazon Web Services noSQL store:) DynamoDB to CSV. I first built a solution for my specific issue (which included some transformations) and then decided it’d be worth building something more generic for people. While you can do this using AWS Pipeline, EMR and S3, I found it immensely complicated for such a simple task.

You can find my solution, DynamoDBtoCSV on github


AWS DynamoDBtoCSV

typically, to use you’d run:

node dynamoDBtoCSV.js -t Hourly_ZEDO_Impressions_by_IP > output.csv

to export to CSV

Use -d to describe the table prior so you can have an idea of the number of rows you are going to export

node dynamoDBtoCSV.js -t Hourly_ZEDO_Impressions_by_IP -d

to get some information about the table.

Full syntax is:

node dynamoDBtoCSV.js --help
    Usage: dynamoDBtoCSV.js [options]

Options:

    -h, --help               output usage information
    -V, --version            output the version number
    -t, --table [tablename]  Add the table you want to output to csv
    -d, --describe

Example output

"HashOf10","DateIPAdID","adcount"
"37693cfc748049e45d87b8c7d8b9aacd","2013011720024058205168000000010002","1"
"37693cfc748049e45d87b8c7d8b9aacd","2013011720050084232194000000010002","1"