GBDX

Manage S3 Contents with GBDXtools

Last Updated: April 24, 2019

You can use GBDXtools to manage the contents of your S3 location. GBDXtools supports downloading a single file or downloading the contents of a directory.

Download the contents of a directory or folder

PYTHON


#Download a directory of files from a GBDX S3 location
from gbdxtools import Interface
gbdx = Interface()

#Path to directory of files to download in GBDX S3 location
#location is relative to user's GBDX S3 location, ie s3://gbd-customer-data/<prefix>/
location = "dir_in_customer_bucket/files_to_download"

#Path to local directory for download
local_dir = "local_dir_name/downloaded_files"

gbdx.s3.download(location=location, local_dir=local_dir)

Download a single file

PYTHON

#Download a single file from a GBDX S3 location

from gbdxtools import Interface
gbdx = Interface()

#Path to file on GBDX S3 location
#location is relative to user's GBDX S3 location, ie s3://gbd-customer-data/<prefix>/
location = "dir_in_customer_bucket/remote_text_file.txt"

#Path to local directory for download
local_dir = "local_dir_name"

gbdx.s3.download(location=location, local_dir=local_dir)

Upload a file

PYTHON

#Upload a local file to a a GBDX S3 location

from gbdxtools import Interface
gbdx = Interface()

#Path to local file to be uploaded
local_file = "local_dir_name/local_text_file.txt"

#s3_path is relative to user's GBDX S3 location ie s3://gbd-customer-data/<prefix>/
#Optional s3 path will upload file to specified path/filename
s3_path = "dir_in_customer_bucket/remote_text_file.txt"
gbdx.s3.upload(local_file=local_file, s3_path=s3_path)

#If s3_path argument is not supplied, the file will be uploaded to the s3://gbd-customer-data/<prefix>/
gbdx.s3.upload(local_file=local_file)

Delete a file

PYTHON


#Delete a file from a GBDX S3 location

from gbdxtools import Interface
gbdx = Interface()

#location can be either a file or directory
#location is relative to user's GBDX S3 location, ie s3://gbd-customer-data/<pefix>/
location = "dir_in_customer_bucket/remote_text_file.txt"
gbdx.s3.delete(location=location)

Manage S3 Contents with GBDXtools


Last Updated: April 24, 2019

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.