Amazon S3
The Amazon S3 provider for Data Sync supports read-write operations against an Amazon S3 bucket.
Since S3 stores binary objects you need to connect this provider with a Data Sync provider that supports binary blob or file data.
The following Data Sync providers can be connected to the Amazon S3 provider.
- FileSystem Provider
- SharePoint Document Library
- SQL Server Table with Blob Column configuration.
- Azure Blob Storage
The Amazon S3 connector also supports incremental sync mode.
Connect
To connect first enter your AccessKeyID and SecretAccessKey, then select the correct region for your bucket, and finally choose the Bucket Name to connect to.
Properties
| Property | Description |
|---|---|
| BucketName | The name of the S3 Bucket to connect to. |
| Prefix | Defines a Prefix to be used with S3 bucket queries, the Delimiter is always '/'. |
| PrefixPathRemove | Removes the Prefix from the Path when reading the S3 bucket list. |
| Region | The AWS region that contains your SimpleDB Domain. |
| AccessKeyID | The AWS AccessKey. |
| SecretAccessKey | The AWS Secret. You can also use AWS IAM Users/Roles when running from an AWS VM with an IAM role assigned. |
| WebFriendlyPaths | Ensures that file paths are returned with forward slashes (/). |
UseCustomMetaDataForDates
Defines two custom metadata fields to contain the file created and modified timestamps in UTC. Data Sync will then use these rather than the Amazon S3 LastModified field which cannot be modified.
- x-amz-meta-sourcemodifieddate
- x-amz-meta-sourcecreateddate
Data Schema
Below is the Data Schema that Data Sync returns for the Amazon S3 provider. The actual file data is not part of the schema and is read from Amazon S3 when required during the Synchronisation operation.
| Column | DataType | Description |
|---|---|---|
| FullFileName | String | The full relative path to the item including filename. |
| Path | String | The full relative path to the item. |
| FileName | String | The filename of the item. |
| DateCreated | DateTime | Date Created [x-amz-meta-sourcecreateddate] or LastModified |
| DateModified | DateTime | Date Modified [x-amz-meta-sourcemodifieddate] or LastModified |
| Length | Int64 | The Length/Size of the file in bytes. |
| Checksum | String | The MD5 checksum of the file. |