Skip to navigation
Build PipelinesAI Guides

AWS S3 Source Configuration

AWS S3 Source Configuration

  1. Navigate to Pipelines: Go to the Pipelines page in the left-hand sidebar and select the pipeline you wish to edit.
  2. Add Source: Click Add Source and choose the AWS S3 option.
  3. Configuration Fields:
  • Title: A descriptive name for the source (e.g., “S3-Logs-Prod”).
  • Description: Optional field for additional context.
  • AWS SQS URL: The AWS SQS queue URL where data is stored (required for ingestion).
  • AWS Access Key: Your AWS credentials (Access Key ID and Secret Access Key) from the AWS Dashboard.
  • Compression: Default is Auto; can be set to GZipNone or ZSTD.
  • Region: Choose an AWS region (default is US East 1).
  1. Save: Click Save to activate the source.

Note: AWS SQS is required for S3 integration, as data is read from the queue.