- Streaming data pipeline whitepapers and/or sample architecture designs?
- Kensis Steams vs MSK
- Performance, not much was shared regarding MSK limits
- Since the max size of a data blob streamed in Kensis is 1MB, what are some architecture patterns for handeling larger blob sizes?
- Refrence data uploaded data to S3, same pattern used in SQS.
- We plan on streaming and batching data directly to Aurora PostgreSQL.
- Batching patterns
- Lambda vs EC2 on-demand instances workers for writing batches
- Lambda limits: https://docs.aws.amazon.com/lambda/latest/dg/limits.html
- Current state of retry patterns for failed Lambda functions
- Lambda function consuming a DLQ?
- Lambda vs EC2 on-demand instances workers for writing batches
- Batching patterns
- What's the general availability ETA for AWS MSK?
- Are there plans encrypt data-in-transit between brokers, clients, and and MSK?
- If yes, is this plan with the general availibity release.
- Whitepapers or additional resources regarding API Gateway Usage Plans with API Keys?
- Or any other resource related to setting up metered API access.
- https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-api-usage-plans.html
- What kind of analytics will the API Gateway provide, if any?
- Peneration testing request: https://aws.amazon.com/security/penetration-testing/
- Some whitepapers or resources on EKS application deployment patterns and tools?
- https://www.spinnaker.io/ (Framework)
- https://www.armory.io/ (Managed Spinnaker)
- https://harness.io/ (Service)
- https://www.spinnaker.io/ (Framework)
Reference:
Abstract diagram of data-pipeline: https://cacoo.com/diagrams/hGZG9Bkp2SJ8wgCw/5FB0B