Chloe McAree (McAteer)
Published on

AWS Kinesis

Authors

This is part of a blog series giving a high level overview of the different services examined on the AWS Solution Architect Associate exam, to view the whole series click here.

Kinesis Summary

Kinesis logo

  • Kinesis is a fully managed platform that can receive streaming data and process/analyse that data in real time.

  • Kinesis can ingest a range of different data such as video, audio, logs, analytics etc.

  • It is very scalable and can handle high amounts of streaming data from hundreds of thousands of sources all with very low latency.

  • 4 types: video streams, data streams, firehose, analytics

Kinesis Video Stream

  • Fully managed service that makes it easy to securely stream live video from devices to the cloud.

  • Enables playback, analytics and machine learning on video data that has been ingested.

  • Can capture data from millions of sources.

  • Automatically stores data and encrypts it at rest.

  • You can specify a custom retention period on data.

Kinesis Data Stream

  • Service for collecting and processing large amounts of real time data at scale.

  • Can capture gigabytes of data per second from hundreds of thousands of sources.

  • Collected data is available in milliseconds to be analysed and create analytics and dashboards.

  • Store data for a minimum of 24 hours

  • Takes data from a producer e.g. EC2 and passes it through a SHARD to the consumer

Shards

  • A shard is a sequence of data records in a stream with a fixed unit of capacity.

  • A shard can support 5 transactions per second for reads with a max 2mb read rate per second.

  • Only data streams have shards

  • The total capacity of a stream is the sum of the capacities of its shards

Kinesis Firehose

  • Fully managed service that allows you to reliably load real time streaming data into data stores like S3, RedShift, Elasticsearch etc.

  • Can compress, transform and batch data to minimise the amount of storage.

  • Encrypts your data streams before loading.

  • Pay for the volume of data that transmits through the service.

Kinesis Analytics

  • Kinesis Analytics allows you to analyse streaming data in real time to gain actionable insights.

  • Allows you to process and analyse data using standard SQL.

  • Can gain realtime dashboards and create real time metrics.

  • Can use with Data Streams or Firehose as the streaming source.