Purpose of the article: This document focuses on high level process flow of MQTT payloads from IoT device to a mobile device by integrating an authentication & authorization mechanism. Step by step creation of AWS services, Terraform and bitbucket runner are out of scope of this document
Intended Audience:Cloud & IoT Enthusiasts, Cloud Architects, Developers and Engineers, Technical Managers and Team Leads
Tools and Technology: Cloud, AWS, IoT, MQTT
Keywords: AWS, IoT, IoT Core, MQTT, Kinesis, Cognito, S3
AWS IoT Low Level Architecture Diagram
Process Flow
- MDM device will select the IoT device to publish the messages.
- IoT devices are registered in AWS IoT Core and generate a certificate for authentication.
- X.509 certificate, private key and root certificate is placed in RFID reader for authenticating to AWS. And these certificates & keys will also be stored in AWS S3 bucket.
- Once authenticated, IoT device acts as a publisher and publishes the messages to a particular client ID and topics of IoT core. An IoT policy is created in AWS IoT core with the same client ID and topic to accept the messages.
- AWS IoT Rules are created to store the MQTT payloads in AWS S3 via AWS Kinesis firehouse.
- Scan App(on mobile device) act as a subscriber & must receive the MQTT payloads/messages which is published by the RFID readers.
- RFID portal application service contains all the RFID details. It is connected to AWS Cognito to get the temporary credentials.
- Once RFID portal application service gets the temporary credentials, scan app will make an API call to get these credentials and gets authenticated to MQTT broker (AWS IoT Core) and starts receiving the MQTT messages/payloads.
Required AWS Resources
- IoT Core
- IoT Thing
- IoT Policy
- IoT Certificate
- IoT Rule
- Kinesis Firehose
- S3
- CloudWatch
- Cognito
- IoT Core
-
- IoT Thing
- IoT Policy
- IoT Certificate
- IoT Rule
- Kinesis Firehose
- S3
- CloudWatch
- Cognito
IoT Core
AWS IoT Core is a managed cloud platform that allows devices to easily and securely interact with applications running on cloud and other devices. Below are the components of IoT core
| IoT Thing | IoT Thing Policy | IoT Thing Certificate | IoT Rule | MQTT Topics |
| A thing is a representation of a specific device or logical entity | Client ID, publisher and subscriber topics are mentioned in the IoT Policy. When the reader publishes with the same client ID and topic then only IoT core will connect and accept the messages. | Contains private key, public key, a x.509 certificate and a root ca certificate. Once these are created these are saved in the S3 bucket to get uploaded them to readers. | Rules are analyzed & based on the MQTT topic stream actions are performed. When reader publishes messages to a topic there will be a SQL query which runs on the topic will trigger IoT rule action and store these messages in S3 bucket via kinesis firehose. IoT rule will create a S3 bucket policy to access and put objects in S3 bucket. | MQTT topics identify AWS IoT messages. MQTT payloads will be sent on MQTT topic names |
| Example: Prod_Thing1 | Example: Prod_Thing1_Iot_Policy | Example: Abc1234******* | Example Prod_Thing1_IoT_rule | Example Prod_Thing1_IoT_topic1 |
IoT Thing Policy Example:
Kinesis Firehose
We are creating an AWS IoT Rule to store the MQTT payloads in S3 bucket via Kinesis Firehose.
Why Kinesis Firehose?
If you create a Direct Rule to save messages from IoT core to S3 by applying IoT core Rule, then only the last payload/message will get stored in S3 and all previous messages will be deleted. By using this method, concatenation of messages will not happen.
Hence, Kinesis Firehose is being used to overcome this issue. By using Kinesis Firehose, messages concatenation can be achieved and complete data/messages will be stored in S3.
Message flow will be like below:
Goto AWS IoT Core → Message Routing → Rules → create Rule
IoT Rule Example:
When the IoT device/reader publishes a message to the topic e.g.: Prod_Thing1_IoT_topic1
Then all messages hitting that topic will get stored in S3 via Kinesis Firehose. An IAM policy is created in AWS so that these services get communicated without any access issues.
S3 Bucket
We will be using two S3 buckets in this project
S3 Bucket 1: To save IoT certificates, private & public keys.
S3 Bucket 2: To save MQTT payloads coming from IoT devices.
CloudWatch
In this project we will be using Cloud watch logs to monitor both Kinesis Data Firehose and IoT logs
Kinesis Data Firehose logs:-
Kinesis Data Firehose logs delivery errors: –
LogGroup Name | /aws/kinesisfirehose/kinesis_logs |
LogStream Name | IoT_Kinesis_logs |
IoT Logs:
After enabling IoT logging, AWS IoT sends progress events about each message as it passes from your devices through the message broker and Rules engine.
Cloudwatch LogGroup Name | AWSIotLogsV2 |
Cloudwatch LogStream Name | <AWS generated LogSTream names> |
Cognito
Scan app wants to receive the messages from AWS IoT core, however to authenticate, scan app will connect to RFID portal application service and it will get AWS temporary credentials which will be provided by AWS Coginto federated identity pool.
Cognito Flow Diagram:
Cognito Deployment
- Create a userpool in Cognito which will generate a userpool ID.
2. Proceed for App client creation.
- Create App client.
App client name | mulesoft-br******* |
App Client ID | 16mv4oia****************** |
3. Create user in userpool.
User Name | c0b9e663-4f*********** |
User Email | <email ID> |
4. IdentityPool Creation.
Create Identity pool in Cognito by providing user pool id & App client id both created above..
- Do not check “Enable access to unauthenticated identities” as we need authenticated identities and won’t allow unauthenticated identities.
- Identity pool creation will create two IAM roles.
- Authenticated role
- Unauthenticated role
- We will be working with an Authenticated role and the credentials for this role are passed back to Application Team in the API call.
Identity Pool Name | Cog-idp-****-**** |
Identity Pool ID | us-east-1:302e3e46-f2**-***** |
5. ID token & AWS temporary credentials creation.
API will get ID token which then will be exchanged with AWS temporary credentials that will be used to connect to IoTCore.
API will return following AWS temporary credentials now –
- Identity token
- Access key
- Secret key
- Session token
- Identity ID
6. Identity token generated from userpool will be exchanged with AWS temporary credentials generated above to get connected with IoTcore.
7. Identity ID generated above can be visible in “Identity Browser” of Cognito.
| Identity ID | us-east-1:47f82ed-*****-****** |
8. IAM role updation.
Update Authenticated IAM role to add required IoT polices.
9. Attach Cognito policy with IoT thing policy.
To attach the Cognito policy with IoT thing policy need to run it with TF codes:-
Format: aws iot attach-principal-policy – -principal <Identity ID (from Identity browser)> –policy-name “<thing policy name>” – -region us-east-1
10. The above AWS CLI command will attach another certificate with the thing policy with label “cognito identity”
11. Finally, user will get authenticated using Cognito and the subscriber will get the MQTT message on the mobile device or on Scan App.
References:
- https://docs.aws.amazon.com/whitepapers/latest/aws-overview/internet-of-things-services.html
- https://docs.aws.amazon.com/cognito/latest/developerguide/authentication.html
- https://docs.aws.amazon.com/iot/latest/developerguide/iot-create-rule.html
- https://docs.aws.amazon.com/iot/latest/developerguide/iot-policies.html
- https://docs.aws.amazon.com/iot/latest/developerguide/device-certs-create.html
Author Bio:
Saurabh Grover
Manager - Cloud & Engineering - Infrastructure team
"Decisive, strategic, and performance-driven professional with expertise in presales and solution architecture. Adept at architecting, designing, deploying, and implementing innovative solutions in cloud, datacentre, and DevOps environments. Proven track record of delivering results that align with business objectives and drive organizational success."
