AWS (Amazon Web Services) provides a wide array of services for queuing and messaging, playing a crucial role in developing scalable and resilient architecture. Queuing shields the system from being overwhelmed by traffic, while messaging ensures the reliable communication between different system components. In this post, we will demystify these two concepts and explain how to use them effectively when preparing for the AWS Certified Solutions Architect – Associate exam.
1. Understanding Queuing with AWS SQS
Amazon Simple Queue Service (SQS) is a fully managed service that helps decouple and scale microservices, distributed systems, and serverless applications. SQS negates the need for controlling message-oriented middleware, providing developers with a highly scalable, reliable, and simple-to-use message queuing service.
The fundamental characteristics of SQS are:
- Highly Scalable: SQS dynamically scales up and down in synchrony with your application traffic.
- Reliable Delivery: SQS ensures the messages are delivered at least once.
- Supports Delay Queues: You can postpone message delivery for a specific duration.
SQS offers two types of queues:
- Standard Queues: Offers maximum throughput, best-effort ordering, and at-least-once delivery.
- FIFO Queues: Designed to ensure that messages are processed exactly once, by maintaining the order of their arrival.
2. Messaging and Pub/Sub model with AWS SNS
Amazon Simple Notification Service (SNS) is a flexible, fully managed pub/sub messaging and mobile notifications service for coordinating the delivery of messages to subscribing endpoints and clients.
The primary features of SNS are:
- Publish/Subscribe Messaging Model: SNS uses this model to fan out messages to a large number of subscribers, which can include SQS queues, AWS Lambda functions, HTTP(S) endpoints, and more.
- Message Filtering: To avoid overwhelming subscribers with irrelevant messages, SNS supports the filtering of messages based on subscription-filter policies.
- Scalability and Redundancy: Just like SQS, SNS scales automatically and offers durable storage of messages across multiple availability zones.
3. SQS and SNS Together
AWS has designed SQS and SNS to work seamlessly together, forming a powerful messaging framework. By combining the two, you can fan out a single published message to multiple SQS queues simultaneously. This improves the reliability of your system and allows multiple separate stakeholders (different services) to process the published message independently.
Example:
//Instantiate an Amazon SNS client and an Amazon SQS client
AmazonSNSClient snsClient = new AmazonSNSClient();
AmazonSQSClient sqsClient = new AmazonSQSClient();
//Create an Amazon SQS queue
CreateQueueRequest createQueueRequest = new CreateQueueRequest("YourQueue");
String myQueueUrl = sqsClient.createQueue(createQueueRequest).getQueueUrl();
//Get ARN of the SQS queue
GetQueueAttributesResult queueAttributes = sqsClient.getQueueAttributes(new GetQueueAttributesRequest(myQueueUrl,
Arrays.asList("QueueArn")));
String myQueueArn = queueAttributes.getAttributes().get("QueueArn");
//Create an Amazon SNS topic
CreateTopicRequest createTopicRequest = new CreateTopicRequest("YourTopic");
String myTopicArn = snsClient.createTopic(createTopicRequest).getTopicArn();
//Subscribe the SQS queue to the SNS topic
SubscribeRequest subscribeRequest = new SubscribeRequest(myTopicArn, "sqs", myQueueArn);
snsClient.subscribe(subscribeRequest);
//Publish message to the SNS topic
PublishRequest publishRequest = new PublishRequest(myTopicArn, "YourMessage");
snsClient.publish(publishRequest);
In the above example, a JavaScript code is used to create an SQS queue and SNS topic, link them, and send a message from SNS to SQS.
Understanding and properly utilizing queuing (SQS) and pub/sub messaging model (SNS) is critical for architecting scalable and resilient infrastructure on AWS. Moreover, these topics feature prominently in the AWS Certified Solutions Architect – Associate exam. By mastering these concepts, you will improve your chances of passing the exam and enhancing your AWS architectural skills.
Practice Test
True or False: Messaging is a form of communication between software components where the sender broadcasts a message to the receiver without waiting for the receiver to respond.
- True
- False
Answer: True
Explanation: Messaging enables application components to communicate and cooperate, but the sender does not need to wait for the receiver to respond before proceeding.
Which of the following statements about the publish/subscribe messaging model is correct?
- A) The sender broadcasts information to all subscribers.
- B) The receiver maintains a connection with the sender until it gets a response.
- C) The sender and receiver must be online at the same time.
- D) There is no option of persistent messaging.
Answer: A) The sender broadcasts information to all subscribers.
Explanation: In the publish/subscribe model, the sender (or publisher) broadcasts a message to all subscribers who have previously expressed interest in receiving such messages.
True or False: AWS SNS stands for Amazon Simple Notification Service and supports the publish/subscribe model.
- True
- False
Answer: True.
Explanation: AWS SNS indeed stands for Amazon Simple Notification Service, and it does support the publish/subscribe messaging model.
Which AWS Service is a fully managed queuing service for decoupling and scaling of microservices?
- A) Amazon SNS
- B) Amazon SQS
- C) Amazon SES
- D) Amazon SWF
Answer: B) Amazon SQS
Explanation: Amazon Simple Queue Service (SQS) is a fully managed service that helps decouple and scale microservices, distributed systems, and serverless applications.
True or False: In a queuing system, messages are pushed to consumers directly.
- True
- False
Answer: False.
Explanation: In a queuing system, messages are not directly pushed to the consumers. Instead, they are pulled by consumers from the queue when the consumers are ready.
True or False: AWS SQS supports both FIFO (First In, First Out) and non-FIFO queues.
- True
- False
Answer: True.
Explanation: Amazon SQS allows you to create FIFO queues as well as Standard (non-FIFO) queues, which offer maximum throughput, best-effort ordering, and at-least-once message delivery.
Which AWS service is used for reliable delivery of large amounts of data by dividing the data into a sequence of logical segments (sometimes called chunks or partitions)?
- A) AWS Kinesis
- B) AWS S3
- C) AWS Redshift
- D) AWS DynamoDB
Answer: A) AWS Kinesis
Explanation: Amazon Kinesis enables real-time processing of streaming data and it divides the data into a sequence of logical segments for reliable delivery.
True or False: In Amazon SNS, subscribers must pull messages from topics.
- True
- False
Answer: False.
Explanation: In Amazon SNS, messages are pushed to subscribers over various supported protocols, subscribers don’t have to pull messages.
In AWS, what’s the maximum visibility timeout for an Amazon SQS message?
- A) 1 Hour
- B) 12 Hours
- C) 24 Hours
- D) 48 Hours
Answer: C) 24 Hours
Explanation: The visibility timeout is the period of time that the message is invisible in the SQS queue after a reader picks up that message. It can be set up to 12 hours.
True or False: Amazon SNS can publish messages to Amazon SQS queues.
- True
- False
Answer: True.
Explanation: Amazon SNS can indeed publish messages to Amazon SQS queues. This allows for message storage, better scalability, and the ability to process messages multiple times.
In a publish/subscribe messaging model, the sender of the message is often referred to as:
- A) Consumer
- B) Subscriber
- C) Broker
- D) Publisher
Answer: D) Publisher
Explanation: In the publish/subscribe pattern, the sender of the message is often referred to as the publisher.
True or False: It is possible to set up Dead Letter Queues (DLQ) with Amazon SQS.
- True
- False
Answer: True.
Explanation: Yes, SQS supports Dead Letter Queues. Dead Letter Queues can process messages that can’t be processed successfully by the consumer.
Which protocol does Amazon SQS support?
- A) MQTT
- B) HTTP
- C) SMTP
- D) SSH
Answer: B) HTTP
Explanation: Amazon SQS is a web service, and as such, it uses the HTTP protocol.
True or False: Publish/Subscribe messaging is a one-to-one communication method.
- True
- False
Answer: False.
Explanation: Publish/Subscribe is a one-to-many communication method, where one entity, the publisher, sends messages to many subscribers.
Which AWS service can be best used for real-time analytics use cases?
- A) Amazon S3
- B) AWS Glue
- C) Amazon Kinesis
- D) AWS Athena
Answer: C) Amazon Kinesis
Explanation: Amazon Kinesis is designed to collect, process and analyze streaming data in real-time.
Interview Questions
What is the publish/subscribe messaging paradigm in AWS?
The publish/subscribe messaging paradigm is a pattern that is characterized by the sender (publisher) of a piece of data (message) not specifically directing it to a receiver. Instead, the publisher categorizes the message into a class, without knowledge of which, if any, subscribers there may be. Similarly, subscribers express interest in one or more classes, and only receive messages that are of interest, without knowledge of which, if any, publishers there are.
In AWS, which service uses the publish/subscribe model for sending messages between applications?
Amazon Simple Notification Service (SNS) uses the publish/subscribe model for sending messages between applications.
How does a queuing system like Amazon SQS work?
Amazon SQS works by storing messages sent between applications. When an application or system sends a message to another, it first sends the message to an Amazon SQS queue. The receiving application then retrieves the message from the queue when it’s ready to process it. This allows applications to run and process messages independently at their own pace.
What are the different types of Amazon SQS queues?
Amazon SQS offers two types of queues – Standard Queues and FIFO (First-In-First-Out) Queues.
What are dead-letter queues in the context of AWS SQS service?
Dead-letter queues are queues that other (source) queues can target for messages that can’t be processed (consumed) successfully.
Can Amazon SNS deliver notifications to Amazon SQS queues?
Yes, Amazon SNS can deliver notifications to Amazon SQS queues.
Can Amazon SNS work with Amazon SQS to fan out messages to multiple SQS queues simultaneously?
Yes, Amazon SNS can work with Amazon SQS to fan out messages to multiple SQS queues simultaneously.
How does a long polling process work in Amazon SQS?
Long polling helps reduce the cost of using Amazon SQS by eliminating the number of empty responses when there are no messages available to return in reply to a ReceiveMessage request sent by the consumer.
What are the main benefits of using a messaging system like Amazon SNS or SQS?
Some main benefits of using a messaging system like Amazon SNS or SQS include: decoupling the components of your system, scalability, reliability (ensuring messages aren’t lost and are delivered at least once), and flexibility to handle peaks of high usage.
What are the limitations of Amazon SNS in terms of message size and delivery mode?
Amazon SNS supports messages up to 256KB in size and supports both push and pull delivery modes. The supported delivery modes include HTTP/S, email, SQS, SMS, and Lambda functions.
Can Amazon SNS and Amazon SQS be used together?
Yes, Amazon SNS and Amazon SQS can be used together. For instance, an Amazon SNS topic can be configured to send messages to multiple Amazon SQS queues to allow simultaneous processing.
What is the maximum visibility timeout for an Amazon SQS message?
The maximum visibility timeout for an Amazon SQS message is 12 hours.
What is the retention period of messages in Amazon SQS?
Amazon SQS retains messages for a period of 1 minute to 14 days. The default retention period is 4 days.
Can the order of messages be preserved in Amazon SQS?
Yes, the order of messages can be preserved in Amazon SQS by using FIFO queues.
What are the delivery guarantees for Amazon SNS and Amazon SQS?
Both Amazon SQS and SNS guarantee delivery of a message at least once.
extutorials.com