It is a fully-managed message queuing service that enables decoupling and asynchronous communication between different components or microservices within a distributed application. Here are some reasons why Amazon SQS is commonly used

• Secure: we can use SSE or SSE keys to encrypt messages.

• Durability: Stores messages on multiple servers to prevent the loss of messages.

• Availability: Redundant infrastructure, to provide highly concurrent access to messages and high availability for producing and consuming messages.

---

Types of queues

There are mainly two types of queues in SQS

FIFO Queues

A FIFO queue ensures that messages are processed in the order they were received. This is particularly important in situations where message order is critical, such as in financial transactions or when processing time-sensitive data.

• FIFO ordering

• Exactly once processing.

• 300 TPS max, without batching 3000 max ( per batch 10 messages ) ( ~ slightly costly ~25%)

Standard Queues

A standard queue does not guarantee the order in which messages are processed. With a standard queue, messages can be processed in any order, and duplicates may be introduced during transmission or processing.

• Order is not guaranteed, only attempt to process in order but not guaranteed.

• At least once delivery. May give the same message in subsequent requests ( Time stamping can help )

• Unlimited throughput

---

Amazon SQS short and long polling

Polling means simply checking out the queue for any messages from the consumer end.

• Short polling: Sends response right away if there is a message in the queue or not.

• Long polling: Sends response after at least one message appears in the queue.

Dead-letter queue

The purpose of a dead-letter queue is to isolate problematic messages and allow you to investigate and troubleshoot the issue without affecting the rest of the queue. Once the message is moved to the DLQ, it can be analyzed to determine the cause of the failure, and the necessary action can be taken to fix the issue.

The dead-letter queue of a FIFO queue must also be a FIFO queue.

Dead-letter queue of a standard queue must also be a standard queue.

You can configure the visibility timeout for your queue, and the default value is 30 seconds. However, you can set a timeout between 0 and 12 hours. When setting the timeout, you should consider the processing time of the messages and the maximum number of retries that you want to allow before sending the message to the Dead Letter Queue (DLQ).

If the visibility timeout is too short, and the application takes longer to process the message, the message may become visible again before processing is complete, resulting in multiple consumers processing the same message, which can cause problems in your system. On the other hand, if the timeout is too long, messages may remain invisible for too long, which can cause delays in processing.

Visibility Timeout

When a consumer application retrieves a message from an SQS queue, the message remains in the queue, but its status is changed to "in-flight" to indicate that it is being processed. During this time, the message is invisible to other consumers, which prevents multiple consumers from processing the same message at the same time.

The visibility timeout starts as soon as the message is retrieved by the consumer application, and if the application doesn't delete or acknowledge the message within this timeout, the message is made visible again in the queue for other consumers to retrieve.

Delay Queue

A delay queue in Amazon SQS is a type of queue that allows you to delay the delivery of messages for a specified period, useful when you want to schedule the processing of messages at a later time.

It is important to note that the maximum delay time for a message in a delay queue is 15 minutes. If you need to delay a message for a longer period, you can use other AWS services like AWS Step Functions or AWS Lambda to trigger the delivery of the message at a specific time in the future.

Message Timers

To set a delay period on an entire queue, rather than on individual messages, use delay queues. A message timer setting for an individual message overrides any DelaySecondsvalue on an Amazon SQS delay queue.

---

Message Processing Workflow

Two different threads cannot compete for the same message.

1. The message is published to the queue

2. The message is received by the receiver end and the visibility countdown starts. And the current message goes aside out of the queue (locked). And it is no longer visible to any other thread except that one which claimed it. A visibility countdown is a time set a threshold,

   • Visibility timeout expires, which means something is not right. and push the message back into the queue and remove the lock.

   • The message is processed and deleted by the consumer. Visibility timeout is cleared.

Why SQS over Api calls?

Back Pressure control: Allows the consumer to choose the rate of consumption request.

Fire and Forget : Publishers don’t have to worry about how consumers handle requests.

Eventual Guaranteed Processing: Good for async or non-realtime apps

Application decoupling: no direct coupling. A middle queue stores all the messages.

That's all about the basics of SQS, next I will be creating another tutorial-based article in which I will connect SQS with lambda and other Node.js servers.

Thanks for reading this article, hope you liked it. Any feedback is welcomed : )