Designing Cloud/Mobile solutions for infinite scale

The Democratization of Scalability

In a world where smartphones are used by the majority of consumers, computing systems that support transactions from thousands of concurrent remote devices are becoming more common.

20 years ago, processing at massive scale to support global deployments was affordable only to the largest IT organizations and tech companies. But with public cloud providers like AWS and Azure reaching mid-market enterprises, massive scale architectures are avalable to anyone able and willing to learn to use them.

The Importance of Queueing Patterns

Any system that has more clients than servers can generate spikes in volume that exceed the capacity of back-end resources. Anyone who has been on hold in a customer service hold queue has experienced a supply/demand imbalance mitigated through the use of queuing. For most customers, hold music&mdashwhile not desirable&mdashis preferable to a busy signal. Call center technology makes phone queues possible.

For an IoT or mobile device architecture, it’s basically the same problem with the same solution. Following the call center analogy, the “callers” in the IoT scenario may be mobile devices, industrial sensors or other IoT devices like household thermostats. The “call center agents” are event processors that pick up application messages from a queue, read the data contained in them, update databases, send notifications, and/or conduct streaming analysis.

Queue/Event Architecture Diagram

The following diagram illustrates a general model of an IoT processing architecture that utilizes a public cloud provider’s infrastructure.

I’ve differentiated between “Message” queues and “Event” queues, as these highlight overlapping but different use cases--even though they are fundamentally similar at a technical level.

Queue Diagram Image

In the diagram above, transactions are sent by a potentially large number of client devices to a finitely sized set of back-end resources. Without a message queue architecture, the solution architect needs to design back-end web front ends and databases to meet peak demand at all times (or use aggressive autoscalling configurations as a demand mitigation).

Even with autoscale, managing traffic spikes with queues can greatly help smooth out the flow of requests, and reduce the cost of meeting scale requirements.

Message Queue Processing

Message Queing isn't new, and is used to model transaction flows that needed to be highly consistent and reliable, and ensure the sequence of processing is perfectly FIFO. Message Queuing typically is done for processes that are waiting for a response, and can be thought of as a buffer to manage demand spikes. In this sense Message Queues are quite similar to the call center queue example.

Message Queues can also be helpful to allow requests that don't require immediate response to yield to other transactions that do. Good examples may be e-mail responses and database updates which need to be done, but can be done after a user interaction is completed.

Event Processing

Events can be clickstream telemetry, temperature or rainfall measurements, and other realtime data streams that are collected.

Event queues can be slighty different in the sense that for event queues there usually isn't a waiting device or user.

Is there an advantage to a technology specifically geared to event processing? The answer is yes—in a public clound environment, an event processing system is less expensive to provision and operate than a a message queue or traditional web application server.

Public Cloud Offerings

Public cloud infrastructures support message and event processing services, often with differentiated pricing tiers. The two most popular services — AWS and Azure — have specific offerings for both Message Queue and Event processing.

Each cloud provider has somewhat different approaches, but in general all support both message and event queue use cases in differentiated offerings depending on the intended use case. Selecting the right service is important to meet application needs appropriately, as well as keeping operating costs minimized.

Further Reading

Want to learn more about cloud message queues and event processing? Check out these other excellent articles:

Event Hubs vs. Azure Messaging

Amazon SQS — Message Queing Service

Azure Service Bus Fundamentals

Azure Event Hubs Overview

Publishing Data from iOS Devices to Azure Event Hubs

Call to Action