Orchestration
Orchestration in Action: Handling Not-So-Happy Paths in Modern Software Systems
In an ideal world, software architecture would be straightforward if only happy paths existed, but real-world applications are complex. Modern architectures have many moving parts, routes, and services that interact with each other. In addition, they must handle different conditions and errors.
One effective way to manage these complexities is through orchestration.
What is Orchestration?
Orchestration is a pattern used to coordinate the execution of distributed services to complete a business process. It is typically used when several independent services must work together to complete a task. Some call them orchestrated transactions.
Why Do We Need Orchestration?
In modern software systems, particularly those built on microservice architectures, workflows often involve multiple services that must work together. Each service may have its own data and business logic, and to complete a workflow, the system needs to coordinate the interactions between these services.
Without a central mechanism to manage these interactions, complexity and potential errors increase significantly. Orchestration addresses these challenges by providing a centralized control mechanism that:
- Manages the workflow
- Ensures that each service is called in the correct order
- Handles errors appropriately
This centralized control facilitates the management of complex workflows, error handling, and consistency assurance between services.
Key Features of Orchestration
Centralized Control
Orchestration provides a single point of control for all workflow decisions. This central entity is the Orchestrator, which manages the sequence of service invocations and controls the logic for error recovery, retries, and state management.
Service Interaction
The orchestrator interacts with many services, calling them in a specific order and handling the data flow between them. This interaction can be synchronous (waiting for a response) or asynchronous (continuing without waiting for a response).
State Management
The orchestrator maintains the state of the workflow, tracking which steps have been completed, their current status, and what needs to be done next. This state management is crucial for handling complex workflows and ensuring consistency.
Extensibility
You can extend orchestrated workflows by adding or modifying new services and updating the central orchestrator to accommodate the changes without significant system disruption.
Error Handling and Compensation
Orchestration enables sophisticated error handling and compensation logic. If something goes wrong, the orchestrator can execute retries, use alternative workflows to handle the failure, or reverse operations. This is one of the strengths of orchestration in system design.
Handling Routes with Orchestration: A Practical Example
In this example, we explore how orchestration can manage the workflow of an e-commerce system involving many services.
For this example, we will focus on:
- Orders
- Shipments
- Notifications
Happy Path
1. Customer Places an Order: The process begins when the customer submits a "Place Order" request to the Order Orchestrator.
2. Order Creation: The Order Orchestrator receives the request and forwards it to the Order Service to create the order. The Order Service processes the request and creates a new order record in its database.
3. Shipment Request: Once the Order Service confirms the order creation, the orchestrator asynchronously calls the Shipment Service to initiate the shipment process. This is done because shipping the order takes time and does not need to block the workflow.
4. Order Shipped: The Shipment Service processes the shipment request. After the order is shipped, it notifies the orchestrator of the successful shipment.
5. Order Status Update: Upon receiving the shipment confirmation, the orchestrator updates the order status to "Shipped" in the Order Service.
6. Notification: Finally, the orchestrator calls the Notification Service to send a success email informing the customer that their order has been shipped.
Not-So-Happy Path (Alternative Flow)
1. Customer Places an Order: The customer submits a "Place Order" request to the Order Orchestrator.
2. Order Creation: The Order Orchestrator forwards the request to the Order Service, which creates the order.
3. Shipment Request: The orchestrator asynchronously calls the Shipment Service to initiate the shipment process.
4. Pending Order Notification: The Shipment Service finds that there is not enough inventory to fulfill the order and notifies the orchestrator of the pending order status.
5. Order Status Update: The orchestrator updates the order status to "Pending" in the Order Service to reflect the issue.
6. Notification: The orchestrator calls the Notification Service to send a pending order notification to the customer, informing them about the issue with their order.
Failed Route (Compensatory Transactions)
When an error occurs during a transaction, it may be necessary to undo previously completed steps to maintain system consistency. This is called compensatory transactions. For example, in systems without support for pending orders, this would look like:
1. Customer Places an Order: The customer submits a "Place Order" request to the Order Orchestrator.
2. Order Creation: The orchestrator calls the Order Service to create the order.
3. Shipment Request: The orchestrator calls the Shipment Service to process the shipment.
4. Inventory Issue: The Shipment Service detects insufficient inventory to fulfill the order and notifies the orchestrator.
5. Compensatory Transactions Initiated:
- Cancel Order: The orchestrator calls the Order Service to cancel the order.
- Reverse Payment: If the payment was processed, the orchestrator may need to send a message to the Payment Service to refund the amount to the customer.
- Order Status Update: The orchestrator updates the order status to "Canceled" in the Order Service.
6. Notification: The orchestrator sends a cancellation notification through the Notification Service.
Not All Operations are Synchronous
As mentioned, this interaction can be asynchronous. While the concept of orchestration suggests a centralized controller and defined steps, it is important to note that not all orchestrators operate synchronously. Event-driven orchestration is an architecture you can use to implement Microservice Orchestration in a completely asynchronous manner.
The orchestrator remains responsible for workflow logic and failure handling in both cases. The only thing that changes is how the orchestrator communicates with downstream microservices.
Benefits of Event-Driven Workflows
- You can use the same monitoring and scaling functionality for the orchestrator as for other event-driven microservices.
- Events can be consumed by other services, including those outside of the orchestration.
- The orchestrator and dependent services are isolated from intermittent failures of each other.
- They have a built-in retry mechanism for failures, as events can stay in the broker for retries.
The orchestrator can continue processing other tasks while waiting for responses from services. This approach is particularly valuable in distributed systems where service latencies can vary and workloads can be unpredictable. The orchestrator is responsible for materializing and maintaining the state of the events.
In this case, event 123 (a representation of an order, for example) has been successfully processed, while event IDs 124 and 125 are at different stages of the workflow. The orchestrator can make decisions based on these results and select the next step according to the workflow logic. Once the events are processed, the orchestrator can also take the necessary data from the results of services A, B, and C and compose the final result.
Assuming the operations in services A, B, and C are independent of each other, you can change the workflow by modifying the order in which you send the events. Additionally, keep in mind that there will be many opportunities to combine direct calls and event-driven flows within the same orchestrator, and this combination will be your sweet spot.
Benefits and Drawbacks of Orchestration
Benefits of Orchestration
- Centralization: The orchestrator acts as a centralized entity where all behaviors, routes, and error handling logic are implemented. This centralization simplifies workflow management by providing a single place to manage all interactions and decisions.
- Resilience: The orchestrator can incorporate retry logic to handle temporary service outages. If a service is momentarily unavailable, the orchestrator can retry the request after a specified interval, improving the system's resilience and reliability. This retry mechanism allows the system to degrade gracefully rather than fail completely, improving the user experience during transient failures.
- State Management: The orchestrator maintains the state of the workflow, making it easy to query the current status of an ongoing process. This state management is critical for monitoring, debugging, and understanding the workflow's progress.
Drawbacks of Orchestration
- Bottleneck: Since all communication must pass through the orchestrator, it can become a bottleneck, especially in high-performance scenarios.
- Performance Overhead: The additional layer of orchestration introduces some performance overhead due to the additional communication and processing required. This can impact system latency and performance.
- Single Point of Failure: The orchestrator is a critical component of the workflow. If it fails, it will disrupt the entire workflow, becoming a single point of failure. However, you can address this with redundancy strategies, such as deploying multiple instances of the orchestrator and using load balancing to distribute the load.
Key Takeaways
- Centralized Management: Orchestration manages workflows from a single location, making it easier to handle different routes and errors.
- Improved Resilience: Orchestrators can retry tasks if a service temporarily fails, enhancing system reliability.
- Progress Visibility: Orchestrators keep track of workflow status, making it easy to check progress and debug issues.
- Asynchronous Scalability: Asynchronous orchestration can handle multiple tasks at once without waiting for responses, making the system more scalable. Tasks can be executed in parallel, speeding up the overall process.
- Potential Bottleneck: The orchestrator can slow down the system if it becomes overloaded.
- Redundancy Needed: If the orchestrator fails, it can halt the entire workflow. Redundancy is necessary to prevent this.
In simple terms, orchestration puts something in charge: the Orchestrators, who are aware of the entire workflow. If there is any problem during the process, the Orchestrator will know about it and take action to handle it or simply notify a failure.
Comments