Adobe has made significant strides in enhancing its service delivery foundation through the implementation of a cell-based architecture called Flex-in-a-Box (FiaB). This innovative approach addresses the challenges faced by their initial architecture and provides a scalable solution for their rapidly growing CI/CD needs.
10 Key Architectural Principles for Scalable and Flexible CI/CD Infrastructure
For Adobe, building a robust and scalable CI/CD infrastructure is to maintain agility and efficiency. Drawing from our experience in redesigning our CI/CD architecture, we’ve identified 10 key principles that can guide similar efforts. Let’s delve into each of these principles and understand their significance.
1. Scalability on Demand
Principle: Ability to predictably add scale (horizontally and vertically) as needed, for short-term and future needs.
In a dynamic environment, your CI/CD infrastructure should be able to grow with your organization. This means having the capability to scale both vertically (adding more resources to existing nodes) and horizontally (adding more nodes) as needed. The key here is predictability – you should be able to forecast and plan for growth, ensuring that your infrastructure can handle increased load without compromising performance.
2. Service Relocation Flexibility
Principle: Ability to relocate services out of Flexbox 1 and relieve pressure.
As your infrastructure grows, you may need to redistribute services to optimize performance. Your architecture should allow for easy relocation of services between different components (in our case, Flexboxes) to balance load and maintain efficiency. This flexibility is crucial for managing resources effectively and preventing any single point of failure.
3. Abstraction of Infrastructure Details
Principle: The services should not decide which Flexbox they are on.
To maintain simplicity and ease of management, services should be agnostic to the underlying infrastructure. This abstraction allows for easier management and relocation of services without requiring changes to the services themselves. It’s the responsibility of the infrastructure layer to handle the routing and placement of services.
4. Backwards Compatibility
Principle: No impact of re architecture on any existing services (runtime or CI/CD) on Flexbox 1.
When evolving your architecture, it’s crucial to ensure that existing services continue to function without disruption. This principle emphasizes the importance of maintaining backwards compatibility, allowing for gradual migration and reducing the risk associated with architectural changes.
5. Runtime Continuity During Relocation
Principle: No impact on service runtime during box-2-box relocation.
When services need to be relocated, it should be done seamlessly without affecting the running instances. This ensures continuous availability and reliability of your services, even during infrastructure changes.
6. Single Association Rule
Principle: A service (app and deploy repos) should be associated with one Flexbox at a time.
To maintain clarity and prevent conflicts, each service should be associated with only one infrastructure component at a time. This simplifies management, troubleshooting, and ensures consistency in service behavior.
7. Minimal Downtime During Relocation
Principle: Minimal impact/downtime during relocation to the service’s CI/CD pipeline, if at all.
While relocating services, the goal should be to minimize or eliminate any downtime in the CI/CD pipeline. This ensures that development teams can continue their work with minimal disruption, maintaining productivity throughout the relocation process.
8. Transparent Communication
Principle: During relocation, the service owners should be aware of the relocation and the downtime.
Clear communication is key during any infrastructure changes. Service owners should be informed about relocations and any potential downtime, allowing them to plan accordingly and manage expectations within their teams.
9. Multi-Cluster Support
Principle: Ability to support more than one physical or virtual clusters inside a Flexbox.
Your architecture should be flexible enough to accommodate multiple clusters within a single logical unit. This allows for better resource utilization and provides options for isolating different types of workloads or environments.
10. Workflow Diversity
Principle: Ability to support non-container workflows as well e.g. Serverless, Cloud Infrastructure etc.
As technology evolves, your CI/CD infrastructure should be capable of supporting various types of workflows beyond just containerized applications. This includes serverless functions, cloud infrastructure provisioning, and other emerging technologies, ensuring your architecture remains relevant and useful in the long term.
Conclusion
In this blog, we consistently highlight industry-leading principles from web scale practitioners such as Adobe, which form a robust foundation for constructing scalable, flexible, and future-proof CI/CD infrastructure. By adhering to guidelines such as these, organizations can architect systems that not only meet current demands but are also well-positioned for future growth and technological advancements.. Our goal is to equip readers with the knowledge to make informed decisions that will position their systems for long-term success in an ever-evolving technological landscape. Through this approach, we strive to offer valuable perspectives that can help teams navigate the complexities of modern software development and delivery, ultimately driving innovation and efficiency in their organizations.
Featured Image by Valter from Pixabay