Lessons Learned as a Programmer: Microservices: Design Patterns

"Microservices: Design Patterns":

Design Patterns for Microservices: Learn about various design patterns that can be leveraged when building and operating microservices, including decomposition, integration, data, and operational patterns.
Problem-Solving with Patterns: Understand how each pattern ties to specific problems and how they can help remedy common issues in microservices architecture.
Specific Patterns Covered: Gain insights into specific patterns such as the API gateway pattern, sidecar pattern, strangler pattern, and more, which help manage complexity and improve the efficiency of microservices.

"Vernacular of microservices":

Service Types: Understanding different types of services is crucial. These include data services, business services, translation services, and edge services.
Platform Definition: The platform encompasses all service operations across multiple data centers, including infrastructure, runtime, ancillary services, networking, and storage.
Operational and Diagnostic Components: These components are essential parts of the platform and should not be overlooked.

"Microservices and cloud native":

Distinct Concepts: Microservices and cloud-native are distinct concepts. Microservices do not inherently make a system cloud-native, and cloud-native does not require microservices.
Cloud-Native Architecture: Cloud-native is an architectural style focused on scalability, portability, and operating in the cloud. It includes practices like externalizing configuration and autoscaling.
Microservices: Microservices are smaller, scoped units of work that allow for independent scaling of different parts of a system. They are often used in cloud-native systems but are not a requirement.

"Decomposition of a system":

Decomposition Approach: Decomposing a system involves breaking down problems into smaller, manageable services, similar to breaking down software problems into reusable functions.
Service Types: Different service types are created to address specific uses across the system, such as domain-based services, business process-based services, and atomic transaction-based services.
Common Patterns: The strangler pattern is used to migrate from monolithic systems to microservices, and the sidecar pattern helps offload operational functions to separate components.

"Domain-based microservices":

Focus on Data Domains: Domain-based microservices are built around data domains, making services more scalable by focusing on specific data patterns and usage within the system.
Design Approach: Start with the domain model, not the database schema. The model should represent the domain to the outside world, and actions should be defined based on how the data is used.
Iterative Process: Designing domain-based services involves trial and error with fast iterations. Telemetry data is crucial for making informed decisions during this process.

"Business process-based microservices":

Higher Level of Abstraction: Business process-based microservices provide a higher level of abstraction around specific business functionality, helping to encapsulate use cases and scale them effectively.
Separation of Concerns: These services typically do not have their own data access, keeping business domains and data domains separate to avoid complexity.
Distinct Functional Uses: It's important to define distinct functional uses for each business process service to prevent creating a monolithic structure within your microservices architecture.

"Atomic transaction-based microservices":

Need for Atomic Transactions: Atomic transactions are necessary when true atomicity is required across multiple data domains, ensuring ACID-compliant transactions.
Complexity and Risks: Implementing atomic transactions adds complexity to the system. It's crucial to ensure that stakeholders understand the risks and constraints involved.
Design Considerations: To build atomic services, data domains must share a database. Clear rules and rollback conditions must be defined, and the service must handle complete commits or rollbacks.

"Strangler pattern":

Concept: The strangler pattern involves gradually migrating from a monolithic system to microservices by incrementally replacing parts of the monolith with microservices.
Implementation: Start by identifying and carving out dependencies from the monolith into new microservice endpoints, then redirect internal clients to these new microservices.
Strategy: You can either start at the API layer and move down to the database or begin with the data store and move up to business processes, depending on which part of the system needs the most attention.
Outcome: Over time, as more functionality is moved to microservices, the monolith is "strangled" and can eventually be deprecated.

"Sidecar pattern":

Purpose: The sidecar pattern offloads processing tasks (like logging, monitoring, and network services) to a separate module, reducing repetitive code across services.
Implementation: Deploy the sidecar as a module with each microservice, allowing it to handle specific tasks without modifying the main application code.
Benefits: This pattern reduces duplicate code, provides unified and focused behavior, and integrates seamlessly with the natural deployment process, enhancing scalability and flexibility

"Gateway pattern":

Purpose: The gateway pattern provides a buffer between underlying services and client needs, preventing chaos by managing how clients interact with system services.
Implementation: It acts as a single ingress point, handling tasks like security, authorization, payload mutation, and aggregation, while ensuring scalability and reliability.
Benefits: The gateway can limit access, aggregate data, and provide a consistent API contract, helping to manage client interactions without exposing internal services directly.

"Process aggregator pattern":

Purpose: The process aggregator pattern is used to combine multiple business processes into a single API call, assembling a composite response for the client.
Implementation: It encapsulates business logic and consolidates payloads, ensuring that clients do not need to reproduce the same set of rules, adhering to the DRY (Don't Repeat Yourself) principle.
Caution: Use this pattern carefully as it can introduce a choke point in your system, potentially causing long-running processes and increased network IO. Consider asynchronous patterns for large data processing operations.

"Edge pattern":

Purpose: The edge pattern is a subset of the gateway pattern, designed to handle scalability concerns and client-specific needs by acting as client-specific gateways.
Benefits: It provides aggregation, consolidation, and complexity isolation tailored to a specific client, improving scalability and flexibility.
Implementation: Focuses on the needs of a single client, allowing for targeted processing, better scalability, and easier maintenance compared to a general gateway.

"Single service database" pattern:

Purpose: This pattern is used to ensure that each microservice has its own dedicated database, which helps in scaling both the service and the database independently.
Implementation: As load on a service increases, the associated database can be scaled independently to meet the demand, ensuring efficient resource utilization.
Benefits: Isolating the data domain and its store allows for independent scaling, reducing the impact on the overall system and potentially enabling regional data isolation for advanced use cases.

"Shared service database" pattern:

Usage: Sometimes necessary due to existing contracts or startup constraints, even though single service databases are generally preferred.
Implementation: Treat data domains as separate databases within a single engine using schemas or logical groupings to maintain isolation.
Best Practices: Ensure each service has unique credentials and avoid cross-schema access to facilitate future database separation and maintain logical isolation.

"Command Query Responsibility Segregation (CQRS)":

Complexity: CQRS separates read and write operations into different models, increasing system complexity but improving data behavior across the system.
Use Case: It's particularly useful for task-based user interfaces and event-driven models where eventual consistency is acceptable.
Implementation: Requires careful design and significant homework to implement correctly, as improper implementation can lead to maintainability and operational challenges.

"Asynchronous eventing":

Purpose: Asynchronous eventing addresses long-running transactions or complex workflows that cannot fit into a single blocking API call.
Implementation: It involves triggering an event through a service API, which then cascades asynchronously, allowing behind-the-scenes processing after the client receives an accepted message.
Benefits: This pattern is powerful in distributed systems, solving complex problems by enabling non-blocking operations and improving overall system efficiency.

"Log aggregation patterns":

Purpose: Log aggregation helps in diagnosing and resolving errors quickly by consolidating log messages from various services into a single stream.
Consistency: Ensuring consistent log structure and format across all services is crucial for effective log aggregation and indexing.
Taxonomy: A common taxonomy for log messages should be shared across all services to facilitate easier parsing and correlation of logs.

"Metrics aggregation patterns":

Importance: Metrics are crucial for diagnosing system issues and can be more useful than logging due to less human interaction and easier instrumentation.
Consistency: Using a common taxonomy for metrics, similar to logs, helps in creating clear and descriptive dashboards.
Dashboards: High-level dashboards for overall system health and detailed dashboards for individual services are essential for effective monitoring and quick issue resolution.

"Tracing patterns":

Purpose: Tracing helps recreate the call stack in a microservices architecture by injecting a trace identifier into every call, spanning from the edge to the database.
Implementation: Use a standards-based approach for tracing to leverage off-the-shelf tools and ensure every log message embeds the trace ID.
Benefits: Tracing aids in diagnosing issues by ensuring no call is lost and providing visual call stacks through application performance monitoring (APM) tools.

"External configuration":

Operational Importance: Externalized configuration is crucial for operational efficiency, especially during load and movement of services, as it helps in quickly resolving issues.
Tooling: Use tools that make external environment variables easy to find and manipulate. Consistent naming of variables is also very helpful.
Security: While exposing configuration, ensure that secrets are protected and accessible securely to the on-call team without exposing them to unauthorized access.

"Service discovery":

Purpose: Service discovery helps manage the complexity of finding appropriate services in a dynamic runtime where service locations can vary.
Functionality: A central location allows new services to advertise themselves and their capabilities, enabling other services to query and find the needed services.
Benefits: This pattern enhances flexibility and scalability, making it easier to manage and locate services as the architecture grows.

"Continuous delivery":

Definition: Continuous delivery is the process of constantly delivering new code to production with full automation, moving artifacts through non-production environments to production using automated gates.
Strategies: Key strategies include automated integration tests, security testing (both penetration and internal security tests), and user acceptance testing (UAT) to ensure confidence in moving code to production.
Importance: CI/CD (Continuous Integration/Continuous Delivery) is crucial for microservices, requiring strong automation patterns to maintain efficiency and reliability.

"Documentation":

Critical Role: Documentation is essential in a microservices architecture for effective inter-team communication, especially when multiple teams are involved.
Tools: Use tools like OpenAPI for documenting APIs, whether you prefer contract-first or code-first development.
Consistency: Ensure each service exposes its documentation at a consistent URL (e.g., /docs) and consider automating the publication of documentation to a central repository.
Enhancements: Include examples of API consumption and team contact information in the documentation to improve usability and collaboration.