Building an eCommerce System with DDD and CQRS

Managing product categories is one of the most important components of an E-Commerce system. Along with order management and shipping, it forms the three main pillars of an E-Commerce system. Therefore, building a product category management system plays a crucial role in the overall architecture. It must meet the following objectives:

Product Model

Managing diverse product attributes:

• This involves effectively managing the diverse attributes associated with each product, such as size, color, material, and any other relevant characteristics.
• The system should provide functionalities to add, edit, and delete attributes as needed.
• Additionally, it should allow for the creation of custom attributes to accommodate unique product specifications.

Ease of maintenance and scalability:

• The system should be designed for easy maintenance and scalability in terms of both business requirements and system infrastructure.
• It should support modular architecture, allowing components to be added or modified without disrupting the overall system functionality.
• Furthermore, the system should be flexible enough to adapt to changing business needs and accommodate future expansions without requiring extensive redevelopment.

High-performance capabilities:

• The system should be capable of meeting high-performance requirements to ensure smooth and efficient operation, even under heavy loads.
• It should optimize database queries, caching mechanisms, and other performance-enhancing techniques to minimize response times and maximize throughput.
• Scalability measures such as load balancing and horizontal scaling should be implemented to handle increasing traffic and user demands.

Integration challenges resolution:

• The system should address integration challenges seamlessly by providing standardized interfaces and protocols for communication with other systems.
• It should support data exchange formats such as JSON, XML, or RESTful APIs to facilitate smooth integration with external systems.
• Robust error handling mechanisms should be in place to handle integration failures gracefully and ensure data consistency across integrated systems.”

A product is an object that displays information for users to make purchases. A product has diverse attributes in terms of quantity, type of attributes, and data types. Each attribute may have different data characteristics such as length, character type, and business requirements. Therefore, the product model is a model with no fixed structure (schemaless structure).

A common approach to address this issue is to use the EAV (Entity-Attribute-Value) model. This model is used by platforms like Magento and WordPress to store diverse structures without predefined schemas. The characteristic of the EAV structure is to separate the storage of attribute values from the structure of the model’s attributes. This allows for defining various types of attribute structures for products

An attribute set is a collection of attributes. Each attribute is distinguished by an attribute code and a data type. An attribute can belong to multiple attribute sets. A product belongs to an attribute set. The attribute values of a product correspond to the attributes, characterized by their data types and attribute codes. The data type of an attribute defines the attribute’s values. By organizing the system in this way, it becomes easy to add new attribute sets as well as to add attributes to existing attribute sets.

Furthermore, a product also has multiple images, belongs to multiple categories, and has various variants (related to other products). For example, an iPhone may have variant products based on color, or a dress may have different sizes, related to other dress products with different sizes, distinguished by the size attribute.

Business logic layer design.

Business logic will be developed around the built model. This is the most complex layer, accounting for the majority of the workload. Product management business focuses on changing the diverse attributes of the product. Design goals:

Ensure the best Don’t Repeat Yourself principle. Business logic is clearly separated, concise, and highly focused. Can extend, modify independent components. Flexibility in development. Capable of easy testing and maintenance. These design goals will influence various software application design patterns. The following section will analyze each software design pattern to provide the most appropriate choice.

MVC Model.

The MVC model is a popular choice for building applications. It is a simple model, easy to understand, and quickly implements features. According to the MVC model, the product catalog management part will have the following architecture:

With the MVC model, the product model will contain all the definitions of product attributes, database access, and business logic execution. Typically, the MVC model will use the Active Record pattern to map with the database. This is a simple approach but has the following limitations:

The product model becomes too heavy when it contains too much logic from attribute definitions, data access, and business logic execution. Due to the inability to separate business and data layers, it’s difficult to perform unit tests on individual components. Changing data access logic and business logic is difficult due to dependencies between components. The encapsulation of business logic is not guaranteed. Business logic will only be implemented according to the CRUD model. Therefore, the more the application progresses, the higher the duplication of logic. Due to these limitations, this is not a suitable model for complex logic applications like ecommerce. It will lead to high maintenance costs in the long run.

Three-tier model.

The next approach is to analyze and design according to the Domain-Driven Design method. The characteristics of this model are:

Separation of the business layer from the application layer and the data access layer. Known as the Domain Layer. This is where all the business logic is centralized. In the business layer, focus on designing the model to reflect the most comprehensive and consistent characteristics of the business. Divide the business layer into two separate components: Domain Model and Domain Service. The Domain Service plays a role in providing business functions externally around the system’s domain models. Design a separate infrastructure layer containing logic for data access, database operations, and message queues.

With this organization, the application, business, and data access layers will be separated and only interact through interfaces. The business functions will revolve around the product model. The data access layer will retrieve or store product objects.

However, applying the three-tier model focusing on the product model also leads to another difficulty, which is providing models for data-reading business functions. These business functions are very diverse, and the most effective way is often to use raw SQL queries. However, if the retrieval tasks are tightly coupled with ORM models, it will affect performance and make it difficult to change data retrieval logic. It also affects development speed, as implementing diverse query business functions for serving clients is hindered and depends on the design of models and the Repository layer.

CQRS Model.

The CQRS (Command Query Responsibility Segregation) model is an extension of the three-tier model in DDD. An important characteristic of CQRS is the separation of two parts of logic: reading and writing data into two separate parts:

1. Writing data: This is done by sending commands to handlers through a command bus. Command handlers act similarly to domain services and interact with models to perform data-changing operations.
2. Reading data: This part is designed independently, not relying on the models of the writing data part. Therefore, it can be flexible in accessing the database, as well as using different data sources to optimize retrieval speed.

The CQRS model addresses the limitations mentioned in the previous section. CQRS brings significant advantages:

1. It allows for separate development and optimization of the reading data part from the writing data part.
2. Modeling data-writing business operations under commands enables effective encapsulation of business logic, facilitating easier scalability. Moreover, these commands can easily switch between synchronous and asynchronous processing through an abstract layer like the command bus without changing the model, ensuring a consistent and coherent model throughout the architecture.

All three models above have their own pros and cons:

1. The MVC model is simple but limited in handling complex business logic, testing, and scalability.
2. The three-tier model is suitable for handling complex business logic but has limitations in optimizing the reading data part.
3. The CQRS model extends from the three-tier model, effectively separating reading and writing, but requires more complex design.

For large ecommerce systems, the CQRS model is the most suitable as it addresses both major requirements: handling complex business logic and meeting high performance demands. The following section will delve deeper into system aspects and an extended variant of CQRS, known as CQRS-ES (Command Query Responsibility Segregation – Event Sourcing).

System Design.

One of the major requirements for managing product catalogs is to design to meet performance needs and integrate with other systems. To meet these requirements effectively, the system must be designed holistically from the outset. This ensures seamless development, consistency, and stability, as well as work progress.

CQRS – ES Model.

The Event Sourcing – ES model is a design pattern where the state of an object is stored as a sequence of events. It differs from conventional design patterns, which typically only store the final state of an object.

When an object is changed, it corresponds to a versioned event storing the object’s change data. These events are stored in an “append-only” fashion in a table structure called the event store. Storing these change events brings several benefits:

1. Preserving the change history of objects.
2. If the events are sent to other systems, those systems can generate the accurate final state of the original object. Therefore, easy integration with other systems is facilitated.
3. Based on the published events, a separate storage can be built for the read side, optimizing data retrieval speed.

Event Bus – Kafka – MySQL Binlog.

The versioned events of objects, if sent, make it easy for the system to integrate with many other systems. The Event Bus is the logic term for a transmission line containing all versioned events in the form of a stream. Consumers can simply subscribe to the event bus to receive events for processing.

Events are stored in a MySQL database. All changes in the database are logged by MySQL into a log file called binlog. By extracting MySQL’s binlog, the system can capture every event generated for transmission.

Kafka is a log-based message broker that allows storing and sending events in the order they are sent. Therefore, it is the best choice for storing the event bus layer.

Overall Architecture.

At this point, the system’s picture has become clear. Each piece from model, business logic to integration has been fully assembled. It can be summarized as follows: the system will build business logic around the product model; structured according to the CQRS – ES model; using MySql Binlog; Kafka to publish change events reliably; based on the event bus, high-speed retrieval structures like Elastic, MongoDB will be built, as well as integration with other systems.

Conclusion.

Designing a product catalog management system in ecommerce requires achieving multiple objectives simultaneously, with a consistent and coherent vision. From the application level to the system level, there must be seamless integration. All aim at achieving two important objectives:

1. Handling complex business processes.
2. Ensuring system performance and stability.

The Domain Driven Design analysis and design method, the CQRS – ES model, the stable replication mechanism of MySql, Kafka’s message ordering guarantee, as well as optimized databases for reads such as Elastic, MongoDB… are the key factors ensuring the quality of the system.