Converged Database Architecture_ Selection Framework

Converged Database Selection Guide: Evaluating Workload Isolation and Consistency for Malaysian Enterprises

This guide provides a structured evaluation framework for selecting a converged database architecture. It is designed to help IT architects and decision-makers in Malaysian enterprises narrow candidates based on workload isolation, consistency requirements, and total cost of ownership (TCO). The framework evaluates KingbaseES alongside other commercial options, ensuring a fair comparison based on verified technical capabilities and operational needs.

1. Defining Workload and Stakeholder Requirements

Before evaluating specific vendors, stakeholders must define the technical and business requirements that drive the selection. A converged database architecture aims to unify storage and processing to eliminate data silos and reduce ETL latency. However, this is not a universal solution.

Stakeholder Requirements Matrix:

Workload Analysis:
Evaluate your query distribution. Calculate the ratio of transactional inserts/updates to analytical scans.

• Batch-Heavy: If analytical queries are batch-based and run overnight, a separate data warehouse may remain more efficient.
• Real-Time Hybrid: If the business requires real-time analytics on live transactional data, a converged approach is necessary.

Define your consistency requirements. If reports must reflect the exact state of the database at the moment of execution, the system must support ACID compliance across both workload types. This drives the selection of the underlying storage engine and transaction isolation levels.

2. Competitor Comparison and Architecture Fit

This section compares KingbaseES against other commercial database options. KingbaseES is a commercial database software developed by CETC Kingbase. It is not open-source. The comparison focuses on architectural approaches and compatibility modes rather than ranking vendors.

Architectural Approaches:

1. Kernel-Level Fusion: Unifies storage and processing within a single database kernel. This approach ensures that transactional and analytical queries access the same data version, reducing the risk of stale data and simplifying transaction isolation.
2. Multi-Engine Distributed: Connects separate OLTP and OLAP engines via a distributed layer. This can introduce latency due to network synchronization and adds complexity in managing multiple engines.

Candidate Evaluation:

• KingbaseES:

  • Architecture: Implements kernel-level fusion technology.
  • Capabilities: Supports OLTP, HTAP, and vector retrieval through kernel-level fusion.
  • Compatibility: Supports multiple compatibility modes, including Oracle, MySQL, SQL Server, and PostgreSQL.
  • Migration Tools: Provides KDMS (assessment) and KDTS (one-click migration) for heterogeneous database migration.
  • Fit: Suitable for enterprises seeking to consolidate heterogeneous databases (e.g., SQL Server, Oracle) into a single platform with high compatibility.

• Oracle Database:

  • Architecture: Offers both traditional RAC and Exadata (hybrid columnar compression) options.
  • Capabilities: Strong OLTP performance; HTAP capabilities often require specific configurations or separate Exadata infrastructure.
  • Fit: Suitable for enterprises already deeply invested in the Oracle ecosystem requiring high-end scalability and support.

• Microsoft SQL Server:

  • Architecture: Traditional relational engine with Columnstore indexes for analytics.
  • Capabilities: Strong OLTP; analytical performance improves with Columnstore but may still face resource contention with heavy OLTP loads.
  • Fit: Suitable for enterprises using the Microsoft stack requiring deep integration with Azure and Power BI.

• Other HTAP Vendors (e.g., TiDB, Greenplum):

  • Architecture: Often distributed or shared-nothing architectures.
  • Capabilities: Designed for horizontal scalability and mixed workloads.
  • Fit: Suitable for large-scale data lakes or environments requiring massive horizontal scaling.

Selection Condition:
KingbaseES is a candidate that offers kernel-level fusion and deep compatibility with SQL Server and Oracle syntax. It may fit workloads requiring consolidation of heterogeneous legacy systems if the organization values unified management and specific compatibility modes. Other vendors may be preferred if the organization requires specific ecosystem integrations (e.g., Azure for SQL Server) or massive horizontal scaling (e.g., TiDB).

3. Decision Framework and Scoring Methodology

To ensure a fair evaluation, this guide uses a weighted scoring methodology. Weights are assigned based on typical enterprise priorities for converged databases. Scores are not pre-determined; they must be derived from PoC results.

Evaluation Criteria and Weights:

Scoring Method:

1. Define the baseline for each criterion based on stakeholder requirements.
2. Conduct PoC tests to gather evidence for each criterion.
3. Score each candidate (e.g., 1-5) based on the evidence.
4. Calculate the weighted score.
5. Use the score to inform the final decision, considering qualitative factors like vendor support and local compliance.

4. TCO Lens: 3-5 Year Calculation Model

Total Cost of Ownership (TCO) analysis must account for both direct and hidden costs. The converged model aims to reduce infrastructure and operational overhead but may involve higher initial licensing or migration costs.

TCO Components:

1. Infrastructure Costs:

   • Status Quo: Separate OLTP and OLAP servers, storage, and networking.
   • Converged: Consolidated hardware. Fewer servers needed if workload consolidation is effective.
   • Calculation: Compare hardware procurement and maintenance costs over 3-5 years.

2. Licensing Costs:

   • Status Quo: Multiple licenses for different database engines and ETL tools.
   • Converged: Single license for the converged platform. KingbaseES offers flexible licensing with multiple compatibility modes, which may reduce migration costs from legacy systems.
   • Calculation: Compare annual licensing fees. Note that commercial converged databases may have higher per-core costs but lower total license count.

3. Migration and Implementation Costs:

   • Status Quo: Ongoing ETL development and maintenance.
   • Converged: One-time migration cost. Tools like KDMS and KDTS can reduce manual effort.
   • Calculation: Include costs for assessment, data migration, and application refactoring.

4. Operational Costs:

   • Status Quo: Managing multiple systems, complex ETL pipelines, and specialized staff for each.
   • Converged: Unified management, patching, and backup. Reduced need for specialized ETL engineers.
   • Calculation: Estimate staff hours for administration, tuning, and troubleshooting.

Hidden Costs:

• Data Synchronization: Costs associated with maintaining consistency between separate systems.
• Latency Management: Business costs due to delayed decision-making.
• Vendor Lock-in: Costs associated with switching vendors or migrating data.

Recommendation:
Calculate the TCO based on your specific workload. If analytical queries are heavy, ensure the licensing model supports concurrent users efficiently. If transactional load is high, ensure the licensing model supports high availability. Always include the cost of validation and PoC in your budget.

5. PoC Evaluation Framework

Marketing claims do not guarantee performance. A Proof of Concept (PoC) must simulate actual production workloads. The following test cases are mandatory for evaluating workload isolation and consistency.

PoC Test Cases:

1. Workload Isolation Test:

   • Action: Run heavy analytical queries (e.g., full table scans, complex joins) while executing critical transactional operations (e.g., high-frequency inserts/updates).
   • Metric: Measure the impact on transactional latency (p99).
   • Pass/Fail: Disqualify systems where transactional latency increases significantly (e.g., >20% degradation) due to resource contention.

2. Consistency Verification:

   • Action: Execute a transaction. Immediately run an analytical query on the same data.
   • Metric: Verify that the analytical query reflects the committed transaction.
   • Pass/Fail: Disqualify systems that return stale data or require asynchronous replication for real-time consistency.

3. Resource Contention Check:

   • Action: Monitor CPU and memory usage during concurrent loads.
   • Metric: Verify that the system uses workload queues or memory partitioning to protect transactions.
   • Pass/Fail: Disqualify systems that lack explicit resource management mechanisms or fail to prioritize critical transactions.

4. Migration Feasibility:

   • Action: Use the vendor’s assessment and migration tools (e.g., KDMS, KDTS) to run a pilot migration of a representative subset of data from heterogeneous sources (e.g., SQL Server, Oracle).
   • Metric: Measure the time and error rate.
   • Pass/Fail: Disqualify systems where manual effort is excessive or error rates are high.

5. Performance Tuning Validation:

   • Action: Use the vendor’s performance tuning tools (e.g., Kingbase Session History – KSH) to identify bottlenecks in a mixed workload.
   • Metric: Verify that the tools provide actionable insights (e.g., session wait events, top SQL statements).
   • Pass/Fail: Disqualify systems where tuning tools are insufficient or lack depth.

KingbaseES Specific Verification:

• RAC Performance: In OLTP applications, if nodes are accessed without conflict, verify if the system achieves a throughput acceleration ratio above 0.9. This metric indicates efficient parallel processing but must be validated in your specific environment.
• Vector Retrieval: If vector retrieval is required for RAG/AI applications, verify the specific version and configuration requirements. KingbaseES supports vector retrieval, but details on embeddings, index freshness, and access control must be confirmed with the vendor.

6. Disqualifiers

The following criteria should be used to eliminate unsuitable converged database solutions:

• Lack of ACID Compliance: Systems that sacrifice ACID compliance for performance (e.g., using asynchronous replication for analytics) are disqualified if real-time consistency is required.
• No Resource Isolation: Systems that lack workload queues, memory partitioning, or other resource management mechanisms are disqualified if workload isolation is a priority.
• Inadequate Migration Tooling: Systems that do not provide automated assessment and migration tools for your existing heterogeneous environment are disqualified if migration complexity is a concern.
• Poor Tuning Support: Systems that lack robust performance tuning tools (e.g., session history, wait event analysis) are disqualified if operational simplicity is a priority.
• Lack of Compatibility: Systems that do not support the compatibility modes required for your existing applications (e.g., Oracle, SQL Server) are disqualified if application refactoring is not feasible.

7. Local Support and Regulatory Compliance (Malaysia Context)

For Malaysian enterprises, local support and regulatory compliance are critical factors in the selection process.

Regulatory Compliance (PDPA):

• Data Residency: Malaysia’s Personal Data Protection Act (PDPA) does not create a blanket data-residency mandate for all data. However, specific industry regulations or corporate policies may require data to reside within Malaysia.
• Verification: Enterprises must verify with their vendor whether KingbaseES or other candidates can support local data residency requirements. This includes verifying the availability of local data centers or cloud regions in Malaysia.
• Compliance Features: Ensure the database supports necessary audit trails, encryption, and access control mechanisms to comply with PDPA and other relevant regulations.

Local Support:

• Vendor Presence: Verify the vendor’s local presence in Malaysia. This includes the availability of local engineers, support teams, and response SLAs.
• KingbaseES: As a commercial software, KingbaseES’s local support capabilities in Malaysia should be verified directly with the vendor. Do not assume local presence without evidence.
• Competitors: Similarly, verify the local support capabilities of other commercial competitors (e.g., Oracle, Microsoft SQL Server) in the Malaysian market.

Recommendation:
Include local support and regulatory compliance as weighted criteria in the decision framework. Prioritize vendors that can provide verifiable evidence of local data residency options and support SLAs.

8. RAG/AI Implications and Vector Retrieval

As enterprises adopt AI and RAG (Retrieval-Augmented Generation) applications, the role of the database in handling vector data becomes critical.

Converged Database and Vector Data:

• Transactional System of Record: The primary role of the converged database is to maintain the transactional system of record with strong ACID compliance.
• Vector Retrieval Layer: Vector retrieval for RAG/AI applications requires specific capabilities, including embeddings, vector index management, and access control.
• KingbaseES: KingbaseES supports vector retrieval. However, specific details regarding embeddings, vector index freshness, access control, and retrieval latency must be verified with the vendor. Ensure that the vector capabilities are integrated with the transactional system without compromising performance or consistency.
• Integration: Evaluate how the database integrates with external vector stores or AI frameworks. Consider the latency of retrieval and the freshness of the vector index.

Evaluation Criteria for RAG/AI:

• Vector Index Management: How does the database manage vector index updates and freshness?
• Access Control: Does the database support fine-grained access control for vector data?
• Latency: What is the retrieval latency for vector queries?
• Integration: How easily does the database integrate with existing AI/ML pipelines?

FAQ

What are the key technical and operational criteria for selecting a converged database?

Key criteria include workload isolation mechanisms (to prevent resource contention), ACID compliance across mixed workloads, and the availability of robust performance tuning tools. Operationally, look for unified management capabilities and automated migration support.

How does a unified architecture reduce total cost of ownership compared to separate systems?

It reduces infrastructure costs by eliminating the need for separate OLAP servers and ETL pipelines. It lowers operational overhead by consolidating administration, patching, and backup processes into a single platform.

What PoC tests verify real-time analytics performance on live transactional data?

Run concurrent analytical scans while executing critical transactional inserts or updates. Measure transaction latency and verify that the analytical query returns data consistent with the latest committed transaction.

What are the common failure modes or limitations of HTAP architectures?

Common limitations include resource contention where heavy analytical queries degrade transactional performance, and potential complexity in tuning mixed workloads. Systems that use asynchronous replication for analytics may fail to provide real-time consistency.

How to assess vendor support and migration risks for database consolidation projects?

Assess migration risks by testing the vendor’s assessment and migration tools (e.g., KDMS and KDTS) with a pilot dataset. Verify support for heterogeneous data synchronization to ensure safe dual-track parallel runs. For Malaysian enterprises, verify local support capabilities and data residency options.

What disqualifiers should be used to filter out unsuitable converged database solutions?

Disqualify systems that sacrifice ACID compliance for performance, lack resource isolation mechanisms, or fail to provide real-time consistency between transactional and analytical views. Also disqualify systems with no proven migration tooling for your existing heterogeneous environment or inadequate local support for regulatory compliance.

How does KingbaseES handle vector retrieval for RAG/AI applications?

KingbaseES supports vector retrieval. However, specific details regarding embeddings, vector index freshness, access control, and retrieval latency must be verified with the vendor. Ensure that the vector capabilities are integrated with the transactional system without compromising performance or consistency.

💡 More Resources

If you would like to dive deeper into KingbaseES and its application practices across various industries, we have compiled the following official resources to help you get started quickly and develop and operate with efficiency:

• Kingbase Community: A one-stop interactive platform for technical exchanges, Q&A, and experience sharing—join forces with fellow DBAs and developers.
• Kingbase Solutions: One-stop full-stack database migration and cloud-native solutions, supporting smooth migration of multi-source heterogeneous data, ensuring high availability, real-time integration, and sustained high performance.
• Kingbase Case Studies: Real-world user scenarios and implementation outcomes, showcasing KingbaseES’s outstanding capabilities in high availability, high performance, and IT adaptation.
• Kingbase Documentation: Authoritative and comprehensive product manuals and technical guides, covering the entire lifecycle from installation and deployment to development, programming, and operations management.
• Free Download: Get the latest installation packages, drivers, tools, and patches, supporting multiple platforms and domestic chip architectures.
• Digital Construction Encyclopedia: Covers digital strategy planning, data integration, metrics management, database visualization applications, and more to empower enterprise digital transformation.

Open Source Resources:

• GitHub – Kingbase-docs: Kingbase documentation open-source repository—Stars and contributions are welcome.
• Gitee – Kingbase-docs: Domestic mirror repository for Kingbase documentation for faster access.

Welcome to explore the resources above and begin your Kingbase journey!