1. Introduction In large-scale Internet applications, as the amount of data continues to grow, the single-database, single-table architecture cannot meet the needs of high concurrency and large data storage. Sub-database and sub-table is a common database architecture optimization solution, which can improve database throughput, reduce the amount of data in a single table, and improve query efficiency. However, sub-database and sub-table also bring many complex problems, such as primary key generation, paging query, distributed transaction, cross-table query, high availability, etc. This article will explain in detail the core concepts, key technologies and implementation methods of sub-database and sub-table…
MySQL data migration knowledge map 1 Core challenges and technology selection 1.1 Three major challenges of data migration Business continuity requirements: Financial-level business requires 99.99% availability throughout the year Data consistency guarantee: Amount data must have zero error (refer to the "Payment System Data Security Specification" of the central bank) Incremental synchronization complexity: Synchronization delay control in the scenario of 10,000 TPS per second 1.2 Tool comparison matrix Tool name Backup type Lock mechanism Recovery speed Applicable scenarios Official document link mysq…
Introduction In database systems, transaction durability is the most critical part of the ACID feature. It promises that once a transaction is successfully committed, the modified data will take effect permanently, and the data will not be lost even if the system crashes or hardware fails. However, the realization of this promise is not as simple as it seems. Will the data really not be lost after the transaction is committed? This depends on the database's internal log mechanism, persistence strategy, distributed architecture design, and hardware redundancy capabilities. This article will start from a single...