vitess v22.0.0-rc3 版本更新介绍
发布日期: 2025-04-24
版本号: v22.0.0-rc3
Vitess v22.0.0版本发布,包含474个合并的PR。主要变更涵盖多个方面:弃用并移除了部分旧指标(如QueriesProcessed等)和CLI标志(如twopc_enable),新增了查询执行指标(如QueryExecutions、QueryRoutes)和表统计指标(如TableRows、IndexCardinality)。配置文件方面,VTOrc改用viper动态配置,VTGate调整了Viper配置键名。VTOrc新增磁盘卡顿恢复功能,并支持通过键范围监视分片。默认MySQL版本升至8.0.40,Docker镜像改用Debian Bookworm。新功能包括更高效的JSON复制、LAST_INSERT_ID(x)直接在VTGate处理、连接池最大空闲连接数配置、错误查询日志过滤、VTGate的多查询RPC支持以及非分片键空间的CREATE PROCEDURE支持。预处理语句优化引入延迟优化机制。新增GetTransactionInfo RPC用于调试分布式事务。紧急主从切换现在倾向于不提升正在执行非内置备份的副本,VTTablet会监控半同步状态并处理卡顿。致命事务错误被包装为VT15001错误,相关新查询会返回VT09032错误。次要变更包括拓扑读取并发行为调整、VTTablet的CLI标志更新和ACL重载逻辑改进,以及VTAdmin Web升级至Node v22.13.1。感谢所有贡献者。
更新内容 (中文)
Vitess v22.0.0 发布说明
概要
目录
- 主要变更
- 次要变更
主要变更
已弃用
指标
| 组件 | 指标名称 | 弃用 PR |
|---|---|---|
vtgate |
QueriesProcessed |
#17727 |
vtgate |
QueriesRouted |
#17727 |
vtgate |
QueriesProcessedByTable |
#17727 |
vtgate |
QueriesRoutedByTable |
#17727 |
CLI 标志
| 组件 | 标志名称 | 备注 | 弃用 PR |
|---|---|---|---|
vttablet |
twopc_enable |
TwoPC 提交的用法将由 VTGate 通过标志或会话变量设置的 transaction_mode 决定。 |
#17279 |
vtgate |
grpc-send-session-in-streaming |
会话始终作为 StreamExecute RPC 响应的一部分发送。 | #17907 |
已删除
指标
| 组件 | 指标名称 | 弃用版本 | 删除 PR |
|---|---|---|---|
vttablet |
QueryCacheLength |
v21.0.0 |
#16289 |
vttablet |
QueryCacheSize |
v21.0.0 |
#16289 |
vttablet |
QueryCacheCapacity |
v21.0.0 |
#16289 |
vttablet |
QueryCacheEvictions |
v21.0.0 |
#16289 |
vttablet |
QueryCacheHits |
v21.0.0 |
#16289 |
vttablet |
QueryCacheMisses |
v21.0.0 |
#16289 |
CLI 标志
| 组件 | 标志名称 | 弃用版本 | 删除 PR |
|---|---|---|---|
vttablet |
queryserver-enable-settings-pool |
v21.0.0 |
#16280 |
vttablet |
remove-sharded-auto-increment |
v21.0.0 |
#16860 |
vttablet |
disable_active_reparents |
v20.0.0 |
#14871 |
vtgate, vtcombo, vtctld |
healthcheck-dial-concurrency |
v21.0.0 |
#16378 |
gh-ost 和 pt-osc 在线 DDL 策略
Vitess 不再支持 gh-ost 和 pt-osc (pt-online-schema-change) 在线 DDL 策略。vitess 策略是进行大规模模式变更的推荐方式。mysql 和 direct 策略继续被支持。
以下 vttablet 标志已被移除:
--gh-ost-path--pt-osc-path
尝试将 gh-ost 或 pt-osc 用作 --ddl-strategy 将会产生错误:
$ vtctldclient ApplySchema --ddl-strategy="gh-ost" ...
$ vtctldclient ApplySchema --ddl-strategy="pt-osc" ...
新增指标
VTGate
| 名称 | 维度 | 描述 | PR |
|---|---|---|---|
QueryExecutions |
Query, Plan, Tablet |
已执行的查询数量。 | #17727 |
QueryRoutes |
Query, Plan, Tablet |
查询执行在其上的 vttablet 数量。 | #17727 |
QueryExecutionsByTable |
Query, Table |
VTGate 执行的查询,按表记录计数。 | #17727 |
VStreamsCount |
Keyspace, ShardName, TabletType |
活跃 vstream 的数量。 | #17858 |
VStreamsEventsStreamed |
Keyspace, ShardName, TabletType |
所有 vstream 发送的事件数量。 | #17858 |
VStreamsEndedWithErrors |
Keyspace, ShardName, TabletType |
因错误结束的 vstream 数量。 | #17858 |
CommitModeTimings |
Mode |
提交计时指标(单次、多次、TwoPC)。 | #16939 |
CommitUnresolved |
N/A | Prepare 后 2PC 事务失败的计数器。 | #16939 |
#17727 中完成的工作引入了新的查询指标。通过这项工作,我们弃用了几个 vtgate 指标,请参阅已弃用指标部分。以下是新指标的计算示例:
查询: select t1.a, t2.b from t1 join t2 on t1.id = t2.id
分片: 2
分片键: 两个表的 id
发布的指标:
1. QueryExecutions – {select, scatter, primary}, 1
2. QueryRoutes – {select, scatter, primary}, 2
3. QueryExecutionsByTable – {select, t1}, 1 以及 {select, t2}, 1
VTTablet
| 名称 | 维度 | 描述 | PR |
|---|---|---|---|
TableRows |
Table |
表中估算的行数。 | #17570 |
TableClusteredIndexSize |
Table |
聚簇索引的字节大小(即行数据)。 | #17570 |
IndexCardinality |
Table, Index |
索引中唯一值的估算数量 | #17570 |
IndexBytes |
Table, Index |
索引的字节大小。 | #17570 |
UnresolvedTransaction |
ManagerType |
当前未解决的事务数量。 | #16939 |
CommitPreparedFail |
FailureType |
Prepare 后提交失败的事务。 | #16939 |
RedoPreparedFail |
FailureType |
重新 Prepare 失败的事务。 | #16939 |
配置文件变更
VTOrc
VTOrc 现在支持使用 viper 进行动态配置。旧的 --config 参数已被移除。请改用 --config-file 参数。配置可以是 json、yaml 或 viper 支持的任何其他格式。
以下字段可以动态更改 -
instance-poll-timeprevent-cross-cell-failoversnapshot-topology-intervalreasonable-replication-lagaudit-to-backendaudit-to-syslogaudit-purge-durationwait-replicas-timeouttolerable-replication-lagtopo-information-refresh-durationrecovery-poll-durationallow-emergency-reparentchange-tablets-with-errant-gtid-to-drained
要升级到新版本的配置文件,请先在升级前切换为在当前部署中使用标志。然后,升级后可以切换为使用配置文件。
VTGate
以下标志的 Viper 配置键已更改为与其标志名称匹配。以前它们有一个 discovery 前缀而不是名称的一部分。
| 标志名称 | 旧配置键 | 新配置键 |
|---|---|---|
discovery_low_replication_lag |
discovery.low_replication_lag |
discovery_low_replication_lag |
discovery_high_replication_lag_minimum_serving |
discovery.high_replication_lag_minimum_serving |
discovery_high_replication_lag_minimum_serving |
discovery_min_number_serving_vttablets |
discovery.min_number_serving_vttablets |
discovery_min_number_serving_vttablets |
discovery_legacy_replication_lag_algorithm |
discovery.legacy_replication_lag_algorithm |
discovery_legacy_replication_lag_algorithm |
要升级到新版本的配置键,请先在升级前切换为在当前部署中使用标志。然后,升级后可以切换为使用新配置键。
VTOrc
磁盘停滞恢复
VTOrc 现在可以识别并从磁盘停滞错误中恢复。
VTTablet 测试磁盘是否可写,并在 FullStatus RPC 响应中将此信息发送给 VTOrc。
如果主 tablet 上的磁盘不可写,VTOrc 将尝试通过提升新的主 tablet 来恢复集群。
这在磁盘停滞且主 vttablet 因此无法接受写入的场景中非常有用。
要启用此功能,必须在 VTOrc 上指定 --enable-primary-disk-stalled-recovery 标志,并且必须在 vttablets 上指定 --disk-write-dir 标志。
--disk-write-interval 和 --disk-write-timeout 标志可用于分别配置轮询间隔和超时。
--clusters_to_watch 中的 KeyRanges
VTOrc 现在支持在 --clusters_to_watch 标志中指定 keyranges。这意味着当你重新分片一个 keyspace 时,无需使用不同的标志值重启 VTOrc 实例。
例如,如果 VTOrc 配置为监视 ks/-80,那么它将监视属于 keyrange -80 的所有分片。
如果执行了重新分片,将 -80 拆分为新的分片 -40 和 40-80,则 VTOrc 实例将自动开始监视新分片,而无需重启。
在之前的逻辑中,为该标志指定 ks/-80 意味着 VTOrc 只会监视 1 个(或没有)分片。
在新系统中,由于我们将 -80 解释为一个 key range,因此它可以监视多个分片,如示例所示。
用户可以继续指定精确的 keyranges。新功能向后兼容。
新增默认版本
MySQL 8.0.40
我们的 vitess/lite:latest 镜像使用的默认 MySQL 主版本从 8.0.30 升级到了 8.0.40。
此变更合并于 #17552。
VTGate 默认宣传的 MySQL 版本也从 8.0.30 变为 8.0.40。如果你运行的版本不是这个,可以设置 mysql_server_version 标志来宣传所需的版本。
⚠️ 使用 vitess-operator 升级到此版本:
如果你使用的是
vitess-operator,考虑到我们将 MySQL 版本从8.0.30提升到了8.0.40,你将需要手动升级:
- 在你的 YAML 文件中,在 extra cnf 中添加
innodb_fast_shutdown=0。- 应用此文件。
- 等待所有 pod 健康。
- 然后更改你的 YAML 文件以使用新的 Docker 镜像 (
vitess/lite:v22.0.0)。- 从你的 YAML 文件中的 extra cnf 中移除
innodb_fast_shutdown=0。- 应用此文件。
这是在
8.0.x系列中最后一次需要这样做,因为从 MySQL8.0.35开始,可以在8.0.x版本之间升级和降级,而无需运行innodb_fast_shutdown=0。
基于 Debian Bookworm 的 Docker vitess/lite 镜像
docker 构建系统现在使用 Debian Bookworm 而不是 Debian Bullseye 来构建 vitess/lite 镜像。此变更合并于 #17552。
新增支持
更高效的 JSON 复制
在 #7345 中,我们添加了对 --binlog-row-value-options=PARTIAL_JSON 的支持。你可以在这里阅读更多关于 [MySQL 8.0 中添加的此功能] 的信息。
如果你使用 MySQL 8.0 或更高版本并使用 JSON 列,你现在可以在你的 Vitess 集群中启用此 MySQL 功能,以减少二进制日志所需的磁盘空间,并降低 mysqld(标准分片内 MySQL 复制)和 vttablet (VReplication) 中的 CPU 和内存使用率,而不会丢失任何功能或特性。
LAST_INSERT_ID(x)
在 #17408 和 #17409 中,我们添加了在 Vitess 中直接在 vtgate 使用 LAST_INSERT_ID(x) 的能力。此改进允许某些查询(如 SELECT last_insert_id(123); 或 SELECT last_insert_id(count(*)) ...)无需依赖 MySQL 获取最终值。
限制:
- 在有序查询(例如
SELECT last_insert_id(col) FROM table ORDER BY foo)中使用LAST_INSERT_ID(x)时,MySQL 会根据最后一行返回值设置会话的 last-insert-id 值。Vitess 不保证相同的行为。
连接池最大空闲连接数
在 #17443 中,我们为连接池引入了一个新的可配置参数 max-idle-count。这允许你指定每个连接池中保留的最大空闲连接数,以优化性能和资源效率。
你可以通过以下标志控制查询服务器的查询池、流池和事务池的空闲连接保留: • –queryserver-config-query-pool-max-idle-count: 定义查询池中保留的最大空闲连接数。 • –queryserver-config-stream-pool-max-idle-count: 定义流池中保留的最大空闲连接数。 • –queryserver-config-txpool-max-idle-count: 定义事务池中保留的最大空闲连接数。
此功能确保在流量高峰期间有空闲连接可用于更快的响应,同时在低流量期间通过限制保留的空闲连接数来最小化开销。它有助于在性能、效率和成本之间取得平衡。
按错误过滤查询日志
querylog-mode 设置可配置为 error,仅记录导致错误的查询。此选项在 VTGate 和 VTTablet 中均受支持。
vtgate 中的 MultiQuery RPC
已向 VTGate 添加新的 RPC,允许用户在单个 sql 字符串中传递多个查询。其行为与 MySQL 相同:RPC 将按查询顺序返回多个结果集,直到遇到错误。新的 RPC 是 ExecuteMulti 和 StreamExecuteMulti。
还添加了一个新标志 --mysql-server-multi-query-protocol,使服务器使用此新实现。此标志默认设置为 false,因此默认使用旧实现。新实现更高效,可在单个 RPC 调用中执行多个查询时提供更好的性能。
非分片 CREATE PROCEDURE 支持
到目前为止,Vitess 不允许用户通过 VTGate 创建过程。它们必须通过直接对底层 MySQL 运行 DDL 来创建。在这个版本中,我们已开始为非分片 keyspace 添加通过 VTGate 运行 CREATE PROCEDURE 语句的支持。解析器中目前并不支持过程的所有构造,因此仍存在一些限制,这些将在未来的版本中得到解决。
优化
预处理语句
预处理语句现在受益于延迟优化,支持参数感知的查询计划。 首先,在 prepare 时创建一个基线计划,在首次执行时,生成一个更高效的参数优化计划。 后续执行会根据输入值动态切换这些计划,在确保正确性的同时提高查询性能。
新的 VtctldServer RPC
GetTransactionInfo RPC 已添加到 VtctldServer 和 TabletManagerClient 接口。这些 RPC 可用于读取未解决的分布式事务的状态。这在调试出了什么问题以及如何修复问题时非常有用。
优先提升未进行备份的副本
紧急主从切换现在优先不提升当前正在使用非 builtin 备份引擎进行备份的副本。请注意,如果只有一个合适的副本可以提升,并且它正在进行备份,它仍然会被提升。
对于计划主从切换,正在进行非 builtin 备份引擎备份的主机会从有效候选列表中过滤掉。这意味着它们永远不会被提升——即使没有其他候选者。
请注意,对于 builtin 备份的行为保持不变:当前正在进行 builtin 备份的副本永远不会被提升,无论是计划还是紧急主从切换。
vttablet 中的半同步监控
主 VTTablet 现在监控其底层 MySQL 实例的半同步状态。 我们观察到,短暂的网络中断可能导致主节点无限期地等待半同步 ACK。 在罕见的情况下,这也会阻止主从切换操作并导致主节点无响应。 更多信息可以在问题 #17709 和 #17749 中找到。
为了解决这个问题,主 VTTablet 持续监控其半同步状态。如果主 MySQL 卡住等待半同步 ACK,监控器会生成合成写入以解除阻塞。如果此操作失败,VTOrc 将收到问题通知,并将启动紧急主从切换操作。
监控间隔可以使用 --semi-sync-monitor-interval 标志进行调整,默认为 10 秒。
封装的致命事务错误
当查询在事务中运行时失败,由于事务不再有效(例如 PRS、滚动更新、主节点宕机等),原始错误现在被封装在 VT15001 错误中。
当查询产生 VT15001 错误时,VTGate 将尝试回滚并清除事务。
同一连接上的任何新查询都将失败并返回 VT09032 错误,直到收到 ROLLBACK 以确认事务已被 VTGate 自动回滚和清除。
VT09032 返回给客户端,以避免应用程序盲目地向 VTGate 发送查询,误以为它们仍在事务中。
此变更由 #17669 引入。
次要变更
--topo_read_concurrency 行为变更
--topo_read_concurrency 标志已添加到所有访问拓扑的组件中,并且提供的限制现在分别应用于每个全局或本地单元格 (默认 32)。
所有拓扑读取调用 (Get, GetVersion, List 和 ListDir) 现在都遵守每个单元格的限制。在此版本之前,对所有单元格调用应用了单个限制,并且许多拓扑调用并未遵守该限制。
VTTablet
CLI 标志
-
twopc_abandon_age标志现在支持 time.Duration 格式的值(例如,1s, 2m, 1h)。 虽然为了向后兼容,该标志将继续接受浮点值(解释为秒), 浮点输入已弃用,并将在未来版本中移除。 -
--consolidator-query-waiter-cap标志设置允许在合并器上等待的最大客户端数量。默认值设置为 0 表示无限制等待。用户可以根据 VTTablet 的性能调整此值,以避免过度内存使用和 OOMKilled 风险,特别是在 Kubernetes 部署中。
ACL 执行与重新加载
当 tablet 使用 --enforce-tableacl-config 启动时,如果文件内容无效,它将退出并报错。在 #17485 中进行更改后,tablet 在收到 SIGHUP 后重新加载文件内容时将不再退出。当重新加载时文件内容无效时,tablet 现在将记录错误,而活动的内存中 ACL 将保持有效。
VTAdmin
更新至 node v22.13.1 (LTS)
构建 vtadmin-web 现在需要 node >= v22.13.0 (LTS)。从 v20 到 v22 的破坏性更改可以在 https://nodejs.org/en/blog/release/v22.13.0 找到 – 没有已知适用于 VTAdmin 的问题。
node v20.12.2 版本的详细信息可以在 https://nodejs.org/en/blog/release/v22.13.1 找到。
此版本的完整变更日志可以在这里找到。
该版本包含 474 个合并的 Pull Request。
感谢我们所有的贡献者:@GrahamCampbell, @GuptaManan100, @L3o-pold, @akagami-harsh, @anirbanmu, @app/dependabot, @app/vitess-bot, @arthmis, @arthurschreiber, @beingnoble03, @c-r-dev, @corbantek, @dbussink, @deepthi, @derekperkins, @ejortegau, @frouioui, @garfthoffman, @gmpify, @gopoto, @harshit-gangal, @huochexizhan, @jeefy, @jwangace, @kbutz, @lmorduch, @mattlord, @mattrobenolt, @maxenglander, @mcrauwel, @mounicasruthi, @niladrix719, @notfelineit, @rafer, @rohit-nayak-ps, @rvrangel, @shailpujan88, @shanth96, @shlomi-noach, @siadat, @systay, @timvaillancourt, @twthorn, @vitess-bot, @vmg, @wiebeytec, @wukuai
更新内容 (原始)
Release of Vitess v22.0.0
Summary
Table of Contents
- Major Changes
- Minor Changes
Major Changes
Deprecations
Metrics
| Component | Metric Name | Deprecation PR |
|---|---|---|
vtgate |
QueriesProcessed |
#17727 |
vtgate |
QueriesRouted |
#17727 |
vtgate |
QueriesProcessedByTable |
#17727 |
vtgate |
QueriesRoutedByTable |
#17727 |
CLI Flags
| Component | Flag Name | Notes | Deprecation PR |
|---|---|---|---|
vttablet |
twopc_enable |
Usage of TwoPC commit will be determined by the transaction_mode set on VTGate via flag or session variable. |
#17279 |
vtgate |
grpc-send-session-in-streaming |
Session will always be sent as part of the response from the StreamExecute RPC. | #17907 |
Deletions
Metrics
| Component | Metric Name | Was Deprecated In | Deletion PR |
|---|---|---|---|
vttablet |
QueryCacheLength |
v21.0.0 |
#16289 |
vttablet |
QueryCacheSize |
v21.0.0 |
#16289 |
vttablet |
QueryCacheCapacity |
v21.0.0 |
#16289 |
vttablet |
QueryCacheEvictions |
v21.0.0 |
#16289 |
vttablet |
QueryCacheHits |
v21.0.0 |
#16289 |
vttablet |
QueryCacheMisses |
v21.0.0 |
#16289 |
CLI Flags
| Component | Flag Name | Was Deprecated In | Deletion PR |
|---|---|---|---|
vttablet |
queryserver-enable-settings-pool |
v21.0.0 |
#16280 |
vttablet |
remove-sharded-auto-increment |
v21.0.0 |
#16860 |
vttablet |
disable_active_reparents |
v20.0.0 |
#14871 |
vtgate, vtcombo, vtctld |
healthcheck-dial-concurrency |
v21.0.0 |
#16378 |
gh-ost and pt-osc Online DDL strategies
Vitess no longer recognizes the gh-ost and pt-osc (pt-online-schema-change) Online DDL strategies. The vitess strategy is the recommended way to make schema changes at scale. mysql and direct strategies continue to be supported.
These vttablet flags have been removed:
--gh-ost-path--pt-osc-path
Attempting to use gh-ost or pt-osc as --ddl-strategy will yield an error:
$ vtctldclient ApplySchema --ddl-strategy="gh-ost" ...
$ vtctldclient ApplySchema --ddl-strategy="pt-osc" ...
New Metrics
VTGate
| Name | Dimensions | Description | PR |
|---|---|---|---|
QueryExecutions |
Query, Plan, Tablet |
Number of queries executed. | #17727 |
QueryRoutes |
Query, Plan, Tablet |
Number of vttablets the query was executed on. | #17727 |
QueryExecutionsByTable |
Query, Table |
Queries executed by vtgate, with counts recorded per table. | #17727 |
VStreamsCount |
Keyspace, ShardName, TabletType |
Number of active vstreams. | #17858 |
VStreamsEventsStreamed |
Keyspace, ShardName, TabletType |
Number of events sent across all vstreams. | #17858 |
VStreamsEndedWithErrors |
Keyspace, ShardName, TabletType |
Number of vstreams that ended with errors. | #17858 |
CommitModeTimings |
Mode |
Timing metrics for commit (Single, Multi, TwoPC). | #16939 |
CommitUnresolved |
N/A | Counter for 2PC transaction failures after Prepare. | #16939 |
The work done in #17727 introduces new metrics for queries. Via this work we have deprecated several vtgate metrics, please see the Deprecated Metrics section. Here is an example of how the new metrics are calculated:
Query: select t1.a, t2.b from t1 join t2 on t1.id = t2.id
Shards: 2
Sharding Key: id for both tables
Metrics Published:
1. QueryExecutions – {select, scatter, primary}, 1
2. QueryRoutes – {select, scatter, primary}, 2
3. QueryExecutionsByTable – {select, t1}, 1 and {select, t2}, 1
VTTablet
| Name | Dimensions | Description | PR |
|---|---|---|---|
TableRows |
Table |
Estimated number of rows in the table. | #17570 |
TableClusteredIndexSize |
Table |
Byte size of the clustered index (i.e. row data). | #17570 |
IndexCardinality |
Table, Index |
Estimated number of unique values in the index | #17570 |
IndexBytes |
Table, Index |
Byte size of the index. | #17570 |
UnresolvedTransaction |
ManagerType |
Current number of unresolved transactions. | #16939 |
CommitPreparedFail |
FailureType |
Transactions that failed to commit after prepare. | #16939 |
RedoPreparedFail |
FailureType |
Transactions that failed to re-prepare. | #16939 |
Config File Changes
VTOrc
VTOrc now supports dynamic configuration using viper. The old --config parameter has been removed. Use the --config-file parameter instead. Configuration can be provided in json, yaml or any other format that viper supports.
The following fields can be dynamically changed -
instance-poll-timeprevent-cross-cell-failoversnapshot-topology-intervalreasonable-replication-lagaudit-to-backendaudit-to-syslogaudit-purge-durationwait-replicas-timeouttolerable-replication-lagtopo-information-refresh-durationrecovery-poll-durationallow-emergency-reparentchange-tablets-with-errant-gtid-to-drained
To upgrade to the new version of the configuration file, first switch to using the flags in your current deployment before upgrading. Then you can switch to using the configuration file after upgrade.
VTGate
The Viper configuration keys for the following flags has been changed to match their flag names. Previously they had a discovery prefix instead of it being part of the name.
| Flag Name | Old Configuration Key | New Configuration Key |
|---|---|---|
discovery_low_replication_lag |
discovery.low_replication_lag |
discovery_low_replication_lag |
discovery_high_replication_lag_minimum_serving |
discovery.high_replication_lag_minimum_serving |
discovery_high_replication_lag_minimum_serving |
discovery_min_number_serving_vttablets |
discovery.min_number_serving_vttablets |
discovery_min_number_serving_vttablets |
discovery_legacy_replication_lag_algorithm |
discovery.legacy_replication_lag_algorithm |
discovery_legacy_replication_lag_algorithm |
To upgrade to the newer version of the configuration keys, first switch to using the flags in your current deployment before upgrading. Then you can switch to using the new configuration keys after upgrade.
VTOrc
Stalled Disk Recovery
VTOrc can now identify and recover from stalled disk errors.
VTTablets test whether the disk is writable and they send this information in the FullStatus RPC response to VTOrc.
If the disk is not writable on the primary tablet, VTOrc will attempt to recover the cluster by promoting a new primary.
This is useful in scenarios where the disk is stalled and the primary vttablet is unable to accept writes because of it.
To opt into this feature, --enable-primary-disk-stalled-recovery flag has to be specified on VTOrc, and --disk-write-dir flag has to be specified on the vttablets.
--disk-write-interval and --disk-write-timeout flags can be used to configure the polling interval and timeout respectively.
KeyRanges in --clusters_to_watch
VTOrc now supports specifying keyranges in the --clusters_to_watch flag. This means that there is no need to restart a VTOrc instance with a different flag value when you reshard a keyspace.
For example, if a VTOrc is configured to watch ks/-80, then it would watch all the shards that fall under the keyrange -80.
If a reshard is performed and -80 is split into new shards -40 and 40-80, the VTOrc instance will automatically start watching the new shards without needing a restart.
In the previous logic, specifying ks/-80 for the flag would mean that VTOrc would watch only 1 (or no) shard.
In the new system, since we interpret -80 as a key range, it can watch multiple shards as described in the example.
Users can continue to specify exact keyranges. The new feature is backward compatible.
New Default Versions
MySQL 8.0.40
The default major MySQL version used by our vitess/lite:latest image is going from 8.0.30 to 8.0.40.
This change was merged in #17552.
VTGate also advertises MySQL version 8.0.40 by default instead of 8.0.30. If that is not what you are running, you can set the mysql_server_version flag to advertise the desired version.
⚠️ Upgrading to this release with vitess-operator:
If you are using the
vitess-operator, considering that we are bumping the MySQL version from8.0.30to8.0.40, you will have to manually upgrade:
- Add
innodb_fast_shutdown=0to your extra cnf in your YAML file.- Apply this file.
- Wait for all the pods to be healthy.
- Then change your YAML file to use the new Docker Images (
vitess/lite:v22.0.0).- Remove
innodb_fast_shutdown=0from your extra cnf in your YAML file.- Apply this file.
This is the last time this will be needed in the
8.0.xseries, as starting with MySQL8.0.35it is possible to upgrade and downgrade between8.0.xversions without needing to runinnodb_fast_shutdown=0.
Docker vitess/lite images with Debian Bookworm
The docker build system now uses Debian Bookworm instead of Debian Bullseye for the vitess/lite images. This change was merged in #17552.
New Support
More Efficient JSON Replication
In #7345 we added support for --binlog-row-value-options=PARTIAL_JSON. You can read more about this feature added to MySQL 8.0 here.
If you are using MySQL 8.0 or later and using JSON columns, you can now enable this MySQL feature across your Vitess cluster(s) to lower the disk space needed for binary logs and improve the CPU and memory usage in both mysqld (standard intrashard MySQL replication) and vttablet (VReplication) without losing any capabilities or features.
LAST_INSERT_ID(x)
In #17408 and #17409, we added the ability to use LAST_INSERT_ID(x) in Vitess directly at vtgate. This improvement allows certain queries—like SELECT last_insert_id(123); or SELECT last_insert_id(count(*)) ...—to be handled without relying on MySQL for the final value.
Limitations:
- When using
LAST_INSERT_ID(x)in ordered queries (e.g.,SELECT last_insert_id(col) FROM table ORDER BY foo), MySQL sets the session’s last-insert-id value according to the last row returned. Vitess does not guarantee the same behavior.
Maximum Idle Connections in the Pool
In #17443 we introduced a new configurable max-idle-count parameter for connection pools. This allows you to specify the maximum number of idle connections retained in each connection pool to optimize performance and resource efficiency.
You can control idle connection retention for the query server’s query pool, stream pool, and transaction pool with the following flags: • –queryserver-config-query-pool-max-idle-count: Defines the maximum number of idle connections retained in the query pool. • –queryserver-config-stream-pool-max-idle-count: Defines the maximum number of idle connections retained in the stream pool. • –queryserver-config-txpool-max-idle-count: Defines the maximum number of idle connections retained in the transaction pool.
This feature ensures that, during traffic spikes, idle connections are available for faster responses, while minimizing overhead in low-traffic periods by limiting the number of idle connections retained. It helps strike a balance between performance, efficiency, and cost.
Filtering Query logs on Error
The querylog-mode setting can be configured to error to log only queries that result in errors. This option is supported in both VTGate and VTTablet.
MultiQuery RPC in vtgate
New RPCs have been added to VTGate that allow users to pass multiple queries in a single sql string. The behavior is the same as that of MySQL: the RPCs will return multiple result sets in the same order as the queries until and unless an error is encountered. The new RPCs are ExecuteMulti and StreamExecuteMulti.
A new flag --mysql-server-multi-query-protocol has also been added that makes the server use this new implementation. This flag is set to false by default, so the old implementation is used by default. The new implementation is more efficient and allows for better performance when executing multiple queries in a single RPC call.
Unsharded CREATE PROCEDURE support
Until now Vitess didn’t allow users to create procedures through VTGate. They had to be created by running a DDL directly against the underlying MySQL. In this release, we have started adding support for running CREATE PROCEDURE statements through VTGate for unsharded keyspaces. Not all constructs of procedures are currently supported in the parser, so there are still some limitations which will be addressed in future releases.
Optimization
Prepared Statement
Prepared statements now benefit from Deferred Optimization, enabling parameter-aware query plans. Initially, a baseline plan is created at prepare-time, and on first execution, a more efficient parameter-optimized plan is generated. Subsequent executions dynamically switch between these plans based on input values, improving query performance while ensuring correctness.
New VtctldServer RPC
GetTransactionInfo RPC has been added to VtctldServer and TabletManagerClient interfaces. These RPCs can be used to read the state of an unresolved distributed transaction. This can be useful in debugging what went wrong and how to fix the problem.
Prefer not promoting a replica that is currently taking a backup
Emergency reparents now prefer to not promote replicas that are currently taking backups with a backup engine other than
builtin. Note that if there’s only one suitable replica to promote, and it is taking a backup, it will still be
promoted.
For planned reparents, hosts taking backups with a backup engine other than builtin are filtered out of the list of
valid candidates. This means they will never get promoted - not even if there are no other candidates.
Note that behavior for builtin backups remains unchanged: a replica that is currently taking a builtin backup will
never be promoted, neither by planned nor by emergency reparents.
Semi-sync monitor in vttablet
Primary VTTablets now monitor the semi-sync status of their underlying MySQL instance . We’ve observed cases where a brief network disruption can cause the primary to get stuck indefinitely waiting for semi-sync ACKs. In rare scenarios, this can also block reparent operations and render the primary unresponsive. More information can be found in the issues #17709 and #17749.
To address this, the primary VTTablets continuously monitor their semi-sync status. If the primary MySQL gets stuck waiting for semi-sync ACKs, the monitor generates synthetic writes to unblock it. If this fails, VTOrc is notified of the issue and it will initiate an emergency reparent operation.
The monitoring interval can be adjusted using the --semi-sync-monitor-interval flag, which defaults to 10 seconds.
Wrapped fatal transaction errors
When a query fails while running in a transaction, due to the transaction no longer being valid (e.g. PRS, rollout, primary down, etc.), the original error is now wrapped in a VT15001 error.
When a query produce a VT15001 error, VTGate will try to rollback and clear the transaction.
Any new queries on the same connection will fail with a VT09032 error, until a ROLLBACK is received
to acknowledge that the transaction was automatically rolled back and cleared by VTGate.
VT09032 is returned to clients to avoid applications blindly sending queries to VTGate thinking they are still in a transaction.
This change was introduced by #17669.
Minor Changes
--topo_read_concurrency behaviour changes
The --topo_read_concurrency flag was added to all components that access the topology and the provided limit is now applied separately for each global or local cell (default 32).
All topology read calls (Get, GetVersion, List and ListDir) now respect this per-cell limit. Previous to this version a single limit was applied to all cell calls and it was not respected by many topology calls.
VTTablet
CLI Flags
-
twopc_abandon_ageflag now supports values in the time.Duration format (e.g., 1s, 2m, 1h). While the flag will continue to accept float values (interpreted as seconds) for backward compatibility, float inputs are deprecated and will be removed in a future release. -
--consolidator-query-waiter-capflag to set the maximum number of clients allowed to wait on the consolidator. The default value is set to 0 for unlimited wait. Users can adjust this value based on the performance of VTTablet to avoid excessive memory usage and the risk of being OOMKilled, particularly in Kubernetes deployments.
ACL enforcement and reloading
When a tablet is started with --enforce-tableacl-config it will exit with an error if the contents of the file are not valid. After the changes made in #17485 the tablet will no longer exit when reloading the contents of the file after receiving a SIGHUP. When the file contents are invalid on reload the tablet will now log an error and the active in-memory ACLs remain in effect.
VTAdmin
vtadmin-web updated to node v22.13.1 (LTS)
Building vtadmin-web now requires node >= v22.13.0 (LTS). Breaking changes from v20 to v22 can be found at https://nodejs.org/en/blog/release/v22.13.0 – with no known issues that apply to VTAdmin.
Full details on the node v20.12.2 release can be found at https://nodejs.org/en/blog/release/v22.13.1.
The entire changelog for this release can be found here.
The release includes 474 merged Pull Requests.
Thanks to all our contributors: @GrahamCampbell, @GuptaManan100, @L3o-pold, @akagami-harsh, @anirbanmu, @app/dependabot, @app/vitess-bot, @arthmis, @arthurschreiber, @beingnoble03, @c-r-dev, @corbantek, @dbussink, @deepthi, @derekperkins, @ejortegau, @frouioui, @garfthoffman, @gmpify, @gopoto, @harshit-gangal, @huochexizhan, @jeefy, @jwangace, @kbutz, @lmorduch, @mattlord, @mattrobenolt, @maxenglander, @mcrauwel, @mounicasruthi, @niladrix719, @notfelineit, @rafer, @rohit-nayak-ps, @rvrangel, @shailpujan88, @shanth96, @shlomi-noach, @siadat, @systay, @timvaillancourt, @twthorn, @vitess-bot, @vmg, @wiebeytec, @wukuai