vitess v22.0.0 版本更新介绍
发布日期: 2025-04-29
版本号: v22.0.0
Vitess v22.0.0 是一个主要版本更新,包含众多功能增强、优化和废弃移除。主要变更包括:废弃了VTGate的多个查询指标(如QueriesProcessed)并替换为新的更细粒度指标(如QueryExecutions);移除了
vttablet的twopc_enable等CLI标志以及gh-ost和pt-osc在线DDL策略;VTOrc支持通过Viper进行动态配置并新增了磁盘卡住恢复功能;默认MySQL版本升级至8.0.40,Docker镜像基础更新为Debian Bookworm;新增了对高效JSON复制、LAST_INSERT_ID(x)、连接池最大空闲连接数、错误查询日志过滤、多查询RPC以及非分片CREATE PROCEDURE的支持;预处理语句引入了延迟优化;在紧急故障转移时优先选择未在备份的副本;VTTablet新增了半同步监控器;事务错误包装提供了更明确的故障状态。次要变更包括拓扑读并发行为调整、VTTablet的CLI标志更新和ACL重载逻辑改进,以及VTAdmin更新至Node.js v22.13.1。
更新内容 (中文)
Vitess v22.0.0 发布
概要
目录
- 主要变更
- 次要变更
<a id="major-changes"/>主要变更
<a id="deprecations"/>弃用项
<a id="deprecated-metrics"/>指标
| 组件 | 指标名称 | 弃用 PR |
|---|---|---|
vtgate |
QueriesProcessed |
#17727 |
vtgate |
QueriesRouted |
#17727 |
vtgate |
QueriesProcessedByTable |
#17727 |
vtgate |
QueriesRoutedByTable |
#17727 |
<a id="deprecated-cli-flags"/>CLI 标志
| 组件 | 标志名称 | 说明 | 弃用 PR |
|---|---|---|---|
vttablet |
twopc_enable |
两阶段提交的使用将由 VTGate 上通过标志或会话变量设置的 transaction_mode 决定。 |
#17279 |
vtgate |
grpc-send-session-in-streaming |
会话将始终作为 StreamExecute RPC 响应的一部分发送。 | #17907 |
<a id="deletions"/>移除项
<a id="deleted-metrics"/>指标
| 组件 | 指标名称 | 弃用于版本 | 移除 PR |
|---|---|---|---|
vttablet |
QueryCacheLength |
v21.0.0 |
#16289 |
vttablet |
QueryCacheSize |
v21.0.0 |
#16289 |
vttablet |
QueryCacheCapacity |
v21.0.0 |
#16289 |
vttablet |
QueryCacheEvictions |
v21.0.0 |
#16289 |
vttablet |
QueryCacheHits |
v21.0.0 |
#16289 |
vttablet |
QueryCacheMisses |
v21.0.0 |
#16289 |
<a id="deleted-cli-flags"/>CLI 标志
| 组件 | 标志名称 | 弃用于版本 | 移除 PR |
|---|---|---|---|
vttablet |
queryserver-enable-settings-pool |
v21.0.0 |
#16280 |
vttablet |
remove-sharded-auto-increment |
v21.0.0 |
#16860 |
vttablet |
disable_active_reparents |
v20.0.0 |
#14871 |
vtgate, vtcombo, vtctld |
healthcheck-dial-concurrency |
v21.0.0 |
#16378 |
<a id="deleted-ghost-ptosc"/>gh-ost 和 pt-osc 在线 DDL 策略
Vitess 不再识别 gh-ost 和 pt-osc (pt-online-schema-change) 在线 DDL 策略。vitess 策略是进行大规模模式变更的推荐方式。mysql 和 direct 策略仍受支持。
以下 vttablet 标志已被移除:
--gh-ost-path--pt-osc-path
尝试使用 gh-ost 或 pt-osc 作为 --ddl-strategy 将会产生错误:
$ vtctldclient ApplySchema --ddl-strategy=\"gh-ost\" ...
$ vtctldclient ApplySchema --ddl-strategy=\"pt-osc\" ...
<a id="new-metrics"/>新增指标
<a id="new-vtgate-metrics"/>VTGate
| 名称 | 维度 | 描述 | PR |
|---|---|---|---|
QueryExecutions |
Query, Plan, Tablet |
已执行的查询数量。 | #17727 |
QueryRoutes |
Query, Plan, Tablet |
查询在其上执行的 vttablet 数量。 | #17727 |
QueryExecutionsByTable |
Query, Table |
vtgate 执行的查询,按表记录计数。 | #17727 |
VStreamsCount |
Keyspace, ShardName, TabletType |
活动的 vstream 数量。 | #17858 |
VStreamsEventsStreamed |
Keyspace, ShardName, TabletType |
所有 vstream 中发送的事件数量。 | #17858 |
VStreamsEndedWithErrors |
Keyspace, ShardName, TabletType |
以错误结束的 vstream 数量。 | #17858 |
CommitModeTimings |
Mode |
提交的时间指标(单事务、多事务、两阶段提交)。 | #16939 |
CommitUnresolved |
N/A | Prepare 后两阶段提交事务失败的计数器。 | #16939 |
#17727 中完成的工作引入了查询的新指标。通过这项工作,我们弃用了几个 vtgate 指标,请参阅弃用指标部分。以下是新指标如何计算的示例:
查询:select t1.a, t2.b from t1 join t2 on t1.id = t2.id
分片:2
分片键:两个表的 id
发布的指标:
1. QueryExecutions – {select, scatter, primary},1
2. QueryRoutes – {select, scatter, primary},2
3. QueryExecutionsByTable – {select, t1},1 和 {select, t2},1
<a id="new-vttablet-metrics"/>VTTablet
| 名称 | 维度 | 描述 | PR |
|---|---|---|---|
TableRows |
Table |
表中估计的行数。 | #17570 |
TableClusteredIndexSize |
Table |
聚簇索引(即行数据)的字节大小。 | #17570 |
IndexCardinality |
Table, Index |
索引中唯一值的估计数量 | #17570 |
IndexBytes |
Table, Index |
索引的字节大小。 | #17570 |
UnresolvedTransaction |
ManagerType |
当前未解析的事务数量。 | #16939 |
CommitPreparedFail |
FailureType |
Prepare 后提交失败的事务。 | #16939 |
RedoPreparedFail |
FailureType |
重新 Prepare 失败的事务。 | #16939 |
<a id="config-file-changes"/>配置文件变更
<a id="vtorc-config-file-changes"/>VTOrc
VTOrc 现在支持使用 viper 进行动态配置。旧的 --config 参数已被移除。请改用 --config-file 参数。配置可以以 json、yaml 或 viper 支持的任何其他格式提供。
以下字段可以动态更改 -
instance-poll-timeprevent-cross-cell-failoversnapshot-topology-intervalreasonable-replication-lagaudit-to-backendaudit-to-syslogaudit-purge-durationwait-replicas-timeouttolerable-replication-lagtopo-information-refresh-durationrecovery-poll-durationallow-emergency-reparentchange-tablets-with-errant-gtid-to-drained
要升级到新版本的配置文件,请先在当前部署中切换到使用标志,然后再进行升级。然后可以在升级后切换到使用配置文件。
<a id="vtgate-config-file-changes"/>VTGate
以下标志的 Viper 配置键已更改为与其标志名称匹配。以前它们有一个 discovery 前缀,而不是作为名称的一部分。
| 标志名称 | 旧配置键 | 新配置键 |
|---|---|---|
discovery_low_replication_lag |
discovery.low_replication_lag |
discovery_low_replication_lag |
discovery_high_replication_lag_minimum_serving |
discovery.high_replication_lag_minimum_serving |
discovery_high_replication_lag_minimum_serving |
discovery_min_number_serving_vttablets |
discovery.min_number_serving_vttablets |
discovery_min_number_serving_vttablets |
discovery_legacy_replication_lag_algorithm |
discovery.legacy_replication_lag_algorithm |
discovery_legacy_replication_lag_algorithm |
要升级到较新版本的配置键,请先在当前部署中切换到使用标志,然后再进行升级。然后可以在升级后切换到使用新的配置键。
<a id="vtorc"/>VTOrc
<a id="stall-disk-recovery-vtorc"/>磁盘停顿恢复
VTOrc 现在可以识别并从磁盘停顿错误中恢复。
VTTablet 测试磁盘是否可写,并将此信息在 FullStatus RPC 响应中发送给 VTOrc。
如果主 tablet 上的磁盘不可写,VTOrc 将尝试通过提升新的主节点来恢复集群。
这在磁盘停顿且主 vttablet 因此无法接受写入的场景中非常有用。
要启用此功能,必须在 VTOrc 上指定 --enable-primary-disk-stalled-recovery 标志,并且必须在 vttablet 上指定 --disk-write-dir 标志。
--disk-write-interval 和 --disk-write-timeout 标志可用于分别配置轮询间隔和超时时间。
<a id="key-range-vtorc"/>--clusters_to_watch 中的 KeyRanges
VTOrc 现在支持在 --clusters_to_watch 标志中指定 keyranges。这意味着当您重新分片一个 keyspace 时,无需使用不同的标志值重启 VTOrc 实例。
例如,如果 VTOrc 配置为监视 ks/-80,那么它将监视 keyrange -80 下的所有分片。
如果执行了重新分片,并且 -80 被拆分为新的分片 -40 和 40-80,VTOrc 实例将自动开始监视新的分片,而无需重启。
在之前的逻辑中,为该标志指定 ks/-80 意味着 VTOrc 将只监视 1 个(或没有)分片。
在新系统中,由于我们将 -80 解释为一个 key range,它可以监视多个分片,如示例所示。
用户可以继续指定精确的 keyranges。新功能是向后兼容的。
<a id="new-default-versions"/>新增默认版本
<a id="mysql-8-0-40"/>MySQL 8.0.40
我们的 vitess/lite:latest 镜像使用的默认主要 MySQL 版本从 8.0.30 升级到了 8.0.40。
此变更在 #17552 中合并。
VTGate 也默认通告 MySQL 版本 8.0.40 而非 8.0.30。如果您运行的不是此版本,可以设置 mysql_server_version 标志来通告所需的版本。
⚠️ 使用 vitess-operator 升级到此版本:
如果您使用
vitess-operator,考虑到我们正在将 MySQL 版本从8.0.30升级到8.0.40,您需要手动升级:
- 在 YAML 文件中的 extra cnf 中添加
innodb_fast_shutdown=0。- 应用此文件。
- 等待所有 Pod 健康。
- 然后更改您的 YAML 文件以使用新的 Docker 镜像 (
vitess/lite:v22.0.0)。- 从 YAML 文件中的 extra cnf 中移除
innodb_fast_shutdown=0。- 应用此文件。
这是在
8.0.x系列中最后一次需要这样做,因为从 MySQL8.0.35开始,可以在8.0.x版本之间进行升级和降级,而无需运行innodb_fast_shutdown=0。
<a id="debian-bookworm"/>使用 Debian Bookworm 的 Docker vitess/lite 镜像
Docker 构建系统现在为 vitess/lite 镜像使用 Debian Bookworm 而非 Debian Bullseye。此变更在 #17552 中合并。
<a id="new-support"/>新增支持
<a id="efficient-json-replication"/>更高效的 JSON 复制
在 #7345 中,我们添加了对 --binlog-row-value-options=PARTIAL_JSON 的支持。您可以在此处阅读有关 MySQL 8.0 中新增此功能的更多信息。
如果您使用 MySQL 8.0 或更高版本并使用 JSON 列,现在可以在整个 Vitess 集群中启用此 MySQL 功能,以减少二进制日志所需的磁盘空间,并降低 mysqld(标准分片内 MySQL 复制)和 vttablet(VReplication)中的 CPU 和内存使用,而不会丢失任何功能或特性。
<a id="last-insert-id"/>LAST_INSERT_ID(x)
在 #17408 和 #17409 中,我们添加了在 vtgate 处直接在 Vitess 中使用 LAST_INSERT_ID(x) 的能力。此改进允许某些查询——例如 SELECT last_insert_id(123); 或 SELECT last_insert_id(count(*)) ...——无需依赖 MySQL 获取最终值即可处理。
限制:
- 在有序查询中使用
LAST_INSERT_ID(x)时(例如SELECT last_insert_id(col) FROM table ORDER BY foo),MySQL 根据返回的最后一行设置会话的 last-insert-id 值。Vitess 不保证相同的行为。
<a id="max-idle-connections"/>连接池中的最大空闲连接数
在 #17443 中,我们为连接池引入了一个新的可配置参数 max-idle-count。这允许您指定每个连接池中保留的最大空闲连接数,以优化性能和资源效率。
您可以使用以下标志控制查询服务器的查询池、流池和事务池的空闲连接保留: • –queryserver-config-query-pool-max-idle-count:定义查询池中保留的最大空闲连接数。 • –queryserver-config-stream-pool-max-idle-count:定义流池中保留的最大空闲连接数。 • –queryserver-config-txpool-max-idle-count:定义事务池中保留的最大空闲连接数。
此功能确保在流量高峰期间,空闲连接可用于更快的响应,同时在低流量期间通过限制保留的空闲连接数来最小化开销。它有助于在性能、效率和成本之间取得平衡。
<a id="query-logs"/>按错误过滤查询日志
可以将 querylog-mode 设置配置为 error,以仅记录导致错误的查询。VTGate 和 VTTablet 均支持此选项。
<a id="multiquery"/>vtgate 中的 MultiQuery RPC
已向 VTGate 添加了新的 RPC,允许用户在单个 sql 字符串中传递多个查询。行为与 MySQL 相同:RPC 将按查询的相同顺序返回多个结果集,除非遇到错误。新的 RPC 是 ExecuteMulti 和 StreamExecuteMulti。
还添加了一个新的标志 --mysql-server-multi-query-protocol,使服务器使用此新实现。此标志默认设置为 false,因此默认情况下使用旧实现。新实现更高效,在单个 RPC 调用中执行多个查询时可实现更好的性能。
<a id="create-procedure"/>非分片 CREATE PROCEDURE 支持
直到现在,Vitess 不允许用户通过 VTGate 创建存储过程。它们必须通过直接对底层 MySQL 运行 DDL 来创建。在此版本中,我们已开始添加对通过 VTGate 运行 CREATE PROCEDURE 语句以支持非分片 keyspace 的支持。解析器中并非所有过程的构造当前都受支持,因此仍存在一些限制,这些限制将在未来的版本中得到解决。
<a id="optimization"/>优化
<a id="prepared-statement"/>预处理语句
预处理语句现在受益于延迟优化,支持参数感知查询计划。 最初,在 prepare 时创建一个基线计划,并在首次执行时生成一个更高效的参数优化计划。 后续执行根据输入值动态切换这些计划,从而在确保正确性的同时提高查询性能。
<a id="rpc-changes"/>新增 VtctldServer RPC
已向 VtctldServer 和 TabletManagerClient 接口添加了 GetTransactionInfo RPC。这些 RPC 可用于读取未解析的分布式事务的状态。这在调试出了什么问题以及如何修复问题时非常有用。
<a id="reparents-prefer-not-backing-up"/>优先不提升当前正在执行备份的副本
紧急切换现在优先不提升当前使用非 builtin 备份引擎执行备份的副本。请注意,如果只有一个合适的副本可提升,而它正在执行备份,则仍然会提升它。
对于计划切换,使用非 builtin 备份引擎执行备份的节点会被过滤出有效候选者列表。这意味着它们永远不会被提升——即使没有其他候选者。
请注意,builtin 备份的行为保持不变:当前正在执行 builtin 备份的副本永远不会被提升,无论是计划切换还是紧急切换。
<a id="semi-sync-monitor"/>vttablet 中的半同步监视器
主 VTTablet 现在监视其底层 MySQL 实例的半同步状态。 我们观察到,短暂的网络中断可能导致主节点无限期地等待半同步 ACK。 在极少数情况下,这还可能阻止切换操作并使主节点无响应。 更多信息可以在问题 #17709 和 #17749 中找到。
为了解决这个问题,主 VTTablet 持续监视其半同步状态。如果主 MySQL 因等待半同步 ACK 而卡住,监视器会生成合成写入以解除其阻塞。如果失败,VTOrc 将收到有关该问题的通知,并将启动紧急切换操作。
可以使用 --semi-sync-monitor-interval 标志调整监视间隔,默认值为 10 秒。
<a id="new-errors-fatal-tx"/>包装致命事务错误
当查询在事务中运行时失败,由于事务不再有效(例如 PRS、回滚、主节点宕机等),原始错误现在包装在 VT15001 错误中。
当查询产生 VT15001 错误时,VTGate 将尝试回滚并清除事务。
同一连接上的任何新查询都将因 VT09032 错误而失败,直到收到 ROLLBACK 以确认事务已被 VTGate 自动回滚和清除。
向客户端返回 VT09032 以避免应用程序盲目地向 VTGate 发送查询,认为它们仍在事务中。
此变更由 #17669 引入。
<a id="minor-changes"/>次要变更
<a id="topo-read-concurrency-changes"/>--topo_read_concurrency 行为变更
--topo_read_concurrency 标志已添加到所有访问拓扑的组件中,提供的限制现在分别应用于每个全局或本地 cell (默认 32)。
所有拓扑读取调用 (Get, GetVersion, List 和 ListDir) 现在都遵守此每个 cell 的限制。在此版本之前,单个限制应用于所有 cell 调用,并且许多拓扑调用并不遵守它。
<a id="minor-changes-vttablet"/>VTTablet
<a id="flags-vttablet"/>CLI 标志
-
twopc_abandon_age标志现在支持 time.Duration 格式的值(例如,1s、2m、1h)。 虽然该标志将继续接受浮点值(解释为秒)以向后兼容, 浮点输入已弃用,并在未来版本中移除。 -
--consolidator-query-waiter-cap标志用于设置允许在合并器上等待的最大客户端数量。默认值设置为 0,表示无限等待。用户可以根据 VTTablet 的性能调整此值,以避免过多的内存使用和被 OOMKilled 的风险,尤其是在 Kubernetes 部署中。
<a id="reloading-vttablet-acl"/>ACL 强制执行与重载
当 tablet 使用 --enforce-tableacl-config 启动时,如果文件内容无效,它将退出并显示错误。在 #17485 中进行更改后,tablet 在收到 SIGHUP 后重载文件内容时将不再退出。当重载时文件内容无效时,tablet 现在将记录错误,并且活动的内存中 ACL 将保持有效。
<a id="vtadmin"/>VTAdmin
<a id="updated-node"/>vtadmin-web 更新至 node v22.13.1 (LTS)
构建 vtadmin-web 现在需要 node >= v22.13.0 (LTS)。从 v20 到 v22 的破坏性变更可在 https://nodejs.org/en/blog/release/v22.13.0 找到——没有已知适用于 VTAdmin 的问题。
有关 node v20.12.2 发布的完整详细信息,请参见 https://nodejs.org/en/blog/release/v22.13.1。
此发布的完整变更日志可在此处找到。
该版本包含 475 个合并的 Pull Request。
感谢所有贡献者:@GrahamCampbell, @GuptaManan100, @L3o-pold, @akagami-harsh, @anirbanmu, @app/dependabot, @app/vitess-bot, @arthmis, @arthurschreiber, @beingnoble03, @c-r-dev, @corbantek, @dbussink, @deepthi, @derekperkins, @ejortegau, @frouioui, @garfthoffman, @gmpify, @gopoto, @harshit-gangal, @huochexizhan, @jeefy, @jwangace, @kbutz, @lmorduch, @mattlord, @mattrobenolt, @maxenglander, @mcrauwel, @mounicasruthi, @niladrix719, @notfelineit, @rafer, @rohit-nayak-ps, @rvrangel, @shailpujan88, @shanth96, @shlomi-noach, @siadat, @systay, @timvaillancourt, @twthorn, @vitess-bot, @vmg, @wiebeytec, @wukuai
更新内容 (原始)
Release of Vitess v22.0.0
Summary
Table of Contents
- Major Changes
- Minor Changes
Major Changes
Deprecations
Metrics
| Component | Metric Name | Deprecation PR |
|---|---|---|
vtgate |
QueriesProcessed |
#17727 |
vtgate |
QueriesRouted |
#17727 |
vtgate |
QueriesProcessedByTable |
#17727 |
vtgate |
QueriesRoutedByTable |
#17727 |
CLI Flags
| Component | Flag Name | Notes | Deprecation PR |
|---|---|---|---|
vttablet |
twopc_enable |
Usage of TwoPC commit will be determined by the transaction_mode set on VTGate via flag or session variable. |
#17279 |
vtgate |
grpc-send-session-in-streaming |
Session will always be sent as part of the response from the StreamExecute RPC. | #17907 |
Deletions
Metrics
| Component | Metric Name | Was Deprecated In | Deletion PR |
|---|---|---|---|
vttablet |
QueryCacheLength |
v21.0.0 |
#16289 |
vttablet |
QueryCacheSize |
v21.0.0 |
#16289 |
vttablet |
QueryCacheCapacity |
v21.0.0 |
#16289 |
vttablet |
QueryCacheEvictions |
v21.0.0 |
#16289 |
vttablet |
QueryCacheHits |
v21.0.0 |
#16289 |
vttablet |
QueryCacheMisses |
v21.0.0 |
#16289 |
CLI Flags
| Component | Flag Name | Was Deprecated In | Deletion PR |
|---|---|---|---|
vttablet |
queryserver-enable-settings-pool |
v21.0.0 |
#16280 |
vttablet |
remove-sharded-auto-increment |
v21.0.0 |
#16860 |
vttablet |
disable_active_reparents |
v20.0.0 |
#14871 |
vtgate, vtcombo, vtctld |
healthcheck-dial-concurrency |
v21.0.0 |
#16378 |
gh-ost and pt-osc Online DDL strategies
Vitess no longer recognizes the gh-ost and pt-osc (pt-online-schema-change) Online DDL strategies. The vitess strategy is the recommended way to make schema changes at scale. mysql and direct strategies continue to be supported.
These vttablet flags have been removed:
--gh-ost-path--pt-osc-path
Attempting to use gh-ost or pt-osc as --ddl-strategy will yield an error:
$ vtctldclient ApplySchema --ddl-strategy="gh-ost" ...
$ vtctldclient ApplySchema --ddl-strategy="pt-osc" ...
New Metrics
VTGate
| Name | Dimensions | Description | PR |
|---|---|---|---|
QueryExecutions |
Query, Plan, Tablet |
Number of queries executed. | #17727 |
QueryRoutes |
Query, Plan, Tablet |
Number of vttablets the query was executed on. | #17727 |
QueryExecutionsByTable |
Query, Table |
Queries executed by vtgate, with counts recorded per table. | #17727 |
VStreamsCount |
Keyspace, ShardName, TabletType |
Number of active vstreams. | #17858 |
VStreamsEventsStreamed |
Keyspace, ShardName, TabletType |
Number of events sent across all vstreams. | #17858 |
VStreamsEndedWithErrors |
Keyspace, ShardName, TabletType |
Number of vstreams that ended with errors. | #17858 |
CommitModeTimings |
Mode |
Timing metrics for commit (Single, Multi, TwoPC). | #16939 |
CommitUnresolved |
N/A | Counter for 2PC transaction failures after Prepare. | #16939 |
The work done in #17727 introduces new metrics for queries. Via this work we have deprecated several vtgate metrics, please see the Deprecated Metrics section. Here is an example of how the new metrics are calculated:
Query: select t1.a, t2.b from t1 join t2 on t1.id = t2.id
Shards: 2
Sharding Key: id for both tables
Metrics Published:
1. QueryExecutions – {select, scatter, primary}, 1
2. QueryRoutes – {select, scatter, primary}, 2
3. QueryExecutionsByTable – {select, t1}, 1 and {select, t2}, 1
VTTablet
| Name | Dimensions | Description | PR |
|---|---|---|---|
TableRows |
Table |
Estimated number of rows in the table. | #17570 |
TableClusteredIndexSize |
Table |
Byte size of the clustered index (i.e. row data). | #17570 |
IndexCardinality |
Table, Index |
Estimated number of unique values in the index | #17570 |
IndexBytes |
Table, Index |
Byte size of the index. | #17570 |
UnresolvedTransaction |
ManagerType |
Current number of unresolved transactions. | #16939 |
CommitPreparedFail |
FailureType |
Transactions that failed to commit after prepare. | #16939 |
RedoPreparedFail |
FailureType |
Transactions that failed to re-prepare. | #16939 |
Config File Changes
VTOrc
VTOrc now supports dynamic configuration using viper. The old --config parameter has been removed. Use the --config-file parameter instead. Configuration can be provided in json, yaml or any other format that viper supports.
The following fields can be dynamically changed -
instance-poll-timeprevent-cross-cell-failoversnapshot-topology-intervalreasonable-replication-lagaudit-to-backendaudit-to-syslogaudit-purge-durationwait-replicas-timeouttolerable-replication-lagtopo-information-refresh-durationrecovery-poll-durationallow-emergency-reparentchange-tablets-with-errant-gtid-to-drained
To upgrade to the new version of the configuration file, first switch to using the flags in your current deployment before upgrading. Then you can switch to using the configuration file after upgrade.
VTGate
The Viper configuration keys for the following flags has been changed to match their flag names. Previously they had a discovery prefix instead of it being part of the name.
| Flag Name | Old Configuration Key | New Configuration Key |
|---|---|---|
discovery_low_replication_lag |
discovery.low_replication_lag |
discovery_low_replication_lag |
discovery_high_replication_lag_minimum_serving |
discovery.high_replication_lag_minimum_serving |
discovery_high_replication_lag_minimum_serving |
discovery_min_number_serving_vttablets |
discovery.min_number_serving_vttablets |
discovery_min_number_serving_vttablets |
discovery_legacy_replication_lag_algorithm |
discovery.legacy_replication_lag_algorithm |
discovery_legacy_replication_lag_algorithm |
To upgrade to the newer version of the configuration keys, first switch to using the flags in your current deployment before upgrading. Then you can switch to using the new configuration keys after upgrade.
VTOrc
Stalled Disk Recovery
VTOrc can now identify and recover from stalled disk errors.
VTTablets test whether the disk is writable and they send this information in the FullStatus RPC response to VTOrc.
If the disk is not writable on the primary tablet, VTOrc will attempt to recover the cluster by promoting a new primary.
This is useful in scenarios where the disk is stalled and the primary vttablet is unable to accept writes because of it.
To opt into this feature, --enable-primary-disk-stalled-recovery flag has to be specified on VTOrc, and --disk-write-dir flag has to be specified on the vttablets.
--disk-write-interval and --disk-write-timeout flags can be used to configure the polling interval and timeout respectively.
KeyRanges in --clusters_to_watch
VTOrc now supports specifying keyranges in the --clusters_to_watch flag. This means that there is no need to restart a VTOrc instance with a different flag value when you reshard a keyspace.
For example, if a VTOrc is configured to watch ks/-80, then it would watch all the shards that fall under the keyrange -80.
If a reshard is performed and -80 is split into new shards -40 and 40-80, the VTOrc instance will automatically start watching the new shards without needing a restart.
In the previous logic, specifying ks/-80 for the flag would mean that VTOrc would watch only 1 (or no) shard.
In the new system, since we interpret -80 as a key range, it can watch multiple shards as described in the example.
Users can continue to specify exact keyranges. The new feature is backward compatible.
New Default Versions
MySQL 8.0.40
The default major MySQL version used by our vitess/lite:latest image is going from 8.0.30 to 8.0.40.
This change was merged in #17552.
VTGate also advertises MySQL version 8.0.40 by default instead of 8.0.30. If that is not what you are running, you can set the mysql_server_version flag to advertise the desired version.
⚠️ Upgrading to this release with vitess-operator:
If you are using the
vitess-operator, considering that we are bumping the MySQL version from8.0.30to8.0.40, you will have to manually upgrade:
- Add
innodb_fast_shutdown=0to your extra cnf in your YAML file.- Apply this file.
- Wait for all the pods to be healthy.
- Then change your YAML file to use the new Docker Images (
vitess/lite:v22.0.0).- Remove
innodb_fast_shutdown=0from your extra cnf in your YAML file.- Apply this file.
This is the last time this will be needed in the
8.0.xseries, as starting with MySQL8.0.35it is possible to upgrade and downgrade between8.0.xversions without needing to runinnodb_fast_shutdown=0.
Docker vitess/lite images with Debian Bookworm
The docker build system now uses Debian Bookworm instead of Debian Bullseye for the vitess/lite images. This change was merged in #17552.
New Support
More Efficient JSON Replication
In #7345 we added support for --binlog-row-value-options=PARTIAL_JSON. You can read more about this feature added to MySQL 8.0 here.
If you are using MySQL 8.0 or later and using JSON columns, you can now enable this MySQL feature across your Vitess cluster(s) to lower the disk space needed for binary logs and improve the CPU and memory usage in both mysqld (standard intrashard MySQL replication) and vttablet (VReplication) without losing any capabilities or features.
LAST_INSERT_ID(x)
In #17408 and #17409, we added the ability to use LAST_INSERT_ID(x) in Vitess directly at vtgate. This improvement allows certain queries—like SELECT last_insert_id(123); or SELECT last_insert_id(count(*)) ...—to be handled without relying on MySQL for the final value.
Limitations:
- When using
LAST_INSERT_ID(x)in ordered queries (e.g.,SELECT last_insert_id(col) FROM table ORDER BY foo), MySQL sets the session’s last-insert-id value according to the last row returned. Vitess does not guarantee the same behavior.
Maximum Idle Connections in the Pool
In #17443 we introduced a new configurable max-idle-count parameter for connection pools. This allows you to specify the maximum number of idle connections retained in each connection pool to optimize performance and resource efficiency.
You can control idle connection retention for the query server’s query pool, stream pool, and transaction pool with the following flags: • –queryserver-config-query-pool-max-idle-count: Defines the maximum number of idle connections retained in the query pool. • –queryserver-config-stream-pool-max-idle-count: Defines the maximum number of idle connections retained in the stream pool. • –queryserver-config-txpool-max-idle-count: Defines the maximum number of idle connections retained in the transaction pool.
This feature ensures that, during traffic spikes, idle connections are available for faster responses, while minimizing overhead in low-traffic periods by limiting the number of idle connections retained. It helps strike a balance between performance, efficiency, and cost.
Filtering Query logs on Error
The querylog-mode setting can be configured to error to log only queries that result in errors. This option is supported in both VTGate and VTTablet.
MultiQuery RPC in vtgate
New RPCs have been added to VTGate that allow users to pass multiple queries in a single sql string. The behavior is the same as that of MySQL: the RPCs will return multiple result sets in the same order as the queries until and unless an error is encountered. The new RPCs are ExecuteMulti and StreamExecuteMulti.
A new flag --mysql-server-multi-query-protocol has also been added that makes the server use this new implementation. This flag is set to false by default, so the old implementation is used by default. The new implementation is more efficient and allows for better performance when executing multiple queries in a single RPC call.
Unsharded CREATE PROCEDURE support
Until now Vitess didn’t allow users to create procedures through VTGate. They had to be created by running a DDL directly against the underlying MySQL. In this release, we have started adding support for running CREATE PROCEDURE statements through VTGate for unsharded keyspaces. Not all constructs of procedures are currently supported in the parser, so there are still some limitations which will be addressed in future releases.
Optimization
Prepared Statement
Prepared statements now benefit from Deferred Optimization, enabling parameter-aware query plans. Initially, a baseline plan is created at prepare-time, and on first execution, a more efficient parameter-optimized plan is generated. Subsequent executions dynamically switch between these plans based on input values, improving query performance while ensuring correctness.
New VtctldServer RPC
GetTransactionInfo RPC has been added to VtctldServer and TabletManagerClient interfaces. These RPCs can be used to read the state of an unresolved distributed transaction. This can be useful in debugging what went wrong and how to fix the problem.
Prefer not promoting a replica that is currently taking a backup
Emergency reparents now prefer to not promote replicas that are currently taking backups with a backup engine other than
builtin. Note that if there’s only one suitable replica to promote, and it is taking a backup, it will still be
promoted.
For planned reparents, hosts taking backups with a backup engine other than builtin are filtered out of the list of
valid candidates. This means they will never get promoted - not even if there are no other candidates.
Note that behavior for builtin backups remains unchanged: a replica that is currently taking a builtin backup will
never be promoted, neither by planned nor by emergency reparents.
Semi-sync monitor in vttablet
Primary VTTablets now monitor the semi-sync status of their underlying MySQL instance . We’ve observed cases where a brief network disruption can cause the primary to get stuck indefinitely waiting for semi-sync ACKs. In rare scenarios, this can also block reparent operations and render the primary unresponsive. More information can be found in the issues #17709 and #17749.
To address this, the primary VTTablets continuously monitor their semi-sync status. If the primary MySQL gets stuck waiting for semi-sync ACKs, the monitor generates synthetic writes to unblock it. If this fails, VTOrc is notified of the issue and it will initiate an emergency reparent operation.
The monitoring interval can be adjusted using the --semi-sync-monitor-interval flag, which defaults to 10 seconds.
Wrapped fatal transaction errors
When a query fails while running in a transaction, due to the transaction no longer being valid (e.g. PRS, rollout, primary down, etc.), the original error is now wrapped in a VT15001 error.
When a query produce a VT15001 error, VTGate will try to rollback and clear the transaction.
Any new queries on the same connection will fail with a VT09032 error, until a ROLLBACK is received
to acknowledge that the transaction was automatically rolled back and cleared by VTGate.
VT09032 is returned to clients to avoid applications blindly sending queries to VTGate thinking they are still in a transaction.
This change was introduced by #17669.
Minor Changes
--topo_read_concurrency behaviour changes
The --topo_read_concurrency flag was added to all components that access the topology and the provided limit is now applied separately for each global or local cell (default 32).
All topology read calls (Get, GetVersion, List and ListDir) now respect this per-cell limit. Previous to this version a single limit was applied to all cell calls and it was not respected by many topology calls.
VTTablet
CLI Flags
-
twopc_abandon_ageflag now supports values in the time.Duration format (e.g., 1s, 2m, 1h). While the flag will continue to accept float values (interpreted as seconds) for backward compatibility, float inputs are deprecated and will be removed in a future release. -
--consolidator-query-waiter-capflag to set the maximum number of clients allowed to wait on the consolidator. The default value is set to 0 for unlimited wait. Users can adjust this value based on the performance of VTTablet to avoid excessive memory usage and the risk of being OOMKilled, particularly in Kubernetes deployments.
ACL enforcement and reloading
When a tablet is started with --enforce-tableacl-config it will exit with an error if the contents of the file are not valid. After the changes made in #17485 the tablet will no longer exit when reloading the contents of the file after receiving a SIGHUP. When the file contents are invalid on reload the tablet will now log an error and the active in-memory ACLs remain in effect.
VTAdmin
vtadmin-web updated to node v22.13.1 (LTS)
Building vtadmin-web now requires node >= v22.13.0 (LTS). Breaking changes from v20 to v22 can be found at https://nodejs.org/en/blog/release/v22.13.0 – with no known issues that apply to VTAdmin.
Full details on the node v20.12.2 release can be found at https://nodejs.org/en/blog/release/v22.13.1.
The entire changelog for this release can be found here.
The release includes 475 merged Pull Requests.
Thanks to all our contributors: @GrahamCampbell, @GuptaManan100, @L3o-pold, @akagami-harsh, @anirbanmu, @app/dependabot, @app/vitess-bot, @arthmis, @arthurschreiber, @beingnoble03, @c-r-dev, @corbantek, @dbussink, @deepthi, @derekperkins, @ejortegau, @frouioui, @garfthoffman, @gmpify, @gopoto, @harshit-gangal, @huochexizhan, @jeefy, @jwangace, @kbutz, @lmorduch, @mattlord, @mattrobenolt, @maxenglander, @mcrauwel, @mounicasruthi, @niladrix719, @notfelineit, @rafer, @rohit-nayak-ps, @rvrangel, @shailpujan88, @shanth96, @shlomi-noach, @siadat, @systay, @timvaillancourt, @twthorn, @vitess-bot, @vmg, @wiebeytec, @wukuai