发布日期: 2025-12-15
版本号: v1.30.0

Meilisearch v1.30.0 增强了企业版的网络分片功能,允许动态调整分片节点数量,无需重新创建集群。这一改进引入了破坏性变更,主要影响启用了自动分片的实验性网络功能用户,稳定版用户不受影响。使用新功能需要指定一个领导者节点来接收所有任务,并通过 PATCH /network 接口配置网络拓扑。此后,设置、文档等任务仅能发送给领导者,并会自动同步到整个网络。新增或移除节点时,系统会自动重新分配文档。此外,版本更新还包括修复 macOS 编译问题、升级 Rust 版本以及移除 CI 中的命令注入风险,并重新提供了 macOS AMD64 架构的二进制文件。

更新内容 (中文)

🌈 改进

网络上下扩展

v1.19.0 起,Meilisearch 企业版 支持在多个 Meilisearch 实例间自动分片文档,从而实现超越单实例容量的扩展。

Meilisearch v1.30.0 新增了修改分片参与节点数量的功能,无需重新开始向包含所需机器数量的新集群发送文档。

为此,Meilisearch v1.30.0 引入了 重大变更。这些变更仅影响启用自动分片(network.sharding = true)的实验性 network 功能用户。稳定功能的用户不受影响。

使用说明

设置初始网络
  1. 选择一台作为主节点接收所有任务的机器。现在,在非主节点的机器上调用创建文档、设置或网络相关任务的路由会返回错误。
  2. 通过调用 PATCH /network 将网络拓扑发送至主节点:
// PATCH /network
{
  "self": "ms0",
  "leader": "ms0", // 必须与 `self` 相同
  "remotes": {
    "ms0": {
      "url": "URL_OF_MS0",
      "searchApiKey": "SEARCH_API_KEY_OF_MS0",
      "writeApiKey": "WRITE_API_KEY_OF_MS0",
    },
    "ms1": {
      "url": "URL_OF_MS1",
      "searchApiKey": "SEARCH_API_KEY_OF_MS1",
      "writeApiKey": "WRITE_API_KEY_OF_MS1",
    }
  }
}
  1. 网络会自动传播到网络中的其他成员。
  2. 像往常一样发送设置和文档,但仅限发送至主节点。它们会传播到网络中的所有参与者,每个参与者处理部分文档。
从 v1.29 或更早版本升级
  1. 我们建议使用实验性的无转储升级功能从 Meilisearch v1.13+ 升级到 Meilisearch v1.29。
  2. 使用实验性无转储升级时,如果 Meilisearch 实例已有 sharding 设置为 truenetwork 实例,则会修改 network 对象,使主节点为字母顺序排列的第一个远程节点。例如,如果您的网络包含远程节点:ABC,则主节点将设置为 A
添加新远程节点
  1. 在主节点上调用 PATCH /network,提供新远程节点的信息:
{
  "remotes": {
  // 添加关于新远程节点的信息
  "ms2": {
      "url": "URL_OF_MS2",
      "searchApiKey": "SEARCH_API_KEY_OF_MS2",
      "writeApiKey": "WRITE_API_KEY_OF_MS2",
    }
  // 无需重复关于现有远程节点的信息
}
  1. 新的网络将从主节点传播到所有远程节点(包括新远程节点 ms2)。
  2. 所有远程节点将注册一个新的 networkTopologyChange 任务,该任务将在现有远程节点与新远程节点之间“重新平衡”文档,即 ms0ms1 会将部分文档发送给 ms2
移除远程节点
  1. 在主节点上调用 PATCH /network,将任何被移除的远程节点设置为 null
{
  "remotes": {
  // 将被移除的远程节点设置为 null
  "ms2": null
  // 无需重复关于现有远程节点的信息
}
  1. 新的网络将从主节点传播到所有远程节点(包括旧的远程节点 ms2,它随后将不再参与网络)。
  2. 所有远程节点将注册一个新的 networkTopologyChange 任务,该任务将在剩余的远程节点之间“重新平衡”文档,即 ms2 将其文档发送给 ms0ms1

变更列表

变更列表,部分为重大变更
  1. 重大变更/network 路由返回或编辑的 Network 对象进行如下修改:
    • 移除了 sharding 布尔字段。
    • 新增 leader 字段,为可选字符串。当其不为 null 时,具有与先前 Network 对象中 sharding: true 相同(且更多)的效果。主节点用于接收任务创建请求时的验证。
    • 新增 version 字段,为 UUID,默认为 null UUID。该版本在处理任务时使用。
  2. 重大变更:当设置了 network.leader 时,如果目标节点的 network.self 与其 network.leader 不同,调用以下路由将返回 not_a_leader 错误:
    • POST /indexes
    • PATCH/DELETE /indexes/{:indexUid}
    • POST/PUT/DELETE /indexes/{:indexUid}/documents
    • POST /indexes/{:indexUid}/documents/delete
    • POST /indexes/{:indexUid}/documents/delete-batch
    • POST /indexes/{:indexUid}/documents/edit
    • PATCH/DELETE /indexes/{:indexUid}/settings 及设置子路由
    • PATCH /network(如果目标是主节点)
    • POST /swap-indexes
  3. 重大变更:设置了 leader 时,PATCH /network 不再返回 Network 对象。而是生成一个新的 networkTopologyChange 任务,并返回摘要任务视图。
  4. 重大变更:调用以下路由时,主节点会将任务复制到整个网络:
    • POST /indexes(本 PR 新增)
    • PATCH/DELETE /indexes/{:indexUid}(本 PR 新增)
    • POST/PUT/DELETE /indexes/{:indexUid}/documents(本 PR 之前已如此)
    • POST /indexes/{:indexUid}/documents/delete(本 PR 之前已如此)
    • POST /indexes/{:indexUid}/documents/delete-batch(本 PR 之前已如此)
    • POST /indexes/{:indexUid}/documents/edit(本 PR 之前已如此)
    • PATCH/DELETE /indexes/{:indexUid}/settings 及设置子路由(本 PR 新增)
    • PATCH /network(如果目标是主节点,本 PR 新增)
    • POST /swap-indexes(本 PR 新增)
  5. 新的 networkTopologyChange 任务执行以下操作:
    1. 执行所有剩余需要处理且 network.version 低于网络任务版本的任务。
    2. 遍历所有索引中的所有文档,确定其新分片,并将文档发送到在新版本中必须持有该文档的远程节点,并从本地数据库中删除该文档
      • 导出路由代码已重构并特化以支持此功能
      • 如果导出到远程节点失败,相应的文档将保留在本地
      • 如果某个索引没有文档需要发送,仍会使用空负载和包含预期元数据的适当头调用文档添加接口
      • 如果整个远程节点没有文档需要发送,会调用 network 路由,并包含预期元数据的特定头
    3. 等待并处理来自旧版本网络远程节点的任务。
  6. 重大变更:导入转储文件时,我们会丢弃网络中的 selfleader
  7. 网络拓扑变更任务可以取消。在这种情况下,状态将是当前状态(任何已移动的文档将保持移动后的状态)。取消操作需要在所有机器上进行。

由 @dureuill 在 https://github.com/meilisearch/meilisearch/pull/6000 中贡献

🛠️ 维护

资源可用性说明

🌈 Meilisearch 二进制文件再次可用于 meilisearch-enterprise-macos-amd64meilisearch-macos-amd64。在 Meilisearch v1.29 中不可用。

完整变更日志: https://github.com/meilisearch/meilisearch/compare/v1.29.0...v1.30.0

更新内容 (原始)

🌈 Improvements

Up and down network scaling

Since v1.19.0, Meilisearch Enterprise Edition allows the automatic sharding of documents over multiple Meilisearch instances, enabling scaling to more documents than a single instance would accommodate.

Meilisearch v1.30.0 adds the ability to modify the number of participants in sharding, without having to start over sending documents to a new cluster containing the number of desired machines.

To make this possible, Meilisearch v1.30.0 introduces breaking changes. These breaking changes only affect the users of the experimental network feature who enabled automatic sharding (network.sharding = true). Users of the stable features are not affected.

Usage

Setting up the initial network
  1. Pick a leader machine that will receive all tasks. Calling a route that creates a dcument, settings or network-related task on a machine that is not the leader will now return an error.
  2. Send your network topology to the leader by calling PATCH /network:
// PATCH /network
{
  "self": "ms0",
  "leader": "ms0", // must be equal to `self`
  "remotes": {
    "ms0": {
      "url": "URL_OF_MS0",
      "searchApiKey": "SEARCH_API_KEY_OF_MS0",
      "writeApiKey": "WRITE_API_KEY_OF_MS0",
    },
    "ms1": {
      "url": "URL_OF_MS1",
      "searchApiKey": "SEARCH_API_KEY_OF_MS1",
      "writeApiKey": "WRITE_API_KEY_OF_MS1",
    }
  }
}
  1. The network is automatically propagated to other members of the network.
  2. Send settings and documents as usual, but exclusively to the leader. They will be propagated to all participants in the network, and each participant will process a piece of the documents.
Upgrading from v1.29 or earlier
  1. We recommend using the experimental dumpless upgrade feature to go from Meilisearch v1.13+ to Meilisearch v1.29.
  2. When using the experimental dumpless upgrade and the Meilisearch instance already has a network instance with sharding set to true, then the network object will be modified so that the leader is the first remote in alphabetic order. For instance, if you network contains remotes: A, B, C, the leader will be set to A.
Adding a new remote
  1. Call PATCH /network on the leader with the information about the new remote:
{
  "remotes": {
  // add information about the new remote
  "ms2": {
      "url": "URL_OF_MS2",
      "searchApiKey": "SEARCH_API_KEY_OF_MS2",
      "writeApiKey": "WRITE_API_KEY_OF_MS2",
    }
  // information about existing remotes does not need to be repeated
}
  1. The new network will be propagated from the leader to all remotes (including the new remote ms2).
  2. All remotes will register a new networkTopologyChange task that will “rebalance” the documents between the existing remotes and the new remote, that is, ms0 and ms1 will send parts of their documents to ms2
Removing a remote
  1. Call PATCH /network on the leader by setting any removed remote to null:
{
  "remotes": {
  // set removed remote to null
  "ms2": null
  // information about existing remotes does not need to be repeated
}
  1. The new network will be propagated from the leader to all remotes (including to the old remote ms2 that will then no longer participate in the network).
  2. All remotes will register a new networkTopologyChange task that will “rebalance” the documents between the remaining remotes, that is, ms2 will send its documents between ms0 and ms1

List of changes

List of changes, some of which are breaking
  1. Breaking change: The Network object returned or edited by the /network route is modified in the following way:
    • the sharding boolean is removed
    • a leader field is added as an optional string. When it is not null, it has the same effect (and more) than having sharding: true in the previous iteration of the Network object. The leader is used as a check when receiving task creation requests.
    • a version field is added as a UUID, defaulting to the null UUID. The version is used when processing tasks.
  2. Breaking change: When a network.leader is set, calling one of the following routes will fail with not_a_leader error if the target’s network.self is not the same as its network.leader:
    • POST /indexes
    • PATCH/DELETE /indexes/{:indexUid}
    • POST/PUT/DELETE /indexes/{:indexUid}/documents
    • POST /indexes/{:indexUid}/documents/delete
    • POST /indexes/{:indexUid}/documents/delete-batch
    • POST /indexes/{:indexUid}/documents/edit
    • PATCH/DELETE /indexes/{:indexUid}/settings and settings sub-routes
    • PATCH /network if the target is the new leader
    • POST /swap-indexes
  3. Breaking change: when a leader is set, PATCH /network no longer returns a Network object. Rather, it spawns a new NetworkTopologyChange task, and returns the summarized task view.
  4. Breaking change: Tasks are duplicated by the leader to the entire network when calling the following routes:
    • POST /indexes (new to this PR)
    • PATCH/DELETE /indexes/{:indexUid} (new to this PR)
    • POST/PUT/DELETE /indexes/{:indexUid}/documents (was already the case before this PR)
    • POST /indexes/{:indexUid}/documents/delete (was already the case before this PR)
    • POST /indexes/{:indexUid}/documents/delete-batch (was already the case before this PR)
    • POST /indexes/{:indexUid}/documents/edit (was already the case before this PR)
    • PATCH/DELETE /indexes/{:indexUid}/settings and settings sub-routes (new to this PR)
    • PATCH /network if the target is the new leader (new to this PR)
    • POST /swap-indexes (new to this PR)
  5. New NetworkTopologyChange tasks that perform the following:
    1. Execute any remaining task to process with a network.version lower than the network task’s version
    2. Iterate over all documents in all indexes, determine their new shard, and send the document to the remote that must now have it in the new version , deleting it from the local DB
      • The export route code has been factored and specialized to allow this
      • Should the export to a remote fail, the corresponding documents are kept locally
      • If there are no documents to send for an index, still call the documents addition with an empty payload and appropriate headers containing the expected metadata
      • If there are no documents to send for an entire remote, call the network route with specific headers containing the expected metadata
    3. Wait for and process tasks from the remotes of the previous version of the network.
  6. Breaking change: When importing dumps, we drop the self and leader from the network
  7. Network topology change tasks can be cancelled. In this case the state will be the current one (any moved documents will stay that way). Cancellation needs to happen on all machines.

by @dureuill in https://github.com/meilisearch/meilisearch/pull/6000

🛠️ Maintenance

Asset availability note

🌈 The Meilisearch binary is available again for meilisearch-enterprise-macos-amd64 and meilisearch-macos-amd64. It was not available for Meilisearch v1.29.

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.29.0...v1.30.0

下载链接