发布日期: 2026-03-26
版本号: v1.17.3

Dapr 1.17.3 版本更新主要包含多项错误修复和安全修复，旨在解决多个关键问题。安全方面，修复了可能导致 gRPC 授权绕过（CVE-2026-33186）和恶意 TIFF 图像引发内存耗尽拒绝服务（CVE-2026-33809）的漏洞，并升级了相关依赖库。功能修复涵盖了多个方面：解决了通过 h2c 协议调用 actor 方法时可能返回空响应体的问题；修复了服务调用和 actor 响应可能转发过时的 Content-Length 头，导致客户端出现 EOF 错误或截断响应的问题；纠正了为非 Dapr Pod 错误记录注入失败指标的情况；优化了 Placement 服务的传播逻辑，防止单个慢响应导致所有副本连接中断，并改进了 DNS 重连机制以避免在过时 DNS 上长时间挂起；解决了 Scheduler 实例在集群扩容后可能静默停止参与集群的问题；最后，修复了 Windows sidecar 容器因镜像清单缺少 OSVersion 而在 AKS 上无法启动的问题。此版本强烈建议用户升级以获取所有修复。

更新内容 （中文）

Dapr 1.17.3

本次更新包含错误修复和安全修复：

Actor方法调用通过h2c返回200但响应体为空

问题

当使用h2c（HTTP/2明文）应用协议时，Actor方法调用可能返回HTTP 200，带有正确的头部（包括Content-Length），但响应体为空。

影响

使用--app-protocol h2c与Actor的应用，尽管Actor处理器返回了数据，仍可能从Actor方法调用中接收到空的响应体。这导致了难以诊断的数据静默丢失，因为HTTP状态码和头部看起来都是正确的。

根本原因

Dapr v1.17.2为服务调用响应体引入了基于管道的流式处理，以避免在内存中缓冲大型负载。响应头部（包括Content-Length）在管道就绪时被捕获，响应体通过io.Pipe经由io.Copy惰性流式传输。

对于HTTP/2，响应体读取与请求上下文绑定。当上下文被取消时——无论是由弹性策略运行器在InvokeMethod返回后执行defer cancel()，还是由Placement传播取消Actor声明——HTTP/2流会被重置（RST_STREAM）。执行从HTTP/2响应体io.Copy的goroutine随后失败，向管道写入0字节。管道正常关闭（EOF），ProtoWithData()读取到空的响应体。调用者收到200 OK和原始的Content-Length头部，但没有数据。

HTTP/1.1不受影响，因为TCP缓冲区读取不检查请求上下文。

解决方案

HTTP应用通道中的两项更改：

  1. 管道错误传播：io.Copy的错误现在被捕获并通过pw.CloseWithError(err)在管道中传播，而不是静默地以EOF关闭。如果上下文取消导致HTTP/2响应体读取失败，调用者将收到错误而不是空的响应体。

  2. h2c上下文分离：仅针对HTTP/2（h2c）传输，现在发送到应用的HTTP请求使用一个与调用者上下文分离的上下文（context.WithoutCancel）。这防止了弹性超时取消和Placement传播在响应体数据传输过程中重置HTTP/2流。分离的上下文在管道读取器关闭时被取消，防止goroutine泄漏。HTTP/1.1行为保持不变。

安全：gRPC授权绕过

问题

Dapr使用的一个上游依赖项（google.golang.org/grpc）引入了一个漏洞，在某些条件下可能允许gRPC授权绕过（CVE-2026-33186）。

影响

运行受影响版本的用户可能暴露于未授权的gRPC请求。

根本原因

该问题源于一个上游库。

解决方案

此发行版将受影响的依赖项升级到已解决CVE-2026-33186的版本。

强烈建议用户升级到此版本。

服务调用和Actor响应转发了过期的Content-Length头部

问题

当Dapr边车转发来自服务调用或Actor方法调用的HTTP响应时，来自上游应用程序的Content-Length头部会被原样传递给调用者，即使边车从内部protobuf表示重建了响应体。 如果上游应用程序发送了不正确的Content-Length（或在序列化过程中响应体大小发生了变化），调用者收到的Content-Length头部与实际响应体不匹配。

影响

使用标准HTTP客户端（如Go的io.ReadAll或Python的aiohttp）读取响应体的调用者，在转发的Content-Length超过实际响应体大小时，会收到unexpected EOF错误或截断的响应体。 当转发的Content-Length小于实际响应体大小时，客户端会静默截断响应。 这同时影响了通过HTTP进行的服务调用（直接消息传递）和Actor方法调用。

根本原因

InternalMetadataToHTTPHeader工具函数将所有内部gRPC元数据头部转换为HTTP响应头部，包括Content-Length。 由于边车从protobuf消息（而不是代理原始HTTP流）重建响应体，来自上游应用程序的原始Content-Length变得过期。 Go的http.ResponseWriter遵循这个预设的Content-Length头部，而不是根据实际写入的数据计算正确的值。

解决方案

两组更改防止了过期的Content-Length头部传播：

  1. 内部元数据路径：在InternalMetadataToHTTPHeader中，将Content-Length添加到跳过列表，与已跳过的Content-Type并列。这防止了来自上游的过期Content-Length被转发到HTTP响应写入器。Go的http.ResponseWriter现在根据实际响应体自动计算正确的Content-Length。更新了HTTP通道的constructRequest，直接从内部元数据读取Content-Length用于传出请求，因为它不再出现在转发的HTTP头部中。

  2. HTTP通道响应路径：HTTP应用通道现在在通过响应管道转发之前，从上游响应头中剥离Content-Length。如果上游应用程序声明的Content-Length大于实际响应体，Go HTTP客户端产生的io.ErrUnexpectedEOF被视为正常完成，因为接收到的数据是有效的。

安全：gRPC授权绕过

问题

Dapr使用的一个上游依赖项（google.golang.org/grpc）引入了一个漏洞，在某些条件下可能允许gRPC授权绕过（CVE-2026-33186）。

影响

运行受影响版本的用户可能暴露于未授权的gRPC请求。

根本原因

该问题源于一个上游库。

解决方案

此发行版将受影响的依赖项升级到已解决CVE-2026-33186的版本。

强烈建议用户升级到此版本。

安全：TIFF图像OOM拒绝服务

问题

Dapr使用的一个上游依赖项（golang.org/x/image）包含一个漏洞，当解码恶意构造的TIFF图像时，可能导致内存溢出崩溃（CVE-2026-33809）。

影响

一个恶意的8字节TIFF文件，其IFD偏移量为0xFFFFFFFF，可能导致golang.org/x/image/tiff.Decode分配多达约4GB的内存，导致内存溢出崩溃。 任何通过此库处理不受信任的TIFF图像输入的Dapr组件或应用程序路径都可能被利用进行拒绝服务攻击。

根本原因

该问题源于上游golang.org/x/image/tiff库。 buffer.fill()函数在分配内存之前未验证IFD偏移量，允许精心构造的偏移量触发无限分配。

解决方案

此发行版将golang.org/x/image从v0.25.0升级到v0.38.0，解决了CVE-2026-33809。

非Dapr Pod的注入失败指标误报

问题

当由未在注入器允许列表中的服务账户创建了一个没有Dapr注解的Pod（例如基础设施Pod如Vault或Nginx）时，注入器会记录错误（service account '...' not on the list of allowed controller accounts）并递增dapr_injector_sidecar_injection_failed_total指标，即使该Pod从未被设计为启用Dapr。

影响

由不在注入器allowedServiceAccounts列表中的服务账户部署的基础设施和非Dapr工作负载，会产生误报错误日志，并使sidecar_injection_failed_total指标（reason="pod_patch"）虚高。这会在监控系统中为与Dapr无关的Pod触发虚假警报。

根本原因

注入器在检查Pod是否有dapr.io/enabled注解之前，先检查了服务账户授权。非Dapr Pod在授权步骤被拒绝，产生了错误日志和失败指标，而不是被静默允许。

解决方案

注入器现在在检查服务账户授权之前检查dapr.io/enabled注解。未将注解设置为"true"的Pod将立即被允许，不进行任何补丁，跳过所有注入和授权逻辑。这确保了无论哪个服务账户创建，非Dapr Pod都不会产生错误日志或递增失败指标。

Placement传播超时在所有副本间级联

问题

当单个慢或无响应的daprd边车在Placement表传播期间未能响应时，传播超时会断开命名空间中所有连接的边车，而不仅仅是那个慢的边车。 这会导致级联故障，健康的副本失去其Placement表，必须重新连接并重新传播。

在滚动更新期间，快速连续的连接/断开周期会生成一个"版本风暴"，其中Placement服务器处理的传播版本速度超过边车的处理速度，导致影响所有副本的重复超时。

影响

任何使用Actor或工作流的多副本部署都会受到影响。 一个慢的边车（由于GC压力、网络延迟或资源争用）可能导致命名空间中所有副本在超时周期内失去Actor路由。 在滚动更新期间，版本风暴可能阻止新副本长时间接收Placement表，导致Actor调用失败。

根本原因

Placement服务器传播逻辑中的三个问题：

  1. 核弹式超时：handleTimeout在超时时关闭所有流，无论它们是否已对当前传播阶段做出响应。 已经确认LOCK/UPDATE的健康流与慢流一起被断开。

  2. 断开时不推进阶段：当流在活跃的传播轮次中断开时，streamsInTargetState计数器未被调整。 如果断开的流是最后一个未响应的，轮次将停滞直到超时触发，即使所有剩余的流都已经响应。

  3. 孤立的存储条目：在滚动更新期间，流关闭事件可能在传播轮次已经完成后才到达传播器，导致Placement表中出现过期的主机条目。

解决方案

  • 选择性超时：handleTimeout现在只关闭未到达当前传播阶段的流。 成功响应的流在超时后存活并参与下一轮。 在超时轮次期间排队的等待连接被添加到新一轮中，而不是被取消。

  • 断开时推进阶段：handleCloseStream现在在已计数的流断开时递减streamsInTargetState，并在移除该流完成当前阶段时调用advancePhase()。 这防止了当慢流断开时轮次停滞。

  • 删除批处理：活跃传播期间的流删除被排队，并在轮次完成时处理，将多个删除合并到一个传播轮次中。 handleAdd也在添加新流之前处理待处理的删除，在滚动更新期间合并delete+add对。

  • 孤立清理：每轮传播完成后，扫描存储中那些流不再活跃的条目。 这些孤立的条目将在下一轮中被清理。

Daprd Placement在DNS过期时重连会挂起20秒

问题

当Placement服务器Pod重启时（由于滚动更新、驱逐或崩溃），连接到旧Pod的daprd边车尝试使用缓存的DNS条目重新连接。 缓存的IP地址指向已终止的Pod，每次连接尝试都会挂起20秒（TCP拨号超时），然后再尝试下一个地址。

影响

在3节点的Placement集群中，daprd最多可能顺序尝试3个过期的IP，在最坏的情况下会导致60秒的重连延迟。 在此期间，Actor调用和工作流操作会失败，因为边车没有Placement表。

根本原因

DNS轮询连接器缓存了初始DNS查找中的IP地址，并且仅在缓存耗尽（所有IP都已尝试）后才重新解析。 在Placement Pod重启后，旧的IP地址仍保留在缓存中。 由于gRPC拨号是惰性的（它们会立即成功，但在第一次RPC调用时失败），过期的IP地址直到边车尝试打开Placement流时才被检测到，然后会挂起整个TCP拨号超时时间。

解决方案

DNS连接器现在在每次Connect调用时重新解析DNS，而不是在调用间缓存IP地址。 Kubernetes无头服务DNS始终返回当前的Pod IP集合，因此每次重连尝试都会立即获得新的地址。 轮询索引在查找之间保持不变，以保持均匀分布。

集群扩容后Scheduler实例静默停止参与

问题

在Scheduler Pod重启、滚动更新或集群扩容事件后，一个或多个Scheduler实例可能会静默停止参与集群。 受影响实例的Pod保持运行并通过健康检查，但其cron引擎退出并且从不重启。 该实例停止向WatchHosts API发布其地址，因此daprd边车永远不会发现它。 分配给受影响实例分区的作业、Actor提醒或工作流永远不会被触发。

从用户的角度来看，工作流、定时作业和Actor提醒在Scheduler Pod重启后随机停止触发。 该问题是间歇性的，取决于重启相对于其他集群活动的确切时间。

影响

任何以多实例（HA）配置运行Scheduler的Dapr部署都会受到影响。 当Scheduler实例加入或重新加入集群，导致领导权仲裁发生变化（例如，分区数从2变为3）时，会触发该问题。 连接的daprd边车数量越多，主机广播所需时间越长，竞态窗口越宽，问题发生的可能性就越大。

当错误触发时：

  • 受影响的Scheduler实例拥有分区，但无法在其上交付作业。分配到这些分区的所有工作流活动计时器、定时作业和Actor提醒都停止触发，并被记录为UNDELIVERABLE。
  • 调用WatchHosts的Daprd边车可能收到不完整的主机列表（缺少受影响的实例），或者可能永远收不到响应——使边车无限期地卡在scheduler-watch-hosts就绪门上，阻止应用程序就绪。
  • 剩余的健康实例无法接管受影响实例的分区，因为其领导权键仍然保留在etcd中（租约保持独立继续）。集群陷入所有实例都在运行但仲裁永远无法收敛到正确分区数的状态。

唯一的恢复方法是同时重启所有Scheduler Pod，这会强制进行新的领导权选举。

根本原因

当Scheduler内部的cron模块在分区变更后达到领导权仲裁时，它会调用runEngine来启动cron引擎。 在启动引擎之前，runEngine会将更新的集群主机地址发送到一个内部无缓冲的Go通道（WatchLeadership），以便WatchHosts API可以将它们广播给连接的边车。

如果通道消费者正忙（例如，向许多WatchHosts订阅者广播之前的主机更新），发送会阻塞。 同时，如果另一个Scheduler实例加入并导致第二次仲裁变化，当选的上下文会在发送仍然阻塞时被取消。

由于cron模块已经退出，它永远不会调用Reelect来用新的分区总数更新其领导权键。 其他Scheduler实例看到这个过期的键，无法达成仲裁协议，从而阻止整个集群收敛。

解决方案

Scheduler cron包装器中的内部通道消费者已被替换为非阻塞事件循环（events/loop）。 循环的Enqueue方法永远不会阻塞。 如果当前段已满，它会分配一个新段。 这意味着无论消费者忙于广播多久，来自cron库的通道发送总是会立即完成。

由于发送不再阻塞，导致静默退出的上下文取消竞态条件不再可能发生。 cron循环继续在每次仲裁变化后调用Reelect，领导权键用正确的分区总数更新，所有实例正常收敛。

Windows边车容器在AKS上因镜像清单缺少OSVersion而启动失败

问题

从Dapr v1.16.9开始，Dapr边车容器（daprd）在AKS Windows节点上启动失败，错误如下：

hcs::CreateComputeSystem daprd: The container operating system does not match the host operating system.

影响

从v1.16.9开始，所有基于Windows的AKS Dapr边车部署都损坏了。daprd容器进入CrashLoopBackOff状态并且永远无法启动。Linux部署不受影响。

根本原因

在v1.16.9中，docker/docker.mk中的docker-manifest-create目标从使用docker manifest create / docker manifest push更改为docker buildx imagetools create。

docker manifest命令会自动从每个源图像的配置中读取os.version，并将其包含在清单列表条目中。 docker buildx imagetools create命令不会将os.version字段传递到清单列表中。

Windows清单条目中缺少os.version，Windows容器运行时无法区分两个windows/amd64镜像（Server 2019和Server 2022），并拉取了与主机操作系统构建不匹配的错误变体。

解决方案

已将docker-manifest-create目标还原为使用docker manifest create和docker manifest push，恢复了Windows镜像清单列表条目中的os.version字段。

更新内容 （原始）

Dapr 1.17.3

This update contains bug fixes and security fixes:

Actor method invocation returns 200 with empty body over h2c

Problem

When using the h2c (HTTP/2 cleartext) app protocol, actor method invocations could return HTTP 200 with correct headers (including Content-Length) but an empty body.

Impact

Applications using --app-protocol h2c with actors could receive empty response bodies from actor method calls, despite the actor handler returning data. This caused silent data loss that was difficult to diagnose because the HTTP status code and headers appeared correct.

Root Cause

Dapr v1.17.2 introduced pipe-based streaming for service invocation response bodies to avoid buffering large payloads in memory. The response headers (including Content-Length) are captured when the pipe is ready, and the body streams lazily through an io.Pipe via io.Copy.

With HTTP/2, response body reads are tied to the request context. When the context is cancelled — either by the resiliency policy runner’s defer cancel() after InvokeMethod returns, or by placement dissemination cancelling actor claims — the HTTP/2 stream is reset (RST_STREAM). The goroutine performing io.Copy from the HTTP/2 response body then fails, writing 0 bytes to the pipe. The pipe closes normally (EOF), and ProtoWithData() reads an empty body. The caller receives 200 OK with the original Content-Length header but no data.

HTTP/1.1 is unaffected because TCP buffer reads do not check the request context.

Solution

Two changes in the HTTP app channel:

  1. Pipe error propagation: io.Copy errors are now captured and propagated through the pipe via pw.CloseWithError(err) instead of silently closing with EOF. If context cancellation causes the HTTP/2 body read to fail, callers receive an error rather than an empty body.

  2. h2c context detachment: For HTTP/2 (h2c) transports only, the HTTP request to the app now uses a context detached from the caller’s context (context.WithoutCancel). This prevents resiliency timeout cancellation and placement dissemination from resetting the HTTP/2 stream while body data is in flight. The detached context is cancelled when the pipe reader is closed, preventing goroutine leaks. HTTP/1.1 behavior is unchanged.

Security: gRPC authorization bypass

Problem

An upstream dependency (google.golang.org/grpc) used by Dapr introduced a vulnerability that could allow gRPC authorization bypass under certain conditions (CVE-2026-33186).

Impact

Users running affected versions could be exposed to unauthorized gRPC requests.

Root Cause

The issue originated in an upstream library.

Solution

This release upgrades the affected dependency to a version that resolves CVE-2026-33186.

Users are strongly encouraged to upgrade to this release.

Service invocation and actor responses forward stale Content-Length header

Problem

When a Dapr sidecar forwarded HTTP responses from service invocation or actor method calls, the Content-Length header from the upstream application was passed through verbatim to the caller, even though the sidecar rebuilds the response body from an internal protobuf representation. If the upstream application sent an incorrect Content-Length (or the body size changed during serialization), the caller received a Content-Length header that did not match the actual response body.

Impact

Callers reading the response body with standard HTTP clients (such as Go’s io.ReadAll or Python’s aiohttp) received unexpected EOF errors or truncated bodies when the forwarded Content-Length exceeded the actual body size. When the forwarded Content-Length was smaller than the actual body, clients silently truncated the response. This affected both service invocation (direct messaging) and actor method invocation via HTTP.

Root Cause

The InternalMetadataToHTTPHeader utility function converted all internal gRPC metadata headers to HTTP response headers, including Content-Length. Since the sidecar reconstructs the response body from a protobuf message (not by proxying the original HTTP stream), the original Content-Length from the upstream application became stale. Go’s http.ResponseWriter honored this pre-set Content-Length header instead of computing the correct value from the data actually written.

Solution

Two sets of changes prevent stale Content-Length headers from propagating:

  1. Internal metadata path: Added Content-Length to the skip list in InternalMetadataToHTTPHeader, alongside Content-Type which was already skipped. This prevents the stale upstream Content-Length from being forwarded to the HTTP response writer. Go’s http.ResponseWriter now computes the correct Content-Length automatically from the actual response body. Updated the HTTP channel’s constructRequest to read Content-Length directly from internal metadata for outgoing requests, since it is no longer present in the forwarded HTTP headers.

  2. HTTP channel response path: The HTTP app channel now strips Content-Length from upstream response headers before forwarding them through the response pipe. If the upstream app declared a Content-Length larger than the actual body, the resulting io.ErrUnexpectedEOF from Go’s HTTP client is treated as normal completion, since the received data is valid.

Security: gRPC authorization bypass

Problem

An upstream dependency (google.golang.org/grpc) used by Dapr introduced a vulnerability that could allow gRPC authorization bypass under certain conditions (CVE-2026-33186).

Impact

Users running affected versions could be exposed to unauthorized gRPC requests.

Root Cause

The issue originated in an upstream library.

Solution

This release upgrades the affected dependency to a version that resolves CVE-2026-33186.

Users are strongly encouraged to upgrade to this release.

Security: TIFF image OOM denial of service

Problem

An upstream dependency (golang.org/x/image) used by Dapr contained a vulnerability that could cause an out-of-memory crash when decoding a maliciously crafted TIFF image (CVE-2026-33809).

Impact

A malicious 8-byte TIFF file with an IFD offset of 0xFFFFFFFF could cause golang.org/x/image/tiff.Decode to allocate up to ~4GB of memory, leading to an out-of-memory crash. Any Dapr component or application path that processes untrusted TIFF image input through this library could be exploited for denial of service.

Root Cause

The issue originated in the upstream golang.org/x/image/tiff library. The buffer.fill() function did not validate the IFD offset before allocating memory, allowing a crafted offset to trigger an unbounded allocation.

Solution

This release upgrades golang.org/x/image from v0.25.0 to v0.38.0, which resolves CVE-2026-33809.

False positive injection failure metrics for non-Dapr pods

Problem

When a pod without Dapr annotations (e.g. infrastructure pods like Vault or Nginx) was created by a service account not in the injector’s allowed list, the injector logged an error (service account '...' not on the list of allowed controller accounts) and incremented the dapr_injector_sidecar_injection_failed_total metric, even though the pod was never meant to be Dapr-enabled.

Impact

Infrastructure and non-Dapr workloads deployed by service accounts not in the injector’s allowedServiceAccounts list caused false-positive error logs and inflated the sidecar_injection_failed_total metric with reason="pod_patch". This triggered spurious alerts in monitoring systems for pods that had nothing to do with Dapr.

Root Cause

The injector checked service account authorization before checking whether the pod had the dapr.io/enabled annotation. Non-Dapr pods were rejected at the authorization step, producing error logs and failure metrics, instead of being silently allowed.

Solution

The injector now checks the dapr.io/enabled annotation before checking service account authorization. Pods without the annotation set to "true" are immediately allowed with no patch, skipping all injection and authorization logic. This ensures non-Dapr pods never produce error logs or increment failure metrics regardless of which service account creates them.

Placement dissemination timeout cascades across all replicas

Problem

When a single slow or unresponsive daprd sidecar fails to respond during placement table dissemination, the dissemination timeout disconnects all connected sidecars in the namespace, not just the slow one. This causes a cascading failure where healthy replicas lose their placement tables and must reconnect and re-disseminate.

During rolling updates, rapid sequential connect/disconnect cycles generate a “version storm” where the placement server churns through many dissemination versions faster than sidecars can process them, leading to repeated timeouts that affect all replicas.

Impact

Any deployment with multiple replicas using actors or workflows is affected. A single slow sidecar (due to GC pressure, network latency, or resource contention) can cause all replicas in the namespace to lose actor routing for the duration of the timeout cycle. During rolling updates, the version storm can prevent new replicas from receiving placement tables for extended periods, causing actor invocations to fail.

Root Cause

Three issues in the placement server’s dissemination logic:

  1. Nuclear timeout: handleTimeout closed ALL streams on timeout, regardless of whether they had responded to the current dissemination phase. Healthy streams that had already acknowledged LOCK/UPDATE were disconnected alongside the slow one.

  2. No phase advancement on disconnect: When a stream disconnected during an active dissemination round, the streamsInTargetState counter was not adjusted. If the disconnected stream was the last one that hadn’t responded, the round would stall until the timeout fired, even though all remaining streams had already responded.

  3. Orphaned store entries: During rolling updates, stream close events could arrive at the disseminator after the dissemination round had already completed, leaving stale host entries in the placement table.

Solution

  • Selective timeout: handleTimeout now only closes streams that have NOT reached the current dissemination phase. Streams that responded successfully survive the timeout and participate in the next round. Waiting connections that were queued during the timed-out round are added to the new round instead of being cancelled.

  • Phase advancement on disconnect: handleCloseStream now decrements streamsInTargetState when a counted stream disconnects and calls advancePhase() if removing the stream completes the current phase. This prevents rounds from stalling when a slow stream disconnects.

  • Delete batching: Stream deletions during active dissemination are queued and processed when the round completes, combining multiple deletes into a single dissemination round. handleAdd also processes pending deletes before adding new streams, coalescing delete+add pairs during rolling updates.

  • Orphan cleanup: After each dissemination round completes, the store is scanned for entries whose streams are no longer active. These orphaned entries are cleaned up in the next round.

Daprd placement reconnect hangs for 20 seconds on stale DNS

Problem

When a placement server pod is restarted (due to rolling update, eviction, or crash), daprd sidecars that were connected to the old pod attempt to reconnect using cached DNS entries. The cached IP address points to the terminated pod, and each connection attempt hangs for 20 seconds (TCP dial timeout) before trying the next address.

Impact

With a 3-node placement cluster, daprd can try up to 3 stale IPs sequentially, causing a 60-second reconnect delay in the worst case. During this time, actor invocations and workflow operations fail because the sidecar has no placement table.

Root Cause

The DNS round-robin connector cached the IP addresses from the initial DNS lookup and only re-resolved when the cache was exhausted (all IPs tried). After a placement pod restart, the old IP remained in the cache. Since gRPC dials are lazy (they succeed immediately and fail on the first RPC), the stale IP was not detected until the sidecar tried to open a placement stream, which then hung for the full TCP dial timeout.

Solution

The DNS connector now re-resolves DNS on every Connect call instead of caching IPs across calls. Kubernetes headless service DNS always returns the current set of pod IPs, so each reconnect attempt immediately gets fresh addresses. The round-robin index is preserved across lookups to maintain even distribution.

Scheduler instance silently stops participating after cluster scale-up

Problem

After a Scheduler pod restart, rolling update, or cluster scale-up event, one or more Scheduler instances can silently stop participating in the cluster. The affected instance’s pod remains running and passes health checks, but its cron engine exits and never restarts. The instance stops publishing its address to the WatchHosts API, so daprd sidecars never discover it. Jobs, Actor Reminders or Workflows assigned to the affected instance’s partitions are never triggered.

From a user’s perspective, workflows, scheduled jobs, and actor reminders randomly stop firing after a Scheduler pod restart. The issue is intermittent and depends on the exact timing of the restart relative to other cluster activity.

Impact

Any Dapr deployment running the Scheduler in a multi-instance (HA) configuration is affected. The issue is triggered when a Scheduler instance joins or rejoins the cluster, causing a leadership quorum change (e.g., partition count changes from 2 to 3). The likelihood increases with the number of connected daprd sidecars, as more WatchHosts subscribers means the host broadcast takes longer, widening the race window.

When the bug is triggered:

  • The affected Scheduler instance owns partitions but cannot deliver jobs on them. All workflow activity timers, scheduled jobs, and actor reminders assigned to those partitions stop firing and are logged as UNDELIVERABLE.
  • Daprd sidecars that call WatchHosts may receive an incomplete host list (missing the affected instance), or may never receive a response at all — leaving the sidecar stuck on the scheduler-watch-hosts readiness gate indefinitely, preventing the application from becoming ready.
  • The remaining healthy instances cannot take over the affected instance’s partitions because its leadership key remains in etcd (the lease keep-alive continues independently). The cluster is stuck in a state where all instances are running but quorum can never converge on the correct partition count.

The only recovery is to restart all Scheduler pods simultaneously, which forces a fresh leadership election.

Root Cause

When the Scheduler’s internal cron module reaches leadership quorum after a partition change, it calls runEngine to start the cron engine. Before starting the engine, runEngine sends the updated cluster host addresses to an internal unbuffered Go channel (WatchLeadership) so that the WatchHosts API can broadcast them to connected sidecars.

If the channel consumer is busy (for example, broadcasting a previous host update to many WatchHosts subscribers), the send blocks. Meanwhile, if another Scheduler instance joins and causes a second quorum change, the elected context is cancelled while the send is still blocked.

Because the cron module has exited, it never calls Reelect to update its leadership key with the new partition total. The other Scheduler instances see this stale key and cannot reach quorum agreement, preventing the entire cluster from converging.

Solution

The internal channel consumer in the Scheduler’s cron wrapper has been replaced with a non-blocking event loop (events/loop). The loop’s Enqueue method never blocks. If the current segment is full, it allocates a new one. This means the channel send from the cron library always completes immediately, regardless of how busy the consumer is with broadcasting.

Since the send no longer blocks, the context cancellation race that caused the silent exit can no longer occur. The cron loop continues to call Reelect after each quorum change, leadership keys are updated with the correct partition total, and all instances converge normally.

Windows sidecar container fails to start on AKS due to missing OSVersion in image manifest

Problem

Starting with Dapr v1.16.9, the Dapr sidecar container (daprd) fails to start on AKS Windows nodes with the error:

hcs::CreateComputeSystem daprd: The container operating system does not match the host operating system.

Impact

All Windows-based Dapr sidecar deployments on AKS are broken from v1.16.9 onward. The daprd container enters CrashLoopBackOff and never starts. Linux deployments are unaffected.

Root Cause

In v1.16.9, the docker-manifest-create target in docker/docker.mk was changed from using docker manifest create / docker manifest push to docker buildx imagetools create.

The docker manifest commands automatically read os.version from each source image’s config and include it in the manifest list entries. The docker buildx imagetools create command does not carry the os.version field through to the manifest list.

Without os.version on the Windows manifest entries, the Windows container runtime cannot distinguish between the two windows/amd64 images (Server 2019 and Server 2022) and pulls the wrong variant for the host OS build.

Solution

Reverted the docker-manifest-create target to use docker manifest create and docker manifest push, restoring the os.version field in the manifest list entries for Windows images.

下载链接