Vihren Atlas

queue_base.go ×11

Code frontier

unlabeled · c_5ff4afb9bd24

5 tests · 4582 LOC · 198 files · introduces 0 tests · 51 LOC · 3 files

Introduces — evidence that enters the hierarchy at this concept

Code
13 ranges51 lines · 3 files
Tests
0 tests

Contains — complete concept membership

All code (extent)
733 ranges4582 lines · 198 files · Browse complete extent
All tests (intent)
5 testsBrowse complete intent

Neighbourhood graph

The orange circle is the focus. Violet and green circles are every ancestor and descendant, broader and narrower, at any distance; blue squares and pink diamonds are the introduced files and exact introduced tests of every visible concept, not only the focus's. Arrows point from broader to narrower concepts and bridge only concepts omitted from this view. Undirected links show source or test introduction. Concept and file size follows LOC; exact test nodes use test-count units.

Introduced files, introduced tests, and structurally relevant concept specialization

In the embedded map, ordinary wheel input scrolls the page; use the visible controls to zoom and drag to pan. Open the full-screen map for canvas navigation: wheel pans, Ctrl/Command plus wheel zooms, and arrow keys pan when this region is focused. On touch screens, open the full-screen map to pan or pinch. If JavaScript or WebGL is unavailable, use the native relationship evidence on this page.

Graph controls are ready.

Interactive rendering requires JavaScript and WebGL. Use the native relationship evidence on this page while the interactive map is unavailable.

Native relationship evidence

Every exact file and test below is linked only from the concept that introduces it.

Introduced tests

Every collected test enters the hierarchy at exactly one concept.

No tests are introduced at this concept. Its intent tests are introduced by other concepts.

Introduced code

Every collected source range enters the hierarchy at exactly one concept.

3 files ranked by introduced lines: 51 introduced LOC across 13 ranges. Expand a file to inspect source; the > gutter marks introduced lines.

go.temporal.io/server/service/history/queues/queue_base.go 43 introduced LOC · 11 ranges

Open complete file

293 }
294
295 > func (p *queueBase) checkpoint() { queue_base.go
296 > var tasksCompleted int
297 > p.readerGroup.ForEach(func(_ int64, r Reader) {
298 tasksCompleted += r.ShrinkSlices()
299 })
300
301 > var checkpointAction Action queue_base.go
302 > maxReaderCount := p.options.MaxReaderCount()
303 > if taskCountBase := p.options.MoveGroupTaskCountBase(); taskCountBase > 0 {
304 // Run an action to proactively move task group with high pending task to non-default reader
305 // so that upon shard reload, those groups won't block other tasks in the default reader from
306 // being loaded.
307 checkpointAction = newMoveGroupAction(maxReaderCount, p.grouper, taskCountBase, p.options.MoveGroupTaskCountMultiplier(), p.logger)
308 > } else { queue_base.go
309 // Run slicePredicateAction to move slices with non-universal predicate to non-default reader
310 // so that upon shard reload, task loading for those slices won't block other slices in the default reader.
312 }
313
314 > runAction(checkpointAction, p.readerGroup, p.metricsHandler) queue_base.go
315 >
316 > readerScopes := make(map[int64][]Scope)
317 > newExclusiveDeletionHighWatermark := p.nonReadableScope.Range.InclusiveMin
318 > for readerID, reader := range p.readerGroup.Readers() {
319 scopes := reader.Scopes()
320
329 }
330 }
331 > metrics.QueueReaderCountHistogram.With(p.metricsHandler).Record(int64(len(readerScopes))) queue_base.go
332 > metrics.QueueSliceCountHistogram.With(p.metricsHandler).Record(int64(p.monitor.GetTotalSliceCount()))
333 > metrics.PendingTasksCounter.With(p.metricsHandler).Record(int64(p.monitor.GetTotalPendingTaskCount()))
334 >
335 > // NOTE: Must range-complete task first.
336 > // Otherwise, if state is updated first, later deletion fails and the shard gets reloaded.
337 > // Some tasks will never be deleted.
338 > //
339 > // Emit metric before the deletion watermark comparison so we have the emit even if there's no task
340 > // for the queue.
341 > metrics.TaskBatchCompleteCounter.With(p.metricsHandler).Record(1)
342 > if newExclusiveDeletionHighWatermark.CompareTo(p.exclusiveDeletionHighWatermark) > 0 ||
343 > (p.updateShardRangeID() && newExclusiveDeletionHighWatermark.CompareTo(tasks.MinimumKey) > 0) {
344 // When shard rangeID is updated, perform range completion again in case the underlying persistence implementation
345 // serves traffic based on the persisted shardInfo.
353 }
354
355 > err := p.updateQueueState(tasksCompleted, readerScopes) queue_base.go
356 > p.resetCheckpointTimer(err)
357 }
358
393 tasksCompleted int,
394 readerScopes map[int64][]Scope,
395 > ) error { queue_base.go
396 > metrics.AckLevelUpdateCounter.With(p.metricsHandler).Record(1)
397 > for readerID, scopes := range readerScopes {
398 if len(scopes) == 0 {
399 delete(readerScopes, readerID)
401 }
402
403 > err := p.shard.SetQueueState(p.category, tasksCompleted, ToPersistenceQueueState(&queueState{ queue_base.go
404 > readerScopes: readerScopes,
405 > exclusiveReaderHighWatermark: p.nonReadableScope.Range.InclusiveMin,
406 > }))
407 > if err != nil {
408 metrics.AckLevelUpdateFailedCounter.With(p.metricsHandler).Record(1)
409 p.logger.Error("Error updating queue state", tag.Error(err), tag.OperationFailed)
410 }
411 > return err queue_base.go
412 }
413
414 > func (p *queueBase) resetCheckpointTimer(checkPointErr error) { queue_base.go
415 > if checkPointErr != nil {
416 delay := p.checkpointRetrier.NextBackOff(checkPointErr)
417 p.checkpointTimer.Reset(delay)
419 }
420
421 > p.checkpointRetrier.Reset() queue_base.go
422 > p.checkpointTimer.Reset(backoff.Jitter(
423 > p.options.CheckpointInterval(),
424 > p.options.CheckpointIntervalJitterCoefficient(),
425 > ))
426 }
427
go.temporal.io/server/service/history/shard/context_impl.go 6 introduced LOC · 1 range

Open complete file

376 tasksCompleted int,
377 state *persistencespb.QueueState,
378 > ) error { context_impl.go
379 > return s.updateShardInfo(tasksCompleted,
380 > func() {
381 > categoryID := category.ID()
382 > s.shardInfo.QueueStates[int32(categoryID)] = state
383 > })
384 }
385
go.temporal.io/server/service/history/queues/mitigator.go 2 introduced LOC · 1 range

Open complete file

97 readerGroup *ReaderGroup,
98 metricsHandler metrics.Handler,
99 > ) { mitigator.go
100 > if !action.Run(readerGroup) {
101 return
102 }

Direct hierarchy

Parents

Children