cmsketch.go ×15

Frontier kind: Joint frontier

unlabeled · c_f57e2916b34e

18 tests · 70 LOC · 1 file · introduces 1 test · 70 LOC · 1 file

Introduces — evidence that enters the hierarchy at this concept

Code
15 ranges70 lines · 1 files
Tests
1 test

Contains — complete concept membership

All code (extent)
15 ranges70 lines · 1 file · Browse complete extent
All tests (intent)
18 testsBrowse complete intent

Neighbourhood graph

The orange circle is the focus. Violet and green circles are every ancestor and descendant, broader and narrower, at any distance; blue squares and pink diamonds are the introduced files and exact introduced tests of every visible concept, not only the focus's. Arrows point from broader to narrower concepts and bridge only concepts omitted from this view. Undirected links show source or test introduction. Concept and file size follows LOC; exact test nodes use test-count units.

Introduced files, introduced tests, and structurally relevant concept specialization

In the embedded map, ordinary wheel input scrolls the page; use the visible controls to zoom and drag to pan. Open the full-screen map for canvas navigation: wheel pans, Ctrl/Command plus wheel zooms, and arrow keys pan when this region is focused. On touch screens, open the full-screen map to pan or pinch. If JavaScript or WebGL is unavailable, use the native relationship evidence on this page.

Introduced files, introduced tests, and structurally relevant concept specializationTestFairness_UniformDistribution_500 · 0 introduced LOCTestFairness_UniformDist…TestFairness_CMSCounter · 0 introduced LOCTestFairness_CMSCountermap.go ×1 · 5 introduced LOCmap.go ×1TestHybridCounter_TopKPreservedOnCMSResize · 0 introduced LOCTestHybridCounter_TopKPr…TestHybridCounter_TopKUpdatesAfterMigration · 0 introduced LOCTestHybridCounter_TopKUp…TestHybridCounter_TopKTracking · 0 introduced LOCTestHybridCounter_TopKTr…hybrid.go ×1 · 5 introduced LOChybrid.go ×1hybrid.go ×2 · 8 introduced LOChybrid.go ×2TestCMSketch_SlideBase_Headroom · 0 introduced LOCTestCMSketch_SlideBase_H…cmsketch.go ×2 · 8 introduced LOCcmsketch.go ×2cmsketch.go ×1 · 1 introduced LOCcmsketch.go ×1TestCMSketch_SlideBase_TargetBelowBase · 0 introduced LOCTestCMSketch_SlideBase_T…cmsketch.go ×2 · 13 introduced LOCcmsketch.go ×2cmsketch.go ×3 · 10 introduced LOCcmsketch.go ×3cmsketch.go ×1 · 3 introduced LOCcmsketch.go ×1cmsketch.go ×1 · 4 introduced LOCcmsketch.go ×1cmsketch.go ×1 · 2 introduced LOCcmsketch.go ×1cmsketch.go ×1 · 3 introduced LOCcmsketch.go ×1cmsketch.go ×1 · 10 introduced LOCcmsketch.go ×1cmsketch.go ×2 · 9 introduced LOCcmsketch.go ×2cmsketch.go ×2 · 4 introduced LOCcmsketch.go ×2cmsketch.go ×1 · 2 introduced LOCcmsketch.go ×1TestOperatorServiceMetadata, TestWorkflowServiceMetadata · 0 introduced LOCTestOperatorServiceMetad…go.temporal.io/server/service/matching/counter/cmsketch.go · 262 LOCcounter/cmsketch.gogo.temporal.io/server/service/matching/counter/hybrid.go · 85 LOCcounter/hybrid.gogo.temporal.io/server/service/matching/counter/map.go · 102 LOCcounter/map.goTestOperatorServiceMetadata · introduced test · go.temporal.io/server/common/api/TestOperatorServiceMetadataTestOperatorServiceMetad…TestWorkflowServiceMetadata · introduced test · go.temporal.io/server/common/api/TestWorkflowServiceMetadataTestWorkflowServiceMetad…TestCMSketch_Basic · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_BasicTestCMSketch_BasicTestCMSketch_CrossMaxInt32 · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_CrossMaxInt32TestCMSketch_CrossMaxInt…TestCMSketch_Grow · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_GrowTestCMSketch_GrowTestCMSketch_Grow_PreservedOnResize · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_Grow_PreservedOnResizeTestCMSketch_Grow_Preser…TestCMSketch_Reseed · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_ReseedTestCMSketch_ReseedTestCMSketch_Reseed_BreaksCollision · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_Reseed_BreaksCollisionTestCMSketch_Reseed_Brea…TestCMSketch_SlideBase · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_SlideBaseTestCMSketch_SlideBaseTestCMSketch_SlideBase_DragUp · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_SlideBase_DragUpTestCMSketch_SlideBase_D…TestCMSketch_SlideBase_Headroom · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_SlideBase_HeadroomTestCMSketch_SlideBase_H…TestCMSketch_SlideBase_LargeDelta · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_SlideBase_LargeDeltaTestCMSketch_SlideBase_L…TestCMSketch_SlideBase_MultipleSlides · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_SlideBase_MultipleSlidesTestCMSketch_SlideBase_M…TestCMSketch_SlideBase_TargetBelowBase · introduced test · go.temporal.io/server/service/matching/counter/TestCMSketch_SlideBase_TargetBelowBaseTestCMSketch_SlideBase_T…TestHybridCounter_Migrates · introduced test · go.temporal.io/server/service/matching/counter/TestHybridCounter_MigratesTestHybridCounter_Migrat…TestHybridCounter_TopKPreservedOnCMSResize · introduced test · go.temporal.io/server/service/matching/counter/TestHybridCounter_TopKPreservedOnCMSResizeTestHybridCounter_TopKPr…TestHybridCounter_TopKTracking · introduced test · go.temporal.io/server/service/matching/counter/TestHybridCounter_TopKTrackingTestHybridCounter_TopKTr…TestHybridCounter_TopKUpdatesAfterMigration · introduced test · go.temporal.io/server/service/matching/counter/TestHybridCounter_TopKUpdatesAfterMigrationTestHybridCounter_TopKUp…TestFairness_CMSCounter · introduced test · go.temporal.io/server/tools/fairsim/TestFairness_CMSCounterTestFairness_CMSCounterTestFairness_UniformDistribution_500 · introduced test · go.temporal.io/server/tools/fairsim/TestFairness_UniformDistribution_500TestFairness_UniformDist…Focused concept · cmsketch.go ×15 · 70 introduced LOCcmsketch.go ×15

Graph controls are ready.

Interactive rendering requires JavaScript and WebGL. Use the native relationship evidence on this page while the interactive map is unavailable.

Native relationship evidence

Every exact file and test below is linked only from the concept that introduces it.

Introduced tests

Every collected test enters the hierarchy at exactly one concept.

1 test introduced at this concept.

Introduced code

Every collected source range enters the hierarchy at exactly one concept.

1 file ranked by introduced lines: 70 introduced LOC across 15 ranges. Expand a file to inspect source; the > gutter marks introduced lines.

go.temporal.io/server/service/matching/counter/cmsketch.go 70 introduced LOC · 15 ranges

Open complete file

59 var _ Counter = (*cmSketch)(nil)
60
61 > func NewCMSketchCounter(params CMSketchParams, src rand.Source, topKProvider topKFunc) *cmSketch { cmsketch.go
62 > params.D = max(1, params.D)
63 > params.W = max(1, params.W)
64 > params.Grow.SkipRateDecay = max(1_000, params.Grow.SkipRateDecay)
65 > numRows := params.D + 1 // + 1 for shadow row
66 > return &cmSketch{
67 > params: params,
68 > seed0: maphash.MakeSeed(),
69 > seeds: makeSeeds(numRows, src),
70 > cells: make([]uint32, params.W*numRows),
71 > shadowRow: 0,
72 > src: src,
73 > topKProvider: topKProvider,
74 > }
75 > }
76
77 > func (s *cmSketch) GetPass(key string, base, inc int64) int64 { cmsketch.go
78 > if inc < 0 {
79 return base // we don't handle negatives here
80 }
81
82 > numRows := s.params.D + 1 cmsketch.go
83 > indexes := make([]int, numRows)
84 > s.fillIndexes(key, indexes)
85 >
86 > current := s.getByIndexes(indexes)
87 > pass := max(base, current+inc)
88 > s.skips += s.ensureByIndexes(indexes, pass)
89 >
90 > if s.incs++; s.incs > s.params.Grow.SkipRateDecay {
91 s.maybeGrow()
92 s.skips >>= 1
94 }
95
96 > if s.reseedOps++; s.params.Reseed.Interval > 0 && s.reseedOps >= s.params.Reseed.Interval { cmsketch.go
97 s.reseed()
98 s.reseedOps = 0
99 }
100
101 > return int64(pass) cmsketch.go
102 }
103
124 // fillIndexes computes cell indexes for all D+1 rows (D active + 1 shadow).
125 // len(indexes) must == len(s.seeds) == D+1
126 > func (s *cmSketch) fillIndexes(k string, indexes []int) { cmsketch.go
127 > w := s.params.W
128 > // get 64 bits of hash
129 > h0 := maphash.String(s.seed0, k)
130 >
131 > for i, seed := range s.seeds {
132 > h1 := bits.RotateLeft64(h0, i*39)
133 > h2l := mix(uint32(h1), uint32(seed))
134 > h2h := mix(uint32(h1>>32), uint32(seed>>32))
135 > h3 := mix(h2l, h2h)
136 > // https://lemire.me/blog/2016/06/27/a-fast-alternative-to-the-modulo-reduction/
137 > indexes[i] = i*w + int((uint64(h3)*uint64(w))>>32)
138 > }
139 }
140
188 }
189
190 > func (s *cmSketch) getByIndexes(indexes []int) int64 { cmsketch.go
191 > // TODO: consider using better estimator: https://dl.acm.org/doi/pdf/10.1145/3219819.3219975
192 > minVal := uint32(math.MaxUint32)
193 > for i, idx := range indexes {
194 > if i == s.shadowRow {
195 > continue // skip shadow row for reads
196 }
197 > minVal = min(minVal, s.cells[idx]) cmsketch.go
198 }
199 > return s.base + int64(minVal) cmsketch.go
200 }
201
202 > func (s *cmSketch) ensureByIndexes(indexes []int, target int64) (skips int) { cmsketch.go
203 > offset := target - s.base
204 > if offset < 0 {
205 // target is below our window floor, all cells are already high enough
206 return s.params.D // only count active rows for skips
207 }
208 > if offset > math.MaxUint32 { cmsketch.go
209 // would overflow uint32, need to slide the base up first
210 s.slideBase(offset + slideHeadroom - math.MaxUint32)
212 }
213
214 > uoffset := uint32(offset) cmsketch.go
215 > for i, idx := range indexes {
216 > if s.cells[idx] < uoffset {
217 > s.cells[idx] = uoffset
218 > } else if i != s.shadowRow {
219 skips++ // only count skips for active rows, not shadow
220 }
221 }
222 > return cmsketch.go
223 }
224
247 }
248
249 > func makeSeeds(rows int, src rand.Source) []uint64 { cmsketch.go
250 > out := make([]uint64, rows)
251 > for i := range out {
252 > out[i] = src.Uint64()
253 > }
254 > return out
255 }
256
257 // from https://www.pcg-random.org/posts/developing-a-seed_seq-alternative.html
258 > func mix(x, y uint32) uint32 { cmsketch.go
259 > result := 0xca01f9dd*x - 0x4973f715*y
260 > result ^= result >> 16
261 > return result
262 > }