visibility_archiver.go ×5

Frontier kind: Joint frontier

unlabeled · c_cce06346348c

1 test · 2275 LOC · 92 files · introduces 1 test · 58 LOC · 1 file

Introduces — evidence that enters the hierarchy at this concept

Code
5 ranges58 lines · 1 files
Tests
1 test

Contains — complete concept membership

All code (extent)
375 ranges2275 lines · 92 files · Browse complete extent
All tests (intent)
1 testBrowse complete intent

Neighbourhood graph

The orange circle is the focus. Violet and green circles are every ancestor and descendant, broader and narrower, at any distance; blue squares and pink diamonds are the introduced files and exact introduced tests of every visible concept, not only the focus's. Arrows point from broader to narrower concepts and bridge only concepts omitted from this view. Undirected links show source or test introduction. Concept and file size follows LOC; exact test nodes use test-count units.

Introduced files, introduced tests, and structurally relevant concept specialization

In the embedded map, ordinary wheel input scrolls the page; use the visible controls to zoom and drag to pan. Open the full-screen map for canvas navigation: wheel pans, Ctrl/Command plus wheel zooms, and arrow keys pan when this region is focused. On touch screens, open the full-screen map to pan or pinch. If JavaScript or WebGL is unavailable, use the native relationship evidence on this page.

Graph controls are ready.

Interactive rendering requires JavaScript and WebGL. Use the native relationship evidence on this page while the interactive map is unavailable.

Native relationship evidence

Every exact file and test below is linked only from the concept that introduces it.

Introduced tests

Every collected test enters the hierarchy at exactly one concept.

1 test introduced at this concept.

Introduced code

Every collected source range enters the hierarchy at exactly one concept.

1 file ranked by introduced lines: 58 introduced LOC across 5 ranges. Expand a file to inspect source; the > gutter marks introduced lines.

go.temporal.io/server/common/archiver/s3store/visibility_archiver.go 58 introduced LOC · 5 ranges

Open complete file

165
166 if strings.TrimSpace(request.Query) == "" {
167 > return v.queryAll(ctx, URI, request, saTypeMap) visibility_archiver.go
168 > }
169
170 parsedQuery, err := v.queryParser.Parse(request.Query)
192 request *archiver.QueryVisibilityRequest,
193 saTypeMap searchattribute.NameTypeMap,
194 > ) (*archiver.QueryVisibilityResponse, error) { visibility_archiver.go
195 > // remaining is the number of workflow executions left to return before we reach pageSize.
196 > remaining := request.PageSize
197 > nextPageToken := request.NextPageToken
198 > var executions []*workflowpb.WorkflowExecutionInfo
199 > // We need to loop because the number of workflow executions returned by each call to query may be fewer than
200 > // pageSize. This is because we may have to skip some workflow executions after querying S3 (client-side filtering)
201 > // because there are 2 entries in S3 for each workflow execution indexed by workflowTypeName (one for closeTimeout
202 > // and one for startTimeout), and we only want to return one entry per workflow execution. See
203 > // createIndexesToArchive for a list of all indexes.
204 > for {
205 > searchPrefix := constructVisibilitySearchPrefix(uri.Path(), request.NamespaceID)
206 > // We suffix searchPrefix with workflowTypeName because the data in S3 is duplicated across combinations of 2
207 > // different primary indices (workflowID and workflowTypeName) and 2 different secondary indices (closeTimeout
208 > // and startTimeout). We only want to return one entry per workflow execution, but the full path to the S3 key
209 > // is <primaryIndexKey>/<primaryIndexValue>/<secondaryIndexKey>/<secondaryIndexValue>/<runID>, and we don't have
210 > // the primaryIndexValue when we make the call to query, so we can only specify the primaryIndexKey.
211 > searchPrefix += "/" + primaryIndexKeyWorkflowTypeName
212 > // The pageSize we supply here is actually the maximum number of keys to fetch from S3. For each execution,
213 > // there should be 2 keys in S3 for this prefix, so you might think that we should multiply the pageSize by 2.
214 > // However, if we do that, we may end up returning more than pageSize workflow executions to the end user of
215 > // this API. This is because we aren't guaranteed that both keys for a given workflow execution will be returned
216 > // in the same call. For example, if the user supplies a pageSize of 1, and we specify a maximum number of keys
217 > // of 2 to S3, we may get back entries from S3 for 2 different workflow executions. You might think that we can
218 > // just truncate this result to 1 workflow execution, but then the nextPageToken would be incorrect. So, we may
219 > // need to make multiple calls to S3 to get the correct number of workflow executions, which will probably make
220 > // this API call slower.
221 > res, err := v.queryPrefix(ctx, uri, &queryVisibilityRequest{
222 > namespaceID: request.NamespaceID,
223 > pageSize: remaining,
224 > nextPageToken: nextPageToken,
225 > parsedQuery: &parsedQuery{},
226 > }, saTypeMap, searchPrefix, func(key string) bool {
227 > // We only want to return entries for the closeTimeout secondary index. Keys for this
228 > // index are always of the form:
229 > // .../closeTimeout/<timestamp>/<runID>
230 > // Walk from the end instead of splitting the whole string to avoid unnecessary
231 > // allocations and to keep the logic clear:
232 > // - drop <runID>
233 > // - drop <timestamp>
234 > // - check the remaining last segment equals "closeTimeout".
235 > dir := path.Dir(key) // drop runID
236 > dir = path.Dir(dir) // drop <timestamp>
237 > return path.Base(dir) == secondaryIndexKeyCloseTimeout
238 > })
239 > if err != nil {
240 return nil, err
241 }
242 > nextPageToken = res.NextPageToken visibility_archiver.go
243 > executions = append(executions, res.Executions...)
244 > remaining -= len(res.Executions)
245 > if len(nextPageToken) == 0 || remaining <= 0 {
246 > break
247 }
248 }
249 > return &archiver.QueryVisibilityResponse{ visibility_archiver.go
250 > Executions: executions,
251 > NextPageToken: nextPageToken,
252 > }, nil
253 }
254
340 for _, item := range results.Contents {
341 if keyFilter != nil && !keyFilter(*item.Key) {
342 > continue visibility_archiver.go
343 }
344