MM-60441: Re-index public channels when a user joins a team (#33400)
* Index all public channels when a user joins a team
* Precompute team members for indexChannelsForTeam
* Refactor RequestContextWithMaster to store package
This way, we can import it from both the sqlstore and the searchlayer
packages. The alternative for this is duplicating the code in those two
packages, but that will *not* work:
The context package expects custom types for the keys stored in it, so
that different packages never clash with each other when trying to
register a new key. See the docs for the WithValue function:
https://pkg.go.dev/context#WithValue
If we try to duplicate the storeContextKey type in both the sqlstore and
searchlayer packages, although they *look* the same, they are not, and
HasMaster will fail to get the value of the storeContextKey(useMaster)
key if it's from the other package.
* Use master in call to GetTeamMembersForChannel
In GetTeamMembersForChannel, use the DB from the newly passed context,
which will be the receiving context everywhere except in the call done
from indexChannelsForTeam, to avoid the read after write issue when
saving a team member.
* Fix GetPublicChannelsForTeam paging
We were using the page and perPage arguments as is in the call to
GetPublicChannelsForTeam, but that function expects and offset and a
limit as understood by SQL. Although perPage and limit are
interchangeable, offset is not equal to page, but to page * perPage.
* Add a synchronous bulk indexer for Opensearch
* Implement Opensearch's SyncBulkIndexChannels
* Add a synchronous bulk indexer for Elasticsearch
* Implement Elasticsearch's SynkBulkIndexChannels
* Test SyncBulkIndexChannels
* make mocks
* Bulk index channels on indexChannelsForTeam
* Handle error from SyncBulkIndexChannels
* Fix style
* Revert indexChannelWithTeamMembers refactor
* Remove defensive code on sync bulk processor
* Revert "Add a synchronous bulk indexer for Opensearch"
This reverts commit bfe4671d96bffa9ca27ed3c655fc5527b72bbafb.
* Revert "Add a synchronous bulk indexer for Elasticsearch"
This reverts commit 6643ae3f30c461544d0861aec7dee1f24e507c37.
* Refactor bulk indexers with a common interface
* Test all the different implementations
Assisted by Claude
* Remove debug statements
* Refactor common code into _stop
* Rename getUserIDsFor{,Private}Channel
* Wrap error
* Make perPage a const
* Fix typos
* Call GetTeamsForUser only if needed
* Differentiate errors for sync/async processors
---------
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-08-25 13:28:19 -04:00
|
|
|
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
|
|
|
|
// See LICENSE.txt for license information.
|
|
|
|
|
|
|
|
|
|
package store
|
|
|
|
|
|
|
|
|
|
import (
|
|
|
|
|
"context"
|
|
|
|
|
|
|
|
|
|
"github.com/mattermost/mattermost/server/public/shared/request"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
// storeContextKey is the base type for all context keys for the store.
|
|
|
|
|
type storeContextKey string
|
|
|
|
|
|
|
|
|
|
// contextValue is a type to hold some pre-determined context values.
|
|
|
|
|
type contextValue string
|
|
|
|
|
|
|
|
|
|
// Different possible values of contextValue.
|
|
|
|
|
const (
|
|
|
|
|
useMaster contextValue = "useMaster"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
// WithMaster adds the context value that master DB should be selected for this request.
|
|
|
|
|
//
|
|
|
|
|
// Deprecated: This method is deprecated and there's ongoing change to use `request.CTX` across
|
|
|
|
|
// instead of `context.Context`. Please use `RequestContextWithMaster` instead.
|
|
|
|
|
func WithMaster(ctx context.Context) context.Context {
|
|
|
|
|
return context.WithValue(ctx, storeContextKey(useMaster), true)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// RequestContextWithMaster adds the context value that master DB should be selected for this request.
|
2025-09-10 09:11:32 -04:00
|
|
|
func RequestContextWithMaster(rctx request.CTX) request.CTX {
|
|
|
|
|
ctx := WithMaster(rctx.Context())
|
|
|
|
|
rctx = rctx.WithContext(ctx)
|
|
|
|
|
return rctx
|
MM-60441: Re-index public channels when a user joins a team (#33400)
* Index all public channels when a user joins a team
* Precompute team members for indexChannelsForTeam
* Refactor RequestContextWithMaster to store package
This way, we can import it from both the sqlstore and the searchlayer
packages. The alternative for this is duplicating the code in those two
packages, but that will *not* work:
The context package expects custom types for the keys stored in it, so
that different packages never clash with each other when trying to
register a new key. See the docs for the WithValue function:
https://pkg.go.dev/context#WithValue
If we try to duplicate the storeContextKey type in both the sqlstore and
searchlayer packages, although they *look* the same, they are not, and
HasMaster will fail to get the value of the storeContextKey(useMaster)
key if it's from the other package.
* Use master in call to GetTeamMembersForChannel
In GetTeamMembersForChannel, use the DB from the newly passed context,
which will be the receiving context everywhere except in the call done
from indexChannelsForTeam, to avoid the read after write issue when
saving a team member.
* Fix GetPublicChannelsForTeam paging
We were using the page and perPage arguments as is in the call to
GetPublicChannelsForTeam, but that function expects and offset and a
limit as understood by SQL. Although perPage and limit are
interchangeable, offset is not equal to page, but to page * perPage.
* Add a synchronous bulk indexer for Opensearch
* Implement Opensearch's SyncBulkIndexChannels
* Add a synchronous bulk indexer for Elasticsearch
* Implement Elasticsearch's SynkBulkIndexChannels
* Test SyncBulkIndexChannels
* make mocks
* Bulk index channels on indexChannelsForTeam
* Handle error from SyncBulkIndexChannels
* Fix style
* Revert indexChannelWithTeamMembers refactor
* Remove defensive code on sync bulk processor
* Revert "Add a synchronous bulk indexer for Opensearch"
This reverts commit bfe4671d96bffa9ca27ed3c655fc5527b72bbafb.
* Revert "Add a synchronous bulk indexer for Elasticsearch"
This reverts commit 6643ae3f30c461544d0861aec7dee1f24e507c37.
* Refactor bulk indexers with a common interface
* Test all the different implementations
Assisted by Claude
* Remove debug statements
* Refactor common code into _stop
* Rename getUserIDsFor{,Private}Channel
* Wrap error
* Make perPage a const
* Fix typos
* Call GetTeamsForUser only if needed
* Differentiate errors for sync/async processors
---------
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-08-25 13:28:19 -04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// HasMaster is a helper function to check whether master DB should be selected or not.
|
|
|
|
|
func HasMaster(ctx context.Context) bool {
|
|
|
|
|
if v := ctx.Value(storeContextKey(useMaster)); v != nil {
|
|
|
|
|
if res, ok := v.(bool); ok && res {
|
|
|
|
|
return true
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
return false
|
|
|
|
|
}
|