mirror of
https://github.com/mattermost/mattermost.git
synced 2026-04-13 13:08:56 -04:00
* ci: enable fullyparallel mode for server tests Replace os.Setenv, os.Chdir, and global state mutations with parallel-safe alternatives (t.Setenv, t.Chdir, test hooks) across 37 files. Refactor GetLogRootPath and MM_INSTALL_TYPE to use package-level test hooks instead of environment variables. This enables gotestsum --fullparallel, allowing all test packages to run with maximum parallelism within each shard. Co-authored-by: Claude <claude@anthropic.com> * ci: split fullyparallel from continue-on-error in workflow template - Add new boolean input 'allow-failure' separate from 'fullyparallel' - Change continue-on-error to use allow-failure instead of fullyparallel - Update server-ci.yml to pass allow-failure: true for test coverage job - Allows independent control of parallel execution and failure tolerance Co-authored-by: Claude <claude@anthropic.com> * fix: protect TestOverrideLogRootPath with sync.Mutex for parallel tests - Replace global var TestOverrideLogRootPath with mutex-protected functions - Add SetTestOverrideLogRootPath() and getTestOverrideLogRootPath() functions - Update GetLogRootPath() to use thread-safe getter - Update all test files to use SetTestOverrideLogRootPath() with t.Cleanup() - Fixes race condition when running tests with t.Parallel() Co-authored-by: Claude <claude@anthropic.com> * fix: configure audit settings before server setup in tests - Move ExperimentalAuditSettings from UpdateConfig() to config defaults - Pass audit config via app.Config() option in SetupWithServerOptions() - Fixes audit test setup ordering to configure BEFORE server initialization - Resolves CodeRabbit's audit config timing issue in api4 tests Co-authored-by: Claude <claude@anthropic.com> * fix: implement SetTestOverrideLogRootPath mutex in logger.go The previous commit updated test callers to use SetTestOverrideLogRootPath() but didn't actually create the function in config/logger.go, causing build failures across all CI shards. This commit: - Replaces the exported var TestOverrideLogRootPath with mutex-protected unexported state (testOverrideLogRootPath + testOverrideLogRootMu) - Adds exported SetTestOverrideLogRootPath() setter - Adds unexported getTestOverrideLogRootPath() getter - Updates GetLogRootPath() to use the thread-safe getter - Fixes log_test.go callers that were missed in the previous commit Co-authored-by: Claude <claude@anthropic.com> * fix(test): use SetupConfig for access_control feature flag registration InitAccessControlPolicy() checks FeatureFlags.AttributeBasedAccessControl at route registration time during server startup. Setting the flag via UpdateConfig after Setup() is too late — routes are never registered and API calls return 404. Use SetupConfig() to pass the feature flag in the initial config before server startup, ensuring routes are properly registered. Co-authored-by: Claude <claude@anthropic.com> * fix(test): restore BurnOnRead flag state in TestRevealPost subtest The 'feature not enabled' subtest disables BurnOnRead without restoring it via t.Cleanup. Subsequent subtests inherit the disabled state, which can cause 501 errors when they expect the feature to be available. Add t.Cleanup to restore FeatureFlags.BurnOnRead = true after the subtest completes. Co-authored-by: Claude <claude@anthropic.com> * fix(test): restore EnableSharedChannelsMemberSync flag via t.Cleanup The test disables EnableSharedChannelsMemberSync without restoring it. If the subtest exits early (e.g., require failure), later sibling subtests inherit a disabled flag and become flaky. Add t.Cleanup to restore the flag after the subtest completes. Co-authored-by: Claude <claude@anthropic.com> * Fix test parallelism: use instance-scoped overrides and init-time audit config Replace package-level test globals (TestOverrideInstallType, SetTestOverrideLogRootPath) with fields on PlatformService so each test gets its own instance without process-wide mutation. Fix three audit tests (TestUserLoginAudit, TestLogoutAuditAuthStatus, TestUpdatePasswordAudit) that configured the audit logger after server init — the audit logger only reads config at startup, so pass audit settings via app.Config() at init time instead. Also revert the Go 1.24.13 downgrade and bump mattermost-govet to v2.0.2 for Go 1.25.8 compatibility. * Fix audit unit tests * Fix MMCLOUDURL unit tests * Fixed unit tests using MM_NOTIFY_ADMIN_COOL_OFF_DAYS * Make app migrations idempotent for parallel test safety Change System().Save() to System().SaveOrUpdate() in all migration completion markers. When two parallel tests share a database pool entry, both may race through the check-then-insert migration pattern. Save() causes a duplicate key fatal crash; SaveOrUpdate() makes the second write a harmless no-op. * test: address review feedback on fullyparallel PR - Use SetLogRootPathOverride() setter instead of direct field access in platform/support_packet_test.go and platform/log_test.go (pvev) - Restore TestGetLogRootPath in config/logger_test.go to keep MM_LOG_PATH env var coverage; test uses t.Setenv so it runs serially which is fine (pvev) - Fix misleading comment in config_test.go: code uses t.Setenv, not os.Setenv (jgheithcock) Co-authored-by: Claude <claude@anthropic.com> * fix: add missing os import in post_test.go The os import was dropped during a merge conflict resolution while burn-on-read shared channel tests from master still use os.Setenv. Co-authored-by: Claude <claude@anthropic.com> --------- Co-authored-by: Claude <claude@anthropic.com> Co-authored-by: wiggin77 <wiggin77@warpmail.net> Co-authored-by: Mattermost Build <build@mattermost.com>
910 lines
33 KiB
Go
910 lines
33 KiB
Go
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
|
// See LICENSE.txt for license information.
|
|
|
|
package app
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"fmt"
|
|
"os"
|
|
"reflect"
|
|
|
|
"github.com/hashicorp/go-multierror"
|
|
"github.com/mattermost/mattermost/server/public/model"
|
|
"github.com/mattermost/mattermost/server/public/shared/mlog"
|
|
"github.com/mattermost/mattermost/server/public/shared/request"
|
|
"github.com/mattermost/mattermost/server/v8/channels/store"
|
|
)
|
|
|
|
const (
|
|
EmojisPermissionsMigrationKey = "EmojisPermissionsMigrationComplete"
|
|
GuestRolesCreationMigrationKey = "GuestRolesCreationMigrationComplete"
|
|
SystemConsoleRolesCreationMigrationKey = "SystemConsoleRolesCreationMigrationComplete"
|
|
CustomGroupAdminRoleCreationMigrationKey = "CustomGroupAdminRoleCreationMigrationComplete"
|
|
SharedChannelManagerRoleCreationMigrationKey = "SystemSharedChannelManagerRoleCreationMigrationComplete"
|
|
SecureConnectionManagerRoleCreationMigrationKey = "SystemSecureConnectionManagerRoleCreationMigrationComplete"
|
|
ContentExtractionConfigDefaultTrueMigrationKey = "ContentExtractionConfigDefaultTrueMigrationComplete"
|
|
PlaybookRolesCreationMigrationKey = "PlaybookRolesCreationMigrationComplete"
|
|
FirstAdminSetupCompleteKey = model.SystemFirstAdminSetupComplete
|
|
remainingSchemaMigrationsKey = "RemainingSchemaMigrations"
|
|
postPriorityConfigDefaultTrueMigrationKey = "PostPriorityConfigDefaultTrueMigrationComplete"
|
|
contentFlaggingSetupDoneKey = "content_flagging_setup_done"
|
|
contentFlaggingMigrationVersion = "v5"
|
|
|
|
contentFlaggingPropertyNameFlaggedPostId = "flagged_post_id"
|
|
ContentFlaggingPropertyNameStatus = "status"
|
|
contentFlaggingPropertyNameReportingUserID = "reporting_user_id"
|
|
contentFlaggingPropertyNameReportingReason = "reporting_reason"
|
|
contentFlaggingPropertyNameReportingComment = "reporting_comment"
|
|
contentFlaggingPropertyNameReportingTime = "reporting_time"
|
|
contentFlaggingPropertyNameReviewerUserID = "reviewer_user_id"
|
|
contentFlaggingPropertyNameActorUserID = "actor_user_id"
|
|
contentFlaggingPropertyNameActorComment = "actor_comment"
|
|
contentFlaggingPropertyNameActionTime = "action_time"
|
|
contentFlaggingPropertyManageByContentFlagging = "content_flagging_managed"
|
|
|
|
contentFlaggingPropertySubTypeTimestamp = "timestamp"
|
|
)
|
|
|
|
// This function migrates the default built in roles from code/config to the database.
|
|
func (a *App) DoAdvancedPermissionsMigration() error {
|
|
return a.Srv().doAdvancedPermissionsMigration()
|
|
}
|
|
|
|
func (s *Server) doAdvancedPermissionsMigration() error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
if _, err := s.Store().System().GetByName(model.AdvancedPermissionsMigrationKey); err == nil {
|
|
return nil
|
|
} else if !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
mlog.Info("Migrating roles to database.")
|
|
roles := model.MakeDefaultRoles()
|
|
|
|
var multiErr *multierror.Error
|
|
for _, role := range roles {
|
|
_, err := s.Store().Role().Save(role)
|
|
if err == nil {
|
|
continue
|
|
}
|
|
mlog.Warn("Couldn't save the role for advanced permissions migration, this can be an expected case", mlog.Err(err))
|
|
|
|
// If this failed for reasons other than the role already existing, don't mark the migration as done.
|
|
fetchedRole, err := s.Store().Role().GetByName(context.Background(), role.Name)
|
|
if err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to migrate role to database: %w", err))
|
|
continue
|
|
}
|
|
|
|
// If the role already existed, check it is the same and update if not.
|
|
if !reflect.DeepEqual(fetchedRole.Permissions, role.Permissions) ||
|
|
fetchedRole.DisplayName != role.DisplayName ||
|
|
fetchedRole.Description != role.Description ||
|
|
fetchedRole.SchemeManaged != role.SchemeManaged {
|
|
role.Id = fetchedRole.Id
|
|
if _, err = s.Store().Role().Save(role); err != nil {
|
|
// Role is not the same, but failed to update.
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to migrate role to database: %w", err))
|
|
}
|
|
}
|
|
}
|
|
|
|
if multiErr != nil {
|
|
return multiErr
|
|
}
|
|
|
|
config := s.platform.Config()
|
|
*config.ServiceSettings.PostEditTimeLimit = -1
|
|
if _, _, err := s.platform.SaveConfig(config, true); err != nil {
|
|
return fmt.Errorf("failed to update config in Advanced Permissions Phase 1 Migration: %w", err)
|
|
}
|
|
|
|
system := model.System{
|
|
Name: model.AdvancedPermissionsMigrationKey,
|
|
Value: "true",
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&system); err != nil {
|
|
return fmt.Errorf("failed to mark advanced permissions migration as completed: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (a *App) SetPhase2PermissionsMigrationStatus(isComplete bool) error {
|
|
if !isComplete {
|
|
if _, err := a.Srv().Store().System().PermanentDeleteByName(model.MigrationKeyAdvancedPermissionsPhase2); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
a.Srv().phase2PermissionsMigrationComplete = isComplete
|
|
return nil
|
|
}
|
|
|
|
func (a *App) DoEmojisPermissionsMigration() error {
|
|
if err := a.Srv().doEmojisPermissionsMigration(); err != nil {
|
|
return fmt.Errorf("Failed to complete emojis permissions migration: %w", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doEmojisPermissionsMigration() error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
if _, err := s.Store().System().GetByName(EmojisPermissionsMigrationKey); err == nil {
|
|
return nil
|
|
} else if !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
var role *model.Role
|
|
var systemAdminRole *model.Role
|
|
var err *model.AppError
|
|
|
|
mlog.Info("Migrating emojis config to database.")
|
|
|
|
// Emoji creation is set to all by default
|
|
role, err = s.GetRoleByName(context.Background(), model.SystemUserRoleId)
|
|
if err != nil {
|
|
return fmt.Errorf("failed to get role for system user: %w", err)
|
|
}
|
|
|
|
if role != nil {
|
|
role.Permissions = append(role.Permissions, model.PermissionCreateEmojis.Id, model.PermissionDeleteEmojis.Id)
|
|
if _, nErr := s.Store().Role().Save(role); nErr != nil {
|
|
return fmt.Errorf("failed to save role: %w", nErr)
|
|
}
|
|
}
|
|
|
|
systemAdminRole, err = s.GetRoleByName(context.Background(), model.SystemAdminRoleId)
|
|
if err != nil {
|
|
return fmt.Errorf("failed to get role for system admin: %w", err)
|
|
}
|
|
|
|
systemAdminRole.Permissions = append(systemAdminRole.Permissions,
|
|
model.PermissionCreateEmojis.Id,
|
|
model.PermissionDeleteEmojis.Id,
|
|
model.PermissionDeleteOthersEmojis.Id,
|
|
)
|
|
if _, err := s.Store().Role().Save(systemAdminRole); err != nil {
|
|
return fmt.Errorf("failed to save role: %w", err)
|
|
}
|
|
|
|
system := model.System{
|
|
Name: EmojisPermissionsMigrationKey,
|
|
Value: "true",
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&system); err != nil {
|
|
return fmt.Errorf("failed to mark emojis permissions migration as completed: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (a *App) DoGuestRolesCreationMigration() error {
|
|
if err := a.Srv().doGuestRolesCreationMigration(); err != nil {
|
|
return fmt.Errorf("Failed to complete guest roles creation migration: %w", err)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doGuestRolesCreationMigration() error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
if _, err := s.Store().System().GetByName(GuestRolesCreationMigrationKey); err == nil {
|
|
return nil
|
|
} else if !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
roles := model.MakeDefaultRoles()
|
|
var multiErr *multierror.Error
|
|
if _, err := s.Store().Role().GetByName(context.Background(), model.ChannelGuestRoleId); err != nil {
|
|
if _, err := s.Store().Role().Save(roles[model.ChannelGuestRoleId]); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new guest role to database: %w", err))
|
|
}
|
|
}
|
|
if _, err := s.Store().Role().GetByName(context.Background(), model.TeamGuestRoleId); err != nil {
|
|
if _, err := s.Store().Role().Save(roles[model.TeamGuestRoleId]); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new guest role to database: %w", err))
|
|
}
|
|
}
|
|
if _, err := s.Store().Role().GetByName(context.Background(), model.SystemGuestRoleId); err != nil {
|
|
if _, err := s.Store().Role().Save(roles[model.SystemGuestRoleId]); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new guest role to database: %w", err))
|
|
}
|
|
}
|
|
|
|
schemes, err := s.Store().Scheme().GetAllPage("", 0, 1000000)
|
|
if err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to get all schemes: %w", err))
|
|
}
|
|
for _, scheme := range schemes {
|
|
if scheme.DefaultTeamGuestRole == "" || scheme.DefaultChannelGuestRole == "" {
|
|
if scheme.Scope == model.SchemeScopeTeam {
|
|
// Team Guest Role
|
|
teamGuestRole := &model.Role{
|
|
Name: model.NewId(),
|
|
DisplayName: fmt.Sprintf("Team Guest Role for Scheme %s", scheme.Name),
|
|
Permissions: roles[model.TeamGuestRoleId].Permissions,
|
|
SchemeManaged: true,
|
|
}
|
|
|
|
if savedRole, err := s.Store().Role().Save(teamGuestRole); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new guest role for custom scheme: %w", err))
|
|
} else {
|
|
scheme.DefaultTeamGuestRole = savedRole.Name
|
|
}
|
|
}
|
|
|
|
// Channel Guest Role
|
|
channelGuestRole := &model.Role{
|
|
Name: model.NewId(),
|
|
DisplayName: fmt.Sprintf("Channel Guest Role for Scheme %s", scheme.Name),
|
|
Permissions: roles[model.ChannelGuestRoleId].Permissions,
|
|
SchemeManaged: true,
|
|
}
|
|
|
|
if savedRole, err := s.Store().Role().Save(channelGuestRole); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new guest role for custom scheme: %w", err))
|
|
} else {
|
|
scheme.DefaultChannelGuestRole = savedRole.Name
|
|
}
|
|
|
|
_, err := s.Store().Scheme().Save(scheme)
|
|
if err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to update custom scheme: %w", err))
|
|
}
|
|
}
|
|
}
|
|
|
|
if multiErr != nil {
|
|
return multiErr
|
|
}
|
|
|
|
system := model.System{
|
|
Name: GuestRolesCreationMigrationKey,
|
|
Value: "true",
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&system); err != nil {
|
|
return fmt.Errorf("failed to mark guest roles creation migration as completed: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (a *App) DoSystemConsoleRolesCreationMigration() error {
|
|
return a.Srv().doSystemConsoleRolesCreationMigration()
|
|
}
|
|
|
|
func (s *Server) doSystemConsoleRolesCreationMigration() error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
if _, err := s.Store().System().GetByName(SystemConsoleRolesCreationMigrationKey); err == nil {
|
|
return nil
|
|
} else if !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
roles := model.MakeDefaultRoles()
|
|
var multiErr *multierror.Error
|
|
if _, err := s.Store().Role().GetByName(context.Background(), model.SystemManagerRoleId); err != nil {
|
|
if _, err := s.Store().Role().Save(roles[model.SystemManagerRoleId]); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new role %q: %w", model.SystemManagerRoleId, err))
|
|
}
|
|
}
|
|
if _, err := s.Store().Role().GetByName(context.Background(), model.SystemReadOnlyAdminRoleId); err != nil {
|
|
if _, err := s.Store().Role().Save(roles[model.SystemReadOnlyAdminRoleId]); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new role %q: %w", model.SystemReadOnlyAdminRoleId, err))
|
|
}
|
|
}
|
|
if _, err := s.Store().Role().GetByName(context.Background(), model.SystemUserManagerRoleId); err != nil {
|
|
if _, err := s.Store().Role().Save(roles[model.SystemUserManagerRoleId]); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new role %q: %w", model.SystemUserManagerRoleId, err))
|
|
}
|
|
}
|
|
|
|
if multiErr != nil {
|
|
return multiErr
|
|
}
|
|
|
|
system := model.System{
|
|
Name: SystemConsoleRolesCreationMigrationKey,
|
|
Value: "true",
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&system); err != nil {
|
|
return fmt.Errorf("failed to mark system console roles creation migration as completed: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doSingleRoleCreationMigration(migrationKey, roleId string) error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
if _, err := s.Store().System().GetByName(migrationKey); err == nil {
|
|
return nil
|
|
} else if !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
roles := model.MakeDefaultRoles()
|
|
role := roles[roleId]
|
|
if role == nil {
|
|
return fmt.Errorf("unknown role id: %q", roleId)
|
|
}
|
|
var nfRoleErr *store.ErrNotFound
|
|
if _, err := s.Store().Role().GetByName(context.Background(), roleId); err != nil {
|
|
if !errors.As(err, &nfRoleErr) {
|
|
return fmt.Errorf("could not query role %q: %w", roleId, err)
|
|
}
|
|
if _, err := s.Store().Role().Save(role); err != nil {
|
|
return fmt.Errorf("failed to create new role %q: %w", roleId, err)
|
|
}
|
|
}
|
|
|
|
system := model.System{
|
|
Name: migrationKey,
|
|
Value: "true",
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&system); err != nil {
|
|
return fmt.Errorf("failed to mark %s migration as completed: %w", migrationKey, err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doCustomGroupAdminRoleCreationMigration() error {
|
|
return s.doSingleRoleCreationMigration(CustomGroupAdminRoleCreationMigrationKey, model.SystemCustomGroupAdminRoleId)
|
|
}
|
|
|
|
func (s *Server) doSharedChannelManagerRoleCreationMigration() error {
|
|
return s.doSingleRoleCreationMigration(SharedChannelManagerRoleCreationMigrationKey, model.SharedChannelManagerRoleId)
|
|
}
|
|
|
|
func (s *Server) doSecureConnectionManagerRoleCreationMigration() error {
|
|
return s.doSingleRoleCreationMigration(SecureConnectionManagerRoleCreationMigrationKey, model.SecureConnectionManagerRoleId)
|
|
}
|
|
|
|
func (s *Server) doContentExtractionConfigDefaultTrueMigration() error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
if _, err := s.Store().System().GetByName(ContentExtractionConfigDefaultTrueMigrationKey); err == nil {
|
|
return nil
|
|
} else if !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
s.platform.UpdateConfig(func(config *model.Config) {
|
|
config.FileSettings.ExtractContent = model.NewPointer(true)
|
|
})
|
|
|
|
system := model.System{
|
|
Name: ContentExtractionConfigDefaultTrueMigrationKey,
|
|
Value: "true",
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&system); err != nil {
|
|
return fmt.Errorf("failed to mark content extraction config migration as completed: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doPlaybooksRolesCreationMigration() error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
if _, err := s.Store().System().GetByName(PlaybookRolesCreationMigrationKey); err == nil {
|
|
return nil
|
|
} else if !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
roles := model.MakeDefaultRoles()
|
|
var multiErr *multierror.Error
|
|
if _, err := s.Store().Role().GetByName(context.Background(), model.PlaybookAdminRoleId); err != nil {
|
|
if _, err := s.Store().Role().Save(roles[model.PlaybookAdminRoleId]); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new playbook %q role to database: %w", model.PlaybookAdminRoleId, err))
|
|
}
|
|
}
|
|
if _, err := s.Store().Role().GetByName(context.Background(), model.PlaybookMemberRoleId); err != nil {
|
|
if _, err := s.Store().Role().Save(roles[model.PlaybookMemberRoleId]); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new playbook %q role to database: %w", model.PlaybookMemberRoleId, err))
|
|
}
|
|
}
|
|
if _, err := s.Store().Role().GetByName(context.Background(), model.RunAdminRoleId); err != nil {
|
|
if _, err := s.Store().Role().Save(roles[model.RunAdminRoleId]); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("ffailed to create new playbook %q role to database: %w", model.RunAdminRoleId, err))
|
|
}
|
|
}
|
|
if _, err := s.Store().Role().GetByName(context.Background(), model.RunMemberRoleId); err != nil {
|
|
if _, err := s.Store().Role().Save(roles[model.RunMemberRoleId]); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new playbook %q role to database: %w", model.RunMemberRoleId, err))
|
|
}
|
|
}
|
|
schemes, err := s.Store().Scheme().GetAllPage(model.SchemeScopeTeam, 0, 1000000)
|
|
if err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to get all schemes: %w", err))
|
|
}
|
|
|
|
for _, scheme := range schemes {
|
|
if scheme.Scope == model.SchemeScopeTeam {
|
|
if scheme.DefaultPlaybookAdminRole == "" {
|
|
playbookAdminRole := &model.Role{
|
|
Name: model.NewId(),
|
|
DisplayName: fmt.Sprintf("Playbook Admin Role for Scheme %s", scheme.Name),
|
|
Permissions: roles[model.PlaybookAdminRoleId].Permissions,
|
|
SchemeManaged: true,
|
|
}
|
|
|
|
if savedRole, err := s.Store().Role().Save(playbookAdminRole); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new playbook %q role for existing custom scheme: %w", model.PlaybookAdminRoleId, err))
|
|
} else {
|
|
scheme.DefaultPlaybookAdminRole = savedRole.Name
|
|
}
|
|
}
|
|
if scheme.DefaultPlaybookMemberRole == "" {
|
|
playbookMember := &model.Role{
|
|
Name: model.NewId(),
|
|
DisplayName: fmt.Sprintf("Playbook Member Role for Scheme %s", scheme.Name),
|
|
Permissions: roles[model.PlaybookMemberRoleId].Permissions,
|
|
SchemeManaged: true,
|
|
}
|
|
|
|
if savedRole, err := s.Store().Role().Save(playbookMember); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new playbook %q role for existing custom scheme: %w", model.PlaybookMemberRoleId, err))
|
|
} else {
|
|
scheme.DefaultPlaybookMemberRole = savedRole.Name
|
|
}
|
|
}
|
|
|
|
if scheme.DefaultRunAdminRole == "" {
|
|
runAdminRole := &model.Role{
|
|
Name: model.NewId(),
|
|
DisplayName: fmt.Sprintf("Run Admin Role for Scheme %s", scheme.Name),
|
|
Permissions: roles[model.RunAdminRoleId].Permissions,
|
|
SchemeManaged: true,
|
|
}
|
|
|
|
if savedRole, err := s.Store().Role().Save(runAdminRole); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new playbook %q role for existing custom scheme: %w", model.RunAdminRoleId, err))
|
|
} else {
|
|
scheme.DefaultRunAdminRole = savedRole.Name
|
|
}
|
|
}
|
|
|
|
if scheme.DefaultRunMemberRole == "" {
|
|
runMemberRole := &model.Role{
|
|
Name: model.NewId(),
|
|
DisplayName: fmt.Sprintf("Run Member Role for Scheme %s", scheme.Name),
|
|
Permissions: roles[model.RunMemberRoleId].Permissions,
|
|
SchemeManaged: true,
|
|
}
|
|
|
|
if savedRole, err := s.Store().Role().Save(runMemberRole); err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to create new playbook %q role for existing custom scheme: %w", model.RunMemberRoleId, err))
|
|
} else {
|
|
scheme.DefaultRunMemberRole = savedRole.Name
|
|
}
|
|
}
|
|
_, err := s.Store().Scheme().Save(scheme)
|
|
if err != nil {
|
|
multiErr = multierror.Append(multiErr, fmt.Errorf("failed to update custom scheme: %w", err))
|
|
}
|
|
}
|
|
}
|
|
|
|
if multiErr != nil {
|
|
return multiErr
|
|
}
|
|
|
|
system := model.System{
|
|
Name: PlaybookRolesCreationMigrationKey,
|
|
Value: "true",
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&system); err != nil {
|
|
return fmt.Errorf("failed to mark playbook roles creation migration as completed: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doFirstAdminSetupCompleteMigration() error {
|
|
// arbitrary choice, though if there is an longstanding installation with less than 10 messages,
|
|
// putting the first admin through onboarding shouldn't be very disruptive.
|
|
const existingInstallationPostsThreshold = 10
|
|
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
if _, err := s.Store().System().GetByName(FirstAdminSetupCompleteKey); err == nil {
|
|
return nil
|
|
} else if !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
teams, err := s.Store().Team().GetAll()
|
|
if err != nil {
|
|
// can not confirm that admin has started in this case.
|
|
return fmt.Errorf("could not get teams: %w", err)
|
|
}
|
|
|
|
if len(teams) == 0 {
|
|
// No teams, and no existing preference. This is most likely a new instance.
|
|
// So do not mark that the admin has already done the first time setup.
|
|
return nil
|
|
}
|
|
|
|
// if there are teams, then if this isn't a new installation, there should be posts
|
|
postCount, err := s.Store().Post().AnalyticsPostCount(&model.PostCountOptions{})
|
|
if err != nil {
|
|
return fmt.Errorf("could not get posts count from the database: %w", err)
|
|
} else if postCount < existingInstallationPostsThreshold {
|
|
mlog.Info("Post count is lower than expected, aborting migration",
|
|
mlog.Int("expected", int(existingInstallationPostsThreshold)),
|
|
mlog.Int("actual", int(postCount)))
|
|
return nil
|
|
}
|
|
|
|
system := model.System{
|
|
Name: FirstAdminSetupCompleteKey,
|
|
Value: "true",
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&system); err != nil {
|
|
return fmt.Errorf("failed to mark first admin setup migration as completed: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doRemainingSchemaMigrations() error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
if _, err := s.Store().System().GetByName(remainingSchemaMigrationsKey); err == nil {
|
|
return nil
|
|
} else if !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
if teams, err := s.Store().Team().GetByEmptyInviteID(); err != nil {
|
|
mlog.Error("Error fetching Teams without InviteID", mlog.Err(err))
|
|
} else {
|
|
for _, team := range teams {
|
|
team.InviteId = model.NewId()
|
|
if _, err := s.Store().Team().Update(team); err != nil {
|
|
return fmt.Errorf("error updating Team InviteIDs %q: %w", team.Id, err)
|
|
}
|
|
}
|
|
}
|
|
|
|
system := model.System{
|
|
Name: remainingSchemaMigrationsKey,
|
|
Value: "true",
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&system); err != nil {
|
|
return fmt.Errorf("failed to mark the remaining schema migrations as completed: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doPostPriorityConfigDefaultTrueMigration() error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
if _, err := s.Store().System().GetByName(postPriorityConfigDefaultTrueMigrationKey); err == nil {
|
|
return nil
|
|
} else if !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
s.platform.UpdateConfig(func(config *model.Config) {
|
|
config.ServiceSettings.PostPriority = model.NewPointer(true)
|
|
})
|
|
|
|
system := model.System{
|
|
Name: postPriorityConfigDefaultTrueMigrationKey,
|
|
Value: "true",
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&system); err != nil {
|
|
return fmt.Errorf("failed to mark post priority config migration as completed: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doSetupContentFlaggingProperties() error {
|
|
// This migration is designed in a way to allow adding more properties in the future.
|
|
// When a new property needs to be added, add it to the expectedPropertiesMap map and
|
|
// update the contentFlaggingMigrationVersion to a new value.
|
|
|
|
// If the migration is already marked as completed, don't do it again.
|
|
var nfErr *store.ErrNotFound
|
|
data, err := s.Store().System().GetByName(contentFlaggingSetupDoneKey)
|
|
if err != nil && !errors.As(err, &nfErr) {
|
|
return fmt.Errorf("could not query migration: %w", err)
|
|
}
|
|
|
|
if data != nil && data.Value == contentFlaggingMigrationVersion {
|
|
return nil
|
|
}
|
|
|
|
// RegisterPropertyGroup is idempotent, so no need to check if group is already registered
|
|
group, err := s.propertyService.RegisterPropertyGroup(model.ContentFlaggingGroupName)
|
|
if err != nil {
|
|
return fmt.Errorf("failed to register Content Flagging group: %w", err)
|
|
}
|
|
|
|
// Using page size of 100 and not iterating through all pages because the
|
|
// number of fields are static and defined here and not expected to be more than 100 for now.
|
|
existingProperties, err := s.propertyService.SearchPropertyFields(nil, group.ID, model.PropertyFieldSearchOpts{PerPage: 100})
|
|
if err != nil {
|
|
return fmt.Errorf("failed to search for existing content flagging properties: %w", err)
|
|
}
|
|
|
|
existingPropertiesMap := map[string]*model.PropertyField{}
|
|
for _, property := range existingProperties {
|
|
existingPropertiesMap[property.Name] = property
|
|
}
|
|
|
|
expectedPropertiesMap := map[string]*model.PropertyField{
|
|
contentFlaggingPropertyNameFlaggedPostId: {
|
|
GroupID: group.ID,
|
|
Name: contentFlaggingPropertyNameFlaggedPostId,
|
|
Type: model.PropertyFieldTypeText,
|
|
},
|
|
ContentFlaggingPropertyNameStatus: {
|
|
GroupID: group.ID,
|
|
Name: ContentFlaggingPropertyNameStatus,
|
|
Type: model.PropertyFieldTypeSelect,
|
|
Attrs: map[string]any{
|
|
"options": []map[string]string{
|
|
{"name": model.ContentFlaggingStatusPending, "color": "light_grey"},
|
|
{"name": model.ContentFlaggingStatusAssigned, "color": "dark_blue"},
|
|
{"name": model.ContentFlaggingStatusRemoved, "color": "dark_red"},
|
|
{"name": model.ContentFlaggingStatusRetained, "color": "light_blue"},
|
|
},
|
|
},
|
|
},
|
|
contentFlaggingPropertyNameReportingUserID: {
|
|
GroupID: group.ID,
|
|
Name: contentFlaggingPropertyNameReportingUserID,
|
|
Type: model.PropertyFieldTypeUser,
|
|
},
|
|
contentFlaggingPropertyNameReportingReason: {
|
|
GroupID: group.ID,
|
|
Name: contentFlaggingPropertyNameReportingReason,
|
|
Type: model.PropertyFieldTypeSelect,
|
|
},
|
|
contentFlaggingPropertyNameReportingComment: {
|
|
GroupID: group.ID,
|
|
Name: contentFlaggingPropertyNameReportingComment,
|
|
Type: model.PropertyFieldTypeText,
|
|
},
|
|
contentFlaggingPropertyNameReportingTime: {
|
|
GroupID: group.ID,
|
|
Name: contentFlaggingPropertyNameReportingTime,
|
|
Type: model.PropertyFieldTypeText,
|
|
Attrs: map[string]any{"subType": contentFlaggingPropertySubTypeTimestamp},
|
|
},
|
|
contentFlaggingPropertyNameReviewerUserID: {
|
|
GroupID: group.ID,
|
|
Name: contentFlaggingPropertyNameReviewerUserID,
|
|
Type: model.PropertyFieldTypeUser,
|
|
Attrs: map[string]any{"editable": true},
|
|
},
|
|
contentFlaggingPropertyNameActorUserID: {
|
|
GroupID: group.ID,
|
|
Name: contentFlaggingPropertyNameActorUserID,
|
|
Type: model.PropertyFieldTypeUser,
|
|
},
|
|
contentFlaggingPropertyNameActorComment: {
|
|
GroupID: group.ID,
|
|
Name: contentFlaggingPropertyNameActorComment,
|
|
Type: model.PropertyFieldTypeText,
|
|
},
|
|
contentFlaggingPropertyNameActionTime: {
|
|
GroupID: group.ID,
|
|
Name: contentFlaggingPropertyNameActionTime,
|
|
Type: model.PropertyFieldTypeText,
|
|
Attrs: map[string]any{"subType": contentFlaggingPropertySubTypeTimestamp},
|
|
},
|
|
contentFlaggingPropertyManageByContentFlagging: {
|
|
GroupID: group.ID,
|
|
Name: contentFlaggingPropertyManageByContentFlagging,
|
|
Type: model.PropertyFieldTypeText,
|
|
},
|
|
}
|
|
|
|
var propertiesToUpdate []*model.PropertyField
|
|
var propertiesToCreate []*model.PropertyField
|
|
|
|
for name, expectedProperty := range expectedPropertiesMap {
|
|
if _, exists := existingPropertiesMap[name]; exists {
|
|
property := existingPropertiesMap[name]
|
|
property.Type = expectedProperty.Type
|
|
property.Attrs = expectedProperty.Attrs
|
|
propertiesToUpdate = append(propertiesToUpdate, property)
|
|
} else {
|
|
propertiesToCreate = append(propertiesToCreate, expectedProperty)
|
|
}
|
|
}
|
|
|
|
for _, property := range propertiesToCreate {
|
|
if _, err := s.propertyService.CreatePropertyField(nil, property); err != nil {
|
|
return fmt.Errorf("failed to create content flagging property: %q, error: %w", property.Name, err)
|
|
}
|
|
}
|
|
|
|
if len(propertiesToUpdate) > 0 {
|
|
if _, err := s.propertyService.UpdatePropertyFields(nil, group.ID, propertiesToUpdate); err != nil {
|
|
return fmt.Errorf("failed to update content flagging property fields: %w", err)
|
|
}
|
|
}
|
|
|
|
if err := s.Store().System().SaveOrUpdate(&model.System{Name: contentFlaggingSetupDoneKey, Value: contentFlaggingMigrationVersion}); err != nil {
|
|
return fmt.Errorf("failed to save content flagging setup done flag in system store %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doCloudS3PathMigrations(rctx request.CTX) error {
|
|
// This migration is only applicable for cloud environments
|
|
if os.Getenv("MM_CLOUD_FILESTORE_BIFROST") == "" {
|
|
return nil
|
|
}
|
|
|
|
// If the migration is already marked as completed, don't do it again.
|
|
if _, err := s.Store().System().GetByName(model.MigrationKeyS3Path); err == nil {
|
|
return nil
|
|
}
|
|
|
|
// If there is a job already pending, no need to schedule again.
|
|
// This is possible if the pod was rolled over.
|
|
jobs, err := s.Store().Job().GetAllByTypeAndStatus(rctx, model.JobTypeS3PathMigration, model.JobStatusPending)
|
|
if err != nil {
|
|
return fmt.Errorf("failed to get jobs by type and status: %w", err)
|
|
}
|
|
if len(jobs) > 0 {
|
|
return nil
|
|
}
|
|
|
|
if _, appErr := s.Jobs.CreateJobOnce(rctx, model.JobTypeS3PathMigration, nil); appErr != nil {
|
|
return fmt.Errorf("failed to start job for migrating s3 file paths: %w", appErr)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doDeleteEmptyDraftsMigration(rctx request.CTX) error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
if _, err := s.Store().System().GetByName(model.MigrationKeyDeleteEmptyDrafts); err == nil {
|
|
return nil
|
|
}
|
|
|
|
jobs, err := s.Store().Job().GetAllByTypeAndStatus(rctx, model.JobTypeDeleteEmptyDraftsMigration, model.JobStatusPending)
|
|
if err != nil {
|
|
return fmt.Errorf("failed to get jobs by type and status: %w", err)
|
|
}
|
|
if len(jobs) > 0 {
|
|
return nil
|
|
}
|
|
|
|
if _, appErr := s.Jobs.CreateJobOnce(rctx, model.JobTypeDeleteEmptyDraftsMigration, nil); appErr != nil {
|
|
return fmt.Errorf("failed to start job for deleting empty drafts: %w", appErr)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doDeleteOrphanDraftsMigration(rctx request.CTX) error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
if _, err := s.Store().System().GetByName(model.MigrationKeyDeleteOrphanDrafts); err == nil {
|
|
return nil
|
|
}
|
|
|
|
jobs, err := s.Store().Job().GetAllByTypeAndStatus(rctx, model.JobTypeDeleteOrphanDraftsMigration, model.JobStatusPending)
|
|
if err != nil {
|
|
return fmt.Errorf("failed to get jobs by type and status: %w", err)
|
|
}
|
|
if len(jobs) > 0 {
|
|
return nil
|
|
}
|
|
|
|
if _, appErr := s.Jobs.CreateJobOnce(rctx, model.JobTypeDeleteOrphanDraftsMigration, nil); appErr != nil {
|
|
return fmt.Errorf("failed to start job for deleting orphan drafts: %w", appErr)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (s *Server) doDeleteDmsPreferencesMigration(rctx request.CTX) error {
|
|
// If the migration is already marked as completed, don't do it again.
|
|
if _, err := s.Store().System().GetByName(model.MigrationKeyDeleteDmsPreferences); err == nil {
|
|
return nil
|
|
}
|
|
|
|
jobs, err := s.Store().Job().GetAllByTypeAndStatus(rctx, model.JobTypeDeleteDmsPreferencesMigration, model.JobStatusPending)
|
|
if err != nil {
|
|
return fmt.Errorf("failed to get jobs by type and status: %w", err)
|
|
}
|
|
if len(jobs) > 0 {
|
|
return nil
|
|
}
|
|
|
|
if _, appErr := s.Jobs.CreateJobOnce(rctx, model.JobTypeDeleteDmsPreferencesMigration, nil); appErr != nil {
|
|
return fmt.Errorf("failed to start job for deleting dm preferences: %w", appErr)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (a *App) DoAppMigrations() {
|
|
a.Srv().doAppMigrations()
|
|
}
|
|
|
|
func (s *Server) doAppMigrations() {
|
|
type migration struct {
|
|
name string
|
|
handler func() error
|
|
}
|
|
m1 := []migration{
|
|
{"Advanced Permissions Migration", s.doAdvancedPermissionsMigration},
|
|
{"Emojis Permissions Migration", s.doEmojisPermissionsMigration},
|
|
{"GuestRolesCreationMigration", s.doGuestRolesCreationMigration},
|
|
{"System Console Roles Creation Migration", s.doSystemConsoleRolesCreationMigration},
|
|
{"Custom Group Admin Role Creation Migration", s.doCustomGroupAdminRoleCreationMigration},
|
|
{"Shared Channel Manager Role Creation Migration", s.doSharedChannelManagerRoleCreationMigration},
|
|
{"Secure Connection Manager Role Creation Migration", s.doSecureConnectionManagerRoleCreationMigration},
|
|
// This migration always run after dependent migrations such as the guest roles migration.
|
|
{"Permissions Migrations", s.doPermissionsMigrations},
|
|
{"Content Extraction Config Default True Migration", s.doContentExtractionConfigDefaultTrueMigration},
|
|
{"Playbooks Roles Creation Migration", s.doPlaybooksRolesCreationMigration},
|
|
{"First Admin Setup Complete Migration", s.doFirstAdminSetupCompleteMigration},
|
|
{"Remaining Schema Migrations", s.doRemainingSchemaMigrations},
|
|
{"Post Priority Config Default True Migration", s.doPostPriorityConfigDefaultTrueMigration},
|
|
{"Content Flagging Properties Setup", s.doSetupContentFlaggingProperties},
|
|
}
|
|
|
|
for i := range m1 {
|
|
err := m1[i].handler()
|
|
if err != nil {
|
|
mlog.Fatal("Failed to run app migration",
|
|
mlog.String("migration", m1[i].name),
|
|
mlog.Err(err),
|
|
)
|
|
}
|
|
}
|
|
|
|
type migrationContext struct {
|
|
name string
|
|
handler func(request.CTX) error
|
|
}
|
|
m2 := []migrationContext{
|
|
{"Encode S3 Image Paths Migration", s.doCloudS3PathMigrations},
|
|
{"Delete Empty Drafts Migration", s.doDeleteEmptyDraftsMigration},
|
|
{"Delete Orphan Drafts Migration", s.doDeleteOrphanDraftsMigration},
|
|
{"Delete Invalid Dms Preferences Migration", s.doDeleteDmsPreferencesMigration},
|
|
}
|
|
|
|
rctx := request.EmptyContext(s.Log())
|
|
for i := range m2 {
|
|
err := m2[i].handler(rctx)
|
|
if err != nil {
|
|
mlog.Fatal("Failed to run app migration",
|
|
mlog.String("migration", m2[i].name),
|
|
mlog.Err(err),
|
|
)
|
|
}
|
|
}
|
|
}
|