58 commits

Author	SHA1	Message	Date
Jesse Hallam	eb8310a30c	simplify CODEOWNERS (#35770) ... * simplify CODEOWNERS * dont .gitignore AGENTS.md * AGENTS.md to document previous CODEOWNERS responsibilities * update from https://developers.mattermost.com/contribute/more-info/server/schema-migration-guide/ * CREATE INDEX CONCURRENTLY now vetted * rewrite and move to README.md * dont limit to 80 chars * rewrite webapp AGENTS.md and add to README.md --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2026-04-01 13:03:36 +00:00
Miguel de la Cruz	48f2fd0873	Merge the Integrated Boards MVP feature branch (#35796) ... * Add CreatedBy and UpdatedBy to the properties fields and values (#34485) * Add CreatedBy and UpdatedBy to the properties fields and values * Fix types --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds ObjectType to the property fields table (#34908) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Update ObjectType migration setting an empty value and marking the column as not null (#34915) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds uniqueness mechanisms to the property fields (#35058) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Fixing retrylayer mocks * Remove retrylayer duplication * Address review comments * Fix comment to avoid linter issues * Address PR comments * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.down.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update field validation to check only for valid target types * Update migrations to avoid concurrent index creation within a transaction * Update migrations to make all index ops concurrent * Update tests to use valid PSAv2 property fields * Adds a helper for valid PSAv2 TargetTypes --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Fix property tests (#35388) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards feature flag (#35378) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards MVP API changes (#34822) This PR includes the necessary changes for channels and posts endpoints and adds a set of generic endpoints to retrieve and manage property fields and values following the new Property System approach. Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> * Property System Architecture permissions for v2 (#35113) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Fix i18n sorting --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Add Views store and app layer (#35361) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor test loops in ViewStore tests for improved readability * change pagination to limit/offset * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * MM-67388, MM-66528, MM-67750: Add View REST API endpoints, websocket events, and sort order (#35442) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Add View API endpoints with OpenAPI spec, client methods, and i18n Implement REST API for channel views (board-type) behind the IntegratedBoards feature flag. Adds CRUD endpoints under /api/v4/channels/{channel_id}/views with permission checks matching the channel bookmark pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add cursor-based pagination to View API for channel views * Enhance cursor handling in getViewsForChannel and update tests for pagination * Refactor test loops in ViewStore tests for improved readability * Refactor loop in TestGetViewsForChannel for improved readability * change pagination to limit/offset * switch to limit/offset pagination * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add view sort order API endpoint Add POST /api/v4/channels/{channel_id}/views/{view_id}/sort_order endpoint following the channel bookmarks reorder pattern. Includes store, app, and API layers with full test coverage at each layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add connectionId to view WebSocket events and sort_order API spec Thread connectionId from request header through all view handlers (create, update, delete, sort_order) to WebSocket events, matching the channel bookmarks pattern. Add sort_order endpoint to OpenAPI spec. Update minimum server version to 11.6. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove duplicate View/ViewPatch definitions from definitions.yaml The merge from integrated-boards-mvp introduced duplicate View and ViewPatch schema definitions that were already defined earlier in the file with more detail (including ViewBoardProps ref and enums). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update minimum server version to 11.6 in views API spec Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add missing translations for view sort order error messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Merge integrated-boards-mvp into ibmvp_api-views; remove spec files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix flaky TestViewStore timestamp test on CI Add sleep before UpdateSortOrder to ensure timestamps differ, preventing same-millisecond comparisons on fast CI machines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove duplicate views.yaml imclude * Use c.boolString() for include_deleted query param in GetViewsForChannel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix views.yaml sort order schema: use integer type and require body Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor view sort order tests to use named IDs instead of array indices Extract idA/idB/idC from views slice and add BEFORE/AFTER comments to make stateful subtest ordering easier to follow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Return 404 instead of 403 for view operations on deleted channels Deleted channels should appear non-existent to callers rather than revealing their existence via a 403. Detailed error text explains the context for debugging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * add missing channel deleteat checks * Use c.Params.Page instead of manual page query param parsing in getViewsForChannel c.Params already validates and defaults page/per_page, so the manual parsing was redundant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add support for total count in views retrieval * Add tests for handling deleted views in GetViewsForChannel and GetView * Short-circuit negative newIndex in UpdateSortOrder before opening transaction Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add per-channel limit on views to bound UpdateSortOrder cost Without a cap, unbounded view creation makes sort-order updates increasingly expensive (CASE WHEN per view, row locks). Adds MaxViewsPerChannel=50 constant and enforces it in the app layer before saving. Includes API and app layer tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove include_deleted support from views API Soft-deleted views are structural metadata with low risk, but no other similar endpoint (e.g. channel bookmarks) exposes deleted records without an admin gate. Rather than adding an admin-only permission check for consistency, remove the feature entirely since there is no current use case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update view permissions to require `create_post` instead of channel management permissions * Remove obsolete view management error messages for direct and group messages --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(migrations): add user tracking and object type to property fields - Introduced user tracking columns (CreatedBy, UpdatedBy) to PropertyFields and PropertyValues. - Added ObjectType column to PropertyFields with associated unique indexes for legacy and typed properties. - Created new migration scripts for adding and dropping these features, including necessary indexes for data integrity. - Established views for managing property fields with new attributes. This update enhances the schema to support better tracking and categorization of property fields. * Add Property System Architecture v2 API endpoints (#35583) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Require target_type filter when searching property fields * Add objectType validation as part of field.IsValid() * Fix linter * Fix test with bad objecttpye * Fix test grouping --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * MM-67968: Flatten view model — remove icon, subviews, typed board props (#35726) * feat(views): flatten view model by removing icon, subview, and board props Simplifies the View data model as part of MM-67968: removes Icon, Subview, and ViewBoardProps types; renames ViewTypeBoard to ViewTypeKanban; replaces typed Props with StringInterface (map[string]any); adds migration 000167 to drop the Icon column from the Views table. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(api): update views OpenAPI spec to reflect flattened model Removes ViewBoardProps, Subview, and icon from the View and ViewPatch schemas. Changes type enum from board to kanban. Replaces typed props with a free-form StringInterface object. Aligns with MM-67968. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * refactor(views): simplify store by dropping dbView and marshalViewProps StringInterface already implements driver.Valuer and sql.Scanner, so the manual JSON marshal/unmarshal and the dbView intermediate struct were redundant. model.View now scans directly from the database. Also removes the dead ViewMaxLinkedProperties constant and wraps the Commit() error in UpdateSortOrder. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(api): allow arbitrary JSON in view props OpenAPI schema The props field was restricted to string values via additionalProperties: { type: string }, conflicting with the Go model's StringInterface (map[string]any). Changed to additionalProperties: true in View, ViewPatch, and inline POST schemas. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Adds basic implementation of the generic redux store for PSAv2 (#35512) * Adds basic implementation of the generic redux store for PSAv2 * Add created_by and updated_by to the test fixtures * Make target_id, target_type and object_type mandatory * Wrap getPropertyFieldsByIds and getPropertyValuesForTargetByFieldIds with createSelector * Address PR comments --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds websocket messages for the PSAv2 API events (#35696) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Adds websocket messages for the PSAv2 API events * Add IsPSAv2 helper to the property field for clarity * Add guard against nil returns on field deletion * Add docs to the websocket endpoints --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * migrations: consolidate views migrations and reorder after master - Merged 000165 (create Views) with 000167 (drop Icon) since Icon was never needed - Renumbered branch migrations 159-166 → 160-167 so master's 000159 (deduplicate_policy_names) runs first - Regenerated migrations.list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add API endpoint to retrieve posts for a specific view (#35604) Automatic Merge * Apply fixes after merge * Return a more specific error from getting multiple fields * Prevent getting broadcast params on field deletion if not needed * Remove duplicated migration code * Update property conflict code to always use master * Adds nil guard when iterating on property fields * Check that permission level is valid before getting rejected by the database * Validate correctness on TargetID for PSAv2 fields * Avoid PSAv1 using permissions or protected * Fix test data after validation change * Fix flaky search test * Adds more posts for filter use cases to properly test exclusions --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> Co-authored-by: Julien Tant <julien@craftyx.fr> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-27 10:36:35 +01:00
Pablo Vélez	95e33dbc72	MM-63848: Enforce unique names for parent access control policies (#35676) ... * MM-63848: Enforce unique names for parent access control policies * Revert accidental package-lock.json change Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix translation file * Fix lint shadows, migration overflow, and unique constraint fallback * Remove pre-check, combine migrations, fix overflow * Combine migrations using regular CREATE INDEX * add missing translation * kip revision bump on policy name-only changes and add test to cover this scenario * MM-63848: Fix tx commit on name-only update, unique test names Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Mattermost Build <build@mattermost.com>	2026-03-25 05:12:13 -05:00
David Krauser	7425c6817b	[MM-67741] Scope role_updated WS events to affected team/channel (#35497) ... With this change, we now scope role_updated websocket events to users that need to receive them. Built-in and unowned role broadcast globally, team-scheme roles emit one event per team using the role, channel-scheme roles emit one event per channel using the role. To efficiently find a role's owning scheme, a schemeid column is added to the roles table. The ID is set when the scheme creates its related roles.	2026-03-16 14:36:55 -04:00
Ben Cooke	76b3528c2b	[MM-67231] Etag fixes for autotranslations (#35196)	2026-02-09 18:32:26 -05:00
Ben Cooke	9ac02ecfdd	Update translation primary key to include objectType (#35040)	2026-02-05 15:00:08 -05:00
Daniel Espino García	1273632d1a	Add endpoint to update channel member autotranslations (#35072) ... * Add endpoint to update channel member autotranslations * Add several improvements and remove unneeded functions * Add user id to audit record * Ensure autotranslation is defined * Update texts * Fix merge * Add new column for channel member autotranslations (#35111) * Minor renamings --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Ben Cooke <benkcooke@gmail.com>	2026-02-05 13:43:50 +01:00
Ben Cooke	a1c85007e1	Autotranslations MVP (#34696) ... --------- Co-authored-by: Elias Nahum <nahumhbl@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Nick Misasi <nick.misasi@mattermost.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Matthew Birtch <mattbirtch@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>	2026-01-26 17:05:34 -05:00
Nick Misasi	8e4cadbc88	[MM-66359] Recaps MVP (#34337) ... * initial commit for POC of Plugin Bridge * Updates * POC for plugin bridge * Updates from collaboration * Fixes * Refactor Plugin Bridge to use HTTP/REST instead of RPC - Remove ExecuteBridgeCall hook and Context.SourcePluginId - Implement HTTP-based bridge using existing PluginHTTP infrastructure - Add CallPlugin API method with endpoint parameter instead of method name - Update CallPluginBridge to construct HTTP POST requests - Add proper headers: Mattermost-User-Id, Mattermost-Plugin-ID - Use 'com.mattermost.server' as plugin ID for core server calls - Update ai.go to use REST endpoint /inter-plugin/v1/completion - Add comprehensive spec documentation in server/spec.md - Add MIGRATION_GUIDE.md for plugin developers - Fix 401/404 issues by setting correct headers and URL paths * Improve Plugin Bridge security and architecture - Create ServeInternalPluginRequest for internal plugin calls (core + plugin-to-plugin) - Move header-setting logic from CallPluginBridge to ServeInternalPluginRequest - Improve separation of concerns: business logic vs HTTP transport - Add security documentation explaining header protection Security Improvements: - ServeInternalPluginRequest is NOT exposed as HTTP route (internal only) - Headers (Mattermost-User-Id, Mattermost-Plugin-ID) are set by trusted server code - External requests cannot spoof these headers (stripped by servePluginRequest) - Core calls use 'com.mattermost.server' as plugin ID for authorization - Plugin-to-plugin calls use real plugin ID (enforced by server) Backward Compatibility: - Keep ServeInterPluginRequest for existing API.PluginHTTP callers (deprecated) - All tests pass Docs: - Update spec.md with security model explanation - Update MIGRATION_GUIDE.md with correct header usage examples * Space * cursor please stop creating markdown files * Fix style * Fix i18n, linter * REMOVE MARKDOWN * Remove CallPlugin method from plugin API interface Per review feedback, this method is no longer needed. Co-authored-by: Nick Misasi <nickmisasi@users.noreply.github.com> * Remove CallPlugin method implementation from PluginAPI Co-authored-by: Nick Misasi <nickmisasi@users.noreply.github.com> * fixes * Add AI OpenAPI spec * fix openapi spec * Use agents client (#34225) * Use agents client * Remove default agent * Fixes * fix: modify system prompts to ensure JSON is being returned * Base implementation for recaps working * small fixes * Adjustments * remove webapp changes * Add feature flags for rewrites and ai bridge, clean up * Remove comments that aren't helpful * Fix i18n * Remove rewrites * Fix tests * Fix i18n * adjust i18n again * Add back translations * Remove leftover mock code * remove model file * Changes from PR review * Make the real substitutions * Include a basic invokation of the client with noop to ensure build works * more fix * Remove unneeded change * Updates from review * Fixes * Remove some logic from rewrites to clean up branch * Use v1.5.0 of agents plugin * A bunch more additions for general UX flow * Add missing files * Add mocks * Fixes for vet-api, i18n, build, types, etc * One more linter fix * Fix i18n and some tests * Refactors and cleanup in backend code * remove rogue markdown file * fixes after refactors from backend * Add back renamed files, and add tests * More self code review * More fixes * More refactors * Fix call stack exceeded bug * Include read messages if there are no unreads * Fix test failure: use correct error message key for recap permission denied The getRecapAndCheckOwnership function was using strings.ToLower(callerName) to generate error keys, which caused 'GetRecap' to become 'getrecap' instead of the expected 'get'. Changed to use the correct static key that matches the en.json localization file. Fixes TestGetRecap/get_recap_by_non-owner test failure. Co-authored-by: Nick Misasi <nickmisasi@users.noreply.github.com> * Consolidate permission errors down to a single string * Fixes for i18n, worktrees making this difficult * Fix i18n * Fix i18n once and for all (for real) (final) * Fix duplicate getAgents method in client4.ts * Remove duplicate ai state from initial_state.ts * Fix types * Fix tests * Fix return type of GetAgents and GetServices * Add tests for recaps components * Fix types * Update i18n * Fixes * Fixes * More cleanup * Revert random file * Use undefined * fix linter * Address feedback * Missed a git add * Fixes * Fix i18n * Remove fallback * Fixes for PR --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Nick Misasi <nickmisasi@users.noreply.github.com> Co-authored-by: Christopher Speller <crspeller@gmail.com> Co-authored-by: Felipe Martin <me@fmartingr.com> Co-authored-by: Mattermost Build <build@mattermost.com>	2026-01-13 11:59:22 -05:00
Ibrahim Serdar Acikgoz	084006c0ea	[MM-61758] Burn on read feature (#34703) ... * Add read receipt store for burn on read message types * update mocks * fix invalidation target * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * add translations * Added logic to associate files of BoR post with the post * Added test * fixes * disable pinning posts and review comments * MM-66594 - Burn on read UI integration (#34647) * MM-66244 - add BoR visual components to message editor * MM-66246 - BoR visual indicator for sender and receiver * MM-66607 - bor - add timer countdown and autodeletion * add the system console max time to live config * use the max expire at and create global scheduler to register bor messages * use seconds for BoR config values in BE * implement the read by text shown in the tooltip logic * unestack the posts from same receiver and BoR and fix styling * avoid opening reply RHS * remove unused dispatchers * persis the BoR label in the drafts * move expiration value to metadata * adjust unit tests to metadata insted of props * code clean up and some performance improvements; add period grace for deletion too * adjust migration serie number * hide bor messages when config is off * performance improvements on post component and code clean up * keep bor existing post functionality if config is disabled * Add read receipt store for burn on read message types * Add temporary posts table * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * avoid reacting to unrevealed bor messages * adjust migration number * Add read receipt store for burn on read message types * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * adjust post reveal and type with backend changes * use real config values, adjust icon usage and style * adjust the delete from from sender and receiver * improve self deleting logic by placing in badge, use burn endpoint * adjust websocket events handling for the read by sender label information * adjust styling for concealed and error state * update burn-on-read post event handling for improved recipient tracking and multi-device sync * replace burn_on_read with type in database migrations and model * remove burn_on_read metadata from PostMetadata and related structures * Added logic to associate files of BoR post with the post * Added test * adjust migration name and fix linter * Add read receipt store for burn on read message types * update mocks * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * Added logic to associate files of BoR post with the post * Added test * disable pinning posts and review comments * show attachment on bor reveal * remove unused translation * Enhance burn-on-read post handling and refine previous post ID retrieval logic * adjust the returning chunk to work with bor messages * read temp post from master db * read from master * show the copy link button to the sender * revert unnecessary check * restore correct json tag * remove unused error handling and clarify burn-on-read comment * improve type safety and use proper selectors * eliminate code duplication in deletion handler * optimize performance and add documentation * delete bor message for sender once all receivers reveal it * add burn on read to scheduled posts * add feature enable check * use master to avoid all read recipients race condition --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> * squash migrations into single file * add configuration for the scheduler * don't run messagehasbeenposted hook * remove parallel tests on burn on read * add clean up for closing opened modals from previous tests * simplify delete menu item rendering * add cleanup step to close open modals after each test to prevent pollution * streamline delete button visibility logic for Burn on Read posts * improve reliability of closing post menu and modals by using body ESC key --------- Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> Co-authored-by: Pablo Vélez <pablovv2012@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>	2025-12-11 07:59:50 +01:00
Elias Nahum	1022cd44c0	MM-65756 Database Migrations, Indexes and Methods for Auto-Translation (#34047) ... * AutoTranslate config settings * comment out Agents provider * Add auto translate timeout config validation * i18n messages for autotranslation config validation * fix test * validate url for libreTranslate * Feedback review * Admin Console UI for Auto-Translation * fix admin console conditional section display * i18n * removed unintentional change Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * update admin.general.localization.autoTranslateProviderDescription newline * fix lint * Fix types * UX feedback review * fix typo in i18n * Fix AutoTranslation feature flag * feedback review * Fix test default values * feedback review * re-add isHidden property to feature discovery * Database Migrations, Indexes and Methods for Auto-Translation * i18n * fix retrylayer and storetest * Fix search query * fix lint * remove the request.CTX and modify Translation model * fix lint and external url * Add settings to playwright * Add empty as a valid value for the Provider * Update jsonb queries * Fix queries and add model methods * fix go lint * go lint fix 2 * fix db migrations * feedback review + store cache * increase migration number * cleanup autotranslation store cache * use NULL as objectType for posts * fix bad merge * fix tests * add missing i18n * Switch prop bags column to boolean * fix lint * fix tests * Remove database search * use Builder methods --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: BenCookie95 <benkcooke@gmail.com>	2025-11-22 09:32:01 +08:00
Ben Cooke	3aad6b0448	Add support for resource parameter with OAuth (#33743)	2025-11-11 15:24:42 -05:00
Ben Cooke	a79ac96b50	OAuth public client support through DCR and PKCE support for public/confidential clients (#33664) ... * public client support along with PKCE for public/confidential clients --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2025-11-11 17:43:37 +00:00
Ben Cooke	a9c9953439	Authorization metadata endpoint and Dynamic Client Registration of Confidential OAuth Apps (#33642) ... * initial DCR and metadata implementation * check for duplicate registrations * tests and other cleanup * dcr fixes * tidy up unused DCR fields * remove initial access token support * remove duplicate client checks * remove unused store function * remove restrictive redirect url checks * create some constants for endpoints * surface support for implicit grant and add system console setting * fix frontend issues with DCR clients * rate limiting the DCR endpoint * lint * lint and cleanup * remove storage of grants, responses and methods. Just enforce in the code * fix lint and tests * docs and test * accidentally removed comments * fix mock * translations * do not advertise public client capability * validate supplied token_endpoint_auth_method * fix pr comments * updates * add metadata endpoint to docs * add definition * lint * fix client4 * fix client methods * fix client again --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2025-11-11 14:27:18 +00:00
Harshil Sharma	3265054ad5	Migrate content flagging settings to database (#33989) ... * lint fix * CI * added new migration mocks * Used setup for tests * some comment * Removed unnecesseery nil check * Form validation * WIP tests * WIP tests * WIP tests * fix: mock content flagging config selector with correct reasons format Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> * fix: add mock for getContentFlaggingConfig in flag post modal test Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> * Updated error code order in API docs * removed empty files * Added tests * lint fixes * minor tweak * lint fix * type fix * fixed test * nit * test enhancements * API WIP * API WIP * creating values * creating content flagging channel and properties * Able to save properties * Added another property field * WIP * WIP * Added validations * Added data validations and hidden post if confifgured to * lint fixes * Added API spec * Added some tests * Added tests for getContentReviewBot * test: add comprehensive tests for getContentReviewChannels function * Added more app layer tests * Added TestCanFlagPost * test: Add comprehensive tests for FlagPost function * Added all app layer tests * Removed a file that was reamoved downstream * test: add content flagging test file * test: add comprehensive tests for FlagContentRequest.IsValid method * Added model tests * test: add comprehensive tests for SqlPropertyValueStore.CreateMany * test: add comprehensive tests for flagPost() API function * Added API tests * linter fix * WIP * sent post flagging confirmation message * fixed i18n nissues * fixed i18n nissues * CI * WIP * WIP * Added API call * test: add test for Client4.flagPost API call in FlagPostModal * fix: remove userEvent.setup() from flag post modal test * test: wrap submit button click in act for proper state updates * Updated tests * lint fix * Updated test * fix: reset contentFlaggingGroupId for test isolation in content flagging tests * removed cached group ID * removed debug log * CI * Updated to allow special characters in comments * Handled empty comment * Created getContentFlaggingFields API * created getPostPropertyValues API * Used finally * WIP * Created useContentFlaggingFields hook * WIP * WIP * Added option to retain data for reviewers * Displayed deleted post's preview * DIsplayed all properties * Adding field name i18n * WIP - managing i18n able texts * Finished displaying all fields * Manual cleanup * lint fixes * team role filter logic fix * Fixed tests * created new API to fetch flagged posts * lint fix * Added new client methods * test: add comprehensive tests for content flagging APIs * Added new API tests * fixed openapi spec * Fixed DataSpillageReport tests * Fixed PostMarkdown test * Fixed PostPreviewPropertyRenderer test * Added metadata to card renderer * test fixes * Added no comment placeholder * Added view detail button * Created RemoveFlaggedMessageConfirmationModal modal * Added key and remove flag request modal * IMplemented delete flagged post * Handled edge cases of deleting flagged post * keep message * UI integration * Added WS event for post report update and handled deleted files of flagged post * Added error handling in keep/remove forms * i18n fixes * Fixed test * Updated OpenAPI specs * fixed types * fixed types * refactoring * refactor: improve test mocking for data spillage report component * test mock updates * Fixed tests * Updated reducer * not resetting mocks * Added migrations for content flagging tables * Created new structure * review fixes * Used correct ot name * WIP * review fixes * review fixes * Added new property translations * CI * CI * CI * Improved test * fixed test * CI * New UI component * WIP * Updated settings APIs * cached DB data * used cached reviewer data * Updated tests * Lint fixes * test: add tests for saveContentFlaggingSettings and getContentFlaggingSettings APIs * test fix * test: add tests for SaveContentFlaggingConfig and GetContentFlaggingConfigReviewerIDs * Updated tests * test: add content flagging test for local cache layer * test: add comprehensive tests for content flagging store cache * Updated tests * lint fix * Updated mobile text * Added content flagging SQL store mocks * Added API specs for new APIs * fixed tests * feat: add TestContentFlaggingStore function for content flagging store testing * feat: add comprehensive tests for content flagging store * Added SQL store tests * test: add content flagging test for local cache layer * test: add tests for content flagging store caching * Added cache layer tests * Updated tests * Fixed * Handled JSON error * fixes * fixes * Fixed retry layer test * fixerdf i18n * Fixed test * CI * building index concurrently * CI * fixed a test * CI * cleanup * Integrate flag post api (#33798) * WIP * WIP * Added API call * test: add test for Client4.flagPost API call in FlagPostModal * fix: remove userEvent.setup() from flag post modal test * test: wrap submit button click in act for proper state updates * Updated tests * lint fix * CI * Updated to allow special characters in comments * Handled empty comment * Used finally * CI * Fixed test * Spillage card integration (#33832) * Created getContentFlaggingFields API * created getPostPropertyValues API * WIP * Created useContentFlaggingFields hook * WIP * WIP * Added option to retain data for reviewers * Displayed deleted post's preview * DIsplayed all properties * Adding field name i18n * WIP - managing i18n able texts * Finished displaying all fields * Manual cleanup * lint fixes * team role filter logic fix * Fixed tests * created new API to fetch flagged posts * lint fix * Added new client methods * test: add comprehensive tests for content flagging APIs * Added new API tests * fixed openapi spec * Fixed DataSpillageReport tests * Fixed PostMarkdown test * Fixed PostPreviewPropertyRenderer test * Added metadata to card renderer * test fixes * Added no comment placeholder * Fixed test * refactor: improve test mocking for data spillage report component * test mock updates * Updated reducer * not resetting mocks * WIP * review fixes * CI * Fixed * fixes * Content flagging actions implementation (#33852) * Added view detail button * Created RemoveFlaggedMessageConfirmationModal modal * Added key and remove flag request modal * IMplemented delete flagged post * Handled edge cases of deleting flagged post * keep message * UI integration * Added WS event for post report update and handled deleted files of flagged post * Added error handling in keep/remove forms * i18n fixes * Updated OpenAPI specs * fixed types * fixed types * refactoring * Fixed tests * review fixes * Added new property translations * Improved test * fixed test * CI * fixes * CI * fixed a test * CI * Review fixes --------- Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>	2025-10-13 12:24:01 +05:30
Agniva De Sarker	9dd8c056e7	MM-63368: Remove MySQL (#33458) ... https://mattermost.atlassian.net/browse/MM-63368 ```release-note Remove MySQL support from the codebase entirely. ```	2025-07-22 20:40:55 +05:30
catalintomai	85391de22a	MM-57326: [Shared Channels] Message priority, acknowledgement and persistent notifications need to be synced (#30736)	2025-06-16 02:30:21 +02:00
catalintomai	fa1c77d9b0	MM-52600: [Shared Channels] Shared channels do not sync channel membership (#30976)	2025-06-15 10:07:56 +02:00
catalintomai	c46ed6c681	MM-62751: [Shared Channels] Allow remote users to be discoverable in the create DM/GM modal (#30918)	2025-06-13 16:51:12 +02:00
Devin Binnie	25a4839a9e	Automatic channel category sorting (#30866) ... * Automatic channel category sorting * Fix types * AIed * Fix issue where categories are updated for all users * Move all logic to server, clean up * PR feedback * Fix lint --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2025-06-11 14:29:36 -04:00
Ibrahim Serdar Acikgoz	1cf2f08108	[MM-64437] Hotfix on attribute view creation error (#31225)	2025-05-30 11:35:31 +02:00
Ibrahim Serdar Acikgoz	a344b3225b	[MM-61756] Attribute Based Access Control - Phase 1 (#30785) ... Attribute Based Access Control - Base * MM-63662 * MM-63919 * MM-63954 * MM-63955 * MM-63425 * MM-63426 * MM-63458 * MM-63459 * MM-63603 * MM-63845 * MM-64146 * MM-64199 * MM-64201 * MM-64233 * MM-64247 * MM-64268 --------- Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> Co-authored-by: Pablo Andrés Vélez Vidal <pablovv2012@gmail.com> Co-authored-by: abhijit-singh <abhijitsingh0702@gmail.com> Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>	2025-05-15 11:33:08 +02:00
Agniva De Sarker	131cf039bb	MM-63756: Added index to sidebarchannels table (#30724) ... The (s SqlChannelStore) getSidebarCategoriesT gets called quite frequently. - Team switch - WS reconnect - Category created - Category updated - Category deleted Of these 1 and 2 are probably the most commonly called sources. Based on that, the sidebarChannels table is not that well-optimized. Even though the query time might be reasonable, without an index, it has to churn a lot of DB CPU for a sequential scan. We add a new index to optimize this. CREATE INDEX idx_sidebarchannels_categoryid ON sidebarchannels(categoryid); ``` Before: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Sort (cost=40854.18..40854.19 rows=4 width=193) (actual time=251.635..251.646 rows=204 loops=1) Sort Key: sidebarcategories.sortorder, sidebarchannels.sortorder Sort Method: quicksort Memory: 65kB Buffers: shared hit=1203 read=23668 -> Nested Loop (cost=8.87..40854.14 rows=4 width=193) (actual time=251.345..251.455 rows=204 loops=1) Buffers: shared hit=1203 read=23668 -> Nested Loop (cost=0.41..9.47 rows=1 width=54) (actual time=0.068..0.074 rows=1 loops=1) Buffers: shared hit=5 -> Seq Scan on teams (cost=0.00..1.03 rows=1 width=27) (actual time=0.024..0.026 rows=1 loops=1) Filter: (((id)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text) AND (deleteat = 0)) Rows Removed by Filter: 1 Buffers: shared hit=1 -> Index Scan using teammembers_pkey on teammembers (cost=0.41..8.43 rows=1 width=27) (actual time=0.039..0.043 rows=1 loops=1) Index Cond: (((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text) AND ((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text)) Filter: (deleteat = 0) Buffers: shared hit=4 -> Hash Right Join (cost=8.45..40844.62 rows=4 width=193) (actual time=251.274..251.361 rows=204 loops=1) Hash Cond: ((sidebarchannels.categoryid)::text = (sidebarcategories.id)::text) Buffers: shared hit=1198 read=23668 -> Seq Scan on sidebarchannels (cost=0.00..37514.77 rows=1265277 width=100) (actual time=0.043..99.345 rows=1265444 loops=1) Buffers: shared hit=1194 read=23668 -> Hash (cost=8.44..8.44 rows=1 width=158) (actual time=0.047..0.047 rows=6 loops=1) Buckets: 1024 Batches: 1 Memory Usage: 10kB Buffers: shared hit=4 -> Index Scan using idx_sidebarcategories_userid_teamid on sidebarcategories (cost=0.42..8.44 rows=1 width=158) (actual time=0.029..0.037 rows=6 loops=1) Index Cond: (((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND ((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text)) Buffers: shared hit=4 Planning: Buffers: shared hit=9 Planning Time: 1.215 ms Execution Time: 251.755 ms (31 rows) After: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Sort (cost=1544.53..1544.54 rows=4 width=192) (actual time=0.834..0.859 rows=204 loops=1) Sort Key: sidebarcategories.sortorder, sidebarchannels.sortorder Sort Method: quicksort Memory: 65kB Buffers: shared hit=58 -> Nested Loop Left Join (cost=8.53..1544.49 rows=4 width=192) (actual time=0.066..0.252 rows=204 loops=1) Buffers: shared hit=58 -> Nested Loop (cost=0.83..17.93 rows=1 width=157) (actual time=0.042..0.098 rows=6 loops=1) Buffers: shared hit=34 -> Nested Loop (cost=0.42..9.48 rows=1 width=157) (actual time=0.030..0.049 rows=6 loops=1) Buffers: shared hit=10 -> Index Scan using idx_sidebarcategories_userid_teamid on sidebarcategories (cost=0.42..8.44 rows=1 width=157) (actual time=0.018..0.022 rows=6 loops=1) Index Cond: (((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND ((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text)) Buffers: shared hit=4 -> Seq Scan on teams (cost=0.00..1.03 rows=1 width=27) (actual time=0.002..0.003 rows=1 loops=6) Filter: (((id)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text) AND (deleteat = 0)) Rows Removed by Filter: 1 Buffers: shared hit=6 -> Index Scan using teammembers_pkey on teammembers (cost=0.41..8.43 rows=1 width=27) (actual time=0.007..0.007 rows=1 loops=6) Index Cond: (((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text) AND ((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text)) Filter: (deleteat = 0) Buffers: shared hit=24 -> Bitmap Heap Scan on sidebarchannels (cost=7.69..1522.35 rows=421 width=100) (actual time=0.012..0.017 rows=34 loops=6) Recheck Cond: ((categoryid)::text = (sidebarcategories.id)::text) Heap Blocks: exact=6 Buffers: shared hit=24 -> Bitmap Index Scan on idx_sidebarchannels_categoryid (cost=0.00..7.58 rows=421 width=0) (actual time=0.010..0.010 rows=34 loops=6) Index Cond: ((categoryid)::text = (sidebarcategories.id)::text) Buffers: shared hit=18 Planning: Buffers: shared hit=18 Planning Time: 0.543 ms Execution Time: 0.968 ms (32 rows) ``` I have also looked at potentially re-ordering the JOINs to make sidebarchannels and sidebarcategories JOIN earlier, but that didn't give a major benefit. Also looked at adding a compound index with (categoryid, sortorder) to improve sorting performance, but that didn't give a major benefit from what the single column index already gives. The `completePopulatingCategoryChannelsT` query also partially benefits from this. But the Postgres optimizer sometimes selects the index on categoryId and sometimes on ChannelId, both giving equivalent performance. So there's no major improvement there, but at the same time, no regression as well. ``` Original: [bigdb] # EXPLAIN (ANALYZE, BUFFERS) SELECT Id FROM ChannelMembers LEFT JOIN Channels ON Channels.Id=ChannelMembers.ChannelId WHERE (ChannelMembers.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND Channels.Type IN ('D' ,'G') AND Channels.DeleteAt = 0 AND NOT EXISTS ( SELECT 1 FROM SidebarChannels JOIN SidebarCategories on SidebarChannels.CategoryId=SidebarCategories.Id WHERE (SidebarChannels.ChannelId = ChannelMembers.ChannelI d AND SidebarCategories.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND SidebarCategories.TeamId = '3ee5y5ok6jgxicrmqstdnghmfr') )) ORDER BY DisplayName ASC; QUERY PLAN -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Sort (cost=5864.68..5865.84 rows=463 width=40) (actual time=9.008..9.022 rows=39 loops=1) Sort Key: channels.displayname Sort Method: quicksort Memory: 27kB Buffers: shared hit=2112 -> Nested Loop Anti Join (cost=1.96..5844.18 rows=463 width=40) (actual time=0.188..8.932 rows=39 loops=1) Buffers: shared hit=2112 -> Nested Loop (cost=0.99..3476.66 rows=463 width=67) (actual time=0.159..7.952 rows=39 loops=1) Buffers: shared hit=1956 -> Index Only Scan using idx_channelmembers_user_id_channel_id_last_viewed_at on channelmembers (cost=0.56..40.78 rows=470 width=27) (actual time=0.036..0.467 rows=437 loops=1) Index Cond: (userid = 'tc3p1yqw67d8idcp3g98awexqe'::text) Heap Fetches: 45 Buffers: shared hit=208 -> Memoize (cost=0.43..7.69 rows=1 width=40) (actual time=0.016..0.016 rows=0 loops=437) Cache Key: channelmembers.channelid Cache Mode: logical Hits: 0 Misses: 437 Evictions: 0 Overflows: 0 Memory Usage: 42kB Buffers: shared hit=1748 -> Index Scan using channels_pkey on channels (cost=0.42..7.68 rows=1 width=40) (actual time=0.015..0.015 rows=0 loops=437) Index Cond: ((id)::text = (channelmembers.channelid)::text) Filter: ((type = ANY ('{D,G}'::channel_type[])) AND (deleteat = 0)) Rows Removed by Filter: 1 Buffers: shared hit=1748 -> Nested Loop (cost=0.97..5.10 rows=1 width=27) (actual time=0.023..0.023 rows=0 loops=39) Buffers: shared hit=156 -> Index Only Scan using sidebarchannels_pkey on sidebarchannels (cost=0.55..4.56 rows=1 width=92) (actual time=0.022..0.022 rows=0 loops=39) Index Cond: (channelid = (channelmembers.channelid)::text) Heap Fetches: 0 Buffers: shared hit=156 -> Index Scan using sidebarcategories_pkey on sidebarcategories (cost=0.42..0.48 rows=1 width=65) (never executed) Index Cond: ((id)::text = (sidebarchannels.categoryid)::text) Filter: (((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND ((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text)) Planning: Buffers: shared hit=48 dirtied=1 Planning Time: 2.222 ms Execution Time: 9.142 ms (35 rows) New: [bigdb] # EXPLAIN (ANALYZE, BUFFERS) SELECT Id FROM ChannelMembers LEFT JOIN Channels ON Channels.Id=ChannelMembers.ChannelId WHERE (ChannelMembers.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND Channels.Type IN ('D' ,'G') AND Channels.DeleteAt = 0 AND NOT EXISTS ( SELECT 1 FROM SidebarChannels JOIN SidebarCategories on SidebarChannels.CategoryId=SidebarCategories.Id WHERE (SidebarChannels.ChannelId = ChannelMembers.ChannelId AND SidebarCategories.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND SidebarCategories.TeamId = '3ee5y5ok6jgxicrmqstdnghmfr') )) ORDER BY DisplayName ASC; QUERY PLAN -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Sort (cost=5059.95..5061.11 rows=463 width=40) (actual time=12.072..12.086 rows=39 loops=1) Sort Key: channels.displayname Sort Method: quicksort Memory: 27kB Buffers: shared hit=1984 -> Nested Loop Anti Join (cost=9.10..5039.45 rows=463 width=40) (actual time=0.751..12.009 rows=39 loops=1) Join Filter: ((sidebarchannels.channelid)::text = (channelmembers.channelid)::text) Rows Removed by Join Filter: 7839 Buffers: shared hit=1984 -> Nested Loop (cost=0.99..3476.66 rows=463 width=67) (actual time=0.161..7.579 rows=39 loops=1) Buffers: shared hit=1956 -> Index Only Scan using idx_channelmembers_user_id_channel_id_last_viewed_at on channelmembers (cost=0.56..40.78 rows=470 width=27) (actual time=0.036..0.449 rows=437 loops=1) Index Cond: (userid = 'tc3p1yqw67d8idcp3g98awexqe'::text) Heap Fetches: 45 Buffers: shared hit=208 -> Memoize (cost=0.43..7.69 rows=1 width=40) (actual time=0.016..0.016 rows=0 loops=437) Cache Key: channelmembers.channelid Cache Mode: logical Hits: 0 Misses: 437 Evictions: 0 Overflows: 0 Memory Usage: 42kB Buffers: shared hit=1748 -> Index Scan using channels_pkey on channels (cost=0.42..7.68 rows=1 width=40) (actual time=0.014..0.014 rows=0 loops=437) Index Cond: ((id)::text = (channelmembers.channelid)::text) Filter: ((type = ANY ('{D,G}'::channel_type[])) AND (deleteat = 0)) Rows Removed by Filter: 1 Buffers: shared hit=1748 -> Materialize (cost=8.11..1535.03 rows=4 width=27) (actual time=0.003..0.046 rows=201 loops=39) Buffers: shared hit=28 -> Nested Loop (cost=8.11..1535.01 rows=4 width=27) (actual time=0.099..0.383 rows=201 loops=1) Buffers: shared hit=28 -> Index Scan using idx_sidebarcategories_userid_teamid on sidebarcategories (cost=0.42..8.44 rows=1 width=65) (actual time=0.047..0.057 rows=6 loops=1) Index Cond: (((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND ((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text)) Buffers: shared hit=4 -> Bitmap Heap Scan on sidebarchannels (cost=7.69..1522.35 rows=421 width=92) (actual time=0.028..0.040 rows=34 loops=6) Recheck Cond: ((categoryid)::text = (sidebarcategories.id)::text) Heap Blocks: exact=6 Buffers: shared hit=24 -> Bitmap Index Scan on idx_sidebarchannels_categoryid (cost=0.00..7.58 rows=421 width=0) (actual time=0.023..0.023 rows=34 loops=6) Index Cond: ((categoryid)::text = (sidebarcategories.id)::text) Buffers: shared hit=18 Planning: Buffers: shared hit=51 Planning Time: 2.240 ms Execution Time: 12.210 ms (42 rows) ``` Analysis on MySQL for completion: ``` Before: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | -> Sort: SidebarCategories.SortOrder, SidebarChannels.SortOrder (actual time=277.675..277.675 rows=4 loops=1) -> Stream results (cost=138558.36 rows=1287808) (actual time=242.506..277.650 rows=4 loops=1) -> Left hash join (<hash>(SidebarChannels.CategoryId)=<hash>(SidebarCategories.Id)), extra conditions: (SidebarChannels.CategoryId = SidebarCategories.Id) (cost=138558.36 rows=1287808) (actual time=242.498..277.626 rows=4 loops=1) -> Index lookup on SidebarCategories using idx_sidebarcategories_userid_teamid (UserId='qdggj9pyobgkjpj8htwzizks1r', TeamId='xmh7bupzajnudqf3h4mm76qapy') (cost=1.40 rows=4) (actual time=0.092..0.094 rows=4 loops=1) -> Hash -> Table scan on SidebarChannels (cost=8394.55 rows=321952) (actual time=0.123..106.334 rows=300002 loops=1) | +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ After: ----------------------------------------------------------------+ | -> Sort: SidebarCategories.SortOrder, SidebarChannels.SortOrder (actual time=0.739..0.742 rows=4 loops=1) -> Stream results (cost=6.80 rows=7) (actual time=0.468..0.703 rows=4 loops=1) -> Nested loop left join (cost=6.80 rows=7) (actual time=0.456..0.673 rows=4 loops=1) -> Index lookup on SidebarCategories using idx_sidebarcategories_userid_teamid (UserId='qdggj9pyobgkjpj8htwzizks1r', TeamId='xmh7bupzajnudqf3h4mm76qapy') (cost=4.38 rows=4) (actual time=0.302..0.313 rows=4 loops=1) -> Index lookup on SidebarChannels using idx_sidebarchannels_categoryid (CategoryId=SidebarCategories.Id) (cost=0.48 rows=2) (actual time=0.085..0.087 rows=0 loops=4) | +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ ``` Timing wise, it takes around 2s to add the index on a table with 1.2M rows for Postgres. And it takes around 5s on MySQL on a table with 300K rows. It looks like it takes longer on MySQL, but since both migrations are non-locking, it should be fine. https://mattermost.atlassian.net/browse/MM-63756 ```release-note NONE ```	2025-04-24 12:11:28 +05:30
Ibrahim Serdar Acikgoz	10b1f4c5ac	[MM-63428] add access control policy store (#30597)	2025-04-02 13:39:28 +02:00
Harshil Sharma	6e738f489f	Channel banner sql migrations (#30274) ... * Adde MySQL and Postgres migrations * Replaced select * with column names * removed all * from channel SQL store * cleanup * Fixed a duplicate column * cleanup * Added migrations and store support * WIP * used channelname slice in a missed place * Handled patch * Added app level tests * Added API layer tests * Added API layer tests * WIP * converted to query builder * cleanupo * added not null and default constraints * Fixed test * fixed file name * review fixes * review fixes * updated migration file * fixed text * Review fixes	2025-02-25 14:52:15 +05:30
Julien Tant	632a60b332	[MM-62553]+[MM-62554] Property Architecture: cursor based pagination (#30119) ... * refactor: Replace pagination with cursor-based pagination for custom profile attributes * remove pagination loop on property value retrieval for CPA * add migrations to optimize pagination on property fields and values * adapt test to remove pagination check * update migrations list * postgres: drop index concurrently * concurrent index manipulation must be done outside of a Tx * fix: Correct SQL index drop syntax from "OM" to "ON" in migration files * test: Add CountForGroup test cases for property field store * refactor: Add CountForGroup method to PropertyFieldStore interface and implementations * Fix style and i18n * feat: Add optional deleted property field filtering to CountForGroup method * refactor: Update CountForGroup to support optional deleted property fields * test: Add comprehensive tests for CountForGroup with includeDeleted parameter * adapt test + gen layers * rename property service method and set the includeDelete to false * refactor: Remove redundant constant and use CustomProfileAttributesFieldLimit directly * fix tests --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2025-02-13 15:23:50 +00:00
Agniva De Sarker	ae9e6174e5	[AI assisted]: Improve system console statistics performance (#29899) ... ```release-note NONE ``` Co-authored-by: Matthew Birtch <mattbirtch@gmail.com>	2025-02-04 21:54:01 +05:30
Miguel de la Cruz	ecdce71fc4	Adds the main Property System Architecture components (#29644) ... * Adds the main Property System Architecture components This change adds the necessary migrations for the Property Groups, Fields and Values tables to be created, the store layer and a Property Service that can be used from the app layer. * Update property field type to use user instead of person * Update PropertyFields to allow for unique nondeleted fields and remove redundant indexes * Update PropertyValues to allow for unique nondeleted fields and remove redundant indexes * Use StringMap instead of the map[string]any on property fields * Add i18n strings * Revert "Use StringMap instead of the map[string]any on property fields" This reverts commit e2735ab0f8. * Cast JSON binary data to string and add todo note for StringMap use * Add mocks to the retrylayer tests * Cast JSON binary data to string in property value store * Check for binary parameter instead of casting to string for JSON data * Check property field type is one of the allowed ones * Avoid reusing err variable to be explicit about the returned value * Merge Property System Migrations into one file * Adds NOT NULL to timestamps at the DB level * Update stores to use tableSelectQuery instead of a slice var * Update PropertyField model translations to be more explicit and avoid repetition * Update PropertyValue model translations to be more explicit and avoid repetition * Use ExecBuilder instead of ToSql&Exec * Update property field errors to add context * Ensure PerPage is greater than zero * Update store errors to give more context * Use ExecBuilder in the property stores where possible * Add an on conflict suffix to the group register to avoid race conditions * Remove badly used translation string * Remove unused get in register group method --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2025-01-13 11:41:44 +00:00
Harshil Sharma	e281b3f37e	Feature scheduled messages (#28932) ... * Create scheduled post api (#27920) * Added migration files for Postgres * Added migrations for MySQL * Added store method * Added API and store tests * Renamed migration after syncing with master * Added app layer tests * API is ready * API is ready * API is ready * Renamed migration after syncing with master * Updated migration list * Fixed retry layer tests * Allowed posts with empty messages * Review fixes * Reverted an incorrect change * Renamed migration and fixed ID assignment * CI * Send post button changes (#28019) * added Split button * WIP * Added core menu options * WIP * WIP * WIP * Handled displaying error in creating scheduled post * lint fixes * webapp i18n fix * Review fixes * Fixed a webapp test * A few more fixes * Removed a duplicate comment * Scheduled post job (#28088) * Added the job function * Added query for fetching scheduled posts for pricessing * WIP * WIP * WIP * WIP * WIP * WIP * Reafactoring of scheduled post job * Lint fixes * Updated i18n files * FInishing touches * Added tests for GetScheduledPosts * Added tests for PermanentlyDeleteScheduledPosts * Updated all layer * Some changes as discussed with team * Added tests for UpdatedScheduledPost * Code review refactoring * Added job test * MM-60120 - Custom time selection (#28120) * Added a common date time picker modal and used it for post reminder * Added a common date time picker modal and used it for post reminderggp * Added modal for custom schedule time and fixed TZ issue * WIP * Removed event from useSubmit hook * Removed event from useSubmit hook * Added timezone handling * fixed type error * Updated i18n strings * Minor cleanup * updated snapshots * review fixes * Handled event * Supported for having a DM thread open in RHS while in a regular channel * Review fixes * MM-60136 - Scheduled messages tab (#28133) * WIP * WIP * Created Tabs and Tab wrapper with added styling * Added API to get scheduled posts * WIP * Displated scheduled post count * i18n fix * Added tests * Handled asetting active tab absed on URL: * Reverted unintended change * Added API to client ad OpenAPI specs * Renamed file * Adding fileinfo to schedule posts * Partial review fixes * Made get scheduled post API return posts by teamID * review fixes * Moved scheduled post redux code to MM-redux package * Usedd selector factory * WIP: * WIP: * Lint fix * Fixed an incorrect openapi spec file * Removed redundent permission check * Clreaed scheduled post data on logout * Removed unused i18n string: * lint fix * Render scheduled posts (#28208) * WIP * WIP * Created Tabs and Tab wrapper with added styling * Added API to get scheduled posts * WIP * Displated scheduled post count * i18n fix * Added tests * Handled asetting active tab absed on URL: * Reverted unintended change * Added API to client ad OpenAPI specs * Renamed file * Created common component for draft list item * WIP * WIP * Adding fileinfo to schedule posts * Basic rendering * Added count badge to tabs * WIP * Made the Drafts LHS iteam appear if no drafts exist but scheduled posts do * Fixed icon size * Partial review fixes * Made get scheduled post API return posts by teamID * Handled initial vs team switch load * Displayed scheduled date in panel header * Added error message and error indiocator * WIP * review fixes * WIP Adding error reason tag * Added error codes * Moved scheduled post redux code to MM-redux package * Usedd selector factory * WIP: * WIP: * Lint fix * Fixed an incorrect openapi spec file * Removed redundent permission check * Clreaed scheduled post data on logout * Removed unused i18n string: * lint fix * Opened rescheduling modal * Updated graphic for empty state of schduled post list * Added delete scheduled post option and modal * Badge and timezone fix * WIP: * Added send now confirmation modal * lint * Webapp i18n fix * Fixed webapp test * Fixed a bug where DM/GM scheduled posts weren't immideatly showing up in UI * Minor fixes * WIP * Review fixes * Review fixes * Optimisations * Fixed reducer name * Moment optimizatin * Updated route check * MM-60144 - added API to update a scheduled post (#28248) * WIP * Added api and ap layer for update scheduled post ̛̦̄ * Added API to OpenAI specs, Go client and TS client * removed permissio check * Added tests * Fixed tests * Added PreUpdate method on scheduled post model * MM-60131 - Reschedule post integration (#28281) * Handled rescheduling post in webapp * Added error handling * MM-60146 - Delete scheduled post api (#28265) * WIP * Added api and ap layer for update scheduled post ̛̦̄ * Added API to OpenAI specs, Go client and TS client * removed permissio check * Added tests * Fixed tests * Added PreUpdate method on scheduled post model * Added delete scheduled post API * Added API to Go client and OpenAPI specs * Added API to TS client * Added tests * CI * Rmeoved two incorrect code comments * MM-60653 - Integrated delete scheduled post API (#28296) * Integrated delete scheduled apost API * Lint fix * Review fixes * Excluded draft checks from scheduled posts (#28370) * Excluded draft checks from scheduled posts * Added a removed todo * MM-60125 - Scheduled post channel indicator (#28320) * Integrated delete scheduled apost API * Lint fix * Added state for storing scheduled posts by channel ID * Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere * Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere * WIP * Added scheduled post indiocator * Handled single and multiple scheudled posts * Review fixes * Fixed styling and handled center channel, RHS and threads view * Lint fix * i18n fix * Fixed a cycling dependency * Lint fix * Added some more comments * Updated styling * Review fixes * Added common component for remote user time and scheduled post indicator * Updated scheduled post count * Minor change * Moved CSS code around * Fixed a bug where files in scheduled post didn't show up until refresh (#28359) --------- Co-authored-by: Daniel Espino García <larkox@gmail.com> * Scheduled post config (#28485) * Added config * Added config on server and webapp side * Added config check in server and webapp * Added license check * Added license check * Added placeholder help text * Added license check to job * Fixed job test * Review fixes * Updated English text * Review fixes * MM-60118 - Added index on ScheduledPosts table (#28579) * Added index * Updated indexes * Scheduled posts misc fixes (#28625) * Added detailed logging for scheduled post job * Limited scheduled posts processing to 24 hours * Marked old scheduled posts as unable to send * Added t5ests * converted some logs to trace level * Fixed a bug causing error message to show up on deleting a scheduled post in a deleted thread (#28630) * Fixed scheduled posts link in RHS (#28659) * Fixed scheduled posts link in RHS * Review fixes * Fix permission name in scheduled posts by team (#28580) * Fix permission name * fix wording --------- Co-authored-by: Mattermost Build <build@mattermost.com> * FIxed width of generic modal header to fix browser channel modal (#28639) * Only consider error-free scheduled posts for indicator in channel and RHS (#28683) * Show only errro free scheudled posts in post box indicator * Fixed a bug to handle no scheduled posts * Fixed draft and scheudled post UI in mobile view (#28680) * MM-60873 and MM-60872 - Fixed a bug with updating scheduled posts (#28656) * Fixed a bug with updating scheduled posts * Better selectors * MOved shceuled post message length validation to app layer * MM-60732 - Scheduled posts channel link now takes you to the first scheduled post in channel/thread in list (#28768) * Ordered scheudle dposts by schgeudled at nad create at * Ordered in client * Added scroll to target * Removed classname prop * Fixed tests * Added doc * Import fix * MM-60961 - Fixed a bug where API used incoming create at date for scheduled post (#28703) * Fixed a bug where API used incoming create at date for scheduled post * Stopped sending created at value for scheduled post * MM-60785 - Fixed a bug where scheduled posts of channel we are no longer member of didn't show up (#28637) * Fixed a bug where scheduled posts of channel we are no longer member of didn't show up * Added a comment * CI * Used data loader to optimise laoding missing channels * Minor refactoring * MM-60963 - Added common checks for post and scheduled posts (#28713) * Added commen checks for post and scheuled posts * Sanitised scheduled posts * Fixed tests * Splitted post checks into app and context functions * Added checks on scheduiled posts job as well: * i18n fix * Fixed a test * Renamed a func * removed duplicate check * Scheduled posts UI fixes (#28828) * Fixed send button and time picker borders * Fixed center alignment of time picker * Removed on for today and tomorrow * Lint fix * Date time modal hover state fix * Badge fix * Fixed a mnerge issue * Scheduled Post send now and add schedule on draft (#28851) * Added send now option on scheduled posts * Minor refactoring * WIP * WIP * WIP * Lint fix * i18n fix * Snapshot update * Review fixes * Scheduled post inline editing (#28893) * Added send now option on scheduled posts * Minor refactoring * WIP * WIP * WIP * Lint fix * i18n fix * Snapshot update * Displayed editing component in scheduled post * Added handling for updating scheduled post * Handle events * Fixed escape key issue in scheudled post editing * Fixes * Displayed error message for editing error * Don't show mention warning * Handled dev mode (#28918) * MInor fixes * client fix * Fixes * CI * Removed dev mode behaviour temperorily (#29008) --------- Co-authored-by: Daniel Espino García <larkox@gmail.com> Co-authored-by: Eva Sarafianou <eva.sarafianou@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>	2024-11-04 11:39:35 +05:30
Julien Tant	1909206e16	[MM-59069] Make sure OTP are actual One Time Password (#28074) ... Automatic Merge	2024-09-17 00:44:32 +02:00
Miguel de la Cruz	f8202309ce	Adds logical deletes to shared channel remotes and remote clusters (#28159) ... * Adds logical deletes to shared channel remotes and remote clusters Instead of physically deleting the shared channel remote and remote clusters records when a channel is unshared, a remote uninvited or a remote cluster is deleted, now those have a logical `DeleteAt` field that is set. This allows us to safely restore shared channels between two remote clusters (as of now resetting the cursor without backfilling their contents) and to know which connections were established in the past and now are severed. * Delete the index in remoteclusters before adding the new column * Fix bad error check	2024-09-12 13:55:11 +02:00
Miguel de la Cruz	cd99c4068d	Removes the backslash in migrations.list (#27882) ... The use of the backslash was causing issues between different versions of make.	2024-08-09 11:17:15 +02:00
Miguel de la Cruz	59873a2a50	Adds default team to the remote cluster entity (#27863) ... * Adds default team to the remote cluster entity A new DefaultTeamId field is added to the RemoteCluster entity and its endpoints, and used when receiving channel invites to choose in which team to create a new channel. This will be later extended with the ability for the system admin to manually accept invites, choosing which team to create the channel on each. This use case will be triggered when the DefaultTeamId field is empty, which now simply chooses the first team it finds in the database as a fallback. * Fix migrations list * Fixes channelinvite test case * Fix i18n * Fix migration list	2024-08-08 16:36:27 +02:00
Scott Bishel	99881b819a	MM-58548 Remove manage_team permissions from System Console Ancillary permissions (#27395) ... * remove manage team permissions from sysconsole_write_user_management_chanels and sysconsole_write_user_management_groups * update migrations, add unit tests * run migrations-extract * make updating ancillary permissions a post * update file names * add new api to doc, update body to just be []string * revert moving ancillary permissions to post * fix queries after final testing * Update channel.go * Update channel.go * Update 000124_remove_manage_team_permission.up.sql * Update 000124_remove_manage_team_permission.up.sql --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2024-07-09 08:21:29 -06:00
Scott Bishel	71c25fb316	MM-58525 Fix upload file permissions (#27298) ... * tie create_post and upload_file permissions together * update tests * update file name * update migration to do in batches * Update 000122_remove_upload_file_permission.up.sql * fix formatting * simplify migrations, fix regex issue * update file names for recent merge --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2024-06-24 10:47:52 -06:00
Harrison Healey	5defc75e46	MM-57913 Increase maximum length of Preferences.Value column (#27297) ... * MM-57913 Increase maximum length of Preferences.Value column * Add missing column name to migration * Lowercased table and column names	2024-06-19 10:39:37 -04:00
Ibrahim Serdar Acikgoz	179d28ee3f	[MM-58341] db/migrations: mysql-8.4 fix for 000027_create_status (#27080)	2024-06-05 18:52:53 +02:00
Nick Misasi	f1019d076e	[CLD-7567] Deprecate Self Serve: Second Pass (#26853) ... * Deprecate Self Serve: First Pass * Fix ci * Fix more ci * Remmove outdated server tests * Fix a missed spot opening purchase modal in Self Hosted * Fix i18n * Clean up some more server code, fix webapp test * Fix alignment of button * Fix linter * Fix i18n server side * Deprecate in product true up * Add back translation * Remove client functions * Put back client functions * webapp deprecation * Deprecate Self Serve: Second Pass * Fix various pipeline issues * Fix linter * Fix pipelines * Fix handlers_test.go * Fix console.error around hostedCustomer in reducer * PICKY LINTER PLEASE * Fix webapp tests, various other fixes for the CI pipelines * Fix i18n * Updates to accomadate enterprise code removal * Fix mocks * More removal * Fix * Adjustments from PR * Fixes for QA Feedback * Update * Add migrations to remove true up review history * Fix migrations check --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: maria.nunez <maria.nunez@mattermost.com>	2024-05-02 09:15:15 -04:00
Elias Nahum	7e9cd04a8b	Channel Bookmarks (#25449) ... * create ChannelBookmarks table * ChannelBookmark model * channel bookamrks Store layer * add GetBookmarksForAllChannelByIdSince * add channel bookmarks to test store * Add channel bookmarks to app layer * remove index for createAt in channel bookmarks migrations * remove createAt from select channel bookmark query and enable store delete bookmark test * update reponse of UpdateBookmark * rename db migration files * channel bookmarks store update sort order * channel bookmarks app layer update sort order * fix lint & tests * Fix lint and introduce util functions to insert / remove from slice * remove model etag * i18n * defer remove file info after test run * Fix tests passing the request context * fix migrations * fix TestRetry * Add bookmark permissions (#25560) * Adds channel bookmarks permissions * Fix linter * Remove unnecessary empty lines * Remove scss change as it's not necessary anymore * Fix mock store * Fix mock store and add role entry * Fix test * Adds cypress test and update permissions migration to update admin roles * Adds channel bookmarks roles to default admin roles * Adds bookmark permissions to default role permissions constant in webapp * Update mmctl test * Update permission test after normalising the roles * fix store tests * fix app layer tests * Add new bookmark endpoint (#25624) * Adds channel bookmarks api scaffold and create endpoint * Applies review comments to the API docs * Adds websocket test to create channel bookmark --------- Co-authored-by: Mattermost Build <build@mattermost.com> * MM-54426 exclude Channel Bookmarks files from data retention (#25656) * Augment channel APIs to include bookmarks (#25567) * update files docs for server 9.4 * Adds update channel bookmark endpoint (#25653) * Adds update channel bookmark sort order endpoint (#25686) * Adds update channel bookmark endpoint * Updates edit app method to return the right deleted bookmark and adds tests * Adds the update channel bookmark sort order endpoint * Fix repeated test after merge * Assign right permissions to each test * Update store and app layer to return specific errors and add tests * Adds delete channel bookmark endpoint (#25693) * Updates edit app method to return the right deleted bookmark and adds tests * Fix repeated test after merge * Updates edit app method to return the right deleted bookmark and adds tests * Adds delete channel bookmark endpoint * Adds list channel bookmarks endpoint (#25700) * Add channel moderation to bookmarks (#25716) * fix migrations index * fix getChannelsForTeamForUser * fix getChannelsForTeamForUser * fix bad merge client4 * fix file api with bookmark permission * add ChannelBookmarks feature flag * add missing translations * Set DB column for type as enum * use custom type for bookmark query using sqlx * use transaction when saving bookmark * return NewErrNotFound instead of Sql.ErrNoRows * use squirrel for IN query * add a limit of 1K for records in GetBookmarksForAllChannelByIdSince * UpdateSortOrder with one single query instead of multiple updates * fix shadow declaration * fix channel bookmarks permission string definition in admin console * fix another shadow declaration * Fix model conversion * add SplitSliceInChunks * remove include bookmarks in channels api * Cap amount of bookmarks per channel * add etag back to get channels * feedback review * update file info when replacing a bookmark file * return 501 not implemented when the license is not available * add detail message when getting channel member on bookmark api * start audit before permission check on create bookmark api * use require.Eventuallyf for testing WS events * remove unnecessary log in app layer * use require instead of assert to avoid panics * enforce limit when querying bookmarks since * prevent to create/update bookmark if file is already attached * fix lint * delete file when a bookmark is deleted * Dot allow to set a fileId and a url at the same time to a bookmark * fix query to delete a file that belongs to a bookmark * do not patch the bookmark type * Server side FeatureFlag check (#26145) * use ff in server, set ff to false * turn on FF for unit tests * defer unset FF for unit tests * turn ff on for testing * only allow attaching files that were uploaded for bookmark * Set feature flag off as default * fix lint * update email templates as PR failed * revert templates * force the assignment of ID when creating a bookmark * Fix unit tests --------- Co-authored-by: Miguel de la Cruz <miguel@mcrx.me> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Caleb Roseland <caleb@calebroseland.com> Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>	2024-03-12 08:36:05 -06:00
Doug Lauder	d56dc9d0ce	Add Options field to RemoteClusters (#25771) ... * add Option flag to RemoteClusters * add Options column to RemoteClusters table	2023-12-19 13:01:35 -05:00
Devin Binnie	109f4643c6	[MM-55017] Add API method to get users for Admin Reporting (#25499) ... * Add store method to get reporting data * Some store changes * Added app layer * Added API call, some miscellaneous fixes * Fix lint * Fix serialized check * Add API docs * Fix user store tests leaking users * Fix test * PR feedback * Add filtering for role/team/activated user, filter out bot users * Fix mock * Fix test * Oops * Switch to using struct filter * More PR feedback * Fix gen * Fix test * Fix API docs * Fix test * Fix possible SQL injection, some query optimization * Fix migrations * Oops * Add role to API * Fix check * Add Client4 API call for load testing * Fix test * Update server/channels/store/storetest/user_store.go Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> * PR feedback --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>	2023-12-08 10:30:08 -05:00
Doug Lauder	8bf9e4c481	Prep for MS Teams plugin API (#25565) ... * - columns added to ShareChannelRemotes: lastpostcreateat, lastpostupdateat - SyncMsg and SyncResponse moved to `model` package - field added to RemoteCluster struct: PluginID * sync new posts before updated posts to ensure post order in MS Teams * add plugid to remoteclusters table and store * don't sync history by default	2023-12-04 13:10:20 -05:00
Felipe Martin	b40366dbdf	[MM-55028] Added OAuthOutgoingConnection store (#25221) ... * OAuthOutgoingConnection model * added store * make generated * add missing license headers * fix receiver name * i18n * i18n sorting * update migrations from master * make migrations-extract * update retrylayer tests * replaced sql query with id pagination * fixed flaky tests * missing columns * missing columns on save/update * typo * improved tests * remove enum from mysql colum * add password credentials to store * renamed migrations * model change suggestions * refactor test functionsn * migration typo * refactor store table names * updated sanitize test * oauthoutgoingconnection -> outgoingoauthconnection * signature change * i18n update * granttype typo * uppercase typo * lowercase store name	2023-11-20 09:42:07 -05:00
Devin Binnie	1bd72bdb99	[MM-55014][MM-55015] Add last login timestamp for users, add materialized view and refresh job to keep track of post stats for Postgres (#25152) ... * [MM-55014][MM-55015] Add last login timestamp for users, add materialized view and refresh job for Postgres * Check fixes * Fix type issue * Add verification that lastlogin was updated * PR feedback * Morge'd * Morge'd again * Merge'd * Update admin setting strings * WIP * PR feedback * Oops * Fix i18n --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2023-11-14 11:26:27 -05:00
Ibrahim Serdar Acikgoz	1cd439f8e0	db/migrations: drop nextsyncat and description columns from shared remotes (#25064)	2023-10-31 15:25:16 +03:00
Ben Cooke	791ee40568	[MM-53291] Data retention improvements (#24253) ... * adding new migration for RetentionIdsForDeletion, changing logic for deleting orphaned reactions. Updating delete user and channel endpoints to remove respective reactions	2023-09-06 08:25:27 -04:00
Devin Binnie	a3b194581f	[MM-37984] Allow Desktop App to authenticate via external providers outside of the app on supported servers (#24140) ... * [MM-37984] Allow Desktop App to authenticate via external providers outside of the app on supported servers * PR feedback * Add support for mattermost-dev protocol for development use * Update server/channels/db/migrations/postgres/000110_create_desktop_tokens.up.sql * Fix silly typo * Update server/channels/db/migrations/postgres/000110_create_desktop_tokens.up.sql * Remove storage of client token, only validate it on the client * Update migrations * Add concurrently create index * Remove CONCURRENTLY for now * Fix issue with changing history * Remove old migration * Use idempotent statement to drop old index * Remove reference to old table	2023-08-30 11:21:43 -04:00
Agniva De Sarker	555009aa82	Bumping up migration number to 111 (#24341) ... PR https://github.com/mattermost/mattermost/pull/24039 was reverted which left an erroneous migration 110. So to workaround this, we create a new migration with number 111. ```release-note NONE ```	2023-08-23 21:15:53 +05:30
Agniva De Sarker	ce92ab7e1c	Update vacuuming parameters for some tables (#23997) ... After some analysis of vacuuming statistics of our large Postgres customers, it was observed that the default vacuuming thresholds for some tables are not optimal enough for our workload. We need more frequent vacuuming for some tables. The analysis showed that the top 4 tables that needed rectification were posts, preferences, threadmemberships and fileinfo. So we reduce the threshold by half for those tables, triggering autovacuum and autoanalyze doubly often. ```release-note NONE ```	2023-08-02 21:18:18 +05:30
Jesse Hallam	89e65257a6	Remove command palette POC (#24044) ... * Revert "POC: Cross-team recent search (#20027)" This reverts commit aa59c28b04, preserving a few code tidyings unrelated to the original PR. * Revert "Add feature flag for command palette (#20011)" This reverts commit c78c5ce3f3.	2023-07-24 10:31:06 -03:00