48 commits

Author	SHA1	Message	Date
Doug Lauder	96e4d7a769	MM-68076 Chunk bulk INSERTs to respect PostgreSQL parameter limit (#35767) ... * MM-68076 Chunk bulk INSERTs to respect PostgreSQL 65,535 parameter limit (#35761) PostgreSQL's wire protocol uses a 16-bit integer for parameter count, causing bulk imports to fail when multi-row INSERTs exceed 65,535 parameters. Add a generic chunkSlice helper that splits rows into sub-batches capped at 50,000 parameters, and apply it to saveMultipleMembers (channel), SaveMultipleMembers (team), and SaveMultipleMemberships (thread). Normal operations (< 3,333 rows) remain a single INSERT with negligible overhead. Wrap all chunked INSERT loops in transactions so multi-chunk batches are atomic — previously channel and team member inserts could leave partial data if a later chunk failed. Add threadMembershipSliceColumns helper so thread membership chunk size is derived dynamically. Includes integration tests for multi-chunk insertion and rollback verification for channel members, team members, posts, and groups.	2026-03-30 22:40:40 -04:00
Miguel de la Cruz	48f2fd0873	Merge the Integrated Boards MVP feature branch (#35796) ... * Add CreatedBy and UpdatedBy to the properties fields and values (#34485) * Add CreatedBy and UpdatedBy to the properties fields and values * Fix types --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds ObjectType to the property fields table (#34908) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Update ObjectType migration setting an empty value and marking the column as not null (#34915) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds uniqueness mechanisms to the property fields (#35058) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Fixing retrylayer mocks * Remove retrylayer duplication * Address review comments * Fix comment to avoid linter issues * Address PR comments * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.down.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update field validation to check only for valid target types * Update migrations to avoid concurrent index creation within a transaction * Update migrations to make all index ops concurrent * Update tests to use valid PSAv2 property fields * Adds a helper for valid PSAv2 TargetTypes --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Fix property tests (#35388) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards feature flag (#35378) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards MVP API changes (#34822) This PR includes the necessary changes for channels and posts endpoints and adds a set of generic endpoints to retrieve and manage property fields and values following the new Property System approach. Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> * Property System Architecture permissions for v2 (#35113) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Fix i18n sorting --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Add Views store and app layer (#35361) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor test loops in ViewStore tests for improved readability * change pagination to limit/offset * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * MM-67388, MM-66528, MM-67750: Add View REST API endpoints, websocket events, and sort order (#35442) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Add View API endpoints with OpenAPI spec, client methods, and i18n Implement REST API for channel views (board-type) behind the IntegratedBoards feature flag. Adds CRUD endpoints under /api/v4/channels/{channel_id}/views with permission checks matching the channel bookmark pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add cursor-based pagination to View API for channel views * Enhance cursor handling in getViewsForChannel and update tests for pagination * Refactor test loops in ViewStore tests for improved readability * Refactor loop in TestGetViewsForChannel for improved readability * change pagination to limit/offset * switch to limit/offset pagination * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add view sort order API endpoint Add POST /api/v4/channels/{channel_id}/views/{view_id}/sort_order endpoint following the channel bookmarks reorder pattern. Includes store, app, and API layers with full test coverage at each layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add connectionId to view WebSocket events and sort_order API spec Thread connectionId from request header through all view handlers (create, update, delete, sort_order) to WebSocket events, matching the channel bookmarks pattern. Add sort_order endpoint to OpenAPI spec. Update minimum server version to 11.6. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove duplicate View/ViewPatch definitions from definitions.yaml The merge from integrated-boards-mvp introduced duplicate View and ViewPatch schema definitions that were already defined earlier in the file with more detail (including ViewBoardProps ref and enums). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update minimum server version to 11.6 in views API spec Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add missing translations for view sort order error messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Merge integrated-boards-mvp into ibmvp_api-views; remove spec files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix flaky TestViewStore timestamp test on CI Add sleep before UpdateSortOrder to ensure timestamps differ, preventing same-millisecond comparisons on fast CI machines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove duplicate views.yaml imclude * Use c.boolString() for include_deleted query param in GetViewsForChannel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix views.yaml sort order schema: use integer type and require body Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor view sort order tests to use named IDs instead of array indices Extract idA/idB/idC from views slice and add BEFORE/AFTER comments to make stateful subtest ordering easier to follow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Return 404 instead of 403 for view operations on deleted channels Deleted channels should appear non-existent to callers rather than revealing their existence via a 403. Detailed error text explains the context for debugging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * add missing channel deleteat checks * Use c.Params.Page instead of manual page query param parsing in getViewsForChannel c.Params already validates and defaults page/per_page, so the manual parsing was redundant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add support for total count in views retrieval * Add tests for handling deleted views in GetViewsForChannel and GetView * Short-circuit negative newIndex in UpdateSortOrder before opening transaction Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add per-channel limit on views to bound UpdateSortOrder cost Without a cap, unbounded view creation makes sort-order updates increasingly expensive (CASE WHEN per view, row locks). Adds MaxViewsPerChannel=50 constant and enforces it in the app layer before saving. Includes API and app layer tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove include_deleted support from views API Soft-deleted views are structural metadata with low risk, but no other similar endpoint (e.g. channel bookmarks) exposes deleted records without an admin gate. Rather than adding an admin-only permission check for consistency, remove the feature entirely since there is no current use case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update view permissions to require `create_post` instead of channel management permissions * Remove obsolete view management error messages for direct and group messages --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(migrations): add user tracking and object type to property fields - Introduced user tracking columns (CreatedBy, UpdatedBy) to PropertyFields and PropertyValues. - Added ObjectType column to PropertyFields with associated unique indexes for legacy and typed properties. - Created new migration scripts for adding and dropping these features, including necessary indexes for data integrity. - Established views for managing property fields with new attributes. This update enhances the schema to support better tracking and categorization of property fields. * Add Property System Architecture v2 API endpoints (#35583) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Require target_type filter when searching property fields * Add objectType validation as part of field.IsValid() * Fix linter * Fix test with bad objecttpye * Fix test grouping --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * MM-67968: Flatten view model — remove icon, subviews, typed board props (#35726) * feat(views): flatten view model by removing icon, subview, and board props Simplifies the View data model as part of MM-67968: removes Icon, Subview, and ViewBoardProps types; renames ViewTypeBoard to ViewTypeKanban; replaces typed Props with StringInterface (map[string]any); adds migration 000167 to drop the Icon column from the Views table. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(api): update views OpenAPI spec to reflect flattened model Removes ViewBoardProps, Subview, and icon from the View and ViewPatch schemas. Changes type enum from board to kanban. Replaces typed props with a free-form StringInterface object. Aligns with MM-67968. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * refactor(views): simplify store by dropping dbView and marshalViewProps StringInterface already implements driver.Valuer and sql.Scanner, so the manual JSON marshal/unmarshal and the dbView intermediate struct were redundant. model.View now scans directly from the database. Also removes the dead ViewMaxLinkedProperties constant and wraps the Commit() error in UpdateSortOrder. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(api): allow arbitrary JSON in view props OpenAPI schema The props field was restricted to string values via additionalProperties: { type: string }, conflicting with the Go model's StringInterface (map[string]any). Changed to additionalProperties: true in View, ViewPatch, and inline POST schemas. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Adds basic implementation of the generic redux store for PSAv2 (#35512) * Adds basic implementation of the generic redux store for PSAv2 * Add created_by and updated_by to the test fixtures * Make target_id, target_type and object_type mandatory * Wrap getPropertyFieldsByIds and getPropertyValuesForTargetByFieldIds with createSelector * Address PR comments --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds websocket messages for the PSAv2 API events (#35696) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Adds websocket messages for the PSAv2 API events * Add IsPSAv2 helper to the property field for clarity * Add guard against nil returns on field deletion * Add docs to the websocket endpoints --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * migrations: consolidate views migrations and reorder after master - Merged 000165 (create Views) with 000167 (drop Icon) since Icon was never needed - Renumbered branch migrations 159-166 → 160-167 so master's 000159 (deduplicate_policy_names) runs first - Regenerated migrations.list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add API endpoint to retrieve posts for a specific view (#35604) Automatic Merge * Apply fixes after merge * Return a more specific error from getting multiple fields * Prevent getting broadcast params on field deletion if not needed * Remove duplicated migration code * Update property conflict code to always use master * Adds nil guard when iterating on property fields * Check that permission level is valid before getting rejected by the database * Validate correctness on TargetID for PSAv2 fields * Avoid PSAv1 using permissions or protected * Fix test data after validation change * Fix flaky search test * Adds more posts for filter use cases to properly test exclusions --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> Co-authored-by: Julien Tant <julien@craftyx.fr> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-27 10:36:35 +01:00
Ben Schumacher	60fbce7d03	[MM-67671] Add CJK Post search support for PostgreSQL (#35260) ... Add LIKE-based CJK (Chinese, Japanese, Korean) search support for PostgreSQL, gated behind a `CJKSearch` feature flag. PostgreSQL's built-in full-text search (`to_tsvector`/`to_tsquery`) does not support CJK languages because it relies on whitespace-based tokenization, which doesn't work for logographic/syllabic scripts that don't use spaces between words. This PR adds a `LIKE`-based fallback for search terms containing CJK characters. **How it works:** When the `CJKSearch` feature flag is enabled and a search term contains CJK characters (Han, Hiragana, Katakana, or Hangul), the query builder generates `LIKE '%term%'` clauses instead of `to_tsvector @@ to_tsquery` expressions. Case-sensitive `LIKE` is used rather than `ILIKE` since CJK scripts have no letter casing, which also allows potential use of standard B-tree indexes.	2026-02-25 10:18:51 -05:00
Jesse Hallam	41e5c7286b	Remove vestigial MySQL support (#34865) ... * Remove legacy quoteColumnName() utility Since Mattermost only supports PostgreSQL, the quoteColumnName() helper that was designed to handle database-specific column quoting is no longer needed. The function was a no-op that simply returned the column name unchanged. Remove the function from utils.go and update status_store.go to use the "Manual" column name directly. * Remove legacy driver checks from store.go Since Mattermost only supports PostgreSQL, remove conditional checks for different database drivers: - Simplify specialSearchChars() to always return PostgreSQL-compatible chars - Remove driver check from computeBinaryParam() - Remove driver check from computeDefaultTextSearchConfig() - Simplify GetDbVersion() to use PostgreSQL syntax directly - Remove switch statement from ensureMinimumDBVersion() - Remove unused driver parameter from versionString() * Remove MySQL alternatives for batch delete operations Since Mattermost only supports PostgreSQL, remove the MySQL-specific DELETE...LIMIT syntax and keep only the PostgreSQL array-based approach: - reaction_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch - file_info_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch - preference_store.go: Use PostgreSQL tuple IN subquery for DeleteInvalidVisibleDmsGms * Remove MySQL alternatives for UPDATE...FROM syntax Since Mattermost only supports PostgreSQL, remove the MySQL-specific UPDATE syntax that joins tables differently: - thread_store.go: Use PostgreSQL UPDATE...FROM syntax in MarkAllAsReadByChannels and MarkAllAsReadByTeam - post_store.go: Use PostgreSQL UPDATE...FROM syntax in deleteThreadFiles * Remove MySQL alternatives for JSON and subquery operations Since Mattermost only supports PostgreSQL, remove the MySQL-specific JSON and subquery syntax: - thread_store.go: Use PostgreSQL JSONB operators for updating participants - access_control_policy_store.go: Use PostgreSQL JSONB @> operator for querying JSON imports - session_store.go: Use PostgreSQL subquery syntax for Cleanup - job_store.go: Use PostgreSQL subquery syntax for Cleanup * Remove MySQL alternatives for CTE queries Since Mattermost only supports PostgreSQL, simplify code that uses CTEs (Common Table Expressions): - channel_store.go: Remove MySQL CASE-based fallback in UpdateLastViewedAt and use PostgreSQL CTE exclusively - draft_store.go: Remove driver checks in DeleteEmptyDraftsByCreateAtAndUserId, DeleteOrphanDraftsByCreateAtAndUserId, and determineMaxDraftSize * Remove driver checks in migrate.go and schema_dump.go Simplify migration code to use PostgreSQL driver directly since PostgreSQL is the only supported database. * Remove driver checks in sqlx_wrapper.go Always apply lowercase named parameter transformation since PostgreSQL is the only supported database. * Remove driver checks in user_store.go Simplify user store functions to use PostgreSQL-only code paths: - Remove isPostgreSQL parameter from helper functions - Use LEFT JOIN pattern instead of subqueries for bot filtering - Always use case-insensitive LIKE with lower() for search - Remove MySQL-specific role filtering alternatives * Remove driver checks in post_store.go Simplify post_store.go to use PostgreSQL-only code paths: - Inline getParentsPostsPostgreSQL into getParentsPosts - Use PostgreSQL TO_CHAR/TO_TIMESTAMP for date formatting in analytics - Use PostgreSQL array syntax for batch deletes - Simplify determineMaxPostSize to always use information_schema - Use PostgreSQL jsonb subtraction for thread participants - Always execute RefreshPostStats (PostgreSQL materialized views) - Use materialized views for AnalyticsPostCountsByDay - Simplify AnalyticsPostCountByTeam to always use countByTeam * Remove driver checks in channel_store.go Simplify channel_store.go to use PostgreSQL-only code paths: - Always use sq.Dollar.ReplacePlaceholders for UNION queries - Use PostgreSQL LEFT JOIN for retention policy exclusion - Use PostgreSQL jsonb @> operator for access control policy imports - Simplify buildLIKEClause to always use LOWER() for case-insensitive search - Simplify buildFulltextClauseX to always use PostgreSQL to_tsvector/to_tsquery - Simplify searchGroupChannelsQuery to use ARRAY_TO_STRING/ARRAY_AGG * Remove driver checks in file_info_store.go Simplify file_info_store.go to use PostgreSQL-only code paths: - Always use PostgreSQL to_tsvector/to_tsquery for file search - Use file_stats materialized view for CountAll() - Use file_stats materialized view for GetStorageUsage() when not including deleted - Always execute RefreshFileStats() for materialized view refresh * Remove driver checks in attributes_store.go Simplify attributes_store.go to use PostgreSQL-only code paths: - Always execute RefreshAttributes() for materialized view refresh - Remove isPostgreSQL parameter from generateSearchQueryForExpression - Always use PostgreSQL LOWER() LIKE LOWER() syntax for case-insensitive search * Remove driver checks in retention_policy_store.go Simplify retention_policy_store.go to use PostgreSQL-only code paths: - Remove isPostgres parameter from scanRetentionIdsForDeletion - Always use pq.Array for scanning retention IDs - Always use pq.Array for inserting retention IDs - Remove unused json import * Remove driver checks in property stores Simplify property_field_store.go and property_value_store.go to use PostgreSQL-only code paths: - Always use PostgreSQL type casts (::text, ::jsonb, ::bigint, etc.) - Remove isPostgres variable and conditionals * Remove driver checks in channel_member_history_store.go Simplify PermanentDeleteBatch to use PostgreSQL-only code path: - Always use ctid-based subquery for DELETE with LIMIT * Remove remaining driver checks in user_store.go Simplify user_store.go to use PostgreSQL-only code paths: - Use LEFT JOIN for bot exclusion in AnalyticsActiveCountForPeriod - Use LEFT JOIN for bot exclusion in IsEmpty * Simplify fulltext search by consolidating buildFulltextClause functions Remove convertMySQLFullTextColumnsToPostgres and consolidate buildFulltextClause and buildFulltextClauseX into a single function that takes variadic column arguments and returns sq.Sqlizer. * Simplify SQL stores leveraging PostgreSQL-only support - Simplify UpdateMembersRole in channel_store.go and team_store.go to use UPDATE...RETURNING instead of SELECT + UPDATE - Simplify GetPostReminders in post_store.go to use DELETE...RETURNING - Simplify DeleteOrphanedRows queries by removing MySQL workarounds for subquery locking issues - Simplify UpdateUserLastSyncAt to use UPDATE...FROM...RETURNING instead of fetching user first then updating - Remove MySQL index hint workarounds in ORDER BY clauses - Update outdated comments referencing MySQL - Consolidate buildFulltextClause and remove convertMySQLFullTextColumnsToPostgres * Remove MySQL-specific test artifacts - Delete unused MySQLStopWords variable and stop_word.go file - Remove redundant testSearchEmailAddressesWithQuotes test (already covered by testSearchEmailAddresses) - Update comment that referenced MySQL query planning * Remove MySQL references from server code outside sqlstore - Update config example and DSN parsing docs to reflect PostgreSQL-only support - Remove mysql:// scheme check from IsDatabaseDSN - Simplify SanitizeDataSource to only handle PostgreSQL - Remove outdated MySQL comments from model and plugin code * Remove MySQL references from test files - Update test DSNs to use PostgreSQL format - Remove dead mysql-replica flag and replicaFlag variable - Simplify tests that had MySQL/PostgreSQL branches * Update docs and test config to use PostgreSQL - Update mmctl config set example to use postgres driver - Update test-config.json to use PostgreSQL DSN format * Remove MySQL migration scripts, test data, and docker image Delete MySQL-related files that are no longer needed: - ESR upgrade scripts (esr.*.mysql.*.sql) - MySQL schema dumps (mattermost-mysql-*.sql) - MySQL replication test scripts (replica-*.sh, mysql-migration-test.sh) - MySQL test warmup data (mysql_migration_warmup.sql) - MySQL docker image reference from mirror-docker-images.json * Remove MySQL references from webapp - Simplify minimumHashtagLength description to remove MySQL-specific configuration note - Remove unused HIDE_MYSQL_STATS_NOTIFICATION preference constant - Update en.json i18n source file * clean up e2e-tests * rm server/tests/template.load * Use teamMemberSliceColumns() in UpdateMembersRole RETURNING clause Refactor to use the existing helper function instead of hardcoding the column names, ensuring consistency if the columns are updated. * u.id -> u.Id * address code review feedback --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2026-01-20 21:01:59 +00:00
Nick Misasi	8e4cadbc88	[MM-66359] Recaps MVP (#34337) ... * initial commit for POC of Plugin Bridge * Updates * POC for plugin bridge * Updates from collaboration * Fixes * Refactor Plugin Bridge to use HTTP/REST instead of RPC - Remove ExecuteBridgeCall hook and Context.SourcePluginId - Implement HTTP-based bridge using existing PluginHTTP infrastructure - Add CallPlugin API method with endpoint parameter instead of method name - Update CallPluginBridge to construct HTTP POST requests - Add proper headers: Mattermost-User-Id, Mattermost-Plugin-ID - Use 'com.mattermost.server' as plugin ID for core server calls - Update ai.go to use REST endpoint /inter-plugin/v1/completion - Add comprehensive spec documentation in server/spec.md - Add MIGRATION_GUIDE.md for plugin developers - Fix 401/404 issues by setting correct headers and URL paths * Improve Plugin Bridge security and architecture - Create ServeInternalPluginRequest for internal plugin calls (core + plugin-to-plugin) - Move header-setting logic from CallPluginBridge to ServeInternalPluginRequest - Improve separation of concerns: business logic vs HTTP transport - Add security documentation explaining header protection Security Improvements: - ServeInternalPluginRequest is NOT exposed as HTTP route (internal only) - Headers (Mattermost-User-Id, Mattermost-Plugin-ID) are set by trusted server code - External requests cannot spoof these headers (stripped by servePluginRequest) - Core calls use 'com.mattermost.server' as plugin ID for authorization - Plugin-to-plugin calls use real plugin ID (enforced by server) Backward Compatibility: - Keep ServeInterPluginRequest for existing API.PluginHTTP callers (deprecated) - All tests pass Docs: - Update spec.md with security model explanation - Update MIGRATION_GUIDE.md with correct header usage examples * Space * cursor please stop creating markdown files * Fix style * Fix i18n, linter * REMOVE MARKDOWN * Remove CallPlugin method from plugin API interface Per review feedback, this method is no longer needed. Co-authored-by: Nick Misasi <nickmisasi@users.noreply.github.com> * Remove CallPlugin method implementation from PluginAPI Co-authored-by: Nick Misasi <nickmisasi@users.noreply.github.com> * fixes * Add AI OpenAPI spec * fix openapi spec * Use agents client (#34225) * Use agents client * Remove default agent * Fixes * fix: modify system prompts to ensure JSON is being returned * Base implementation for recaps working * small fixes * Adjustments * remove webapp changes * Add feature flags for rewrites and ai bridge, clean up * Remove comments that aren't helpful * Fix i18n * Remove rewrites * Fix tests * Fix i18n * adjust i18n again * Add back translations * Remove leftover mock code * remove model file * Changes from PR review * Make the real substitutions * Include a basic invokation of the client with noop to ensure build works * more fix * Remove unneeded change * Updates from review * Fixes * Remove some logic from rewrites to clean up branch * Use v1.5.0 of agents plugin * A bunch more additions for general UX flow * Add missing files * Add mocks * Fixes for vet-api, i18n, build, types, etc * One more linter fix * Fix i18n and some tests * Refactors and cleanup in backend code * remove rogue markdown file * fixes after refactors from backend * Add back renamed files, and add tests * More self code review * More fixes * More refactors * Fix call stack exceeded bug * Include read messages if there are no unreads * Fix test failure: use correct error message key for recap permission denied The getRecapAndCheckOwnership function was using strings.ToLower(callerName) to generate error keys, which caused 'GetRecap' to become 'getrecap' instead of the expected 'get'. Changed to use the correct static key that matches the en.json localization file. Fixes TestGetRecap/get_recap_by_non-owner test failure. Co-authored-by: Nick Misasi <nickmisasi@users.noreply.github.com> * Consolidate permission errors down to a single string * Fixes for i18n, worktrees making this difficult * Fix i18n * Fix i18n once and for all (for real) (final) * Fix duplicate getAgents method in client4.ts * Remove duplicate ai state from initial_state.ts * Fix types * Fix tests * Fix return type of GetAgents and GetServices * Add tests for recaps components * Fix types * Update i18n * Fixes * Fixes * More cleanup * Revert random file * Use undefined * fix linter * Address feedback * Missed a git add * Fixes * Fix i18n * Remove fallback * Fixes for PR --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Nick Misasi <nickmisasi@users.noreply.github.com> Co-authored-by: Christopher Speller <crspeller@gmail.com> Co-authored-by: Felipe Martin <me@fmartingr.com> Co-authored-by: Mattermost Build <build@mattermost.com>	2026-01-13 11:59:22 -05:00
Ibrahim Serdar Acikgoz	084006c0ea	[MM-61758] Burn on read feature (#34703) ... * Add read receipt store for burn on read message types * update mocks * fix invalidation target * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * add translations * Added logic to associate files of BoR post with the post * Added test * fixes * disable pinning posts and review comments * MM-66594 - Burn on read UI integration (#34647) * MM-66244 - add BoR visual components to message editor * MM-66246 - BoR visual indicator for sender and receiver * MM-66607 - bor - add timer countdown and autodeletion * add the system console max time to live config * use the max expire at and create global scheduler to register bor messages * use seconds for BoR config values in BE * implement the read by text shown in the tooltip logic * unestack the posts from same receiver and BoR and fix styling * avoid opening reply RHS * remove unused dispatchers * persis the BoR label in the drafts * move expiration value to metadata * adjust unit tests to metadata insted of props * code clean up and some performance improvements; add period grace for deletion too * adjust migration serie number * hide bor messages when config is off * performance improvements on post component and code clean up * keep bor existing post functionality if config is disabled * Add read receipt store for burn on read message types * Add temporary posts table * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * avoid reacting to unrevealed bor messages * adjust migration number * Add read receipt store for burn on read message types * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * adjust post reveal and type with backend changes * use real config values, adjust icon usage and style * adjust the delete from from sender and receiver * improve self deleting logic by placing in badge, use burn endpoint * adjust websocket events handling for the read by sender label information * adjust styling for concealed and error state * update burn-on-read post event handling for improved recipient tracking and multi-device sync * replace burn_on_read with type in database migrations and model * remove burn_on_read metadata from PostMetadata and related structures * Added logic to associate files of BoR post with the post * Added test * adjust migration name and fix linter * Add read receipt store for burn on read message types * update mocks * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * Added logic to associate files of BoR post with the post * Added test * disable pinning posts and review comments * show attachment on bor reveal * remove unused translation * Enhance burn-on-read post handling and refine previous post ID retrieval logic * adjust the returning chunk to work with bor messages * read temp post from master db * read from master * show the copy link button to the sender * revert unnecessary check * restore correct json tag * remove unused error handling and clarify burn-on-read comment * improve type safety and use proper selectors * eliminate code duplication in deletion handler * optimize performance and add documentation * delete bor message for sender once all receivers reveal it * add burn on read to scheduled posts * add feature enable check * use master to avoid all read recipients race condition --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> * squash migrations into single file * add configuration for the scheduler * don't run messagehasbeenposted hook * remove parallel tests on burn on read * add clean up for closing opened modals from previous tests * simplify delete menu item rendering * add cleanup step to close open modals after each test to prevent pollution * streamline delete button visibility logic for Burn on Read posts * improve reliability of closing post menu and modals by using body ESC key --------- Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> Co-authored-by: Pablo Vélez <pablovv2012@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>	2025-12-11 07:59:50 +01:00
Elias Nahum	1022cd44c0	MM-65756 Database Migrations, Indexes and Methods for Auto-Translation (#34047) ... * AutoTranslate config settings * comment out Agents provider * Add auto translate timeout config validation * i18n messages for autotranslation config validation * fix test * validate url for libreTranslate * Feedback review * Admin Console UI for Auto-Translation * fix admin console conditional section display * i18n * removed unintentional change Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * update admin.general.localization.autoTranslateProviderDescription newline * fix lint * Fix types * UX feedback review * fix typo in i18n * Fix AutoTranslation feature flag * feedback review * Fix test default values * feedback review * re-add isHidden property to feature discovery * Database Migrations, Indexes and Methods for Auto-Translation * i18n * fix retrylayer and storetest * Fix search query * fix lint * remove the request.CTX and modify Translation model * fix lint and external url * Add settings to playwright * Add empty as a valid value for the Provider * Update jsonb queries * Fix queries and add model methods * fix go lint * go lint fix 2 * fix db migrations * feedback review + store cache * increase migration number * cleanup autotranslation store cache * use NULL as objectType for posts * fix bad merge * fix tests * add missing i18n * Switch prop bags column to boolean * fix lint * fix tests * Remove database search * use Builder methods --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: BenCookie95 <benkcooke@gmail.com>	2025-11-22 09:32:01 +08:00
Ben Schumacher	549be3d2b1	Remove context.Context from Store (#34413)	2025-11-07 08:41:43 +01:00
Ben Schumacher	892a7c9c69	Use golangci-lints's build-in modernize linter (#34341)	2025-11-04 12:09:11 +01:00
Harshil Sharma	3265054ad5	Migrate content flagging settings to database (#33989) ... * lint fix * CI * added new migration mocks * Used setup for tests * some comment * Removed unnecesseery nil check * Form validation * WIP tests * WIP tests * WIP tests * fix: mock content flagging config selector with correct reasons format Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> * fix: add mock for getContentFlaggingConfig in flag post modal test Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> * Updated error code order in API docs * removed empty files * Added tests * lint fixes * minor tweak * lint fix * type fix * fixed test * nit * test enhancements * API WIP * API WIP * creating values * creating content flagging channel and properties * Able to save properties * Added another property field * WIP * WIP * Added validations * Added data validations and hidden post if confifgured to * lint fixes * Added API spec * Added some tests * Added tests for getContentReviewBot * test: add comprehensive tests for getContentReviewChannels function * Added more app layer tests * Added TestCanFlagPost * test: Add comprehensive tests for FlagPost function * Added all app layer tests * Removed a file that was reamoved downstream * test: add content flagging test file * test: add comprehensive tests for FlagContentRequest.IsValid method * Added model tests * test: add comprehensive tests for SqlPropertyValueStore.CreateMany * test: add comprehensive tests for flagPost() API function * Added API tests * linter fix * WIP * sent post flagging confirmation message * fixed i18n nissues * fixed i18n nissues * CI * WIP * WIP * Added API call * test: add test for Client4.flagPost API call in FlagPostModal * fix: remove userEvent.setup() from flag post modal test * test: wrap submit button click in act for proper state updates * Updated tests * lint fix * Updated test * fix: reset contentFlaggingGroupId for test isolation in content flagging tests * removed cached group ID * removed debug log * CI * Updated to allow special characters in comments * Handled empty comment * Created getContentFlaggingFields API * created getPostPropertyValues API * Used finally * WIP * Created useContentFlaggingFields hook * WIP * WIP * Added option to retain data for reviewers * Displayed deleted post's preview * DIsplayed all properties * Adding field name i18n * WIP - managing i18n able texts * Finished displaying all fields * Manual cleanup * lint fixes * team role filter logic fix * Fixed tests * created new API to fetch flagged posts * lint fix * Added new client methods * test: add comprehensive tests for content flagging APIs * Added new API tests * fixed openapi spec * Fixed DataSpillageReport tests * Fixed PostMarkdown test * Fixed PostPreviewPropertyRenderer test * Added metadata to card renderer * test fixes * Added no comment placeholder * Added view detail button * Created RemoveFlaggedMessageConfirmationModal modal * Added key and remove flag request modal * IMplemented delete flagged post * Handled edge cases of deleting flagged post * keep message * UI integration * Added WS event for post report update and handled deleted files of flagged post * Added error handling in keep/remove forms * i18n fixes * Fixed test * Updated OpenAPI specs * fixed types * fixed types * refactoring * refactor: improve test mocking for data spillage report component * test mock updates * Fixed tests * Updated reducer * not resetting mocks * Added migrations for content flagging tables * Created new structure * review fixes * Used correct ot name * WIP * review fixes * review fixes * Added new property translations * CI * CI * CI * Improved test * fixed test * CI * New UI component * WIP * Updated settings APIs * cached DB data * used cached reviewer data * Updated tests * Lint fixes * test: add tests for saveContentFlaggingSettings and getContentFlaggingSettings APIs * test fix * test: add tests for SaveContentFlaggingConfig and GetContentFlaggingConfigReviewerIDs * Updated tests * test: add content flagging test for local cache layer * test: add comprehensive tests for content flagging store cache * Updated tests * lint fix * Updated mobile text * Added content flagging SQL store mocks * Added API specs for new APIs * fixed tests * feat: add TestContentFlaggingStore function for content flagging store testing * feat: add comprehensive tests for content flagging store * Added SQL store tests * test: add content flagging test for local cache layer * test: add tests for content flagging store caching * Added cache layer tests * Updated tests * Fixed * Handled JSON error * fixes * fixes * Fixed retry layer test * fixerdf i18n * Fixed test * CI * building index concurrently * CI * fixed a test * CI * cleanup * Integrate flag post api (#33798) * WIP * WIP * Added API call * test: add test for Client4.flagPost API call in FlagPostModal * fix: remove userEvent.setup() from flag post modal test * test: wrap submit button click in act for proper state updates * Updated tests * lint fix * CI * Updated to allow special characters in comments * Handled empty comment * Used finally * CI * Fixed test * Spillage card integration (#33832) * Created getContentFlaggingFields API * created getPostPropertyValues API * WIP * Created useContentFlaggingFields hook * WIP * WIP * Added option to retain data for reviewers * Displayed deleted post's preview * DIsplayed all properties * Adding field name i18n * WIP - managing i18n able texts * Finished displaying all fields * Manual cleanup * lint fixes * team role filter logic fix * Fixed tests * created new API to fetch flagged posts * lint fix * Added new client methods * test: add comprehensive tests for content flagging APIs * Added new API tests * fixed openapi spec * Fixed DataSpillageReport tests * Fixed PostMarkdown test * Fixed PostPreviewPropertyRenderer test * Added metadata to card renderer * test fixes * Added no comment placeholder * Fixed test * refactor: improve test mocking for data spillage report component * test mock updates * Updated reducer * not resetting mocks * WIP * review fixes * CI * Fixed * fixes * Content flagging actions implementation (#33852) * Added view detail button * Created RemoveFlaggedMessageConfirmationModal modal * Added key and remove flag request modal * IMplemented delete flagged post * Handled edge cases of deleting flagged post * keep message * UI integration * Added WS event for post report update and handled deleted files of flagged post * Added error handling in keep/remove forms * i18n fixes * Updated OpenAPI specs * fixed types * fixed types * refactoring * Fixed tests * review fixes * Added new property translations * Improved test * fixed test * CI * fixes * CI * fixed a test * CI * Review fixes --------- Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>	2025-10-13 12:24:01 +05:30
Alejandro García Montoro	6eacd7c534	Bump Postgres minimum supported version to 14 (#34010)	2025-10-01 09:49:05 +02:00
Agniva De Sarker	9dd8c056e7	MM-63368: Remove MySQL (#33458) ... https://mattermost.atlassian.net/browse/MM-63368 ```release-note Remove MySQL support from the codebase entirely. ```	2025-07-22 20:40:55 +05:30
Ben Schumacher	04a60b6609	[MM-57693] Add schema dump to Support Packet (#31162) ... Co-authored-by: Claude <noreply@anthropic.com>	2025-06-19 11:33:55 +02:00
Jesse Hallam	31a8047973	Disable morph logging during TestMain (#30948) ... * rm "No TEST_DATABASE... override" log message Let's only log if this value is actually overridden. * rm "(Created|Dropped) temporary database" message * only log "Pinging SQL" on subsequent attempts * disable morph logging from TestMain * Fix style issues in store test files - Add missing parameter to migrate() function calls in tests - Remove unused log function in settings.go - Fix formatting with go fmt 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * within sqlstore, use "enable" vs "disable" for clarity * remove trailing newline from morph logs --------- Co-authored-by: Claude <noreply@anthropic.com>	2025-05-21 17:31:18 +00:00
Ibrahim Serdar Acikgoz	a344b3225b	[MM-61756] Attribute Based Access Control - Phase 1 (#30785) ... Attribute Based Access Control - Base * MM-63662 * MM-63919 * MM-63954 * MM-63955 * MM-63425 * MM-63426 * MM-63458 * MM-63459 * MM-63603 * MM-63845 * MM-64146 * MM-64199 * MM-64201 * MM-64233 * MM-64247 * MM-64268 --------- Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> Co-authored-by: Pablo Andrés Vélez Vidal <pablovv2012@gmail.com> Co-authored-by: abhijit-singh <abhijitsingh0702@gmail.com> Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>	2025-05-15 11:33:08 +02:00
Ibrahim Serdar Acikgoz	10b1f4c5ac	[MM-63428] add access control policy store (#30597)	2025-04-02 13:39:28 +02:00
Agniva De Sarker	a51a8e9b60	MM-62964: Skip setting replica lag handle in case of an error (#30498) ... Previously, we would set it to nil pointer which would eventually cause a panic when ReplicaLagAbs/ReplicaLagTime would get called. https://mattermost.atlassian.net/browse/MM-62964 ```release-note NONE ``` Co-authored-by: Mattermost Build <build@mattermost.com>	2025-03-20 11:38:29 +05:30
Agniva De Sarker	341f919fa8	MM-62782: Bumping minimum Postgres version to 13 (#30039) ... As discussed in https://docs.mattermost.com/install/prepare-mattermost-database.html#minimum-supported-version-policy https://mattermost.atlassian.net/browse/MM-62782 ```release-note Minimum postgres version has been bumped to 13. ``` Co-authored-by: Mattermost Build <build@mattermost.com>	2025-01-31 09:57:35 +05:30
Miguel de la Cruz	ecdce71fc4	Adds the main Property System Architecture components (#29644) ... * Adds the main Property System Architecture components This change adds the necessary migrations for the Property Groups, Fields and Values tables to be created, the store layer and a Property Service that can be used from the app layer. * Update property field type to use user instead of person * Update PropertyFields to allow for unique nondeleted fields and remove redundant indexes * Update PropertyValues to allow for unique nondeleted fields and remove redundant indexes * Use StringMap instead of the map[string]any on property fields * Add i18n strings * Revert "Use StringMap instead of the map[string]any on property fields" This reverts commit e2735ab0f8. * Cast JSON binary data to string and add todo note for StringMap use * Add mocks to the retrylayer tests * Cast JSON binary data to string in property value store * Check for binary parameter instead of casting to string for JSON data * Check property field type is one of the allowed ones * Avoid reusing err variable to be explicit about the returned value * Merge Property System Migrations into one file * Adds NOT NULL to timestamps at the DB level * Update stores to use tableSelectQuery instead of a slice var * Update PropertyField model translations to be more explicit and avoid repetition * Update PropertyValue model translations to be more explicit and avoid repetition * Use ExecBuilder instead of ToSql&Exec * Update property field errors to add context * Ensure PerPage is greater than zero * Update store errors to give more context * Use ExecBuilder in the property stores where possible * Add an on conflict suffix to the group register to avoid race conditions * Remove badly used translation string * Remove unused get in register group method --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2025-01-13 11:41:44 +00:00
Christopher Poile	aba4434dab	MM-59966 - Compliance Export overhaul - feature branch (#29789) ... * [MM-59089] Add a compliance export constant (#27919) * add a useful constant * i18n * another constant * another i18n * [MM-60422] Add GetChannelsWithActivityDuring (#28301) * modify GetUsersInChannelDuring to accept a slice of channelIds * add GetChannelsWithActivityDuring * add compliance export progress message; remove unused custom status * linting * tests running too fast * add batch size config settings * add store tests * linting * empty commit * i18n changes * fix i18n ordering * MM-60570 - Server-side changes consolidating the export CLI with server/ent code (#28640) * add an i18n field; add the CLI's export directory * int64 -> int * Add UntilUpdateAt for MessageExport and AnalyticsPostCount to merge * remove now-unused i18n strings * add TranslationsPreInitFromBuffer to allow CLI to use i18n * use GetBuilder to simplify; rename TranslationsPreInitFromFileBytes * [MM-59089] Improve compliance export timings (#1733 - Enterprise repo) * MM-60422 - Performance and logic fixes for Compliance Exports (#1757 - Enterprise repo) * MM-60570 - Enterprise-side changes consolidating the export CLI with server/ent code (#1769 - Enterprise repo) * merge conflicts; missed file from ent branch * MM-61038 - Add an option to sqlstore.New (#28702) remove useless comment add test add an option to sqlstore.New * MM-60976: Remove RunExport command from Mattermost binary (#28805) * remove RunExport command from mattermost binary * remove the code it was calling * fix i18n * remove test (was only testing license, not functionality) * empty commit * fix flaky GetChannelsWithActivityDuring test * MM-60063: Dedicated Export Filestore fix, redo of #1772 (enterprise) (#28803) * redo filestore fix #1772 (enterprise repo) on top of MM-59966 feature * add new e2e tests for export filestore * golint * ok, note to self: shadowing bad, actually (when there's a defer) * empty commit * MM-61137 - Message export: Support 7.8.11 era dbs (#28824) * support 7.8.11 era dbs by wrapping the store using only what we need * fix flaky GetChannelsWithActivityDuring test * add a comment * only need to define the MEFileInfoStore (the one that'll be overridden) * blank commit * MM-60974 - Message Export: Add performance metrics (#28836) * support 7.8.11 era dbs by wrapping the store using only what we need * fix flaky GetChannelsWithActivityDuring test * add a comment * only need to define the MEFileInfoStore (the one that'll be overridden) * performance metrics * cleanup unneeded named returns * blank commit * MM-60975 - Message export: Add startTime and endTime to export folder name (#28840) * support 7.8.11 era dbs by wrapping the store using only what we need * fix flaky GetChannelsWithActivityDuring test * add a comment * only need to define the MEFileInfoStore (the one that'll be overridden) * performance metrics * output startTime and endTime in export folder * empty commit * merge conflict * MM-60978 - Message export: Improve xml fields; fix delete semantics (#28873) * support 7.8.11 era dbs by wrapping the store using only what we need * fix flaky GetChannelsWithActivityDuring test * add a comment * only need to define the MEFileInfoStore (the one that'll be overridden) * performance metrics * output startTime and endTime in export folder * empty commit * add xml fields, omit when empty, tests * fix delete semantics; test (and test for update semantics) * clarify comments * simplify edited post detection, now there's no edge case. * add some spacing to help fast running tests * merge conflicts/updates needed for new deleted post semantics * linting; fixing tests from upstream merge * use SafeDereference * linting * stronger typing; better wrapped errors; better formatting * blank commit * goimports formatting * fix merge mistake * minor fixes due to changes in master * MM-61755 - Simplifying and Support reporting to the db from the CLI (#29281) * finally clean up JobData struct and stringMap; prep for CLI using db * and now simplify using StringMapToJobDataWithZeroValues * remove unused fn * create JobDataExported; clean up errors * MM-60176 - Message Export: Global relay cleanup (#29168) * move global relay logic into global_relay_export * blank commit * blank commit * improve errors * MM-60693 - Refactor CSV to use same codepath as Actiance (#29191) * move global relay logic into global_relay_export * blank commit * refactor (and simplify) ExportParams into shared * blank commit * remove unused fn * csv now uses pre-calculated joins/leaves like actiance * improve errors * remove nil post check; remove ignoredPosts metric * remove unneeded copy * MM-61696 - Refactor GlobalRelay to use same codepath as Actiance (#29225) * move global relay logic into global_relay_export * blank commit * refactor (and simplify) ExportParams into shared * blank commit * remove unused fn * csv now uses pre-calculated joins/leaves like actiance * remove newly unneeded function and its test. goodbye. * refactor GetPostAttachments for csv + global relay to share * refactor global_relay_export and fix tests (no changes to output) * improve errors * remove nil post check; remove ignoredPosts metric * remove unneeded copy * remove unneeded nil check * PR comments * MM-61715 - Generalize e2e to all export types 🤖 (#29369) * move global relay logic into global_relay_export * blank commit * refactor (and simplify) ExportParams into shared * blank commit * remove unused fn * csv now uses pre-calculated joins/leaves like actiance * remove newly unneeded function and its test. goodbye. * refactor GetPostAttachments for csv + global relay to share * refactor global_relay_export and fix tests (no changes to output) * improve errors * remove nil post check; remove ignoredPosts metric * remove unneeded copy * remove unneeded nil check * PR comments * refactor isDeletedMsg for all export types * fix start and endtime, nasty csv createAt bug; bring closer to Actiance * align unit tests with new logic (e.g. starttime / endtime) * refactor a TimestampConvert fn for code + tests * bug: pass templates to global relay (hurray for e2e tests, otherwise...) * add global relay zip to allowed list (only for tests) * test helpers * new templates for e2e tests * e2e tests... phew. * linting * merge conflicts * unexport PostToRow; add test helper marker * cleanup, shortening, thanks to PR comments * MM-61972 - Generalize export data path - Actiance (#29399) * extract and generalize the export data generation functions * finish moving test (bc of previous extraction) * lift a function from common -> shared (to break an import cycle) * actiance now takes general export data, processes it into actiance data * bring tests in line with correct sorting rules (upadateAt, messageId) * fixups, PR comments * turn strings.Repeat into a more descriptive const amended: one letter fix; bad rebase * MM-62009 - e2e clock heisenbug (#29434) * consolidate assertions; output debuggable diffs (keeping for future) * refactor test output generator to generators file * waitUntilZeroPosts + pass through until to job = fix all clock issues * simplify messages to model.NewId(); remove unneeded waitUntilZeroPosts * model.NewId() -> storetest.NewTestID() * MM-61980 - Generalize export data path - CSV (#29482) * simple refactoring * increase sleep times for (very) rare test failures * add extra information to the generic export for CSV * adj Actiance to handle new generic export (no difference in its output) * no longer need mergePosts (yay), move getJoinLeavePosts for everyone * adjust tests for new csv semantics (detailed in summary) * and need to add the new exported data to the export_data_tests * rearrange csv writing to happen after data export (more logical) * linting * remove debug statements * figured out what was wrong with global relay e2e test 3; solid now * PR comments * MM-61718 - Generalize export data path - Global Relay (#29508) * move global relay over to using the generalized export data * performance pass -- not much can be done * Update server/enterprise/message_export/global_relay_export/global_relay_export.go Co-authored-by: Claudio Costa <cstcld91@gmail.com> --------- Co-authored-by: Claudio Costa <cstcld91@gmail.com> * MM-62058 - Align CSV with Actiance (#29551) * refactoring actiance files and var names for clarity * bug found in exported attachments (we used to miss some start/ends) * changes needed for actiance due to new generic exports * bringing CSV up to actiance standards * fixing global relay b/c of new semantics (adding a note on an edge case) * aligning e2e tests, adding comments to clarify what is expected/tested * necessary changes; 1 more test for added functionality (ignoreDeleted) * comment style * MM-62059 - Align Global Relay with Actiance/CSV; many fixes (#29665) * core logic changes to general export_data and the specific export paths * unit tests and e2e tests, covering all new edge cases and all logic * linting * better var naming, const value, and cleaning up functions calls * MM-62436 - Temporarily skip cypress tests that require download link (#29772) --------- Co-authored-by: Claudio Costa <cstcld91@gmail.com>	2025-01-10 16:56:02 -05:00
Jesse Hallam	5369f8b36b	s/Get(Master|Replica)X/Get\1/g (#29520) ... Drop the legacy `X` suffix from `GetMasterX` and `GetReplicaX`. The presence of the suffix suggests there's a `non-X` version: but in fact we migrated these away a long time ago, so remove the cognitive overhead. As an aside, this additionally helps avoid trip up LLMs that interpret this as "something to fix".	2024-12-10 09:57:19 -04:00
Agniva De Sarker	63c959f1d3	Fail fast while connecting to replicas (#29212) ... Since we have the auto-retry mechanism for replicas, there is not much benefit in trying 5 times before deciding to move on. In the earlier model, we would fail the server startup, so it made sense to check as many times as possible. Also reducing the sleep interval so that we can improve the boot up time in case of a bad replica. ```release-note NONE ``` * fix: Add job name to the publish report step --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Antonis Stamatiou <stamatiou.antonis@gmail.com>	2024-11-14 09:45:52 +05:30
Harshil Sharma	e281b3f37e	Feature scheduled messages (#28932) ... * Create scheduled post api (#27920) * Added migration files for Postgres * Added migrations for MySQL * Added store method * Added API and store tests * Renamed migration after syncing with master * Added app layer tests * API is ready * API is ready * API is ready * Renamed migration after syncing with master * Updated migration list * Fixed retry layer tests * Allowed posts with empty messages * Review fixes * Reverted an incorrect change * Renamed migration and fixed ID assignment * CI * Send post button changes (#28019) * added Split button * WIP * Added core menu options * WIP * WIP * WIP * Handled displaying error in creating scheduled post * lint fixes * webapp i18n fix * Review fixes * Fixed a webapp test * A few more fixes * Removed a duplicate comment * Scheduled post job (#28088) * Added the job function * Added query for fetching scheduled posts for pricessing * WIP * WIP * WIP * WIP * WIP * WIP * Reafactoring of scheduled post job * Lint fixes * Updated i18n files * FInishing touches * Added tests for GetScheduledPosts * Added tests for PermanentlyDeleteScheduledPosts * Updated all layer * Some changes as discussed with team * Added tests for UpdatedScheduledPost * Code review refactoring * Added job test * MM-60120 - Custom time selection (#28120) * Added a common date time picker modal and used it for post reminder * Added a common date time picker modal and used it for post reminderggp * Added modal for custom schedule time and fixed TZ issue * WIP * Removed event from useSubmit hook * Removed event from useSubmit hook * Added timezone handling * fixed type error * Updated i18n strings * Minor cleanup * updated snapshots * review fixes * Handled event * Supported for having a DM thread open in RHS while in a regular channel * Review fixes * MM-60136 - Scheduled messages tab (#28133) * WIP * WIP * Created Tabs and Tab wrapper with added styling * Added API to get scheduled posts * WIP * Displated scheduled post count * i18n fix * Added tests * Handled asetting active tab absed on URL: * Reverted unintended change * Added API to client ad OpenAPI specs * Renamed file * Adding fileinfo to schedule posts * Partial review fixes * Made get scheduled post API return posts by teamID * review fixes * Moved scheduled post redux code to MM-redux package * Usedd selector factory * WIP: * WIP: * Lint fix * Fixed an incorrect openapi spec file * Removed redundent permission check * Clreaed scheduled post data on logout * Removed unused i18n string: * lint fix * Render scheduled posts (#28208) * WIP * WIP * Created Tabs and Tab wrapper with added styling * Added API to get scheduled posts * WIP * Displated scheduled post count * i18n fix * Added tests * Handled asetting active tab absed on URL: * Reverted unintended change * Added API to client ad OpenAPI specs * Renamed file * Created common component for draft list item * WIP * WIP * Adding fileinfo to schedule posts * Basic rendering * Added count badge to tabs * WIP * Made the Drafts LHS iteam appear if no drafts exist but scheduled posts do * Fixed icon size * Partial review fixes * Made get scheduled post API return posts by teamID * Handled initial vs team switch load * Displayed scheduled date in panel header * Added error message and error indiocator * WIP * review fixes * WIP Adding error reason tag * Added error codes * Moved scheduled post redux code to MM-redux package * Usedd selector factory * WIP: * WIP: * Lint fix * Fixed an incorrect openapi spec file * Removed redundent permission check * Clreaed scheduled post data on logout * Removed unused i18n string: * lint fix * Opened rescheduling modal * Updated graphic for empty state of schduled post list * Added delete scheduled post option and modal * Badge and timezone fix * WIP: * Added send now confirmation modal * lint * Webapp i18n fix * Fixed webapp test * Fixed a bug where DM/GM scheduled posts weren't immideatly showing up in UI * Minor fixes * WIP * Review fixes * Review fixes * Optimisations * Fixed reducer name * Moment optimizatin * Updated route check * MM-60144 - added API to update a scheduled post (#28248) * WIP * Added api and ap layer for update scheduled post ̛̦̄ * Added API to OpenAI specs, Go client and TS client * removed permissio check * Added tests * Fixed tests * Added PreUpdate method on scheduled post model * MM-60131 - Reschedule post integration (#28281) * Handled rescheduling post in webapp * Added error handling * MM-60146 - Delete scheduled post api (#28265) * WIP * Added api and ap layer for update scheduled post ̛̦̄ * Added API to OpenAI specs, Go client and TS client * removed permissio check * Added tests * Fixed tests * Added PreUpdate method on scheduled post model * Added delete scheduled post API * Added API to Go client and OpenAPI specs * Added API to TS client * Added tests * CI * Rmeoved two incorrect code comments * MM-60653 - Integrated delete scheduled post API (#28296) * Integrated delete scheduled apost API * Lint fix * Review fixes * Excluded draft checks from scheduled posts (#28370) * Excluded draft checks from scheduled posts * Added a removed todo * MM-60125 - Scheduled post channel indicator (#28320) * Integrated delete scheduled apost API * Lint fix * Added state for storing scheduled posts by channel ID * Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere * Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere * WIP * Added scheduled post indiocator * Handled single and multiple scheudled posts * Review fixes * Fixed styling and handled center channel, RHS and threads view * Lint fix * i18n fix * Fixed a cycling dependency * Lint fix * Added some more comments * Updated styling * Review fixes * Added common component for remote user time and scheduled post indicator * Updated scheduled post count * Minor change * Moved CSS code around * Fixed a bug where files in scheduled post didn't show up until refresh (#28359) --------- Co-authored-by: Daniel Espino García <larkox@gmail.com> * Scheduled post config (#28485) * Added config * Added config on server and webapp side * Added config check in server and webapp * Added license check * Added license check * Added placeholder help text * Added license check to job * Fixed job test * Review fixes * Updated English text * Review fixes * MM-60118 - Added index on ScheduledPosts table (#28579) * Added index * Updated indexes * Scheduled posts misc fixes (#28625) * Added detailed logging for scheduled post job * Limited scheduled posts processing to 24 hours * Marked old scheduled posts as unable to send * Added t5ests * converted some logs to trace level * Fixed a bug causing error message to show up on deleting a scheduled post in a deleted thread (#28630) * Fixed scheduled posts link in RHS (#28659) * Fixed scheduled posts link in RHS * Review fixes * Fix permission name in scheduled posts by team (#28580) * Fix permission name * fix wording --------- Co-authored-by: Mattermost Build <build@mattermost.com> * FIxed width of generic modal header to fix browser channel modal (#28639) * Only consider error-free scheduled posts for indicator in channel and RHS (#28683) * Show only errro free scheudled posts in post box indicator * Fixed a bug to handle no scheduled posts * Fixed draft and scheudled post UI in mobile view (#28680) * MM-60873 and MM-60872 - Fixed a bug with updating scheduled posts (#28656) * Fixed a bug with updating scheduled posts * Better selectors * MOved shceuled post message length validation to app layer * MM-60732 - Scheduled posts channel link now takes you to the first scheduled post in channel/thread in list (#28768) * Ordered scheudle dposts by schgeudled at nad create at * Ordered in client * Added scroll to target * Removed classname prop * Fixed tests * Added doc * Import fix * MM-60961 - Fixed a bug where API used incoming create at date for scheduled post (#28703) * Fixed a bug where API used incoming create at date for scheduled post * Stopped sending created at value for scheduled post * MM-60785 - Fixed a bug where scheduled posts of channel we are no longer member of didn't show up (#28637) * Fixed a bug where scheduled posts of channel we are no longer member of didn't show up * Added a comment * CI * Used data loader to optimise laoding missing channels * Minor refactoring * MM-60963 - Added common checks for post and scheduled posts (#28713) * Added commen checks for post and scheuled posts * Sanitised scheduled posts * Fixed tests * Splitted post checks into app and context functions * Added checks on scheduiled posts job as well: * i18n fix * Fixed a test * Renamed a func * removed duplicate check * Scheduled posts UI fixes (#28828) * Fixed send button and time picker borders * Fixed center alignment of time picker * Removed on for today and tomorrow * Lint fix * Date time modal hover state fix * Badge fix * Fixed a mnerge issue * Scheduled Post send now and add schedule on draft (#28851) * Added send now option on scheduled posts * Minor refactoring * WIP * WIP * WIP * Lint fix * i18n fix * Snapshot update * Review fixes * Scheduled post inline editing (#28893) * Added send now option on scheduled posts * Minor refactoring * WIP * WIP * WIP * Lint fix * i18n fix * Snapshot update * Displayed editing component in scheduled post * Added handling for updating scheduled post * Handle events * Fixed escape key issue in scheudled post editing * Fixes * Displayed error message for editing error * Don't show mention warning * Handled dev mode (#28918) * MInor fixes * client fix * Fixes * CI * Removed dev mode behaviour temperorily (#29008) --------- Co-authored-by: Daniel Espino García <larkox@gmail.com> Co-authored-by: Eva Sarafianou <eva.sarafianou@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>	2024-11-04 11:39:35 +05:30
Nick Misasi	f1019d076e	[CLD-7567] Deprecate Self Serve: Second Pass (#26853) ... * Deprecate Self Serve: First Pass * Fix ci * Fix more ci * Remmove outdated server tests * Fix a missed spot opening purchase modal in Self Hosted * Fix i18n * Clean up some more server code, fix webapp test * Fix alignment of button * Fix linter * Fix i18n server side * Deprecate in product true up * Add back translation * Remove client functions * Put back client functions * webapp deprecation * Deprecate Self Serve: Second Pass * Fix various pipeline issues * Fix linter * Fix pipelines * Fix handlers_test.go * Fix console.error around hostedCustomer in reducer * PICKY LINTER PLEASE * Fix webapp tests, various other fixes for the CI pipelines * Fix i18n * Updates to accomadate enterprise code removal * Fix mocks * More removal * Fix * Adjustments from PR * Fixes for QA Feedback * Update * Add migrations to remove true up review history * Fix migrations check --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: maria.nunez <maria.nunez@mattermost.com>	2024-05-02 09:15:15 -04:00
Agniva De Sarker	9431239b2e	MM-57378: Bump up golangci version (#26535) ... https://mattermost.atlassian.net/browse/MM-57378 ```release-note NONE ``` Co-authored-by: Mattermost Build <build@mattermost.com>	2024-03-22 10:23:21 +05:30
Elias Nahum	7e9cd04a8b	Channel Bookmarks (#25449) ... * create ChannelBookmarks table * ChannelBookmark model * channel bookamrks Store layer * add GetBookmarksForAllChannelByIdSince * add channel bookmarks to test store * Add channel bookmarks to app layer * remove index for createAt in channel bookmarks migrations * remove createAt from select channel bookmark query and enable store delete bookmark test * update reponse of UpdateBookmark * rename db migration files * channel bookmarks store update sort order * channel bookmarks app layer update sort order * fix lint & tests * Fix lint and introduce util functions to insert / remove from slice * remove model etag * i18n * defer remove file info after test run * Fix tests passing the request context * fix migrations * fix TestRetry * Add bookmark permissions (#25560) * Adds channel bookmarks permissions * Fix linter * Remove unnecessary empty lines * Remove scss change as it's not necessary anymore * Fix mock store * Fix mock store and add role entry * Fix test * Adds cypress test and update permissions migration to update admin roles * Adds channel bookmarks roles to default admin roles * Adds bookmark permissions to default role permissions constant in webapp * Update mmctl test * Update permission test after normalising the roles * fix store tests * fix app layer tests * Add new bookmark endpoint (#25624) * Adds channel bookmarks api scaffold and create endpoint * Applies review comments to the API docs * Adds websocket test to create channel bookmark --------- Co-authored-by: Mattermost Build <build@mattermost.com> * MM-54426 exclude Channel Bookmarks files from data retention (#25656) * Augment channel APIs to include bookmarks (#25567) * update files docs for server 9.4 * Adds update channel bookmark endpoint (#25653) * Adds update channel bookmark sort order endpoint (#25686) * Adds update channel bookmark endpoint * Updates edit app method to return the right deleted bookmark and adds tests * Adds the update channel bookmark sort order endpoint * Fix repeated test after merge * Assign right permissions to each test * Update store and app layer to return specific errors and add tests * Adds delete channel bookmark endpoint (#25693) * Updates edit app method to return the right deleted bookmark and adds tests * Fix repeated test after merge * Updates edit app method to return the right deleted bookmark and adds tests * Adds delete channel bookmark endpoint * Adds list channel bookmarks endpoint (#25700) * Add channel moderation to bookmarks (#25716) * fix migrations index * fix getChannelsForTeamForUser * fix getChannelsForTeamForUser * fix bad merge client4 * fix file api with bookmark permission * add ChannelBookmarks feature flag * add missing translations * Set DB column for type as enum * use custom type for bookmark query using sqlx * use transaction when saving bookmark * return NewErrNotFound instead of Sql.ErrNoRows * use squirrel for IN query * add a limit of 1K for records in GetBookmarksForAllChannelByIdSince * UpdateSortOrder with one single query instead of multiple updates * fix shadow declaration * fix channel bookmarks permission string definition in admin console * fix another shadow declaration * Fix model conversion * add SplitSliceInChunks * remove include bookmarks in channels api * Cap amount of bookmarks per channel * add etag back to get channels * feedback review * update file info when replacing a bookmark file * return 501 not implemented when the license is not available * add detail message when getting channel member on bookmark api * start audit before permission check on create bookmark api * use require.Eventuallyf for testing WS events * remove unnecessary log in app layer * use require instead of assert to avoid panics * enforce limit when querying bookmarks since * prevent to create/update bookmark if file is already attached * fix lint * delete file when a bookmark is deleted * Dot allow to set a fileId and a url at the same time to a bookmark * fix query to delete a file that belongs to a bookmark * do not patch the bookmark type * Server side FeatureFlag check (#26145) * use ff in server, set ff to false * turn on FF for unit tests * defer unset FF for unit tests * turn ff on for testing * only allow attaching files that were uploaded for bookmark * Set feature flag off as default * fix lint * update email templates as PR failed * revert templates * force the assignment of ID when creating a bookmark * Fix unit tests --------- Co-authored-by: Miguel de la Cruz <miguel@mcrx.me> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Caleb Roseland <caleb@calebroseland.com> Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>	2024-03-12 08:36:05 -06:00
Harshil Sharma	4fda7e6f34	Moved some common SQL function to public utils as they are used in plugins (#26412) ... * Moved some common SQL function tu public utls as they are used in plugins * goimported file * Added tests * Created sub-package * MOved SetupConnection to public sql utils	2024-03-11 09:54:23 +05:30
Agniva De Sarker	04cf1ed114	Bump up the minimum required MySQL version (#25465) ... MySQL 5.7 is at end of life. https://mattermost.atlassian.net/browse/MM-55589 ```release-note We bump up minimum MySQL version to be 8.0.0 ``` Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>	2024-01-11 22:18:20 +05:30
Agniva De Sarker	88bd7b65f0	MM-55589: Throw warn logs for MySQL < 8 (#25637) ... We just throw a warn log for now. Support will be completely removed next ESR. https://mattermost.atlassian.net/browse/MM-55589 ```release-note MySQL 5.7 is at EOL. We recommend all customers to upgrade to atleast 8.x. For now, we are logging a warning. From version 9.5 onwards, which is the next ESR, we will stop supporting 5.7 altogether. ``` Co-authored-by: Neil B <93996140+nab-77@users.noreply.github.com> Co-authored-by: Mattermost Build <build@mattermost.com>	2023-12-11 09:21:18 +05:30
Ben Schumacher	b2ec1ff8ae	[MM-55595] Use annotated logger in search layer (#25468)	2023-12-04 18:34:57 +01:00
Felipe Martin	b40366dbdf	[MM-55028] Added OAuthOutgoingConnection store (#25221) ... * OAuthOutgoingConnection model * added store * make generated * add missing license headers * fix receiver name * i18n * i18n sorting * update migrations from master * make migrations-extract * update retrylayer tests * replaced sql query with id pagination * fixed flaky tests * missing columns * missing columns on save/update * typo * improved tests * remove enum from mysql colum * add password credentials to store * renamed migrations * model change suggestions * refactor test functionsn * migration typo * refactor store table names * updated sanitize test * oauthoutgoingconnection -> outgoingoauthconnection * signature change * i18n update * granttype typo * uppercase typo * lowercase store name	2023-11-20 09:42:07 -05:00
Ben Schumacher	0d5a8b8841	Add whitespace linter (#24855)	2023-10-11 10:13:36 +02:00
Devin Binnie	a3b194581f	[MM-37984] Allow Desktop App to authenticate via external providers outside of the app on supported servers (#24140) ... * [MM-37984] Allow Desktop App to authenticate via external providers outside of the app on supported servers * PR feedback * Add support for mattermost-dev protocol for development use * Update server/channels/db/migrations/postgres/000110_create_desktop_tokens.up.sql * Fix silly typo * Update server/channels/db/migrations/postgres/000110_create_desktop_tokens.up.sql * Remove storage of client token, only validate it on the client * Update migrations * Add concurrently create index * Remove CONCURRENTLY for now * Fix issue with changing history * Remove old migration * Use idempotent statement to drop old index * Remove reference to old table	2023-08-30 11:21:43 -04:00
Agniva De Sarker	45a14e23a9	MM-53228: Adding schema version to cluster info (#24209) ... With the schema version available, a job can query for the cluster info to confirm whether or not all nodes in a cluster are upgraded to the same version or not. This will help it in determining whether to start the job or not. https://mattermost.atlassian.net/browse/MM-53228 ```release-note NONE ```	2023-08-10 09:40:49 +05:30
Jesse Hallam	e39b485c4b	Fail vs. fatal on store startup (#24170) ... If the store fails to initialize (e.g. run a migration), it would `log.Fatal` and then `os.Exit`. Unfortunately, this trips up `TestMain`, which happily keeps running tests, now guaranteed to fail. Avoid this by instead returning an error from the store initialization, handling appropriately at the layer above.	2023-08-04 23:05:01 -03:00
Miguel de la Cruz	150c6e7aef	Remove boards product references (#23855) ... Automatic Merge	2023-07-18 15:17:29 +03:00
Devin Binnie	823ef27d6c	Revert "[MM-37984] Allow Desktop App to authenticate via external providers outside of the app on supported servers (#23795)" (#24039) ... This reverts commit abdf4e58c3.	2023-07-17 10:54:53 -04:00
Devin Binnie	abdf4e58c3	[MM-37984] Allow Desktop App to authenticate via external providers outside of the app on supported servers (#23795) ... * WIP * Add rate limiting for desktop token API * Missing mocks * Style fixes * Update snapshots * Maybe use an actual redirect link :P * Refactoring for tests * Add tests for server * Fix lint issue * Fix tests * Fix lint * Add front-end screen component * Component logic * Style changes * Quick style fix * Lint fixes * Initial PR feedback * Enable logging into the browser as well when completing the login process * Refactor to push more logic to the other component * Remove unnecessary helper code * Fix i18n --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2023-07-12 09:25:05 -04:00
Ibrahim Serdar Acikgoz	4546a2eebb	[MM-47976] cmd/mattermost/db: add downgrade commands and enable plan saving (#21779)	2023-06-12 12:48:50 +03:00
Agniva De Sarker	efaa6264cc	MM-53032: Fix module path after repo rename (#23689) ... It was a good decision in hindsight to keep the public module as 0.x because this would have been a breaking change again. https://mattermost.atlassian.net/browse/MM-53032 ```release-note Changed the Go module path from github.com/mattermost/mattermost-server/server/v8 to github.com/mattermost/mattermost/server/v8. For the public facing module, it's path is also changed from github.com/mattermost/mattermost-server/server/public to github.com/mattermost/mattermost/server/public ```	2023-06-11 10:54:35 +05:30
Vishal	9399ce8637	[MM-47751][MM-48102] MPA: Send Persistent Notifications (#21619) ... * MM-46410: adds urgency on mention counts We have introduced priority for posts in https://github.com/mattermost/mattermost-webapp/pull/10951. We do need to color the mention badges in the webapp with a prominent color when a mention is posted in an urgent message. A thread has urgent mentions if the root post is marked as urgent, and the replies contain mentions to the user viewing the thread. This PR adds two columns, urgentmentioncount, and isurgent, in channelmembers, and threads tables respectively. Furthermore when asking for team/thread mention counts, we also return urgent mention counts for the user. * Adds PostAcknowledgements table and apis * job init and fetch mentions * add-migrations * delete-expired * send-notifications * Fetches post priority in batches * stop-notifications * stop-notification-on-reply * MM-47750: Adds PostAcknowledgements table and apis - Adds post acknowledgement api/app/store methods to be able to save and delete post acknowledgements by users. - Adds wesbsocket events for acknowledgement created/deleted - Returns post acknowledgements in the post's metadata * add-license-check * add-pagination * delete on channel and team * validate guests * add configs * move create priority post check from app to api * Add desktop notifications * check status * use config in job * add IsUrgent check * Add last-sent-at * validate max recipients * Update lastSentAt * Validate min. recipient * send email notification only once * remove email notifications * use latest time from config to run job * Add notifications counter * publish events to mentioned users only * pickup license updates in scheduler * don't allow post owner to stop notifications * follow normal notifications behaviour * Validates persistent notifications interval * move logic of handling valid and expired posts into sql * Adds persistent notifications in the webapp --------- Co-authored-by: koox00 <3829551+koox00@users.noreply.github.com> Co-authored-by: Mattermod <mattermod@users.noreply.github.com> Co-authored-by: Mattermost Build <build@mattermost.com>	2023-05-18 23:44:12 +05:30
Jesse Hallam	bb02b35048	Expose public/ API as submodule (#23345) ... * model -> public/model * plugin -> public/plugin * public/model/utils -> public/utils * platform/shared/mlog -> public/shared/mlog * platform/shared/i18n -> public/shared/i18n * platform/shared/markdown -> public/shared/markdown * platform/services/timezones -> public/shared/timezones * channels/einterfaces -> einterfaces * expose public/ submodule * go mod tidy * .github: cache-dependency-path, setup-go-work * modules-tidy for public/ too * remove old gomodtidy	2023-05-10 13:07:02 -03:00
Agniva De Sarker	a3e4ea2e7a	MM-52679: Bump up minimum postgres version to 11 (#23315) ... https://mattermost.atlassian.net/browse/MM-52679 ```release-note NONE ``` --------- Co-authored-by: Mattermost Build <build@mattermost.com>	2023-05-10 08:26:59 +05:30
Agniva De Sarker	c34a50a6c7	MM-50427: Make MM survive DB replica outage (#22888) ... We monitor the health of DB replicas, and on a fatal error, take them out of the pool. On a separate goroutine, we keep pinging the unhealthy replicas, and on getting a good response back, we add them back to the pool. https://mattermost.atlassian.net/browse/MM-50427 ```release-note Mattermost is now resilient against DB replica outages and will dynamically choose a replica if it's alive. Also added a config parameter ReplicaMonitorIntervalSeconds whose default value is 5. This controls how frequently unhealthy replicas will be monitored for liveness check. ``` Co-authored-by: Mattermost Build <build@mattermost.com>	2023-04-19 17:03:18 +05:30
Miguel de la Cruz	067e36c23c	Enable products for tests (#22757) ... * Enable products for channels tests * increase unit test timeout; check IsConfigReadOnly * make app-layers * Avoid loading boards tempaltes between tests to improve speed * Fix delete query to be compatible with both databases * Avoid preserving the templates for boards store tests * Run all tests in one command * Revert "Run all tests in one command" This reverts commit 0330f7cd8f. * concurrent pkg group tests in CI * Revert "Revert "Run all tests in one command"" This reverts commit 73892fec77. * Revert "concurrent pkg group tests in CI" This reverts commit 550fb6cdd4. * try testing 3 subsets of packages concurrently to improve time taken * Revert "try testing 3 subsets of packages concurrently to improve time taken" This reverts commit 97475f3c4e. --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: wiggin77 <wiggin77@warpmail.net>	2023-04-18 13:58:33 +02:00
Agniva De Sarker	b200a07881	v8.0 module release (#22975) ... https://mattermost.atlassian.net/browse/MM-52079 ```release-note We upgrade the module version to 8.0. The new module path is github.com/mattermost-server/server/v8. ``` Co-authored-by: Doug Lauder <wiggin77@warpmail.net>	2023-04-18 11:05:28 +05:30
Agniva De Sarker	dacac2e3ed	MM-51768: Scrub username/password from SQL datasource (#22731) ... https://mattermost.atlassian.net/browse/MM-51768 ```release-note NONE ```	2023-04-03 23:41:51 +05:30
Doug Lauder	c943ed6859	Mono repo -> Master (#22553) ... Combines the following repositories into one: https://github.com/mattermost/mattermost-server https://github.com/mattermost/mattermost-webapp https://github.com/mattermost/focalboard https://github.com/mattermost/mattermost-plugin-playbooks	2023-03-22 17:22:27 -04:00