mirror of
https://github.com/mattermost/mattermost.git
synced 2026-04-13 21:17:49 -04:00
15 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Jesse Hallam
|
71ca373de7
|
Generate instead of hard-coding test passwords, enforce new minimum for FIPS, shard CI, fix FIPS builds (#35905)
Some checks are pending
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (shard 0) (push) Blocked by required conditions
Server CI / Postgres (shard 1) (push) Blocked by required conditions
Server CI / Postgres (shard 2) (push) Blocked by required conditions
Server CI / Postgres (shard 3) (push) Blocked by required conditions
Server CI / Merge Postgres Test Results (push) Blocked by required conditions
Server CI / Postgres FIPS (shard 0) (push) Blocked by required conditions
Server CI / Postgres FIPS (shard 1) (push) Blocked by required conditions
Server CI / Postgres FIPS (shard 2) (push) Blocked by required conditions
Server CI / Postgres FIPS (shard 3) (push) Blocked by required conditions
Server CI / Merge Postgres FIPS Test Results (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Tools CI / check-style (mattermost-govet) (push) Waiting to run
Tools CI / Test (mattermost-govet) (push) Waiting to run
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Blocked by required conditions
Web App CI / check-external-links (push) Blocked by required conditions
Web App CI / check-types (push) Blocked by required conditions
Web App CI / test (platform) (push) Blocked by required conditions
Web App CI / test (mattermost-redux) (push) Blocked by required conditions
Web App CI / test (channels shard 1/4) (push) Blocked by required conditions
Web App CI / test (channels shard 2/4) (push) Blocked by required conditions
Web App CI / test (channels shard 3/4) (push) Blocked by required conditions
Web App CI / test (channels shard 4/4) (push) Blocked by required conditions
Web App CI / upload-coverage (push) Blocked by required conditions
Web App CI / build (push) Blocked by required conditions
* Replace hardcoded test passwords with model.NewTestPassword() Add model.NewTestPassword() utility that generates 14+ character passwords meeting complexity requirements for FIPS compliance. Replace all short hardcoded test passwords across the test suite with calls to this function. * Enforce FIPS compliance for passwords and HMAC keys FIPS OpenSSL requires HMAC keys to be at least 14 bytes. PBKDF2 uses the password as the HMAC key internally, so short passwords cause PKCS5_PBKDF2_HMAC to fail. - Add FIPSEnabled and PasswordFIPSMinimumLength build-tag constants - Raise the password minimum length floor to 14 when compiled with requirefips, applied in SetDefaults only when unset and validated independently in IsValid - Return ErrMismatchedHashAndPassword for too-short passwords in PBKDF2 CompareHashAndPassword rather than a cryptic OpenSSL error - Validate atmos/camo HMAC key length under FIPS and lengthen test keys accordingly - Adjust password validation tests to use PasswordFIPSMinimumLength so they work under both FIPS and non-FIPS builds * CI: shard FIPS test suite and extract merge template Run FIPS tests on PRs that touch go.mod or have 'fips' in the branch name. Shard FIPS tests across 4 runners matching the normal Postgres suite. Extract the test result merge logic into a reusable workflow template to deduplicate the normal and FIPS merge jobs. * more * Fix email test helper to respect FIPS minimum password length * Fix test helpers to respect FIPS minimum password length * Remove unnecessary "disable strict password requirements" blocks from test helpers * Fix CodeRabbit review comments on PR #35905 - Add server-test-merge-template.yml to server-ci.yml pull_request.paths so changes to the reusable merge workflow trigger Server CI validation - Skip merge-postgres-fips-test-results job when test-postgres-normal-fips was skipped, preventing failures due to missing artifacts - Set guest.Password on returned guest in CreateGuestAndClient helper to keep contract consistent with CreateUserWithClient - Use shared LowercaseLetters/UppercaseLetters/NUMBERS/PasswordFIPSMinimumLength constants in NewTestPassword() to avoid drift if FIPS floor changes https://claude.ai/code/session_01HmE9QkZM3cAoXn2J7XrK2f * Rename FIPS test artifact to match server-ci-report pattern The server-ci-report job searches for artifacts matching "*-test-logs", so rename from postgres-server-test-logs-fips to postgres-server-fips-test-logs to be included in the report. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
Miguel de la Cruz
|
48f2fd0873
|
Merge the Integrated Boards MVP feature branch (#35796)
* Add CreatedBy and UpdatedBy to the properties fields and values (#34485) * Add CreatedBy and UpdatedBy to the properties fields and values * Fix types --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds ObjectType to the property fields table (#34908) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Update ObjectType migration setting an empty value and marking the column as not null (#34915) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds uniqueness mechanisms to the property fields (#35058) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Fixing retrylayer mocks * Remove retrylayer duplication * Address review comments * Fix comment to avoid linter issues * Address PR comments * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.down.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update field validation to check only for valid target types * Update migrations to avoid concurrent index creation within a transaction * Update migrations to make all index ops concurrent * Update tests to use valid PSAv2 property fields * Adds a helper for valid PSAv2 TargetTypes --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Fix property tests (#35388) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards feature flag (#35378) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards MVP API changes (#34822) This PR includes the necessary changes for channels and posts endpoints and adds a set of generic endpoints to retrieve and manage property fields and values following the new Property System approach. Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> * Property System Architecture permissions for v2 (#35113) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Fix i18n sorting --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Add Views store and app layer (#35361) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor test loops in ViewStore tests for improved readability * change pagination to limit/offset * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * MM-67388, MM-66528, MM-67750: Add View REST API endpoints, websocket events, and sort order (#35442) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Add View API endpoints with OpenAPI spec, client methods, and i18n Implement REST API for channel views (board-type) behind the IntegratedBoards feature flag. Adds CRUD endpoints under /api/v4/channels/{channel_id}/views with permission checks matching the channel bookmark pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add cursor-based pagination to View API for channel views * Enhance cursor handling in getViewsForChannel and update tests for pagination * Refactor test loops in ViewStore tests for improved readability * Refactor loop in TestGetViewsForChannel for improved readability * change pagination to limit/offset * switch to limit/offset pagination * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add view sort order API endpoint Add POST /api/v4/channels/{channel_id}/views/{view_id}/sort_order endpoint following the channel bookmarks reorder pattern. Includes store, app, and API layers with full test coverage at each layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add connectionId to view WebSocket events and sort_order API spec Thread connectionId from request header through all view handlers (create, update, delete, sort_order) to WebSocket events, matching the channel bookmarks pattern. Add sort_order endpoint to OpenAPI spec. Update minimum server version to 11.6. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove duplicate View/ViewPatch definitions from definitions.yaml The merge from integrated-boards-mvp introduced duplicate View and ViewPatch schema definitions that were already defined earlier in the file with more detail (including ViewBoardProps ref and enums). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update minimum server version to 11.6 in views API spec Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add missing translations for view sort order error messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Merge integrated-boards-mvp into ibmvp_api-views; remove spec files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix flaky TestViewStore timestamp test on CI Add sleep before UpdateSortOrder to ensure timestamps differ, preventing same-millisecond comparisons on fast CI machines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove duplicate views.yaml imclude * Use c.boolString() for include_deleted query param in GetViewsForChannel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix views.yaml sort order schema: use integer type and require body Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor view sort order tests to use named IDs instead of array indices Extract idA/idB/idC from views slice and add BEFORE/AFTER comments to make stateful subtest ordering easier to follow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Return 404 instead of 403 for view operations on deleted channels Deleted channels should appear non-existent to callers rather than revealing their existence via a 403. Detailed error text explains the context for debugging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * add missing channel deleteat checks * Use c.Params.Page instead of manual page query param parsing in getViewsForChannel c.Params already validates and defaults page/per_page, so the manual parsing was redundant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add support for total count in views retrieval * Add tests for handling deleted views in GetViewsForChannel and GetView * Short-circuit negative newIndex in UpdateSortOrder before opening transaction Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add per-channel limit on views to bound UpdateSortOrder cost Without a cap, unbounded view creation makes sort-order updates increasingly expensive (CASE WHEN per view, row locks). Adds MaxViewsPerChannel=50 constant and enforces it in the app layer before saving. Includes API and app layer tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove include_deleted support from views API Soft-deleted views are structural metadata with low risk, but no other similar endpoint (e.g. channel bookmarks) exposes deleted records without an admin gate. Rather than adding an admin-only permission check for consistency, remove the feature entirely since there is no current use case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update view permissions to require `create_post` instead of channel management permissions * Remove obsolete view management error messages for direct and group messages --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(migrations): add user tracking and object type to property fields - Introduced user tracking columns (CreatedBy, UpdatedBy) to PropertyFields and PropertyValues. - Added ObjectType column to PropertyFields with associated unique indexes for legacy and typed properties. - Created new migration scripts for adding and dropping these features, including necessary indexes for data integrity. - Established views for managing property fields with new attributes. This update enhances the schema to support better tracking and categorization of property fields. * Add Property System Architecture v2 API endpoints (#35583) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Require target_type filter when searching property fields * Add objectType validation as part of field.IsValid() * Fix linter * Fix test with bad objecttpye * Fix test grouping --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * MM-67968: Flatten view model — remove icon, subviews, typed board props (#35726) * feat(views): flatten view model by removing icon, subview, and board props Simplifies the View data model as part of MM-67968: removes Icon, Subview, and ViewBoardProps types; renames ViewTypeBoard to ViewTypeKanban; replaces typed Props with StringInterface (map[string]any); adds migration 000167 to drop the Icon column from the Views table. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(api): update views OpenAPI spec to reflect flattened model Removes ViewBoardProps, Subview, and icon from the View and ViewPatch schemas. Changes type enum from board to kanban. Replaces typed props with a free-form StringInterface object. Aligns with MM-67968. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * refactor(views): simplify store by dropping dbView and marshalViewProps StringInterface already implements driver.Valuer and sql.Scanner, so the manual JSON marshal/unmarshal and the dbView intermediate struct were redundant. model.View now scans directly from the database. Also removes the dead ViewMaxLinkedProperties constant and wraps the Commit() error in UpdateSortOrder. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(api): allow arbitrary JSON in view props OpenAPI schema The props field was restricted to string values via additionalProperties: { type: string }, conflicting with the Go model's StringInterface (map[string]any). Changed to additionalProperties: true in View, ViewPatch, and inline POST schemas. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Adds basic implementation of the generic redux store for PSAv2 (#35512) * Adds basic implementation of the generic redux store for PSAv2 * Add created_by and updated_by to the test fixtures * Make target_id, target_type and object_type mandatory * Wrap getPropertyFieldsByIds and getPropertyValuesForTargetByFieldIds with createSelector * Address PR comments --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds websocket messages for the PSAv2 API events (#35696) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Adds websocket messages for the PSAv2 API events * Add IsPSAv2 helper to the property field for clarity * Add guard against nil returns on field deletion * Add docs to the websocket endpoints --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * migrations: consolidate views migrations and reorder after master - Merged 000165 (create Views) with 000167 (drop Icon) since Icon was never needed - Renumbered branch migrations 159-166 → 160-167 so master's 000159 (deduplicate_policy_names) runs first - Regenerated migrations.list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add API endpoint to retrieve posts for a specific view (#35604) Automatic Merge * Apply fixes after merge * Return a more specific error from getting multiple fields * Prevent getting broadcast params on field deletion if not needed * Remove duplicated migration code * Update property conflict code to always use master * Adds nil guard when iterating on property fields * Check that permission level is valid before getting rejected by the database * Validate correctness on TargetID for PSAv2 fields * Avoid PSAv1 using permissions or protected * Fix test data after validation change * Fix flaky search test * Adds more posts for filter use cases to properly test exclusions --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> Co-authored-by: Julien Tant <julien@craftyx.fr> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Miguel de la Cruz
|
58dd9e1bb4
|
Add property system app layer architecture (#35157)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Blocked by required conditions
Web App CI / check-external-links (push) Blocked by required conditions
Web App CI / check-types (push) Blocked by required conditions
Web App CI / test (platform) (push) Blocked by required conditions
Web App CI / test (mattermost-redux) (push) Blocked by required conditions
Web App CI / test (channels shard 1/4) (push) Blocked by required conditions
Web App CI / test (channels shard 2/4) (push) Blocked by required conditions
Web App CI / test (channels shard 3/4) (push) Blocked by required conditions
Web App CI / test (channels shard 4/4) (push) Blocked by required conditions
Web App CI / upload-coverage (push) Blocked by required conditions
Web App CI / build (push) Blocked by required conditions
* Refactor property system with app layer routing and access control separation Establish the app layer as the primary entry point for property operations with intelligent routing based on group type. This architecture separates access-controlled operations (CPA groups) from standard operations, improving performance and code clarity. Architecture Changes: - App layer now routes operations based on group type: - CPA groups -> PropertyAccessService (enforces access control) - Non-CPA groups -> PropertyService (direct, no access control) - PropertyAccessService simplified to handle only CPA operations - Eliminated redundant group type checks throughout the codebase * Move access control routing into PropertyService This change makes the PropertyService the main entrypoint for property related operations, and adds a routing mechanism to decide if extra behaviors or checks should run for each operation, in this case, the property access service logic. To add specific payloads that pluggable checks and operations may need, we use the request context. When the request comes from the API, the endpoints are in charge of adding the caller ID to the payload, and in the case of the plugin API, on receiving a request, the server automatically tags the context with the plugin ID so the property service can react accordingly. Finally, the new design enforces all these checks migrating the actual property logic to internal, non-exposed methods, so any caller from the App layer needs to go through the service checks that decide if pluggable logic is needed, avoiding any possibility of a bypass. * Fix i18n * Fix bad error string * Added nil guards to property methods * Add check for multiple group IDs on value operations * Add nil guard to the plugin checker * Fix build error * Update value tests * Fix linter * Adds early return when content flaggin a thread with no replies * Fix mocks * Clean the state of plugin property tests before each run * Do not wrap appErr on API response and fix i18n * Fix create property field test * Remove the need to cache cpaGroupID as part of the property service * Split the property.go file into multiple * Not found group doesn't bypass access control check * Unexport SetPluginCheckerForTests * Rename plugin context getter to be more PSA specific --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> |
||
|
Miguel de la Cruz
|
e39be2b7e5
|
Improves the Property System Architecture groups (#35395)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Blocked by required conditions
Web App CI / check-external-links (push) Blocked by required conditions
Web App CI / check-types (push) Blocked by required conditions
Web App CI / test (platform) (push) Blocked by required conditions
Web App CI / test (mattermost-redux) (push) Blocked by required conditions
Web App CI / test (channels shard 1/4) (push) Blocked by required conditions
Web App CI / test (channels shard 2/4) (push) Blocked by required conditions
Web App CI / test (channels shard 3/4) (push) Blocked by required conditions
Web App CI / test (channels shard 4/4) (push) Blocked by required conditions
Web App CI / upload-coverage (push) Blocked by required conditions
Web App CI / build (push) Blocked by required conditions
* Improves the Property System Architecture groups The group creation for builtin property groups is moved from behaving like a singleton in the app layer (first call creates the group) to register groups and making sure they're present at server startup time. At the same time, it adds a groups cache as a sync map in the property service, to avoid having individual caches per feature as package variables, making the group caching part of the system. * Fix i18n * Fix test and calls after updating the branch * Avoid panics by controlling the errors * Adjust translations after merge --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> |
||
Harshil Sharma
|
c21ef29f02
|
Flag post API (#33765)
* Added enable/disable setting and feature flag * added rest of notifgication settings * Added backend for content flagging setting and populated notification values from server side defaults * WIP user selector * Added common reviewers UI * Added additonal reviewers section * WIP * WIP * Team table base * Added search in teams * Added search in teams * Added additional settings section * WIP * Inbtegrated reviewers settings * WIP * WIP * Added server side validation * cleanup * cleanup * [skip ci] * Some refactoring * type fixes * lint fix * test: add content flagging settings test file * test: add comprehensive unit tests for content flagging settings * enhanced tests * test: add test file for content flagging additional settings * test: add comprehensive unit tests for ContentFlaggingAdditionalSettingsSection * Added additoonal settings test * test: add empty test file for team reviewers section * test: add comprehensive unit tests for TeamReviewersSection component * test: update tests to handle async data fetching in team reviewers section * test: add empty test file for content reviewers component * feat: add comprehensive unit tests for ContentFlaggingContentReviewers component * Added ContentFlaggingContentReviewersContentFlaggingContentReviewers test * test: add notification settings test file for content flagging * test: add comprehensive unit tests for content flagging notification settings * Added ContentFlaggingNotificationSettingsSection tests * test: add user profile pill test file * test: add comprehensive unit tests for UserProfilePill component * refactor: Replace enzyme shallow with renderWithContext in user_profile_pill tests * Added UserProfilePill tests * test: add empty test file for content reviewers team option * test: add comprehensive unit tests for TeamOptionComponent * Added TeamOptionComponent tests * test: add empty test file for reason_option component * test: add comprehensive unit tests for ReasonOption component * Added ReasonOption tests * cleanup * Fixed i18n error * fixed e2e test lijnt issues * Updated test cases * Added snaoshot * Updated snaoshot * lint fix * WIP * lint fix * Added post flagging properties setup * review fixes * updated snapshot * CI * Added base APIs * Fetched team status data on load and team switch * WIP * Review fixes * wip * WIP * Removed an test, updated comment * CI * Added tests * Added tests * Lint fix * Added API specs * Fixed types * CI fixes * API tests * lint fixes * Set env variable so API routes are regiustered * Test update * term renaming and disabling API tests on MySQL * typo * Updated store type definition * Minor tweaks * Added tests * Removed error in app startup when content flaghging setup fails * Updated sync condition: * Flag message modal basE * added post preview * displaying options * Adde comment input * Updated tests and docs * finction rename * WIP * Updated tests * refactor * lint fix * MOved to data migration * lint fix * CI * added new migration mocks * Used setup for tests * some comment * Removed unnecesseery nil check * Form validation * WIP tests * WIP tests * WIP tests * fix: mock content flagging config selector with correct reasons format Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> * fix: add mock for getContentFlaggingConfig in flag post modal test Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> * Updated error code order in API docs * removed empty files * Added tests * lint fixes * minor tweak * lint fix * type fix * fixed test * nit * test enhancements * API WIP * API WIP * creating values * creating content flagging channel and properties * Able to save properties * Added another property field * WIP * WIP * Added validations * Added data validations and hidden post if confifgured to * lint fixes * Added API spec * Added some tests * Added tests for getContentReviewBot * test: add comprehensive tests for getContentReviewChannels function * Added more app layer tests * Added TestCanFlagPost * test: Add comprehensive tests for FlagPost function * Added all app layer tests * Removed a file that was reamoved downstream * test: add content flagging test file * test: add comprehensive tests for FlagContentRequest.IsValid method * Added model tests * test: add comprehensive tests for SqlPropertyValueStore.CreateMany * test: add comprehensive tests for flagPost() API function * Added API tests * linter fix * WIP * sent post flagging confirmation message * fixed i18n nissues * fixed i18n nissues * CI * Updated test * fix: reset contentFlaggingGroupId for test isolation in content flagging tests * removed cached group ID * removed debug log * review fixes * Used correct ot name * CI * Updated mobile text * Handled JSON error * fixerdf i18n * CI * Integrate flag post api (#33798) * WIP * WIP * Added API call * test: add test for Client4.flagPost API call in FlagPostModal * fix: remove userEvent.setup() from flag post modal test * test: wrap submit button click in act for proper state updates * Updated tests * lint fix * CI * Updated to allow special characters in comments * Handled empty comment * Used finally * CI * Fixed test * Spillage card integration (#33832) * Created getContentFlaggingFields API * created getPostPropertyValues API * WIP * Created useContentFlaggingFields hook * WIP * WIP * Added option to retain data for reviewers * Displayed deleted post's preview * DIsplayed all properties * Adding field name i18n * WIP - managing i18n able texts * Finished displaying all fields * Manual cleanup * lint fixes * team role filter logic fix * Fixed tests * created new API to fetch flagged posts * lint fix * Added new client methods * test: add comprehensive tests for content flagging APIs * Added new API tests * fixed openapi spec * Fixed DataSpillageReport tests * Fixed PostMarkdown test * Fixed PostPreviewPropertyRenderer test * Added metadata to card renderer * test fixes * Added no comment placeholder * Fixed test * refactor: improve test mocking for data spillage report component * test mock updates * Updated reducer * not resetting mocks * WIP * review fixes * CI * Fixed * fixes * Content flagging actions implementation (#33852) * Added view detail button * Created RemoveFlaggedMessageConfirmationModal modal * Added key and remove flag request modal * IMplemented delete flagged post * Handled edge cases of deleting flagged post * keep message * UI integration * Added WS event for post report update and handled deleted files of flagged post * Added error handling in keep/remove forms * i18n fixes * Updated OpenAPI specs * fixed types * fixed types * refactoring * Fixed tests * review fixes * Added new property translations * Improved test * fixed test * CI * fixes * CI * fixed a test * CI --------- Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> |
||
Julien Tant
|
78050bb0d3
|
Change properties search signature to support multiple TargetIDs (#33873)
* change properties search * add tests * Fix calls to to the search methods * Fix SearchPropertyFields call with wrong signature |
||
Guillermo Vayá
|
d15b933888
|
[MM-64683] Implement property field counting functionality in Plugin API (#33438)
* Implement property field limit enforcement and counting functionality in Plugin API - Added a limit of 20 property fields per group in the CreatePropertyField method. - Introduced CountPropertyFields method to count active and all property fields, including deleted ones. - Enhanced tests to validate the new property field limit and counting behavior. - Updated related API and service methods to support the new functionality. * Update server/channels/app/properties/property_field.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix vet * fix lint error * fix test * fix tests * fix test * count properties + targets * Update server/channels/app/plugin_api.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * remove test for limit * fix more tests * improve testing messages now that the limit is removed * Apply suggestion from @calebroseland Co-authored-by: Caleb Roseland <caleb@calebroseland.com> * Apply suggestion from @calebroseland Co-authored-by: Caleb Roseland <caleb@calebroseland.com> * Apply suggestion from @calebroseland Co-authored-by: Caleb Roseland <caleb@calebroseland.com> * Apply suggestion from @calebroseland Co-authored-by: Caleb Roseland <caleb@calebroseland.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com> Co-authored-by: Caleb Roseland <caleb@calebroseland.com> |
||
|
Miguel de la Cruz
|
e51ea025db
|
Deletes CPA values on CPA field type change (#31122)
* Deletes CPA values on CPA field type change * Fix error method name reference * Cleans the state when a CPA field's type is updated * Fix types * Fix linter * Webapp no longer makes a decision on the change and server sents a flag in the WS message * Fix linter --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> |
||
|
Miguel de la Cruz
|
6ab6a008e6
|
Adds the capability to retrieve a property field by name (#30859)
* Adds the capability to retrieve a property field by name Allows to retrieve a property field by name and groupID. As the name is only unique within the context of a group, and we can have multiple fields with the same name in the store, for this method the groupID is directly included in the query instead of being an optional field. * Adds the targetID parameter to correctly filter fields * Ensure the method only retrieves non-deleted fields --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
Miguel de la Cruz
|
ca9fd45408
|
Adds a mechanism to delete CPA values for a given user (#30330)
* Adds a mechanism to delete CPA values for a given user This requires improving the Property Value service to enable delete all values for a given target, so a new method was created that allows to delete filtering by targetType and targetID (required) and optionally for a specific groupID in case the caller wants to affect all values for a target (useful in case you remove a post for example and want to delete all values pointing to that post regardless of the feature they belong to) or only those that belong to a specific feature. * Fix property value tests * Fix after merge and update method name * Fix linter --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
Miguel de la Cruz
|
1ca6f6d6fb
|
Adds a groupID filter to the property service methods (#30420)
* Adds a groupID filter to the property service methods This allows the property service caller to directly ensure that a given call is only going to affect a field or value that belongs to a given group, instead of (for example) retrieving a property value before deleting it by id to ensure that the value belongs to a specific property group. The groupID filter is optional and has no effect if called with the empty string value. The changes also remove references to input sanitization on trimming the whitespace for the CPA field names and validate at the API level the input for the field patch endpoint. * Fix linter --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
Miguel de la Cruz
|
c5e6d9f570
|
Updates the property service and store method signatures (#30103)
* Updates the property service and store method signatures Getters can now receive a `groupID` that narrows down the query if present, so it's not necessary to check for the group ID on the returning values from the outside layers. The Search methods now receive the `groupID` and the `targetID` explicitly as parameters, incentivizing the use of the indexes that the underlying tables have on the searches. * Fix tests --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
Julien Tant
|
632a60b332
|
[MM-62553]+[MM-62554] Property Architecture: cursor based pagination (#30119)
* refactor: Replace pagination with cursor-based pagination for custom profile attributes * remove pagination loop on property value retrieval for CPA * add migrations to optimize pagination on property fields and values * adapt test to remove pagination check * update migrations list * postgres: drop index concurrently * concurrent index manipulation must be done outside of a Tx * fix: Correct SQL index drop syntax from "OM" to "ON" in migration files * test: Add CountForGroup test cases for property field store * refactor: Add CountForGroup method to PropertyFieldStore interface and implementations * Fix style and i18n * feat: Add optional deleted property field filtering to CountForGroup method * refactor: Update CountForGroup to support optional deleted property fields * test: Add comprehensive tests for CountForGroup with includeDeleted parameter * adapt test + gen layers * rename property service method and set the includeDelete to false * refactor: Remove redundant constant and use CustomProfileAttributesFieldLimit directly * fix tests --------- Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
Miguel de la Cruz
|
f85a8c61a4
|
Adds websocket messages to Custom Profile Attributes (#30163)
* Adds websocket messages to Custom Profile Attributes The app layer now fires a websocket event as part of the operations over Custom Profile Attribute fields and values. It updates as well the Patch method for CPA values so all the changes are commited as part of the same transaction. To be able to do this last operation, the change adds methods to upsert CPA values in both the store and the property service. * Fix i18n strings --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> |
||
|
Miguel de la Cruz
|
ecdce71fc4
|
Adds the main Property System Architecture components (#29644)
* Adds the main Property System Architecture components
This change adds the necessary migrations for the Property Groups,
Fields and Values tables to be created, the store layer and a Property
Service that can be used from the app layer.
* Update property field type to use user instead of person
* Update PropertyFields to allow for unique nondeleted fields and remove redundant indexes
* Update PropertyValues to allow for unique nondeleted fields and remove redundant indexes
* Use StringMap instead of the map[string]any on property fields
* Add i18n strings
* Revert "Use StringMap instead of the map[string]any on property fields"
This reverts commit
|