Commit graph

67 commits

Author SHA1 Message Date
Doug Lauder
5b76fb11a5
MM-67647: Rename shared_channel_manager roles to follow system_ prefix convention (#35944)
* Rename shared_channel_manager and secure_connection_manager roles to use system_ prefix

  The new roles added in PR #35354 broke the naming convention that all
  system-level roles stored in Users.Roles are prefixed with "system_".
  Client-side code (role.includes('system')) and server-side code (explicit
  switch cases in applyMultiRoleFilters) relied on this convention, causing
  users assigned to these roles to not appear in the System Console.

  Also adds both roles to the applyMultiRoleFilters switch statement in
  user_store.go, which was missing them entirely.
2026-04-08 08:01:33 -04:00
Miguel de la Cruz
48f2fd0873
Merge the Integrated Boards MVP feature branch (#35796)
* Add CreatedBy and UpdatedBy to the properties fields and values (#34485)

* Add CreatedBy and UpdatedBy to the properties fields and values

* Fix types

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>

* Adds ObjectType to the property fields table (#34908)

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>

* Update ObjectType migration setting an empty value and marking the column as not null (#34915)

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>

* Adds uniqueness mechanisms to the property fields (#35058)

* Adds uniqueness mechanisms to the property fields

After adding ObjectType, this commit ensures that both the PSAv1 and
PSAv2 schemas are supported, and enforces property uniqueness through
both database indexes and a logical check when creating new property
fields.

* Adds uniqueness check to property updates

Updates are covered on this commit and we refactor as well the SQL
code to use the squirrel builder and work better with the conditional
addition of the `existingID` piece of the query.

* Add translations to error messages

* Fixing retrylayer mocks

* Remove retrylayer duplication

* Address review comments

* Fix comment to avoid linter issues

* Address PR comments

* Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.down.sql

Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>

* Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql

Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>

* Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql

Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>

* Update field validation to check only for valid target types

* Update migrations to avoid concurrent index creation within a transaction

* Update migrations to make all index ops concurrent

* Update tests to use valid PSAv2 property fields

* Adds a helper for valid PSAv2 TargetTypes

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>

* Fix property tests (#35388)

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>

* Adds Integrated Boards feature flag (#35378)

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>

* Adds Integrated Boards MVP API changes (#34822)

This PR includes the necessary changes for channels and posts
endpoints and adds a set of generic endpoints to retrieve and manage
property fields and values following the new Property System approach.

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>

* Property System Architecture permissions for v2 (#35113)

* Adds uniqueness mechanisms to the property fields

After adding ObjectType, this commit ensures that both the PSAv1 and
PSAv2 schemas are supported, and enforces property uniqueness through
both database indexes and a logical check when creating new property
fields.

* Adds uniqueness check to property updates

Updates are covered on this commit and we refactor as well the SQL
code to use the squirrel builder and work better with the conditional
addition of the `existingID` piece of the query.

* Add translations to error messages

* Add the permissions to the migrations, model and update the store calls

* Adds the property field and property group app layer

* Adds authorization helpers for property fields and values

* Make sure that users cannot lock themselves out of property fields

* Migrate permissions from a JSON column to three normalized columns

* Remove the audit comment

* Use target level constants in authorization

* Log authorization membership failures

* Rename admin to sysadmin

* Fix i18n sorting

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>

* Add Views store and app layer (#35361)

* Add Views store and app layer for Integrated Boards

Implements the View entity (model, SQL store, service, app) as described
in the Integrated Boards tech spec. Views are channel-scoped board
configurations with typed props (board, kanban subviews) and soft-delete.

- public/model: View, ViewBoardProps, Subview, ViewPatch types with
  PreSave/PreUpdate/IsValid/Patch/Clone/Auditable
- Migration 158: Views table with jsonb Props column and indexes
- SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag)
- ViewService: CreateView seeds default kanban subview and links the
  boards property field; caches boardPropertyFieldID at startup
- App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView
  with channel-membership permission checks and WebSocket events
  (view_created, view_updated, view_deleted)
- doSetupBoardsPropertyField: registers the Boards property group and
  board field in NewServer() before ViewService construction
- GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows

* Move permission checks out of App layer for views

- Remove HasPermissionToChannel calls from all App view methods
- Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView
- Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field

* Make View service generic and enforce board validation in model

- Remove board-specific auto-setup from service and server startup
- Enforce that board views require Props, at least one subview, and at least one linked property in IsValid()
- Move default subview seeding out of app layer; callers must provide valid props
- Call PreSave on subviews during PreUpdate to assign IDs to new subviews
- Update all tests to reflect the new validation requirements

* Restore migrations files to match base branch

* Distinguish ErrNotFound from other errors in view store Get

* Use CONCURRENTLY and nontransactional for index operations in views migration

* Split views index creation into separate nontransactional migrations

* Update migrations.list

* Update i18n translations for views

* Fix makeView helper to include required Props for board view validation

* Rename ctx parameter from c to rctx in OAuthProvider mock

* Remove views service layer, call store directly from app

* Return 500 for unexpected DB errors in GetView, 404 only for not-found

* Harden View model: deep-copy Props, validate linked property IDs

- Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews
- Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing
- Iterate over LinkedProperties in View.IsValid() and reject invalid IDs
  with a dedicated i18n key
- Register ViewStore in storetest AssertExpectations so mock expectations
  are enforced
- Add tests covering all new behaviours

* Restore autotranslation worker_stopped i18n translation

* Fix view store test IDs and improve error handling in app layer

- Use model.NewId() for linked property IDs in testUpdateView to fix
  validation failure (IsValid rejects non-UUID strings)
- Fix import grouping in app/view.go (stdlib imports in one block)
- Return 404 instead of 500 when Update/Delete store calls return
  ErrNotFound (e.g. concurrent deletion TOCTOU race)

* Add View store mock to retrylayer test genStore helper

The View store was added to the store interface but the genStore()
helper in retrylayer_test.go was not updated, causing TestRetry to panic.
Also removes the duplicate Recap mock registration.

* Refactor view deletion and websocket event handling; update SQL store methods to use query builder

* revert property field store

* Remove useless migrations

* Add cursor-based pagination to View store GetForChannel

- Add ViewQueryCursor and ViewQueryOpts types with validation
- Return (views, cursor, error) for caller-driven pagination
- PerPage clamping: <=0 defaults to 20, >200 clamps to 200
- Support IncludeDeleted filter
- Add comprehensive store tests for pagination, cursor edge cases,
  PerPage clamping, and invalid input rejection
- Add app layer test for empty channelID → 400
- Update interface, retrylayer, timerlayer, and mock signatures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Refactor test loops in ViewStore tests for improved readability

* change pagination to limit/offset

* Add upper-bound limits on View Subviews and LinkedProperties

Defense-in-depth validation: cap Subviews at 50 and LinkedProperties
at 500 to prevent abuse below the 300KB payload limit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* MM-67388, MM-66528, MM-67750: Add View REST API endpoints, websocket events, and sort order (#35442)

* Add Views store and app layer for Integrated Boards

Implements the View entity (model, SQL store, service, app) as described
in the Integrated Boards tech spec. Views are channel-scoped board
configurations with typed props (board, kanban subviews) and soft-delete.

- public/model: View, ViewBoardProps, Subview, ViewPatch types with
  PreSave/PreUpdate/IsValid/Patch/Clone/Auditable
- Migration 158: Views table with jsonb Props column and indexes
- SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag)
- ViewService: CreateView seeds default kanban subview and links the
  boards property field; caches boardPropertyFieldID at startup
- App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView
  with channel-membership permission checks and WebSocket events
  (view_created, view_updated, view_deleted)
- doSetupBoardsPropertyField: registers the Boards property group and
  board field in NewServer() before ViewService construction
- GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows

* Move permission checks out of App layer for views

- Remove HasPermissionToChannel calls from all App view methods
- Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView
- Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field

* Make View service generic and enforce board validation in model

- Remove board-specific auto-setup from service and server startup
- Enforce that board views require Props, at least one subview, and at least one linked property in IsValid()
- Move default subview seeding out of app layer; callers must provide valid props
- Call PreSave on subviews during PreUpdate to assign IDs to new subviews
- Update all tests to reflect the new validation requirements

* Restore migrations files to match base branch

* Distinguish ErrNotFound from other errors in view store Get

* Use CONCURRENTLY and nontransactional for index operations in views migration

* Split views index creation into separate nontransactional migrations

* Update migrations.list

* Update i18n translations for views

* Fix makeView helper to include required Props for board view validation

* Rename ctx parameter from c to rctx in OAuthProvider mock

* Remove views service layer, call store directly from app

* Return 500 for unexpected DB errors in GetView, 404 only for not-found

* Harden View model: deep-copy Props, validate linked property IDs

- Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews
- Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing
- Iterate over LinkedProperties in View.IsValid() and reject invalid IDs
  with a dedicated i18n key
- Register ViewStore in storetest AssertExpectations so mock expectations
  are enforced
- Add tests covering all new behaviours

* Restore autotranslation worker_stopped i18n translation

* Fix view store test IDs and improve error handling in app layer

- Use model.NewId() for linked property IDs in testUpdateView to fix
  validation failure (IsValid rejects non-UUID strings)
- Fix import grouping in app/view.go (stdlib imports in one block)
- Return 404 instead of 500 when Update/Delete store calls return
  ErrNotFound (e.g. concurrent deletion TOCTOU race)

* Add View store mock to retrylayer test genStore helper

The View store was added to the store interface but the genStore()
helper in retrylayer_test.go was not updated, causing TestRetry to panic.
Also removes the duplicate Recap mock registration.

* Refactor view deletion and websocket event handling; update SQL store methods to use query builder

* revert property field store

* Add View API endpoints with OpenAPI spec, client methods, and i18n

Implement REST API for channel views (board-type) behind the
IntegratedBoards feature flag. Adds CRUD endpoints under
/api/v4/channels/{channel_id}/views with permission checks
matching the channel bookmark pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove useless migrations

* Add cursor-based pagination to View store GetForChannel

- Add ViewQueryCursor and ViewQueryOpts types with validation
- Return (views, cursor, error) for caller-driven pagination
- PerPage clamping: <=0 defaults to 20, >200 clamps to 200
- Support IncludeDeleted filter
- Add comprehensive store tests for pagination, cursor edge cases,
  PerPage clamping, and invalid input rejection
- Add app layer test for empty channelID → 400
- Update interface, retrylayer, timerlayer, and mock signatures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add cursor-based pagination to View API for channel views

* Enhance cursor handling in getViewsForChannel and update tests for pagination

* Refactor test loops in ViewStore tests for improved readability

* Refactor loop in TestGetViewsForChannel for improved readability

* change pagination to limit/offset

* switch to limit/offset pagination

* Add upper-bound limits on View Subviews and LinkedProperties

Defense-in-depth validation: cap Subviews at 50 and LinkedProperties
at 500 to prevent abuse below the 300KB payload limit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add view sort order API endpoint

Add POST /api/v4/channels/{channel_id}/views/{view_id}/sort_order
endpoint following the channel bookmarks reorder pattern. Includes
store, app, and API layers with full test coverage at each layer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add connectionId to view WebSocket events and sort_order API spec

Thread connectionId from request header through all view handlers
(create, update, delete, sort_order) to WebSocket events, matching
the channel bookmarks pattern. Add sort_order endpoint to OpenAPI
spec. Update minimum server version to 11.6.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove duplicate View/ViewPatch definitions from definitions.yaml

The merge from integrated-boards-mvp introduced duplicate View and
ViewPatch schema definitions that were already defined earlier in
the file with more detail (including ViewBoardProps ref and enums).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update minimum server version to 11.6 in views API spec

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add missing translations for view sort order error messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Merge integrated-boards-mvp into ibmvp_api-views; remove spec files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix flaky TestViewStore timestamp test on CI

Add sleep before UpdateSortOrder to ensure timestamps differ,
preventing same-millisecond comparisons on fast CI machines.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* remove duplicate views.yaml imclude

* Use c.boolString() for include_deleted query param in GetViewsForChannel

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix views.yaml sort order schema: use integer type and require body

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Refactor view sort order tests to use named IDs instead of array indices

Extract idA/idB/idC from views slice and add BEFORE/AFTER comments
to make stateful subtest ordering easier to follow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Return 404 instead of 403 for view operations on deleted channels

Deleted channels should appear non-existent to callers rather than
revealing their existence via a 403. Detailed error text explains
the context for debugging.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* add missing channel deleteat checks

* Use c.Params.Page instead of manual page query param parsing in getViewsForChannel

c.Params already validates and defaults page/per_page, so the manual
parsing was redundant.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add support for total count in views retrieval

* Add tests for handling deleted views in GetViewsForChannel and GetView

* Short-circuit negative newIndex in UpdateSortOrder before opening transaction

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add per-channel limit on views to bound UpdateSortOrder cost

Without a cap, unbounded view creation makes sort-order updates
increasingly expensive (CASE WHEN per view, row locks). Adds
MaxViewsPerChannel=50 constant and enforces it in the app layer
before saving. Includes API and app layer tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Remove include_deleted support from views API

Soft-deleted views are structural metadata with low risk, but no other
similar endpoint (e.g. channel bookmarks) exposes deleted records without
an admin gate. Rather than adding an admin-only permission check for
consistency, remove the feature entirely since there is no current use case.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update view permissions to require `create_post` instead of channel management permissions

* Remove obsolete view management error messages for direct and group messages

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat(migrations): add user tracking and object type to property fields

- Introduced user tracking columns (CreatedBy, UpdatedBy) to PropertyFields and PropertyValues.
- Added ObjectType column to PropertyFields with associated unique indexes for legacy and typed properties.
- Created new migration scripts for adding and dropping these features, including necessary indexes for data integrity.
- Established views for managing property fields with new attributes.

This update enhances the schema to support better tracking and categorization of property fields.

* Add Property System Architecture v2 API endpoints (#35583)

* Adds uniqueness mechanisms to the property fields

After adding ObjectType, this commit ensures that both the PSAv1 and
PSAv2 schemas are supported, and enforces property uniqueness through
both database indexes and a logical check when creating new property
fields.

* Adds uniqueness check to property updates

Updates are covered on this commit and we refactor as well the SQL
code to use the squirrel builder and work better with the conditional
addition of the `existingID` piece of the query.

* Add translations to error messages

* Add the permissions to the migrations, model and update the store calls

* Adds the property field and property group app layer

* Adds authorization helpers for property fields and values

* Make sure that users cannot lock themselves out of property fields

* Migrate permissions from a JSON column to three normalized columns

* Remove the audit comment

* Use target level constants in authorization

* Log authorization membership failures

* Rename admin to sysadmin

* Adds the Property System Architecture v2 API endpoints

* Adds permission checks to the create field endpoint

* Add target access checks to value endpoints

* Add default branches for object_type and target_type and extra guards for cursor client4 methods

* Fix vet API mismatch

* Fix error checks

* Fix linter

* Add merge semantics for property patch logic and API endpoint

* Fix i18n

* Fix duplicated patch elements and early return on bad cursor

* Update docs to use enums

* Fix i18n sorting

* Update app layer to return model.AppError

* Adds a limit to the number of property values that can be patched in the same request

* Require target_type filter when searching property fields

* Add objectType validation as part of field.IsValid()

* Fix linter

* Fix test with bad objecttpye

* Fix test grouping

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>

* MM-67968: Flatten view model — remove icon, subviews, typed board props (#35726)

* feat(views): flatten view model by removing icon, subview, and board props

Simplifies the View data model as part of MM-67968: removes Icon, Subview,
and ViewBoardProps types; renames ViewTypeBoard to ViewTypeKanban; replaces
typed Props with StringInterface (map[string]any); adds migration 000167
to drop the Icon column from the Views table.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* feat(api): update views OpenAPI spec to reflect flattened model

Removes ViewBoardProps, Subview, and icon from the View and ViewPatch
schemas. Changes type enum from board to kanban. Replaces typed props
with a free-form StringInterface object. Aligns with MM-67968.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* refactor(views): simplify store by dropping dbView and marshalViewProps

StringInterface already implements driver.Valuer and sql.Scanner, so the
manual JSON marshal/unmarshal and the dbView intermediate struct were
redundant. model.View now scans directly from the database. Also removes
the dead ViewMaxLinkedProperties constant and wraps the Commit() error in
UpdateSortOrder.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix(api): allow arbitrary JSON in view props OpenAPI schema

The props field was restricted to string values via
additionalProperties: { type: string }, conflicting with the Go model's
StringInterface (map[string]any). Changed to additionalProperties: true
in View, ViewPatch, and inline POST schemas.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* Adds basic implementation of the generic redux store for PSAv2 (#35512)

* Adds basic implementation of the generic redux store for PSAv2

* Add created_by and updated_by to the test fixtures

* Make target_id, target_type and object_type mandatory

* Wrap getPropertyFieldsByIds and getPropertyValuesForTargetByFieldIds with createSelector

* Address PR comments

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>

* Adds websocket messages for the PSAv2 API events (#35696)

* Adds uniqueness mechanisms to the property fields

After adding ObjectType, this commit ensures that both the PSAv1 and
PSAv2 schemas are supported, and enforces property uniqueness through
both database indexes and a logical check when creating new property
fields.

* Adds uniqueness check to property updates

Updates are covered on this commit and we refactor as well the SQL
code to use the squirrel builder and work better with the conditional
addition of the `existingID` piece of the query.

* Add translations to error messages

* Add the permissions to the migrations, model and update the store calls

* Adds the property field and property group app layer

* Adds authorization helpers for property fields and values

* Make sure that users cannot lock themselves out of property fields

* Migrate permissions from a JSON column to three normalized columns

* Remove the audit comment

* Use target level constants in authorization

* Log authorization membership failures

* Rename admin to sysadmin

* Adds the Property System Architecture v2 API endpoints

* Adds permission checks to the create field endpoint

* Add target access checks to value endpoints

* Add default branches for object_type and target_type and extra guards for cursor client4 methods

* Fix vet API mismatch

* Fix error checks

* Fix linter

* Add merge semantics for property patch logic and API endpoint

* Fix i18n

* Fix duplicated patch elements and early return on bad cursor

* Update docs to use enums

* Fix i18n sorting

* Update app layer to return model.AppError

* Adds a limit to the number of property values that can be patched in the same request

* Adds websocket messages for the PSAv2 API events

* Add IsPSAv2 helper to the property field for clarity

* Add guard against nil returns on field deletion

* Add docs to the websocket endpoints

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>

* migrations: consolidate views migrations and reorder after master

- Merged 000165 (create Views) with 000167 (drop Icon) since Icon was never needed
- Renumbered branch migrations 159-166 → 160-167 so master's 000159 (deduplicate_policy_names) runs first
- Regenerated migrations.list

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add API endpoint to retrieve posts for a specific view (#35604)

Automatic Merge

* Apply fixes after merge

* Return a more specific error from getting multiple fields

* Prevent getting broadcast params on field deletion if not needed

* Remove duplicated migration code

* Update property conflict code to always use master

* Adds nil guard when iterating on property fields

* Check that permission level is valid before getting rejected by the database

* Validate correctness on TargetID for PSAv2 fields

* Avoid PSAv1 using permissions or protected

* Fix test data after validation change

* Fix flaky search test

* Adds more posts for filter use cases to properly test exclusions

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com>
Co-authored-by: Julien Tant <julien@craftyx.fr>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 10:36:35 +01:00
Miguel de la Cruz
58dd9e1bb4
Add property system app layer architecture (#35157)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Blocked by required conditions
Web App CI / check-external-links (push) Blocked by required conditions
Web App CI / check-types (push) Blocked by required conditions
Web App CI / test (platform) (push) Blocked by required conditions
Web App CI / test (mattermost-redux) (push) Blocked by required conditions
Web App CI / test (channels shard 1/4) (push) Blocked by required conditions
Web App CI / test (channels shard 2/4) (push) Blocked by required conditions
Web App CI / test (channels shard 3/4) (push) Blocked by required conditions
Web App CI / test (channels shard 4/4) (push) Blocked by required conditions
Web App CI / upload-coverage (push) Blocked by required conditions
Web App CI / build (push) Blocked by required conditions
* Refactor property system with app layer routing and access control separation

Establish the app layer as the primary entry point for property operations
with intelligent routing based on group type. This architecture separates
access-controlled operations (CPA groups) from standard operations,
improving performance and code clarity.

Architecture Changes:
- App layer now routes operations based on group type:
  - CPA groups -> PropertyAccessService (enforces access control)
  - Non-CPA groups -> PropertyService (direct, no access control)
- PropertyAccessService simplified to handle only CPA operations
- Eliminated redundant group type checks throughout the codebase

* Move access control routing into PropertyService

This change makes the PropertyService the main entrypoint for property
related operations, and adds a routing mechanism to decide if extra
behaviors or checks should run for each operation, in this case, the
property access service logic.

To add specific payloads that pluggable checks and operations may
need, we use the request context. When the request comes from the API,
the endpoints are in charge of adding the caller ID to the payload,
and in the case of the plugin API, on receiving a request, the server
automatically tags the context with the plugin ID so the property
service can react accordingly.

Finally, the new design enforces all these checks migrating the actual
property logic to internal, non-exposed methods, so any caller from
the App layer needs to go through the service checks that decide if
pluggable logic is needed, avoiding any possibility of a bypass.

* Fix i18n

* Fix bad error string

* Added nil guards to property methods

* Add check for multiple group IDs on value operations

* Add nil guard to the plugin checker

* Fix build error

* Update value tests

* Fix linter

* Adds early return when content flaggin a thread with no replies

* Fix mocks

* Clean the state of plugin property tests before each run

* Do not wrap appErr on API response and fix i18n

* Fix create property field test

* Remove the need to cache cpaGroupID as part of the property service

* Split the property.go file into multiple

* Not found group doesn't bypass access control check

* Unexport SetPluginCheckerForTests

* Rename plugin context getter to be more PSA specific

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
2026-03-26 07:54:50 +00:00
Nick Misasi
0192d529ed
PermissionManageOauth removal impact (#35554)
Some checks are pending
API / build (push) Waiting to run
Server CI / Compute Go Version (push) Waiting to run
Server CI / Check mocks (push) Blocked by required conditions
Server CI / Check go mod tidy (push) Blocked by required conditions
Server CI / check-style (push) Blocked by required conditions
Server CI / Check serialization methods for hot structs (push) Blocked by required conditions
Server CI / Vet API (push) Blocked by required conditions
Server CI / Check migration files (push) Blocked by required conditions
Server CI / Generate email templates (push) Blocked by required conditions
Server CI / Check store layers (push) Blocked by required conditions
Server CI / Check mmctl docs (push) Blocked by required conditions
Server CI / Postgres with binary parameters (push) Blocked by required conditions
Server CI / Postgres (push) Blocked by required conditions
Server CI / Postgres (FIPS) (push) Blocked by required conditions
Server CI / Generate Test Coverage (push) Blocked by required conditions
Server CI / Run mmctl tests (push) Blocked by required conditions
Server CI / Run mmctl tests (FIPS) (push) Blocked by required conditions
Server CI / Build mattermost server app (push) Blocked by required conditions
Web App CI / check-lint (push) Waiting to run
Web App CI / check-i18n (push) Blocked by required conditions
Web App CI / check-external-links (push) Blocked by required conditions
Web App CI / check-types (push) Blocked by required conditions
Web App CI / test (platform) (push) Blocked by required conditions
Web App CI / test (mattermost-redux) (push) Blocked by required conditions
Web App CI / test (channels shard 1/4) (push) Blocked by required conditions
Web App CI / test (channels shard 2/4) (push) Blocked by required conditions
Web App CI / test (channels shard 3/4) (push) Blocked by required conditions
Web App CI / test (channels shard 4/4) (push) Blocked by required conditions
Web App CI / upload-coverage (push) Blocked by required conditions
Web App CI / build (push) Blocked by required conditions
* Restore manage oauth permission

Co-authored-by: Nick Misasi <nick13misasi@gmail.com>

* Fix migration test lint assertion

Co-authored-by: Nick Misasi <nick13misasi@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-03-15 12:05:29 -04:00
Doug Lauder
e31f471498
MM-67647 Add roles for shared channels management (#35354)
* Add shared_channel_manager and secure_connection_manager built-in roles

  Introduce two new delegated admin roles for granular Shared Channels
  permission management, allowing admins to assign shared channel and
  secure connection management to specific non-admin users without
  granting full System Admin or System Manager access.

  - shared_channel_manager: grants manage_shared_channels permission
  - secure_connection_manager: grants manage_secure_connections permission

  Includes server role definitions, app migrations, permissions migrations,
  System Console UI support, and API permission tests.
2026-03-06 10:51:21 -05:00
David Krauser
1cfe3d92b6
[MM-66836] Integrate PropertyAccessService into API and app layers (#34818)
Updates all Custom Profile Attribute endpoints and app layer methods to pass caller user IDs through to the PropertyAccessService. This connects the access control service introduced in #34812 to the REST API, Plugin API, and internal app operations.

Also updates the OpenAPI spec to document the new field attributes (protected, source_plugin_id, access_mode) and adds notes about protected field restrictions.
2026-02-06 18:06:51 -05:00
Daniel Espino García
2bd29c0359
Add the ability to patch channel autotranslations (#35078)
* Add the ability to patch channel autotranslations

* Fix lint

* Update docs

* Fix CI

* Fix CI

* Fix mmctl test

* Check whether the channel is translated for the user when checking user enabled

* Fix wrong uses of patch acrros e2e and frontend

* Fix test

* Fix wording

* Fix tests and column name

* Move group constrained test so they don't mess with the basic entities

* Fix patch sending too much information
2026-02-06 18:19:06 +01:00
Doug Lauder
7f6a98fd7a
MM-66789 Restrict log downloads to a root path for support packets (#35014)
* [MM-66789] Fix arbitrary file read vulnerability in advanced logging

  Add path validation to prevent reading files outside the logging root
  directory via GetAdvancedLogs (used in support packet generation).

  Security controls:
  - Validate file paths are within logging root before reading
  - Support MM_LOG_PATH environment variable to allow system admins
    to configure a custom logging root directory
  - Resolve symlinks to prevent bypass attacks
  - Detect and block path traversal attempts

  Also adds:
  - Audit logging for support packet generation
  - Config-time validation that logs errors for paths outside logging
    root (will become blocking in future version)
  - Comprehensive test coverage for path validation

* Update server/channels/app/platform/log_test.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix linter errors

* Update server/channels/api4/system.go

Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>

* Simplify unit tests for platform/log_test.go by moving some test logic to config/logger_test.go

* Fix unit tests requiring logging root to be set

* enforce LogSettings.FileLocation path validation; simplify path checking

* fix linter errors

* use dir in logging root for all unit test logging

* MM_LOG_PATH is set once, centrally, for all tests

* fix flaky test

* fix flaky test

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2026-01-29 13:29:55 -05:00
Jesse Hallam
41e5c7286b
Remove vestigial MySQL support (#34865)
* Remove legacy quoteColumnName() utility

Since Mattermost only supports PostgreSQL, the quoteColumnName() helper
that was designed to handle database-specific column quoting is no longer
needed. The function was a no-op that simply returned the column name
unchanged.

Remove the function from utils.go and update status_store.go to use
the "Manual" column name directly.

* Remove legacy driver checks from store.go

Since Mattermost only supports PostgreSQL, remove conditional checks
for different database drivers:

- Simplify specialSearchChars() to always return PostgreSQL-compatible chars
- Remove driver check from computeBinaryParam()
- Remove driver check from computeDefaultTextSearchConfig()
- Simplify GetDbVersion() to use PostgreSQL syntax directly
- Remove switch statement from ensureMinimumDBVersion()
- Remove unused driver parameter from versionString()

* Remove MySQL alternatives for batch delete operations

Since Mattermost only supports PostgreSQL, remove the MySQL-specific
DELETE...LIMIT syntax and keep only the PostgreSQL array-based approach:

- reaction_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch
- file_info_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch
- preference_store.go: Use PostgreSQL tuple IN subquery for DeleteInvalidVisibleDmsGms

* Remove MySQL alternatives for UPDATE...FROM syntax

Since Mattermost only supports PostgreSQL, remove the MySQL-specific
UPDATE syntax that joins tables differently:

- thread_store.go: Use PostgreSQL UPDATE...FROM syntax in
  MarkAllAsReadByChannels and MarkAllAsReadByTeam
- post_store.go: Use PostgreSQL UPDATE...FROM syntax in deleteThreadFiles

* Remove MySQL alternatives for JSON and subquery operations

Since Mattermost only supports PostgreSQL, remove the MySQL-specific
JSON and subquery syntax:

- thread_store.go: Use PostgreSQL JSONB operators for updating participants
- access_control_policy_store.go: Use PostgreSQL JSONB @> operator for
  querying JSON imports
- session_store.go: Use PostgreSQL subquery syntax for Cleanup
- job_store.go: Use PostgreSQL subquery syntax for Cleanup

* Remove MySQL alternatives for CTE queries

Since Mattermost only supports PostgreSQL, simplify code that
uses CTEs (Common Table Expressions):

- channel_store.go: Remove MySQL CASE-based fallback in
  UpdateLastViewedAt and use PostgreSQL CTE exclusively
- draft_store.go: Remove driver checks in DeleteEmptyDraftsByCreateAtAndUserId,
  DeleteOrphanDraftsByCreateAtAndUserId, and determineMaxDraftSize

* Remove driver checks in migrate.go and schema_dump.go

Simplify migration code to use PostgreSQL driver directly since
PostgreSQL is the only supported database.

* Remove driver checks in sqlx_wrapper.go

Always apply lowercase named parameter transformation since PostgreSQL
is the only supported database.

* Remove driver checks in user_store.go

Simplify user store functions to use PostgreSQL-only code paths:
- Remove isPostgreSQL parameter from helper functions
- Use LEFT JOIN pattern instead of subqueries for bot filtering
- Always use case-insensitive LIKE with lower() for search
- Remove MySQL-specific role filtering alternatives

* Remove driver checks in post_store.go

Simplify post_store.go to use PostgreSQL-only code paths:
- Inline getParentsPostsPostgreSQL into getParentsPosts
- Use PostgreSQL TO_CHAR/TO_TIMESTAMP for date formatting in analytics
- Use PostgreSQL array syntax for batch deletes
- Simplify determineMaxPostSize to always use information_schema
- Use PostgreSQL jsonb subtraction for thread participants
- Always execute RefreshPostStats (PostgreSQL materialized views)
- Use materialized views for AnalyticsPostCountsByDay
- Simplify AnalyticsPostCountByTeam to always use countByTeam

* Remove driver checks in channel_store.go

Simplify channel_store.go to use PostgreSQL-only code paths:
- Always use sq.Dollar.ReplacePlaceholders for UNION queries
- Use PostgreSQL LEFT JOIN for retention policy exclusion
- Use PostgreSQL jsonb @> operator for access control policy imports
- Simplify buildLIKEClause to always use LOWER() for case-insensitive search
- Simplify buildFulltextClauseX to always use PostgreSQL to_tsvector/to_tsquery
- Simplify searchGroupChannelsQuery to use ARRAY_TO_STRING/ARRAY_AGG

* Remove driver checks in file_info_store.go

Simplify file_info_store.go to use PostgreSQL-only code paths:
- Always use PostgreSQL to_tsvector/to_tsquery for file search
- Use file_stats materialized view for CountAll()
- Use file_stats materialized view for GetStorageUsage() when not including deleted
- Always execute RefreshFileStats() for materialized view refresh

* Remove driver checks in attributes_store.go

Simplify attributes_store.go to use PostgreSQL-only code paths:
- Always execute RefreshAttributes() for materialized view refresh
- Remove isPostgreSQL parameter from generateSearchQueryForExpression
- Always use PostgreSQL LOWER() LIKE LOWER() syntax for case-insensitive search

* Remove driver checks in retention_policy_store.go

Simplify retention_policy_store.go to use PostgreSQL-only code paths:
- Remove isPostgres parameter from scanRetentionIdsForDeletion
- Always use pq.Array for scanning retention IDs
- Always use pq.Array for inserting retention IDs
- Remove unused json import

* Remove driver checks in property stores

Simplify property_field_store.go and property_value_store.go to use
PostgreSQL-only code paths:
- Always use PostgreSQL type casts (::text, ::jsonb, ::bigint, etc.)
- Remove isPostgres variable and conditionals

* Remove driver checks in channel_member_history_store.go

Simplify PermanentDeleteBatch to use PostgreSQL-only code path:
- Always use ctid-based subquery for DELETE with LIMIT

* Remove remaining driver checks in user_store.go

Simplify user_store.go to use PostgreSQL-only code paths:
- Use LEFT JOIN for bot exclusion in AnalyticsActiveCountForPeriod
- Use LEFT JOIN for bot exclusion in IsEmpty

* Simplify fulltext search by consolidating buildFulltextClause functions

Remove convertMySQLFullTextColumnsToPostgres and consolidate
buildFulltextClause and buildFulltextClauseX into a single function
that takes variadic column arguments and returns sq.Sqlizer.

* Simplify SQL stores leveraging PostgreSQL-only support

- Simplify UpdateMembersRole in channel_store.go and team_store.go
  to use UPDATE...RETURNING instead of SELECT + UPDATE
- Simplify GetPostReminders in post_store.go to use DELETE...RETURNING
- Simplify DeleteOrphanedRows queries by removing MySQL workarounds
  for subquery locking issues
- Simplify UpdateUserLastSyncAt to use UPDATE...FROM...RETURNING
  instead of fetching user first then updating
- Remove MySQL index hint workarounds in ORDER BY clauses
- Update outdated comments referencing MySQL
- Consolidate buildFulltextClause and remove convertMySQLFullTextColumnsToPostgres

* Remove MySQL-specific test artifacts

- Delete unused MySQLStopWords variable and stop_word.go file
- Remove redundant testSearchEmailAddressesWithQuotes test
  (already covered by testSearchEmailAddresses)
- Update comment that referenced MySQL query planning

* Remove MySQL references from server code outside sqlstore

- Update config example and DSN parsing docs to reflect PostgreSQL-only support
- Remove mysql:// scheme check from IsDatabaseDSN
- Simplify SanitizeDataSource to only handle PostgreSQL
- Remove outdated MySQL comments from model and plugin code

* Remove MySQL references from test files

- Update test DSNs to use PostgreSQL format
- Remove dead mysql-replica flag and replicaFlag variable
- Simplify tests that had MySQL/PostgreSQL branches

* Update docs and test config to use PostgreSQL

- Update mmctl config set example to use postgres driver
- Update test-config.json to use PostgreSQL DSN format

* Remove MySQL migration scripts, test data, and docker image

Delete MySQL-related files that are no longer needed:
- ESR upgrade scripts (esr.*.mysql.*.sql)
- MySQL schema dumps (mattermost-mysql-*.sql)
- MySQL replication test scripts (replica-*.sh, mysql-migration-test.sh)
- MySQL test warmup data (mysql_migration_warmup.sql)
- MySQL docker image reference from mirror-docker-images.json

* Remove MySQL references from webapp

- Simplify minimumHashtagLength description to remove MySQL-specific configuration note
- Remove unused HIDE_MYSQL_STATS_NOTIFICATION preference constant
- Update en.json i18n source file

* clean up e2e-tests

* rm server/tests/template.load

* Use teamMemberSliceColumns() in UpdateMembersRole RETURNING clause

Refactor to use the existing helper function instead of hardcoding
the column names, ensuring consistency if the columns are updated.

* u.id -> u.Id

* address code review feedback

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2026-01-20 21:01:59 +00:00
Jesse Hallam
6ef73af2cc
MM-65960: Avoid replica race lag when accessing TelemetryID (#34586)
* avoid replica race lag when remembering ServerID

In an HA environment, with a master and read replica, querying the server id from the store runs the risk of returning a value saved to master but not yet replicated. Avoid this by using the telemetry service value directly when available.

Fixes: MM-65960

* Add Get(ByName)WithContext

* explicitly use master for ServerId

* mock GetByNameWithContext

* more mocking

* more mocks
2025-12-11 17:34:36 -04:00
Jesse Hallam
219530c82c
Add fast test hasher to speed up CI tests (#34707)
The production password hasher uses PBKDF2 with 600,000 iterations,
which is slow especially when combined with race detection. This
adds a fast test hasher (work factor 1) that can be used during tests
to speed up user creation.

The fast hasher is only available in non-production builds via build
tags, ensuring it cannot be used in production.
2025-12-11 09:46:21 -04:00
Nick Misasi
91dfcbbdd1
Integration permission management changes (#34421)
* Support for permissions allowing end users to create and manage their own integrations if sysadmin deems necessary

* Adjustments based on new understanding

* remove extra functions now that we've consolidated

* Fix webapp i18n

* Update snapshots

* Fix test

* Fix some tests, refactor some more, and add a few extra

* fix linter

* Update snapshots

* Fix test

* Missed some cleanup

* Fix e2e

* Fi

* Fix

* Fixes from PR feedback

* Update snapshots

* Fix tests

* Fix slash command list endpoint per PR feedback. Remove changes around OAuth Apps

* Further reversions of oauth stuff

* Update tests

* Small changes to fix when customOnly=false

* Remove extra perm from cypress

* Fixes from Eva's feedback

* Fix i18n

* More fixing

* More fixing
2025-11-13 11:12:30 +00:00
Jesse Hallam
8cace74692
MM-64486: Remove telemetry (#33606)
* MM-64486: Remove telemetry

Remove telemetry from Mattermost. We're no longer relying on Rudder upstream, and no longer making use of this information.

* recover mock for SystemStore.Get

* Fix TestClearPushNotificationSync by adding missing SystemStore mock

The test was failing because the SystemStore mock was missing the Get()
method that's required by the ServerId() function. Added the missing mock
to return a StringMap with SystemServerId.

* fix mocking issue

* Remove now-unused telemetry and constants

* Remove "Disable telemetry events" debug setting

* Remove empty functions

* Remove most "Telemetry tracking removed" comments

* Remove remains of DataPrefetch telemetry

* Remove now-unused prop from InviteMembersButton

* Remove trackDotMenuEvent

* Remove some more leftover comments

* Remove lingering logic related to trackingLocation

* Remove now-unused argument from useCopyText

* Remove lingering telemetry references from PreparingWorkspace

* fixup Remove trackDotMenuEvent

* Remove lingering telemetry references from signup page and password check

* Update snapshots and fix test broken by my changes

* Fix unintended behavior change in thread list filtering

Remove handleSetFilter wrapper that was accidentally modified during
telemetry removal. The function was calling clear() when switching to
unread filter, which was not the original behavior. Use setFilter
directly instead, restoring the original functionality.

* Remove unused useOpenDowngradeModal hook

The useOpenDowngradeModal hook was not being used anywhere in the codebase.

* Remove unused expandableLink from useExpandOverageUsersCheck

The expandableLink return value was not being used by any components.

* Re-add missing TeamLinkClicked performance telemetry

The mark(Mark.TeamLinkClicked) call was accidentally removed from the
handleSwitch function. This telemetry is needed for Looker-based
performance tracking.

* drop LogSettings.VerboseDiagnostics

---------

Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-09-04 18:46:18 +00:00
Harshil Sharma
bcfd7eff86
Post properties (#33395)
* WIP

* Added post flagging properties setup

* Added tests

* Removed error in app startup when content flaghging setup fails

* Updated sync condition:

* WIP

* MOved to data migration

* lint fix

* CI

* added new migration mocks

* Used setup for tests

* some comment

* removed empty files

* Added another property field

* WIP

* Updated test

* Stored version in system key

* fixed tests
2025-08-20 12:22:13 +05:30
Ben Schumacher
ac90cdbb97
[MM-63805] Don't throw a MFA warning for unauthenticated plugin requests (#30795)
* Don't throw a MFA warning for unauthenticated requests

* Always clean Authorization header

* Remove log message from GetSession

* Rewrite ServePluginPublicRequest for clarity

* Move CSRF validation into seperate method

* Update test

* linter

* Fix logger access

* Add log message if check fails

* Improve error messanges for internal errors

* linter fixes

* Add comprehensive tests

* Cleanup tests and token parser

* Add case-insensitive authentication header tests

Tests authentication with lowercase 'bearer' and uppercase 'TOKEN'
prefixes to ensure header parsing is case-insensitive.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* casing

* Update server/channels/app/plugin_requests.go

Co-authored-by: Eva Sarafianou <eva.sarafianou@gmail.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-08-18 11:17:33 +02:00
Agniva De Sarker
41e88b74ac
MM-61407: Remove Bleve (#33430)
https://mattermost.atlassian.net/browse/MM-61407

```release-note
NONE
```

* webapp i18n

```release-note
NONE
```

* Fix e2e tests

```release-note
NONE
```

* fix roles in e2e tests

```release-note
NONE
```

* some review comments

```release-note
NONE
```

* add back permissions to deprecated list

```release-note
NONE
```
2025-07-31 09:35:29 +05:30
Pablo Vélez
206c741226
Mm 64495 manage access rules permissions (#31658)
* MM-6449 - manage channel access rules permissions backend part

* add the system console changes to show the new permission

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-07-25 12:30:10 +02:00
Agniva De Sarker
9dd8c056e7
MM-63368: Remove MySQL (#33458)
https://mattermost.atlassian.net/browse/MM-63368

```release-note
Remove MySQL support from the codebase entirely.
```
2025-07-22 20:40:55 +05:30
Claudio Costa
611b2a8e79
[MM-62408] Server Code Coverage with Fully Parallel Tests (#30078)
* TestPool

* Store infra

* Store tests updates

* Bump maximum concurrent postgres connections

* More infra

* channels/jobs

* channels/app

* channels/api4

* Protect i18n from concurrent access

* Replace some use of os.Setenv

* Remove debug

* Lint fixes

* Fix more linting

* Fix test

* Remove use of Setenv in drafts tests

* Fix flaky TestWebHubCloseConnOnDBFail

* Fix merge

* [MM-62408] Add CI job to generate test coverage (#30284)

* Add CI job to generate test coverage

* Remove use of Setenv in drafts tests

* Fix flaky TestWebHubCloseConnOnDBFail

* Fix more Setenv usage

* Fix more potential flakyness

* Remove parallelism from flaky test

* Remove conflicting env var

* Fix

* Disable parallelism

* Test atomic covermode

* Disable parallelism

* Enable parallelism

* Add upload coverage step

* Fix codecov.yml

* Add codecov.yml

* Remove redundant workspace field

* Add Parallel() util methods and refactor

* Fix formatting

* More formatting fixes

* Fix reporting
2025-05-30 13:58:26 +02:00
Jesse Hallam
31a8047973
Disable morph logging during TestMain (#30948)
* rm "No TEST_DATABASE... override" log message

Let's only log if this value is actually overridden.

* rm "(Created|Dropped) temporary database" message

* only log "Pinging SQL" on subsequent attempts

* disable morph logging from TestMain

* Fix style issues in store test files

- Add missing parameter to migrate() function calls in tests
- Remove unused log function in settings.go
- Fix formatting with go fmt

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* within sqlstore, use "enable" vs "disable" for clarity

* remove trailing newline from morph logs

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-05-21 17:31:18 +00:00
David Krauser
4b64eb0e39
Handle error returned by GetClusterInfos() (#30919)
A recent change to the enterprise cluster code introduced a change to the enterprise API interface. GetClusterInfos() can now return an error. This commit introduces code to handle that error.
2025-05-12 13:37:58 -04:00
Harshil Sharma
a5e68639c2
Channel banner permissions (#30917)
* Fixed save state panel for channel banner

* Defined default background color

* Updated test

* WIP

* wip

* removed unused param

* Updated tests

* CI

* Fixed mmctl test

* Fixed TestDoAdvancedPermissionsMigration test

* Test update

* lint fix

* lint fix

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-05-06 14:03:35 +05:30
Daniel Espino García
7999239ccf
Add system console settings for mobile security (#30456)
* Add config settings for additional security features on mobile

* Add system console settings for mobile security

* Update svg and link

* Fix strings

* Add test for the discovery feature

* Fix tests

* Add permission migrations

* Add relevant e2e tests

* Fix key alignment

* fix tests

* Fix lint

* Mock new migration

* Fix playwright prettier

* Add new section to delegated permissions

* Update snapshots

* Fix flakyness in playwright test

---------

Co-authored-by: Elias Nahum <nahumhbl@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-03-27 13:13:20 +01:00
Scott Bishel
b26c43e716
Permission changes (#29570)
* update permission mistakes

* add getAnalytics to TeamStatistics

* add PermissionGetAnalytics to ReadTeamStatistics

* add mocks for migrations

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-27 16:28:27 +00:00
Julien Tant
b41b968eb8
restrict access to channel converstion to public for non team admin+ (#29819) 2025-01-23 09:42:20 -07:00
Agniva De Sarker
cb75a20c54
MM-61904: Make reliable websockets work in HA (#29489)
We do a cluster request to get the active and dead queues
from other nodes in the cluster to sync any missing
information.

We check the dead queue in the other nodes to see
if there's been any message loss or not. Accordingly,
we send just the active queue or both active and dead queues.

There's still an edge case that is left out where
a client could have potentially connected and reconnected
to multiple nodes leaving multiple active queues
in multiple nodes. We don't handle this scenario
because then potentially we need to create
a slice of sendQueueSize * number_of_nodes. And then
this can happen again, leading to an infinite increase
in sendQueueSize.

We leave this edge-case to Redis, acknowledging
a limitation in our architecture.

In this PR, when there's no message loss, we just
take the active queue from the last node it connected
to.

And if there's message loss where the client's
seqNum is within the last node's dead queue, we also
handle that.

But if there's severe message loss where the client's
seqNum falls within the dead queue of another node, then
we just send the data from that node to reconstruct the
data as much as possible. It could be possible to set
a new connection ID in this case, but this involves
more data transfer always from all nodes and recomputing
the state in the requestor node.

https://mattermost.atlassian.net/browse/MM-61904

```release-note
NONE
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-17 11:11:32 +05:30
Miguel de la Cruz
ecdce71fc4
Adds the main Property System Architecture components (#29644)
* Adds the main Property System Architecture components

This change adds the necessary migrations for the Property Groups,
Fields and Values tables to be created, the store layer and a Property
Service that can be used from the app layer.

* Update property field type to use user instead of person

* Update PropertyFields to allow for unique nondeleted fields and remove redundant indexes

* Update PropertyValues to allow for unique nondeleted fields and remove redundant indexes

* Use StringMap instead of the map[string]any on property fields

* Add i18n strings

* Revert "Use StringMap instead of the map[string]any on property fields"

This reverts commit e2735ab0f8.

* Cast JSON binary data to string and add todo note for StringMap use

* Add mocks to the retrylayer tests

* Cast JSON binary data to string in property value store

* Check for binary parameter instead of casting to string for JSON data

* Check property field type is one of the allowed ones

* Avoid reusing err variable to be explicit about the returned value

* Merge Property System Migrations into one file

* Adds NOT NULL to timestamps at the DB level

* Update stores to use tableSelectQuery instead of a slice var

* Update PropertyField model translations to be more explicit and avoid repetition

* Update PropertyValue model translations to be more explicit and avoid repetition

* Use ExecBuilder instead of ToSql&Exec

* Update property field errors to add context

* Ensure PerPage is greater than zero

* Update store errors to give more context

* Use ExecBuilder in the property stores where possible

* Add an on conflict suffix to the group register to avoid race conditions

* Remove badly used translation string

* Remove unused get in register group method

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-01-13 11:41:44 +00:00
Jesse Hallam
5369f8b36b
s/Get(Master|Replica)X/Get\1/g (#29520)
Drop the legacy `X` suffix from `GetMasterX` and `GetReplicaX`. The
presence of the suffix suggests there's a `non-X` version: but in fact
we migrated these away a long time ago, so remove the cognitive
overhead.

As an aside, this additionally helps avoid trip up LLMs that interpret
this as "something to fix".
2024-12-10 09:57:19 -04:00
Agniva De Sarker
a6d37fa14c
MM-61887: Log the userID if a metric exceeds the last histogram bucket (#29448)
We create a custom histogram metric that logs the userID
when the observed value is greater or equal to the last bucket value.

This allows us to start tracking the slowest users of a system
while at the same time not polluting the Prometheus metrics
by storing a userID for every observation.

https://mattermost.atlassian.net/browse/MM-61887

```release-note
NONE
```
2024-12-05 09:12:54 +05:30
Agniva De Sarker
7f032b0b39
MM-61524: Fix flaky test RedisPubSub (#29311)
- We refactor some of the testlib assertion code
and add a new function to just return true or false.

https://mattermost.atlassian.net/browse/MM-61524
```release-note
NONE
```
2024-11-18 14:55:34 +05:30
mas-who
f48fd7b791
GH-28752 fix errcheck lint issues in server/channels/testlib/helper.go (#29260) 2024-11-14 13:13:45 +01:00
Ben Schumacher
2b426573cd
[MM-60619] Annotate cluster logs messages (#28268) 2024-09-27 09:17:16 +02:00
Scott Bishel
0c585bdac6
MM-59529 Set Channel/Team Admin permissions if All members get set (#28104)
* initial fix

* cleanup code

* move struct back

* fix unit test

* add unit tests

* add comments

* add manage_bookmark permissions

* revert package-lock.json

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-09-10 11:26:55 -06:00
daveseo901
388dfdfbca
[MM-56792] Clarify large plugin upload error (#26271)
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2024-09-09 14:00:25 +02:00
Ben Schumacher
1158e6358c
[MM-54593] HA aware Support Packet (#27598) 2024-08-03 16:11:13 +02:00
Ben Cooke
b244bb621d
[MM-56904] Reduce the number of api requests made to fetch user information for GMs on page load (#27149)
* use new endpoint to fetch group members
2024-07-25 15:57:23 -04:00
Ben Cooke
9187c772b6
[MM-56074] mmctl job commands (#26855)
* add job list and update  job status command to mmctl
2024-06-17 12:07:05 -04:00
Jesse Hallam
cd51dec6e5
Centralize the GetPackagePath (#27004) 2024-05-15 12:05:13 -03:00
Jesse Hallam
630bd40141
Eliminate MM_SERVER_PATH (#24968) 2024-05-10 18:13:05 -03:00
Agniva De Sarker
0cee332001
MM-57152: Get webconn count from the whole cluster (#26813)
We were setting the user status to offline without
checking for connections on other nodes in a cluster.

Now we implement a request-response mechanism for the whole
cluster and we check that before setting a user to offline.

https://mattermost.atlassian.net/browse/MM-57153

```release-note
Fix a bug where the user status would incorrectly be set to offline
without checking for connections in other nodes in an HA cluster.
```

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2024-04-30 19:28:55 +05:30
Elias Nahum
7e9cd04a8b
Channel Bookmarks (#25449)
* create ChannelBookmarks table

* ChannelBookmark model

* channel bookamrks Store layer

* add GetBookmarksForAllChannelByIdSince

* add channel bookmarks to test store

* Add channel bookmarks to app layer

* remove index for createAt in channel bookmarks migrations

* remove createAt from select channel bookmark query and enable store delete bookmark test

* update reponse of UpdateBookmark

* rename db migration files

* channel bookmarks store update sort order

* channel bookmarks app layer update sort order

* fix lint & tests

* Fix lint and introduce util functions to insert / remove from slice

* remove model etag

* i18n

* defer remove file info after test run

* Fix tests passing the request context

* fix migrations

* fix TestRetry

* Add bookmark permissions (#25560)

* Adds channel bookmarks permissions

* Fix linter

* Remove unnecessary empty lines

* Remove scss change as it's not necessary anymore

* Fix mock store

* Fix mock store and add role entry

* Fix test

* Adds cypress test and update permissions migration to update admin roles

* Adds channel bookmarks roles to default admin roles

* Adds bookmark permissions to default role permissions constant in webapp

* Update mmctl test

* Update permission test after normalising the roles

* fix store tests

* fix app layer tests

* Add new bookmark endpoint (#25624)

* Adds channel bookmarks api scaffold and create endpoint

* Applies review comments to the API docs

* Adds websocket test to create channel bookmark

---------

Co-authored-by: Mattermost Build <build@mattermost.com>

* MM-54426 exclude Channel Bookmarks files from data retention (#25656)

* Augment channel APIs to include bookmarks (#25567)

* update files docs for server 9.4

* Adds update channel bookmark endpoint (#25653)

* Adds update channel bookmark sort order endpoint (#25686)

* Adds update channel bookmark endpoint

* Updates edit app method to return the right deleted bookmark and adds tests

* Adds the update channel bookmark sort order endpoint

* Fix repeated test after merge

* Assign right permissions to each test

* Update store and app layer to return specific errors and add tests

* Adds delete channel bookmark endpoint (#25693)

* Updates edit app method to return the right deleted bookmark and adds tests

* Fix repeated test after merge

* Updates edit app method to return the right deleted bookmark and adds tests

* Adds delete channel bookmark endpoint

* Adds list channel bookmarks endpoint (#25700)

* Add channel moderation to bookmarks (#25716)

* fix migrations index

* fix getChannelsForTeamForUser

* fix getChannelsForTeamForUser

* fix bad merge client4

* fix file api with bookmark permission

* add ChannelBookmarks feature flag

* add missing translations

* Set DB column for type as enum

* use custom type for bookmark query using sqlx

* use transaction when saving bookmark

* return NewErrNotFound instead of Sql.ErrNoRows

* use squirrel for IN query

* add a limit of 1K for records in GetBookmarksForAllChannelByIdSince

* UpdateSortOrder with one single query instead of multiple updates

* fix shadow declaration

* fix channel bookmarks permission string definition in admin console

* fix another shadow declaration

* Fix model conversion

* add SplitSliceInChunks

* remove include bookmarks in channels api

* Cap amount of bookmarks per channel

* add etag back to get channels

* feedback review

* update file info when replacing a bookmark file

* return 501 not implemented when the license is not available

* add detail message when getting channel member on bookmark api

* start audit before permission check on create bookmark api

* use require.Eventuallyf for testing WS events

* remove unnecessary log in app layer

* use require instead of assert to avoid panics

* enforce limit when querying bookmarks since

* prevent to create/update bookmark if file is already attached

* fix lint

* delete file when a bookmark is deleted

* Dot allow to set a fileId and a url at the same time to a bookmark

* fix query to delete a file that belongs to a bookmark

* do not patch the bookmark type

* Server side FeatureFlag check (#26145)

* use ff in server, set ff to false

* turn on FF for unit tests

* defer unset FF for unit tests

* turn ff on for testing

* only allow attaching files that were uploaded for bookmark

* Set feature flag off as default

* fix lint

* update email templates as PR failed

* revert templates

* force the assignment of ID when creating a bookmark

* Fix unit tests

---------

Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
2024-03-12 08:36:05 -06:00
Michael Kochell
4e071e861c
Webapp - Outgoing OAuth Connections (#25507)
* added store

* make generated

* add missing license headers

* fix receiver name

* i18n

* i18n sorting

* update migrations from master

* make migrations-extract

* update retrylayer tests

* replaced sql query with id pagination

* fixed flaky tests

* missing columns

* missing columns on save/update

* typo

* improved tests

* remove enum from mysql colum

* add password credentials to store

* license changes

* OAuthOutgoingConnectionInterface

* Oauth -> OAuth

* make generated

* copied over installed_oauth_apps component and renamed things to installed_outgoing_oauth_connections

* merge migrations

* renamed migrations

* model change suggestions

* refactor test functionsn

* migration typo

* refactor store table names

* updated sanitize test

* cleanup merge

* refactor symbol

* "installed outgoing oauth connections" page works

* move things into a nested folder

* add and edit page stubs work

* list endpoint

* oauthoutgoingconnection -> outgoingoauthconnection

* signature change

* i18n update

* granttype typo

* naming

* api list

* uppercase typo

* i18n

* missing license header

* fixed path in comments

* updated openapi definitions

* changes to support selecting command request url

* sanitize connections

* make generated

* test license and no feature flag

* removed t.fatal

* updated testhelper calls

* yaml schema fixes

* switched interface name

* suggested translation

* missing i18n translation

* management permission

* moved permission initalization to proper place

* endpoints

* put tests

* error check typo

* fixed specific enttity urls

* tests

* read permission check

* updated openapi definitions

* i18n

* GetConnectionByAudience method

* notes

* replaced GetConnectionsByAudience with a filter

* added custom oauth token object

* updated interface and usage

* properly set enterprise interface

* move retrieval logic to impl

* webhook tests

* translations

* i18n: updates

* address comments

* endpoint and tests

* i18n

* api docs

* fixed endpoint path

* sq.like

* use filter object instead of parameters

* set url values if not empty

* typos

* converted some components to function components, and move around files

* correctly check token url

* restore flag to previous value

* added command oauth handler

* update enterprise imports

* migrate last component to function component

* Added enterprise import

* refactor permissions and add necessary webapp code

* Check correct flag in permission tree

* allow partial updates

* sort i18n webapp

* missing test modification

* fixed webapp i18n sorting

* allow validating stored connections

* added missing translation

* fix finished adding connection link and text on result page

* added missing permission to smoke tests

* missing role in smoke test

* updated translations

* updated translations

* support editing client secret on existing connection

* fix some i18n strings

* updated translations

* better error messages

* progress on using react select for command request url while maintaining typed in value

* remove writeheader, test

* HasValidGrantType

* end early to avoid nil pointer errors

* move slash command request url input box into its own component

* wrap components related to oauth connections in config check

* fix tests

* i18n-extract

* change some i18n strings to say "Outgoing OAuth 2.0 Connections"

* remove debug code

* fixed i18n

* updated i18n file

* feature configuration backend

* typo

* add system console setting

* Revert "typo"

This reverts commit 669da23e8e.

* Revert "updated i18n file"

This reverts commit d0882c0dd7.

* Revert "fixed i18n"

This reverts commit 3108866bc1.

* fixed i18n

* updated i18n file

* typo

* updated i18n

* updated i18n

* updated i18n

* updated version to 9.6

* replace feature flag with system console configuration

* i18n

* updated tests

* pr feedback

* fix styling of disabled text box

* fix styling of action links in integration console

* server changes for validation feature

* webapp changes for validation feature

* pencil icon styling

* styling fixes for oauth audience correct configuration message

* fix sanitize test

* remove max lengths from outgoing oauth connection form

* use config var in webapp instead of feature flag

* change asterisks to bullets

* update api docs for validate endpoint

* feedback from ux review

* fix lint, types, tests

* fix stylelint

* implement validation button under the token url input

* support wildcard for matching audience urls

* updates for styling

* update snapshots

* add doc links for the outgoing oauth connections feature

* change doc links to use permalink

* add docs link to system console

* fix: use limitedreader in json decoding

* fix: form error in validation

* management permission can read now

* updated api documentation

* doc typo

* require one permission to read only

* fix api connection list audience filter

* fix audience matching and add loading indicator

* fix team permissions on outgoing oauth connection api calls

* fix api doc and test, for adding team id to query params

* handle read permissions by adding a team in the payload

* missing teamid query parameter in test

* change validate button logic to not require audience urls to be filled out

* fix redux type

---------

Co-authored-by: Felipe Martin <me@fmartingr.com>
2024-02-09 14:49:49 -05:00
Utsav Ladani
3ac6edb406
[GH-25484] Fix draft removal on post deletion (#25715)
* [GH-25484] Fix draft removal on post deletion

* [GH-25484] Add batch migration to remove orphan drafts

* [GH-25484] Fix tests of migration and draft store

* [GH-25484] Remove translation file changes.

* [GH-25484] Remove translation file changes.

---------

Co-authored-by: Devin Binnie <52460000+devinbinnie@users.noreply.github.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
2024-01-29 09:55:34 -05:00
Ben Schumacher
b2ec1ff8ae
[MM-55595] Use annotated logger in search layer (#25468) 2023-12-04 18:34:57 +01:00
Nick Misasi
e1c851a3ca
[CLD-6324] Cloud IP Filtering (#24726)
* Initial comit for ip filtering service implementation

* Add audit logs for IP Filters

* start of webapp work

* Stashing

* Updates based on Agniva's feedback around service vs einterface

* Updates completed

* Commit before refactoring, everything's working

* First pass of cleanup complete, front-end tests added

* actually add files

* Updates to some translation strings, running i18n-extract

* Lock everything behind a feature flag

* Fix tests, try to fix some linter stuff

* Fixed linter for JS, on to scss

* Fixed linter for scss

* Fix linter

* More fixes for pipeline

* Support for IPV6

* Remove tsx file that was removed in masteR

* Revert package.json and package-lock.json to master, add cidr-regex dep into channels/package.json

* Another commit to force fix Github

* Fixes around IPV6. Some suggestions from Matt re: UX review. Fixing pipelines for tests and types on new cidr-regex package

* Changes to address Matt's feedback

* A few more changes for clean up

* Add support for permissions

* Fix vet for OpenAPI spec

* Actually add the yaml file for openapi

* Add permission migration to allow support for IP Filtering

* Fix tests

* Final fixes from Matt

* Remove cancel button from page, update link outs to documentation

* Update test to account for removed cancel button

* Adjustments based on feedback from Harrison

* More fixes from PR feedback

* Add a t to fix translations that doesn't seem to be breaking anyone else?

* More fix

* updates for PR feedback

* Fix linter

* Fix types

* Now fix the linter again

* Add back tests because Harrison was able to get them running

* Adjustments for PR feedback

* Remove admin_definition.jsx

* Fix linter

* [CLD-6453] IP Filtering notification email for sysadmins (#25224)

* Initial commit for IP filtering alert email

* Updates to style for email, addition of ip_filtering email:

* Fix pipelines

* Adjustments from Matt's feedback

* Padding changes

* template diff (#25249)

Co-authored-by: Gabe Jackson <3694686+gabrieljackson@users.noreply.github.com>

* Fix hardcoded true, remove bool return value

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Gabe Jackson <3694686+gabrieljackson@users.noreply.github.com>

* Lock feature behind enterprise license. Drop cidr-regex in favour of ipaddr.js dependency. Refactor isIpAddressWithinRanges to use ipaddr.js

* Add a couple server tests

* fix linter

* Fix types from merge conflicts

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Gabe Jackson <3694686+gabrieljackson@users.noreply.github.com>
2023-11-14 09:12:04 -05:00
Harrison Healey
3d0fd16666
MM-54640 Add API to get multiple emojis by name at once (#24651)
* MM-54640 Add API to get multiple emojis by name at once

* Fix status code when too many names are requested

* Address feedback

* Update unit tests

* Fix styling

* Fix more styling

* Fix mismatched i18n id
2023-10-17 12:03:28 -04:00
Devin Binnie
89492a6a46
[MM-53428] Delete empty drafts on upsert (#24046)
* [MM-53428] Delete empty drafts on upsert

* Add migrations to fix existing drafts

* Fix CI

* Delete empty drafts entirely from the DB

* Fix lint

* Implement batch migration for deleting drafts

* Missing store layers

* Add updated mock

* Remove unnecessary test

* PR feedback

* Add check for cluster migration

* Fix MySQL

* Don't check for len<2

* Bit of PR feedback

* Use query builder for parameters

* PR feedback

* More PR feedback

* Merge'd

* unit test GetLastCreateAtAndUserIdValuesForEmptyDraftsMigration

* simplified builder interface

* fix DeleteEmptyDraftsByCreateAtAndUserId for MySQL

* rework as batch migration worker

* fix typo

* log ip address on version mismatches too

* simplify reset semantics

* remove trace log in favour of low spam

* document parameters for clarity

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
2023-10-12 10:52:10 -04:00
Ben Schumacher
0d5a8b8841
Add whitespace linter (#24855) 2023-10-11 10:13:36 +02:00
Jesse Hallam
e39b485c4b
Fail vs. fatal on store startup (#24170)
If the store fails to initialize (e.g. run a migration), it would `log.Fatal` and then `os.Exit`. Unfortunately, this trips up `TestMain`, which happily keeps running tests, now guaranteed to fail.

Avoid this by instead returning an error from the store initialization, handling appropriately at the layer above.
2023-08-04 23:05:01 -03:00
Elias Nahum
aa88f8bf59
Add read_channel_content permission (#24118)
* Add read_channel_content permission

* fix tests

* update system console default permissions per role

* add read_channel_content to e2e defaultRolesPermissions

* Migration to include custom roles

* change deprecated isNotExactRole for isNotRole

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2023-07-31 09:12:53 -04:00