mattermost/server/public/model/post.go

1388 lines
45 KiB
Go
Raw Permalink Normal View History

// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
// See LICENSE.txt for license information.
2015-06-15 03:53:32 -04:00
package model
import (
Add cursor-based Posts Reporting API for compliance and auditing (#34252) * Add cursor-based Posts Reporting API for compliance and auditing Implements a new admin-only endpoint for retrieving posts with efficient cursor-based pagination, designed for compliance, auditing, and archival workflows. Key Features: - Cursor-based pagination using composite (time, ID) keys for consistent performance regardless of dataset size (~10ms per page at any depth) - Flexible time range queries with optional upper/lower bounds - Support for both create_at and update_at time fields - Ascending or descending sort order - Optional metadata enrichment (files, reactions, acknowledgements) - System admin only access (requires manage_system permission) - License enforcement for compliance features API Endpoint: POST /api/v4/reports/posts - Request: JSON body with channel_id, cursor_time, cursor_id, and options - Response: Posts map + next_cursor object (null when pagination complete) - Max page size: 1000 posts per request (MaxReportingPerPage constant) Implementation: - Store Layer: Direct SQL queries with composite index on (ChannelId, CreateAt, Id) - App Layer: Permission checks, optional metadata enrichment, post hooks - API Layer: Parameter validation, system admin enforcement, license checks - Data Model: ReportPostOptions, ReportPostOptionsCursor, ReportPostListResponse Code Quality Improvements: - Added MaxReportingPerPage constant (1000) to eliminate magic numbers - Removed unused StartTime field from ReportPostOptions - Added fmt import for dynamic error messages Testing: - 14 comprehensive store layer unit tests - 12 API layer integration tests covering permissions, pagination, filters - All tests passing Documentation: - POSTS_REPORTING.md: Developer reference with Go structs and usage examples - POSTS_REPORTING_API_SPEC.md: Complete technical specification - GET_POSTS_API_IMPROVEMENTS.md: Implementation analysis and design rationale - POSTS_TIME_RANGE_FEATURE.md: Archived time range feature for future use Performance: Cursor-based pagination maintains consistent ~10ms query time at any dataset depth, compared to offset-based pagination which degrades significantly (Page 1 = 10ms, Page 1000 = 10 seconds). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * lint fixes * lint fixes * gofmt * i18n-extract * Add Enterprise license requirement to posts reporting API Enforce Enterprise license (tier 20+) for the new posts reporting endpoint to align with compliance feature licensing. Professional tier is insufficient. Changes: - Add MinimumEnterpriseLicense check in GetPostsForReporting app layer - Add test coverage for license validation (no license and Professional tier) All existing tests pass with new license enforcement. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * i18n-extract * add licensing to api documentation * Test SSH signing * Add mmctl command for posts reporting API Adds mmctl report posts command to retrieve posts from a channel for administrative reporting purposes. Supports cursor-based pagination with configurable sorting, filtering, and time range options. Includes database migration for updateat+id index to support efficient cursor-based queries when sorting by update_at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor posts reporting API cursor to opaque token and improve layer separation This addresses code review feedback by transforming the cursor from exposed fields to an opaque token and improving architectural layer separation. **Key Changes:** 1. **Opaque Cursor Implementation** - Transform cursor from split fields (cursor_time, cursor_id) to single opaque base64-encoded string - Cursor now self-contained with all query parameters embedded - When cursor provided, embedded parameters take precedence over request body - Clients treat cursor as opaque token and pass unchanged 2. **Field Naming** - Rename ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Now excludes ALL system posts (any type starting with "system_") - Clearer and more consistent naming 3. **Layer Separation** - Move cursor decoding from store layer to model layer - Create ReportPostQueryParams struct for resolved parameters - Store layer receives pre-resolved parameters (no business logic) - Add ResolveReportPostQueryParams() function in model layer 4. **Code Quality** - Add type-safe constants (ReportingTimeFieldCreateAt, ReportingSortDirectionAsc, etc.) - Replace magic number 9223372036854775807 with math.MaxInt64 - Remove debug SQL logging (info disclosure risk) - Update mmctl to use constants and fix NextCursor pointer access 5. **Tests** - Update all 17 store test calls to use new resolution pattern - Add comprehensive test for DESC + end_time boundary behavior 6. **API Documentation** - Update OpenAPI spec to reflect opaque cursor format - Update all request/response examples - Clarify end_time behavior with sort directions **Files Changed:** - Model layer: public/model/post.go - App layer: channels/app/report.go - Store layer: channels/store/store.go, channels/store/sqlstore/post_store.go - Tests: channels/store/storetest/post_store.go - Mocks: channels/store/storetest/mocks/PostStore.go - API: channels/api4/report.go, channels/api4/report_test.go - mmctl: cmd/mmctl/commands/report.go - Docs: api/v4/source/reports.yaml 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix unhandled parse errors in cursor decoding Address security finding: cursor decoding was silently ignoring parse errors from strconv functions, which could lead to unexpected behavior when malformed cursors are provided. Changes: - Add explicit error handling for strconv.Atoi (version parsing) - Add explicit error handling for strconv.ParseBool (includeDeleted, excludeSystemPosts) - Add explicit error handling for strconv.ParseInt (timestamp parsing) - Return clear error messages indicating which field failed to parse This prevents silent failures where malformed values would default to zero-values (0, false) and potentially alter query behavior without warning. Addresses DryRun Security finding: "Unhandled Errors in Cursor Parsing" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix linting issues - Remove unused reportPostCursorV1 struct (unused) - Remove obsolete +build comment (buildtag) - Use maps.Copy instead of manual loop (mapsloop) - Modernize for loop with range over int (rangeint) - Apply gofmt formatting 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix gofmt formatting issues Fix alignment in struct literals and constant declarations: - Align map keys in report_test.go request bodies - Align struct fields in ReportPostOptions initialization - Align reporting constant declarations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update mmctl tests for opaque cursor and add i18n translations Update report_test.go to align with the refactored Posts Reporting API: - Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag - Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Update all mock expectations to use new ReportPostOptionsCursor structure - Replace test cursor values with base64-encoded opaque cursor strings Add English translations for cursor decoding error messages in i18n/en.json. Minor API documentation fix in reports.yaml (remove "all" from description). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update mmctl tests for opaque cursor and add i18n translations Update report_test.go to align with the refactored Posts Reporting API: - Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag - Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Update all mock expectations to use new ReportPostOptionsCursor structure - Replace test cursor values with base64-encoded opaque cursor strings Add English translations for cursor decoding error messages in i18n/en.json. Minor API documentation fix in reports.yaml (remove "all" from description). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * more lint fixes * remove index update files * Remove end_time parameter from Posts Reporting API Align with other cursor-based APIs in the codebase by removing the end_time parameter. The caller now controls when to stop pagination by simply not making another request, which is the same pattern used by GetPostsSinceForSync, MessageExport, and GetPostsBatchForIndexing. Changes: - Remove EndTime field from ReportPostOptions and ReportPostQueryParams - Remove EndTime filtering logic from store layer - Remove tests that used end_time parameter 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor posts reporting API for security and validation Address security review feedback by consolidating parameter resolution and validation in the API layer, with comprehensive validation of all cursor fields to prevent SQL injection and invalid queries. Changes: - Move parameter resolution from model to API layer for clearer separation - Add ReportPostQueryParams.Validate() with inline validation for all fields - Validate ChannelId, TimeField, SortDirection, and CursorId format - Add start_time parameter for time-bounded queries - Cap per_page at 100-1000 instead of rejecting invalid values - Export DecodeReportPostCursorV1() for API layer use - Simplify app layer to receive pre-validated parameters - Check channel existence when results are empty (better error messages) Testing: - Add 10 model tests for validation and malformed cursor scenarios - Add 4 API tests for cursors with invalid field values - Refactor 13 store tests to use buildReportPostQueryParams() helper - All 31 tests pass Documentation: - Update OpenAPI spec with start_time, remove unused end_time - Update markdown docs with start_time examples Security improvements: - Whitelist validation prevents SQL injection in TimeField/SortDirection - Format validation ensures ChannelId and CursorId are valid IDs - Single validation point for both cursor and options paths - Defense in depth: validation + parameterized queries + store layer whitelist 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Improve posts reporting query efficiency and safety Replace SELECT * and nested OR/AND conditions with explicit column selection and PostgreSQL row value comparison for better performance and maintainability. Changes: - Use postSliceColumns() instead of SELECT * for explicit column selection - Replace Squirrel OR/AND with row value comparison: (timeField, Id) > (?, ?) - Use fmt.Sprintf for safer string formatting in WHERE clause Query improvements: Before: WHERE (CreateAt > ?) OR (CreateAt = ? AND Id > ?) After: WHERE (CreateAt, Id) > (?, ?) Benefits: - Explicit column selection prevents issues if table schema changes - Row value comparison is more concise and better optimized by PostgreSQL - Follows existing patterns in post_store.go (postSliceColumns) - Standard SQL:2003 syntax 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Change posts reporting response from map to ordered array Replace the Posts map with an ordered array to preserve query sort order and provide a more natural API response for sequential processing. Changes: - ReportPostListResponse.Posts: map[string]*Post → []*Post - Store layer returns posts array directly (already sorted by query) - App layer iterates by index for metadata enrichment - Remove applyPostsWillBeConsumedHook call (not applicable to reporting) - Update API tests to iterate arrays instead of map lookups - Update store tests to convert array to map for deduplication checks - Remove unused "maps" import Benefits: - Preserves query sort order (ASC/DESC, create_at/update_at) - More natural for sequential processing/export workflows - Simpler response structure for reporting/compliance use cases - Aligns with message export/compliance patterns (no plugin hooks) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix linting issues in posts reporting tests Replace inefficient loops with append(...) for better performance. Changes: - Use append(postSlice, result.Posts...) instead of loop - Simplifies code and follows staticcheck recommendations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix store test AppError nil checking Use require.Nil instead of require.NoError for *AppError returns to avoid Go interface nil pointer issues. When DecodeReportPostCursorV1 returns nil *AppError and it's assigned to error interface, the interface becomes non-nil even though the pointer is nil. This causes require.NoError to fail incorrectly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-17 11:02:19 -05:00
"encoding/base64"
2015-06-15 03:53:32 -04:00
"encoding/json"
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
"errors"
"fmt"
"io"
"maps"
"net/http"
"regexp"
"sort"
Add cursor-based Posts Reporting API for compliance and auditing (#34252) * Add cursor-based Posts Reporting API for compliance and auditing Implements a new admin-only endpoint for retrieving posts with efficient cursor-based pagination, designed for compliance, auditing, and archival workflows. Key Features: - Cursor-based pagination using composite (time, ID) keys for consistent performance regardless of dataset size (~10ms per page at any depth) - Flexible time range queries with optional upper/lower bounds - Support for both create_at and update_at time fields - Ascending or descending sort order - Optional metadata enrichment (files, reactions, acknowledgements) - System admin only access (requires manage_system permission) - License enforcement for compliance features API Endpoint: POST /api/v4/reports/posts - Request: JSON body with channel_id, cursor_time, cursor_id, and options - Response: Posts map + next_cursor object (null when pagination complete) - Max page size: 1000 posts per request (MaxReportingPerPage constant) Implementation: - Store Layer: Direct SQL queries with composite index on (ChannelId, CreateAt, Id) - App Layer: Permission checks, optional metadata enrichment, post hooks - API Layer: Parameter validation, system admin enforcement, license checks - Data Model: ReportPostOptions, ReportPostOptionsCursor, ReportPostListResponse Code Quality Improvements: - Added MaxReportingPerPage constant (1000) to eliminate magic numbers - Removed unused StartTime field from ReportPostOptions - Added fmt import for dynamic error messages Testing: - 14 comprehensive store layer unit tests - 12 API layer integration tests covering permissions, pagination, filters - All tests passing Documentation: - POSTS_REPORTING.md: Developer reference with Go structs and usage examples - POSTS_REPORTING_API_SPEC.md: Complete technical specification - GET_POSTS_API_IMPROVEMENTS.md: Implementation analysis and design rationale - POSTS_TIME_RANGE_FEATURE.md: Archived time range feature for future use Performance: Cursor-based pagination maintains consistent ~10ms query time at any dataset depth, compared to offset-based pagination which degrades significantly (Page 1 = 10ms, Page 1000 = 10 seconds). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * lint fixes * lint fixes * gofmt * i18n-extract * Add Enterprise license requirement to posts reporting API Enforce Enterprise license (tier 20+) for the new posts reporting endpoint to align with compliance feature licensing. Professional tier is insufficient. Changes: - Add MinimumEnterpriseLicense check in GetPostsForReporting app layer - Add test coverage for license validation (no license and Professional tier) All existing tests pass with new license enforcement. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * i18n-extract * add licensing to api documentation * Test SSH signing * Add mmctl command for posts reporting API Adds mmctl report posts command to retrieve posts from a channel for administrative reporting purposes. Supports cursor-based pagination with configurable sorting, filtering, and time range options. Includes database migration for updateat+id index to support efficient cursor-based queries when sorting by update_at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor posts reporting API cursor to opaque token and improve layer separation This addresses code review feedback by transforming the cursor from exposed fields to an opaque token and improving architectural layer separation. **Key Changes:** 1. **Opaque Cursor Implementation** - Transform cursor from split fields (cursor_time, cursor_id) to single opaque base64-encoded string - Cursor now self-contained with all query parameters embedded - When cursor provided, embedded parameters take precedence over request body - Clients treat cursor as opaque token and pass unchanged 2. **Field Naming** - Rename ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Now excludes ALL system posts (any type starting with "system_") - Clearer and more consistent naming 3. **Layer Separation** - Move cursor decoding from store layer to model layer - Create ReportPostQueryParams struct for resolved parameters - Store layer receives pre-resolved parameters (no business logic) - Add ResolveReportPostQueryParams() function in model layer 4. **Code Quality** - Add type-safe constants (ReportingTimeFieldCreateAt, ReportingSortDirectionAsc, etc.) - Replace magic number 9223372036854775807 with math.MaxInt64 - Remove debug SQL logging (info disclosure risk) - Update mmctl to use constants and fix NextCursor pointer access 5. **Tests** - Update all 17 store test calls to use new resolution pattern - Add comprehensive test for DESC + end_time boundary behavior 6. **API Documentation** - Update OpenAPI spec to reflect opaque cursor format - Update all request/response examples - Clarify end_time behavior with sort directions **Files Changed:** - Model layer: public/model/post.go - App layer: channels/app/report.go - Store layer: channels/store/store.go, channels/store/sqlstore/post_store.go - Tests: channels/store/storetest/post_store.go - Mocks: channels/store/storetest/mocks/PostStore.go - API: channels/api4/report.go, channels/api4/report_test.go - mmctl: cmd/mmctl/commands/report.go - Docs: api/v4/source/reports.yaml 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix unhandled parse errors in cursor decoding Address security finding: cursor decoding was silently ignoring parse errors from strconv functions, which could lead to unexpected behavior when malformed cursors are provided. Changes: - Add explicit error handling for strconv.Atoi (version parsing) - Add explicit error handling for strconv.ParseBool (includeDeleted, excludeSystemPosts) - Add explicit error handling for strconv.ParseInt (timestamp parsing) - Return clear error messages indicating which field failed to parse This prevents silent failures where malformed values would default to zero-values (0, false) and potentially alter query behavior without warning. Addresses DryRun Security finding: "Unhandled Errors in Cursor Parsing" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix linting issues - Remove unused reportPostCursorV1 struct (unused) - Remove obsolete +build comment (buildtag) - Use maps.Copy instead of manual loop (mapsloop) - Modernize for loop with range over int (rangeint) - Apply gofmt formatting 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix gofmt formatting issues Fix alignment in struct literals and constant declarations: - Align map keys in report_test.go request bodies - Align struct fields in ReportPostOptions initialization - Align reporting constant declarations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update mmctl tests for opaque cursor and add i18n translations Update report_test.go to align with the refactored Posts Reporting API: - Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag - Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Update all mock expectations to use new ReportPostOptionsCursor structure - Replace test cursor values with base64-encoded opaque cursor strings Add English translations for cursor decoding error messages in i18n/en.json. Minor API documentation fix in reports.yaml (remove "all" from description). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update mmctl tests for opaque cursor and add i18n translations Update report_test.go to align with the refactored Posts Reporting API: - Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag - Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Update all mock expectations to use new ReportPostOptionsCursor structure - Replace test cursor values with base64-encoded opaque cursor strings Add English translations for cursor decoding error messages in i18n/en.json. Minor API documentation fix in reports.yaml (remove "all" from description). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * more lint fixes * remove index update files * Remove end_time parameter from Posts Reporting API Align with other cursor-based APIs in the codebase by removing the end_time parameter. The caller now controls when to stop pagination by simply not making another request, which is the same pattern used by GetPostsSinceForSync, MessageExport, and GetPostsBatchForIndexing. Changes: - Remove EndTime field from ReportPostOptions and ReportPostQueryParams - Remove EndTime filtering logic from store layer - Remove tests that used end_time parameter 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor posts reporting API for security and validation Address security review feedback by consolidating parameter resolution and validation in the API layer, with comprehensive validation of all cursor fields to prevent SQL injection and invalid queries. Changes: - Move parameter resolution from model to API layer for clearer separation - Add ReportPostQueryParams.Validate() with inline validation for all fields - Validate ChannelId, TimeField, SortDirection, and CursorId format - Add start_time parameter for time-bounded queries - Cap per_page at 100-1000 instead of rejecting invalid values - Export DecodeReportPostCursorV1() for API layer use - Simplify app layer to receive pre-validated parameters - Check channel existence when results are empty (better error messages) Testing: - Add 10 model tests for validation and malformed cursor scenarios - Add 4 API tests for cursors with invalid field values - Refactor 13 store tests to use buildReportPostQueryParams() helper - All 31 tests pass Documentation: - Update OpenAPI spec with start_time, remove unused end_time - Update markdown docs with start_time examples Security improvements: - Whitelist validation prevents SQL injection in TimeField/SortDirection - Format validation ensures ChannelId and CursorId are valid IDs - Single validation point for both cursor and options paths - Defense in depth: validation + parameterized queries + store layer whitelist 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Improve posts reporting query efficiency and safety Replace SELECT * and nested OR/AND conditions with explicit column selection and PostgreSQL row value comparison for better performance and maintainability. Changes: - Use postSliceColumns() instead of SELECT * for explicit column selection - Replace Squirrel OR/AND with row value comparison: (timeField, Id) > (?, ?) - Use fmt.Sprintf for safer string formatting in WHERE clause Query improvements: Before: WHERE (CreateAt > ?) OR (CreateAt = ? AND Id > ?) After: WHERE (CreateAt, Id) > (?, ?) Benefits: - Explicit column selection prevents issues if table schema changes - Row value comparison is more concise and better optimized by PostgreSQL - Follows existing patterns in post_store.go (postSliceColumns) - Standard SQL:2003 syntax 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Change posts reporting response from map to ordered array Replace the Posts map with an ordered array to preserve query sort order and provide a more natural API response for sequential processing. Changes: - ReportPostListResponse.Posts: map[string]*Post → []*Post - Store layer returns posts array directly (already sorted by query) - App layer iterates by index for metadata enrichment - Remove applyPostsWillBeConsumedHook call (not applicable to reporting) - Update API tests to iterate arrays instead of map lookups - Update store tests to convert array to map for deduplication checks - Remove unused "maps" import Benefits: - Preserves query sort order (ASC/DESC, create_at/update_at) - More natural for sequential processing/export workflows - Simpler response structure for reporting/compliance use cases - Aligns with message export/compliance patterns (no plugin hooks) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix linting issues in posts reporting tests Replace inefficient loops with append(...) for better performance. Changes: - Use append(postSlice, result.Posts...) instead of loop - Simplifies code and follows staticcheck recommendations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix store test AppError nil checking Use require.Nil instead of require.NoError for *AppError returns to avoid Go interface nil pointer issues. When DecodeReportPostCursorV1 returns nil *AppError and it's assigned to error interface, the interface becomes non-nil even though the pointer is nil. This causes require.NoError to fail incorrectly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-17 11:02:19 -05:00
"strconv"
2017-09-27 11:09:09 -04:00
"strings"
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
"sync"
"unicode/utf8"
"github.com/hashicorp/go-multierror"
"github.com/mattermost/mattermost/server/public/shared/markdown"
"github.com/mattermost/mattermost/server/public/shared/mlog"
2015-06-15 03:53:32 -04:00
)
[MM-61758] Burn on read feature (#34703) * Add read receipt store for burn on read message types * update mocks * fix invalidation target * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * add translations * Added logic to associate files of BoR post with the post * Added test * fixes * disable pinning posts and review comments * MM-66594 - Burn on read UI integration (#34647) * MM-66244 - add BoR visual components to message editor * MM-66246 - BoR visual indicator for sender and receiver * MM-66607 - bor - add timer countdown and autodeletion * add the system console max time to live config * use the max expire at and create global scheduler to register bor messages * use seconds for BoR config values in BE * implement the read by text shown in the tooltip logic * unestack the posts from same receiver and BoR and fix styling * avoid opening reply RHS * remove unused dispatchers * persis the BoR label in the drafts * move expiration value to metadata * adjust unit tests to metadata insted of props * code clean up and some performance improvements; add period grace for deletion too * adjust migration serie number * hide bor messages when config is off * performance improvements on post component and code clean up * keep bor existing post functionality if config is disabled * Add read receipt store for burn on read message types * Add temporary posts table * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * avoid reacting to unrevealed bor messages * adjust migration number * Add read receipt store for burn on read message types * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * adjust post reveal and type with backend changes * use real config values, adjust icon usage and style * adjust the delete from from sender and receiver * improve self deleting logic by placing in badge, use burn endpoint * adjust websocket events handling for the read by sender label information * adjust styling for concealed and error state * update burn-on-read post event handling for improved recipient tracking and multi-device sync * replace burn_on_read with type in database migrations and model * remove burn_on_read metadata from PostMetadata and related structures * Added logic to associate files of BoR post with the post * Added test * adjust migration name and fix linter * Add read receipt store for burn on read message types * update mocks * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * Added logic to associate files of BoR post with the post * Added test * disable pinning posts and review comments * show attachment on bor reveal * remove unused translation * Enhance burn-on-read post handling and refine previous post ID retrieval logic * adjust the returning chunk to work with bor messages * read temp post from master db * read from master * show the copy link button to the sender * revert unnecessary check * restore correct json tag * remove unused error handling and clarify burn-on-read comment * improve type safety and use proper selectors * eliminate code duplication in deletion handler * optimize performance and add documentation * delete bor message for sender once all receivers reveal it * add burn on read to scheduled posts * add feature enable check * use master to avoid all read recipients race condition --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> * squash migrations into single file * add configuration for the scheduler * don't run messagehasbeenposted hook * remove parallel tests on burn on read * add clean up for closing opened modals from previous tests * simplify delete menu item rendering * add cleanup step to close open modals after each test to prevent pollution * streamline delete button visibility logic for Burn on Read posts * improve reliability of closing post menu and modals by using body ESC key --------- Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> Co-authored-by: Pablo Vélez <pablovv2012@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2025-12-11 01:59:50 -05:00
type PostContextKey string
2015-06-15 03:53:32 -04:00
const (
PostSystemMessagePrefix = "system_"
PostTypeDefault = ""
PostTypeMessageAttachment = "slack_attachment"
PostTypeSystemGeneric = "system_generic"
PostTypeJoinLeave = "system_join_leave" // Deprecated, use PostJoinChannel or PostLeaveChannel instead
PostTypeJoinChannel = "system_join_channel"
PostTypeGuestJoinChannel = "system_guest_join_channel"
PostTypeLeaveChannel = "system_leave_channel"
PostTypeJoinTeam = "system_join_team"
PostTypeLeaveTeam = "system_leave_team"
PostTypeAutoResponder = "system_auto_responder"
PostTypeAutotranslationChange = "system_autotranslation"
PostTypeAddRemove = "system_add_remove" // Deprecated, use PostAddToChannel or PostRemoveFromChannel instead
PostTypeAddToChannel = "system_add_to_channel"
PostTypeAddGuestToChannel = "system_add_guest_to_chan"
PostTypeRemoveFromChannel = "system_remove_from_channel"
PostTypeMoveChannel = "system_move_channel"
PostTypeAddToTeam = "system_add_to_team"
PostTypeRemoveFromTeam = "system_remove_from_team"
PostTypeHeaderChange = "system_header_change"
PostTypeDisplaynameChange = "system_displayname_change"
PostTypeConvertChannel = "system_convert_channel"
PostTypePurposeChange = "system_purpose_change"
PostTypeChannelDeleted = "system_channel_deleted"
PostTypeChannelRestored = "system_channel_restored"
PostTypeEphemeral = "system_ephemeral"
PostTypeChangeChannelPrivacy = "system_change_chan_privacy"
PostTypeWrangler = "system_wrangler"
PostTypeGMConvertedToChannel = "system_gm_to_channel"
PostTypeAddBotTeamsChannels = "add_bot_teams_channels"
PostTypeMe = "me"
PostCustomTypePrefix = "custom_"
PostTypeReminder = "reminder"
PostTypeBurnOnRead = "burn_on_read"
Merge the Integrated Boards MVP feature branch (#35796) * Add CreatedBy and UpdatedBy to the properties fields and values (#34485) * Add CreatedBy and UpdatedBy to the properties fields and values * Fix types --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds ObjectType to the property fields table (#34908) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Update ObjectType migration setting an empty value and marking the column as not null (#34915) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds uniqueness mechanisms to the property fields (#35058) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Fixing retrylayer mocks * Remove retrylayer duplication * Address review comments * Fix comment to avoid linter issues * Address PR comments * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.down.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update field validation to check only for valid target types * Update migrations to avoid concurrent index creation within a transaction * Update migrations to make all index ops concurrent * Update tests to use valid PSAv2 property fields * Adds a helper for valid PSAv2 TargetTypes --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Fix property tests (#35388) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards feature flag (#35378) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards MVP API changes (#34822) This PR includes the necessary changes for channels and posts endpoints and adds a set of generic endpoints to retrieve and manage property fields and values following the new Property System approach. Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> * Property System Architecture permissions for v2 (#35113) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Fix i18n sorting --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Add Views store and app layer (#35361) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor test loops in ViewStore tests for improved readability * change pagination to limit/offset * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * MM-67388, MM-66528, MM-67750: Add View REST API endpoints, websocket events, and sort order (#35442) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Add View API endpoints with OpenAPI spec, client methods, and i18n Implement REST API for channel views (board-type) behind the IntegratedBoards feature flag. Adds CRUD endpoints under /api/v4/channels/{channel_id}/views with permission checks matching the channel bookmark pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add cursor-based pagination to View API for channel views * Enhance cursor handling in getViewsForChannel and update tests for pagination * Refactor test loops in ViewStore tests for improved readability * Refactor loop in TestGetViewsForChannel for improved readability * change pagination to limit/offset * switch to limit/offset pagination * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add view sort order API endpoint Add POST /api/v4/channels/{channel_id}/views/{view_id}/sort_order endpoint following the channel bookmarks reorder pattern. Includes store, app, and API layers with full test coverage at each layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add connectionId to view WebSocket events and sort_order API spec Thread connectionId from request header through all view handlers (create, update, delete, sort_order) to WebSocket events, matching the channel bookmarks pattern. Add sort_order endpoint to OpenAPI spec. Update minimum server version to 11.6. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove duplicate View/ViewPatch definitions from definitions.yaml The merge from integrated-boards-mvp introduced duplicate View and ViewPatch schema definitions that were already defined earlier in the file with more detail (including ViewBoardProps ref and enums). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update minimum server version to 11.6 in views API spec Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add missing translations for view sort order error messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Merge integrated-boards-mvp into ibmvp_api-views; remove spec files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix flaky TestViewStore timestamp test on CI Add sleep before UpdateSortOrder to ensure timestamps differ, preventing same-millisecond comparisons on fast CI machines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove duplicate views.yaml imclude * Use c.boolString() for include_deleted query param in GetViewsForChannel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix views.yaml sort order schema: use integer type and require body Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor view sort order tests to use named IDs instead of array indices Extract idA/idB/idC from views slice and add BEFORE/AFTER comments to make stateful subtest ordering easier to follow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Return 404 instead of 403 for view operations on deleted channels Deleted channels should appear non-existent to callers rather than revealing their existence via a 403. Detailed error text explains the context for debugging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * add missing channel deleteat checks * Use c.Params.Page instead of manual page query param parsing in getViewsForChannel c.Params already validates and defaults page/per_page, so the manual parsing was redundant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add support for total count in views retrieval * Add tests for handling deleted views in GetViewsForChannel and GetView * Short-circuit negative newIndex in UpdateSortOrder before opening transaction Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add per-channel limit on views to bound UpdateSortOrder cost Without a cap, unbounded view creation makes sort-order updates increasingly expensive (CASE WHEN per view, row locks). Adds MaxViewsPerChannel=50 constant and enforces it in the app layer before saving. Includes API and app layer tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove include_deleted support from views API Soft-deleted views are structural metadata with low risk, but no other similar endpoint (e.g. channel bookmarks) exposes deleted records without an admin gate. Rather than adding an admin-only permission check for consistency, remove the feature entirely since there is no current use case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update view permissions to require `create_post` instead of channel management permissions * Remove obsolete view management error messages for direct and group messages --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(migrations): add user tracking and object type to property fields - Introduced user tracking columns (CreatedBy, UpdatedBy) to PropertyFields and PropertyValues. - Added ObjectType column to PropertyFields with associated unique indexes for legacy and typed properties. - Created new migration scripts for adding and dropping these features, including necessary indexes for data integrity. - Established views for managing property fields with new attributes. This update enhances the schema to support better tracking and categorization of property fields. * Add Property System Architecture v2 API endpoints (#35583) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Require target_type filter when searching property fields * Add objectType validation as part of field.IsValid() * Fix linter * Fix test with bad objecttpye * Fix test grouping --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * MM-67968: Flatten view model — remove icon, subviews, typed board props (#35726) * feat(views): flatten view model by removing icon, subview, and board props Simplifies the View data model as part of MM-67968: removes Icon, Subview, and ViewBoardProps types; renames ViewTypeBoard to ViewTypeKanban; replaces typed Props with StringInterface (map[string]any); adds migration 000167 to drop the Icon column from the Views table. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(api): update views OpenAPI spec to reflect flattened model Removes ViewBoardProps, Subview, and icon from the View and ViewPatch schemas. Changes type enum from board to kanban. Replaces typed props with a free-form StringInterface object. Aligns with MM-67968. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * refactor(views): simplify store by dropping dbView and marshalViewProps StringInterface already implements driver.Valuer and sql.Scanner, so the manual JSON marshal/unmarshal and the dbView intermediate struct were redundant. model.View now scans directly from the database. Also removes the dead ViewMaxLinkedProperties constant and wraps the Commit() error in UpdateSortOrder. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(api): allow arbitrary JSON in view props OpenAPI schema The props field was restricted to string values via additionalProperties: { type: string }, conflicting with the Go model's StringInterface (map[string]any). Changed to additionalProperties: true in View, ViewPatch, and inline POST schemas. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Adds basic implementation of the generic redux store for PSAv2 (#35512) * Adds basic implementation of the generic redux store for PSAv2 * Add created_by and updated_by to the test fixtures * Make target_id, target_type and object_type mandatory * Wrap getPropertyFieldsByIds and getPropertyValuesForTargetByFieldIds with createSelector * Address PR comments --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds websocket messages for the PSAv2 API events (#35696) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Adds websocket messages for the PSAv2 API events * Add IsPSAv2 helper to the property field for clarity * Add guard against nil returns on field deletion * Add docs to the websocket endpoints --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * migrations: consolidate views migrations and reorder after master - Merged 000165 (create Views) with 000167 (drop Icon) since Icon was never needed - Renumbered branch migrations 159-166 → 160-167 so master's 000159 (deduplicate_policy_names) runs first - Regenerated migrations.list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add API endpoint to retrieve posts for a specific view (#35604) Automatic Merge * Apply fixes after merge * Return a more specific error from getting multiple fields * Prevent getting broadcast params on field deletion if not needed * Remove duplicated migration code * Update property conflict code to always use master * Adds nil guard when iterating on property fields * Check that permission level is valid before getting rejected by the database * Validate correctness on TargetID for PSAv2 fields * Avoid PSAv1 using permissions or protected * Fix test data after validation change * Fix flaky search test * Adds more posts for filter use cases to properly test exclusions --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> Co-authored-by: Julien Tant <julien@craftyx.fr> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 05:36:35 -04:00
PostTypeCard = "card"
2021-07-12 14:05:36 -04:00
PostFileidsMaxRunes = 300
PostFilenamesMaxRunes = 4000
PostHashtagsMaxRunes = 1000
PostMessageMaxRunesV1 = 4000
Remove vestigial MySQL support (#34865) * Remove legacy quoteColumnName() utility Since Mattermost only supports PostgreSQL, the quoteColumnName() helper that was designed to handle database-specific column quoting is no longer needed. The function was a no-op that simply returned the column name unchanged. Remove the function from utils.go and update status_store.go to use the "Manual" column name directly. * Remove legacy driver checks from store.go Since Mattermost only supports PostgreSQL, remove conditional checks for different database drivers: - Simplify specialSearchChars() to always return PostgreSQL-compatible chars - Remove driver check from computeBinaryParam() - Remove driver check from computeDefaultTextSearchConfig() - Simplify GetDbVersion() to use PostgreSQL syntax directly - Remove switch statement from ensureMinimumDBVersion() - Remove unused driver parameter from versionString() * Remove MySQL alternatives for batch delete operations Since Mattermost only supports PostgreSQL, remove the MySQL-specific DELETE...LIMIT syntax and keep only the PostgreSQL array-based approach: - reaction_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch - file_info_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch - preference_store.go: Use PostgreSQL tuple IN subquery for DeleteInvalidVisibleDmsGms * Remove MySQL alternatives for UPDATE...FROM syntax Since Mattermost only supports PostgreSQL, remove the MySQL-specific UPDATE syntax that joins tables differently: - thread_store.go: Use PostgreSQL UPDATE...FROM syntax in MarkAllAsReadByChannels and MarkAllAsReadByTeam - post_store.go: Use PostgreSQL UPDATE...FROM syntax in deleteThreadFiles * Remove MySQL alternatives for JSON and subquery operations Since Mattermost only supports PostgreSQL, remove the MySQL-specific JSON and subquery syntax: - thread_store.go: Use PostgreSQL JSONB operators for updating participants - access_control_policy_store.go: Use PostgreSQL JSONB @> operator for querying JSON imports - session_store.go: Use PostgreSQL subquery syntax for Cleanup - job_store.go: Use PostgreSQL subquery syntax for Cleanup * Remove MySQL alternatives for CTE queries Since Mattermost only supports PostgreSQL, simplify code that uses CTEs (Common Table Expressions): - channel_store.go: Remove MySQL CASE-based fallback in UpdateLastViewedAt and use PostgreSQL CTE exclusively - draft_store.go: Remove driver checks in DeleteEmptyDraftsByCreateAtAndUserId, DeleteOrphanDraftsByCreateAtAndUserId, and determineMaxDraftSize * Remove driver checks in migrate.go and schema_dump.go Simplify migration code to use PostgreSQL driver directly since PostgreSQL is the only supported database. * Remove driver checks in sqlx_wrapper.go Always apply lowercase named parameter transformation since PostgreSQL is the only supported database. * Remove driver checks in user_store.go Simplify user store functions to use PostgreSQL-only code paths: - Remove isPostgreSQL parameter from helper functions - Use LEFT JOIN pattern instead of subqueries for bot filtering - Always use case-insensitive LIKE with lower() for search - Remove MySQL-specific role filtering alternatives * Remove driver checks in post_store.go Simplify post_store.go to use PostgreSQL-only code paths: - Inline getParentsPostsPostgreSQL into getParentsPosts - Use PostgreSQL TO_CHAR/TO_TIMESTAMP for date formatting in analytics - Use PostgreSQL array syntax for batch deletes - Simplify determineMaxPostSize to always use information_schema - Use PostgreSQL jsonb subtraction for thread participants - Always execute RefreshPostStats (PostgreSQL materialized views) - Use materialized views for AnalyticsPostCountsByDay - Simplify AnalyticsPostCountByTeam to always use countByTeam * Remove driver checks in channel_store.go Simplify channel_store.go to use PostgreSQL-only code paths: - Always use sq.Dollar.ReplacePlaceholders for UNION queries - Use PostgreSQL LEFT JOIN for retention policy exclusion - Use PostgreSQL jsonb @> operator for access control policy imports - Simplify buildLIKEClause to always use LOWER() for case-insensitive search - Simplify buildFulltextClauseX to always use PostgreSQL to_tsvector/to_tsquery - Simplify searchGroupChannelsQuery to use ARRAY_TO_STRING/ARRAY_AGG * Remove driver checks in file_info_store.go Simplify file_info_store.go to use PostgreSQL-only code paths: - Always use PostgreSQL to_tsvector/to_tsquery for file search - Use file_stats materialized view for CountAll() - Use file_stats materialized view for GetStorageUsage() when not including deleted - Always execute RefreshFileStats() for materialized view refresh * Remove driver checks in attributes_store.go Simplify attributes_store.go to use PostgreSQL-only code paths: - Always execute RefreshAttributes() for materialized view refresh - Remove isPostgreSQL parameter from generateSearchQueryForExpression - Always use PostgreSQL LOWER() LIKE LOWER() syntax for case-insensitive search * Remove driver checks in retention_policy_store.go Simplify retention_policy_store.go to use PostgreSQL-only code paths: - Remove isPostgres parameter from scanRetentionIdsForDeletion - Always use pq.Array for scanning retention IDs - Always use pq.Array for inserting retention IDs - Remove unused json import * Remove driver checks in property stores Simplify property_field_store.go and property_value_store.go to use PostgreSQL-only code paths: - Always use PostgreSQL type casts (::text, ::jsonb, ::bigint, etc.) - Remove isPostgres variable and conditionals * Remove driver checks in channel_member_history_store.go Simplify PermanentDeleteBatch to use PostgreSQL-only code path: - Always use ctid-based subquery for DELETE with LIMIT * Remove remaining driver checks in user_store.go Simplify user_store.go to use PostgreSQL-only code paths: - Use LEFT JOIN for bot exclusion in AnalyticsActiveCountForPeriod - Use LEFT JOIN for bot exclusion in IsEmpty * Simplify fulltext search by consolidating buildFulltextClause functions Remove convertMySQLFullTextColumnsToPostgres and consolidate buildFulltextClause and buildFulltextClauseX into a single function that takes variadic column arguments and returns sq.Sqlizer. * Simplify SQL stores leveraging PostgreSQL-only support - Simplify UpdateMembersRole in channel_store.go and team_store.go to use UPDATE...RETURNING instead of SELECT + UPDATE - Simplify GetPostReminders in post_store.go to use DELETE...RETURNING - Simplify DeleteOrphanedRows queries by removing MySQL workarounds for subquery locking issues - Simplify UpdateUserLastSyncAt to use UPDATE...FROM...RETURNING instead of fetching user first then updating - Remove MySQL index hint workarounds in ORDER BY clauses - Update outdated comments referencing MySQL - Consolidate buildFulltextClause and remove convertMySQLFullTextColumnsToPostgres * Remove MySQL-specific test artifacts - Delete unused MySQLStopWords variable and stop_word.go file - Remove redundant testSearchEmailAddressesWithQuotes test (already covered by testSearchEmailAddresses) - Update comment that referenced MySQL query planning * Remove MySQL references from server code outside sqlstore - Update config example and DSN parsing docs to reflect PostgreSQL-only support - Remove mysql:// scheme check from IsDatabaseDSN - Simplify SanitizeDataSource to only handle PostgreSQL - Remove outdated MySQL comments from model and plugin code * Remove MySQL references from test files - Update test DSNs to use PostgreSQL format - Remove dead mysql-replica flag and replicaFlag variable - Simplify tests that had MySQL/PostgreSQL branches * Update docs and test config to use PostgreSQL - Update mmctl config set example to use postgres driver - Update test-config.json to use PostgreSQL DSN format * Remove MySQL migration scripts, test data, and docker image Delete MySQL-related files that are no longer needed: - ESR upgrade scripts (esr.*.mysql.*.sql) - MySQL schema dumps (mattermost-mysql-*.sql) - MySQL replication test scripts (replica-*.sh, mysql-migration-test.sh) - MySQL test warmup data (mysql_migration_warmup.sql) - MySQL docker image reference from mirror-docker-images.json * Remove MySQL references from webapp - Simplify minimumHashtagLength description to remove MySQL-specific configuration note - Remove unused HIDE_MYSQL_STATS_NOTIFICATION preference constant - Update en.json i18n source file * clean up e2e-tests * rm server/tests/template.load * Use teamMemberSliceColumns() in UpdateMembersRole RETURNING clause Refactor to use the existing helper function instead of hardcoding the column names, ensuring consistency if the columns are updated. * u.id -> u.Id * address code review feedback --------- Co-authored-by: Mattermost Build <build@mattermost.com>
2026-01-20 16:01:59 -05:00
PostMessageMaxBytesV2 = 65535
2021-07-12 14:05:36 -04:00
PostMessageMaxRunesV2 = PostMessageMaxBytesV2 / 4 // Assume a worst-case representation
Add cursor-based Posts Reporting API for compliance and auditing (#34252) * Add cursor-based Posts Reporting API for compliance and auditing Implements a new admin-only endpoint for retrieving posts with efficient cursor-based pagination, designed for compliance, auditing, and archival workflows. Key Features: - Cursor-based pagination using composite (time, ID) keys for consistent performance regardless of dataset size (~10ms per page at any depth) - Flexible time range queries with optional upper/lower bounds - Support for both create_at and update_at time fields - Ascending or descending sort order - Optional metadata enrichment (files, reactions, acknowledgements) - System admin only access (requires manage_system permission) - License enforcement for compliance features API Endpoint: POST /api/v4/reports/posts - Request: JSON body with channel_id, cursor_time, cursor_id, and options - Response: Posts map + next_cursor object (null when pagination complete) - Max page size: 1000 posts per request (MaxReportingPerPage constant) Implementation: - Store Layer: Direct SQL queries with composite index on (ChannelId, CreateAt, Id) - App Layer: Permission checks, optional metadata enrichment, post hooks - API Layer: Parameter validation, system admin enforcement, license checks - Data Model: ReportPostOptions, ReportPostOptionsCursor, ReportPostListResponse Code Quality Improvements: - Added MaxReportingPerPage constant (1000) to eliminate magic numbers - Removed unused StartTime field from ReportPostOptions - Added fmt import for dynamic error messages Testing: - 14 comprehensive store layer unit tests - 12 API layer integration tests covering permissions, pagination, filters - All tests passing Documentation: - POSTS_REPORTING.md: Developer reference with Go structs and usage examples - POSTS_REPORTING_API_SPEC.md: Complete technical specification - GET_POSTS_API_IMPROVEMENTS.md: Implementation analysis and design rationale - POSTS_TIME_RANGE_FEATURE.md: Archived time range feature for future use Performance: Cursor-based pagination maintains consistent ~10ms query time at any dataset depth, compared to offset-based pagination which degrades significantly (Page 1 = 10ms, Page 1000 = 10 seconds). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * lint fixes * lint fixes * gofmt * i18n-extract * Add Enterprise license requirement to posts reporting API Enforce Enterprise license (tier 20+) for the new posts reporting endpoint to align with compliance feature licensing. Professional tier is insufficient. Changes: - Add MinimumEnterpriseLicense check in GetPostsForReporting app layer - Add test coverage for license validation (no license and Professional tier) All existing tests pass with new license enforcement. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * i18n-extract * add licensing to api documentation * Test SSH signing * Add mmctl command for posts reporting API Adds mmctl report posts command to retrieve posts from a channel for administrative reporting purposes. Supports cursor-based pagination with configurable sorting, filtering, and time range options. Includes database migration for updateat+id index to support efficient cursor-based queries when sorting by update_at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor posts reporting API cursor to opaque token and improve layer separation This addresses code review feedback by transforming the cursor from exposed fields to an opaque token and improving architectural layer separation. **Key Changes:** 1. **Opaque Cursor Implementation** - Transform cursor from split fields (cursor_time, cursor_id) to single opaque base64-encoded string - Cursor now self-contained with all query parameters embedded - When cursor provided, embedded parameters take precedence over request body - Clients treat cursor as opaque token and pass unchanged 2. **Field Naming** - Rename ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Now excludes ALL system posts (any type starting with "system_") - Clearer and more consistent naming 3. **Layer Separation** - Move cursor decoding from store layer to model layer - Create ReportPostQueryParams struct for resolved parameters - Store layer receives pre-resolved parameters (no business logic) - Add ResolveReportPostQueryParams() function in model layer 4. **Code Quality** - Add type-safe constants (ReportingTimeFieldCreateAt, ReportingSortDirectionAsc, etc.) - Replace magic number 9223372036854775807 with math.MaxInt64 - Remove debug SQL logging (info disclosure risk) - Update mmctl to use constants and fix NextCursor pointer access 5. **Tests** - Update all 17 store test calls to use new resolution pattern - Add comprehensive test for DESC + end_time boundary behavior 6. **API Documentation** - Update OpenAPI spec to reflect opaque cursor format - Update all request/response examples - Clarify end_time behavior with sort directions **Files Changed:** - Model layer: public/model/post.go - App layer: channels/app/report.go - Store layer: channels/store/store.go, channels/store/sqlstore/post_store.go - Tests: channels/store/storetest/post_store.go - Mocks: channels/store/storetest/mocks/PostStore.go - API: channels/api4/report.go, channels/api4/report_test.go - mmctl: cmd/mmctl/commands/report.go - Docs: api/v4/source/reports.yaml 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix unhandled parse errors in cursor decoding Address security finding: cursor decoding was silently ignoring parse errors from strconv functions, which could lead to unexpected behavior when malformed cursors are provided. Changes: - Add explicit error handling for strconv.Atoi (version parsing) - Add explicit error handling for strconv.ParseBool (includeDeleted, excludeSystemPosts) - Add explicit error handling for strconv.ParseInt (timestamp parsing) - Return clear error messages indicating which field failed to parse This prevents silent failures where malformed values would default to zero-values (0, false) and potentially alter query behavior without warning. Addresses DryRun Security finding: "Unhandled Errors in Cursor Parsing" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix linting issues - Remove unused reportPostCursorV1 struct (unused) - Remove obsolete +build comment (buildtag) - Use maps.Copy instead of manual loop (mapsloop) - Modernize for loop with range over int (rangeint) - Apply gofmt formatting 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix gofmt formatting issues Fix alignment in struct literals and constant declarations: - Align map keys in report_test.go request bodies - Align struct fields in ReportPostOptions initialization - Align reporting constant declarations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update mmctl tests for opaque cursor and add i18n translations Update report_test.go to align with the refactored Posts Reporting API: - Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag - Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Update all mock expectations to use new ReportPostOptionsCursor structure - Replace test cursor values with base64-encoded opaque cursor strings Add English translations for cursor decoding error messages in i18n/en.json. Minor API documentation fix in reports.yaml (remove "all" from description). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update mmctl tests for opaque cursor and add i18n translations Update report_test.go to align with the refactored Posts Reporting API: - Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag - Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Update all mock expectations to use new ReportPostOptionsCursor structure - Replace test cursor values with base64-encoded opaque cursor strings Add English translations for cursor decoding error messages in i18n/en.json. Minor API documentation fix in reports.yaml (remove "all" from description). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * more lint fixes * remove index update files * Remove end_time parameter from Posts Reporting API Align with other cursor-based APIs in the codebase by removing the end_time parameter. The caller now controls when to stop pagination by simply not making another request, which is the same pattern used by GetPostsSinceForSync, MessageExport, and GetPostsBatchForIndexing. Changes: - Remove EndTime field from ReportPostOptions and ReportPostQueryParams - Remove EndTime filtering logic from store layer - Remove tests that used end_time parameter 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor posts reporting API for security and validation Address security review feedback by consolidating parameter resolution and validation in the API layer, with comprehensive validation of all cursor fields to prevent SQL injection and invalid queries. Changes: - Move parameter resolution from model to API layer for clearer separation - Add ReportPostQueryParams.Validate() with inline validation for all fields - Validate ChannelId, TimeField, SortDirection, and CursorId format - Add start_time parameter for time-bounded queries - Cap per_page at 100-1000 instead of rejecting invalid values - Export DecodeReportPostCursorV1() for API layer use - Simplify app layer to receive pre-validated parameters - Check channel existence when results are empty (better error messages) Testing: - Add 10 model tests for validation and malformed cursor scenarios - Add 4 API tests for cursors with invalid field values - Refactor 13 store tests to use buildReportPostQueryParams() helper - All 31 tests pass Documentation: - Update OpenAPI spec with start_time, remove unused end_time - Update markdown docs with start_time examples Security improvements: - Whitelist validation prevents SQL injection in TimeField/SortDirection - Format validation ensures ChannelId and CursorId are valid IDs - Single validation point for both cursor and options paths - Defense in depth: validation + parameterized queries + store layer whitelist 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Improve posts reporting query efficiency and safety Replace SELECT * and nested OR/AND conditions with explicit column selection and PostgreSQL row value comparison for better performance and maintainability. Changes: - Use postSliceColumns() instead of SELECT * for explicit column selection - Replace Squirrel OR/AND with row value comparison: (timeField, Id) > (?, ?) - Use fmt.Sprintf for safer string formatting in WHERE clause Query improvements: Before: WHERE (CreateAt > ?) OR (CreateAt = ? AND Id > ?) After: WHERE (CreateAt, Id) > (?, ?) Benefits: - Explicit column selection prevents issues if table schema changes - Row value comparison is more concise and better optimized by PostgreSQL - Follows existing patterns in post_store.go (postSliceColumns) - Standard SQL:2003 syntax 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Change posts reporting response from map to ordered array Replace the Posts map with an ordered array to preserve query sort order and provide a more natural API response for sequential processing. Changes: - ReportPostListResponse.Posts: map[string]*Post → []*Post - Store layer returns posts array directly (already sorted by query) - App layer iterates by index for metadata enrichment - Remove applyPostsWillBeConsumedHook call (not applicable to reporting) - Update API tests to iterate arrays instead of map lookups - Update store tests to convert array to map for deduplication checks - Remove unused "maps" import Benefits: - Preserves query sort order (ASC/DESC, create_at/update_at) - More natural for sequential processing/export workflows - Simpler response structure for reporting/compliance use cases - Aligns with message export/compliance patterns (no plugin hooks) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix linting issues in posts reporting tests Replace inefficient loops with append(...) for better performance. Changes: - Use append(postSlice, result.Posts...) instead of loop - Simplifies code and follows staticcheck recommendations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix store test AppError nil checking Use require.Nil instead of require.NoError for *AppError returns to avoid Go interface nil pointer issues. When DecodeReportPostCursorV1 returns nil *AppError and it's assigned to error interface, the interface becomes non-nil even though the pointer is nil. This causes require.NoError to fail incorrectly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-17 11:02:19 -05:00
// Reporting API constants
MaxReportingPerPage = 1000 // Maximum number of posts that can be requested per page in reporting endpoints
ReportingTimeFieldCreateAt = "create_at"
ReportingTimeFieldUpdateAt = "update_at"
ReportingSortDirectionAsc = "asc"
ReportingSortDirectionDesc = "desc"
PostPropsMaxRunes = 800000
PostPropsMaxUserRunes = PostPropsMaxRunes - 40000 // Leave some room for system / pre-save modifications
2021-07-12 14:05:36 -04:00
PropsAddChannelMember = "add_channel_member"
PostPropsAddedUserId = "addedUserId"
PostPropsDeleteBy = "deleteBy"
PostPropsOverrideIconURL = "override_icon_url"
PostPropsOverrideIconEmoji = "override_icon_emoji"
PostPropsOverrideUsername = "override_username"
PostPropsFromWebhook = "from_webhook"
PostPropsFromBot = "from_bot"
PostPropsFromOAuthApp = "from_oauth_app"
PostPropsWebhookDisplayName = "webhook_display_name"
PostPropsAttachments = "attachments"
PostPropsFromPlugin = "from_plugin"
2021-07-12 14:05:36 -04:00
PostPropsMentionHighlightDisabled = "mentionHighlightDisabled"
PostPropsGroupHighlightDisabled = "disable_group_highlight"
PostPropsPreviewedPost = "previewed_post"
PostPropsForceNotification = "force_notification"
PostPropsChannelMentions = "channel_mentions"
PostPropsCurrentTeamId = "current_team_id"
PostPropsUnsafeLinks = "unsafe_links"
PostPropsAIGeneratedByUserID = "ai_generated_by"
PostPropsAIGeneratedByUsername = "ai_generated_by_username"
[MM-61758] Burn on read feature (#34703) * Add read receipt store for burn on read message types * update mocks * fix invalidation target * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * add translations * Added logic to associate files of BoR post with the post * Added test * fixes * disable pinning posts and review comments * MM-66594 - Burn on read UI integration (#34647) * MM-66244 - add BoR visual components to message editor * MM-66246 - BoR visual indicator for sender and receiver * MM-66607 - bor - add timer countdown and autodeletion * add the system console max time to live config * use the max expire at and create global scheduler to register bor messages * use seconds for BoR config values in BE * implement the read by text shown in the tooltip logic * unestack the posts from same receiver and BoR and fix styling * avoid opening reply RHS * remove unused dispatchers * persis the BoR label in the drafts * move expiration value to metadata * adjust unit tests to metadata insted of props * code clean up and some performance improvements; add period grace for deletion too * adjust migration serie number * hide bor messages when config is off * performance improvements on post component and code clean up * keep bor existing post functionality if config is disabled * Add read receipt store for burn on read message types * Add temporary posts table * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * avoid reacting to unrevealed bor messages * adjust migration number * Add read receipt store for burn on read message types * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * adjust post reveal and type with backend changes * use real config values, adjust icon usage and style * adjust the delete from from sender and receiver * improve self deleting logic by placing in badge, use burn endpoint * adjust websocket events handling for the read by sender label information * adjust styling for concealed and error state * update burn-on-read post event handling for improved recipient tracking and multi-device sync * replace burn_on_read with type in database migrations and model * remove burn_on_read metadata from PostMetadata and related structures * Added logic to associate files of BoR post with the post * Added test * adjust migration name and fix linter * Add read receipt store for burn on read message types * update mocks * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * Added logic to associate files of BoR post with the post * Added test * disable pinning posts and review comments * show attachment on bor reveal * remove unused translation * Enhance burn-on-read post handling and refine previous post ID retrieval logic * adjust the returning chunk to work with bor messages * read temp post from master db * read from master * show the copy link button to the sender * revert unnecessary check * restore correct json tag * remove unused error handling and clarify burn-on-read comment * improve type safety and use proper selectors * eliminate code duplication in deletion handler * optimize performance and add documentation * delete bor message for sender once all receivers reveal it * add burn on read to scheduled posts * add feature enable check * use master to avoid all read recipients race condition --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> * squash migrations into single file * add configuration for the scheduler * don't run messagehasbeenposted hook * remove parallel tests on burn on read * add clean up for closing opened modals from previous tests * simplify delete menu item rendering * add cleanup step to close open modals after each test to prevent pollution * streamline delete button visibility logic for Burn on Read posts * improve reliability of closing post menu and modals by using body ESC key --------- Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> Co-authored-by: Pablo Vélez <pablovv2012@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2025-12-11 01:59:50 -05:00
PostPropsExpireAt = "expire_at"
PostPropsReadDurationSeconds = "read_duration"
PostPriorityUrgent = "urgent"
[MM-61758] Burn on read feature (#34703) * Add read receipt store for burn on read message types * update mocks * fix invalidation target * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * add translations * Added logic to associate files of BoR post with the post * Added test * fixes * disable pinning posts and review comments * MM-66594 - Burn on read UI integration (#34647) * MM-66244 - add BoR visual components to message editor * MM-66246 - BoR visual indicator for sender and receiver * MM-66607 - bor - add timer countdown and autodeletion * add the system console max time to live config * use the max expire at and create global scheduler to register bor messages * use seconds for BoR config values in BE * implement the read by text shown in the tooltip logic * unestack the posts from same receiver and BoR and fix styling * avoid opening reply RHS * remove unused dispatchers * persis the BoR label in the drafts * move expiration value to metadata * adjust unit tests to metadata insted of props * code clean up and some performance improvements; add period grace for deletion too * adjust migration serie number * hide bor messages when config is off * performance improvements on post component and code clean up * keep bor existing post functionality if config is disabled * Add read receipt store for burn on read message types * Add temporary posts table * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * avoid reacting to unrevealed bor messages * adjust migration number * Add read receipt store for burn on read message types * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * adjust post reveal and type with backend changes * use real config values, adjust icon usage and style * adjust the delete from from sender and receiver * improve self deleting logic by placing in badge, use burn endpoint * adjust websocket events handling for the read by sender label information * adjust styling for concealed and error state * update burn-on-read post event handling for improved recipient tracking and multi-device sync * replace burn_on_read with type in database migrations and model * remove burn_on_read metadata from PostMetadata and related structures * Added logic to associate files of BoR post with the post * Added test * adjust migration name and fix linter * Add read receipt store for burn on read message types * update mocks * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * Added logic to associate files of BoR post with the post * Added test * disable pinning posts and review comments * show attachment on bor reveal * remove unused translation * Enhance burn-on-read post handling and refine previous post ID retrieval logic * adjust the returning chunk to work with bor messages * read temp post from master db * read from master * show the copy link button to the sender * revert unnecessary check * restore correct json tag * remove unused error handling and clarify burn-on-read comment * improve type safety and use proper selectors * eliminate code duplication in deletion handler * optimize performance and add documentation * delete bor message for sender once all receivers reveal it * add burn on read to scheduled posts * add feature enable check * use master to avoid all read recipients race condition --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> * squash migrations into single file * add configuration for the scheduler * don't run messagehasbeenposted hook * remove parallel tests on burn on read * add clean up for closing opened modals from previous tests * simplify delete menu item rendering * add cleanup step to close open modals after each test to prevent pollution * streamline delete button visibility logic for Burn on Read posts * improve reliability of closing post menu and modals by using body ESC key --------- Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> Co-authored-by: Pablo Vélez <pablovv2012@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2025-12-11 01:59:50 -05:00
DefaultExpirySeconds = 60 * 60 * 24 * 7 // 7 days
DefaultReadDurationSeconds = 10 * 60 // 10 minutes
PostContextKeyIsScheduledPost PostContextKey = "isScheduledPost"
2015-06-15 03:53:32 -04:00
)
type Post struct {
Id string `json:"id"`
CreateAt int64 `json:"create_at"`
UpdateAt int64 `json:"update_at"`
EditAt int64 `json:"edit_at"`
DeleteAt int64 `json:"delete_at"`
IsPinned bool `json:"is_pinned"`
UserId string `json:"user_id"`
ChannelId string `json:"channel_id"`
RootId string `json:"root_id"`
OriginalId string `json:"original_id"`
Message string `json:"message"`
// MessageSource will contain the message as submitted by the user if Message has been modified
// by Mattermost for presentation (e.g if an image proxy is being used). It should be used to
// populate edit boxes if present.
MessageSource string `json:"message_source,omitempty"`
Type string `json:"type"`
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
propsMu sync.RWMutex `db:"-"` // Unexported mutex used to guard Post.Props.
Props StringInterface `json:"props"` // Deprecated: use GetProps()
Hashtags string `json:"hashtags"`
Filenames StringArray `json:"-"` // Deprecated, do not use this field any more
Feature edit attachments (#29769) * Updated patch/update post API to allow file modification (#29447) * WIP * WIP * Atatched new files ton post * WIP: deleting removed files * Deleted removed files and invalidated file metadata cache * removed file ignore logif from update post API * Added TestFindExclusives * Added tests for DeleteForPostByIds * Added app layer tests * Added tests * Added API level tests * test enhancements * Fixed a test * Edit history include file metadata (#29505) * Send file metadata in edit history metadata * Added app tests * Added store tests * Added tests for populateEditHistoryFileMetadata{ * Added cache to avoid repetitigve DB calls for edits with only message changes * Added API tests * i18m fix * removed commented code * Improved test helper * Show attachments in edit history RHS (#29519) * Send file metadata in edit history metadata * Added app tests * Added store tests * Added tests for populateEditHistoryFileMetadata{ * Added cache to avoid repetitigve DB calls for edits with only message changes * Added API tests * i18m fix * WIUP: displa files in edit * removed commented code * Displayed file in edit history * Handled file icon * Fixed closing history component on clicking on file * Simplified selector * Simplified selector * Improved test helper * Disabled action menu on edit history file * Added tests * Improved selector * Updated snapshot * review Fixes * restructured componnets * Updated test * Updated test * Restore post api (#29643) * Restore post version API WIP * Undelete files WIP * Added store tests * Created post restore API * Updated updatepost safeUpdate signature * review fixex and improvements * Fixed an app test * Added API laer tests * Added API tests and OpenAPI specs * Fixed a typo * Allow editing files when editing posts (#29709) * WIP - basic view files when editing post * Cleanup * bg color * Added text editor tests for files * WIP * WIP * removed debug log * Allowed admin to add and remove files on someone else's post * Handled drafts and scheduled posts * linter fixes * Updated snapshot * server test fix * CI * Added doc * Restore post api integration (#29719) * WIP - basic view files when editing post * Cleanup * bg color * Added text editor tests for files * WIP * WIP * removed debug log * Allowed admin to add and remove files on someone else's post * Handled drafts and scheduled posts * linter fixes * Updated snapshot * server test fix * Used new API to restore post * handled edut limit and undo * lint fix * added comments * Fixed edit post item tests * Fixed buttons * Aded snapshots * fix test * Updated snapshot * Minor fixes * fixed snapshot * Edit file dnd area (#29763) * dnd wip * DND continued * Supported multiple unbind dragster funcs * lint fixes * Got center channel file drop working when editing a post * file dnd working with center channel and rhs * file dnd working with center channel and rhs * removed unneeded stopPropogation calls * cleanup * DND overlay fix * Lint fix * Advanced text editor test updates for file upload overlay * fixed use upload hook tests * Updated some more snapshots * minor cleanup * Updated i18n * removed need of array for dragster unbind events * lint fixes * edit history cursor * Fixed bugu causing faliure to delete empty posts (#29778) * Files in restore confirmation (#29781) * Added files to restore post confirmation dialog * Fixed post restore toast colors * Fixed restore bug * Fixed restore confirmation toast tests * a11y improvement and modal width fix * Edit attachment misc fixes (#29808) * Removed single image actions in restore post confirmation dialog * Fixed file drop overlay size and position * Made edit indiator accessible * Lint fix * Added bunch of more tests * ANother test migrated from enzyme to react testing library * More test enhancements * More test enhancements * More test enhancements * lint fixes * Fixed a test * Added missing snapshots * Test fixes
2025-01-13 07:46:56 -05:00
FileIds StringArray `json:"file_ids"`
PendingPostId string `json:"pending_post_id"`
HasReactions bool `json:"has_reactions,omitempty"`
RemoteId *string `json:"remote_id,omitempty"`
// Transient data populated before sending a post to the client
ReplyCount int64 `json:"reply_count"`
LastReplyAt int64 `json:"last_reply_at"`
Participants []*User `json:"participants"`
IsFollowing *bool `json:"is_following,omitempty"` // for root posts in collapsed thread mode indicates if the current user is following this thread
Metadata *PostMetadata `json:"metadata,omitempty"`
2015-06-15 03:53:32 -04:00
}
func (o *Post) Auditable() map[string]any {
var metaData map[string]any
if o.Metadata != nil {
metaData = o.Metadata.Auditable()
}
return map[string]any{
Audit logging -- convert audit logs to use the new schema (#20526) * Audit logging - new schema added, old schema removed. * fix linter error by running goimports * Address review comments * Address review comments * Example usage of new audit logging API for the updateUserAuth call * fixed unit test on auditing updating user record * Changed the `TestUpdateConfigDiffInAuditRecord` testcase---it failed, because this PR changes how the `meta` field is serialized into the audit log records. * fix linter error * use string constants for record keys * new audit api calls for api4/bot * `Auditable` interface implementations for model classes * New audit calls for channel api * New audit calls for channel_local * renamed receivers for required style reasons * New audit calls for api4/command * renamed receiver * New audit calls for api4/command_local * renamed receiver * fix unit test to reflect changes in the Auditable implementation of the user class * new audit calls for compliance * new audit calls for configs * remove auditRec.addMeta from updateConfig and patchConfig * new audit calls for config_local * new audit calls * new audit calls for ldap, license apis * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * fix linter error * fixed linter error * fixed "user update" test * Don't include all of config when audit logging config changes. Also fix unit test on TestUpdateConfigDiffInAuditRecord * address review comments * Added Auditable() method for UserPatch * Fix duplicative method declaration from merge * Fix styling and API changes issues introduced with merge * Fix broken test Co-authored-by: Daniel Schalla <daniel@schalla.me>
2022-07-14 07:52:46 -04:00
"id": o.Id,
"create_at": o.CreateAt,
"update_at": o.UpdateAt,
"edit_at": o.EditAt,
"delete_at": o.DeleteAt,
"is_pinned": o.IsPinned,
"user_id": o.UserId,
"channel_id": o.ChannelId,
"root_id": o.RootId,
"original_id": o.OriginalId,
"type": o.Type,
"props": o.GetProps(),
"file_ids": o.FileIds,
"pending_post_id": o.PendingPostId,
"remote_id": o.RemoteId,
"reply_count": o.ReplyCount,
"last_reply_at": o.LastReplyAt,
"is_following": o.IsFollowing,
"metadata": metaData,
Audit logging -- convert audit logs to use the new schema (#20526) * Audit logging - new schema added, old schema removed. * fix linter error by running goimports * Address review comments * Address review comments * Example usage of new audit logging API for the updateUserAuth call * fixed unit test on auditing updating user record * Changed the `TestUpdateConfigDiffInAuditRecord` testcase---it failed, because this PR changes how the `meta` field is serialized into the audit log records. * fix linter error * use string constants for record keys * new audit api calls for api4/bot * `Auditable` interface implementations for model classes * New audit calls for channel api * New audit calls for channel_local * renamed receivers for required style reasons * New audit calls for api4/command * renamed receiver * New audit calls for api4/command_local * renamed receiver * fix unit test to reflect changes in the Auditable implementation of the user class * new audit calls for compliance * new audit calls for configs * remove auditRec.addMeta from updateConfig and patchConfig * new audit calls for config_local * new audit calls * new audit calls for ldap, license apis * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * new audit calls * fix linter error * fixed linter error * fixed "user update" test * Don't include all of config when audit logging config changes. Also fix unit test on TestUpdateConfigDiffInAuditRecord * address review comments * Added Auditable() method for UserPatch * Fix duplicative method declaration from merge * Fix styling and API changes issues introduced with merge * Fix broken test Co-authored-by: Daniel Schalla <daniel@schalla.me>
2022-07-14 07:52:46 -04:00
}
}
func (o *Post) LogClone() any {
return o.Auditable()
}
type PostEphemeral struct {
UserID string `json:"user_id"`
Post *Post `json:"post"`
}
type PostPatch struct {
IsPinned *bool `json:"is_pinned"`
Message *string `json:"message"`
Props *StringInterface `json:"props"`
FileIds *StringArray `json:"file_ids"`
HasReactions *bool `json:"has_reactions"`
}
func (o *PostPatch) IsEmpty() bool {
return o.IsPinned == nil && o.Message == nil && o.Props == nil && o.FileIds == nil && o.HasReactions == nil
}
type PostReminder struct {
TargetTime int64 `json:"target_time"`
// These fields are only used internally for interacting with DB.
PostId string `json:",omitempty"`
UserId string `json:",omitempty"`
}
type PostPriority struct {
Priority *string `json:"priority"`
RequestedAck *bool `json:"requested_ack"`
PersistentNotifications *bool `json:"persistent_notifications"`
// These fields are only used internally for interacting with DB.
PostId string `json:",omitempty"`
ChannelId string `json:",omitempty"`
}
[MM-47751][MM-48102] MPA: Send Persistent Notifications (#21619) * MM-46410: adds urgency on mention counts We have introduced priority for posts in https://github.com/mattermost/mattermost-webapp/pull/10951. We do need to color the mention badges in the webapp with a prominent color when a mention is posted in an urgent message. A thread has urgent mentions if the root post is marked as urgent, and the replies contain mentions to the user viewing the thread. This PR adds two columns, urgentmentioncount, and isurgent, in channelmembers, and threads tables respectively. Furthermore when asking for team/thread mention counts, we also return urgent mention counts for the user. * Adds PostAcknowledgements table and apis * job init and fetch mentions * add-migrations * delete-expired * send-notifications * Fetches post priority in batches * stop-notifications * stop-notification-on-reply * MM-47750: Adds PostAcknowledgements table and apis - Adds post acknowledgement api/app/store methods to be able to save and delete post acknowledgements by users. - Adds wesbsocket events for acknowledgement created/deleted - Returns post acknowledgements in the post's metadata * add-license-check * add-pagination * delete on channel and team * validate guests * add configs * move create priority post check from app to api * Add desktop notifications * check status * use config in job * add IsUrgent check * Add last-sent-at * validate max recipients * Update lastSentAt * Validate min. recipient * send email notification only once * remove email notifications * use latest time from config to run job * Add notifications counter * publish events to mentioned users only * pickup license updates in scheduler * don't allow post owner to stop notifications * follow normal notifications behaviour * Validates persistent notifications interval * move logic of handling valid and expired posts into sql * Adds persistent notifications in the webapp --------- Co-authored-by: koox00 <3829551+koox00@users.noreply.github.com> Co-authored-by: Mattermod <mattermod@users.noreply.github.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2023-05-18 14:14:12 -04:00
type PostPersistentNotifications struct {
PostId string
CreateAt int64
LastSentAt int64
DeleteAt int64
SentCount int16
}
type GetPersistentNotificationsPostsParams struct {
MaxTime int64
MaxSentCount int16
PerPage int
}
Feature: Wrangler (#23602) * Migrate feature/wrangler to mono-repo * Add wrangler files * Fix linters, types, etc * Fix snapshots * Fix playwright * Fix pipelines * Fix more pipeline * Fixes for pipelines * More changes for pipeline * Fix types * Add support for a feature flag, but leave it defaulted on for spinwick usage for now * Update snapshot * fix js error when removing last value of multiselect, support CSV marshaling to string array for textsetting * Fix linter * Remove TODO * Remove another TODO * fix tests * Fix i18n * Add server tests * Fix linter * Fix linter * Use proper icon for dot menu * Update snapshot * Add Cypress UI tests for various entrypoints to move thread modal, split SCSS out from forward post into its own thing * clean up * fix linter * More cleanup * Revert files to master * Fix linter for e2e tests * Make ForwardPostChannelSelect channel types configurable with a prop * Add missing return * Fixes from PR feedback * First batch of PR Feedback * Another batch of PR changes * Fix linter * Update snapshots * Wrangler system messages are translated to each user's locale * Initially translate Wrangler into system locale rather than initiating user * More fixes for PR Feedback * Fix some server tests * More updates with master. Fixes around pipelines. Enforce Enterprise license on front/back end * Add tests for dot_menu * More pipeline fixes * Fix e2etests prettier * Update cypress tests, change occurrences of 'Wrangler' with 'Move Thread' * Fix linter * Remove enterprise lock * A couple more occurrences of wrangler strings, and one more enterprise lock * Fix server tests * Fix i18n * Fix e2e linter * Feature flag shouldn't be on by default * Enable move threads feature in smoke tests (#25657) * enable move threads feature * add @prod tag * Fix move_thread_from_public_channel e2e test * Fix e2e style --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: yasserfaraazkhan <attitude3cena.yf@gmail.com>
2023-12-11 15:27:34 -05:00
type MoveThreadParams struct {
ChannelId string `json:"channel_id"`
}
type SearchParameter struct {
Terms *string `json:"terms"`
IsOrSearch *bool `json:"is_or_search"`
TimeZoneOffset *int `json:"time_zone_offset"`
Page *int `json:"page"`
PerPage *int `json:"per_page"`
IncludeDeletedChannels *bool `json:"include_deleted_channels"`
}
func (sp SearchParameter) Auditable() map[string]any {
return map[string]any{
"terms": sp.Terms,
"is_or_search": sp.IsOrSearch,
"time_zone_offset": sp.TimeZoneOffset,
"page": sp.Page,
"per_page": sp.PerPage,
"include_deleted_channels": sp.IncludeDeletedChannels,
}
}
func (sp SearchParameter) LogClone() any {
return sp.Auditable()
}
type AnalyticsPostCountsOptions struct {
TeamId string
BotsOnly bool
YesterdayOnly bool
}
func (o *PostPatch) WithRewrittenImageURLs(f func(string) string) *PostPatch {
pCopy := *o //nolint:revive
if pCopy.Message != nil {
*pCopy.Message = RewriteImageURLs(*o.Message, f)
}
return &pCopy
}
func (o *PostPatch) Auditable() map[string]any {
return map[string]any{
"is_pinned": o.IsPinned,
"props": o.Props,
"file_ids": o.FileIds,
"has_reactions": o.HasReactions,
}
}
type PostForExport struct {
Post
TeamName string
ChannelName string
Username string
ReplyCount int
FlaggedBy StringArray
}
[MM 12464] Include DM/GM Channels and Their Posts in the Bulk Export (#10421) * transplant the existing PR into the working tree * start addressing review comments * move existing direct channel export code into this branch * modify channel exporter to use squirell and populate members in two steps * use squirrel to build sql queries for channel and dm/gm export methods * remove debug helpers and use Username instead of UserId * unit test for DM Channel exporter * add more unit tests for channel export * add test for DM/GM post export * checkpoint with failing test for postgres * use getQueryBuilder to make sure squirrel uses the correct formatting for each database * add a test for post export * fix shadowed vars that broke the build * address review comments and add tests to support it * address review comments and add a mlog call * s/Info/Debug/ * address review comments in post_store * address review comments in channel_store * address review comments in export * address review comment in post_store: drop GroupBy * address review comment on supplier: move getQueryBuilder to sqlstore * address review comments: explicit TearDown * address review comments: improve test coverage * address review comments: make sure public and private channels are excluded * address review comments: improve test coverage * address review comments: make sure Channels table gets truncated after each test * more cleanups and better assertions * wrap PostStore in a StoreTestWithSqlSupplier * last minute changes: improve post export test coverage and check members * address review comments: make sure all posts have their channel members set * address review comments: make sure all posts have their ChannelMembers exported correctly * gofmt fix * sort channels so it's possible to assert on index
2019-03-15 11:28:43 -04:00
type DirectPostForExport struct {
Post
User string
ChannelMembers *[]string
FlaggedBy StringArray
[MM 12464] Include DM/GM Channels and Their Posts in the Bulk Export (#10421) * transplant the existing PR into the working tree * start addressing review comments * move existing direct channel export code into this branch * modify channel exporter to use squirell and populate members in two steps * use squirrel to build sql queries for channel and dm/gm export methods * remove debug helpers and use Username instead of UserId * unit test for DM Channel exporter * add more unit tests for channel export * add test for DM/GM post export * checkpoint with failing test for postgres * use getQueryBuilder to make sure squirrel uses the correct formatting for each database * add a test for post export * fix shadowed vars that broke the build * address review comments and add tests to support it * address review comments and add a mlog call * s/Info/Debug/ * address review comments in post_store * address review comments in channel_store * address review comments in export * address review comment in post_store: drop GroupBy * address review comment on supplier: move getQueryBuilder to sqlstore * address review comments: explicit TearDown * address review comments: improve test coverage * address review comments: make sure public and private channels are excluded * address review comments: improve test coverage * address review comments: make sure Channels table gets truncated after each test * more cleanups and better assertions * wrap PostStore in a StoreTestWithSqlSupplier * last minute changes: improve post export test coverage and check members * address review comments: make sure all posts have their channel members set * address review comments: make sure all posts have their ChannelMembers exported correctly * gofmt fix * sort channels so it's possible to assert on index
2019-03-15 11:28:43 -04:00
}
type ReplyForExport struct {
Post
Username string
FlaggedBy StringArray
}
type PostForIndexing struct {
Post
TeamId string `json:"team_id"`
ParentCreateAt *int64 `json:"parent_create_at"`
ChannelType string `json:"channel_type"`
}
type FileForIndexing struct {
FileInfo
ChannelId string `json:"channel_id"`
Content string `json:"content"`
}
// ShouldIndex tells if a file should be indexed or not.
// index files which are-
// a. not deleted
// b. have an associated post ID, if no post ID, then,
// b.i. the file should belong to the channel's bookmarks, as indicated by the "CreatorId" field.
//
// Files not passing this criteria will be deleted from ES index.
// We're deleting those files from ES index instead of simply skipping them while fetching a batch of files
// because existing ES indexes might have these files already indexed, so we need to remove them from index.
func (file *FileForIndexing) ShouldIndex() bool {
// NOTE - this function is used in server as well as Enterprise code.
// Make sure to update public package dependency in both server and Enterprise code when
// updating the logic here and to test both places.
return file != nil && file.DeleteAt == 0 && (file.PostId != "" || file.CreatorId == BookmarkFileOwner)
}
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
// ShallowCopy is an utility function to shallow copy a Post to the given
// destination without touching the internal RWMutex.
func (o *Post) ShallowCopy(dst *Post) error {
if dst == nil {
return errors.New("dst cannot be nil")
}
o.propsMu.RLock()
defer o.propsMu.RUnlock()
dst.propsMu.Lock()
defer dst.propsMu.Unlock()
dst.Id = o.Id
dst.CreateAt = o.CreateAt
dst.UpdateAt = o.UpdateAt
dst.EditAt = o.EditAt
dst.DeleteAt = o.DeleteAt
dst.IsPinned = o.IsPinned
dst.UserId = o.UserId
dst.ChannelId = o.ChannelId
dst.RootId = o.RootId
dst.OriginalId = o.OriginalId
dst.Message = o.Message
dst.MessageSource = o.MessageSource
dst.Type = o.Type
dst.Props = o.Props
dst.Hashtags = o.Hashtags
dst.Filenames = o.Filenames
dst.FileIds = o.FileIds
dst.PendingPostId = o.PendingPostId
dst.HasReactions = o.HasReactions
dst.ReplyCount = o.ReplyCount
dst.Participants = o.Participants
dst.LastReplyAt = o.LastReplyAt
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
dst.Metadata = o.Metadata
if o.IsFollowing != nil {
dst.IsFollowing = NewPointer(*o.IsFollowing)
}
dst.RemoteId = o.RemoteId
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
return nil
}
// Clone shallowly copies the post and returns the copy.
func (o *Post) Clone() *Post {
pCopy := &Post{} //nolint:revive
o.ShallowCopy(pCopy)
return pCopy
}
[MM-22051] Remove To/From JSON (#18070) * Posts * Add missing translation * Fix internal store marshaling * [MM-22051] Remove To/From JSON (Channels) (#18116) * Channels * Channel members * ChannelSearch * Channel categories, list, sidebar, stats, view * Fix conversions * [MM-22051] Remove To/From JSON (Users) (#18121) * User related structs * Fix return * Team related structures (#18127) * [MM-22051] Remove To/From JSON (Status, Bot, Reaction, Thread, FileInfo) (#18130) * Status * Bot * Reaction * Thread * FileInfo * Some fixes * Translations update from Weblate (#18143) * Translated using Weblate (German) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/de/ * Translated using Weblate (Turkish) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/tr/ * Translated using Weblate (Hungarian) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/hu/ * Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ * Translated using Weblate (German) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/de/ * Translated using Weblate (Turkish) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/tr/ * Translated using Weblate (Hungarian) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/hu/ * Translated using Weblate (English (Australia)) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/en_AU/ * Translated using Weblate (Bulgarian) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/bg/ * Translated using Weblate (Japanese) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ja/ * Translated using Weblate (Chinese (Simplified)) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/zh_Hans/ Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> Co-authored-by: Weblate (bot) <hosted@weblate.org> Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> * [MM-22051] Remove To/From JSON methods from model (#18138) * Scheme * Role * Session * Config * Status * Fix logic * Emoji * GuestsInvite * Group * Command * ClusterInfo * License * Job * System * Plugin * Command2 * IncomingWebhook * OutgoingWebhook * Fix tests * Update traslation * Some fixes * Add missing return * Simplify * Make Config.ToJSONFiltered() return []byte * Make Busy.ToJSON() return []byte * Include error in log * Split logic * [MM-22051] Remove To/From JSON (final) (#18150) * SwitchRequest * PluginEventData * Permalink * PushNotification * SuggestCommand * PluginsResponse * WebSocketMessage * RemoteCluster * SharedChannel * PluginStatuses * InitialLoad * ClusterDiscovery * ClusterStats * MfaSecret * GroupSyncable * SAML * WebSocketRequest * TypingRequest * SecurityBulletin * OAuthApp * IntegrationAction * DataRetention * Preference * FileInfoList * Compliance * Preferences * FileInfoSearchResults * TermsOfService * InstallMarketplacePluginRequest * GitLabUser * UploadSessions * Remove unused helpers * Fix tests * [MM-23280] Fix linting for ToJSON/FromJSON (#18153) * SwitchRequest * PluginEventData * Permalink * PushNotification * SuggestCommand * PluginsResponse * WebSocketMessage * RemoteCluster * SharedChannel * PluginStatuses * InitialLoad * ClusterDiscovery * ClusterStats * MfaSecret * GroupSyncable * SAML * WebSocketRequest * TypingRequest * SecurityBulletin * OAuthApp * IntegrationAction * DataRetention * Preference * FileInfoList * Compliance * Preferences * FileInfoSearchResults * TermsOfService * InstallMarketplacePluginRequest * GitLabUser * UploadSessions * Remove unused helpers * Fix tests * Fix linting for ToJSON/FromJSON * Fix conversions Co-authored-by: Weblate (bot) <hosted@weblate.org> Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-09-01 08:43:12 -04:00
func (o *Post) ToJSON() (string, error) {
pCopy := o.Clone() //nolint:revive
pCopy.StripActionIntegrations()
b, err := json.Marshal(pCopy)
[MM-22051] Remove To/From JSON (#18070) * Posts * Add missing translation * Fix internal store marshaling * [MM-22051] Remove To/From JSON (Channels) (#18116) * Channels * Channel members * ChannelSearch * Channel categories, list, sidebar, stats, view * Fix conversions * [MM-22051] Remove To/From JSON (Users) (#18121) * User related structs * Fix return * Team related structures (#18127) * [MM-22051] Remove To/From JSON (Status, Bot, Reaction, Thread, FileInfo) (#18130) * Status * Bot * Reaction * Thread * FileInfo * Some fixes * Translations update from Weblate (#18143) * Translated using Weblate (German) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/de/ * Translated using Weblate (Turkish) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/tr/ * Translated using Weblate (Hungarian) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/hu/ * Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ * Translated using Weblate (German) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/de/ * Translated using Weblate (Turkish) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/tr/ * Translated using Weblate (Hungarian) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/hu/ * Translated using Weblate (English (Australia)) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/en_AU/ * Translated using Weblate (Bulgarian) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/bg/ * Translated using Weblate (Japanese) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ja/ * Translated using Weblate (Chinese (Simplified)) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/zh_Hans/ Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> Co-authored-by: Weblate (bot) <hosted@weblate.org> Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> * [MM-22051] Remove To/From JSON methods from model (#18138) * Scheme * Role * Session * Config * Status * Fix logic * Emoji * GuestsInvite * Group * Command * ClusterInfo * License * Job * System * Plugin * Command2 * IncomingWebhook * OutgoingWebhook * Fix tests * Update traslation * Some fixes * Add missing return * Simplify * Make Config.ToJSONFiltered() return []byte * Make Busy.ToJSON() return []byte * Include error in log * Split logic * [MM-22051] Remove To/From JSON (final) (#18150) * SwitchRequest * PluginEventData * Permalink * PushNotification * SuggestCommand * PluginsResponse * WebSocketMessage * RemoteCluster * SharedChannel * PluginStatuses * InitialLoad * ClusterDiscovery * ClusterStats * MfaSecret * GroupSyncable * SAML * WebSocketRequest * TypingRequest * SecurityBulletin * OAuthApp * IntegrationAction * DataRetention * Preference * FileInfoList * Compliance * Preferences * FileInfoSearchResults * TermsOfService * InstallMarketplacePluginRequest * GitLabUser * UploadSessions * Remove unused helpers * Fix tests * [MM-23280] Fix linting for ToJSON/FromJSON (#18153) * SwitchRequest * PluginEventData * Permalink * PushNotification * SuggestCommand * PluginsResponse * WebSocketMessage * RemoteCluster * SharedChannel * PluginStatuses * InitialLoad * ClusterDiscovery * ClusterStats * MfaSecret * GroupSyncable * SAML * WebSocketRequest * TypingRequest * SecurityBulletin * OAuthApp * IntegrationAction * DataRetention * Preference * FileInfoList * Compliance * Preferences * FileInfoSearchResults * TermsOfService * InstallMarketplacePluginRequest * GitLabUser * UploadSessions * Remove unused helpers * Fix tests * Fix linting for ToJSON/FromJSON * Fix conversions Co-authored-by: Weblate (bot) <hosted@weblate.org> Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-09-01 08:43:12 -04:00
return string(b), err
2015-06-15 03:53:32 -04:00
}
func (o *Post) EncodeJSON(w io.Writer) error {
o.StripActionIntegrations()
return json.NewEncoder(w).Encode(o)
}
type CreatePostFlags struct {
TriggerWebhooks bool
SetOnline bool
ForceNotification bool
}
type GetPostsSinceOptions struct {
UserId string
ChannelId string
Time int64
SkipFetchThreads bool
CollapsedThreads bool
CollapsedThreadsExtended bool
SortAscending bool
}
type GetPostsSinceForSyncCursor struct {
LastPostUpdateAt int64
LastPostUpdateID string
LastPostCreateAt int64
LastPostCreateID string
}
func (c GetPostsSinceForSyncCursor) IsEmpty() bool {
return c.LastPostCreateAt == 0 && c.LastPostCreateID == "" && c.LastPostUpdateAt == 0 && c.LastPostUpdateID == ""
}
type GetPostsSinceForSyncOptions struct {
ChannelId string
ExcludeRemoteId string
IncludeDeleted bool
Merge the Integrated Boards MVP feature branch (#35796) * Add CreatedBy and UpdatedBy to the properties fields and values (#34485) * Add CreatedBy and UpdatedBy to the properties fields and values * Fix types --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds ObjectType to the property fields table (#34908) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Update ObjectType migration setting an empty value and marking the column as not null (#34915) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds uniqueness mechanisms to the property fields (#35058) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Fixing retrylayer mocks * Remove retrylayer duplication * Address review comments * Fix comment to avoid linter issues * Address PR comments * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.down.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update field validation to check only for valid target types * Update migrations to avoid concurrent index creation within a transaction * Update migrations to make all index ops concurrent * Update tests to use valid PSAv2 property fields * Adds a helper for valid PSAv2 TargetTypes --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Fix property tests (#35388) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards feature flag (#35378) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards MVP API changes (#34822) This PR includes the necessary changes for channels and posts endpoints and adds a set of generic endpoints to retrieve and manage property fields and values following the new Property System approach. Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> * Property System Architecture permissions for v2 (#35113) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Fix i18n sorting --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Add Views store and app layer (#35361) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor test loops in ViewStore tests for improved readability * change pagination to limit/offset * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * MM-67388, MM-66528, MM-67750: Add View REST API endpoints, websocket events, and sort order (#35442) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Add View API endpoints with OpenAPI spec, client methods, and i18n Implement REST API for channel views (board-type) behind the IntegratedBoards feature flag. Adds CRUD endpoints under /api/v4/channels/{channel_id}/views with permission checks matching the channel bookmark pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add cursor-based pagination to View API for channel views * Enhance cursor handling in getViewsForChannel and update tests for pagination * Refactor test loops in ViewStore tests for improved readability * Refactor loop in TestGetViewsForChannel for improved readability * change pagination to limit/offset * switch to limit/offset pagination * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add view sort order API endpoint Add POST /api/v4/channels/{channel_id}/views/{view_id}/sort_order endpoint following the channel bookmarks reorder pattern. Includes store, app, and API layers with full test coverage at each layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add connectionId to view WebSocket events and sort_order API spec Thread connectionId from request header through all view handlers (create, update, delete, sort_order) to WebSocket events, matching the channel bookmarks pattern. Add sort_order endpoint to OpenAPI spec. Update minimum server version to 11.6. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove duplicate View/ViewPatch definitions from definitions.yaml The merge from integrated-boards-mvp introduced duplicate View and ViewPatch schema definitions that were already defined earlier in the file with more detail (including ViewBoardProps ref and enums). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update minimum server version to 11.6 in views API spec Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add missing translations for view sort order error messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Merge integrated-boards-mvp into ibmvp_api-views; remove spec files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix flaky TestViewStore timestamp test on CI Add sleep before UpdateSortOrder to ensure timestamps differ, preventing same-millisecond comparisons on fast CI machines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove duplicate views.yaml imclude * Use c.boolString() for include_deleted query param in GetViewsForChannel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix views.yaml sort order schema: use integer type and require body Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor view sort order tests to use named IDs instead of array indices Extract idA/idB/idC from views slice and add BEFORE/AFTER comments to make stateful subtest ordering easier to follow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Return 404 instead of 403 for view operations on deleted channels Deleted channels should appear non-existent to callers rather than revealing their existence via a 403. Detailed error text explains the context for debugging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * add missing channel deleteat checks * Use c.Params.Page instead of manual page query param parsing in getViewsForChannel c.Params already validates and defaults page/per_page, so the manual parsing was redundant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add support for total count in views retrieval * Add tests for handling deleted views in GetViewsForChannel and GetView * Short-circuit negative newIndex in UpdateSortOrder before opening transaction Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add per-channel limit on views to bound UpdateSortOrder cost Without a cap, unbounded view creation makes sort-order updates increasingly expensive (CASE WHEN per view, row locks). Adds MaxViewsPerChannel=50 constant and enforces it in the app layer before saving. Includes API and app layer tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove include_deleted support from views API Soft-deleted views are structural metadata with low risk, but no other similar endpoint (e.g. channel bookmarks) exposes deleted records without an admin gate. Rather than adding an admin-only permission check for consistency, remove the feature entirely since there is no current use case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update view permissions to require `create_post` instead of channel management permissions * Remove obsolete view management error messages for direct and group messages --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(migrations): add user tracking and object type to property fields - Introduced user tracking columns (CreatedBy, UpdatedBy) to PropertyFields and PropertyValues. - Added ObjectType column to PropertyFields with associated unique indexes for legacy and typed properties. - Created new migration scripts for adding and dropping these features, including necessary indexes for data integrity. - Established views for managing property fields with new attributes. This update enhances the schema to support better tracking and categorization of property fields. * Add Property System Architecture v2 API endpoints (#35583) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Require target_type filter when searching property fields * Add objectType validation as part of field.IsValid() * Fix linter * Fix test with bad objecttpye * Fix test grouping --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * MM-67968: Flatten view model — remove icon, subviews, typed board props (#35726) * feat(views): flatten view model by removing icon, subview, and board props Simplifies the View data model as part of MM-67968: removes Icon, Subview, and ViewBoardProps types; renames ViewTypeBoard to ViewTypeKanban; replaces typed Props with StringInterface (map[string]any); adds migration 000167 to drop the Icon column from the Views table. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(api): update views OpenAPI spec to reflect flattened model Removes ViewBoardProps, Subview, and icon from the View and ViewPatch schemas. Changes type enum from board to kanban. Replaces typed props with a free-form StringInterface object. Aligns with MM-67968. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * refactor(views): simplify store by dropping dbView and marshalViewProps StringInterface already implements driver.Valuer and sql.Scanner, so the manual JSON marshal/unmarshal and the dbView intermediate struct were redundant. model.View now scans directly from the database. Also removes the dead ViewMaxLinkedProperties constant and wraps the Commit() error in UpdateSortOrder. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(api): allow arbitrary JSON in view props OpenAPI schema The props field was restricted to string values via additionalProperties: { type: string }, conflicting with the Go model's StringInterface (map[string]any). Changed to additionalProperties: true in View, ViewPatch, and inline POST schemas. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Adds basic implementation of the generic redux store for PSAv2 (#35512) * Adds basic implementation of the generic redux store for PSAv2 * Add created_by and updated_by to the test fixtures * Make target_id, target_type and object_type mandatory * Wrap getPropertyFieldsByIds and getPropertyValuesForTargetByFieldIds with createSelector * Address PR comments --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds websocket messages for the PSAv2 API events (#35696) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Adds websocket messages for the PSAv2 API events * Add IsPSAv2 helper to the property field for clarity * Add guard against nil returns on field deletion * Add docs to the websocket endpoints --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * migrations: consolidate views migrations and reorder after master - Merged 000165 (create Views) with 000167 (drop Icon) since Icon was never needed - Renumbered branch migrations 159-166 → 160-167 so master's 000159 (deduplicate_policy_names) runs first - Regenerated migrations.list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add API endpoint to retrieve posts for a specific view (#35604) Automatic Merge * Apply fixes after merge * Return a more specific error from getting multiple fields * Prevent getting broadcast params on field deletion if not needed * Remove duplicated migration code * Update property conflict code to always use master * Adds nil guard when iterating on property fields * Check that permission level is valid before getting rejected by the database * Validate correctness on TargetID for PSAv2 fields * Avoid PSAv1 using permissions or protected * Fix test data after validation change * Fix flaky search test * Adds more posts for filter use cases to properly test exclusions --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> Co-authored-by: Julien Tant <julien@craftyx.fr> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 05:36:35 -04:00
SinceCreateAt bool // determines whether the cursor will be based on CreateAt or UpdateAt
ExcludeChannelMetadataSystemPosts bool // if true, exclude channel metadata system posts (header, display name, purpose changes)
ExcludedPostTypes []string // post types to exclude from sync
}
type GetPostsOptions struct {
UserId string
ChannelId string
PostId string
Page int
PerPage int
SkipFetchThreads bool
CollapsedThreads bool
CollapsedThreadsExtended bool
FromPost string // PostId after which to send the items
FromCreateAt int64 // CreateAt after which to send the items
MM-56548: [AI assisted]Add support for incremental thread loading using UpdateAt timestamp (#30486) Every time we load the RHS, we used to load the FULL thread always. Although the actual ThreadViewer React component is virtualized, and the server side API call is paginated, we still went through all the pages, to get the full thread and passed it on to the ThreadViewer. This would be for first loads, and subsequent loads of the same thread. This was a bug originally, but then it was a necessity after we applied websocket event scope because now we won't get emoji reactions of a thread if the user is not on the thread. To fix that, we enhance the thread loading functionality by adding support for fetching thread updates based on the UpdateAt timestamp. Now, for subsequent loads, we only get the changed posts in a thread. The implementation: - Adds new API parameters: fromUpdateAt and updatesOnly to the GetPostThread endpoint - Updates database queries to support sorting and filtering by UpdateAt - Implements thread state management to track the last update timestamp - Adds client-side support to use incremental loading for improved performance - Ensures proper validation for parameter combinations and error handling This change enables more efficient thread loading, particularly for long threads with frequent updates, by only fetching posts that have been updated since the last view. Caveats: For delta updates, the SQL query won't use the best index possible because we have an index for (CreateAt, Id), but no index for (UpdateAt, Id). However, from my tests, it is not as bad as it looks: ``` [loadtest] # EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM Posts WHERE Posts.DeleteAt = 0 AND Posts.RootId = 'qbr5gctu9iyg8c36hpcq6f3w8e' AND Posts.UpdateAt > 1623445795824 ORDER BY UpdateAt ASC, Id ASC LIMIT 61; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------------- Limit (cost=8.31..8.31 rows=1 width=216) (actual time=0.047..0.049 rows=0 loops=1) Buffers: shared hit=2 -> Sort (cost=8.31..8.31 rows=1 width=216) (actual time=0.044..0.045 rows=0 loops=1) Sort Key: updateat, id Sort Method: quicksort Memory: 25kB Buffers: shared hit=2 -> Index Scan using idx_posts_root_id_delete_at on posts (cost=0.28..8.30 rows=1 width=216) (actual time=0.031..0.032 rows=0 loops=1) Index Cond: (((rootid)::text = 'qbr5gctu9iyg8c36hpcq6f3w8e'::text) AND (deleteat = 0)) Filter: (updateat > '1623445795824'::bigint) Buffers: shared hit=2 Planning: Buffers: shared hit=3 Planning Time: 0.508 ms Execution Time: 0.106 ms (14 rows) ``` We still get an index scan with index cond. Although there's a filter element, but atleast we get the whole thread with the index. My thinking is that while the whole thread might be large, but after that, updates on a thread should be incremental. Therefore, we should be okay without adding yet another index on the posts table. This is just the first step in what could be potentially improved further. 1. We shouldn't even be loading the full thread always. But rather let the virtualized viewer load more posts on demand. 2. If a post has been just reacted to, then we need not send the whole post down, but just the reaction. This further saves bandwidth. https://mattermost.atlassian.net/browse/MM-56548 TBD: Add load-test coverage to update the thread loading code ```release-note NONE ``` --------- Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-22 01:13:13 -04:00
FromUpdateAt int64 // UpdateAt after which to send the items. This cannot be used with FromCreateAt.
Direction string // Only accepts up|down. Indicates the order in which to send the items.
MM-56548: [AI assisted]Add support for incremental thread loading using UpdateAt timestamp (#30486) Every time we load the RHS, we used to load the FULL thread always. Although the actual ThreadViewer React component is virtualized, and the server side API call is paginated, we still went through all the pages, to get the full thread and passed it on to the ThreadViewer. This would be for first loads, and subsequent loads of the same thread. This was a bug originally, but then it was a necessity after we applied websocket event scope because now we won't get emoji reactions of a thread if the user is not on the thread. To fix that, we enhance the thread loading functionality by adding support for fetching thread updates based on the UpdateAt timestamp. Now, for subsequent loads, we only get the changed posts in a thread. The implementation: - Adds new API parameters: fromUpdateAt and updatesOnly to the GetPostThread endpoint - Updates database queries to support sorting and filtering by UpdateAt - Implements thread state management to track the last update timestamp - Adds client-side support to use incremental loading for improved performance - Ensures proper validation for parameter combinations and error handling This change enables more efficient thread loading, particularly for long threads with frequent updates, by only fetching posts that have been updated since the last view. Caveats: For delta updates, the SQL query won't use the best index possible because we have an index for (CreateAt, Id), but no index for (UpdateAt, Id). However, from my tests, it is not as bad as it looks: ``` [loadtest] # EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM Posts WHERE Posts.DeleteAt = 0 AND Posts.RootId = 'qbr5gctu9iyg8c36hpcq6f3w8e' AND Posts.UpdateAt > 1623445795824 ORDER BY UpdateAt ASC, Id ASC LIMIT 61; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------------- Limit (cost=8.31..8.31 rows=1 width=216) (actual time=0.047..0.049 rows=0 loops=1) Buffers: shared hit=2 -> Sort (cost=8.31..8.31 rows=1 width=216) (actual time=0.044..0.045 rows=0 loops=1) Sort Key: updateat, id Sort Method: quicksort Memory: 25kB Buffers: shared hit=2 -> Index Scan using idx_posts_root_id_delete_at on posts (cost=0.28..8.30 rows=1 width=216) (actual time=0.031..0.032 rows=0 loops=1) Index Cond: (((rootid)::text = 'qbr5gctu9iyg8c36hpcq6f3w8e'::text) AND (deleteat = 0)) Filter: (updateat > '1623445795824'::bigint) Buffers: shared hit=2 Planning: Buffers: shared hit=3 Planning Time: 0.508 ms Execution Time: 0.106 ms (14 rows) ``` We still get an index scan with index cond. Although there's a filter element, but atleast we get the whole thread with the index. My thinking is that while the whole thread might be large, but after that, updates on a thread should be incremental. Therefore, we should be okay without adding yet another index on the posts table. This is just the first step in what could be potentially improved further. 1. We shouldn't even be loading the full thread always. But rather let the virtualized viewer load more posts on demand. 2. If a post has been just reacted to, then we need not send the whole post down, but just the reaction. This further saves bandwidth. https://mattermost.atlassian.net/browse/MM-56548 TBD: Add load-test coverage to update the thread loading code ```release-note NONE ``` --------- Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-22 01:13:13 -04:00
UpdatesOnly bool // This flag is used to make the API work with the updateAt value.
IncludeDeleted bool
IncludePostPriority bool
}
type PostCountOptions struct {
// Only include posts on a specific team. "" for any team.
TeamId string
MustHaveFile bool
MustHaveHashtag bool
ExcludeDeleted bool
ExcludeSystemPosts bool
UsersPostsOnly bool
// AllowFromCache looks up cache only when ExcludeDeleted and UsersPostsOnly are true and rest are falsy.
AllowFromCache bool
MM-59966 - Compliance Export overhaul - feature branch (#29789) * [MM-59089] Add a compliance export constant (#27919) * add a useful constant * i18n * another constant * another i18n * [MM-60422] Add GetChannelsWithActivityDuring (#28301) * modify GetUsersInChannelDuring to accept a slice of channelIds * add GetChannelsWithActivityDuring * add compliance export progress message; remove unused custom status * linting * tests running too fast * add batch size config settings * add store tests * linting * empty commit * i18n changes * fix i18n ordering * MM-60570 - Server-side changes consolidating the export CLI with server/ent code (#28640) * add an i18n field; add the CLI's export directory * int64 -> int * Add UntilUpdateAt for MessageExport and AnalyticsPostCount to merge * remove now-unused i18n strings * add TranslationsPreInitFromBuffer to allow CLI to use i18n * use GetBuilder to simplify; rename TranslationsPreInitFromFileBytes * [MM-59089] Improve compliance export timings (#1733 - Enterprise repo) * MM-60422 - Performance and logic fixes for Compliance Exports (#1757 - Enterprise repo) * MM-60570 - Enterprise-side changes consolidating the export CLI with server/ent code (#1769 - Enterprise repo) * merge conflicts; missed file from ent branch * MM-61038 - Add an option to sqlstore.New (#28702) remove useless comment add test add an option to sqlstore.New * MM-60976: Remove RunExport command from Mattermost binary (#28805) * remove RunExport command from mattermost binary * remove the code it was calling * fix i18n * remove test (was only testing license, not functionality) * empty commit * fix flaky GetChannelsWithActivityDuring test * MM-60063: Dedicated Export Filestore fix, redo of #1772 (enterprise) (#28803) * redo filestore fix #1772 (enterprise repo) on top of MM-59966 feature * add new e2e tests for export filestore * golint * ok, note to self: shadowing bad, actually (when there's a defer) * empty commit * MM-61137 - Message export: Support 7.8.11 era dbs (#28824) * support 7.8.11 era dbs by wrapping the store using only what we need * fix flaky GetChannelsWithActivityDuring test * add a comment * only need to define the MEFileInfoStore (the one that'll be overridden) * blank commit * MM-60974 - Message Export: Add performance metrics (#28836) * support 7.8.11 era dbs by wrapping the store using only what we need * fix flaky GetChannelsWithActivityDuring test * add a comment * only need to define the MEFileInfoStore (the one that'll be overridden) * performance metrics * cleanup unneeded named returns * blank commit * MM-60975 - Message export: Add startTime and endTime to export folder name (#28840) * support 7.8.11 era dbs by wrapping the store using only what we need * fix flaky GetChannelsWithActivityDuring test * add a comment * only need to define the MEFileInfoStore (the one that'll be overridden) * performance metrics * output startTime and endTime in export folder * empty commit * merge conflict * MM-60978 - Message export: Improve xml fields; fix delete semantics (#28873) * support 7.8.11 era dbs by wrapping the store using only what we need * fix flaky GetChannelsWithActivityDuring test * add a comment * only need to define the MEFileInfoStore (the one that'll be overridden) * performance metrics * output startTime and endTime in export folder * empty commit * add xml fields, omit when empty, tests * fix delete semantics; test (and test for update semantics) * clarify comments * simplify edited post detection, now there's no edge case. * add some spacing to help fast running tests * merge conflicts/updates needed for new deleted post semantics * linting; fixing tests from upstream merge * use SafeDereference * linting * stronger typing; better wrapped errors; better formatting * blank commit * goimports formatting * fix merge mistake * minor fixes due to changes in master * MM-61755 - Simplifying and Support reporting to the db from the CLI (#29281) * finally clean up JobData struct and stringMap; prep for CLI using db * and now simplify using StringMapToJobDataWithZeroValues * remove unused fn * create JobDataExported; clean up errors * MM-60176 - Message Export: Global relay cleanup (#29168) * move global relay logic into global_relay_export * blank commit * blank commit * improve errors * MM-60693 - Refactor CSV to use same codepath as Actiance (#29191) * move global relay logic into global_relay_export * blank commit * refactor (and simplify) ExportParams into shared * blank commit * remove unused fn * csv now uses pre-calculated joins/leaves like actiance * improve errors * remove nil post check; remove ignoredPosts metric * remove unneeded copy * MM-61696 - Refactor GlobalRelay to use same codepath as Actiance (#29225) * move global relay logic into global_relay_export * blank commit * refactor (and simplify) ExportParams into shared * blank commit * remove unused fn * csv now uses pre-calculated joins/leaves like actiance * remove newly unneeded function and its test. goodbye. * refactor GetPostAttachments for csv + global relay to share * refactor global_relay_export and fix tests (no changes to output) * improve errors * remove nil post check; remove ignoredPosts metric * remove unneeded copy * remove unneeded nil check * PR comments * MM-61715 - Generalize e2e to all export types 🤖 (#29369) * move global relay logic into global_relay_export * blank commit * refactor (and simplify) ExportParams into shared * blank commit * remove unused fn * csv now uses pre-calculated joins/leaves like actiance * remove newly unneeded function and its test. goodbye. * refactor GetPostAttachments for csv + global relay to share * refactor global_relay_export and fix tests (no changes to output) * improve errors * remove nil post check; remove ignoredPosts metric * remove unneeded copy * remove unneeded nil check * PR comments * refactor isDeletedMsg for all export types * fix start and endtime, nasty csv createAt bug; bring closer to Actiance * align unit tests with new logic (e.g. starttime / endtime) * refactor a TimestampConvert fn for code + tests * bug: pass templates to global relay (hurray for e2e tests, otherwise...) * add global relay zip to allowed list (only for tests) * test helpers * new templates for e2e tests * e2e tests... phew. * linting * merge conflicts * unexport PostToRow; add test helper marker * cleanup, shortening, thanks to PR comments * MM-61972 - Generalize export data path - Actiance (#29399) * extract and generalize the export data generation functions * finish moving test (bc of previous extraction) * lift a function from common -> shared (to break an import cycle) * actiance now takes general export data, processes it into actiance data * bring tests in line with correct sorting rules (upadateAt, messageId) * fixups, PR comments * turn strings.Repeat into a more descriptive const amended: one letter fix; bad rebase * MM-62009 - e2e clock heisenbug (#29434) * consolidate assertions; output debuggable diffs (keeping for future) * refactor test output generator to generators file * waitUntilZeroPosts + pass through until to job = fix all clock issues * simplify messages to model.NewId(); remove unneeded waitUntilZeroPosts * model.NewId() -> storetest.NewTestID() * MM-61980 - Generalize export data path - CSV (#29482) * simple refactoring * increase sleep times for (very) rare test failures * add extra information to the generic export for CSV * adj Actiance to handle new generic export (no difference in its output) * no longer need mergePosts (yay), move getJoinLeavePosts for everyone * adjust tests for new csv semantics (detailed in summary) * and need to add the new exported data to the export_data_tests * rearrange csv writing to happen after data export (more logical) * linting * remove debug statements * figured out what was wrong with global relay e2e test 3; solid now * PR comments * MM-61718 - Generalize export data path - Global Relay (#29508) * move global relay over to using the generalized export data * performance pass -- not much can be done * Update server/enterprise/message_export/global_relay_export/global_relay_export.go Co-authored-by: Claudio Costa <cstcld91@gmail.com> --------- Co-authored-by: Claudio Costa <cstcld91@gmail.com> * MM-62058 - Align CSV with Actiance (#29551) * refactoring actiance files and var names for clarity * bug found in exported attachments (we used to miss some start/ends) * changes needed for actiance due to new generic exports * bringing CSV up to actiance standards * fixing global relay b/c of new semantics (adding a note on an edge case) * aligning e2e tests, adding comments to clarify what is expected/tested * necessary changes; 1 more test for added functionality (ignoreDeleted) * comment style * MM-62059 - Align Global Relay with Actiance/CSV; many fixes (#29665) * core logic changes to general export_data and the specific export paths * unit tests and e2e tests, covering all new edge cases and all logic * linting * better var naming, const value, and cleaning up functions calls * MM-62436 - Temporarily skip cypress tests that require download link (#29772) --------- Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2025-01-10 16:56:02 -05:00
// retrieves posts in the inclusive range: [SinceUpdateAt + LastPostId, UntilUpdateAt]
SincePostID string
SinceUpdateAt int64
UntilUpdateAt int64
}
2015-06-15 03:53:32 -04:00
func (o *Post) Etag() string {
return Etag(o.Id, o.UpdateAt)
}
Relax 4k post message limit (#8478) * MM-9661: rename POST_MESSAGE_MAX_RUNES to \0_v1 * MM-9661: s/4000/POST_MESSAGE_MAX_RUNES_V1/ in tests * MM-9661: introduce POST_MESSAGE_MAX_RUNES_V2 * MM-9661: migrate Postgres Posts.Message column to TEXT from VARCHAR(4000) This is safe to do in a production instance since the underyling type is not changing. We explicitly don't do this automatically for MySQL, but also don't need to since the ORM would have already created a TEXT column for MySQL in that case. * MM-9661: emit MaxPostSize in client config This value remains unconfigurable at this time, but exposes the current limit to the client. The limit remains at 4k in this commit. * MM-9661: introduce and use SqlPostStore.GetMaxPostSize Enforce a byte limitation in the database, and use 1/4 of that value as the rune count limitation (assuming a worst case UTF-8 representation). * move maxPostSizeCached, lastPostsCache and lastPostTimeCache out of the global context and onto the SqlPostStore * address feedback from code review: * ensure sqlstore unit tests are actually being run * move global caches into SqlPostStore * leverage sync.Once to address a race condition * modify upgrade semantics to match new db semantics gorp's behaviour on creating columns with a maximum length on Postgres differs from MySQL: * Postgres * gorp uses TEXT for string columns without a maximum length * gorp uses VARCHAR(N) for string columns with a maximum length of N * MySQL * gorp uses TEXT for string columns with a maximum length >= 256 * gorp uses VARCHAR(N) for string columns with a maximum length of N * gorp defaults to a maximum length of 255, implying VARCHAR(255) So the Message column has been TEXT on MySQL but VARCHAR(4000) on Postgres. With the new, longer limits of 65535, and without changes to gorp, the expected behaviour is TEXT on MySQL and VARCHAR(65535) on Postgres. This commit makes the upgrade semantics match the new database semantics. Ideally, we'd revisit the gorp behaviour at a later time. * allow TestMaxPostSize test cases to actually run in parallel * default maxPostSizeCached to POST_MESSAGE_MAX_RUNES_V1 in case the once initializer panics * fix casting error * MM-9661: skip the schema migration for Postgres It turns out resizing VARCHAR requires a rewrite in some versions of Postgres, but migrating VARCHAR to TEXT does not. Given the increasing complexity, let's defer the migration to the enduser instead.
2018-03-26 17:55:35 -04:00
func (o *Post) IsValid(maxPostSize int) *AppError {
if !IsValidId(o.Id) {
return NewAppError("Post.IsValid", "model.post.is_valid.id.app_error", nil, "", http.StatusBadRequest)
2015-06-15 03:53:32 -04:00
}
if o.CreateAt == 0 {
return NewAppError("Post.IsValid", "model.post.is_valid.create_at.app_error", nil, "id="+o.Id, http.StatusBadRequest)
2015-06-15 03:53:32 -04:00
}
if o.UpdateAt == 0 {
return NewAppError("Post.IsValid", "model.post.is_valid.update_at.app_error", nil, "id="+o.Id, http.StatusBadRequest)
2015-06-15 03:53:32 -04:00
}
if !IsValidId(o.UserId) {
return NewAppError("Post.IsValid", "model.post.is_valid.user_id.app_error", nil, "", http.StatusBadRequest)
2015-06-15 03:53:32 -04:00
}
if !IsValidId(o.ChannelId) {
return NewAppError("Post.IsValid", "model.post.is_valid.channel_id.app_error", nil, "", http.StatusBadRequest)
2015-06-15 03:53:32 -04:00
}
if !(IsValidId(o.RootId) || o.RootId == "") {
return NewAppError("Post.IsValid", "model.post.is_valid.root_id.app_error", nil, "", http.StatusBadRequest)
2015-06-15 03:53:32 -04:00
}
if !(len(o.OriginalId) == 26 || o.OriginalId == "") {
return NewAppError("Post.IsValid", "model.post.is_valid.original_id.app_error", nil, "", http.StatusBadRequest)
2015-06-15 03:53:32 -04:00
}
Relax 4k post message limit (#8478) * MM-9661: rename POST_MESSAGE_MAX_RUNES to \0_v1 * MM-9661: s/4000/POST_MESSAGE_MAX_RUNES_V1/ in tests * MM-9661: introduce POST_MESSAGE_MAX_RUNES_V2 * MM-9661: migrate Postgres Posts.Message column to TEXT from VARCHAR(4000) This is safe to do in a production instance since the underyling type is not changing. We explicitly don't do this automatically for MySQL, but also don't need to since the ORM would have already created a TEXT column for MySQL in that case. * MM-9661: emit MaxPostSize in client config This value remains unconfigurable at this time, but exposes the current limit to the client. The limit remains at 4k in this commit. * MM-9661: introduce and use SqlPostStore.GetMaxPostSize Enforce a byte limitation in the database, and use 1/4 of that value as the rune count limitation (assuming a worst case UTF-8 representation). * move maxPostSizeCached, lastPostsCache and lastPostTimeCache out of the global context and onto the SqlPostStore * address feedback from code review: * ensure sqlstore unit tests are actually being run * move global caches into SqlPostStore * leverage sync.Once to address a race condition * modify upgrade semantics to match new db semantics gorp's behaviour on creating columns with a maximum length on Postgres differs from MySQL: * Postgres * gorp uses TEXT for string columns without a maximum length * gorp uses VARCHAR(N) for string columns with a maximum length of N * MySQL * gorp uses TEXT for string columns with a maximum length >= 256 * gorp uses VARCHAR(N) for string columns with a maximum length of N * gorp defaults to a maximum length of 255, implying VARCHAR(255) So the Message column has been TEXT on MySQL but VARCHAR(4000) on Postgres. With the new, longer limits of 65535, and without changes to gorp, the expected behaviour is TEXT on MySQL and VARCHAR(65535) on Postgres. This commit makes the upgrade semantics match the new database semantics. Ideally, we'd revisit the gorp behaviour at a later time. * allow TestMaxPostSize test cases to actually run in parallel * default maxPostSizeCached to POST_MESSAGE_MAX_RUNES_V1 in case the once initializer panics * fix casting error * MM-9661: skip the schema migration for Postgres It turns out resizing VARCHAR requires a rewrite in some versions of Postgres, but migrating VARCHAR to TEXT does not. Given the increasing complexity, let's defer the migration to the enduser instead.
2018-03-26 17:55:35 -04:00
if utf8.RuneCountInString(o.Message) > maxPostSize {
return NewAppError("Post.IsValid", "model.post.is_valid.message_length.app_error",
map[string]any{"Length": utf8.RuneCountInString(o.Message), "MaxLength": maxPostSize}, "id="+o.Id, http.StatusBadRequest)
2015-06-15 03:53:32 -04:00
}
2021-07-12 14:05:36 -04:00
if utf8.RuneCountInString(o.Hashtags) > PostHashtagsMaxRunes {
return NewAppError("Post.IsValid", "model.post.is_valid.hashtags.app_error", nil, "id="+o.Id, http.StatusBadRequest)
2015-06-15 03:53:32 -04:00
}
switch o.Type {
case
2021-07-12 14:05:36 -04:00
PostTypeDefault,
PostTypeSystemGeneric,
PostTypeJoinLeave,
PostTypeAutoResponder,
PostTypeAddRemove,
PostTypeJoinChannel,
PostTypeGuestJoinChannel,
PostTypeLeaveChannel,
PostTypeJoinTeam,
PostTypeLeaveTeam,
PostTypeAddToChannel,
PostTypeAddGuestToChannel,
PostTypeRemoveFromChannel,
PostTypeMoveChannel,
PostTypeAddToTeam,
PostTypeRemoveFromTeam,
PostTypeMessageAttachment,
2021-07-12 14:05:36 -04:00
PostTypeHeaderChange,
PostTypePurposeChange,
PostTypeDisplaynameChange,
PostTypeConvertChannel,
PostTypeChannelDeleted,
PostTypeChannelRestored,
PostTypeChangeChannelPrivacy,
PostTypeAddBotTeamsChannels,
PostTypeReminder,
MM-53125 Add feature to convert group message to private channel (#24421) * Added convert to channel menu item * WIP * refactored channel name input field and created conversion modal * style * style * WIP * wip * Created API to fetch common teams of GM members * Added UI for all members deactivated * Fetched common teams in client * WIP * Added a required attribute to DropdownInput component * Fixed a case with dropdown input required flag * WIP * API first draft * Genetayed layers and mocks * Fixed create channel bug * WIP * Added cache invalidation * Calling API from client * Updated API to accept name and display name as well * WIP * Moved converted GM to correct category * Style fixes * Added logic to move user to new team/channel after GM conversion * Prevented guest user from performing action * Added loading indicator * Added smoother height transistion when loading finishes * UI imporvements * WIP * Formatted GM conversion message on client side * lint fix * Moved convert option from sidebar menu to channel header menu * Some cleanup * Updated server layers * Fixed i18n * Fixed types * Fix server i18n * Fixed channel creation bug * Added store test for GetCommonTeamIDsForMultipleUsers * Server tests done * Updated snapshots * Updated layers * lint fix * Update tests * For CI * lint * restored debug code * Used user ID instead of username in channel conversion post * WIP * Review fixes * LInt fixes * Test fix * WIP * WIP * WIP * wip * Review fixes, lots of them * Review fix * Disabled WIP test * test * Cleanup * Test fix * removed testing line * Fixed incorrect default message * Review fixes * Fixes * lint and i18n fix * Setting category on server side * updated i18n * Updated tests * Added tests * Refs cleanup * added test --------- Co-authored-by: Harshil Sharma <harshilsharma@Harshils-MacBook-Pro.local>
2023-09-19 08:41:34 -04:00
PostTypeMe,
Feature: Wrangler (#23602) * Migrate feature/wrangler to mono-repo * Add wrangler files * Fix linters, types, etc * Fix snapshots * Fix playwright * Fix pipelines * Fix more pipeline * Fixes for pipelines * More changes for pipeline * Fix types * Add support for a feature flag, but leave it defaulted on for spinwick usage for now * Update snapshot * fix js error when removing last value of multiselect, support CSV marshaling to string array for textsetting * Fix linter * Remove TODO * Remove another TODO * fix tests * Fix i18n * Add server tests * Fix linter * Fix linter * Use proper icon for dot menu * Update snapshot * Add Cypress UI tests for various entrypoints to move thread modal, split SCSS out from forward post into its own thing * clean up * fix linter * More cleanup * Revert files to master * Fix linter for e2e tests * Make ForwardPostChannelSelect channel types configurable with a prop * Add missing return * Fixes from PR feedback * First batch of PR Feedback * Another batch of PR changes * Fix linter * Update snapshots * Wrangler system messages are translated to each user's locale * Initially translate Wrangler into system locale rather than initiating user * More fixes for PR Feedback * Fix some server tests * More updates with master. Fixes around pipelines. Enforce Enterprise license on front/back end * Add tests for dot_menu * More pipeline fixes * Fix e2etests prettier * Update cypress tests, change occurrences of 'Wrangler' with 'Move Thread' * Fix linter * Remove enterprise lock * A couple more occurrences of wrangler strings, and one more enterprise lock * Fix server tests * Fix i18n * Fix e2e linter * Feature flag shouldn't be on by default * Enable move threads feature in smoke tests (#25657) * enable move threads feature * add @prod tag * Fix move_thread_from_public_channel e2e test * Fix e2e style --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: yasserfaraazkhan <attitude3cena.yf@gmail.com>
2023-12-11 15:27:34 -05:00
PostTypeWrangler,
[MM-61758] Burn on read feature (#34703) * Add read receipt store for burn on read message types * update mocks * fix invalidation target * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * add translations * Added logic to associate files of BoR post with the post * Added test * fixes * disable pinning posts and review comments * MM-66594 - Burn on read UI integration (#34647) * MM-66244 - add BoR visual components to message editor * MM-66246 - BoR visual indicator for sender and receiver * MM-66607 - bor - add timer countdown and autodeletion * add the system console max time to live config * use the max expire at and create global scheduler to register bor messages * use seconds for BoR config values in BE * implement the read by text shown in the tooltip logic * unestack the posts from same receiver and BoR and fix styling * avoid opening reply RHS * remove unused dispatchers * persis the BoR label in the drafts * move expiration value to metadata * adjust unit tests to metadata insted of props * code clean up and some performance improvements; add period grace for deletion too * adjust migration serie number * hide bor messages when config is off * performance improvements on post component and code clean up * keep bor existing post functionality if config is disabled * Add read receipt store for burn on read message types * Add temporary posts table * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * avoid reacting to unrevealed bor messages * adjust migration number * Add read receipt store for burn on read message types * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * adjust post reveal and type with backend changes * use real config values, adjust icon usage and style * adjust the delete from from sender and receiver * improve self deleting logic by placing in badge, use burn endpoint * adjust websocket events handling for the read by sender label information * adjust styling for concealed and error state * update burn-on-read post event handling for improved recipient tracking and multi-device sync * replace burn_on_read with type in database migrations and model * remove burn_on_read metadata from PostMetadata and related structures * Added logic to associate files of BoR post with the post * Added test * adjust migration name and fix linter * Add read receipt store for burn on read message types * update mocks * have consistent case on index creation * Add temporary posts table * add mock * add transaction support * reflect review comments * wip: Add reveal endpoint * user check error id instead * wip: Add ws events and cleanup for burn on read posts * add burn endpoint for explicitly burning messages * Added logic to associate files of BoR post with the post * Added test * disable pinning posts and review comments * show attachment on bor reveal * remove unused translation * Enhance burn-on-read post handling and refine previous post ID retrieval logic * adjust the returning chunk to work with bor messages * read temp post from master db * read from master * show the copy link button to the sender * revert unnecessary check * restore correct json tag * remove unused error handling and clarify burn-on-read comment * improve type safety and use proper selectors * eliminate code duplication in deletion handler * optimize performance and add documentation * delete bor message for sender once all receivers reveal it * add burn on read to scheduled posts * add feature enable check * use master to avoid all read recipients race condition --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com> Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> * squash migrations into single file * add configuration for the scheduler * don't run messagehasbeenposted hook * remove parallel tests on burn on read * add clean up for closing opened modals from previous tests * simplify delete menu item rendering * add cleanup step to close open modals after each test to prevent pollution * streamline delete button visibility logic for Burn on Read posts * improve reliability of closing post menu and modals by using body ESC key --------- Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com> Co-authored-by: Pablo Vélez <pablovv2012@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2025-12-11 01:59:50 -05:00
PostTypeGMConvertedToChannel,
PostTypeAutotranslationChange,
Merge the Integrated Boards MVP feature branch (#35796) * Add CreatedBy and UpdatedBy to the properties fields and values (#34485) * Add CreatedBy and UpdatedBy to the properties fields and values * Fix types --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds ObjectType to the property fields table (#34908) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Update ObjectType migration setting an empty value and marking the column as not null (#34915) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds uniqueness mechanisms to the property fields (#35058) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Fixing retrylayer mocks * Remove retrylayer duplication * Address review comments * Fix comment to avoid linter issues * Address PR comments * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.down.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update server/channels/db/migrations/postgres/000157_add_object_type_to_property_fields.up.sql Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Update field validation to check only for valid target types * Update migrations to avoid concurrent index creation within a transaction * Update migrations to make all index ops concurrent * Update tests to use valid PSAv2 property fields * Adds a helper for valid PSAv2 TargetTypes --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> * Fix property tests (#35388) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards feature flag (#35378) Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds Integrated Boards MVP API changes (#34822) This PR includes the necessary changes for channels and posts endpoints and adds a set of generic endpoints to retrieve and manage property fields and values following the new Property System approach. Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Mattermost Build <build@mattermost.com> * Property System Architecture permissions for v2 (#35113) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Fix i18n sorting --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Add Views store and app layer (#35361) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor test loops in ViewStore tests for improved readability * change pagination to limit/offset * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * MM-67388, MM-66528, MM-67750: Add View REST API endpoints, websocket events, and sort order (#35442) * Add Views store and app layer for Integrated Boards Implements the View entity (model, SQL store, service, app) as described in the Integrated Boards tech spec. Views are channel-scoped board configurations with typed props (board, kanban subviews) and soft-delete. - public/model: View, ViewBoardProps, Subview, ViewPatch types with PreSave/PreUpdate/IsValid/Patch/Clone/Auditable - Migration 158: Views table with jsonb Props column and indexes - SqlViewStore: CRUD with nil-safe Props marshaling (AppendBinaryFlag) - ViewService: CreateView seeds default kanban subview and links the boards property field; caches boardPropertyFieldID at startup - App layer: CreateView/GetView/GetViewsForChannel/UpdateView/DeleteView with channel-membership permission checks and WebSocket events (view_created, view_updated, view_deleted) - doSetupBoardsPropertyField: registers the Boards property group and board field in NewServer() before ViewService construction - GetFieldByName now returns store.ErrNotFound instead of raw sql.ErrNoRows * Move permission checks out of App layer for views - Remove HasPermissionToChannel calls from all App view methods - Drop userID params from GetView, GetViewsForChannel, UpdateView, DeleteView - Fix doSetupBoardsPropertyField to include required TargetType for PSAv2 field * Make View service generic and enforce board validation in model - Remove board-specific auto-setup from service and server startup - Enforce that board views require Props, at least one subview, and at least one linked property in IsValid() - Move default subview seeding out of app layer; callers must provide valid props - Call PreSave on subviews during PreUpdate to assign IDs to new subviews - Update all tests to reflect the new validation requirements * Restore migrations files to match base branch * Distinguish ErrNotFound from other errors in view store Get * Use CONCURRENTLY and nontransactional for index operations in views migration * Split views index creation into separate nontransactional migrations * Update migrations.list * Update i18n translations for views * Fix makeView helper to include required Props for board view validation * Rename ctx parameter from c to rctx in OAuthProvider mock * Remove views service layer, call store directly from app * Return 500 for unexpected DB errors in GetView, 404 only for not-found * Harden View model: deep-copy Props, validate linked property IDs - Add ViewBoardProps.Clone() to deep-copy LinkedProperties and Subviews - Use it in View.Clone() and View.Patch() to prevent shared-slice aliasing - Iterate over LinkedProperties in View.IsValid() and reject invalid IDs with a dedicated i18n key - Register ViewStore in storetest AssertExpectations so mock expectations are enforced - Add tests covering all new behaviours * Restore autotranslation worker_stopped i18n translation * Fix view store test IDs and improve error handling in app layer - Use model.NewId() for linked property IDs in testUpdateView to fix validation failure (IsValid rejects non-UUID strings) - Fix import grouping in app/view.go (stdlib imports in one block) - Return 404 instead of 500 when Update/Delete store calls return ErrNotFound (e.g. concurrent deletion TOCTOU race) * Add View store mock to retrylayer test genStore helper The View store was added to the store interface but the genStore() helper in retrylayer_test.go was not updated, causing TestRetry to panic. Also removes the duplicate Recap mock registration. * Refactor view deletion and websocket event handling; update SQL store methods to use query builder * revert property field store * Add View API endpoints with OpenAPI spec, client methods, and i18n Implement REST API for channel views (board-type) behind the IntegratedBoards feature flag. Adds CRUD endpoints under /api/v4/channels/{channel_id}/views with permission checks matching the channel bookmark pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove useless migrations * Add cursor-based pagination to View store GetForChannel - Add ViewQueryCursor and ViewQueryOpts types with validation - Return (views, cursor, error) for caller-driven pagination - PerPage clamping: <=0 defaults to 20, >200 clamps to 200 - Support IncludeDeleted filter - Add comprehensive store tests for pagination, cursor edge cases, PerPage clamping, and invalid input rejection - Add app layer test for empty channelID → 400 - Update interface, retrylayer, timerlayer, and mock signatures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add cursor-based pagination to View API for channel views * Enhance cursor handling in getViewsForChannel and update tests for pagination * Refactor test loops in ViewStore tests for improved readability * Refactor loop in TestGetViewsForChannel for improved readability * change pagination to limit/offset * switch to limit/offset pagination * Add upper-bound limits on View Subviews and LinkedProperties Defense-in-depth validation: cap Subviews at 50 and LinkedProperties at 500 to prevent abuse below the 300KB payload limit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add view sort order API endpoint Add POST /api/v4/channels/{channel_id}/views/{view_id}/sort_order endpoint following the channel bookmarks reorder pattern. Includes store, app, and API layers with full test coverage at each layer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add connectionId to view WebSocket events and sort_order API spec Thread connectionId from request header through all view handlers (create, update, delete, sort_order) to WebSocket events, matching the channel bookmarks pattern. Add sort_order endpoint to OpenAPI spec. Update minimum server version to 11.6. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove duplicate View/ViewPatch definitions from definitions.yaml The merge from integrated-boards-mvp introduced duplicate View and ViewPatch schema definitions that were already defined earlier in the file with more detail (including ViewBoardProps ref and enums). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update minimum server version to 11.6 in views API spec Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add missing translations for view sort order error messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Merge integrated-boards-mvp into ibmvp_api-views; remove spec files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix flaky TestViewStore timestamp test on CI Add sleep before UpdateSortOrder to ensure timestamps differ, preventing same-millisecond comparisons on fast CI machines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove duplicate views.yaml imclude * Use c.boolString() for include_deleted query param in GetViewsForChannel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix views.yaml sort order schema: use integer type and require body Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor view sort order tests to use named IDs instead of array indices Extract idA/idB/idC from views slice and add BEFORE/AFTER comments to make stateful subtest ordering easier to follow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Return 404 instead of 403 for view operations on deleted channels Deleted channels should appear non-existent to callers rather than revealing their existence via a 403. Detailed error text explains the context for debugging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * add missing channel deleteat checks * Use c.Params.Page instead of manual page query param parsing in getViewsForChannel c.Params already validates and defaults page/per_page, so the manual parsing was redundant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add support for total count in views retrieval * Add tests for handling deleted views in GetViewsForChannel and GetView * Short-circuit negative newIndex in UpdateSortOrder before opening transaction Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add per-channel limit on views to bound UpdateSortOrder cost Without a cap, unbounded view creation makes sort-order updates increasingly expensive (CASE WHEN per view, row locks). Adds MaxViewsPerChannel=50 constant and enforces it in the app layer before saving. Includes API and app layer tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove include_deleted support from views API Soft-deleted views are structural metadata with low risk, but no other similar endpoint (e.g. channel bookmarks) exposes deleted records without an admin gate. Rather than adding an admin-only permission check for consistency, remove the feature entirely since there is no current use case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update view permissions to require `create_post` instead of channel management permissions * Remove obsolete view management error messages for direct and group messages --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(migrations): add user tracking and object type to property fields - Introduced user tracking columns (CreatedBy, UpdatedBy) to PropertyFields and PropertyValues. - Added ObjectType column to PropertyFields with associated unique indexes for legacy and typed properties. - Created new migration scripts for adding and dropping these features, including necessary indexes for data integrity. - Established views for managing property fields with new attributes. This update enhances the schema to support better tracking and categorization of property fields. * Add Property System Architecture v2 API endpoints (#35583) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Require target_type filter when searching property fields * Add objectType validation as part of field.IsValid() * Fix linter * Fix test with bad objecttpye * Fix test grouping --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * MM-67968: Flatten view model — remove icon, subviews, typed board props (#35726) * feat(views): flatten view model by removing icon, subview, and board props Simplifies the View data model as part of MM-67968: removes Icon, Subview, and ViewBoardProps types; renames ViewTypeBoard to ViewTypeKanban; replaces typed Props with StringInterface (map[string]any); adds migration 000167 to drop the Icon column from the Views table. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(api): update views OpenAPI spec to reflect flattened model Removes ViewBoardProps, Subview, and icon from the View and ViewPatch schemas. Changes type enum from board to kanban. Replaces typed props with a free-form StringInterface object. Aligns with MM-67968. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * refactor(views): simplify store by dropping dbView and marshalViewProps StringInterface already implements driver.Valuer and sql.Scanner, so the manual JSON marshal/unmarshal and the dbView intermediate struct were redundant. model.View now scans directly from the database. Also removes the dead ViewMaxLinkedProperties constant and wraps the Commit() error in UpdateSortOrder. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(api): allow arbitrary JSON in view props OpenAPI schema The props field was restricted to string values via additionalProperties: { type: string }, conflicting with the Go model's StringInterface (map[string]any). Changed to additionalProperties: true in View, ViewPatch, and inline POST schemas. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Adds basic implementation of the generic redux store for PSAv2 (#35512) * Adds basic implementation of the generic redux store for PSAv2 * Add created_by and updated_by to the test fixtures * Make target_id, target_type and object_type mandatory * Wrap getPropertyFieldsByIds and getPropertyValuesForTargetByFieldIds with createSelector * Address PR comments --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * Adds websocket messages for the PSAv2 API events (#35696) * Adds uniqueness mechanisms to the property fields After adding ObjectType, this commit ensures that both the PSAv1 and PSAv2 schemas are supported, and enforces property uniqueness through both database indexes and a logical check when creating new property fields. * Adds uniqueness check to property updates Updates are covered on this commit and we refactor as well the SQL code to use the squirrel builder and work better with the conditional addition of the `existingID` piece of the query. * Add translations to error messages * Add the permissions to the migrations, model and update the store calls * Adds the property field and property group app layer * Adds authorization helpers for property fields and values * Make sure that users cannot lock themselves out of property fields * Migrate permissions from a JSON column to three normalized columns * Remove the audit comment * Use target level constants in authorization * Log authorization membership failures * Rename admin to sysadmin * Adds the Property System Architecture v2 API endpoints * Adds permission checks to the create field endpoint * Add target access checks to value endpoints * Add default branches for object_type and target_type and extra guards for cursor client4 methods * Fix vet API mismatch * Fix error checks * Fix linter * Add merge semantics for property patch logic and API endpoint * Fix i18n * Fix duplicated patch elements and early return on bad cursor * Update docs to use enums * Fix i18n sorting * Update app layer to return model.AppError * Adds a limit to the number of property values that can be patched in the same request * Adds websocket messages for the PSAv2 API events * Add IsPSAv2 helper to the property field for clarity * Add guard against nil returns on field deletion * Add docs to the websocket endpoints --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> * migrations: consolidate views migrations and reorder after master - Merged 000165 (create Views) with 000167 (drop Icon) since Icon was never needed - Renumbered branch migrations 159-166 → 160-167 so master's 000159 (deduplicate_policy_names) runs first - Regenerated migrations.list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add API endpoint to retrieve posts for a specific view (#35604) Automatic Merge * Apply fixes after merge * Return a more specific error from getting multiple fields * Prevent getting broadcast params on field deletion if not needed * Remove duplicated migration code * Update property conflict code to always use master * Adds nil guard when iterating on property fields * Check that permission level is valid before getting rejected by the database * Validate correctness on TargetID for PSAv2 fields * Avoid PSAv1 using permissions or protected * Fix test data after validation change * Fix flaky search test * Adds more posts for filter use cases to properly test exclusions --------- Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es> Co-authored-by: Alejandro García Montoro <alejandro.garciamontoro@gmail.com> Co-authored-by: Julien Tant <julien@craftyx.fr> Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: Julien Tant <785518+JulienTant@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 05:36:35 -04:00
PostTypeBurnOnRead,
PostTypeCard:
default:
2021-07-12 14:05:36 -04:00
if !strings.HasPrefix(o.Type, PostCustomTypePrefix) {
return NewAppError("Post.IsValid", "model.post.is_valid.type.app_error", nil, "id="+o.Type, http.StatusBadRequest)
}
2015-06-15 03:53:32 -04:00
}
[MM-22051] Remove To/From JSON (#18070) * Posts * Add missing translation * Fix internal store marshaling * [MM-22051] Remove To/From JSON (Channels) (#18116) * Channels * Channel members * ChannelSearch * Channel categories, list, sidebar, stats, view * Fix conversions * [MM-22051] Remove To/From JSON (Users) (#18121) * User related structs * Fix return * Team related structures (#18127) * [MM-22051] Remove To/From JSON (Status, Bot, Reaction, Thread, FileInfo) (#18130) * Status * Bot * Reaction * Thread * FileInfo * Some fixes * Translations update from Weblate (#18143) * Translated using Weblate (German) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/de/ * Translated using Weblate (Turkish) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/tr/ * Translated using Weblate (Hungarian) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/hu/ * Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ * Translated using Weblate (German) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/de/ * Translated using Weblate (Turkish) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/tr/ * Translated using Weblate (Hungarian) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/hu/ * Translated using Weblate (English (Australia)) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/en_AU/ * Translated using Weblate (Bulgarian) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/bg/ * Translated using Weblate (Japanese) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ja/ * Translated using Weblate (Chinese (Simplified)) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/zh_Hans/ Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> Co-authored-by: Weblate (bot) <hosted@weblate.org> Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> * [MM-22051] Remove To/From JSON methods from model (#18138) * Scheme * Role * Session * Config * Status * Fix logic * Emoji * GuestsInvite * Group * Command * ClusterInfo * License * Job * System * Plugin * Command2 * IncomingWebhook * OutgoingWebhook * Fix tests * Update traslation * Some fixes * Add missing return * Simplify * Make Config.ToJSONFiltered() return []byte * Make Busy.ToJSON() return []byte * Include error in log * Split logic * [MM-22051] Remove To/From JSON (final) (#18150) * SwitchRequest * PluginEventData * Permalink * PushNotification * SuggestCommand * PluginsResponse * WebSocketMessage * RemoteCluster * SharedChannel * PluginStatuses * InitialLoad * ClusterDiscovery * ClusterStats * MfaSecret * GroupSyncable * SAML * WebSocketRequest * TypingRequest * SecurityBulletin * OAuthApp * IntegrationAction * DataRetention * Preference * FileInfoList * Compliance * Preferences * FileInfoSearchResults * TermsOfService * InstallMarketplacePluginRequest * GitLabUser * UploadSessions * Remove unused helpers * Fix tests * [MM-23280] Fix linting for ToJSON/FromJSON (#18153) * SwitchRequest * PluginEventData * Permalink * PushNotification * SuggestCommand * PluginsResponse * WebSocketMessage * RemoteCluster * SharedChannel * PluginStatuses * InitialLoad * ClusterDiscovery * ClusterStats * MfaSecret * GroupSyncable * SAML * WebSocketRequest * TypingRequest * SecurityBulletin * OAuthApp * IntegrationAction * DataRetention * Preference * FileInfoList * Compliance * Preferences * FileInfoSearchResults * TermsOfService * InstallMarketplacePluginRequest * GitLabUser * UploadSessions * Remove unused helpers * Fix tests * Fix linting for ToJSON/FromJSON * Fix conversions Co-authored-by: Weblate (bot) <hosted@weblate.org> Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-09-01 08:43:12 -04:00
if utf8.RuneCountInString(ArrayToJSON(o.Filenames)) > PostFilenamesMaxRunes {
return NewAppError("Post.IsValid", "model.post.is_valid.filenames.app_error", nil, "id="+o.Id, http.StatusBadRequest)
2015-06-15 03:53:32 -04:00
}
[MM-22051] Remove To/From JSON (#18070) * Posts * Add missing translation * Fix internal store marshaling * [MM-22051] Remove To/From JSON (Channels) (#18116) * Channels * Channel members * ChannelSearch * Channel categories, list, sidebar, stats, view * Fix conversions * [MM-22051] Remove To/From JSON (Users) (#18121) * User related structs * Fix return * Team related structures (#18127) * [MM-22051] Remove To/From JSON (Status, Bot, Reaction, Thread, FileInfo) (#18130) * Status * Bot * Reaction * Thread * FileInfo * Some fixes * Translations update from Weblate (#18143) * Translated using Weblate (German) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/de/ * Translated using Weblate (Turkish) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/tr/ * Translated using Weblate (Hungarian) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/hu/ * Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ * Translated using Weblate (German) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/de/ * Translated using Weblate (Turkish) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/tr/ * Translated using Weblate (Hungarian) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/hu/ * Translated using Weblate (English (Australia)) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/en_AU/ * Translated using Weblate (Bulgarian) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/bg/ * Translated using Weblate (Japanese) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ja/ * Translated using Weblate (Chinese (Simplified)) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/zh_Hans/ Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> Co-authored-by: Weblate (bot) <hosted@weblate.org> Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> * [MM-22051] Remove To/From JSON methods from model (#18138) * Scheme * Role * Session * Config * Status * Fix logic * Emoji * GuestsInvite * Group * Command * ClusterInfo * License * Job * System * Plugin * Command2 * IncomingWebhook * OutgoingWebhook * Fix tests * Update traslation * Some fixes * Add missing return * Simplify * Make Config.ToJSONFiltered() return []byte * Make Busy.ToJSON() return []byte * Include error in log * Split logic * [MM-22051] Remove To/From JSON (final) (#18150) * SwitchRequest * PluginEventData * Permalink * PushNotification * SuggestCommand * PluginsResponse * WebSocketMessage * RemoteCluster * SharedChannel * PluginStatuses * InitialLoad * ClusterDiscovery * ClusterStats * MfaSecret * GroupSyncable * SAML * WebSocketRequest * TypingRequest * SecurityBulletin * OAuthApp * IntegrationAction * DataRetention * Preference * FileInfoList * Compliance * Preferences * FileInfoSearchResults * TermsOfService * InstallMarketplacePluginRequest * GitLabUser * UploadSessions * Remove unused helpers * Fix tests * [MM-23280] Fix linting for ToJSON/FromJSON (#18153) * SwitchRequest * PluginEventData * Permalink * PushNotification * SuggestCommand * PluginsResponse * WebSocketMessage * RemoteCluster * SharedChannel * PluginStatuses * InitialLoad * ClusterDiscovery * ClusterStats * MfaSecret * GroupSyncable * SAML * WebSocketRequest * TypingRequest * SecurityBulletin * OAuthApp * IntegrationAction * DataRetention * Preference * FileInfoList * Compliance * Preferences * FileInfoSearchResults * TermsOfService * InstallMarketplacePluginRequest * GitLabUser * UploadSessions * Remove unused helpers * Fix tests * Fix linting for ToJSON/FromJSON * Fix conversions Co-authored-by: Weblate (bot) <hosted@weblate.org> Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-09-01 08:43:12 -04:00
if utf8.RuneCountInString(ArrayToJSON(o.FileIds)) > PostFileidsMaxRunes {
return NewAppError("Post.IsValid", "model.post.is_valid.file_ids.app_error", nil, "id="+o.Id, http.StatusBadRequest)
}
[MM-22051] Remove To/From JSON (#18070) * Posts * Add missing translation * Fix internal store marshaling * [MM-22051] Remove To/From JSON (Channels) (#18116) * Channels * Channel members * ChannelSearch * Channel categories, list, sidebar, stats, view * Fix conversions * [MM-22051] Remove To/From JSON (Users) (#18121) * User related structs * Fix return * Team related structures (#18127) * [MM-22051] Remove To/From JSON (Status, Bot, Reaction, Thread, FileInfo) (#18130) * Status * Bot * Reaction * Thread * FileInfo * Some fixes * Translations update from Weblate (#18143) * Translated using Weblate (German) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/de/ * Translated using Weblate (Turkish) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/tr/ * Translated using Weblate (Hungarian) Currently translated at 100.0% (2309 of 2309 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/hu/ * Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ Update translation files Updated by "Cleanup translation files" hook in Weblate. Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ * Translated using Weblate (German) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/de/ * Translated using Weblate (Turkish) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/tr/ * Translated using Weblate (Hungarian) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/hu/ * Translated using Weblate (English (Australia)) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/en_AU/ * Translated using Weblate (Bulgarian) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/bg/ * Translated using Weblate (Japanese) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/ja/ * Translated using Weblate (Chinese (Simplified)) Currently translated at 100.0% (2301 of 2301 strings) Translation: mattermost-languages-shipped/mattermost-server Translate-URL: https://translate.mattermost.com/projects/mattermost/mattermost-server_master/zh_Hans/ Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> Co-authored-by: Weblate (bot) <hosted@weblate.org> Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> * [MM-22051] Remove To/From JSON methods from model (#18138) * Scheme * Role * Session * Config * Status * Fix logic * Emoji * GuestsInvite * Group * Command * ClusterInfo * License * Job * System * Plugin * Command2 * IncomingWebhook * OutgoingWebhook * Fix tests * Update traslation * Some fixes * Add missing return * Simplify * Make Config.ToJSONFiltered() return []byte * Make Busy.ToJSON() return []byte * Include error in log * Split logic * [MM-22051] Remove To/From JSON (final) (#18150) * SwitchRequest * PluginEventData * Permalink * PushNotification * SuggestCommand * PluginsResponse * WebSocketMessage * RemoteCluster * SharedChannel * PluginStatuses * InitialLoad * ClusterDiscovery * ClusterStats * MfaSecret * GroupSyncable * SAML * WebSocketRequest * TypingRequest * SecurityBulletin * OAuthApp * IntegrationAction * DataRetention * Preference * FileInfoList * Compliance * Preferences * FileInfoSearchResults * TermsOfService * InstallMarketplacePluginRequest * GitLabUser * UploadSessions * Remove unused helpers * Fix tests * [MM-23280] Fix linting for ToJSON/FromJSON (#18153) * SwitchRequest * PluginEventData * Permalink * PushNotification * SuggestCommand * PluginsResponse * WebSocketMessage * RemoteCluster * SharedChannel * PluginStatuses * InitialLoad * ClusterDiscovery * ClusterStats * MfaSecret * GroupSyncable * SAML * WebSocketRequest * TypingRequest * SecurityBulletin * OAuthApp * IntegrationAction * DataRetention * Preference * FileInfoList * Compliance * Preferences * FileInfoSearchResults * TermsOfService * InstallMarketplacePluginRequest * GitLabUser * UploadSessions * Remove unused helpers * Fix tests * Fix linting for ToJSON/FromJSON * Fix conversions Co-authored-by: Weblate (bot) <hosted@weblate.org> Co-authored-by: JtheBAB <srast@bioc.uzh.ch> Co-authored-by: Kaya Zeren <kayazeren@gmail.com> Co-authored-by: Tóth Csaba // Online ERP Hungary Kft <csaba.toth@online-erp.hu> Co-authored-by: Matthew Williams <Matthew.Williams@outlook.com.au> Co-authored-by: Nikolai Zahariev <nikolaiz@yahoo.com> Co-authored-by: kaakaa <stooner.hoe@gmail.com> Co-authored-by: aeomin <lin@aeomin.net> Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-09-01 08:43:12 -04:00
if utf8.RuneCountInString(StringInterfaceToJSON(o.GetProps())) > PostPropsMaxRunes {
return NewAppError("Post.IsValid", "model.post.is_valid.props.app_error", nil, "id="+o.Id, http.StatusBadRequest)
2015-12-03 11:57:20 -05:00
}
2015-06-15 03:53:32 -04:00
return nil
}
func (o *Post) SanitizeProps() {
[MM-30831] granular data retention wireup (#17417) * pre-checkout commit * add API endpoints for retention policies * allow deleting multiple teams/channels from a policy in a single request * pre-checkout commit * add auditing in API functions * add permission checks * update the store layers * update storetest * add check constraint on PostDuration column * pre-checkout commit * add query to delete posts under the scope of a granular retention policy * add suggestions from sbishel * allow clients to specify channels/teams when creating a new policy * remove foreign keys referencing Channels and Teams tables * add checks for whether teams and channels exist * pre-checkout commit * remove data referencing the Posts table * pre-checkout commit * write data store tests * sort results of buildGetPoliciesQuery * add missing test cases for teams * pre-checkout commit * add Client4 methods for data retention policy endpoints * add uint and uint64 to app/layer_generators * make granular policies override global policies * fix lint errors * pre-checkout commit * add license to top of files * add tests for data store layer * add missing test cases for store layer * run make i18n-extract * add query to delete ChannelMemberHistory * work in progress * add test for old reply to old post * fix lint error * use COALESCE on each Posts column * begin implementing orphaned rows worker * split PR * pre-checkout commit * use RetentionPolicyWithTeamAndChannelCounts * update app and api layers * run make i18n-extract * add RetentionPolicy to retrylayer_test.go * Revert "split PR" This reverts commit b316f03dd307a30deae931944ca7e4a1cc904605. * fix errors caused by revert * add suggestions from sbishel * fix copy-paste error * fix lint errors * pre-checkout commit * add function to delete orphaned rows * use -1 for infinite retention * remove check constraint * copy i18n entries from master * re-run tests with newer enterprise branch * add team data to channel list * add search for channels and teams in a policy * add store tests for channel and team search * add suggestions from mkraft * run make einterfaces-mocks * fix lint errors * add suggestions from mkraft * move removeOrphanedRows method to wireup branch * Revert "move removeOrphanedRows method to wireup branch" This reverts commit 94605c9b4a5378ffa44a3dec4d3f8e3306b9d33e. * use DeleteOrphanedRows where possible * run make i18n-extract * use COMPLIANCE permissions * run make migrations-bindadta * clean up teams before test * fix tests for TestRetentionPolicyStore * add API endpoints for mobile * fix lint error * fix some of the lint errors * move user/data_retention endpoints to data_retention.go * Revert "fix some of the lint errors" This reverts commit b5b2dc27566c427187db942c5c0afe319e7679c4. * add exclude_policy_constrained parameter for /channels and /teams * fix lint errors * add policy_id field to GET endpoints for channels and teams * use PolicyWithTeamID in RetentionPolicy layer * fix lint errors * run make i18n-extract * update mock call in telemetry_test.go * return status:OK in JSON instead of 204 * pre-checkout commit * add policy_id field on channels/teams * fix lint errors * use sq.Eq instead of '?' * use new subsection permissions * update channels and teams endpoints to use new subsection permissions * add extra search opts for channels in a policy * fix lint errors * allow negative post duration in patch * remove DELETE FROM query in retention policy tests * use *int64 for PostDuration * re-run CI tests * use 3-step deletion strategy for each table * fix lint errors * run make store-layers * re-run CI tests * add test with channel, team and global policies * use common function for SQL queries * add pagination test * use struct for args to common SQL function * fix lint errors * run make i18n-extract * check if Channels.TeamId is "" or nil * use three OR clauses * write separate genericRetentionPoliciesDeletion function * add config setting for BatchSize * add telemetry for BatchSize * use feature flag * add old i18n messages back in * re-run CI tests * update call signature in storetest * MM-30831: Adds constant for retention default batch size. * MM-30831: Removes comment re: optimization. * MM-30831: Converts days to milliseconds. * MM-30831: Reverts change to test. * Revert "MM-30831: Reverts change to test." This reverts commit 6d14275a1ceae682bb9e17ec69b39252b44e0c0c. * Revert "MM-30831: Converts days to milliseconds." This reverts commit a0cb6ec09d854a05194c1daee1c5333f260231c3. * MM-30831: Fixes tests. * MM-30381: Fix for change to method sig. Co-authored-by: Max Erenberg <max.erenberg@mattermost.com> Co-authored-by: Mattermod <mattermod@users.noreply.github.com> Co-authored-by: Martin Kraft <martin@upspin.org>
2021-06-23 07:55:12 -04:00
if o == nil {
return
}
membersToSanitize := []string{
2021-07-12 14:05:36 -04:00
PropsAddChannelMember,
PostPropsForceNotification,
}
for _, member := range membersToSanitize {
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
if _, ok := o.GetProps()[member]; ok {
o.DelProp(member)
}
}
for _, p := range o.Participants {
p.Sanitize(map[string]bool{})
}
}
// Remove any input data from the post object that is not user controlled
func (o *Post) SanitizeInput() {
o.DeleteAt = 0
o.RemoteId = NewPointer("")
if o.Metadata != nil {
o.Metadata.Embeds = nil
}
}
func (o *Post) ContainsIntegrationsReservedProps() []string {
Feature scheduled messages (#28932) * Create scheduled post api (#27920) * Added migration files for Postgres * Added migrations for MySQL * Added store method * Added API and store tests * Renamed migration after syncing with master * Added app layer tests * API is ready * API is ready * API is ready * Renamed migration after syncing with master * Updated migration list * Fixed retry layer tests * Allowed posts with empty messages * Review fixes * Reverted an incorrect change * Renamed migration and fixed ID assignment * CI * Send post button changes (#28019) * added Split button * WIP * Added core menu options * WIP * WIP * WIP * Handled displaying error in creating scheduled post * lint fixes * webapp i18n fix * Review fixes * Fixed a webapp test * A few more fixes * Removed a duplicate comment * Scheduled post job (#28088) * Added the job function * Added query for fetching scheduled posts for pricessing * WIP * WIP * WIP * WIP * WIP * WIP * Reafactoring of scheduled post job * Lint fixes * Updated i18n files * FInishing touches * Added tests for GetScheduledPosts * Added tests for PermanentlyDeleteScheduledPosts * Updated all layer * Some changes as discussed with team * Added tests for UpdatedScheduledPost * Code review refactoring * Added job test * MM-60120 - Custom time selection (#28120) * Added a common date time picker modal and used it for post reminder * Added a common date time picker modal and used it for post reminderggp * Added modal for custom schedule time and fixed TZ issue * WIP * Removed event from useSubmit hook * Removed event from useSubmit hook * Added timezone handling * fixed type error * Updated i18n strings * Minor cleanup * updated snapshots * review fixes * Handled event * Supported for having a DM thread open in RHS while in a regular channel * Review fixes * MM-60136 - Scheduled messages tab (#28133) * WIP * WIP * Created Tabs and Tab wrapper with added styling * Added API to get scheduled posts * WIP * Displated scheduled post count * i18n fix * Added tests * Handled asetting active tab absed on URL: * Reverted unintended change * Added API to client ad OpenAPI specs * Renamed file * Adding fileinfo to schedule posts * Partial review fixes * Made get scheduled post API return posts by teamID * review fixes * Moved scheduled post redux code to MM-redux package * Usedd selector factory * WIP: * WIP: * Lint fix * Fixed an incorrect openapi spec file * Removed redundent permission check * Clreaed scheduled post data on logout * Removed unused i18n string: * lint fix * Render scheduled posts (#28208) * WIP * WIP * Created Tabs and Tab wrapper with added styling * Added API to get scheduled posts * WIP * Displated scheduled post count * i18n fix * Added tests * Handled asetting active tab absed on URL: * Reverted unintended change * Added API to client ad OpenAPI specs * Renamed file * Created common component for draft list item * WIP * WIP * Adding fileinfo to schedule posts * Basic rendering * Added count badge to tabs * WIP * Made the Drafts LHS iteam appear if no drafts exist but scheduled posts do * Fixed icon size * Partial review fixes * Made get scheduled post API return posts by teamID * Handled initial vs team switch load * Displayed scheduled date in panel header * Added error message and error indiocator * WIP * review fixes * WIP Adding error reason tag * Added error codes * Moved scheduled post redux code to MM-redux package * Usedd selector factory * WIP: * WIP: * Lint fix * Fixed an incorrect openapi spec file * Removed redundent permission check * Clreaed scheduled post data on logout * Removed unused i18n string: * lint fix * Opened rescheduling modal * Updated graphic for empty state of schduled post list * Added delete scheduled post option and modal * Badge and timezone fix * WIP: * Added send now confirmation modal * lint * Webapp i18n fix * Fixed webapp test * Fixed a bug where DM/GM scheduled posts weren't immideatly showing up in UI * Minor fixes * WIP * Review fixes * Review fixes * Optimisations * Fixed reducer name * Moment optimizatin * Updated route check * MM-60144 - added API to update a scheduled post (#28248) * WIP * Added api and ap layer for update scheduled post ̛̦̄ * Added API to OpenAI specs, Go client and TS client * removed permissio check * Added tests * Fixed tests * Added PreUpdate method on scheduled post model * MM-60131 - Reschedule post integration (#28281) * Handled rescheduling post in webapp * Added error handling * MM-60146 - Delete scheduled post api (#28265) * WIP * Added api and ap layer for update scheduled post ̛̦̄ * Added API to OpenAI specs, Go client and TS client * removed permissio check * Added tests * Fixed tests * Added PreUpdate method on scheduled post model * Added delete scheduled post API * Added API to Go client and OpenAPI specs * Added API to TS client * Added tests * CI * Rmeoved two incorrect code comments * MM-60653 - Integrated delete scheduled post API (#28296) * Integrated delete scheduled apost API * Lint fix * Review fixes * Excluded draft checks from scheduled posts (#28370) * Excluded draft checks from scheduled posts * Added a removed todo * MM-60125 - Scheduled post channel indicator (#28320) * Integrated delete scheduled apost API * Lint fix * Added state for storing scheduled posts by channel ID * Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere * Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere * WIP * Added scheduled post indiocator * Handled single and multiple scheudled posts * Review fixes * Fixed styling and handled center channel, RHS and threads view * Lint fix * i18n fix * Fixed a cycling dependency * Lint fix * Added some more comments * Updated styling * Review fixes * Added common component for remote user time and scheduled post indicator * Updated scheduled post count * Minor change * Moved CSS code around * Fixed a bug where files in scheduled post didn't show up until refresh (#28359) --------- Co-authored-by: Daniel Espino García <larkox@gmail.com> * Scheduled post config (#28485) * Added config * Added config on server and webapp side * Added config check in server and webapp * Added license check * Added license check * Added placeholder help text * Added license check to job * Fixed job test * Review fixes * Updated English text * Review fixes * MM-60118 - Added index on ScheduledPosts table (#28579) * Added index * Updated indexes * Scheduled posts misc fixes (#28625) * Added detailed logging for scheduled post job * Limited scheduled posts processing to 24 hours * Marked old scheduled posts as unable to send * Added t5ests * converted some logs to trace level * Fixed a bug causing error message to show up on deleting a scheduled post in a deleted thread (#28630) * Fixed scheduled posts link in RHS (#28659) * Fixed scheduled posts link in RHS * Review fixes * Fix permission name in scheduled posts by team (#28580) * Fix permission name * fix wording --------- Co-authored-by: Mattermost Build <build@mattermost.com> * FIxed width of generic modal header to fix browser channel modal (#28639) * Only consider error-free scheduled posts for indicator in channel and RHS (#28683) * Show only errro free scheudled posts in post box indicator * Fixed a bug to handle no scheduled posts * Fixed draft and scheudled post UI in mobile view (#28680) * MM-60873 and MM-60872 - Fixed a bug with updating scheduled posts (#28656) * Fixed a bug with updating scheduled posts * Better selectors * MOved shceuled post message length validation to app layer * MM-60732 - Scheduled posts channel link now takes you to the first scheduled post in channel/thread in list (#28768) * Ordered scheudle dposts by schgeudled at nad create at * Ordered in client * Added scroll to target * Removed classname prop * Fixed tests * Added doc * Import fix * MM-60961 - Fixed a bug where API used incoming create at date for scheduled post (#28703) * Fixed a bug where API used incoming create at date for scheduled post * Stopped sending created at value for scheduled post * MM-60785 - Fixed a bug where scheduled posts of channel we are no longer member of didn't show up (#28637) * Fixed a bug where scheduled posts of channel we are no longer member of didn't show up * Added a comment * CI * Used data loader to optimise laoding missing channels * Minor refactoring * MM-60963 - Added common checks for post and scheduled posts (#28713) * Added commen checks for post and scheuled posts * Sanitised scheduled posts * Fixed tests * Splitted post checks into app and context functions * Added checks on scheduiled posts job as well: * i18n fix * Fixed a test * Renamed a func * removed duplicate check * Scheduled posts UI fixes (#28828) * Fixed send button and time picker borders * Fixed center alignment of time picker * Removed on for today and tomorrow * Lint fix * Date time modal hover state fix * Badge fix * Fixed a mnerge issue * Scheduled Post send now and add schedule on draft (#28851) * Added send now option on scheduled posts * Minor refactoring * WIP * WIP * WIP * Lint fix * i18n fix * Snapshot update * Review fixes * Scheduled post inline editing (#28893) * Added send now option on scheduled posts * Minor refactoring * WIP * WIP * WIP * Lint fix * i18n fix * Snapshot update * Displayed editing component in scheduled post * Added handling for updating scheduled post * Handle events * Fixed escape key issue in scheudled post editing * Fixes * Displayed error message for editing error * Don't show mention warning * Handled dev mode (#28918) * MInor fixes * client fix * Fixes * CI * Removed dev mode behaviour temperorily (#29008) --------- Co-authored-by: Daniel Espino García <larkox@gmail.com> Co-authored-by: Eva Sarafianou <eva.sarafianou@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2024-11-04 01:09:35 -05:00
return ContainsIntegrationsReservedProps(o.GetProps())
}
func (o *PostPatch) ContainsIntegrationsReservedProps() []string {
if o == nil || o.Props == nil {
return nil
}
Feature scheduled messages (#28932) * Create scheduled post api (#27920) * Added migration files for Postgres * Added migrations for MySQL * Added store method * Added API and store tests * Renamed migration after syncing with master * Added app layer tests * API is ready * API is ready * API is ready * Renamed migration after syncing with master * Updated migration list * Fixed retry layer tests * Allowed posts with empty messages * Review fixes * Reverted an incorrect change * Renamed migration and fixed ID assignment * CI * Send post button changes (#28019) * added Split button * WIP * Added core menu options * WIP * WIP * WIP * Handled displaying error in creating scheduled post * lint fixes * webapp i18n fix * Review fixes * Fixed a webapp test * A few more fixes * Removed a duplicate comment * Scheduled post job (#28088) * Added the job function * Added query for fetching scheduled posts for pricessing * WIP * WIP * WIP * WIP * WIP * WIP * Reafactoring of scheduled post job * Lint fixes * Updated i18n files * FInishing touches * Added tests for GetScheduledPosts * Added tests for PermanentlyDeleteScheduledPosts * Updated all layer * Some changes as discussed with team * Added tests for UpdatedScheduledPost * Code review refactoring * Added job test * MM-60120 - Custom time selection (#28120) * Added a common date time picker modal and used it for post reminder * Added a common date time picker modal and used it for post reminderggp * Added modal for custom schedule time and fixed TZ issue * WIP * Removed event from useSubmit hook * Removed event from useSubmit hook * Added timezone handling * fixed type error * Updated i18n strings * Minor cleanup * updated snapshots * review fixes * Handled event * Supported for having a DM thread open in RHS while in a regular channel * Review fixes * MM-60136 - Scheduled messages tab (#28133) * WIP * WIP * Created Tabs and Tab wrapper with added styling * Added API to get scheduled posts * WIP * Displated scheduled post count * i18n fix * Added tests * Handled asetting active tab absed on URL: * Reverted unintended change * Added API to client ad OpenAPI specs * Renamed file * Adding fileinfo to schedule posts * Partial review fixes * Made get scheduled post API return posts by teamID * review fixes * Moved scheduled post redux code to MM-redux package * Usedd selector factory * WIP: * WIP: * Lint fix * Fixed an incorrect openapi spec file * Removed redundent permission check * Clreaed scheduled post data on logout * Removed unused i18n string: * lint fix * Render scheduled posts (#28208) * WIP * WIP * Created Tabs and Tab wrapper with added styling * Added API to get scheduled posts * WIP * Displated scheduled post count * i18n fix * Added tests * Handled asetting active tab absed on URL: * Reverted unintended change * Added API to client ad OpenAPI specs * Renamed file * Created common component for draft list item * WIP * WIP * Adding fileinfo to schedule posts * Basic rendering * Added count badge to tabs * WIP * Made the Drafts LHS iteam appear if no drafts exist but scheduled posts do * Fixed icon size * Partial review fixes * Made get scheduled post API return posts by teamID * Handled initial vs team switch load * Displayed scheduled date in panel header * Added error message and error indiocator * WIP * review fixes * WIP Adding error reason tag * Added error codes * Moved scheduled post redux code to MM-redux package * Usedd selector factory * WIP: * WIP: * Lint fix * Fixed an incorrect openapi spec file * Removed redundent permission check * Clreaed scheduled post data on logout * Removed unused i18n string: * lint fix * Opened rescheduling modal * Updated graphic for empty state of schduled post list * Added delete scheduled post option and modal * Badge and timezone fix * WIP: * Added send now confirmation modal * lint * Webapp i18n fix * Fixed webapp test * Fixed a bug where DM/GM scheduled posts weren't immideatly showing up in UI * Minor fixes * WIP * Review fixes * Review fixes * Optimisations * Fixed reducer name * Moment optimizatin * Updated route check * MM-60144 - added API to update a scheduled post (#28248) * WIP * Added api and ap layer for update scheduled post ̛̦̄ * Added API to OpenAI specs, Go client and TS client * removed permissio check * Added tests * Fixed tests * Added PreUpdate method on scheduled post model * MM-60131 - Reschedule post integration (#28281) * Handled rescheduling post in webapp * Added error handling * MM-60146 - Delete scheduled post api (#28265) * WIP * Added api and ap layer for update scheduled post ̛̦̄ * Added API to OpenAI specs, Go client and TS client * removed permissio check * Added tests * Fixed tests * Added PreUpdate method on scheduled post model * Added delete scheduled post API * Added API to Go client and OpenAPI specs * Added API to TS client * Added tests * CI * Rmeoved two incorrect code comments * MM-60653 - Integrated delete scheduled post API (#28296) * Integrated delete scheduled apost API * Lint fix * Review fixes * Excluded draft checks from scheduled posts (#28370) * Excluded draft checks from scheduled posts * Added a removed todo * MM-60125 - Scheduled post channel indicator (#28320) * Integrated delete scheduled apost API * Lint fix * Added state for storing scheduled posts by channel ID * Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere * Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere * WIP * Added scheduled post indiocator * Handled single and multiple scheudled posts * Review fixes * Fixed styling and handled center channel, RHS and threads view * Lint fix * i18n fix * Fixed a cycling dependency * Lint fix * Added some more comments * Updated styling * Review fixes * Added common component for remote user time and scheduled post indicator * Updated scheduled post count * Minor change * Moved CSS code around * Fixed a bug where files in scheduled post didn't show up until refresh (#28359) --------- Co-authored-by: Daniel Espino García <larkox@gmail.com> * Scheduled post config (#28485) * Added config * Added config on server and webapp side * Added config check in server and webapp * Added license check * Added license check * Added placeholder help text * Added license check to job * Fixed job test * Review fixes * Updated English text * Review fixes * MM-60118 - Added index on ScheduledPosts table (#28579) * Added index * Updated indexes * Scheduled posts misc fixes (#28625) * Added detailed logging for scheduled post job * Limited scheduled posts processing to 24 hours * Marked old scheduled posts as unable to send * Added t5ests * converted some logs to trace level * Fixed a bug causing error message to show up on deleting a scheduled post in a deleted thread (#28630) * Fixed scheduled posts link in RHS (#28659) * Fixed scheduled posts link in RHS * Review fixes * Fix permission name in scheduled posts by team (#28580) * Fix permission name * fix wording --------- Co-authored-by: Mattermost Build <build@mattermost.com> * FIxed width of generic modal header to fix browser channel modal (#28639) * Only consider error-free scheduled posts for indicator in channel and RHS (#28683) * Show only errro free scheudled posts in post box indicator * Fixed a bug to handle no scheduled posts * Fixed draft and scheudled post UI in mobile view (#28680) * MM-60873 and MM-60872 - Fixed a bug with updating scheduled posts (#28656) * Fixed a bug with updating scheduled posts * Better selectors * MOved shceuled post message length validation to app layer * MM-60732 - Scheduled posts channel link now takes you to the first scheduled post in channel/thread in list (#28768) * Ordered scheudle dposts by schgeudled at nad create at * Ordered in client * Added scroll to target * Removed classname prop * Fixed tests * Added doc * Import fix * MM-60961 - Fixed a bug where API used incoming create at date for scheduled post (#28703) * Fixed a bug where API used incoming create at date for scheduled post * Stopped sending created at value for scheduled post * MM-60785 - Fixed a bug where scheduled posts of channel we are no longer member of didn't show up (#28637) * Fixed a bug where scheduled posts of channel we are no longer member of didn't show up * Added a comment * CI * Used data loader to optimise laoding missing channels * Minor refactoring * MM-60963 - Added common checks for post and scheduled posts (#28713) * Added commen checks for post and scheuled posts * Sanitised scheduled posts * Fixed tests * Splitted post checks into app and context functions * Added checks on scheduiled posts job as well: * i18n fix * Fixed a test * Renamed a func * removed duplicate check * Scheduled posts UI fixes (#28828) * Fixed send button and time picker borders * Fixed center alignment of time picker * Removed on for today and tomorrow * Lint fix * Date time modal hover state fix * Badge fix * Fixed a mnerge issue * Scheduled Post send now and add schedule on draft (#28851) * Added send now option on scheduled posts * Minor refactoring * WIP * WIP * WIP * Lint fix * i18n fix * Snapshot update * Review fixes * Scheduled post inline editing (#28893) * Added send now option on scheduled posts * Minor refactoring * WIP * WIP * WIP * Lint fix * i18n fix * Snapshot update * Displayed editing component in scheduled post * Added handling for updating scheduled post * Handle events * Fixed escape key issue in scheudled post editing * Fixes * Displayed error message for editing error * Don't show mention warning * Handled dev mode (#28918) * MInor fixes * client fix * Fixes * CI * Removed dev mode behaviour temperorily (#29008) --------- Co-authored-by: Daniel Espino García <larkox@gmail.com> Co-authored-by: Eva Sarafianou <eva.sarafianou@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2024-11-04 01:09:35 -05:00
return ContainsIntegrationsReservedProps(*o.Props)
}
Feature scheduled messages (#28932) * Create scheduled post api (#27920) * Added migration files for Postgres * Added migrations for MySQL * Added store method * Added API and store tests * Renamed migration after syncing with master * Added app layer tests * API is ready * API is ready * API is ready * Renamed migration after syncing with master * Updated migration list * Fixed retry layer tests * Allowed posts with empty messages * Review fixes * Reverted an incorrect change * Renamed migration and fixed ID assignment * CI * Send post button changes (#28019) * added Split button * WIP * Added core menu options * WIP * WIP * WIP * Handled displaying error in creating scheduled post * lint fixes * webapp i18n fix * Review fixes * Fixed a webapp test * A few more fixes * Removed a duplicate comment * Scheduled post job (#28088) * Added the job function * Added query for fetching scheduled posts for pricessing * WIP * WIP * WIP * WIP * WIP * WIP * Reafactoring of scheduled post job * Lint fixes * Updated i18n files * FInishing touches * Added tests for GetScheduledPosts * Added tests for PermanentlyDeleteScheduledPosts * Updated all layer * Some changes as discussed with team * Added tests for UpdatedScheduledPost * Code review refactoring * Added job test * MM-60120 - Custom time selection (#28120) * Added a common date time picker modal and used it for post reminder * Added a common date time picker modal and used it for post reminderggp * Added modal for custom schedule time and fixed TZ issue * WIP * Removed event from useSubmit hook * Removed event from useSubmit hook * Added timezone handling * fixed type error * Updated i18n strings * Minor cleanup * updated snapshots * review fixes * Handled event * Supported for having a DM thread open in RHS while in a regular channel * Review fixes * MM-60136 - Scheduled messages tab (#28133) * WIP * WIP * Created Tabs and Tab wrapper with added styling * Added API to get scheduled posts * WIP * Displated scheduled post count * i18n fix * Added tests * Handled asetting active tab absed on URL: * Reverted unintended change * Added API to client ad OpenAPI specs * Renamed file * Adding fileinfo to schedule posts * Partial review fixes * Made get scheduled post API return posts by teamID * review fixes * Moved scheduled post redux code to MM-redux package * Usedd selector factory * WIP: * WIP: * Lint fix * Fixed an incorrect openapi spec file * Removed redundent permission check * Clreaed scheduled post data on logout * Removed unused i18n string: * lint fix * Render scheduled posts (#28208) * WIP * WIP * Created Tabs and Tab wrapper with added styling * Added API to get scheduled posts * WIP * Displated scheduled post count * i18n fix * Added tests * Handled asetting active tab absed on URL: * Reverted unintended change * Added API to client ad OpenAPI specs * Renamed file * Created common component for draft list item * WIP * WIP * Adding fileinfo to schedule posts * Basic rendering * Added count badge to tabs * WIP * Made the Drafts LHS iteam appear if no drafts exist but scheduled posts do * Fixed icon size * Partial review fixes * Made get scheduled post API return posts by teamID * Handled initial vs team switch load * Displayed scheduled date in panel header * Added error message and error indiocator * WIP * review fixes * WIP Adding error reason tag * Added error codes * Moved scheduled post redux code to MM-redux package * Usedd selector factory * WIP: * WIP: * Lint fix * Fixed an incorrect openapi spec file * Removed redundent permission check * Clreaed scheduled post data on logout * Removed unused i18n string: * lint fix * Opened rescheduling modal * Updated graphic for empty state of schduled post list * Added delete scheduled post option and modal * Badge and timezone fix * WIP: * Added send now confirmation modal * lint * Webapp i18n fix * Fixed webapp test * Fixed a bug where DM/GM scheduled posts weren't immideatly showing up in UI * Minor fixes * WIP * Review fixes * Review fixes * Optimisations * Fixed reducer name * Moment optimizatin * Updated route check * MM-60144 - added API to update a scheduled post (#28248) * WIP * Added api and ap layer for update scheduled post ̛̦̄ * Added API to OpenAI specs, Go client and TS client * removed permissio check * Added tests * Fixed tests * Added PreUpdate method on scheduled post model * MM-60131 - Reschedule post integration (#28281) * Handled rescheduling post in webapp * Added error handling * MM-60146 - Delete scheduled post api (#28265) * WIP * Added api and ap layer for update scheduled post ̛̦̄ * Added API to OpenAI specs, Go client and TS client * removed permissio check * Added tests * Fixed tests * Added PreUpdate method on scheduled post model * Added delete scheduled post API * Added API to Go client and OpenAPI specs * Added API to TS client * Added tests * CI * Rmeoved two incorrect code comments * MM-60653 - Integrated delete scheduled post API (#28296) * Integrated delete scheduled apost API * Lint fix * Review fixes * Excluded draft checks from scheduled posts (#28370) * Excluded draft checks from scheduled posts * Added a removed todo * MM-60125 - Scheduled post channel indicator (#28320) * Integrated delete scheduled apost API * Lint fix * Added state for storing scheduled posts by channel ID * Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere * Refactored redux store to store scheudled posts by ID, thens tore IDs everywhere * WIP * Added scheduled post indiocator * Handled single and multiple scheudled posts * Review fixes * Fixed styling and handled center channel, RHS and threads view * Lint fix * i18n fix * Fixed a cycling dependency * Lint fix * Added some more comments * Updated styling * Review fixes * Added common component for remote user time and scheduled post indicator * Updated scheduled post count * Minor change * Moved CSS code around * Fixed a bug where files in scheduled post didn't show up until refresh (#28359) --------- Co-authored-by: Daniel Espino García <larkox@gmail.com> * Scheduled post config (#28485) * Added config * Added config on server and webapp side * Added config check in server and webapp * Added license check * Added license check * Added placeholder help text * Added license check to job * Fixed job test * Review fixes * Updated English text * Review fixes * MM-60118 - Added index on ScheduledPosts table (#28579) * Added index * Updated indexes * Scheduled posts misc fixes (#28625) * Added detailed logging for scheduled post job * Limited scheduled posts processing to 24 hours * Marked old scheduled posts as unable to send * Added t5ests * converted some logs to trace level * Fixed a bug causing error message to show up on deleting a scheduled post in a deleted thread (#28630) * Fixed scheduled posts link in RHS (#28659) * Fixed scheduled posts link in RHS * Review fixes * Fix permission name in scheduled posts by team (#28580) * Fix permission name * fix wording --------- Co-authored-by: Mattermost Build <build@mattermost.com> * FIxed width of generic modal header to fix browser channel modal (#28639) * Only consider error-free scheduled posts for indicator in channel and RHS (#28683) * Show only errro free scheudled posts in post box indicator * Fixed a bug to handle no scheduled posts * Fixed draft and scheudled post UI in mobile view (#28680) * MM-60873 and MM-60872 - Fixed a bug with updating scheduled posts (#28656) * Fixed a bug with updating scheduled posts * Better selectors * MOved shceuled post message length validation to app layer * MM-60732 - Scheduled posts channel link now takes you to the first scheduled post in channel/thread in list (#28768) * Ordered scheudle dposts by schgeudled at nad create at * Ordered in client * Added scroll to target * Removed classname prop * Fixed tests * Added doc * Import fix * MM-60961 - Fixed a bug where API used incoming create at date for scheduled post (#28703) * Fixed a bug where API used incoming create at date for scheduled post * Stopped sending created at value for scheduled post * MM-60785 - Fixed a bug where scheduled posts of channel we are no longer member of didn't show up (#28637) * Fixed a bug where scheduled posts of channel we are no longer member of didn't show up * Added a comment * CI * Used data loader to optimise laoding missing channels * Minor refactoring * MM-60963 - Added common checks for post and scheduled posts (#28713) * Added commen checks for post and scheuled posts * Sanitised scheduled posts * Fixed tests * Splitted post checks into app and context functions * Added checks on scheduiled posts job as well: * i18n fix * Fixed a test * Renamed a func * removed duplicate check * Scheduled posts UI fixes (#28828) * Fixed send button and time picker borders * Fixed center alignment of time picker * Removed on for today and tomorrow * Lint fix * Date time modal hover state fix * Badge fix * Fixed a mnerge issue * Scheduled Post send now and add schedule on draft (#28851) * Added send now option on scheduled posts * Minor refactoring * WIP * WIP * WIP * Lint fix * i18n fix * Snapshot update * Review fixes * Scheduled post inline editing (#28893) * Added send now option on scheduled posts * Minor refactoring * WIP * WIP * WIP * Lint fix * i18n fix * Snapshot update * Displayed editing component in scheduled post * Added handling for updating scheduled post * Handle events * Fixed escape key issue in scheudled post editing * Fixes * Displayed error message for editing error * Don't show mention warning * Handled dev mode (#28918) * MInor fixes * client fix * Fixes * CI * Removed dev mode behaviour temperorily (#29008) --------- Co-authored-by: Daniel Espino García <larkox@gmail.com> Co-authored-by: Eva Sarafianou <eva.sarafianou@gmail.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2024-11-04 01:09:35 -05:00
func ContainsIntegrationsReservedProps(props StringInterface) []string {
foundProps := []string{}
if props != nil {
reservedProps := []string{
PostPropsFromWebhook,
PostPropsOverrideUsername,
PostPropsWebhookDisplayName,
PostPropsOverrideIconURL,
PostPropsOverrideIconEmoji,
}
for _, key := range reservedProps {
if _, ok := props[key]; ok {
foundProps = append(foundProps, key)
}
}
}
return foundProps
}
2015-06-15 03:53:32 -04:00
func (o *Post) PreSave() {
if o.Id == "" {
o.Id = NewId()
}
o.OriginalId = ""
if o.CreateAt == 0 {
o.CreateAt = GetMillis()
}
2015-06-15 03:53:32 -04:00
o.UpdateAt = o.CreateAt
2017-09-19 17:05:20 -04:00
o.PreCommit()
}
2015-06-15 03:53:32 -04:00
2017-09-19 17:05:20 -04:00
func (o *Post) PreCommit() {
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
if o.GetProps() == nil {
o.SetProps(make(map[string]any))
2015-06-15 03:53:32 -04:00
}
if o.Filenames == nil {
o.Filenames = []string{}
}
PLT-3105 Files table migration (#4068) * Implemented initial changes for files table * Removed *_benchmark_test.go files * Re-implemented GetPublicFile and added support for old path * Localization for files table * Moved file system code into utils package * Finished server-side changes and added initial upgrade script * Added getPostFiles api * Re-add Extension and HasPreviewImage fields to FileInfo * Removed unused translation * Fixed merge conflicts left over after permissions changes * Forced FileInfo.extension to be lower case * Changed FileUploadResponse to contain the FileInfos instead of FileIds * Fixed permissions on getFile* calls * Fixed notifications for file uploads * Added initial version of client code for files changes * Permanently added FileIds field to Post object and removed Post.HasFiles * Updated PostStore.Update to be usable in more circumstances * Re-added Filenames field and switched file migration to be entirely lazy-loaded * Increased max listener count for FileStore * Removed unused fileInfoCache * Moved file system code back into api * Removed duplicate test case * Fixed unit test running on ports other than 8065 * Renamed HasPermissionToPostContext to HasPermissionToChannelByPostContext * Refactored handleImages to make it more easily understandable * Renamed getPostFiles to getFileInfosForPost * Re-added pre-FileIds posts to analytics * Changed files to be saved as their ids as opposed to id/filename.ext * Renamed FileInfo.UserId to FileInfo.CreatorId * Fixed detection of language in CodePreview * Fixed switching between threads in the RHS not loading new files * Add serverside protection against a rare bug where the client sends the same file twice for a single post * Refactored the important parts of uploadFile api call into a function that can be called without a web context
2016-09-30 11:06:30 -04:00
if o.FileIds == nil {
o.FileIds = []string{}
}
2017-09-19 17:05:20 -04:00
o.GenerateActionIds()
// There's a rare bug where the client sends up duplicate FileIds so protect against that
o.FileIds = RemoveDuplicateStrings(o.FileIds)
2015-06-15 03:53:32 -04:00
}
func (o *Post) MakeNonNil() {
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
if o.GetProps() == nil {
o.SetProps(make(map[string]any))
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
}
}
func (o *Post) DelProp(key string) {
o.propsMu.Lock()
defer o.propsMu.Unlock()
propsCopy := make(map[string]any, len(o.Props)-1)
maps.Copy(propsCopy, o.Props)
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
delete(propsCopy, key)
o.Props = propsCopy
2015-06-15 03:53:32 -04:00
}
func (o *Post) AddProp(key string, value any) {
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
o.propsMu.Lock()
defer o.propsMu.Unlock()
propsCopy := make(map[string]any, len(o.Props)+1)
maps.Copy(propsCopy, o.Props)
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
propsCopy[key] = value
o.Props = propsCopy
}
2015-06-15 03:53:32 -04:00
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
func (o *Post) GetProps() StringInterface {
o.propsMu.RLock()
defer o.propsMu.RUnlock()
return o.Props
}
func (o *Post) SetProps(props StringInterface) {
o.propsMu.Lock()
defer o.propsMu.Unlock()
o.Props = props
}
2015-06-15 03:53:32 -04:00
func (o *Post) GetProp(key string) any {
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
o.propsMu.RLock()
defer o.propsMu.RUnlock()
return o.Props[key]
2015-06-15 03:53:32 -04:00
}
// ValidateProps checks all known props for validity.
// Currently, it logs warnings for invalid props rather than returning an error.
// In a future version, this will be updated to return errors for invalid props.
func (o *Post) ValidateProps(logger mlog.LoggerIFace) {
if err := o.propsIsValid(); err != nil {
logger.Warn(
"Invalid post props. In a future version this will result in an error. Please update your integration to be compliant.",
mlog.String("post_id", o.Id),
mlog.Err(err),
)
}
}
func (o *Post) propsIsValid() error {
var multiErr *multierror.Error
props := o.GetProps()
// Check basic props validity
if props == nil {
return nil
}
if props[PostPropsAddedUserId] != nil {
if addedUserID, ok := props[PostPropsAddedUserId].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("added_user_id prop must be a string"))
} else if !IsValidId(addedUserID) {
multiErr = multierror.Append(multiErr, fmt.Errorf("added_user_id prop must be a valid user ID"))
}
}
if props[PostPropsDeleteBy] != nil {
if deleteByID, ok := props[PostPropsDeleteBy].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("delete_by prop must be a string"))
} else if !IsValidId(deleteByID) {
multiErr = multierror.Append(multiErr, fmt.Errorf("delete_by prop must be a valid user ID"))
}
}
// Validate integration props
if props[PostPropsOverrideIconURL] != nil {
if iconURL, ok := props[PostPropsOverrideIconURL].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("override_icon_url prop must be a string"))
} else if iconURL == "" || !IsValidHTTPURL(iconURL) {
multiErr = multierror.Append(multiErr, fmt.Errorf("override_icon_url prop must be a valid URL"))
}
}
if props[PostPropsOverrideIconEmoji] != nil {
if _, ok := props[PostPropsOverrideIconEmoji].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("override_icon_emoji prop must be a string"))
}
}
if props[PostPropsOverrideUsername] != nil {
if _, ok := props[PostPropsOverrideUsername].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("override_username prop must be a string"))
}
}
if props[PostPropsFromWebhook] != nil {
if fromWebhook, ok := props[PostPropsFromWebhook].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("from_webhook prop must be a string"))
} else if fromWebhook != "true" {
multiErr = multierror.Append(multiErr, fmt.Errorf("from_webhook prop must be \"true\""))
}
}
if props[PostPropsFromBot] != nil {
if fromBot, ok := props[PostPropsFromBot].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("from_bot prop must be a string"))
} else if fromBot != "true" {
multiErr = multierror.Append(multiErr, fmt.Errorf("from_bot prop must be \"true\""))
}
}
if props[PostPropsFromOAuthApp] != nil {
if fromOAuthApp, ok := props[PostPropsFromOAuthApp].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("from_oauth_app prop must be a string"))
} else if fromOAuthApp != "true" {
multiErr = multierror.Append(multiErr, fmt.Errorf("from_oauth_app prop must be \"true\""))
}
}
if props[PostPropsFromPlugin] != nil {
if fromPlugin, ok := props[PostPropsFromPlugin].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("from_plugin prop must be a string"))
} else if fromPlugin != "true" {
multiErr = multierror.Append(multiErr, fmt.Errorf("from_plugin prop must be \"true\""))
}
}
if props[PostPropsUnsafeLinks] != nil {
if unsafeLinks, ok := props[PostPropsUnsafeLinks].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("unsafe_links prop must be a string"))
} else if unsafeLinks != "true" {
multiErr = multierror.Append(multiErr, fmt.Errorf("unsafe_links prop must be \"true\""))
}
}
if props[PostPropsWebhookDisplayName] != nil {
if _, ok := props[PostPropsWebhookDisplayName].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("webhook_display_name prop must be a string"))
}
}
if props[PostPropsMentionHighlightDisabled] != nil {
if _, ok := props[PostPropsMentionHighlightDisabled].(bool); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("mention_highlight_disabled prop must be a boolean"))
}
}
if props[PostPropsGroupHighlightDisabled] != nil {
if _, ok := props[PostPropsGroupHighlightDisabled].(bool); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("disable_group_highlight prop must be a boolean"))
}
}
if props[PostPropsPreviewedPost] != nil {
if previewedPostID, ok := props[PostPropsPreviewedPost].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("previewed_post prop must be a string"))
} else if !IsValidId(previewedPostID) {
multiErr = multierror.Append(multiErr, fmt.Errorf("previewed_post prop must be a valid post ID"))
}
}
if props[PostPropsForceNotification] != nil {
if _, ok := props[PostPropsForceNotification].(bool); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("force_notification prop must be a boolean"))
}
}
if props[PostPropsAIGeneratedByUserID] != nil {
if aiGenUserID, ok := props[PostPropsAIGeneratedByUserID].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("ai_generated_by prop must be a string"))
} else if !IsValidId(aiGenUserID) {
multiErr = multierror.Append(multiErr, fmt.Errorf("ai_generated_by prop must be a valid user ID"))
}
}
if props[PostPropsAIGeneratedByUsername] != nil {
if _, ok := props[PostPropsAIGeneratedByUsername].(string); !ok {
multiErr = multierror.Append(multiErr, fmt.Errorf("ai_generated_by_username prop must be a string"))
}
}
for i, a := range o.Attachments() {
if err := a.IsValid(); err != nil {
multiErr = multierror.Append(multiErr, multierror.Prefix(err, fmt.Sprintf("message attachtment at index %d is invalid:", i)))
}
}
return multiErr.ErrorOrNil()
}
func (o *Post) IsSystemMessage() bool {
2021-07-12 14:05:36 -04:00
return len(o.Type) >= len(PostSystemMessagePrefix) && o.Type[:len(PostSystemMessagePrefix)] == PostSystemMessagePrefix
}
// IsRemote returns true if the post originated on a remote cluster.
func (o *Post) IsRemote() bool {
return o.RemoteId != nil && *o.RemoteId != ""
}
// GetRemoteID safely returns the remoteID or empty string if not remote.
func (o *Post) GetRemoteID() string {
if o.RemoteId != nil {
return *o.RemoteId
}
return ""
}
func (o *Post) IsJoinLeaveMessage() bool {
2021-07-12 14:05:36 -04:00
return o.Type == PostTypeJoinLeave ||
o.Type == PostTypeAddRemove ||
o.Type == PostTypeJoinChannel ||
o.Type == PostTypeLeaveChannel ||
o.Type == PostTypeJoinTeam ||
o.Type == PostTypeLeaveTeam ||
o.Type == PostTypeAddToChannel ||
o.Type == PostTypeRemoveFromChannel ||
o.Type == PostTypeAddToTeam ||
o.Type == PostTypeRemoveFromTeam
}
func (o *Post) Patch(patch *PostPatch) {
if patch.IsPinned != nil {
o.IsPinned = *patch.IsPinned
}
if patch.Message != nil {
o.Message = *patch.Message
}
if patch.Props != nil {
[MM-21378] Add mutex to model.Post to guard against race conditions on Post.Props (#13884) * Add mutex to model.Post to guard against race conditions on Post.Props * Rename mutex * Add GetProp() method to Post * Fix more tests * Fix flaky test Benchmarks: BenchmarkPostPropsGet_indirect BenchmarkPostPropsGet_indirect-2 85026746 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-4 90273747 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-8 88324293 13.0 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_indirect-16 91427720 13.1 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct BenchmarkPostPropsGet_direct-2 1000000000 0.242 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-4 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-8 1000000000 0.240 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsGet_direct-16 1000000000 0.241 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_indirect BenchmarkPostPropsAdd_indirect-2 5602224 203 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-4 5959496 206 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-8 5833999 205 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_indirect-16 5802493 225 ns/op 336 B/op 2 allocs/op BenchmarkPostPropsAdd_direct BenchmarkPostPropsAdd_direct-2 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-4 100000000 11.3 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-8 100000000 11.6 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsAdd_direct-16 99840794 11.4 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_indirect BenchmarkPostPropsDel_indirect-2 18824002 61.9 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-4 19470736 63.8 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-8 17640460 65.3 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_indirect-16 18692962 65.4 ns/op 48 B/op 1 allocs/op BenchmarkPostPropsDel_direct BenchmarkPostPropsDel_direct-2 516257440 2.34 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-4 514865216 2.43 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-8 511330477 2.37 ns/op 0 B/op 0 allocs/op BenchmarkPostPropsDel_direct-16 499504010 2.38 ns/op 0 B/op 0 allocs/op
2020-03-13 16:12:20 -04:00
newProps := *patch.Props
o.SetProps(newProps)
}
if patch.FileIds != nil {
o.FileIds = *patch.FileIds
}
if patch.HasReactions != nil {
o.HasReactions = *patch.HasReactions
}
}
func (o *Post) ChannelMentions() []string {
return ChannelMentions(o.Message)
}
// ChannelMentionsAll returns all channel mentions from both the message and attachments.
// This is used by FillInPostProps to populate channel_mentions for rendering.
func (o *Post) ChannelMentionsAll() []string {
// Get mentions from message
messageMentions := ChannelMentions(o.Message)
// Get mentions from attachments
attachmentMentions := ChannelMentionsFromAttachments(o.Attachments())
// Combine and deduplicate
alreadyMentioned := make(map[string]bool)
var allMentions []string
for _, mention := range messageMentions {
if !alreadyMentioned[mention] {
allMentions = append(allMentions, mention)
alreadyMentioned[mention] = true
}
}
for _, mention := range attachmentMentions {
if !alreadyMentioned[mention] {
allMentions = append(allMentions, mention)
alreadyMentioned[mention] = true
}
}
return allMentions
}
// DisableMentionHighlights disables a posts mention highlighting and returns the first channel mention that was present in the message.
func (o *Post) DisableMentionHighlights() string {
mention, hasMentions := findAtChannelMention(o.Message)
if hasMentions {
2021-07-12 14:05:36 -04:00
o.AddProp(PostPropsMentionHighlightDisabled, true)
}
return mention
}
// DisableMentionHighlights disables mention highlighting for a post patch if required.
func (o *PostPatch) DisableMentionHighlights() {
if o.Message == nil {
return
}
if _, hasMentions := findAtChannelMention(*o.Message); hasMentions {
if o.Props == nil {
o.Props = &StringInterface{}
}
2021-07-12 14:05:36 -04:00
(*o.Props)[PostPropsMentionHighlightDisabled] = true
}
}
func findAtChannelMention(message string) (mention string, found bool) {
re := regexp.MustCompile(`(?i)\B@(channel|all|here)\b`)
matched := re.FindStringSubmatch(message)
if found = (len(matched) > 0); found {
mention = strings.ToLower(matched[0])
}
return
}
func (o *Post) Attachments() []*MessageAttachment {
if attachments, ok := o.GetProp(PostPropsAttachments).([]*MessageAttachment); ok {
return attachments
}
var ret []*MessageAttachment
if attachments, ok := o.GetProp(PostPropsAttachments).([]any); ok {
for _, attachment := range attachments {
if enc, err := json.Marshal(attachment); err == nil {
var decoded MessageAttachment
if json.Unmarshal(enc, &decoded) == nil {
// Ignoring nil actions
i := 0
for _, action := range decoded.Actions {
if action != nil {
decoded.Actions[i] = action
i++
}
}
decoded.Actions = decoded.Actions[:i]
// Ignoring nil fields
i = 0
for _, field := range decoded.Fields {
if field != nil {
decoded.Fields[i] = field
i++
}
}
decoded.Fields = decoded.Fields[:i]
ret = append(ret, &decoded)
}
}
}
}
return ret
}
func (o *Post) AttachmentsEqual(input *Post) bool {
attachments := o.Attachments()
inputAttachments := input.Attachments()
if len(attachments) != len(inputAttachments) {
return false
}
for i := range attachments {
if !attachments[i].Equals(inputAttachments[i]) {
return false
}
}
return true
}
var markdownDestinationEscaper = strings.NewReplacer(
`\`, `\\`,
`<`, `\<`,
`>`, `\>`,
`(`, `\(`,
`)`, `\)`,
)
// WithRewrittenImageURLs returns a new shallow copy of the post where the message has been
// rewritten via RewriteImageURLs.
func (o *Post) WithRewrittenImageURLs(f func(string) string) *Post {
pCopy := o.Clone()
pCopy.Message = RewriteImageURLs(o.Message, f)
if pCopy.MessageSource == "" && pCopy.Message != o.Message {
pCopy.MessageSource = o.Message
}
return pCopy
}
// RewriteImageURLs takes a message and returns a copy that has all of the image URLs replaced
// according to the function f. For each image URL, f will be invoked, and the resulting markdown
// will contain the URL returned by that invocation instead.
//
// Image URLs are destination URLs used in inline images or reference definitions that are used
// anywhere in the input markdown as an image.
func RewriteImageURLs(message string, f func(string) string) string {
if !strings.Contains(message, "![") {
return message
}
var ranges []markdown.Range
markdown.Inspect(message, func(blockOrInline any) bool {
switch v := blockOrInline.(type) {
case *markdown.ReferenceImage:
ranges = append(ranges, v.ReferenceDefinition.RawDestination)
case *markdown.InlineImage:
ranges = append(ranges, v.RawDestination)
default:
return true
}
return true
})
if ranges == nil {
return message
}
sort.Slice(ranges, func(i, j int) bool {
return ranges[i].Position < ranges[j].Position
})
copyRanges := make([]markdown.Range, 0, len(ranges))
urls := make([]string, 0, len(ranges))
resultLength := len(message)
start := 0
for i, r := range ranges {
switch {
case i == 0:
case r.Position != ranges[i-1].Position:
start = ranges[i-1].End
default:
continue
}
original := message[r.Position:r.End]
replacement := markdownDestinationEscaper.Replace(f(markdown.Unescape(original)))
resultLength += len(replacement) - len(original)
copyRanges = append(copyRanges, markdown.Range{Position: start, End: r.Position})
urls = append(urls, replacement)
}
result := make([]byte, resultLength)
offset := 0
for i, r := range copyRanges {
offset += copy(result[offset:], message[r.Position:r.End])
offset += copy(result[offset:], urls[i])
}
copy(result[offset:], message[ranges[len(ranges)-1].End:])
return string(result)
}
func (o *Post) IsFromOAuthBot() bool {
props := o.GetProps()
return props[PostPropsFromWebhook] == "true" && props[PostPropsOverrideUsername] != ""
}
[MM-30831] granular data retention wireup (#17417) * pre-checkout commit * add API endpoints for retention policies * allow deleting multiple teams/channels from a policy in a single request * pre-checkout commit * add auditing in API functions * add permission checks * update the store layers * update storetest * add check constraint on PostDuration column * pre-checkout commit * add query to delete posts under the scope of a granular retention policy * add suggestions from sbishel * allow clients to specify channels/teams when creating a new policy * remove foreign keys referencing Channels and Teams tables * add checks for whether teams and channels exist * pre-checkout commit * remove data referencing the Posts table * pre-checkout commit * write data store tests * sort results of buildGetPoliciesQuery * add missing test cases for teams * pre-checkout commit * add Client4 methods for data retention policy endpoints * add uint and uint64 to app/layer_generators * make granular policies override global policies * fix lint errors * pre-checkout commit * add license to top of files * add tests for data store layer * add missing test cases for store layer * run make i18n-extract * add query to delete ChannelMemberHistory * work in progress * add test for old reply to old post * fix lint error * use COALESCE on each Posts column * begin implementing orphaned rows worker * split PR * pre-checkout commit * use RetentionPolicyWithTeamAndChannelCounts * update app and api layers * run make i18n-extract * add RetentionPolicy to retrylayer_test.go * Revert "split PR" This reverts commit b316f03dd307a30deae931944ca7e4a1cc904605. * fix errors caused by revert * add suggestions from sbishel * fix copy-paste error * fix lint errors * pre-checkout commit * add function to delete orphaned rows * use -1 for infinite retention * remove check constraint * copy i18n entries from master * re-run tests with newer enterprise branch * add team data to channel list * add search for channels and teams in a policy * add store tests for channel and team search * add suggestions from mkraft * run make einterfaces-mocks * fix lint errors * add suggestions from mkraft * move removeOrphanedRows method to wireup branch * Revert "move removeOrphanedRows method to wireup branch" This reverts commit 94605c9b4a5378ffa44a3dec4d3f8e3306b9d33e. * use DeleteOrphanedRows where possible * run make i18n-extract * use COMPLIANCE permissions * run make migrations-bindadta * clean up teams before test * fix tests for TestRetentionPolicyStore * add API endpoints for mobile * fix lint error * fix some of the lint errors * move user/data_retention endpoints to data_retention.go * Revert "fix some of the lint errors" This reverts commit b5b2dc27566c427187db942c5c0afe319e7679c4. * add exclude_policy_constrained parameter for /channels and /teams * fix lint errors * add policy_id field to GET endpoints for channels and teams * use PolicyWithTeamID in RetentionPolicy layer * fix lint errors * run make i18n-extract * update mock call in telemetry_test.go * return status:OK in JSON instead of 204 * pre-checkout commit * add policy_id field on channels/teams * fix lint errors * use sq.Eq instead of '?' * use new subsection permissions * update channels and teams endpoints to use new subsection permissions * add extra search opts for channels in a policy * fix lint errors * allow negative post duration in patch * remove DELETE FROM query in retention policy tests * use *int64 for PostDuration * re-run CI tests * use 3-step deletion strategy for each table * fix lint errors * run make store-layers * re-run CI tests * add test with channel, team and global policies * use common function for SQL queries * add pagination test * use struct for args to common SQL function * fix lint errors * run make i18n-extract * check if Channels.TeamId is "" or nil * use three OR clauses * write separate genericRetentionPoliciesDeletion function * add config setting for BatchSize * add telemetry for BatchSize * use feature flag * add old i18n messages back in * re-run CI tests * update call signature in storetest * MM-30831: Adds constant for retention default batch size. * MM-30831: Removes comment re: optimization. * MM-30831: Converts days to milliseconds. * MM-30831: Reverts change to test. * Revert "MM-30831: Reverts change to test." This reverts commit 6d14275a1ceae682bb9e17ec69b39252b44e0c0c. * Revert "MM-30831: Converts days to milliseconds." This reverts commit a0cb6ec09d854a05194c1daee1c5333f260231c3. * MM-30831: Fixes tests. * MM-30381: Fix for change to method sig. Co-authored-by: Max Erenberg <max.erenberg@mattermost.com> Co-authored-by: Mattermod <mattermod@users.noreply.github.com> Co-authored-by: Martin Kraft <martin@upspin.org>
2021-06-23 07:55:12 -04:00
func (o *Post) ToNilIfInvalid() *Post {
if o.Id == "" {
return nil
}
return o
}
2021-08-09 11:33:21 -04:00
func (o *Post) ForPlugin() *Post {
p := o.Clone()
p.Metadata = nil
if p.Type == fmt.Sprintf("%sup_notification", PostCustomTypePrefix) {
p.DelProp("requested_features")
}
return p
}
2021-08-09 11:33:21 -04:00
func (o *Post) GetPreviewPost() *PreviewPost {
if o.Metadata == nil {
return nil
}
2021-08-09 11:33:21 -04:00
for _, embed := range o.Metadata.Embeds {
if embed != nil && embed.Type == PostEmbedPermalink {
2021-08-09 11:33:21 -04:00
if previewPost, ok := embed.Data.(*PreviewPost); ok {
return previewPost
}
}
}
return nil
}
func (o *Post) GetPreviewedPostProp() string {
if val, ok := o.GetProp(PostPropsPreviewedPost).(string); ok {
return val
}
return ""
}
func (o *Post) GetPriority() *PostPriority {
[MM-47751][MM-48102] MPA: Send Persistent Notifications (#21619) * MM-46410: adds urgency on mention counts We have introduced priority for posts in https://github.com/mattermost/mattermost-webapp/pull/10951. We do need to color the mention badges in the webapp with a prominent color when a mention is posted in an urgent message. A thread has urgent mentions if the root post is marked as urgent, and the replies contain mentions to the user viewing the thread. This PR adds two columns, urgentmentioncount, and isurgent, in channelmembers, and threads tables respectively. Furthermore when asking for team/thread mention counts, we also return urgent mention counts for the user. * Adds PostAcknowledgements table and apis * job init and fetch mentions * add-migrations * delete-expired * send-notifications * Fetches post priority in batches * stop-notifications * stop-notification-on-reply * MM-47750: Adds PostAcknowledgements table and apis - Adds post acknowledgement api/app/store methods to be able to save and delete post acknowledgements by users. - Adds wesbsocket events for acknowledgement created/deleted - Returns post acknowledgements in the post's metadata * add-license-check * add-pagination * delete on channel and team * validate guests * add configs * move create priority post check from app to api * Add desktop notifications * check status * use config in job * add IsUrgent check * Add last-sent-at * validate max recipients * Update lastSentAt * Validate min. recipient * send email notification only once * remove email notifications * use latest time from config to run job * Add notifications counter * publish events to mentioned users only * pickup license updates in scheduler * don't allow post owner to stop notifications * follow normal notifications behaviour * Validates persistent notifications interval * move logic of handling valid and expired posts into sql * Adds persistent notifications in the webapp --------- Co-authored-by: koox00 <3829551+koox00@users.noreply.github.com> Co-authored-by: Mattermod <mattermod@users.noreply.github.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2023-05-18 14:14:12 -04:00
if o.Metadata == nil {
return nil
}
[MM-47751][MM-48102] MPA: Send Persistent Notifications (#21619) * MM-46410: adds urgency on mention counts We have introduced priority for posts in https://github.com/mattermost/mattermost-webapp/pull/10951. We do need to color the mention badges in the webapp with a prominent color when a mention is posted in an urgent message. A thread has urgent mentions if the root post is marked as urgent, and the replies contain mentions to the user viewing the thread. This PR adds two columns, urgentmentioncount, and isurgent, in channelmembers, and threads tables respectively. Furthermore when asking for team/thread mention counts, we also return urgent mention counts for the user. * Adds PostAcknowledgements table and apis * job init and fetch mentions * add-migrations * delete-expired * send-notifications * Fetches post priority in batches * stop-notifications * stop-notification-on-reply * MM-47750: Adds PostAcknowledgements table and apis - Adds post acknowledgement api/app/store methods to be able to save and delete post acknowledgements by users. - Adds wesbsocket events for acknowledgement created/deleted - Returns post acknowledgements in the post's metadata * add-license-check * add-pagination * delete on channel and team * validate guests * add configs * move create priority post check from app to api * Add desktop notifications * check status * use config in job * add IsUrgent check * Add last-sent-at * validate max recipients * Update lastSentAt * Validate min. recipient * send email notification only once * remove email notifications * use latest time from config to run job * Add notifications counter * publish events to mentioned users only * pickup license updates in scheduler * don't allow post owner to stop notifications * follow normal notifications behaviour * Validates persistent notifications interval * move logic of handling valid and expired posts into sql * Adds persistent notifications in the webapp --------- Co-authored-by: koox00 <3829551+koox00@users.noreply.github.com> Co-authored-by: Mattermod <mattermod@users.noreply.github.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2023-05-18 14:14:12 -04:00
return o.Metadata.Priority
}
[MM-47751][MM-48102] MPA: Send Persistent Notifications (#21619) * MM-46410: adds urgency on mention counts We have introduced priority for posts in https://github.com/mattermost/mattermost-webapp/pull/10951. We do need to color the mention badges in the webapp with a prominent color when a mention is posted in an urgent message. A thread has urgent mentions if the root post is marked as urgent, and the replies contain mentions to the user viewing the thread. This PR adds two columns, urgentmentioncount, and isurgent, in channelmembers, and threads tables respectively. Furthermore when asking for team/thread mention counts, we also return urgent mention counts for the user. * Adds PostAcknowledgements table and apis * job init and fetch mentions * add-migrations * delete-expired * send-notifications * Fetches post priority in batches * stop-notifications * stop-notification-on-reply * MM-47750: Adds PostAcknowledgements table and apis - Adds post acknowledgement api/app/store methods to be able to save and delete post acknowledgements by users. - Adds wesbsocket events for acknowledgement created/deleted - Returns post acknowledgements in the post's metadata * add-license-check * add-pagination * delete on channel and team * validate guests * add configs * move create priority post check from app to api * Add desktop notifications * check status * use config in job * add IsUrgent check * Add last-sent-at * validate max recipients * Update lastSentAt * Validate min. recipient * send email notification only once * remove email notifications * use latest time from config to run job * Add notifications counter * publish events to mentioned users only * pickup license updates in scheduler * don't allow post owner to stop notifications * follow normal notifications behaviour * Validates persistent notifications interval * move logic of handling valid and expired posts into sql * Adds persistent notifications in the webapp --------- Co-authored-by: koox00 <3829551+koox00@users.noreply.github.com> Co-authored-by: Mattermod <mattermod@users.noreply.github.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2023-05-18 14:14:12 -04:00
func (o *Post) GetPersistentNotification() *bool {
priority := o.GetPriority()
if priority == nil {
return nil
}
return priority.PersistentNotifications
}
func (o *Post) GetRequestedAck() *bool {
priority := o.GetPriority()
if priority == nil {
return nil
}
return priority.RequestedAck
}
func (o *Post) IsUrgent() bool {
postPriority := o.GetPriority()
if postPriority == nil {
return false
}
if postPriority.Priority == nil {
return false
}
return *postPriority.Priority == PostPriorityUrgent
}
Feature: Wrangler (#23602) * Migrate feature/wrangler to mono-repo * Add wrangler files * Fix linters, types, etc * Fix snapshots * Fix playwright * Fix pipelines * Fix more pipeline * Fixes for pipelines * More changes for pipeline * Fix types * Add support for a feature flag, but leave it defaulted on for spinwick usage for now * Update snapshot * fix js error when removing last value of multiselect, support CSV marshaling to string array for textsetting * Fix linter * Remove TODO * Remove another TODO * fix tests * Fix i18n * Add server tests * Fix linter * Fix linter * Use proper icon for dot menu * Update snapshot * Add Cypress UI tests for various entrypoints to move thread modal, split SCSS out from forward post into its own thing * clean up * fix linter * More cleanup * Revert files to master * Fix linter for e2e tests * Make ForwardPostChannelSelect channel types configurable with a prop * Add missing return * Fixes from PR feedback * First batch of PR Feedback * Another batch of PR changes * Fix linter * Update snapshots * Wrangler system messages are translated to each user's locale * Initially translate Wrangler into system locale rather than initiating user * More fixes for PR Feedback * Fix some server tests * More updates with master. Fixes around pipelines. Enforce Enterprise license on front/back end * Add tests for dot_menu * More pipeline fixes * Fix e2etests prettier * Update cypress tests, change occurrences of 'Wrangler' with 'Move Thread' * Fix linter * Remove enterprise lock * A couple more occurrences of wrangler strings, and one more enterprise lock * Fix server tests * Fix i18n * Fix e2e linter * Feature flag shouldn't be on by default * Enable move threads feature in smoke tests (#25657) * enable move threads feature * add @prod tag * Fix move_thread_from_public_channel e2e test * Fix e2e style --------- Co-authored-by: Mattermost Build <build@mattermost.com> Co-authored-by: yasserfaraazkhan <attitude3cena.yf@gmail.com>
2023-12-11 15:27:34 -05:00
func (o *Post) CleanPost() *Post {
o.Id = ""
o.CreateAt = 0
o.UpdateAt = 0
o.EditAt = 0
return o
}
Feature edit attachments (#29769) * Updated patch/update post API to allow file modification (#29447) * WIP * WIP * Atatched new files ton post * WIP: deleting removed files * Deleted removed files and invalidated file metadata cache * removed file ignore logif from update post API * Added TestFindExclusives * Added tests for DeleteForPostByIds * Added app layer tests * Added tests * Added API level tests * test enhancements * Fixed a test * Edit history include file metadata (#29505) * Send file metadata in edit history metadata * Added app tests * Added store tests * Added tests for populateEditHistoryFileMetadata{ * Added cache to avoid repetitigve DB calls for edits with only message changes * Added API tests * i18m fix * removed commented code * Improved test helper * Show attachments in edit history RHS (#29519) * Send file metadata in edit history metadata * Added app tests * Added store tests * Added tests for populateEditHistoryFileMetadata{ * Added cache to avoid repetitigve DB calls for edits with only message changes * Added API tests * i18m fix * WIUP: displa files in edit * removed commented code * Displayed file in edit history * Handled file icon * Fixed closing history component on clicking on file * Simplified selector * Simplified selector * Improved test helper * Disabled action menu on edit history file * Added tests * Improved selector * Updated snapshot * review Fixes * restructured componnets * Updated test * Updated test * Restore post api (#29643) * Restore post version API WIP * Undelete files WIP * Added store tests * Created post restore API * Updated updatepost safeUpdate signature * review fixex and improvements * Fixed an app test * Added API laer tests * Added API tests and OpenAPI specs * Fixed a typo * Allow editing files when editing posts (#29709) * WIP - basic view files when editing post * Cleanup * bg color * Added text editor tests for files * WIP * WIP * removed debug log * Allowed admin to add and remove files on someone else's post * Handled drafts and scheduled posts * linter fixes * Updated snapshot * server test fix * CI * Added doc * Restore post api integration (#29719) * WIP - basic view files when editing post * Cleanup * bg color * Added text editor tests for files * WIP * WIP * removed debug log * Allowed admin to add and remove files on someone else's post * Handled drafts and scheduled posts * linter fixes * Updated snapshot * server test fix * Used new API to restore post * handled edut limit and undo * lint fix * added comments * Fixed edit post item tests * Fixed buttons * Aded snapshots * fix test * Updated snapshot * Minor fixes * fixed snapshot * Edit file dnd area (#29763) * dnd wip * DND continued * Supported multiple unbind dragster funcs * lint fixes * Got center channel file drop working when editing a post * file dnd working with center channel and rhs * file dnd working with center channel and rhs * removed unneeded stopPropogation calls * cleanup * DND overlay fix * Lint fix * Advanced text editor test updates for file upload overlay * fixed use upload hook tests * Updated some more snapshots * minor cleanup * Updated i18n * removed need of array for dragster unbind events * lint fixes * edit history cursor * Fixed bugu causing faliure to delete empty posts (#29778) * Files in restore confirmation (#29781) * Added files to restore post confirmation dialog * Fixed post restore toast colors * Fixed restore bug * Fixed restore confirmation toast tests * a11y improvement and modal width fix * Edit attachment misc fixes (#29808) * Removed single image actions in restore post confirmation dialog * Fixed file drop overlay size and position * Made edit indiator accessible * Lint fix * Added bunch of more tests * ANother test migrated from enzyme to react testing library * More test enhancements * More test enhancements * More test enhancements * lint fixes * Fixed a test * Added missing snapshots * Test fixes
2025-01-13 07:46:56 -05:00
type UpdatePostOptions struct {
SafeUpdate bool
IsRestorePost bool
}
func DefaultUpdatePostOptions() *UpdatePostOptions {
return &UpdatePostOptions{
SafeUpdate: false,
IsRestorePost: false,
}
}
Flag post API (#33765) * Added enable/disable setting and feature flag * added rest of notifgication settings * Added backend for content flagging setting and populated notification values from server side defaults * WIP user selector * Added common reviewers UI * Added additonal reviewers section * WIP * WIP * Team table base * Added search in teams * Added search in teams * Added additional settings section * WIP * Inbtegrated reviewers settings * WIP * WIP * Added server side validation * cleanup * cleanup * [skip ci] * Some refactoring * type fixes * lint fix * test: add content flagging settings test file * test: add comprehensive unit tests for content flagging settings * enhanced tests * test: add test file for content flagging additional settings * test: add comprehensive unit tests for ContentFlaggingAdditionalSettingsSection * Added additoonal settings test * test: add empty test file for team reviewers section * test: add comprehensive unit tests for TeamReviewersSection component * test: update tests to handle async data fetching in team reviewers section * test: add empty test file for content reviewers component * feat: add comprehensive unit tests for ContentFlaggingContentReviewers component * Added ContentFlaggingContentReviewersContentFlaggingContentReviewers test * test: add notification settings test file for content flagging * test: add comprehensive unit tests for content flagging notification settings * Added ContentFlaggingNotificationSettingsSection tests * test: add user profile pill test file * test: add comprehensive unit tests for UserProfilePill component * refactor: Replace enzyme shallow with renderWithContext in user_profile_pill tests * Added UserProfilePill tests * test: add empty test file for content reviewers team option * test: add comprehensive unit tests for TeamOptionComponent * Added TeamOptionComponent tests * test: add empty test file for reason_option component * test: add comprehensive unit tests for ReasonOption component * Added ReasonOption tests * cleanup * Fixed i18n error * fixed e2e test lijnt issues * Updated test cases * Added snaoshot * Updated snaoshot * lint fix * WIP * lint fix * Added post flagging properties setup * review fixes * updated snapshot * CI * Added base APIs * Fetched team status data on load and team switch * WIP * Review fixes * wip * WIP * Removed an test, updated comment * CI * Added tests * Added tests * Lint fix * Added API specs * Fixed types * CI fixes * API tests * lint fixes * Set env variable so API routes are regiustered * Test update * term renaming and disabling API tests on MySQL * typo * Updated store type definition * Minor tweaks * Added tests * Removed error in app startup when content flaghging setup fails * Updated sync condition: * Flag message modal basE * added post preview * displaying options * Adde comment input * Updated tests and docs * finction rename * WIP * Updated tests * refactor * lint fix * MOved to data migration * lint fix * CI * added new migration mocks * Used setup for tests * some comment * Removed unnecesseery nil check * Form validation * WIP tests * WIP tests * WIP tests * fix: mock content flagging config selector with correct reasons format Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> * fix: add mock for getContentFlaggingConfig in flag post modal test Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat> * Updated error code order in API docs * removed empty files * Added tests * lint fixes * minor tweak * lint fix * type fix * fixed test * nit * test enhancements * API WIP * API WIP * creating values * creating content flagging channel and properties * Able to save properties * Added another property field * WIP * WIP * Added validations * Added data validations and hidden post if confifgured to * lint fixes * Added API spec * Added some tests * Added tests for getContentReviewBot * test: add comprehensive tests for getContentReviewChannels function * Added more app layer tests * Added TestCanFlagPost * test: Add comprehensive tests for FlagPost function * Added all app layer tests * Removed a file that was reamoved downstream * test: add content flagging test file * test: add comprehensive tests for FlagContentRequest.IsValid method * Added model tests * test: add comprehensive tests for SqlPropertyValueStore.CreateMany * test: add comprehensive tests for flagPost() API function * Added API tests * linter fix * WIP * sent post flagging confirmation message * fixed i18n nissues * fixed i18n nissues * CI * Updated test * fix: reset contentFlaggingGroupId for test isolation in content flagging tests * removed cached group ID * removed debug log * review fixes * Used correct ot name * CI * Updated mobile text * Handled JSON error * fixerdf i18n * CI * Integrate flag post api (#33798) * WIP * WIP * Added API call * test: add test for Client4.flagPost API call in FlagPostModal * fix: remove userEvent.setup() from flag post modal test * test: wrap submit button click in act for proper state updates * Updated tests * lint fix * CI * Updated to allow special characters in comments * Handled empty comment * Used finally * CI * Fixed test * Spillage card integration (#33832) * Created getContentFlaggingFields API * created getPostPropertyValues API * WIP * Created useContentFlaggingFields hook * WIP * WIP * Added option to retain data for reviewers * Displayed deleted post's preview * DIsplayed all properties * Adding field name i18n * WIP - managing i18n able texts * Finished displaying all fields * Manual cleanup * lint fixes * team role filter logic fix * Fixed tests * created new API to fetch flagged posts * lint fix * Added new client methods * test: add comprehensive tests for content flagging APIs * Added new API tests * fixed openapi spec * Fixed DataSpillageReport tests * Fixed PostMarkdown test * Fixed PostPreviewPropertyRenderer test * Added metadata to card renderer * test fixes * Added no comment placeholder * Fixed test * refactor: improve test mocking for data spillage report component * test mock updates * Updated reducer * not resetting mocks * WIP * review fixes * CI * Fixed * fixes * Content flagging actions implementation (#33852) * Added view detail button * Created RemoveFlaggedMessageConfirmationModal modal * Added key and remove flag request modal * IMplemented delete flagged post * Handled edge cases of deleting flagged post * keep message * UI integration * Added WS event for post report update and handled deleted files of flagged post * Added error handling in keep/remove forms * i18n fixes * Updated OpenAPI specs * fixed types * fixed types * refactoring * Fixed tests * review fixes * Added new property translations * Improved test * fixed test * CI * fixes * CI * fixed a test * CI --------- Co-authored-by: aider (anthropic/claude-sonnet-4-20250514) <aider@aider.chat>
2025-10-02 10:54:29 -04:00
type PreparePostForClientOpts struct {
IsNewPost bool
IsEditPost bool
IncludePriority bool
RetainContent bool
IncludeDeleted bool
}
Add cursor-based Posts Reporting API for compliance and auditing (#34252) * Add cursor-based Posts Reporting API for compliance and auditing Implements a new admin-only endpoint for retrieving posts with efficient cursor-based pagination, designed for compliance, auditing, and archival workflows. Key Features: - Cursor-based pagination using composite (time, ID) keys for consistent performance regardless of dataset size (~10ms per page at any depth) - Flexible time range queries with optional upper/lower bounds - Support for both create_at and update_at time fields - Ascending or descending sort order - Optional metadata enrichment (files, reactions, acknowledgements) - System admin only access (requires manage_system permission) - License enforcement for compliance features API Endpoint: POST /api/v4/reports/posts - Request: JSON body with channel_id, cursor_time, cursor_id, and options - Response: Posts map + next_cursor object (null when pagination complete) - Max page size: 1000 posts per request (MaxReportingPerPage constant) Implementation: - Store Layer: Direct SQL queries with composite index on (ChannelId, CreateAt, Id) - App Layer: Permission checks, optional metadata enrichment, post hooks - API Layer: Parameter validation, system admin enforcement, license checks - Data Model: ReportPostOptions, ReportPostOptionsCursor, ReportPostListResponse Code Quality Improvements: - Added MaxReportingPerPage constant (1000) to eliminate magic numbers - Removed unused StartTime field from ReportPostOptions - Added fmt import for dynamic error messages Testing: - 14 comprehensive store layer unit tests - 12 API layer integration tests covering permissions, pagination, filters - All tests passing Documentation: - POSTS_REPORTING.md: Developer reference with Go structs and usage examples - POSTS_REPORTING_API_SPEC.md: Complete technical specification - GET_POSTS_API_IMPROVEMENTS.md: Implementation analysis and design rationale - POSTS_TIME_RANGE_FEATURE.md: Archived time range feature for future use Performance: Cursor-based pagination maintains consistent ~10ms query time at any dataset depth, compared to offset-based pagination which degrades significantly (Page 1 = 10ms, Page 1000 = 10 seconds). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * lint fixes * lint fixes * gofmt * i18n-extract * Add Enterprise license requirement to posts reporting API Enforce Enterprise license (tier 20+) for the new posts reporting endpoint to align with compliance feature licensing. Professional tier is insufficient. Changes: - Add MinimumEnterpriseLicense check in GetPostsForReporting app layer - Add test coverage for license validation (no license and Professional tier) All existing tests pass with new license enforcement. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * i18n-extract * add licensing to api documentation * Test SSH signing * Add mmctl command for posts reporting API Adds mmctl report posts command to retrieve posts from a channel for administrative reporting purposes. Supports cursor-based pagination with configurable sorting, filtering, and time range options. Includes database migration for updateat+id index to support efficient cursor-based queries when sorting by update_at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor posts reporting API cursor to opaque token and improve layer separation This addresses code review feedback by transforming the cursor from exposed fields to an opaque token and improving architectural layer separation. **Key Changes:** 1. **Opaque Cursor Implementation** - Transform cursor from split fields (cursor_time, cursor_id) to single opaque base64-encoded string - Cursor now self-contained with all query parameters embedded - When cursor provided, embedded parameters take precedence over request body - Clients treat cursor as opaque token and pass unchanged 2. **Field Naming** - Rename ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Now excludes ALL system posts (any type starting with "system_") - Clearer and more consistent naming 3. **Layer Separation** - Move cursor decoding from store layer to model layer - Create ReportPostQueryParams struct for resolved parameters - Store layer receives pre-resolved parameters (no business logic) - Add ResolveReportPostQueryParams() function in model layer 4. **Code Quality** - Add type-safe constants (ReportingTimeFieldCreateAt, ReportingSortDirectionAsc, etc.) - Replace magic number 9223372036854775807 with math.MaxInt64 - Remove debug SQL logging (info disclosure risk) - Update mmctl to use constants and fix NextCursor pointer access 5. **Tests** - Update all 17 store test calls to use new resolution pattern - Add comprehensive test for DESC + end_time boundary behavior 6. **API Documentation** - Update OpenAPI spec to reflect opaque cursor format - Update all request/response examples - Clarify end_time behavior with sort directions **Files Changed:** - Model layer: public/model/post.go - App layer: channels/app/report.go - Store layer: channels/store/store.go, channels/store/sqlstore/post_store.go - Tests: channels/store/storetest/post_store.go - Mocks: channels/store/storetest/mocks/PostStore.go - API: channels/api4/report.go, channels/api4/report_test.go - mmctl: cmd/mmctl/commands/report.go - Docs: api/v4/source/reports.yaml 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix unhandled parse errors in cursor decoding Address security finding: cursor decoding was silently ignoring parse errors from strconv functions, which could lead to unexpected behavior when malformed cursors are provided. Changes: - Add explicit error handling for strconv.Atoi (version parsing) - Add explicit error handling for strconv.ParseBool (includeDeleted, excludeSystemPosts) - Add explicit error handling for strconv.ParseInt (timestamp parsing) - Return clear error messages indicating which field failed to parse This prevents silent failures where malformed values would default to zero-values (0, false) and potentially alter query behavior without warning. Addresses DryRun Security finding: "Unhandled Errors in Cursor Parsing" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix linting issues - Remove unused reportPostCursorV1 struct (unused) - Remove obsolete +build comment (buildtag) - Use maps.Copy instead of manual loop (mapsloop) - Modernize for loop with range over int (rangeint) - Apply gofmt formatting 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix gofmt formatting issues Fix alignment in struct literals and constant declarations: - Align map keys in report_test.go request bodies - Align struct fields in ReportPostOptions initialization - Align reporting constant declarations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update mmctl tests for opaque cursor and add i18n translations Update report_test.go to align with the refactored Posts Reporting API: - Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag - Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Update all mock expectations to use new ReportPostOptionsCursor structure - Replace test cursor values with base64-encoded opaque cursor strings Add English translations for cursor decoding error messages in i18n/en.json. Minor API documentation fix in reports.yaml (remove "all" from description). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update mmctl tests for opaque cursor and add i18n translations Update report_test.go to align with the refactored Posts Reporting API: - Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag - Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Update all mock expectations to use new ReportPostOptionsCursor structure - Replace test cursor values with base64-encoded opaque cursor strings Add English translations for cursor decoding error messages in i18n/en.json. Minor API documentation fix in reports.yaml (remove "all" from description). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * more lint fixes * remove index update files * Remove end_time parameter from Posts Reporting API Align with other cursor-based APIs in the codebase by removing the end_time parameter. The caller now controls when to stop pagination by simply not making another request, which is the same pattern used by GetPostsSinceForSync, MessageExport, and GetPostsBatchForIndexing. Changes: - Remove EndTime field from ReportPostOptions and ReportPostQueryParams - Remove EndTime filtering logic from store layer - Remove tests that used end_time parameter 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor posts reporting API for security and validation Address security review feedback by consolidating parameter resolution and validation in the API layer, with comprehensive validation of all cursor fields to prevent SQL injection and invalid queries. Changes: - Move parameter resolution from model to API layer for clearer separation - Add ReportPostQueryParams.Validate() with inline validation for all fields - Validate ChannelId, TimeField, SortDirection, and CursorId format - Add start_time parameter for time-bounded queries - Cap per_page at 100-1000 instead of rejecting invalid values - Export DecodeReportPostCursorV1() for API layer use - Simplify app layer to receive pre-validated parameters - Check channel existence when results are empty (better error messages) Testing: - Add 10 model tests for validation and malformed cursor scenarios - Add 4 API tests for cursors with invalid field values - Refactor 13 store tests to use buildReportPostQueryParams() helper - All 31 tests pass Documentation: - Update OpenAPI spec with start_time, remove unused end_time - Update markdown docs with start_time examples Security improvements: - Whitelist validation prevents SQL injection in TimeField/SortDirection - Format validation ensures ChannelId and CursorId are valid IDs - Single validation point for both cursor and options paths - Defense in depth: validation + parameterized queries + store layer whitelist 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Improve posts reporting query efficiency and safety Replace SELECT * and nested OR/AND conditions with explicit column selection and PostgreSQL row value comparison for better performance and maintainability. Changes: - Use postSliceColumns() instead of SELECT * for explicit column selection - Replace Squirrel OR/AND with row value comparison: (timeField, Id) > (?, ?) - Use fmt.Sprintf for safer string formatting in WHERE clause Query improvements: Before: WHERE (CreateAt > ?) OR (CreateAt = ? AND Id > ?) After: WHERE (CreateAt, Id) > (?, ?) Benefits: - Explicit column selection prevents issues if table schema changes - Row value comparison is more concise and better optimized by PostgreSQL - Follows existing patterns in post_store.go (postSliceColumns) - Standard SQL:2003 syntax 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Change posts reporting response from map to ordered array Replace the Posts map with an ordered array to preserve query sort order and provide a more natural API response for sequential processing. Changes: - ReportPostListResponse.Posts: map[string]*Post → []*Post - Store layer returns posts array directly (already sorted by query) - App layer iterates by index for metadata enrichment - Remove applyPostsWillBeConsumedHook call (not applicable to reporting) - Update API tests to iterate arrays instead of map lookups - Update store tests to convert array to map for deduplication checks - Remove unused "maps" import Benefits: - Preserves query sort order (ASC/DESC, create_at/update_at) - More natural for sequential processing/export workflows - Simpler response structure for reporting/compliance use cases - Aligns with message export/compliance patterns (no plugin hooks) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix linting issues in posts reporting tests Replace inefficient loops with append(...) for better performance. Changes: - Use append(postSlice, result.Posts...) instead of loop - Simplifies code and follows staticcheck recommendations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix store test AppError nil checking Use require.Nil instead of require.NoError for *AppError returns to avoid Go interface nil pointer issues. When DecodeReportPostCursorV1 returns nil *AppError and it's assigned to error interface, the interface becomes non-nil even though the pointer is nil. This causes require.NoError to fail incorrectly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-17 11:02:19 -05:00
// ReportPostOptions contains options for querying posts for reporting/compliance purposes
type ReportPostOptions struct {
ChannelId string `json:"channel_id"`
StartTime int64 `json:"start_time,omitempty"` // Optional: Start time for query range (unix timestamp in milliseconds)
TimeField string `json:"time_field,omitempty"` // "create_at" or "update_at" (default: "create_at")
SortDirection string `json:"sort_direction,omitempty"` // "asc" or "desc" (default: "asc")
PerPage int `json:"per_page,omitempty"` // Number of posts per page (default: 100, max: MaxReportingPerPage)
IncludeDeleted bool `json:"include_deleted,omitempty"` // Include deleted posts
ExcludeSystemPosts bool `json:"exclude_system_posts,omitempty"` // Exclude all system posts (any type starting with "system_")
IncludeMetadata bool `json:"include_metadata,omitempty"` // Include file info, reactions, etc.
}
type RewriteAction string
const (
RewriteActionCustom RewriteAction = "custom"
RewriteActionShorten RewriteAction = "shorten"
RewriteActionElaborate RewriteAction = "elaborate"
RewriteActionImproveWriting RewriteAction = "improve_writing"
RewriteActionFixSpelling RewriteAction = "fix_spelling"
RewriteActionSimplify RewriteAction = "simplify"
RewriteActionSummarize RewriteAction = "summarize"
)
type RewriteRequest struct {
AgentID string `json:"agent_id"`
Message string `json:"message"`
Action RewriteAction `json:"action"`
CustomPrompt string `json:"custom_prompt,omitempty"`
RootID string `json:"root_id,omitempty"`
}
type RewriteResponse struct {
RewrittenText string `json:"rewritten_text"`
}
const RewriteSystemPrompt = `You are a JSON API that rewrites text. Your response must be valid JSON only.
Return this exact format: {"rewritten_text":"content"}.
Do not use markdown, code blocks, or any formatting. Start with { and end with }.`
Add cursor-based Posts Reporting API for compliance and auditing (#34252) * Add cursor-based Posts Reporting API for compliance and auditing Implements a new admin-only endpoint for retrieving posts with efficient cursor-based pagination, designed for compliance, auditing, and archival workflows. Key Features: - Cursor-based pagination using composite (time, ID) keys for consistent performance regardless of dataset size (~10ms per page at any depth) - Flexible time range queries with optional upper/lower bounds - Support for both create_at and update_at time fields - Ascending or descending sort order - Optional metadata enrichment (files, reactions, acknowledgements) - System admin only access (requires manage_system permission) - License enforcement for compliance features API Endpoint: POST /api/v4/reports/posts - Request: JSON body with channel_id, cursor_time, cursor_id, and options - Response: Posts map + next_cursor object (null when pagination complete) - Max page size: 1000 posts per request (MaxReportingPerPage constant) Implementation: - Store Layer: Direct SQL queries with composite index on (ChannelId, CreateAt, Id) - App Layer: Permission checks, optional metadata enrichment, post hooks - API Layer: Parameter validation, system admin enforcement, license checks - Data Model: ReportPostOptions, ReportPostOptionsCursor, ReportPostListResponse Code Quality Improvements: - Added MaxReportingPerPage constant (1000) to eliminate magic numbers - Removed unused StartTime field from ReportPostOptions - Added fmt import for dynamic error messages Testing: - 14 comprehensive store layer unit tests - 12 API layer integration tests covering permissions, pagination, filters - All tests passing Documentation: - POSTS_REPORTING.md: Developer reference with Go structs and usage examples - POSTS_REPORTING_API_SPEC.md: Complete technical specification - GET_POSTS_API_IMPROVEMENTS.md: Implementation analysis and design rationale - POSTS_TIME_RANGE_FEATURE.md: Archived time range feature for future use Performance: Cursor-based pagination maintains consistent ~10ms query time at any dataset depth, compared to offset-based pagination which degrades significantly (Page 1 = 10ms, Page 1000 = 10 seconds). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * lint fixes * lint fixes * gofmt * i18n-extract * Add Enterprise license requirement to posts reporting API Enforce Enterprise license (tier 20+) for the new posts reporting endpoint to align with compliance feature licensing. Professional tier is insufficient. Changes: - Add MinimumEnterpriseLicense check in GetPostsForReporting app layer - Add test coverage for license validation (no license and Professional tier) All existing tests pass with new license enforcement. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * i18n-extract * add licensing to api documentation * Test SSH signing * Add mmctl command for posts reporting API Adds mmctl report posts command to retrieve posts from a channel for administrative reporting purposes. Supports cursor-based pagination with configurable sorting, filtering, and time range options. Includes database migration for updateat+id index to support efficient cursor-based queries when sorting by update_at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor posts reporting API cursor to opaque token and improve layer separation This addresses code review feedback by transforming the cursor from exposed fields to an opaque token and improving architectural layer separation. **Key Changes:** 1. **Opaque Cursor Implementation** - Transform cursor from split fields (cursor_time, cursor_id) to single opaque base64-encoded string - Cursor now self-contained with all query parameters embedded - When cursor provided, embedded parameters take precedence over request body - Clients treat cursor as opaque token and pass unchanged 2. **Field Naming** - Rename ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Now excludes ALL system posts (any type starting with "system_") - Clearer and more consistent naming 3. **Layer Separation** - Move cursor decoding from store layer to model layer - Create ReportPostQueryParams struct for resolved parameters - Store layer receives pre-resolved parameters (no business logic) - Add ResolveReportPostQueryParams() function in model layer 4. **Code Quality** - Add type-safe constants (ReportingTimeFieldCreateAt, ReportingSortDirectionAsc, etc.) - Replace magic number 9223372036854775807 with math.MaxInt64 - Remove debug SQL logging (info disclosure risk) - Update mmctl to use constants and fix NextCursor pointer access 5. **Tests** - Update all 17 store test calls to use new resolution pattern - Add comprehensive test for DESC + end_time boundary behavior 6. **API Documentation** - Update OpenAPI spec to reflect opaque cursor format - Update all request/response examples - Clarify end_time behavior with sort directions **Files Changed:** - Model layer: public/model/post.go - App layer: channels/app/report.go - Store layer: channels/store/store.go, channels/store/sqlstore/post_store.go - Tests: channels/store/storetest/post_store.go - Mocks: channels/store/storetest/mocks/PostStore.go - API: channels/api4/report.go, channels/api4/report_test.go - mmctl: cmd/mmctl/commands/report.go - Docs: api/v4/source/reports.yaml 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix unhandled parse errors in cursor decoding Address security finding: cursor decoding was silently ignoring parse errors from strconv functions, which could lead to unexpected behavior when malformed cursors are provided. Changes: - Add explicit error handling for strconv.Atoi (version parsing) - Add explicit error handling for strconv.ParseBool (includeDeleted, excludeSystemPosts) - Add explicit error handling for strconv.ParseInt (timestamp parsing) - Return clear error messages indicating which field failed to parse This prevents silent failures where malformed values would default to zero-values (0, false) and potentially alter query behavior without warning. Addresses DryRun Security finding: "Unhandled Errors in Cursor Parsing" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix linting issues - Remove unused reportPostCursorV1 struct (unused) - Remove obsolete +build comment (buildtag) - Use maps.Copy instead of manual loop (mapsloop) - Modernize for loop with range over int (rangeint) - Apply gofmt formatting 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix gofmt formatting issues Fix alignment in struct literals and constant declarations: - Align map keys in report_test.go request bodies - Align struct fields in ReportPostOptions initialization - Align reporting constant declarations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update mmctl tests for opaque cursor and add i18n translations Update report_test.go to align with the refactored Posts Reporting API: - Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag - Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Update all mock expectations to use new ReportPostOptionsCursor structure - Replace test cursor values with base64-encoded opaque cursor strings Add English translations for cursor decoding error messages in i18n/en.json. Minor API documentation fix in reports.yaml (remove "all" from description). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Update mmctl tests for opaque cursor and add i18n translations Update report_test.go to align with the refactored Posts Reporting API: - Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag - Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts - Update all mock expectations to use new ReportPostOptionsCursor structure - Replace test cursor values with base64-encoded opaque cursor strings Add English translations for cursor decoding error messages in i18n/en.json. Minor API documentation fix in reports.yaml (remove "all" from description). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * more lint fixes * remove index update files * Remove end_time parameter from Posts Reporting API Align with other cursor-based APIs in the codebase by removing the end_time parameter. The caller now controls when to stop pagination by simply not making another request, which is the same pattern used by GetPostsSinceForSync, MessageExport, and GetPostsBatchForIndexing. Changes: - Remove EndTime field from ReportPostOptions and ReportPostQueryParams - Remove EndTime filtering logic from store layer - Remove tests that used end_time parameter 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor posts reporting API for security and validation Address security review feedback by consolidating parameter resolution and validation in the API layer, with comprehensive validation of all cursor fields to prevent SQL injection and invalid queries. Changes: - Move parameter resolution from model to API layer for clearer separation - Add ReportPostQueryParams.Validate() with inline validation for all fields - Validate ChannelId, TimeField, SortDirection, and CursorId format - Add start_time parameter for time-bounded queries - Cap per_page at 100-1000 instead of rejecting invalid values - Export DecodeReportPostCursorV1() for API layer use - Simplify app layer to receive pre-validated parameters - Check channel existence when results are empty (better error messages) Testing: - Add 10 model tests for validation and malformed cursor scenarios - Add 4 API tests for cursors with invalid field values - Refactor 13 store tests to use buildReportPostQueryParams() helper - All 31 tests pass Documentation: - Update OpenAPI spec with start_time, remove unused end_time - Update markdown docs with start_time examples Security improvements: - Whitelist validation prevents SQL injection in TimeField/SortDirection - Format validation ensures ChannelId and CursorId are valid IDs - Single validation point for both cursor and options paths - Defense in depth: validation + parameterized queries + store layer whitelist 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Improve posts reporting query efficiency and safety Replace SELECT * and nested OR/AND conditions with explicit column selection and PostgreSQL row value comparison for better performance and maintainability. Changes: - Use postSliceColumns() instead of SELECT * for explicit column selection - Replace Squirrel OR/AND with row value comparison: (timeField, Id) > (?, ?) - Use fmt.Sprintf for safer string formatting in WHERE clause Query improvements: Before: WHERE (CreateAt > ?) OR (CreateAt = ? AND Id > ?) After: WHERE (CreateAt, Id) > (?, ?) Benefits: - Explicit column selection prevents issues if table schema changes - Row value comparison is more concise and better optimized by PostgreSQL - Follows existing patterns in post_store.go (postSliceColumns) - Standard SQL:2003 syntax 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Change posts reporting response from map to ordered array Replace the Posts map with an ordered array to preserve query sort order and provide a more natural API response for sequential processing. Changes: - ReportPostListResponse.Posts: map[string]*Post → []*Post - Store layer returns posts array directly (already sorted by query) - App layer iterates by index for metadata enrichment - Remove applyPostsWillBeConsumedHook call (not applicable to reporting) - Update API tests to iterate arrays instead of map lookups - Update store tests to convert array to map for deduplication checks - Remove unused "maps" import Benefits: - Preserves query sort order (ASC/DESC, create_at/update_at) - More natural for sequential processing/export workflows - Simpler response structure for reporting/compliance use cases - Aligns with message export/compliance patterns (no plugin hooks) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix linting issues in posts reporting tests Replace inefficient loops with append(...) for better performance. Changes: - Use append(postSlice, result.Posts...) instead of loop - Simplifies code and follows staticcheck recommendations 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix store test AppError nil checking Use require.Nil instead of require.NoError for *AppError returns to avoid Go interface nil pointer issues. When DecodeReportPostCursorV1 returns nil *AppError and it's assigned to error interface, the interface becomes non-nil even though the pointer is nil. This causes require.NoError to fail incorrectly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-17 11:02:19 -05:00
// ReportPostOptionsCursor contains cursor information for pagination.
// The cursor is an opaque base64-encoded string that encodes all pagination state.
// Clients should treat this as an opaque token and pass it back unchanged.
//
// Internal format (before base64 encoding):
//
// v1: "version:channel_id:time_field:include_deleted:exclude_system_posts:sort_direction:timestamp:post_id"
//
// Field order (general to specific):
// - version: Allows format evolution
// - channel_id: Which channel to query (filter)
// - time_field: Which timestamp column to use for ordering (filter/config)
// - include_deleted: Whether to include deleted posts (filter)
// - exclude_system_posts: Whether to exclude channel metadata system posts (filter)
// - sort_direction: Query direction ASC vs DESC (filter/config)
// - timestamp: The cursor position in time (pagination state)
// - post_id: Tie-breaker for posts with identical timestamps (pagination state)
//
// Version history:
// - v1: Initial format with all query-affecting parameters ordered general→specific, base64-encoded for opacity
// ReportPostOptionsCursor contains the pagination cursor for posts reporting.
//
// The cursor is opaque and self-contained:
// - It's base64-encoded and contains all query parameters (channel_id, time_field, sort_direction, etc.)
// - When a cursor is provided, query parameters in the request body are IGNORED
// - The cursor's embedded parameters take precedence over request body parameters
// - This allows clients to keep sending the same parameters on every page without errors
// - For the first page, omit the cursor field or set it to ""
type ReportPostOptionsCursor struct {
Cursor string `json:"cursor,omitempty"` // Optional: Opaque base64-encoded cursor string (omit or use "" for first request)
}
// ReportPostListResponse contains the response for cursor-based post reporting queries
type ReportPostListResponse struct {
Posts []*Post `json:"posts"`
NextCursor *ReportPostOptionsCursor `json:"next_cursor,omitempty"` // nil if no more pages
}
// ReportPostQueryParams contains the fully resolved query parameters for the store layer.
// This struct is used internally after cursor decoding and parameter resolution.
// The store layer receives these concrete parameters and executes the query.
type ReportPostQueryParams struct {
ChannelId string // Required: Channel to query
CursorTime int64 // Pagination cursor time position
CursorId string // Pagination cursor ID for tie-breaking
TimeField string // Resolved: "create_at" or "update_at"
SortDirection string // Resolved: "asc" or "desc"
IncludeDeleted bool // Resolved: include deleted posts
ExcludeSystemPosts bool // Resolved: exclude system posts
PerPage int // Number of posts per page (already validated)
}
// Validate validates the ReportPostQueryParams fields.
// This should be called after parameter resolution (from cursor or options) and before passing to the store layer.
// Note: PerPage is handled separately in the API layer (capped at 100-1000 range).
func (q *ReportPostQueryParams) Validate() *AppError {
// Validate ChannelId
if !IsValidId(q.ChannelId) {
return NewAppError("ReportPostQueryParams.Validate", "model.post.query_params.invalid_channel_id", nil, "channel_id must be a valid 26-character ID", 400)
}
// Validate TimeField
if q.TimeField != ReportingTimeFieldCreateAt && q.TimeField != ReportingTimeFieldUpdateAt {
return NewAppError("ReportPostQueryParams.Validate", "model.post.query_params.invalid_time_field", nil, fmt.Sprintf("time_field must be %q or %q", ReportingTimeFieldCreateAt, ReportingTimeFieldUpdateAt), 400)
}
// Validate SortDirection
if q.SortDirection != ReportingSortDirectionAsc && q.SortDirection != ReportingSortDirectionDesc {
return NewAppError("ReportPostQueryParams.Validate", "model.post.query_params.invalid_sort_direction", nil, fmt.Sprintf("sort_direction must be %q or %q", ReportingSortDirectionAsc, ReportingSortDirectionDesc), 400)
}
// Validate CursorId - can be empty (first page) or must be a valid ID format (subsequent pages)
if q.CursorId != "" && !IsValidId(q.CursorId) {
return NewAppError("ReportPostQueryParams.Validate", "model.post.query_params.invalid_cursor_id", nil, "cursor_id must be a valid 26-character ID", 400)
}
// CursorTime is validated by the fact it's an int64
// PerPage is handled in API layer before calling Validate()
return nil
}
// EncodeReportPostCursor creates an opaque cursor string from pagination state.
// The cursor encodes all query-affecting parameters to ensure consistency across pages.
// The cursor is base64-encoded to ensure it's truly opaque and URL-safe.
//
// Internal format: "version:channel_id:time_field:include_deleted:exclude_system_posts:sort_direction:timestamp:post_id"
// Example (before encoding): "1:abc123xyz:create_at:false:true:asc:1635724800000:post456def"
func EncodeReportPostCursor(channelId string, timeField string, includeDeleted bool, excludeSystemPosts bool, sortDirection string, timestamp int64, postId string) string {
plainText := fmt.Sprintf("1:%s:%s:%t:%t:%s:%d:%s",
channelId,
timeField,
includeDeleted,
excludeSystemPosts,
sortDirection,
timestamp,
postId)
return base64.URLEncoding.EncodeToString([]byte(plainText))
}
// DecodeReportPostCursorV1 parses an opaque cursor string into query parameters.
// Returns a partially populated ReportPostQueryParams (missing PerPage which comes from the request).
func DecodeReportPostCursorV1(cursor string) (*ReportPostQueryParams, *AppError) {
decoded, err := base64.URLEncoding.DecodeString(cursor)
if err != nil {
return nil, NewAppError("DecodeReportPostCursorV1", "model.post.decode_cursor.invalid_base64", nil, err.Error(), 400)
}
parts := strings.Split(string(decoded), ":")
if len(parts) != 8 {
return nil, NewAppError("DecodeReportPostCursorV1", "model.post.decode_cursor.invalid_format", nil, fmt.Sprintf("expected 8 parts, got %d", len(parts)), 400)
}
version, err := strconv.Atoi(parts[0])
if err != nil {
return nil, NewAppError("DecodeReportPostCursorV1", "model.post.decode_cursor.invalid_version", nil, fmt.Sprintf("version must be an integer: %s", err.Error()), 400)
}
if version != 1 {
return nil, NewAppError("DecodeReportPostCursorV1", "model.post.decode_cursor.unsupported_version", nil, fmt.Sprintf("version %d", version), 400)
}
includeDeleted, err := strconv.ParseBool(parts[3])
if err != nil {
return nil, NewAppError("DecodeReportPostCursorV1", "model.post.decode_cursor.invalid_include_deleted", nil, fmt.Sprintf("include_deleted must be a boolean: %s", err.Error()), 400)
}
excludeSystemPosts, err := strconv.ParseBool(parts[4])
if err != nil {
return nil, NewAppError("DecodeReportPostCursorV1", "model.post.decode_cursor.invalid_exclude_system_posts", nil, fmt.Sprintf("exclude_system_posts must be a boolean: %s", err.Error()), 400)
}
timestamp, err := strconv.ParseInt(parts[6], 10, 64)
if err != nil {
return nil, NewAppError("DecodeReportPostCursorV1", "model.post.decode_cursor.invalid_timestamp", nil, fmt.Sprintf("timestamp must be an integer: %s", err.Error()), 400)
}
return &ReportPostQueryParams{
ChannelId: parts[1],
CursorTime: timestamp,
CursorId: parts[7],
TimeField: parts[2],
SortDirection: parts[5],
IncludeDeleted: includeDeleted,
ExcludeSystemPosts: excludeSystemPosts,
}, nil
}