2023-12-14 10:49:19 -05:00
|
|
|
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
|
|
|
|
// See LICENSE.txt for license information.
|
|
|
|
|
|
|
|
|
|
package api4
|
|
|
|
|
|
|
|
|
|
import (
|
|
|
|
|
"encoding/json"
|
Add cursor-based Posts Reporting API for compliance and auditing (#34252)
* Add cursor-based Posts Reporting API for compliance and auditing
Implements a new admin-only endpoint for retrieving posts with efficient
cursor-based pagination, designed for compliance, auditing, and archival
workflows.
Key Features:
- Cursor-based pagination using composite (time, ID) keys for consistent
performance regardless of dataset size (~10ms per page at any depth)
- Flexible time range queries with optional upper/lower bounds
- Support for both create_at and update_at time fields
- Ascending or descending sort order
- Optional metadata enrichment (files, reactions, acknowledgements)
- System admin only access (requires manage_system permission)
- License enforcement for compliance features
API Endpoint:
POST /api/v4/reports/posts
- Request: JSON body with channel_id, cursor_time, cursor_id, and options
- Response: Posts map + next_cursor object (null when pagination complete)
- Max page size: 1000 posts per request (MaxReportingPerPage constant)
Implementation:
- Store Layer: Direct SQL queries with composite index on (ChannelId, CreateAt, Id)
- App Layer: Permission checks, optional metadata enrichment, post hooks
- API Layer: Parameter validation, system admin enforcement, license checks
- Data Model: ReportPostOptions, ReportPostOptionsCursor, ReportPostListResponse
Code Quality Improvements:
- Added MaxReportingPerPage constant (1000) to eliminate magic numbers
- Removed unused StartTime field from ReportPostOptions
- Added fmt import for dynamic error messages
Testing:
- 14 comprehensive store layer unit tests
- 12 API layer integration tests covering permissions, pagination, filters
- All tests passing
Documentation:
- POSTS_REPORTING.md: Developer reference with Go structs and usage examples
- POSTS_REPORTING_API_SPEC.md: Complete technical specification
- GET_POSTS_API_IMPROVEMENTS.md: Implementation analysis and design rationale
- POSTS_TIME_RANGE_FEATURE.md: Archived time range feature for future use
Performance:
Cursor-based pagination maintains consistent ~10ms query time at any dataset
depth, compared to offset-based pagination which degrades significantly
(Page 1 = 10ms, Page 1000 = 10 seconds).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* lint fixes
* lint fixes
* gofmt
* i18n-extract
* Add Enterprise license requirement to posts reporting API
Enforce Enterprise license (tier 20+) for the new posts reporting endpoint
to align with compliance feature licensing. Professional tier is insufficient.
Changes:
- Add MinimumEnterpriseLicense check in GetPostsForReporting app layer
- Add test coverage for license validation (no license and Professional tier)
All existing tests pass with new license enforcement.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* i18n-extract
* add licensing to api documentation
* Test SSH signing
* Add mmctl command for posts reporting API
Adds mmctl report posts command to retrieve posts from a channel for
administrative reporting purposes. Supports cursor-based pagination with
configurable sorting, filtering, and time range options.
Includes database migration for updateat+id index to support efficient
cursor-based queries when sorting by update_at.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor posts reporting API cursor to opaque token and improve layer separation
This addresses code review feedback by transforming the cursor from exposed fields
to an opaque token and improving architectural layer separation.
**Key Changes:**
1. **Opaque Cursor Implementation**
- Transform cursor from split fields (cursor_time, cursor_id) to single opaque base64-encoded string
- Cursor now self-contained with all query parameters embedded
- When cursor provided, embedded parameters take precedence over request body
- Clients treat cursor as opaque token and pass unchanged
2. **Field Naming**
- Rename ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Now excludes ALL system posts (any type starting with "system_")
- Clearer and more consistent naming
3. **Layer Separation**
- Move cursor decoding from store layer to model layer
- Create ReportPostQueryParams struct for resolved parameters
- Store layer receives pre-resolved parameters (no business logic)
- Add ResolveReportPostQueryParams() function in model layer
4. **Code Quality**
- Add type-safe constants (ReportingTimeFieldCreateAt, ReportingSortDirectionAsc, etc.)
- Replace magic number 9223372036854775807 with math.MaxInt64
- Remove debug SQL logging (info disclosure risk)
- Update mmctl to use constants and fix NextCursor pointer access
5. **Tests**
- Update all 17 store test calls to use new resolution pattern
- Add comprehensive test for DESC + end_time boundary behavior
6. **API Documentation**
- Update OpenAPI spec to reflect opaque cursor format
- Update all request/response examples
- Clarify end_time behavior with sort directions
**Files Changed:**
- Model layer: public/model/post.go
- App layer: channels/app/report.go
- Store layer: channels/store/store.go, channels/store/sqlstore/post_store.go
- Tests: channels/store/storetest/post_store.go
- Mocks: channels/store/storetest/mocks/PostStore.go
- API: channels/api4/report.go, channels/api4/report_test.go
- mmctl: cmd/mmctl/commands/report.go
- Docs: api/v4/source/reports.yaml
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix unhandled parse errors in cursor decoding
Address security finding: cursor decoding was silently ignoring parse errors
from strconv functions, which could lead to unexpected behavior when malformed
cursors are provided.
Changes:
- Add explicit error handling for strconv.Atoi (version parsing)
- Add explicit error handling for strconv.ParseBool (includeDeleted, excludeSystemPosts)
- Add explicit error handling for strconv.ParseInt (timestamp parsing)
- Return clear error messages indicating which field failed to parse
This prevents silent failures where malformed values would default to zero-values
(0, false) and potentially alter query behavior without warning.
Addresses DryRun Security finding: "Unhandled Errors in Cursor Parsing"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix linting issues
- Remove unused reportPostCursorV1 struct (unused)
- Remove obsolete +build comment (buildtag)
- Use maps.Copy instead of manual loop (mapsloop)
- Modernize for loop with range over int (rangeint)
- Apply gofmt formatting
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix gofmt formatting issues
Fix alignment in struct literals and constant declarations:
- Align map keys in report_test.go request bodies
- Align struct fields in ReportPostOptions initialization
- Align reporting constant declarations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update mmctl tests for opaque cursor and add i18n translations
Update report_test.go to align with the refactored Posts Reporting API:
- Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag
- Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Update all mock expectations to use new ReportPostOptionsCursor structure
- Replace test cursor values with base64-encoded opaque cursor strings
Add English translations for cursor decoding error messages in i18n/en.json.
Minor API documentation fix in reports.yaml (remove "all" from description).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update mmctl tests for opaque cursor and add i18n translations
Update report_test.go to align with the refactored Posts Reporting API:
- Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag
- Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Update all mock expectations to use new ReportPostOptionsCursor structure
- Replace test cursor values with base64-encoded opaque cursor strings
Add English translations for cursor decoding error messages in i18n/en.json.
Minor API documentation fix in reports.yaml (remove "all" from description).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* more lint fixes
* remove index update files
* Remove end_time parameter from Posts Reporting API
Align with other cursor-based APIs in the codebase by removing the end_time
parameter. The caller now controls when to stop pagination by simply not
making another request, which is the same pattern used by GetPostsSinceForSync,
MessageExport, and GetPostsBatchForIndexing.
Changes:
- Remove EndTime field from ReportPostOptions and ReportPostQueryParams
- Remove EndTime filtering logic from store layer
- Remove tests that used end_time parameter
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor posts reporting API for security and validation
Address security review feedback by consolidating parameter resolution
and validation in the API layer, with comprehensive validation of all
cursor fields to prevent SQL injection and invalid queries.
Changes:
- Move parameter resolution from model to API layer for clearer separation
- Add ReportPostQueryParams.Validate() with inline validation for all fields
- Validate ChannelId, TimeField, SortDirection, and CursorId format
- Add start_time parameter for time-bounded queries
- Cap per_page at 100-1000 instead of rejecting invalid values
- Export DecodeReportPostCursorV1() for API layer use
- Simplify app layer to receive pre-validated parameters
- Check channel existence when results are empty (better error messages)
Testing:
- Add 10 model tests for validation and malformed cursor scenarios
- Add 4 API tests for cursors with invalid field values
- Refactor 13 store tests to use buildReportPostQueryParams() helper
- All 31 tests pass
Documentation:
- Update OpenAPI spec with start_time, remove unused end_time
- Update markdown docs with start_time examples
Security improvements:
- Whitelist validation prevents SQL injection in TimeField/SortDirection
- Format validation ensures ChannelId and CursorId are valid IDs
- Single validation point for both cursor and options paths
- Defense in depth: validation + parameterized queries + store layer whitelist
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Improve posts reporting query efficiency and safety
Replace SELECT * and nested OR/AND conditions with explicit column
selection and PostgreSQL row value comparison for better performance
and maintainability.
Changes:
- Use postSliceColumns() instead of SELECT * for explicit column selection
- Replace Squirrel OR/AND with row value comparison: (timeField, Id) > (?, ?)
- Use fmt.Sprintf for safer string formatting in WHERE clause
Query improvements:
Before: WHERE (CreateAt > ?) OR (CreateAt = ? AND Id > ?)
After: WHERE (CreateAt, Id) > (?, ?)
Benefits:
- Explicit column selection prevents issues if table schema changes
- Row value comparison is more concise and better optimized by PostgreSQL
- Follows existing patterns in post_store.go (postSliceColumns)
- Standard SQL:2003 syntax
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Change posts reporting response from map to ordered array
Replace the Posts map with an ordered array to preserve query sort order
and provide a more natural API response for sequential processing.
Changes:
- ReportPostListResponse.Posts: map[string]*Post → []*Post
- Store layer returns posts array directly (already sorted by query)
- App layer iterates by index for metadata enrichment
- Remove applyPostsWillBeConsumedHook call (not applicable to reporting)
- Update API tests to iterate arrays instead of map lookups
- Update store tests to convert array to map for deduplication checks
- Remove unused "maps" import
Benefits:
- Preserves query sort order (ASC/DESC, create_at/update_at)
- More natural for sequential processing/export workflows
- Simpler response structure for reporting/compliance use cases
- Aligns with message export/compliance patterns (no plugin hooks)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix linting issues in posts reporting tests
Replace inefficient loops with append(...) for better performance.
Changes:
- Use append(postSlice, result.Posts...) instead of loop
- Simplifies code and follows staticcheck recommendations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix store test AppError nil checking
Use require.Nil instead of require.NoError for *AppError returns
to avoid Go interface nil pointer issues.
When DecodeReportPostCursorV1 returns nil *AppError and it's assigned
to error interface, the interface becomes non-nil even though the
pointer is nil. This causes require.NoError to fail incorrectly.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-17 11:02:19 -05:00
|
|
|
"fmt"
|
|
|
|
|
"math"
|
2023-12-14 10:49:19 -05:00
|
|
|
"net/http"
|
2024-01-10 09:08:23 -05:00
|
|
|
"net/url"
|
2023-12-14 10:49:19 -05:00
|
|
|
"strconv"
|
|
|
|
|
"time"
|
|
|
|
|
|
|
|
|
|
"github.com/mattermost/mattermost/server/public/model"
|
|
|
|
|
"github.com/mattermost/mattermost/server/public/shared/mlog"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
func (api *API) InitReports() {
|
2024-07-15 10:52:03 -04:00
|
|
|
api.BaseRoutes.Reports.Handle("/users", api.APISessionRequired(getUsersForReporting)).Methods(http.MethodGet)
|
|
|
|
|
api.BaseRoutes.Reports.Handle("/users/count", api.APISessionRequired(getUserCountForReporting)).Methods(http.MethodGet)
|
|
|
|
|
api.BaseRoutes.Reports.Handle("/users/export", api.APISessionRequired(startUsersBatchExport)).Methods(http.MethodPost)
|
Add cursor-based Posts Reporting API for compliance and auditing (#34252)
* Add cursor-based Posts Reporting API for compliance and auditing
Implements a new admin-only endpoint for retrieving posts with efficient
cursor-based pagination, designed for compliance, auditing, and archival
workflows.
Key Features:
- Cursor-based pagination using composite (time, ID) keys for consistent
performance regardless of dataset size (~10ms per page at any depth)
- Flexible time range queries with optional upper/lower bounds
- Support for both create_at and update_at time fields
- Ascending or descending sort order
- Optional metadata enrichment (files, reactions, acknowledgements)
- System admin only access (requires manage_system permission)
- License enforcement for compliance features
API Endpoint:
POST /api/v4/reports/posts
- Request: JSON body with channel_id, cursor_time, cursor_id, and options
- Response: Posts map + next_cursor object (null when pagination complete)
- Max page size: 1000 posts per request (MaxReportingPerPage constant)
Implementation:
- Store Layer: Direct SQL queries with composite index on (ChannelId, CreateAt, Id)
- App Layer: Permission checks, optional metadata enrichment, post hooks
- API Layer: Parameter validation, system admin enforcement, license checks
- Data Model: ReportPostOptions, ReportPostOptionsCursor, ReportPostListResponse
Code Quality Improvements:
- Added MaxReportingPerPage constant (1000) to eliminate magic numbers
- Removed unused StartTime field from ReportPostOptions
- Added fmt import for dynamic error messages
Testing:
- 14 comprehensive store layer unit tests
- 12 API layer integration tests covering permissions, pagination, filters
- All tests passing
Documentation:
- POSTS_REPORTING.md: Developer reference with Go structs and usage examples
- POSTS_REPORTING_API_SPEC.md: Complete technical specification
- GET_POSTS_API_IMPROVEMENTS.md: Implementation analysis and design rationale
- POSTS_TIME_RANGE_FEATURE.md: Archived time range feature for future use
Performance:
Cursor-based pagination maintains consistent ~10ms query time at any dataset
depth, compared to offset-based pagination which degrades significantly
(Page 1 = 10ms, Page 1000 = 10 seconds).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* lint fixes
* lint fixes
* gofmt
* i18n-extract
* Add Enterprise license requirement to posts reporting API
Enforce Enterprise license (tier 20+) for the new posts reporting endpoint
to align with compliance feature licensing. Professional tier is insufficient.
Changes:
- Add MinimumEnterpriseLicense check in GetPostsForReporting app layer
- Add test coverage for license validation (no license and Professional tier)
All existing tests pass with new license enforcement.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* i18n-extract
* add licensing to api documentation
* Test SSH signing
* Add mmctl command for posts reporting API
Adds mmctl report posts command to retrieve posts from a channel for
administrative reporting purposes. Supports cursor-based pagination with
configurable sorting, filtering, and time range options.
Includes database migration for updateat+id index to support efficient
cursor-based queries when sorting by update_at.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor posts reporting API cursor to opaque token and improve layer separation
This addresses code review feedback by transforming the cursor from exposed fields
to an opaque token and improving architectural layer separation.
**Key Changes:**
1. **Opaque Cursor Implementation**
- Transform cursor from split fields (cursor_time, cursor_id) to single opaque base64-encoded string
- Cursor now self-contained with all query parameters embedded
- When cursor provided, embedded parameters take precedence over request body
- Clients treat cursor as opaque token and pass unchanged
2. **Field Naming**
- Rename ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Now excludes ALL system posts (any type starting with "system_")
- Clearer and more consistent naming
3. **Layer Separation**
- Move cursor decoding from store layer to model layer
- Create ReportPostQueryParams struct for resolved parameters
- Store layer receives pre-resolved parameters (no business logic)
- Add ResolveReportPostQueryParams() function in model layer
4. **Code Quality**
- Add type-safe constants (ReportingTimeFieldCreateAt, ReportingSortDirectionAsc, etc.)
- Replace magic number 9223372036854775807 with math.MaxInt64
- Remove debug SQL logging (info disclosure risk)
- Update mmctl to use constants and fix NextCursor pointer access
5. **Tests**
- Update all 17 store test calls to use new resolution pattern
- Add comprehensive test for DESC + end_time boundary behavior
6. **API Documentation**
- Update OpenAPI spec to reflect opaque cursor format
- Update all request/response examples
- Clarify end_time behavior with sort directions
**Files Changed:**
- Model layer: public/model/post.go
- App layer: channels/app/report.go
- Store layer: channels/store/store.go, channels/store/sqlstore/post_store.go
- Tests: channels/store/storetest/post_store.go
- Mocks: channels/store/storetest/mocks/PostStore.go
- API: channels/api4/report.go, channels/api4/report_test.go
- mmctl: cmd/mmctl/commands/report.go
- Docs: api/v4/source/reports.yaml
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix unhandled parse errors in cursor decoding
Address security finding: cursor decoding was silently ignoring parse errors
from strconv functions, which could lead to unexpected behavior when malformed
cursors are provided.
Changes:
- Add explicit error handling for strconv.Atoi (version parsing)
- Add explicit error handling for strconv.ParseBool (includeDeleted, excludeSystemPosts)
- Add explicit error handling for strconv.ParseInt (timestamp parsing)
- Return clear error messages indicating which field failed to parse
This prevents silent failures where malformed values would default to zero-values
(0, false) and potentially alter query behavior without warning.
Addresses DryRun Security finding: "Unhandled Errors in Cursor Parsing"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix linting issues
- Remove unused reportPostCursorV1 struct (unused)
- Remove obsolete +build comment (buildtag)
- Use maps.Copy instead of manual loop (mapsloop)
- Modernize for loop with range over int (rangeint)
- Apply gofmt formatting
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix gofmt formatting issues
Fix alignment in struct literals and constant declarations:
- Align map keys in report_test.go request bodies
- Align struct fields in ReportPostOptions initialization
- Align reporting constant declarations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update mmctl tests for opaque cursor and add i18n translations
Update report_test.go to align with the refactored Posts Reporting API:
- Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag
- Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Update all mock expectations to use new ReportPostOptionsCursor structure
- Replace test cursor values with base64-encoded opaque cursor strings
Add English translations for cursor decoding error messages in i18n/en.json.
Minor API documentation fix in reports.yaml (remove "all" from description).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update mmctl tests for opaque cursor and add i18n translations
Update report_test.go to align with the refactored Posts Reporting API:
- Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag
- Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Update all mock expectations to use new ReportPostOptionsCursor structure
- Replace test cursor values with base64-encoded opaque cursor strings
Add English translations for cursor decoding error messages in i18n/en.json.
Minor API documentation fix in reports.yaml (remove "all" from description).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* more lint fixes
* remove index update files
* Remove end_time parameter from Posts Reporting API
Align with other cursor-based APIs in the codebase by removing the end_time
parameter. The caller now controls when to stop pagination by simply not
making another request, which is the same pattern used by GetPostsSinceForSync,
MessageExport, and GetPostsBatchForIndexing.
Changes:
- Remove EndTime field from ReportPostOptions and ReportPostQueryParams
- Remove EndTime filtering logic from store layer
- Remove tests that used end_time parameter
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor posts reporting API for security and validation
Address security review feedback by consolidating parameter resolution
and validation in the API layer, with comprehensive validation of all
cursor fields to prevent SQL injection and invalid queries.
Changes:
- Move parameter resolution from model to API layer for clearer separation
- Add ReportPostQueryParams.Validate() with inline validation for all fields
- Validate ChannelId, TimeField, SortDirection, and CursorId format
- Add start_time parameter for time-bounded queries
- Cap per_page at 100-1000 instead of rejecting invalid values
- Export DecodeReportPostCursorV1() for API layer use
- Simplify app layer to receive pre-validated parameters
- Check channel existence when results are empty (better error messages)
Testing:
- Add 10 model tests for validation and malformed cursor scenarios
- Add 4 API tests for cursors with invalid field values
- Refactor 13 store tests to use buildReportPostQueryParams() helper
- All 31 tests pass
Documentation:
- Update OpenAPI spec with start_time, remove unused end_time
- Update markdown docs with start_time examples
Security improvements:
- Whitelist validation prevents SQL injection in TimeField/SortDirection
- Format validation ensures ChannelId and CursorId are valid IDs
- Single validation point for both cursor and options paths
- Defense in depth: validation + parameterized queries + store layer whitelist
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Improve posts reporting query efficiency and safety
Replace SELECT * and nested OR/AND conditions with explicit column
selection and PostgreSQL row value comparison for better performance
and maintainability.
Changes:
- Use postSliceColumns() instead of SELECT * for explicit column selection
- Replace Squirrel OR/AND with row value comparison: (timeField, Id) > (?, ?)
- Use fmt.Sprintf for safer string formatting in WHERE clause
Query improvements:
Before: WHERE (CreateAt > ?) OR (CreateAt = ? AND Id > ?)
After: WHERE (CreateAt, Id) > (?, ?)
Benefits:
- Explicit column selection prevents issues if table schema changes
- Row value comparison is more concise and better optimized by PostgreSQL
- Follows existing patterns in post_store.go (postSliceColumns)
- Standard SQL:2003 syntax
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Change posts reporting response from map to ordered array
Replace the Posts map with an ordered array to preserve query sort order
and provide a more natural API response for sequential processing.
Changes:
- ReportPostListResponse.Posts: map[string]*Post → []*Post
- Store layer returns posts array directly (already sorted by query)
- App layer iterates by index for metadata enrichment
- Remove applyPostsWillBeConsumedHook call (not applicable to reporting)
- Update API tests to iterate arrays instead of map lookups
- Update store tests to convert array to map for deduplication checks
- Remove unused "maps" import
Benefits:
- Preserves query sort order (ASC/DESC, create_at/update_at)
- More natural for sequential processing/export workflows
- Simpler response structure for reporting/compliance use cases
- Aligns with message export/compliance patterns (no plugin hooks)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix linting issues in posts reporting tests
Replace inefficient loops with append(...) for better performance.
Changes:
- Use append(postSlice, result.Posts...) instead of loop
- Simplifies code and follows staticcheck recommendations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix store test AppError nil checking
Use require.Nil instead of require.NoError for *AppError returns
to avoid Go interface nil pointer issues.
When DecodeReportPostCursorV1 returns nil *AppError and it's assigned
to error interface, the interface becomes non-nil even though the
pointer is nil. This causes require.NoError to fail incorrectly.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-17 11:02:19 -05:00
|
|
|
api.BaseRoutes.Reports.Handle("/posts", api.APISessionRequired(getPostsForReporting)).Methods(http.MethodPost)
|
2023-12-14 10:49:19 -05:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func getUsersForReporting(c *Context, w http.ResponseWriter, r *http.Request) {
|
2024-07-12 00:52:04 -04:00
|
|
|
if !c.App.SessionHasPermissionTo(*c.AppContext.Session(), model.PermissionSysconsoleReadUserManagementUsers) {
|
2023-12-14 10:49:19 -05:00
|
|
|
c.SetPermissionError(model.PermissionSysconsoleReadUserManagementUsers)
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
2024-01-10 09:08:23 -05:00
|
|
|
baseOptions := fillReportingBaseOptions(r.URL.Query())
|
|
|
|
|
options, err := fillUserReportOptions(r.URL.Query())
|
2024-10-02 06:23:39 -04:00
|
|
|
|
2024-01-10 09:08:23 -05:00
|
|
|
if err != nil {
|
|
|
|
|
c.Err = err
|
|
|
|
|
return
|
2023-12-17 19:26:06 -05:00
|
|
|
}
|
2024-01-10 09:08:23 -05:00
|
|
|
options.ReportingBaseOptions = baseOptions
|
2023-12-17 19:26:06 -05:00
|
|
|
|
2024-01-10 09:08:23 -05:00
|
|
|
// Don't allow fetching more than 100 users at a time from the normal query endpoint
|
|
|
|
|
if options.PageSize <= 0 || options.PageSize > model.ReportingMaxPageSize {
|
|
|
|
|
c.Err = model.NewAppError("getUsersForReporting", "api.getUsersForReporting.invalid_page_size", nil, "", http.StatusBadRequest)
|
|
|
|
|
return
|
2023-12-14 10:49:19 -05:00
|
|
|
}
|
|
|
|
|
|
2024-01-10 09:08:23 -05:00
|
|
|
userReports, err := c.App.GetUsersForReporting(options)
|
|
|
|
|
if err != nil {
|
|
|
|
|
c.Err = err
|
2023-12-14 10:49:19 -05:00
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
2024-01-10 09:08:23 -05:00
|
|
|
if jsonErr := json.NewEncoder(w).Encode(userReports); jsonErr != nil {
|
|
|
|
|
c.Logger.Warn("Error writing response", mlog.Err(jsonErr))
|
2023-12-14 10:49:19 -05:00
|
|
|
}
|
2024-01-10 09:08:23 -05:00
|
|
|
}
|
2023-12-14 10:49:19 -05:00
|
|
|
|
2024-01-10 09:08:23 -05:00
|
|
|
func getUserCountForReporting(c *Context, w http.ResponseWriter, r *http.Request) {
|
2024-07-12 00:52:04 -04:00
|
|
|
if !c.App.SessionHasPermissionTo(*c.AppContext.Session(), model.PermissionSysconsoleReadUserManagementUsers) {
|
2024-01-10 09:08:23 -05:00
|
|
|
c.SetPermissionError(model.PermissionSysconsoleReadUserManagementUsers)
|
|
|
|
|
return
|
2023-12-14 10:49:19 -05:00
|
|
|
}
|
|
|
|
|
|
2024-01-10 09:08:23 -05:00
|
|
|
options, err := fillUserReportOptions(r.URL.Query())
|
|
|
|
|
if err != nil {
|
|
|
|
|
c.Err = err
|
2023-12-14 10:49:19 -05:00
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
2024-01-10 09:08:23 -05:00
|
|
|
count, err := c.App.GetUserCountForReport(options)
|
2023-12-14 10:49:19 -05:00
|
|
|
if err != nil {
|
|
|
|
|
c.Err = err
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
2024-01-10 09:08:23 -05:00
|
|
|
if jsonErr := json.NewEncoder(w).Encode(count); jsonErr != nil {
|
2023-12-14 10:49:19 -05:00
|
|
|
c.Logger.Warn("Error writing response", mlog.Err(jsonErr))
|
|
|
|
|
}
|
|
|
|
|
}
|
2024-01-10 09:08:23 -05:00
|
|
|
|
2024-01-19 15:22:17 -05:00
|
|
|
func startUsersBatchExport(c *Context, w http.ResponseWriter, r *http.Request) {
|
|
|
|
|
if !(c.IsSystemAdmin()) {
|
2024-07-12 01:43:09 -04:00
|
|
|
c.SetPermissionError(model.PermissionManageSystem)
|
2024-01-19 15:22:17 -05:00
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
2024-10-02 06:23:39 -04:00
|
|
|
baseOptions := fillReportingBaseOptions(r.URL.Query())
|
|
|
|
|
options, err := fillUserReportOptions(r.URL.Query())
|
|
|
|
|
|
|
|
|
|
if err != nil {
|
|
|
|
|
c.Err = err
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
options.ReportingBaseOptions = baseOptions
|
|
|
|
|
dateRange := options.ReportingBaseOptions.DateRange
|
2024-01-29 09:52:33 -05:00
|
|
|
if dateRange == "" {
|
|
|
|
|
dateRange = "all_time"
|
2024-01-19 15:22:17 -05:00
|
|
|
}
|
|
|
|
|
|
2024-01-29 09:52:33 -05:00
|
|
|
startAt, endAt := model.GetReportDateRange(dateRange, time.Now())
|
2024-10-02 06:23:39 -04:00
|
|
|
if err := c.App.StartUsersBatchExport(c.AppContext, options, startAt, endAt); err != nil {
|
2024-01-19 15:22:17 -05:00
|
|
|
c.Err = err
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
2024-01-29 09:52:33 -05:00
|
|
|
ReturnStatusOK(w)
|
2024-01-19 15:22:17 -05:00
|
|
|
}
|
|
|
|
|
|
2024-01-10 09:08:23 -05:00
|
|
|
func fillReportingBaseOptions(values url.Values) model.ReportingBaseOptions {
|
|
|
|
|
sortColumn := "Username"
|
|
|
|
|
if values.Get("sort_column") != "" {
|
|
|
|
|
sortColumn = values.Get("sort_column")
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
direction := "next"
|
|
|
|
|
if values.Get("direction") == "prev" {
|
|
|
|
|
direction = "prev"
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pageSize := 50
|
|
|
|
|
if pageSizeStr, err := strconv.ParseInt(values.Get("page_size"), 10, 64); err == nil {
|
|
|
|
|
pageSize = int(pageSizeStr)
|
|
|
|
|
}
|
|
|
|
|
|
2024-01-18 14:06:30 -05:00
|
|
|
options := model.ReportingBaseOptions{
|
2024-01-10 09:08:23 -05:00
|
|
|
Direction: direction,
|
|
|
|
|
SortColumn: sortColumn,
|
|
|
|
|
SortDesc: values.Get("sort_direction") == "desc",
|
|
|
|
|
PageSize: pageSize,
|
|
|
|
|
FromColumnValue: values.Get("from_column_value"),
|
|
|
|
|
FromId: values.Get("from_id"),
|
|
|
|
|
DateRange: values.Get("date_range"),
|
|
|
|
|
}
|
2024-01-18 14:06:30 -05:00
|
|
|
options.PopulateDateRange(time.Now())
|
|
|
|
|
return options
|
2024-01-10 09:08:23 -05:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func fillUserReportOptions(values url.Values) (*model.UserReportOptions, *model.AppError) {
|
|
|
|
|
teamFilter := values.Get("team_filter")
|
|
|
|
|
if !(teamFilter == "" || model.IsValidId(teamFilter)) {
|
|
|
|
|
return nil, model.NewAppError("getUsersForReporting", "api.getUsersForReporting.invalid_team_filter", nil, "", http.StatusBadRequest)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
hideActive := values.Get("hide_active") == "true"
|
|
|
|
|
hideInactive := values.Get("hide_inactive") == "true"
|
|
|
|
|
if hideActive && hideInactive {
|
|
|
|
|
return nil, model.NewAppError("getUsersForReporting", "api.getUsersForReporting.invalid_active_filter", nil, "", http.StatusBadRequest)
|
|
|
|
|
}
|
|
|
|
|
|
2024-01-18 14:06:30 -05:00
|
|
|
return &model.UserReportOptions{
|
2024-01-10 09:08:23 -05:00
|
|
|
Team: teamFilter,
|
|
|
|
|
Role: values.Get("role_filter"),
|
|
|
|
|
HasNoTeam: values.Get("has_no_team") == "true",
|
|
|
|
|
HideActive: hideActive,
|
|
|
|
|
HideInactive: hideInactive,
|
|
|
|
|
SearchTerm: values.Get("search_term"),
|
Add single-channel guests filter and channel count column to System Console Users (#35517)
* Add single-channel guests filter and channel count column to System Console Users
- Add guest_filter query parameter to Reports API with store-level
filtering by guest channel membership count (all, single_channel,
multi_channel)
- Add channel_count field to user report responses and CSV exports
- Add grouped guest role filter options in the filter popover
- Add toggleable Channel count column to the users table
- Add GuestFilter and SearchTerm to Go client GetUsersForReporting
- Add tests: API parsing, API integration, app job dedup, webapp utils,
E2E column data rendering
Made-with: Cursor
* Fix gofmt alignment and isolate guest store tests
- Align GuestFilter constants to satisfy gofmt
- Move guest user/channel setup into a nested sub-test to avoid
breaking existing ordering and role filter assertions
Made-with: Cursor
* Exclude archived channels from guest filter queries and ChannelCount
The ChannelMembers subqueries for guest_filter (single/multi channel)
and the ChannelCount column did not join with Channels to check
DeleteAt = 0. Since channel archival soft-deletes (sets DeleteAt) but
leaves ChannelMembers rows intact, archived channel memberships were
incorrectly counted, potentially misclassifying guests between
single-channel and multi-channel filters and inflating ChannelCount.
- Join ChannelMembers with Channels (DeleteAt = 0) in all three
subqueries in applyUserReportFilter and GetUserReport
- Add store test covering archived channel exclusion
- Tighten existing guest filter test assertions with found-flags
and exact count checks
Made-with: Cursor
* Exclude DM/GM from guest channel counts, validate GuestFilter, fix dropdown divider
- Scope ChannelCount and guest filter subqueries to Open/Private channel
types only (exclude DM and GM), so a guest with one team channel plus
a DM is correctly classified as single-channel
- Add GuestFilter validation in UserReportOptions.IsValid with
AllowedGuestFilters whitelist
- Add API test for invalid guest_filter rejection (400)
- Add store regression test for DM/GM exclusion
- Fix role filter dropdown: hide the divider above the first group
heading via CSS rule on DropDown__group:first-child
- Update E2E test label to match "Guests in a single channel" wording
Made-with: Cursor
* Add store test coverage for private and GM channel types
Private channels (type P) should be counted in ChannelCount and guest
filters, while GM channels (type G) should not. Add a test that creates
a guest with memberships in an open channel, a private channel, and a
GM, then asserts ChannelCount = 2, multi-channel filter includes the
guest, and single-channel filter excludes them.
Made-with: Cursor
* Add server i18n translation for invalid_guest_filter error
The new error ID model.user_report_options.is_valid.invalid_guest_filter
was missing from server/i18n/en.json, causing CI to fail.
Made-with: Cursor
* Make filter dropdown dividers full width
Remove the horizontal inset from grouped dropdown separators so the
system user role filter dividers span edge to edge across the menu.
Leave the unrelated webapp/package-lock.json change uncommitted.
Made-with: Cursor
* Optimize guest channel report filters.
Use per-user channel count subqueries for the single- and multi-channel guest filters so the report avoids aggregating all channel memberships before filtering guests.
2026-03-12 12:50:53 -04:00
|
|
|
GuestFilter: values.Get("guest_filter"),
|
2024-01-18 14:06:30 -05:00
|
|
|
}, nil
|
2024-01-10 09:08:23 -05:00
|
|
|
}
|
Add cursor-based Posts Reporting API for compliance and auditing (#34252)
* Add cursor-based Posts Reporting API for compliance and auditing
Implements a new admin-only endpoint for retrieving posts with efficient
cursor-based pagination, designed for compliance, auditing, and archival
workflows.
Key Features:
- Cursor-based pagination using composite (time, ID) keys for consistent
performance regardless of dataset size (~10ms per page at any depth)
- Flexible time range queries with optional upper/lower bounds
- Support for both create_at and update_at time fields
- Ascending or descending sort order
- Optional metadata enrichment (files, reactions, acknowledgements)
- System admin only access (requires manage_system permission)
- License enforcement for compliance features
API Endpoint:
POST /api/v4/reports/posts
- Request: JSON body with channel_id, cursor_time, cursor_id, and options
- Response: Posts map + next_cursor object (null when pagination complete)
- Max page size: 1000 posts per request (MaxReportingPerPage constant)
Implementation:
- Store Layer: Direct SQL queries with composite index on (ChannelId, CreateAt, Id)
- App Layer: Permission checks, optional metadata enrichment, post hooks
- API Layer: Parameter validation, system admin enforcement, license checks
- Data Model: ReportPostOptions, ReportPostOptionsCursor, ReportPostListResponse
Code Quality Improvements:
- Added MaxReportingPerPage constant (1000) to eliminate magic numbers
- Removed unused StartTime field from ReportPostOptions
- Added fmt import for dynamic error messages
Testing:
- 14 comprehensive store layer unit tests
- 12 API layer integration tests covering permissions, pagination, filters
- All tests passing
Documentation:
- POSTS_REPORTING.md: Developer reference with Go structs and usage examples
- POSTS_REPORTING_API_SPEC.md: Complete technical specification
- GET_POSTS_API_IMPROVEMENTS.md: Implementation analysis and design rationale
- POSTS_TIME_RANGE_FEATURE.md: Archived time range feature for future use
Performance:
Cursor-based pagination maintains consistent ~10ms query time at any dataset
depth, compared to offset-based pagination which degrades significantly
(Page 1 = 10ms, Page 1000 = 10 seconds).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* lint fixes
* lint fixes
* gofmt
* i18n-extract
* Add Enterprise license requirement to posts reporting API
Enforce Enterprise license (tier 20+) for the new posts reporting endpoint
to align with compliance feature licensing. Professional tier is insufficient.
Changes:
- Add MinimumEnterpriseLicense check in GetPostsForReporting app layer
- Add test coverage for license validation (no license and Professional tier)
All existing tests pass with new license enforcement.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* i18n-extract
* add licensing to api documentation
* Test SSH signing
* Add mmctl command for posts reporting API
Adds mmctl report posts command to retrieve posts from a channel for
administrative reporting purposes. Supports cursor-based pagination with
configurable sorting, filtering, and time range options.
Includes database migration for updateat+id index to support efficient
cursor-based queries when sorting by update_at.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor posts reporting API cursor to opaque token and improve layer separation
This addresses code review feedback by transforming the cursor from exposed fields
to an opaque token and improving architectural layer separation.
**Key Changes:**
1. **Opaque Cursor Implementation**
- Transform cursor from split fields (cursor_time, cursor_id) to single opaque base64-encoded string
- Cursor now self-contained with all query parameters embedded
- When cursor provided, embedded parameters take precedence over request body
- Clients treat cursor as opaque token and pass unchanged
2. **Field Naming**
- Rename ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Now excludes ALL system posts (any type starting with "system_")
- Clearer and more consistent naming
3. **Layer Separation**
- Move cursor decoding from store layer to model layer
- Create ReportPostQueryParams struct for resolved parameters
- Store layer receives pre-resolved parameters (no business logic)
- Add ResolveReportPostQueryParams() function in model layer
4. **Code Quality**
- Add type-safe constants (ReportingTimeFieldCreateAt, ReportingSortDirectionAsc, etc.)
- Replace magic number 9223372036854775807 with math.MaxInt64
- Remove debug SQL logging (info disclosure risk)
- Update mmctl to use constants and fix NextCursor pointer access
5. **Tests**
- Update all 17 store test calls to use new resolution pattern
- Add comprehensive test for DESC + end_time boundary behavior
6. **API Documentation**
- Update OpenAPI spec to reflect opaque cursor format
- Update all request/response examples
- Clarify end_time behavior with sort directions
**Files Changed:**
- Model layer: public/model/post.go
- App layer: channels/app/report.go
- Store layer: channels/store/store.go, channels/store/sqlstore/post_store.go
- Tests: channels/store/storetest/post_store.go
- Mocks: channels/store/storetest/mocks/PostStore.go
- API: channels/api4/report.go, channels/api4/report_test.go
- mmctl: cmd/mmctl/commands/report.go
- Docs: api/v4/source/reports.yaml
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix unhandled parse errors in cursor decoding
Address security finding: cursor decoding was silently ignoring parse errors
from strconv functions, which could lead to unexpected behavior when malformed
cursors are provided.
Changes:
- Add explicit error handling for strconv.Atoi (version parsing)
- Add explicit error handling for strconv.ParseBool (includeDeleted, excludeSystemPosts)
- Add explicit error handling for strconv.ParseInt (timestamp parsing)
- Return clear error messages indicating which field failed to parse
This prevents silent failures where malformed values would default to zero-values
(0, false) and potentially alter query behavior without warning.
Addresses DryRun Security finding: "Unhandled Errors in Cursor Parsing"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix linting issues
- Remove unused reportPostCursorV1 struct (unused)
- Remove obsolete +build comment (buildtag)
- Use maps.Copy instead of manual loop (mapsloop)
- Modernize for loop with range over int (rangeint)
- Apply gofmt formatting
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix gofmt formatting issues
Fix alignment in struct literals and constant declarations:
- Align map keys in report_test.go request bodies
- Align struct fields in ReportPostOptions initialization
- Align reporting constant declarations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update mmctl tests for opaque cursor and add i18n translations
Update report_test.go to align with the refactored Posts Reporting API:
- Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag
- Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Update all mock expectations to use new ReportPostOptionsCursor structure
- Replace test cursor values with base64-encoded opaque cursor strings
Add English translations for cursor decoding error messages in i18n/en.json.
Minor API documentation fix in reports.yaml (remove "all" from description).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Update mmctl tests for opaque cursor and add i18n translations
Update report_test.go to align with the refactored Posts Reporting API:
- Replace split cursor flags (cursor-time, cursor-id) with single opaque cursor flag
- Update field name: ExcludeChannelMetadataSystemPosts → ExcludeSystemPosts
- Update all mock expectations to use new ReportPostOptionsCursor structure
- Replace test cursor values with base64-encoded opaque cursor strings
Add English translations for cursor decoding error messages in i18n/en.json.
Minor API documentation fix in reports.yaml (remove "all" from description).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* more lint fixes
* remove index update files
* Remove end_time parameter from Posts Reporting API
Align with other cursor-based APIs in the codebase by removing the end_time
parameter. The caller now controls when to stop pagination by simply not
making another request, which is the same pattern used by GetPostsSinceForSync,
MessageExport, and GetPostsBatchForIndexing.
Changes:
- Remove EndTime field from ReportPostOptions and ReportPostQueryParams
- Remove EndTime filtering logic from store layer
- Remove tests that used end_time parameter
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Refactor posts reporting API for security and validation
Address security review feedback by consolidating parameter resolution
and validation in the API layer, with comprehensive validation of all
cursor fields to prevent SQL injection and invalid queries.
Changes:
- Move parameter resolution from model to API layer for clearer separation
- Add ReportPostQueryParams.Validate() with inline validation for all fields
- Validate ChannelId, TimeField, SortDirection, and CursorId format
- Add start_time parameter for time-bounded queries
- Cap per_page at 100-1000 instead of rejecting invalid values
- Export DecodeReportPostCursorV1() for API layer use
- Simplify app layer to receive pre-validated parameters
- Check channel existence when results are empty (better error messages)
Testing:
- Add 10 model tests for validation and malformed cursor scenarios
- Add 4 API tests for cursors with invalid field values
- Refactor 13 store tests to use buildReportPostQueryParams() helper
- All 31 tests pass
Documentation:
- Update OpenAPI spec with start_time, remove unused end_time
- Update markdown docs with start_time examples
Security improvements:
- Whitelist validation prevents SQL injection in TimeField/SortDirection
- Format validation ensures ChannelId and CursorId are valid IDs
- Single validation point for both cursor and options paths
- Defense in depth: validation + parameterized queries + store layer whitelist
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Improve posts reporting query efficiency and safety
Replace SELECT * and nested OR/AND conditions with explicit column
selection and PostgreSQL row value comparison for better performance
and maintainability.
Changes:
- Use postSliceColumns() instead of SELECT * for explicit column selection
- Replace Squirrel OR/AND with row value comparison: (timeField, Id) > (?, ?)
- Use fmt.Sprintf for safer string formatting in WHERE clause
Query improvements:
Before: WHERE (CreateAt > ?) OR (CreateAt = ? AND Id > ?)
After: WHERE (CreateAt, Id) > (?, ?)
Benefits:
- Explicit column selection prevents issues if table schema changes
- Row value comparison is more concise and better optimized by PostgreSQL
- Follows existing patterns in post_store.go (postSliceColumns)
- Standard SQL:2003 syntax
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Change posts reporting response from map to ordered array
Replace the Posts map with an ordered array to preserve query sort order
and provide a more natural API response for sequential processing.
Changes:
- ReportPostListResponse.Posts: map[string]*Post → []*Post
- Store layer returns posts array directly (already sorted by query)
- App layer iterates by index for metadata enrichment
- Remove applyPostsWillBeConsumedHook call (not applicable to reporting)
- Update API tests to iterate arrays instead of map lookups
- Update store tests to convert array to map for deduplication checks
- Remove unused "maps" import
Benefits:
- Preserves query sort order (ASC/DESC, create_at/update_at)
- More natural for sequential processing/export workflows
- Simpler response structure for reporting/compliance use cases
- Aligns with message export/compliance patterns (no plugin hooks)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix linting issues in posts reporting tests
Replace inefficient loops with append(...) for better performance.
Changes:
- Use append(postSlice, result.Posts...) instead of loop
- Simplifies code and follows staticcheck recommendations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix store test AppError nil checking
Use require.Nil instead of require.NoError for *AppError returns
to avoid Go interface nil pointer issues.
When DecodeReportPostCursorV1 returns nil *AppError and it's assigned
to error interface, the interface becomes non-nil even though the
pointer is nil. This causes require.NoError to fail incorrectly.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-11-17 11:02:19 -05:00
|
|
|
|
|
|
|
|
// getPostsForReporting retrieves posts for reporting purposes with cursor-based pagination.
|
|
|
|
|
//
|
|
|
|
|
// API Endpoint: POST /api/v4/reports/posts
|
|
|
|
|
//
|
|
|
|
|
// Cursor Behavior:
|
|
|
|
|
// - The cursor is opaque and self-contained (base64-encoded)
|
|
|
|
|
// - When a cursor is provided, it contains all query parameters from the initial request
|
|
|
|
|
// - Query parameters in the request body (time_field, sort_direction, include_deleted, exclude_system_posts)
|
|
|
|
|
// are IGNORED when a cursor is present - the cursor's parameters take precedence
|
|
|
|
|
// - This allows clients to keep sending the same parameters on every request without causing errors
|
|
|
|
|
// - For the first page or to start a new query, omit the cursor or send an empty string
|
|
|
|
|
//
|
|
|
|
|
// Required permissions: System Admin (PERMISSION_MANAGE_SYSTEM)
|
|
|
|
|
func getPostsForReporting(c *Context, w http.ResponseWriter, r *http.Request) {
|
|
|
|
|
// Require system admin permission for accessing posts reporting
|
|
|
|
|
if !c.IsSystemAdmin() {
|
|
|
|
|
c.SetPermissionError(model.PermissionManageSystem)
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Request body contains both options and cursor
|
|
|
|
|
var request struct {
|
|
|
|
|
model.ReportPostOptions
|
|
|
|
|
model.ReportPostOptionsCursor
|
|
|
|
|
}
|
|
|
|
|
if jsonErr := json.NewDecoder(r.Body).Decode(&request); jsonErr != nil {
|
|
|
|
|
c.SetInvalidParamWithErr("body", jsonErr)
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Resolve query parameters: either from cursor (if provided) or from request options
|
|
|
|
|
var queryParams *model.ReportPostQueryParams
|
|
|
|
|
var appErr *model.AppError
|
|
|
|
|
|
|
|
|
|
if request.Cursor != "" {
|
|
|
|
|
// Decode cursor - it contains all query parameters
|
|
|
|
|
queryParams, appErr = model.DecodeReportPostCursorV1(request.Cursor)
|
|
|
|
|
if appErr != nil {
|
|
|
|
|
c.Err = appErr
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
} else {
|
|
|
|
|
// First request - build params from options
|
|
|
|
|
// Set defaults for optional parameters
|
|
|
|
|
timeField := model.ReportingTimeFieldCreateAt
|
|
|
|
|
if request.TimeField == model.ReportingTimeFieldUpdateAt {
|
|
|
|
|
timeField = model.ReportingTimeFieldUpdateAt
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
sortDirection := model.ReportingSortDirectionAsc
|
|
|
|
|
if request.SortDirection == model.ReportingSortDirectionDesc {
|
|
|
|
|
sortDirection = model.ReportingSortDirectionDesc
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Set initial cursor position based on sort direction and start_time
|
|
|
|
|
cursorTime := int64(0) // ASC: start from beginning (or user-specified start_time)
|
|
|
|
|
if request.StartTime > 0 {
|
|
|
|
|
cursorTime = request.StartTime
|
|
|
|
|
} else if sortDirection == model.ReportingSortDirectionDesc {
|
|
|
|
|
cursorTime = math.MaxInt64 // DESC: start from end
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Build query params
|
|
|
|
|
queryParams = &model.ReportPostQueryParams{
|
|
|
|
|
ChannelId: request.ChannelId,
|
|
|
|
|
CursorTime: cursorTime,
|
|
|
|
|
CursorId: "",
|
|
|
|
|
TimeField: timeField,
|
|
|
|
|
SortDirection: sortDirection,
|
|
|
|
|
IncludeDeleted: request.IncludeDeleted,
|
|
|
|
|
ExcludeSystemPosts: request.ExcludeSystemPosts,
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Handle PerPage - applies to both cursor and non-cursor requests
|
|
|
|
|
// Cap at min 100, max 1000
|
|
|
|
|
if request.PerPage <= 0 {
|
|
|
|
|
queryParams.PerPage = 100
|
|
|
|
|
} else if request.PerPage > model.MaxReportingPerPage {
|
|
|
|
|
queryParams.PerPage = model.MaxReportingPerPage
|
|
|
|
|
} else {
|
|
|
|
|
queryParams.PerPage = request.PerPage
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Validate the fully resolved query parameters (shared validation for both cursor and options)
|
|
|
|
|
if appErr = queryParams.Validate(); appErr != nil {
|
|
|
|
|
c.Err = appErr
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Call app layer with resolved and validated parameters
|
|
|
|
|
response, appErr := c.App.GetPostsForReporting(c.AppContext, *queryParams, request.IncludeMetadata)
|
|
|
|
|
if appErr != nil {
|
|
|
|
|
c.Err = appErr
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if len(response.Posts) == 0 {
|
|
|
|
|
// Determine if channel exists.
|
|
|
|
|
// This provides a better error message than returning an empty result set
|
|
|
|
|
channel, appErr := c.App.GetChannel(c.AppContext, request.ChannelId)
|
|
|
|
|
if appErr != nil {
|
|
|
|
|
c.Err = appErr
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
if channel == nil {
|
|
|
|
|
c.Err = model.NewAppError("getPostsForReporting", "api.post.get_posts_for_reporting.channel_not_found", nil, fmt.Sprintf("channel_id=%s", request.ChannelId), http.StatusNotFound)
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
if jsonErr := json.NewEncoder(w).Encode(response); jsonErr != nil {
|
|
|
|
|
c.Logger.Warn("Error writing response", mlog.Err(jsonErr))
|
|
|
|
|
}
|
|
|
|
|
}
|