mirror of
https://github.com/mattermost/mattermost.git
synced 2026-04-15 22:12:19 -04:00
4 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Andre Vasconcelos
|
13a0d63b3c
|
MM-67372: Improve link preview metadata handling and filtering (#35178)
* MM-67372: Filter SVG images from OpenGraph metadata to prevent DoS This commit adds server-side filtering of SVG images from OpenGraph metadata to mitigate a DoS vulnerability where malicious SVG images in og:image tags can crash Chromium-based browsers and Safari. Changes: - Add IsSVGImageURL() helper function in model package to detect SVG URLs - Filter SVG images in parseOpenGraphMetadata() for regular HTML pages - Filter SVG images in parseOpenGraphFromOEmbed() for oEmbed responses - Add defense-in-depth filtering in TruncateOpenGraph() and getImagesForPost() - Add comprehensive tests for all SVG filtering functionality SVG detection is based on: - File extension (.svg, .svgz) - case-insensitive - MIME type (image/svg+xml) Reference: https://issues.chromium.org/issues/40057345 * MM-67372: Filter SVG images from cache/DB and direct SVG URLs This commit addresses remaining attack vectors for the SVG DoS vulnerability: 1. Cache/DB filtering: Apply TruncateOpenGraph when returning OpenGraph from cache or database to filter stale data that was stored before the initial fix was deployed. 2. Direct SVG URLs: Filter PostImage entries with Format="svg" to prevent browser crashes when someone posts a direct link to an SVG file. 3. Embed creation: Skip creating image embeds for SVG images and create link embeds instead. 4. New SVG detection: Return nil instead of creating PostImage when fetching direct SVG URLs to prevent storing them in the database. These changes ensure that even environments with pre-existing malicious link metadata will be protected after a server restart. * MM-67372: Fix test expectation for SVG image handling * Removed duplicate logic in favor of already implemented FilterSVGImages in model * Addressing PR comments * Replacing exact match comparison with prefix check * Added new test cases for unit tests |
||
Jesse Hallam
|
41e5c7286b
|
Remove vestigial MySQL support (#34865)
* Remove legacy quoteColumnName() utility Since Mattermost only supports PostgreSQL, the quoteColumnName() helper that was designed to handle database-specific column quoting is no longer needed. The function was a no-op that simply returned the column name unchanged. Remove the function from utils.go and update status_store.go to use the "Manual" column name directly. * Remove legacy driver checks from store.go Since Mattermost only supports PostgreSQL, remove conditional checks for different database drivers: - Simplify specialSearchChars() to always return PostgreSQL-compatible chars - Remove driver check from computeBinaryParam() - Remove driver check from computeDefaultTextSearchConfig() - Simplify GetDbVersion() to use PostgreSQL syntax directly - Remove switch statement from ensureMinimumDBVersion() - Remove unused driver parameter from versionString() * Remove MySQL alternatives for batch delete operations Since Mattermost only supports PostgreSQL, remove the MySQL-specific DELETE...LIMIT syntax and keep only the PostgreSQL array-based approach: - reaction_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch - file_info_store.go: Use PostgreSQL array syntax for PermanentDeleteBatch - preference_store.go: Use PostgreSQL tuple IN subquery for DeleteInvalidVisibleDmsGms * Remove MySQL alternatives for UPDATE...FROM syntax Since Mattermost only supports PostgreSQL, remove the MySQL-specific UPDATE syntax that joins tables differently: - thread_store.go: Use PostgreSQL UPDATE...FROM syntax in MarkAllAsReadByChannels and MarkAllAsReadByTeam - post_store.go: Use PostgreSQL UPDATE...FROM syntax in deleteThreadFiles * Remove MySQL alternatives for JSON and subquery operations Since Mattermost only supports PostgreSQL, remove the MySQL-specific JSON and subquery syntax: - thread_store.go: Use PostgreSQL JSONB operators for updating participants - access_control_policy_store.go: Use PostgreSQL JSONB @> operator for querying JSON imports - session_store.go: Use PostgreSQL subquery syntax for Cleanup - job_store.go: Use PostgreSQL subquery syntax for Cleanup * Remove MySQL alternatives for CTE queries Since Mattermost only supports PostgreSQL, simplify code that uses CTEs (Common Table Expressions): - channel_store.go: Remove MySQL CASE-based fallback in UpdateLastViewedAt and use PostgreSQL CTE exclusively - draft_store.go: Remove driver checks in DeleteEmptyDraftsByCreateAtAndUserId, DeleteOrphanDraftsByCreateAtAndUserId, and determineMaxDraftSize * Remove driver checks in migrate.go and schema_dump.go Simplify migration code to use PostgreSQL driver directly since PostgreSQL is the only supported database. * Remove driver checks in sqlx_wrapper.go Always apply lowercase named parameter transformation since PostgreSQL is the only supported database. * Remove driver checks in user_store.go Simplify user store functions to use PostgreSQL-only code paths: - Remove isPostgreSQL parameter from helper functions - Use LEFT JOIN pattern instead of subqueries for bot filtering - Always use case-insensitive LIKE with lower() for search - Remove MySQL-specific role filtering alternatives * Remove driver checks in post_store.go Simplify post_store.go to use PostgreSQL-only code paths: - Inline getParentsPostsPostgreSQL into getParentsPosts - Use PostgreSQL TO_CHAR/TO_TIMESTAMP for date formatting in analytics - Use PostgreSQL array syntax for batch deletes - Simplify determineMaxPostSize to always use information_schema - Use PostgreSQL jsonb subtraction for thread participants - Always execute RefreshPostStats (PostgreSQL materialized views) - Use materialized views for AnalyticsPostCountsByDay - Simplify AnalyticsPostCountByTeam to always use countByTeam * Remove driver checks in channel_store.go Simplify channel_store.go to use PostgreSQL-only code paths: - Always use sq.Dollar.ReplacePlaceholders for UNION queries - Use PostgreSQL LEFT JOIN for retention policy exclusion - Use PostgreSQL jsonb @> operator for access control policy imports - Simplify buildLIKEClause to always use LOWER() for case-insensitive search - Simplify buildFulltextClauseX to always use PostgreSQL to_tsvector/to_tsquery - Simplify searchGroupChannelsQuery to use ARRAY_TO_STRING/ARRAY_AGG * Remove driver checks in file_info_store.go Simplify file_info_store.go to use PostgreSQL-only code paths: - Always use PostgreSQL to_tsvector/to_tsquery for file search - Use file_stats materialized view for CountAll() - Use file_stats materialized view for GetStorageUsage() when not including deleted - Always execute RefreshFileStats() for materialized view refresh * Remove driver checks in attributes_store.go Simplify attributes_store.go to use PostgreSQL-only code paths: - Always execute RefreshAttributes() for materialized view refresh - Remove isPostgreSQL parameter from generateSearchQueryForExpression - Always use PostgreSQL LOWER() LIKE LOWER() syntax for case-insensitive search * Remove driver checks in retention_policy_store.go Simplify retention_policy_store.go to use PostgreSQL-only code paths: - Remove isPostgres parameter from scanRetentionIdsForDeletion - Always use pq.Array for scanning retention IDs - Always use pq.Array for inserting retention IDs - Remove unused json import * Remove driver checks in property stores Simplify property_field_store.go and property_value_store.go to use PostgreSQL-only code paths: - Always use PostgreSQL type casts (::text, ::jsonb, ::bigint, etc.) - Remove isPostgres variable and conditionals * Remove driver checks in channel_member_history_store.go Simplify PermanentDeleteBatch to use PostgreSQL-only code path: - Always use ctid-based subquery for DELETE with LIMIT * Remove remaining driver checks in user_store.go Simplify user_store.go to use PostgreSQL-only code paths: - Use LEFT JOIN for bot exclusion in AnalyticsActiveCountForPeriod - Use LEFT JOIN for bot exclusion in IsEmpty * Simplify fulltext search by consolidating buildFulltextClause functions Remove convertMySQLFullTextColumnsToPostgres and consolidate buildFulltextClause and buildFulltextClauseX into a single function that takes variadic column arguments and returns sq.Sqlizer. * Simplify SQL stores leveraging PostgreSQL-only support - Simplify UpdateMembersRole in channel_store.go and team_store.go to use UPDATE...RETURNING instead of SELECT + UPDATE - Simplify GetPostReminders in post_store.go to use DELETE...RETURNING - Simplify DeleteOrphanedRows queries by removing MySQL workarounds for subquery locking issues - Simplify UpdateUserLastSyncAt to use UPDATE...FROM...RETURNING instead of fetching user first then updating - Remove MySQL index hint workarounds in ORDER BY clauses - Update outdated comments referencing MySQL - Consolidate buildFulltextClause and remove convertMySQLFullTextColumnsToPostgres * Remove MySQL-specific test artifacts - Delete unused MySQLStopWords variable and stop_word.go file - Remove redundant testSearchEmailAddressesWithQuotes test (already covered by testSearchEmailAddresses) - Update comment that referenced MySQL query planning * Remove MySQL references from server code outside sqlstore - Update config example and DSN parsing docs to reflect PostgreSQL-only support - Remove mysql:// scheme check from IsDatabaseDSN - Simplify SanitizeDataSource to only handle PostgreSQL - Remove outdated MySQL comments from model and plugin code * Remove MySQL references from test files - Update test DSNs to use PostgreSQL format - Remove dead mysql-replica flag and replicaFlag variable - Simplify tests that had MySQL/PostgreSQL branches * Update docs and test config to use PostgreSQL - Update mmctl config set example to use postgres driver - Update test-config.json to use PostgreSQL DSN format * Remove MySQL migration scripts, test data, and docker image Delete MySQL-related files that are no longer needed: - ESR upgrade scripts (esr.*.mysql.*.sql) - MySQL schema dumps (mattermost-mysql-*.sql) - MySQL replication test scripts (replica-*.sh, mysql-migration-test.sh) - MySQL test warmup data (mysql_migration_warmup.sql) - MySQL docker image reference from mirror-docker-images.json * Remove MySQL references from webapp - Simplify minimumHashtagLength description to remove MySQL-specific configuration note - Remove unused HIDE_MYSQL_STATS_NOTIFICATION preference constant - Update en.json i18n source file * clean up e2e-tests * rm server/tests/template.load * Use teamMemberSliceColumns() in UpdateMembersRole RETURNING clause Refactor to use the existing helper function instead of hardcoding the column names, ensuring consistency if the columns are updated. * u.id -> u.Id * address code review feedback --------- Co-authored-by: Mattermost Build <build@mattermost.com> |
||
Nadav Tasher
|
baf9bf82c9
|
Bugfix - Enforce URL length check in LinkMetadata before attempting to insert to database (#29589)
* Add missing URL length validation in LinkMetadata * Added MaxURLLength constant * Add test to validate new URL length limit * Add comment about table constraint * Use a seperate error string for URL length errors * Added error string fpr URL length * Updated error message * Update en.json * Fixed gofmt error in link_metadata.go * use right txn id ```release-note NONE ``` --------- Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in> Co-authored-by: Mattermost Build <build@mattermost.com> |
||
|
Jesse Hallam
|
bb02b35048
|
Expose public/ API as submodule (#23345)
* model -> public/model * plugin -> public/plugin * public/model/utils -> public/utils * platform/shared/mlog -> public/shared/mlog * platform/shared/i18n -> public/shared/i18n * platform/shared/markdown -> public/shared/markdown * platform/services/timezones -> public/shared/timezones * channels/einterfaces -> einterfaces * expose public/ submodule * go mod tidy * .github: cache-dependency-path, setup-go-work * modules-tidy for public/ too * remove old gomodtidy |
Renamed from server/model/link_metadata.go (Browse further)