* Excludes remote channels from channel search
This is done through a new body parameter in the SearchAllChannels
endpoint that allows to search for local only channels, which are
either channels that are shared but marked as homed locally, or
channels that are not shared at all.
* fix lint
* Fix tests
---------
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
- Improve some code to use ToSQL
- Fix a bug in SqlReactionStore.GetForPostSince
where OrderBy wasn't added properly. Added a test.
- Improve a method to return a slice rather than
pass a pointer to slice.
```release-note
NONE
```
* Add metrics for mobile versions snapshots
* Add notifications disabled and fix lint
* Address feedback
* Verify all references to JobTypeActiveUsers
* Fix typos
* Improve platform values
* Add test and MySQL support
* ignore bot accounts when counting deactivated users
* moved query to squirrel
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
* set is null right
* Run a different query depending on driver due to performance reasons.
---------
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Allows invites to be sent to offline remotes
Invites sent to remotes marked as offline will be stored as pending,
and when the remote comes back online, it will process the invites as
part of the synchronization process.
* Update condition name for excluding confirmed invites
* Adds logical deletes to shared channel remotes and remote clusters
Instead of physically deleting the shared channel remote and remote
clusters records when a channel is unshared, a remote uninvited or a
remote cluster is deleted, now those have a logical `DeleteAt` field
that is set.
This allows us to safely restore shared channels between two remote
clusters (as of now resetting the cursor without backfilling their
contents) and to know which connections were established in the past
and now are severed.
* Delete the index in remoteclusters before adding the new column
* Fix bad error check
* Adds Shared Channel management API endpoints
New endpoints for the following routes are added:
- Get Shared Channel Remotes by Remote Cluster at `GET
/api/v4/remotecluster/{remote_id}/sharedchannelremotes`
- Invite Remote Cluster to Channel at `POST
/api/v4/remotecluster/{remote_id}/channels/invite`
- Uninvite Remote Cluster to Channel at `POST
/api/v4/remotecluster/{remote_id}/channels/uninvite`
These endpoints are planned to be used from the system console, and
gated through the `manage_secure_connections` permission.
* Adds i18n messages for API errors
* Fix pagination flaky test
* Fix linter
* Adds the posibility of filtering shared channel remotes by home
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Adds default team to the remote cluster entity
A new DefaultTeamId field is added to the RemoteCluster entity and its
endpoints, and used when receiving channel invites to choose in which
team to create a new channel.
This will be later extended with the ability for the system admin to
manually accept invites, choosing which team to create the channel on
each. This use case will be triggered when the DefaultTeamId field is
empty, which now simply chooses the first team it finds in the
database as a fallback.
* Fix migrations list
* Fixes channelinvite test case
* Fix i18n
* Fix migration list
* Refactored channel saving logic
The code for saving channels has been refactored handling different database drivers. The query string is now dynamically generated based on the driver in use. Additionally, error handling has been improved to account for cases where no rows are affected by the insert operation. Because a conflict should be detected due to the complex flow under the call of the function.
* Refactor variable name and error handling in channel store
The variable 'q' has been renamed to 'insert' for better readability. Error handling after executing the insert statement has been improved by checking for errors immediately after execution, rather than waiting until rows affected are checked. This provides a more immediate response to potential issues during the insert operation.
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Fixed the issue for DB layer, ES to go
* Handled channel bookmarks
* Handled Bleve
* Lint fix
* Added channel bookmark test
* Skip bleve test
* Used common function
* SKipping ES as indexing logic in unavailable in test
* Opened modal from system console
* WIP
* WIP
* WIP
* Handled saving user
* Successfully updated user based settings
* WIP
* WIP
* All settings are updating well
* Fixed modal style
* Added admin mode indicators in modal
* Added confirmation dialog
* Lint fixes
* Added license check
* Added permission check
* Fixed i18n file order
* type fix
* Updated snapshots
* Handled performance debugging setting
* Some styling tweaks
* Fixed text alighnment
* Updated license required from professional to enterprise
* Handled long user names
* review fixes
* Added manage setting option in user list page context menu
* Added loader
* Minor reordering
* Removed confirm modal
* Updated snapshots for removed modal
* Added some tests
* Lint fix
* Used new selector in user detail page
* Used new selector in user list page
* Updated tests
* Fixed an incorrect default test
* Adds Remote Cluster related API endpoints
New endpoints for the following routes are added:
- Get Remote Clusters at `GET /api/v4/remotecluster`
- Create Remote Cluster at `POST /api/v4/remotecluster`
- Accept Remote Cluster invite at `POST
/api/v4/remotecluster/accept_invite`
- Generate Remote Cluster invite at `POST
/api/v4/remotecluster/{remote_id}/generate_invite`
- Get Remote Cluster at `GET /api/v4/remotecluster/{remote_id}`
- Patch Remote Cluster at `PATCH /api/v4/remotecluster/{remote_id}`
- Delete Remote Cluster at `DELETE /api/v4/remotecluster/{remote_id}`
These endpoints are planned to be used from the system console, and
gated through the `manage_secure_connections` permission.
* Update server/channels/api4/remote_cluster_test.go
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
* Fix AppError names
---------
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
Co-authored-by: Mattermost Build <build@mattermost.com>
* - ensure that posts and reactions can only be added via sync when coming from a remote that the target channel is shared with.
- ensure that posts and reactions are only modified/deleted by the remote that owns them.
* check that reaction belongs to post that belongs to channel that is shared with remote; check that posts belong to channel shared with remote
* check for correct error type in unit test
* tweak unit test
* [MM-58355] Send invalidate cache message across the cluster so that websocket connections on other instances are invalidated correctly
* Add suggestion to clear the session cache on the local node as well
* Force read from master DB when gettting channel members for websocket to avoid any DB sync issues
* PR feedback
* Missed generated files
Calling app.DeletePost immediately after creating a post
is susceptible to replica lag because we were calling the
replica to check for the post.
We fix this by passing a context to always query master.
https://mattermost.atlassian.net/browse/MM-58038
```release-note
NONE
```
* Deprecate Self Serve: First Pass
* Fix ci
* Fix more ci
* Remmove outdated server tests
* Fix a missed spot opening purchase modal in Self Hosted
* Fix i18n
* Clean up some more server code, fix webapp test
* Fix alignment of button
* Fix linter
* Fix i18n server side
* Deprecate in product true up
* Add back translation
* Remove client functions
* Put back client functions
* webapp deprecation
* Deprecate Self Serve: Second Pass
* Fix various pipeline issues
* Fix linter
* Fix pipelines
* Fix handlers_test.go
* Fix console.error around hostedCustomer in reducer
* PICKY LINTER PLEASE
* Fix webapp tests, various other fixes for the CI pipelines
* Fix i18n
* Updates to accomadate enterprise code removal
* Fix mocks
* More removal
* Fix
* Adjustments from PR
* Fixes for QA Feedback
* Update
* Add migrations to remove true up review history
* Fix migrations check
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: maria.nunez <maria.nunez@mattermost.com>
The error reporte when moving channels and failing has been improved to show
that the problem was the repeated name on the team.
The error message has been unified with MM-53756
Co-authored-by: Mattermost Build <build@mattermost.com>
* always ping on plugin registration; SharedChannel.IsValid allow no team for GM
* wait for services to start before ping
* ping plugin remotes synchronously on startup
* remove the waitForInterClusterServices stuff
* don't set remoteid when inviting remote to channel
* Update server/public/model/remote_cluster_test.go
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* address review comments
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* add ratelimit to /sessions; cap userAgent version length; tests
* add MaxSessionsLimit; remove oldest session first; tests
* can't use slices in 1.20; improve test
* nits
* add GetLRUSessions; move limiting to CreateSession; remove rate limiting
* use queryBuilder
* mysql needs a limit when using offset
* update i18n
* refactor into limitNumberOfSessions; protect createSessionForUserAccessToken
* add comment to GetLRUSessions
* add limit to oauth path; PR comments
* Use a row constructor comparison to avoid a filter
The original query had a WHERE condition that looked like:
(x > a) OR (x = a AND y < b)
This commit changes it to a compound comparison that, given the
lexicographical order it uses, is semantically the same:
(x, y) > (a, b)
This makes the plan in Postgres use an Index Cond instead of a Filter,
having two main performance improvements:
1. The query's time is more or less constant, not increasing every time
this query is executed. This is important because the query is executed
repeatedly until all posts are indexed.
2. The query's time is much shorter: while the original query eventually
takes 30s, hitting the timeout, the new one takes ~30ms.
Notice the difference in the plans below:
- the original query uses a `Filter` step, while the new one uses an
`IndexCond` + `Filter` step.
- the original query has a shared hit count of 40557115 (109GiB), while
the new one has a shared hit count of 13951 (109MiBA). This does not
show exactly how much data is read from disk, given it's a nested loop
node, but it gives us an idea of the amount of data processed in the
server.
Original plan:
agnivaltdb=> EXPLAIN (ANALYZE, BUFFERS) SELECT Posts.*, Channels.TeamId FROM Posts LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE Posts.CreateAt > '1687424888405' OR (Posts.CreateAt = '1687424888405' AND Posts.Id > 'tpomh9yu1tffmdp6dopobwuc9h') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=1.22..2287.94 rows=10000 width=464) (actual time=23921.236..23954.229 rows=10000 loops=1)
Buffers: shared hit=40557115
-> Incremental Sort (cost=1.22..13998881.93 rows=61217938 width=464) (actual time=23921.235..23953.205 rows=10000 loops=1)
Sort Key: posts.createat, posts.id
Presorted Key: posts.createat
Full-sort Groups: 311 Sort Method: quicksort Average Memory: 45kB Peak Memory: 45kB
Buffers: shared hit=40557115
-> Nested Loop Left Join (cost=1.00..11421751.31 rows=61217938 width=464) (actual time=23920.970..23947.067 rows=10001 loops=1)
Buffers: shared hit=40557115
-> Index Scan using idx_posts_create_at on posts (cost=0.57..9889434.54 rows=61217938 width=461) (actual time=23920.930..23931.063 rows=10001 loops=1)
Filter: ((createat > '1687424888405'::bigint) OR ((createat = '1687424888405'::bigint) AND ((id)::text > 'tpomh9yu1tffmdp6dopobwuc9h'::text)))
Rows Removed by Filter: 40920000
Buffers: shared hit=40553119
-> Memoize (cost=0.43..0.70 rows=1 width=30) (actual time=0.001..0.001 rows=1 loops=10001)
Cache Key: posts.channelid
Cache Mode: logical
Hits: 9002 Misses: 999 Evictions: 0 Overflows: 0 Memory Usage: 151kB
Buffers: shared hit=3996
-> Index Scan using channels_pkey on channels (cost=0.42..0.69 rows=1 width=30) (actual time=0.007..0.007 rows=1 loops=999)
Index Cond: ((id)::text = (posts.channelid)::text)
Buffers: shared hit=3996
Planning:
Buffers: shared hit=112
Planning Time: 0.501 ms
Execution Time: 23954.974 ms
(25 rows)
New plan:
agnivaltdb=> EXPLAIN (ANALYZE, BUFFERS) SELECT Posts.*, Channels.TeamId FROM Posts LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE (Posts.CreateAt, Posts.Id) > ('1687424888405', 'tpomh9yu1tffmdp6dopobwuc9h') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=1.15..1666.83 rows=10000 width=464) (actual time=0.366..34.053 rows=10000 loops=1)
Buffers: shared hit=13951
-> Incremental Sort (cost=1.15..10196977.36 rows=61217938 width=464) (actual time=0.365..33.031 rows=10000 loops=1)
Sort Key: posts.createat, posts.id
Presorted Key: posts.createat
Full-sort Groups: 311 Sort Method: quicksort Average Memory: 45kB Peak Memory: 45kB
Buffers: shared hit=13951
-> Nested Loop Left Join (cost=1.00..7619846.74 rows=61217938 width=464) (actual time=0.059..26.840 rows=10001 loops=1)
Buffers: shared hit=13951
-> Index Scan using idx_posts_create_at on posts (cost=0.57..6087529.97 rows=61217938 width=461) (actual time=0.040..10.548 rows=10001 loops=1)
Index Cond: (createat >= '1687424888405'::bigint)
Filter: (ROW(createat, (id)::text) > ROW('1687424888405'::bigint, 'tpomh9yu1tffmdp6dopobwuc9h'::text))
Rows Removed by Filter: 2
Buffers: shared hit=9955
-> Memoize (cost=0.43..0.70 rows=1 width=30) (actual time=0.001..0.001 rows=1 loops=10001)
Cache Key: posts.channelid
Cache Mode: logical
Hits: 9002 Misses: 999 Evictions: 0 Overflows: 0 Memory Usage: 151kB
Buffers: shared hit=3996
-> Index Scan using channels_pkey on channels (cost=0.42..0.69 rows=1 width=30) (actual time=0.007..0.007 rows=1 loops=999)
Index Cond: ((id)::text = (posts.channelid)::text)
Buffers: shared hit=3996
Planning:
Buffers: shared hit=112
Planning Time: 0.471 ms
Execution Time: 34.716 ms
(26 rows)
* Go back to the old query for MySQL
As one could expect, the two databases have completely opposite
behaviours, and for MySQL, the old query is significantly faster than
the new one, exactly for the same reason than in Postgres, but the other
way around: the old query uses an Index range scan while the new one
would use a Filter.
Old query (0.18s):
mysql> EXPLAIN ANALYZE SELECT Posts.*, Channels.TeamId FROM Posts USE INDEX(idx_posts_create_at_id) LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE Posts.CreateAt > 1557752415221 OR (Posts.CreateAt = 1557752415221 AND Posts.Id > 'ad59ire57tfwmjr5r8xqxc75qw') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000;
--------------
EXPLAIN ANALYZE SELECT Posts.*, Channels.TeamId FROM Posts USE INDEX(idx_posts_create_at_id) LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE Posts.CreateAt > 1557752415221 OR (Posts.CreateAt = 1557752415221 AND Posts.Id > 'ad59ire57tfwmjr5r8xqxc75qw') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000
--------------
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| EXPLAIN |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| -> Limit: 10000 row(s) (cost=7472068.91 rows=10000) (actual time=0.063..164.174 rows=10000 loops=1)
-> Nested loop left join (cost=7472068.91 rows=5559093) (actual time=0.062..163.450 rows=10000 loops=1)
-> Index range scan on Posts using idx_posts_create_at_id over (CreateAt = 1557752415221 AND 'ad59ire57tfwmjr5r8xqxc75qw' < Id) OR (1557752415221 < CreateAt), with index condition: ((Posts.CreateAt > 1557752415221) or ((Posts.CreateAt = 1557752415221) and (Posts.Id > 'ad59ire57tfwmjr5r8xqxc75qw'))) (cost=1357066.29 rows=5559093) (actual time=0.043..97.358 rows=10000 loops=1)
-> Single-row index lookup on Channels using PRIMARY (Id=Posts.ChannelId) (cost=1.00 rows=1) (actual time=0.006..0.006 rows=1 loops=10000)
|
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.18 sec)
New query (5s):
mysql> EXPLAIN ANALYZE SELECT Posts.*, Channels.TeamId FROM Posts USE INDEX(idx_posts_create_at_id) LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE (Posts.CreateAt, Posts.Id) > (1557752415221, 'ad
59ire57tfwmjr5r8xqxc75qw') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000;
--------------
EXPLAIN ANALYZE SELECT Posts.*, Channels.TeamId FROM Posts USE INDEX(idx_posts_create_at_id) LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE (Posts.CreateAt, Posts.Id) > (1557752415221, 'ad59ire57tfwmjr5r8xqxc75qw') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000
--------------
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| EXPLAIN |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| -> Limit: 10000 row(s) (cost=11119405.48 rows=10000) (actual time=5295.106..5455.285 rows=10000 loops=1)
-> Nested loop left join (cost=11119405.48 rows=10000) (actual time=5295.105..5454.572 rows=10000 loops=1)
-> Filter: ((Posts.CreateAt,Posts.Id) > (1557752415221,'ad59ire57tfwmjr5r8xqxc75qw')) (cost=221.48 rows=10000) (actual time=5295.078..5388.668 rows=10000 loops=1)
-> Index scan on Posts using idx_posts_create_at_id (cost=221.48 rows=10000) (actual time=0.055..5314.753 rows=600000 loops=1)
-> Single-row index lookup on Channels using PRIMARY (Id=Posts.ChannelId) (cost=1.00 rows=1) (actual time=0.006..0.006 rows=1 loops=10000)
|
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (5.46 sec)
```
* Add comment explaining the difference between DBs
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* create ChannelBookmarks table
* ChannelBookmark model
* channel bookamrks Store layer
* add GetBookmarksForAllChannelByIdSince
* add channel bookmarks to test store
* Add channel bookmarks to app layer
* remove index for createAt in channel bookmarks migrations
* remove createAt from select channel bookmark query and enable store delete bookmark test
* update reponse of UpdateBookmark
* rename db migration files
* channel bookmarks store update sort order
* channel bookmarks app layer update sort order
* fix lint & tests
* Fix lint and introduce util functions to insert / remove from slice
* remove model etag
* i18n
* defer remove file info after test run
* Fix tests passing the request context
* fix migrations
* fix TestRetry
* Add bookmark permissions (#25560)
* Adds channel bookmarks permissions
* Fix linter
* Remove unnecessary empty lines
* Remove scss change as it's not necessary anymore
* Fix mock store
* Fix mock store and add role entry
* Fix test
* Adds cypress test and update permissions migration to update admin roles
* Adds channel bookmarks roles to default admin roles
* Adds bookmark permissions to default role permissions constant in webapp
* Update mmctl test
* Update permission test after normalising the roles
* fix store tests
* fix app layer tests
* Add new bookmark endpoint (#25624)
* Adds channel bookmarks api scaffold and create endpoint
* Applies review comments to the API docs
* Adds websocket test to create channel bookmark
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-54426 exclude Channel Bookmarks files from data retention (#25656)
* Augment channel APIs to include bookmarks (#25567)
* update files docs for server 9.4
* Adds update channel bookmark endpoint (#25653)
* Adds update channel bookmark sort order endpoint (#25686)
* Adds update channel bookmark endpoint
* Updates edit app method to return the right deleted bookmark and adds tests
* Adds the update channel bookmark sort order endpoint
* Fix repeated test after merge
* Assign right permissions to each test
* Update store and app layer to return specific errors and add tests
* Adds delete channel bookmark endpoint (#25693)
* Updates edit app method to return the right deleted bookmark and adds tests
* Fix repeated test after merge
* Updates edit app method to return the right deleted bookmark and adds tests
* Adds delete channel bookmark endpoint
* Adds list channel bookmarks endpoint (#25700)
* Add channel moderation to bookmarks (#25716)
* fix migrations index
* fix getChannelsForTeamForUser
* fix getChannelsForTeamForUser
* fix bad merge client4
* fix file api with bookmark permission
* add ChannelBookmarks feature flag
* add missing translations
* Set DB column for type as enum
* use custom type for bookmark query using sqlx
* use transaction when saving bookmark
* return NewErrNotFound instead of Sql.ErrNoRows
* use squirrel for IN query
* add a limit of 1K for records in GetBookmarksForAllChannelByIdSince
* UpdateSortOrder with one single query instead of multiple updates
* fix shadow declaration
* fix channel bookmarks permission string definition in admin console
* fix another shadow declaration
* Fix model conversion
* add SplitSliceInChunks
* remove include bookmarks in channels api
* Cap amount of bookmarks per channel
* add etag back to get channels
* feedback review
* update file info when replacing a bookmark file
* return 501 not implemented when the license is not available
* add detail message when getting channel member on bookmark api
* start audit before permission check on create bookmark api
* use require.Eventuallyf for testing WS events
* remove unnecessary log in app layer
* use require instead of assert to avoid panics
* enforce limit when querying bookmarks since
* prevent to create/update bookmark if file is already attached
* fix lint
* delete file when a bookmark is deleted
* Dot allow to set a fileId and a url at the same time to a bookmark
* fix query to delete a file that belongs to a bookmark
* do not patch the bookmark type
* Server side FeatureFlag check (#26145)
* use ff in server, set ff to false
* turn on FF for unit tests
* defer unset FF for unit tests
* turn ff on for testing
* only allow attaching files that were uploaded for bookmark
* Set feature flag off as default
* fix lint
* update email templates as PR failed
* revert templates
* force the assignment of ID when creating a bookmark
* Fix unit tests
---------
Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Moved some common SQL function tu public utls as they are used in plugins
* goimported file
* Added tests
* Created sub-package
* MOved SetupConnection to public sql utils
There were 3 remaining caches which were there in the store layer.
We migrate them to make the store layer fully free
from any caches.
https://mattermost.atlassian.net/browse/MM-56879
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
There were 2 separate bugs here:
1. We were not including deactivated users while fetching posts from
DM channels.
2. We were not respecting the includeArchivedChannels flag while
fetching DM channels.
We fix both of these issues here.
```release-note
Include posts from deactivated users in DM channel export. Also
respect the --include-archived-channels flag for DM channels.
```
https://mattermost.atlassian.net/browse/MM-56579
Co-authored-by: Mattermost Build <build@mattermost.com>
* added store
* make generated
* add missing license headers
* fix receiver name
* i18n
* i18n sorting
* update migrations from master
* make migrations-extract
* update retrylayer tests
* replaced sql query with id pagination
* fixed flaky tests
* missing columns
* missing columns on save/update
* typo
* improved tests
* remove enum from mysql colum
* add password credentials to store
* license changes
* OAuthOutgoingConnectionInterface
* Oauth -> OAuth
* make generated
* copied over installed_oauth_apps component and renamed things to installed_outgoing_oauth_connections
* merge migrations
* renamed migrations
* model change suggestions
* refactor test functionsn
* migration typo
* refactor store table names
* updated sanitize test
* cleanup merge
* refactor symbol
* "installed outgoing oauth connections" page works
* move things into a nested folder
* add and edit page stubs work
* list endpoint
* oauthoutgoingconnection -> outgoingoauthconnection
* signature change
* i18n update
* granttype typo
* naming
* api list
* uppercase typo
* i18n
* missing license header
* fixed path in comments
* updated openapi definitions
* changes to support selecting command request url
* sanitize connections
* make generated
* test license and no feature flag
* removed t.fatal
* updated testhelper calls
* yaml schema fixes
* switched interface name
* suggested translation
* missing i18n translation
* management permission
* moved permission initalization to proper place
* endpoints
* put tests
* error check typo
* fixed specific enttity urls
* tests
* read permission check
* updated openapi definitions
* i18n
* GetConnectionByAudience method
* notes
* replaced GetConnectionsByAudience with a filter
* added custom oauth token object
* updated interface and usage
* properly set enterprise interface
* move retrieval logic to impl
* webhook tests
* translations
* i18n: updates
* address comments
* endpoint and tests
* i18n
* api docs
* fixed endpoint path
* sq.like
* use filter object instead of parameters
* set url values if not empty
* typos
* converted some components to function components, and move around files
* correctly check token url
* restore flag to previous value
* added command oauth handler
* update enterprise imports
* migrate last component to function component
* Added enterprise import
* refactor permissions and add necessary webapp code
* Check correct flag in permission tree
* allow partial updates
* sort i18n webapp
* missing test modification
* fixed webapp i18n sorting
* allow validating stored connections
* added missing translation
* fix finished adding connection link and text on result page
* added missing permission to smoke tests
* missing role in smoke test
* updated translations
* updated translations
* support editing client secret on existing connection
* fix some i18n strings
* updated translations
* better error messages
* progress on using react select for command request url while maintaining typed in value
* remove writeheader, test
* HasValidGrantType
* end early to avoid nil pointer errors
* move slash command request url input box into its own component
* wrap components related to oauth connections in config check
* fix tests
* i18n-extract
* change some i18n strings to say "Outgoing OAuth 2.0 Connections"
* remove debug code
* fixed i18n
* updated i18n file
* feature configuration backend
* typo
* add system console setting
* Revert "typo"
This reverts commit 669da23e8e.
* Revert "updated i18n file"
This reverts commit d0882c0dd7.
* Revert "fixed i18n"
This reverts commit 3108866bc1.
* fixed i18n
* updated i18n file
* typo
* updated i18n
* updated i18n
* updated i18n
* updated version to 9.6
* replace feature flag with system console configuration
* i18n
* updated tests
* pr feedback
* fix styling of disabled text box
* fix styling of action links in integration console
* server changes for validation feature
* webapp changes for validation feature
* pencil icon styling
* styling fixes for oauth audience correct configuration message
* fix sanitize test
* remove max lengths from outgoing oauth connection form
* use config var in webapp instead of feature flag
* change asterisks to bullets
* update api docs for validate endpoint
* feedback from ux review
* fix lint, types, tests
* fix stylelint
* implement validation button under the token url input
* support wildcard for matching audience urls
* updates for styling
* update snapshots
* add doc links for the outgoing oauth connections feature
* change doc links to use permalink
* add docs link to system console
* fix: use limitedreader in json decoding
* fix: form error in validation
* management permission can read now
* updated api documentation
* doc typo
* require one permission to read only
* fix api connection list audience filter
* fix audience matching and add loading indicator
* fix team permissions on outgoing oauth connection api calls
* fix api doc and test, for adding team id to query params
* handle read permissions by adding a team in the payload
* missing teamid query parameter in test
* change validate button logic to not require audience urls to be filled out
* fix redux type
---------
Co-authored-by: Felipe Martin <me@fmartingr.com>
* added rctx to reuired files
* added more file changes after issues on startup
* added context to draft_store
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
We were incorrectly excluding deactivated users
while getting channel members for a DM channel, whereas
we were actually exporting all users in the users array.
We fix this and also correctly honor the includeArchivedChannels
flag as well.
https://mattermost.atlassian.net/browse/MM-55524
```release-note
Include deactivated members in a favorited DM channel export.
```
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* option for auto inviting plugin to all shared channels.
* auto-invite remotes to shared channels when flag set
* fix unit test
* immediately ping new remotes; fix unique siteurl bug
* make i18n-extract
* fix translations
* plugin hooks for file attachments
* hook for profile image sync
* fix profile image sync
* fix unit test
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Add ChannelStore.UpdateMultipleMembersNotifyProps
* Make UpdateMultipleMembersNotifyProps return updated values from the DB
* Add UpdateChannelMembersNotifications plugin API
* Extract i18n
* Fix style
* Make layers
* Change to PatchMultipleMembersNotifyProps
* Add limit to PatchChannelMembersNotifyProps
* Add additional unit tests
* Address feedback
* Lowercase decodeJSON
* Have PatchMultipleMembersNotifyProps update LastUpdateAt
* Fix tests that relied on unreliable return order
* Fix i18n
MySQL 5.7 is at end of life.
https://mattermost.atlassian.net/browse/MM-55589
```release-note
We bump up minimum MySQL version to be 8.0.0
```
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* [MM-56399] Add user count endpoint for reporting
* [MM-56397] Added search term to user report filter
* Missing translation
* [MM-56456] Rename up/down to prev/next for reporting cursoring
* [MM-56269] Add DeleteAt, MfaActive and AuthService fields to UserReport
* PR feedback
* Fix test
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* option for auto inviting plugin to all shared channels.
* auto-invite remotes to shared channels when flag set
* fix unit test
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
While loading a thread, we were unnecessarily re-counting the replies
for all posts in a thread even if they would be the same number.
While this is needed in other queries where the posts can be from different
threads or they can be random ids, but to load a single post thread, there
is no need to recompute it again and again.
Therefore, we use a CTE to precompute the replycount and then just plug in
the value in the subsequent query. This gives an improvement in the query
plan as well:
OLD:
```
explain (analyze, buffers) SELECT p.id, p.rootid, p.createat, (SELECT count(*) FROM Posts WHERE Posts.RootId = (CASE WHEN p.RootId = '' THEN p.Id ELSE p.RootId END) AND Posts.DeleteAt = 0) as ReplyC
ount FROM Posts p WHERE (p.Id = 'h3cer597jb8abbcbitpghpomua' OR p.RootId = 'h3cer597jb8abbcbitpghpomua') AND p.DeleteAt = 0;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------
Bitmap Heap Scan on posts p (cost=45.39..1042149.61 rows=2748 width=49) (actual time=3.156..7906.215 rows=5353 loops=1)
Recheck Cond: (((id)::text = 'h3cer597jb8abbcbitpghpomua'::text) OR (((rootid)::text = 'h3cer597jb8abbcbitpghpomua'::text) AND (deleteat = 0)))
Filter: (deleteat = 0)
Heap Blocks: exact=5308
Buffers: shared hit=610244
-> BitmapOr (cost=45.39..45.39 rows=2748 width=0) (actual time=0.918..0.920 rows=0 loops=1)
Buffers: shared hit=47
-> Bitmap Index Scan on posts_pkey (cost=0.00..1.68 rows=1 width=0) (actual time=0.028..0.028 rows=1 loops=1)
Index Cond: ((id)::text = 'h3cer597jb8abbcbitpghpomua'::text)
Buffers: shared hit=4
-> Bitmap Index Scan on idx_posts_root_id_delete_at (cost=0.00..42.34 rows=2747 width=0) (actual time=0.889..0.890 rows=5352 loops=1)
Index Cond: (((rootid)::text = 'h3cer597jb8abbcbitpghpomua'::text) AND (deleteat = 0))
Buffers: shared hit=43
SubPlan 1
-> Aggregate (cost=378.10..378.11 rows=1 width=8) (actual time=1.474..1.474 rows=1 loops=5353)
Buffers: shared hit=604889
-> Index Only Scan using idx_posts_root_id_delete_at on posts (cost=0.57..343.85 rows=13699 width=0) (actual time=0.016..1.039 rows=5352 loops=5353)
Index Cond: ((rootid = (CASE WHEN ((p.rootid)::text = ''::text) THEN p.id ELSE p.rootid END)::text) AND (deleteat = 0))
Heap Fetches: 0
Buffers: shared hit=604889
Planning Time: 0.194 ms
Execution Time: 7906.846 ms
```
NEW:
```
explain analyze with replycount as (select count(*) as num from posts where rootid='h3cer597jb8abbcbitpghpomua' and deleteat=0)
select id, rootid, createat, replycount.num from posts, replycount where id='h3cer597jb8abbcbitpghpomua' or rootid='h3cer597jb8abbcbitpghpomua' and deleteat=0;
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=123.16..3215.48 rows=2748 width=49) (actual time=2.960..9.775 rows=5353 loops=1)
-> Aggregate (cost=77.78..77.79 rows=1 width=8) (actual time=1.455..1.456 rows=1 loops=1)
-> Index Only Scan using idx_posts_root_id_delete_at on posts posts_1 (cost=0.57..70.91 rows=2747 width=0) (actual time=0.056..1.145 rows=5352 loops=1)
Index Cond: ((rootid = 'h3cer597jb8abbcbitpghpomua'::text) AND (deleteat = 0))
Heap Fetches: 0
-> Bitmap Heap Scan on posts (cost=45.39..3110.20 rows=2748 width=41) (actual time=1.501..7.747 rows=5353 loops=1)
Recheck Cond: (((id)::text = 'h3cer597jb8abbcbitpghpomua'::text) OR (((rootid)::text = 'h3cer597jb8abbcbitpghpomua'::text) AND (deleteat = 0)))
Heap Blocks: exact=5308
-> BitmapOr (cost=45.39..45.39 rows=2748 width=0) (actual time=0.797..0.798 rows=0 loops=1)
-> Bitmap Index Scan on posts_pkey (cost=0.00..1.68 rows=1 width=0) (actual time=0.014..0.014 rows=1 loops=1)
Index Cond: ((id)::text = 'h3cer597jb8abbcbitpghpomua'::text)
-> Bitmap Index Scan on idx_posts_root_id_delete_at (cost=0.00..42.34 rows=2747 width=0) (actual time=0.782..0.782 rows=5352 loops=1)
Index Cond: (((rootid)::text = 'h3cer597jb8abbcbitpghpomua'::text) AND (deleteat = 0))
Planning Time: 0.220 ms
Execution Time: 10.052 ms
(15 rows)
```
Observe the `loops=5353` in the first query, and `loops=1` in the next.
https://mattermost.atlassian.net/browse/MM-55476
```release-note
Optimize createPost performance
```
Co-authored-by: Mattermost Build <build@mattermost.com>
We just throw a warn log for now. Support will be completely
removed next ESR.
https://mattermost.atlassian.net/browse/MM-55589
```release-note
MySQL 5.7 is at EOL. We recommend all customers to upgrade to atleast 8.x. For now, we are logging a warning. From version 9.5 onwards, which is the next ESR, we will stop supporting 5.7 altogether.
```
Co-authored-by: Neil B <93996140+nab-77@users.noreply.github.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Add store method to get reporting data
* Some store changes
* Added app layer
* Added API call, some miscellaneous fixes
* Fix lint
* Fix serialized check
* Add API docs
* Fix user store tests leaking users
* Fix test
* PR feedback
* Add filtering for role/team/activated user, filter out bot users
* Fix mock
* Fix test
* Oops
* Switch to using struct filter
* More PR feedback
* Fix gen
* Fix test
* Fix API docs
* Fix test
* Fix possible SQL injection, some query optimization
* Fix migrations
* Oops
* Add role to API
* Fix check
* Add Client4 API call for load testing
* Fix test
* Update server/channels/store/storetest/user_store.go
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* PR feedback
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* - columns added to ShareChannelRemotes: lastpostcreateat, lastpostupdateat
- SyncMsg and SyncResponse moved to `model` package
- field added to RemoteCluster struct: PluginID
* sync new posts before updated posts to ensure post order in MS Teams
* add plugid to remoteclusters table and store
* don't sync history by default
* [MM-55143] Disallow reacting with an emoji that does not exist
* WIP for server limit on emoji reactions
* WIP
* Implement default limit of 25 unique emoji reactions
* Add modal for reaction limit
* Fix test
* PR feedback
* Fix i18n
* Update admin string
* Merge'd
* Fixing some issues, check limits correctly based on other users reactions
* Fix typos
* Fix lint/test
* Add tests, fix other tests
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Load tests show that channelstore.GetMember and
channelstore.GetMembersForUser are among the chief
queries that take up CPU in the DB.
In this PR, we attempt some strategic optimizations to
reduce/optimize calls to channelstore.GetMember
1. Optimize `(a *App) HasPermissionToChannel`
We replace GetChannelMember with GetAllChannelMembersForUser
because it's cache backed. So although it gets more data,
it does not hit the DB and saves some latency.
2. Optimize getPostsForChannelAroundLastUnread
We repace getChannelMember with getChannelMemberOnly
which is a lite version of the store call which queries
just the ChannelMembers table. This is because
in the app layer, we just use the LastViewedAt attribute.
Therefore, there is no reason to join with 5 tables when
a single table can do the job.
3. Optimize publishWebsocketEventForPermalinkPost
We use GetAllChannelMembersById instead of GetChannelMembersPage
which again joins with a lot of other tables.
4. Optimize countMentionsFromPost
Again, we replace GetChannelMember which is a costly call joining
multiple tables, with GetAllChannelMembersNotifyPropsForChannel
which is cache-backed and gives us just what we need in the app
layer - notify props.
```release-note
Make small optimizations in several DB calls:
- App.HasPermissionToChannel
- getPostsForChannelAroundLastUnread
- publishWebsocketEventForPermalinkPost
- countMentionsFromPost
```
https://mattermost.atlassian.net/browse/MM-55295
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Fix mark as unread on GMs
* Don't count own messages in gms when marking as unread
* Change argument name
* Rename userId
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* [MM-53428] Delete empty drafts on upsert
* Add migrations to fix existing drafts
* Fix CI
* Delete empty drafts entirely from the DB
* Fix lint
* Implement batch migration for deleting drafts
* Missing store layers
* Add updated mock
* Remove unnecessary test
* PR feedback
* Add check for cluster migration
* Fix MySQL
* Don't check for len<2
* Bit of PR feedback
* Use query builder for parameters
* PR feedback
* More PR feedback
* Merge'd
* unit test GetLastCreateAtAndUserIdValuesForEmptyDraftsMigration
* simplified builder interface
* fix DeleteEmptyDraftsByCreateAtAndUserId for MySQL
* rework as batch migration worker
* fix typo
* log ip address on version mismatches too
* simplify reset semantics
* remove trace log in favour of low spam
* document parameters for clarity
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
* pass a logger instead of embedding on job
* leverage mlog.Millis
* use worker logger with HandleJobPanic
* rely on existing LogClone instead
* guard Job.LogClone against nil Job
* s/workername/worker_name
* Revert "rely on existing LogClone instead"
This reverts commit 17303cbac9.
* Revert "guard Job.LogClone against nil Job"
This reverts commit f1ae22dee5.
* Added convert to channel menu item
* WIP
* refactored channel name input field and created conversion modal
* style
* style
* WIP
* wip
* Created API to fetch common teams of GM members
* Added UI for all members deactivated
* Fetched common teams in client
* WIP
* Added a required attribute to DropdownInput component
* Fixed a case with dropdown input required flag
* WIP
* API first draft
* Genetayed layers and mocks
* Fixed create channel bug
* WIP
* Added cache invalidation
* Calling API from client
* Updated API to accept name and display name as well
* WIP
* Moved converted GM to correct category
* Style fixes
* Added logic to move user to new team/channel after GM conversion
* Prevented guest user from performing action
* Added loading indicator
* Added smoother height transistion when loading finishes
* UI imporvements
* WIP
* Formatted GM conversion message on client side
* lint fix
* Moved convert option from sidebar menu to channel header menu
* Some cleanup
* Updated server layers
* Fixed i18n
* Fixed types
* Fix server i18n
* Fixed channel creation bug
* Added store test for GetCommonTeamIDsForMultipleUsers
* Server tests done
* Updated snapshots
* Updated layers
* lint fix
* Update tests
* For CI
* lint
* restored debug code
* Used user ID instead of username in channel conversion post
* WIP
* Review fixes
* LInt fixes
* Test fix
* WIP
* WIP
* WIP
* wip
* Review fixes, lots of them
* Review fix
* Disabled WIP test
* test
* Cleanup
* Test fix
* removed testing line
* Fixed incorrect default message
* Review fixes
* Fixes
* lint and i18n fix
* Setting category on server side
* updated i18n
* Updated tests
* Added tests
* Refs cleanup
* added test
---------
Co-authored-by: Harshil Sharma <harshilsharma@Harshils-MacBook-Pro.local>
* Use builder in GetMember
* Use query builder in GetByUser
* Use query builder in GetMemberUsers
* Use query builder in GetNonMemberUsersPage
* Use query builder in PermanentDeleteMembersByUser
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* adding new migration for RetentionIdsForDeletion, changing logic for deleting orphaned reactions. Updating delete user and channel endpoints to remove respective reactions
* add ability to restore groups from the user group modal
* factory selector for groups to reduce number of renders across the app
* react window and infinite scroll for user groups
* adding archive groups to dropdown
* restore user group from the view modal
* component cleanup
* lint
* adding websocket for archiveGroup
* updating tests
* adding some tests and fixing types
* lint
* fixing broken test
* fixing snapshot
* fixing infinitescroll
* lint
* increasing max-height and updating snapshots
* fixing PR comments
* fixing case for button
* snapshot and translation
* fixing PR comments
* tiding up repition and creating new hook
* fixing tests
* add additional parammeter for call to getGroups()
* make sure popup is visible for all rows
* update text for admin console
* update css for lint
* fix edge cases found in review
* revert package-lock.json
* revert adding query to GetGroupsParam
* fixing lint
* change include_archived to false in team_controller
---------
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* [MM-37984] Allow Desktop App to authenticate via external providers outside of the app on supported servers
* PR feedback
* Add support for mattermost-dev protocol for development use
* Update server/channels/db/migrations/postgres/000110_create_desktop_tokens.up.sql
* Fix silly typo
* Update server/channels/db/migrations/postgres/000110_create_desktop_tokens.up.sql
* Remove storage of client token, only validate it on the client
* Update migrations
* Add concurrently create index
* Remove CONCURRENTLY for now
* Fix issue with changing history
* Remove old migration
* Use idempotent statement to drop old index
* Remove reference to old table
* Update group_store.go
Replace raw SQL with query builder for Get, Update, and Delete methods in GroupStore
* Update group_store.go
Replace raw SQL with query builder for GetByRemoteID, GetAllBySource, and Restore methods in GroupStore
* Add error handling when using query builder in group_store.go
* Use builder methods to cut down on boilerplate
* Update server/channels/store/sqlstore/group_store.go
Implement suggestion from PR review
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
---------
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Making all the counts aware of Remote users
* Disable login for remote users
* Adding tests for login remote_users error
* Adding tests for the store
* Adding frontend part of not counting remote users in the license
* Addressing PR review comment
* Adding the new ExternaUserId field to users
* Running make migrations-extract
* Running make app-layers and make gen-serialized
* Revert "Adding the new ExternaUserId field to users"
This reverts commit 12e5fd5189.
* Adding GetUserByRemoteID methods
* Adding needed migration for users
* i18n-extract
* Fixing postgres increase remote user id field size migration up and down
* run make gen-serialized
* Removing migration code
* Not count remote users as part of the cloud pricing
* Add the cloud subscription when a user gets promote from remote to not-remote
* Fixing merge problems
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Mark category as read
* Fix lint and test
* Fix tests
* Fix test and remove wrong aria
* Address server issues and add mark as read for unreads
* Missing changes
* Fix tests
* fix tests
* Add confirmation popup to mark as read category
* Always use viewMultipleChannels and other fixes
* Remove unneeded code
* Fix test
* Address feedback
* Address feedback
* Fix tests
* Fix test
* Fix tests
* Update aria-haspopup depending on the number of channels to mark as viewed
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
With the schema version available, a job can query for
the cluster info to confirm whether or not all nodes
in a cluster are upgraded to the same version or not.
This will help it in determining whether to start
the job or not.
https://mattermost.atlassian.net/browse/MM-53228
```release-note
NONE
```
If the store fails to initialize (e.g. run a migration), it would `log.Fatal` and then `os.Exit`. Unfortunately, this trips up `TestMain`, which happily keeps running tests, now guaranteed to fail.
Avoid this by instead returning an error from the store initialization, handling appropriately at the layer above.
We missed this out last time. It's possible in an HA
scenario for a second pod to start later while the other
job is in-progress. In that case, it would schedule
two jobs.
https://mattermost.atlassian.net/browse/MM-53747
```release-note
NONE
```
Bifrost now encodes all image paths. Due to this
one-way translation, we need to encode all the older
image paths as well.
After this is done, we can remove the double-lookup.
https://mattermost.atlassian.net/browse/MM-53747
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* Revert "POC: Cross-team recent search (#20027)"
This reverts commit aa59c28b04, preserving
a few code tidyings unrelated to the original PR.
* Revert "Add feature flag for command palette (#20011)"
This reverts commit c78c5ce3f3.
* add base for calling the endpoint
* add endpoint and handler
* update store and layers
* call the endpoint
* align types
* update app layers
* generate mocks
* complete handler
* finish store query
* add todos
* add ui for member count
* add selector
* add a todo
* add cache layer
* optimize calls in FE
* handle invalidation of the cache
* fix go style
* fix test
* use existing channel layer count
* fix import error
* delete unnecessary code
* write tests for channel cache layer
* fix testname
* fix mocks
* fix cache layer test
* fix a test
* really fix the test
* write more tests for server
* address PR comments
* remove comment
* rename more_channels to browse_channels
* fix style
* update snapshot
* add translations
* Revert "add translations"
This reverts commit 56476a5dab.
* add only related translations
* address PR review points
* add test
* fix test
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
There was already a cache present for emoji names. But we weren't using
it for the GetMultipleByName method. Now we implement that method
to look up the cache for every emoji name passed.
Secondly, a bigger problem was that we were making the DB call for system
emojis as well. Since system emojis aren't stored in the DB, it would
fall through the cache layer and always make a redundant DB call. In the
profiles, this should up as taking 16% of the total time to serve
a getPostsForChannel API endpoint.
We fix this by filtering the emojis to only custom emojis
before making the call.
https://mattermost.atlassian.net/browse/MM-53669
```release-note
NONE
```
* [MM-53192] Patch full name leak in Insights team_members API
* Update server/channels/app/team.go
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
---------
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* WIP
* Add rate limiting for desktop token API
* Missing mocks
* Style fixes
* Update snapshots
* Maybe use an actual redirect link :P
* Refactoring for tests
* Add tests for server
* Fix lint issue
* Fix tests
* Fix lint
* Add front-end screen component
* Component logic
* Style changes
* Quick style fix
* Lint fixes
* Initial PR feedback
* Enable logging into the browser as well when completing the login process
* Refactor to push more logic to the other component
* Remove unnecessary helper code
* Fix i18n
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* update for guest demotion and sysadmin edits
* update unit test
* update unit test
* use existing disabled flag
* remove commented line
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
For MySQL, a query for type LEFT JOIN .. IS NULL
leads to a nested antijoin which leads to poor performance.
We fix this by rewriting the query to avoid the antijoin.
See the JIRA epic for more context behind this.
We also make another improvement to remove the DISTINCT
clause. It didn't serve any purpose since userids would
already be unique.
https://mattermost.atlassian.net/browse/MM-53406
```release-note
NONE
```
It was a good decision in hindsight to keep the public module as 0.x
because this would have been a breaking change again.
https://mattermost.atlassian.net/browse/MM-53032
```release-note
Changed the Go module path from github.com/mattermost/mattermost-server/server/v8 to github.com/mattermost/mattermost/server/v8.
For the public facing module, it's path is also changed from github.com/mattermost/mattermost-server/server/public to github.com/mattermost/mattermost/server/public
```
* mark thread files as deleted
* add missing check
* improve query
* Stopped rendering post preview if post is deleted
* Fixed lint error
* Fixed test
* updated types
* Removed deleted post from other post's embed data
* Added tests
* Apply suggestions from code review
Co-authored-by: Daniel Espino García <larkox@gmail.com>
* lint fix
---------
Co-authored-by: Konstantinos Pittas <konstantinos.pittas@mattermost.com>
Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Harshil Sharma <18575143+harshilsharma63@users.noreply.github.com>
Co-authored-by: Daniel Espino García <larkox@gmail.com>
* MM-46410: adds urgency on mention counts
We have introduced priority for posts in
https://github.com/mattermost/mattermost-webapp/pull/10951.
We do need to color the mention badges in the webapp with a prominent
color when a mention is posted in an urgent message.
A thread has urgent mentions if the root post is marked as urgent, and
the replies contain mentions to the user viewing the thread.
This PR adds two columns, urgentmentioncount, and isurgent, in
channelmembers, and threads tables respectively.
Furthermore when asking for team/thread mention counts, we also return
urgent mention counts for the user.
* Adds PostAcknowledgements table and apis
* job init and fetch mentions
* add-migrations
* delete-expired
* send-notifications
* Fetches post priority in batches
* stop-notifications
* stop-notification-on-reply
* MM-47750: Adds PostAcknowledgements table and apis
- Adds post acknowledgement api/app/store methods to be able to save and
delete post acknowledgements by users.
- Adds wesbsocket events for acknowledgement created/deleted
- Returns post acknowledgements in the post's metadata
* add-license-check
* add-pagination
* delete on channel and team
* validate guests
* add configs
* move create priority post check from app to api
* Add desktop notifications
* check status
* use config in job
* add IsUrgent check
* Add last-sent-at
* validate max recipients
* Update lastSentAt
* Validate min. recipient
* send email notification only once
* remove email notifications
* use latest time from config to run job
* Add notifications counter
* publish events to mentioned users only
* pickup license updates in scheduler
* don't allow post owner to stop notifications
* follow normal notifications behaviour
* Validates persistent notifications interval
* move logic of handling valid and expired posts into sql
* Adds persistent notifications in the webapp
---------
Co-authored-by: koox00 <3829551+koox00@users.noreply.github.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Regenerate default profile picture if username has changed
- Only actions is profile picture has not been changed
- Adjusts ResetLastPictureUpdate store function to store
-curTime instead of 0
- This is to support updating the default picture while still
retaining the ability to discern a default image from a set one.
- Changes SetDefaultProfileImage to leverage UpdateDefaultProfileImage
- Test updates around updating user default profile pictures
* App interface updates
* Only display picture update date if non-negative
- Ensures we don't display negative timestamps (default images)
- Change ported for mono-repo changes
* Remove duplicate test assertion
---------
Co-authored-by: Nathan Geist <ngeist@spiria.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-50550: Filter out threads from "left" channels v2
Currently leaving a channel doesn't affect the thread memberships of
that user/channel combination.
This PR aims to filter out all threads from those channels for the user.
Adds a DeleteAt column in the ThreadMemberships table, and filter out
all thread memberships that are "deleted".
Each time a user leaves a channel all thread memberships are going to be
marked as deleted, and when a user joins a channel again all those
existing thread memberships will be re-instantiated.
Adds a migration to mark all existing thread memberships as deleted
depending on whether there exists a channel membership for that
channel/user.
* Added migration files into list
* Fixes tests
* Fixes case where DeleteAt would be null
* Guard thread API endpoints with appropriate perms
* Deletes ThreadMembership rows upon leaving channel
* Minor style changes
* Use NoTranslation error
* Refactors tests
* Adds API tests to assert permissions on Team
* Adds tests, and fixes migrations
* Fixes test description
* Fix test
* Removes check on DM/GMs
* Change the MySQL query in the migration
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
We monitor the health of DB replicas, and on a fatal error,
take them out of the pool.
On a separate goroutine, we keep pinging the unhealthy replicas,
and on getting a good response back, we add them back to the pool.
https://mattermost.atlassian.net/browse/MM-50427
```release-note
Mattermost is now resilient against DB replica outages and will
dynamically choose a replica if it's alive.
Also added a config parameter ReplicaMonitorIntervalSeconds
whose default value is 5. This controls how frequently unhealthy
replicas will be monitored for liveness check.
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* Enable products for channels tests
* increase unit test timeout; check IsConfigReadOnly
* make app-layers
* Avoid loading boards tempaltes between tests to improve speed
* Fix delete query to be compatible with both databases
* Avoid preserving the templates for boards store tests
* Run all tests in one command
* Revert "Run all tests in one command"
This reverts commit 0330f7cd8f.
* concurrent pkg group tests in CI
* Revert "Revert "Run all tests in one command""
This reverts commit 73892fec77.
* Revert "concurrent pkg group tests in CI"
This reverts commit 550fb6cdd4.
* try testing 3 subsets of packages concurrently to improve time taken
* Revert "try testing 3 subsets of packages concurrently to improve time taken"
This reverts commit 97475f3c4e.
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: wiggin77 <wiggin77@warpmail.net>
https://mattermost.atlassian.net/browse/MM-52079
```release-note
We upgrade the module version to 8.0. The new module path is github.com/mattermost-server/server/v8.
```
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
* MM-49564: Upsert in the Store vs App layer
Refactor drafts so that Upserting a draft would happen in the DB and not
in the app layer.
* Fixes mocks
* Fixes tests
* Fixes translations
* Fixes tests
* Update tests
* Fixes tests
* Addresses review comments
- renames Save => Upsert
- removes Sleep from tests
* Fixes flaky test
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-45956: Optimize FileInfo stats query
We Denormalize Post.ChannelId on FileInfo.ChannelId
```release-note
The file info stats query is now optimized by denormalizing the channelID column into the table itself. This will speed up the query to get the file count for a channel on clicking the RHS.
Migration times:
On a MySQL 8.0.31 DB with
1405 rows in FileInfo and 11M posts, it took around 0.3s
On a Postgres 12.14 DB with
1731 rows in FileInfo and 11M posts, it took around 0.27s
```
https://mattermost.atlassian.net/browse/MM-45956
