Having cache layer as part of the store call
Prometheus metrics can be somewhat misleading
because cache-backed methods can have very high
volume and yet not cause any load on the DB at all.
However, if the admin is not aware which calls
are cache-backed and which are not, then
it can be hard to understand why a store call
of high volume isn't causing any stress on the DB at all.
Therefore, for simplicity we reorder the layers
such that cache layer is handled outside the timer layer.
We already have cache related metrics like hit/miss
which can be used to monitor cache usage.
https://mattermost.atlassian.net/browse/MM-53058
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* Remove build references
* Remove playbooks webapp and server, and add the prepackaged plugin
* Remove translations
* Add ProductSettings to the playwright type
* Restore playbooks as a prepackaged plugin for cypress e2e tests
https://mattermost.atlassian.net/browse/MM-52532
- Replace golint with revive
- Add makezero linter
- Fix all the required linter failures
Some issues in enterprise and public modules
are yet to be fixed. We send this to expediate things.
It was a good decision in hindsight to keep the public module as 0.x
because this would have been a breaking change again.
https://mattermost.atlassian.net/browse/MM-53032
```release-note
Changed the Go module path from github.com/mattermost/mattermost-server/server/v8 to github.com/mattermost/mattermost/server/v8.
For the public facing module, it's path is also changed from github.com/mattermost/mattermost-server/server/public to github.com/mattermost/mattermost/server/public
```
* disable coverage
This reduces runtime of the server test suite from ~30m to ~10m, and as far as I can see: we discarded the coverage output anyway.
* allow morph 60s to migrate when running tests
* fix fileutils.TestFindFile on MacOS
* introduce model.ExternalServiceEnvironment
* pick license public key from external service env
* pick Stripe public key from external service env
* pick Rudder key from external service env
* configure Sentry DSN from external service env
* always log external_service_environment, Unsetenv
* clear faked BuildEnv, improve logging
* strip out unset GOTAGS
* fix Sentry tests
* simplify to just ServiceEnvironment
* relocate ServiceEnvironment in client config
* initialize CWS URLs based on service environment
* unset rudder key for boards dev
* harden service environment to avoid accidental production
* fix TestSentry again
* fix DEFAULT -> ENTERPRISE
* s/dev/test when naming playbooks rudder key
* simplify boards rudder key switch
* use uniform rudderKey variable names
* retain compatibility with existing pipeline
* reduce to just production/test
* unit test with valid test license
* simplify Playbooks telemetry initialization
* restore dev service environment
* emit ServiceEnvironment when running e2e tests
* Migrate all method in model/client4.go to accept a context.Context
* Fix th.*Client
* Fix remaining issues
* Empty commit to triger CI
* Fix test
* Add cancellation test
* Test that returned error is context.Canceled
* Fix bad merge
* Update mmctl code
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* create config setting to enable/disable playbooks product
* fix to config name
* fix typo
* revert changes to package-lock.json
* update name of test
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* mark thread files as deleted
* add missing check
* improve query
* Stopped rendering post preview if post is deleted
* Fixed lint error
* Fixed test
* updated types
* Removed deleted post from other post's embed data
* Added tests
* Apply suggestions from code review
Co-authored-by: Daniel Espino García <larkox@gmail.com>
* lint fix
---------
Co-authored-by: Konstantinos Pittas <konstantinos.pittas@mattermost.com>
Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Harshil Sharma <18575143+harshilsharma63@users.noreply.github.com>
Co-authored-by: Daniel Espino García <larkox@gmail.com>
* Avoid calling the user count query in future if we get a count > 0
* re-adding mock session to avoid adding the old mitigation in future
* adjustments based on feedback
* MM-46410: adds urgency on mention counts
We have introduced priority for posts in
https://github.com/mattermost/mattermost-webapp/pull/10951.
We do need to color the mention badges in the webapp with a prominent
color when a mention is posted in an urgent message.
A thread has urgent mentions if the root post is marked as urgent, and
the replies contain mentions to the user viewing the thread.
This PR adds two columns, urgentmentioncount, and isurgent, in
channelmembers, and threads tables respectively.
Furthermore when asking for team/thread mention counts, we also return
urgent mention counts for the user.
* Adds PostAcknowledgements table and apis
* job init and fetch mentions
* add-migrations
* delete-expired
* send-notifications
* Fetches post priority in batches
* stop-notifications
* stop-notification-on-reply
* MM-47750: Adds PostAcknowledgements table and apis
- Adds post acknowledgement api/app/store methods to be able to save and
delete post acknowledgements by users.
- Adds wesbsocket events for acknowledgement created/deleted
- Returns post acknowledgements in the post's metadata
* add-license-check
* add-pagination
* delete on channel and team
* validate guests
* add configs
* move create priority post check from app to api
* Add desktop notifications
* check status
* use config in job
* add IsUrgent check
* Add last-sent-at
* validate max recipients
* Update lastSentAt
* Validate min. recipient
* send email notification only once
* remove email notifications
* use latest time from config to run job
* Add notifications counter
* publish events to mentioned users only
* pickup license updates in scheduler
* don't allow post owner to stop notifications
* follow normal notifications behaviour
* Validates persistent notifications interval
* move logic of handling valid and expired posts into sql
* Adds persistent notifications in the webapp
---------
Co-authored-by: koox00 <3829551+koox00@users.noreply.github.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
I mistakenly assumed that those packages are naturally imported.
And also atleast one test would have broken.
None of them are true. :)
So we just import it again at a better place
This reverts commit 30a053314b.
```release-note
NONE
```
* platform/web_conn: fix a possible data race issue
* reflect review comments
---------
Co-authored-by: Ibrahim Serdar Acikgoz <ibrahim@ibrahims-mac.local>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Regenerate default profile picture if username has changed
- Only actions is profile picture has not been changed
- Adjusts ResetLastPictureUpdate store function to store
-curTime instead of 0
- This is to support updating the default picture while still
retaining the ability to discern a default image from a set one.
- Changes SetDefaultProfileImage to leverage UpdateDefaultProfileImage
- Test updates around updating user default profile pictures
* App interface updates
* Only display picture update date if non-negative
- Ensures we don't display negative timestamps (default images)
- Change ported for mono-repo changes
* Remove duplicate test assertion
---------
Co-authored-by: Nathan Geist <ngeist@spiria.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
We strengthen some checks, forcing them to be changed
only via config.
https://mattermost.atlassian.net/browse/MM-37691
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* [MM-47492] - Don't count system messages as replies to a thread
* Add test assertion
* revert store changes`
---------
Co-authored-by: Nevyana Angelova <nevyangelova@Nevyanas-MacBook-Pro.local>
Co-authored-by: Mattermost Build <build@mattermost.com>
* WIP
* Fetched ES index for type checking
* WIP
* Moved migration key to model
* CI
* CI
* Fixed existing tests
* Updated einterface mocks
* CI
* WIP
* Reverted test makefile changes
* Fixed error level
* CI
* Not attempting to launch job if not enterprise ready
* CI
* Fixing job trigger condition
* CI
* Updated eemocks
* CI
* MM-50960 - store system organization name
* restore the preparing workspace plugins and invite screens
* add back the page lines for the design
* add lines back and organize styles
* set back documentation to monorepo style and disable board as a product
* fix organization link and style skip button
* create team on organization name screen continue button click
* make sure there are not already created team and if so just update team name
* update the team display name if team has already been created
* cover error scenarios during team creation
* add pr feedback and add a couple of unit tests
* fix translation server error; make sure only update display name if it has changed in the form
* temp advances
* rewrite unit tests using react-testing library; fix unit tests
* fix translations
* make sure the launching workspace finish in cloud installations
* remove redundant validation
* fix unit tests
* remove unintended config value left after merge conflict
* MM-50550: Filter out threads from "left" channels v2
Currently leaving a channel doesn't affect the thread memberships of
that user/channel combination.
This PR aims to filter out all threads from those channels for the user.
Adds a DeleteAt column in the ThreadMemberships table, and filter out
all thread memberships that are "deleted".
Each time a user leaves a channel all thread memberships are going to be
marked as deleted, and when a user joins a channel again all those
existing thread memberships will be re-instantiated.
Adds a migration to mark all existing thread memberships as deleted
depending on whether there exists a channel membership for that
channel/user.
* Added migration files into list
* Fixes tests
* Fixes case where DeleteAt would be null
* Guard thread API endpoints with appropriate perms
* Deletes ThreadMembership rows upon leaving channel
* Minor style changes
* Use NoTranslation error
* Refactors tests
* Adds API tests to assert permissions on Team
* Adds tests, and fixes migrations
* Fixes test description
* Fix test
* Removes check on DM/GMs
* Change the MySQL query in the migration
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
We monitor the health of DB replicas, and on a fatal error,
take them out of the pool.
On a separate goroutine, we keep pinging the unhealthy replicas,
and on getting a good response back, we add them back to the pool.
https://mattermost.atlassian.net/browse/MM-50427
```release-note
Mattermost is now resilient against DB replica outages and will
dynamically choose a replica if it's alive.
Also added a config parameter ReplicaMonitorIntervalSeconds
whose default value is 5. This controls how frequently unhealthy
replicas will be monitored for liveness check.
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* Enable products for channels tests
* increase unit test timeout; check IsConfigReadOnly
* make app-layers
* Avoid loading boards tempaltes between tests to improve speed
* Fix delete query to be compatible with both databases
* Avoid preserving the templates for boards store tests
* Run all tests in one command
* Revert "Run all tests in one command"
This reverts commit 0330f7cd8f.
* concurrent pkg group tests in CI
* Revert "Revert "Run all tests in one command""
This reverts commit 73892fec77.
* Revert "concurrent pkg group tests in CI"
This reverts commit 550fb6cdd4.
* try testing 3 subsets of packages concurrently to improve time taken
* Revert "try testing 3 subsets of packages concurrently to improve time taken"
This reverts commit 97475f3c4e.
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: wiggin77 <wiggin77@warpmail.net>
https://mattermost.atlassian.net/browse/MM-52079
```release-note
We upgrade the module version to 8.0. The new module path is github.com/mattermost-server/server/v8.
```
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
This will be shown in the system console to let users know
which search backend is active.
Implemented via adding an extra param in the ping response.
https://mattermost.atlassian.net/browse/MM-49984
```release-note
The database section in the system console now has an additional
read-only section which shows the active search backend in use.
This can be helpful to confirm which is the currently active
search engine when there are multiple of them configured.
```
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
We would return "nil, nil" if a plugin was on the blocklist.
This would cause a nil dereference panic while trying to add
struct fields from manifest to the audit logs.
To avoid it, we return an error explicitly and ignore the
error ids to unnecessarily have it log an error.
https://mattermost.atlassian.net/browse/MM-51786
```release-note
NONE
```
After the monorepo integration, the boards product
was started before we started the hub.
This led to a data race where the GetHubForUserId was actually
called before the hub was initialized.
To fix this we initialize the platform first, and then the products.
https://mattermost.atlassian.net/browse/MM-52009
```release-note
NONE
```
Mockery started to throw errors like "Unexpected package creation during export data loading".
This was consistently reproducible locally and on upgrade, it has gone away.
```release-note
NONE
```
* bind to localhost:0 for tests
Explicitly bind to `localhost:0` instead of just `:0` to avoid binding to all available interfaces and in turn avoid triggering firewall warnings on MacOS.
* fix Playbooks to use dynamic port
* TestMetrics: handle ipv4 localhost too
* TestMetrics: linting
* MM-49564: Upsert in the Store vs App layer
Refactor drafts so that Upserting a draft would happen in the DB and not
in the app layer.
* Fixes mocks
* Fixes tests
* Fixes translations
* Fixes tests
* Update tests
* Fixes tests
* Addresses review comments
- renames Save => Upsert
- removes Sleep from tests
* Fixes flaky test
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* remove Boards feature flag from server, makefile, and docker
* Update server/boards/server/boards_service_util.go
Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
---------
Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
* Add new trial form to enrich trial requests with more customer info
* Update snapshots
* One more addition
* Fixes from PR feedback
* Fix types, i18n, update e2e tests
* Fix linter
* Fix i18n?
* Fix blank translation
* update snapshot
* Update snapshot properly
* Remove inapplicable test
* Fix business email validation only happening once
* UX Feedback
* Fix linter
* Fix linter again, not working locally
* FIX LINTER
* Move isvalid check until after some fields are set
* Fix for overlapping modals
* Fix linter
* UX feedback
* UX Feedback
* Fix typo in error modal
* [MM-51551] Add new Trial Form to Playbooks trial requests (#22650)
* Playbooks start trial entrypoints opens new trial form modal
* Fix style
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* [MM-51347] Trial form error modal for embargoed and air gapped entities (#22656)
* Playbooks start trial entrypoints opens new trial form modal
* Add support for air gapped environments when making trial requests
* Add specific handling for embargoed entities
* undo some code
* Fix linter
* Fix types
* Fix style
* Updates because TE has to upgrade to E0 before it can activate a trial
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
We log warnings whenever our websocket buffer sizes exceed
certain thresholds. The problem with that is, when this happens,
the logs are completely spammed with these lines
making it annoying for the customer.
To improve the situation, we use a timer that only gets reset
every minute.
https://mattermost.atlassian.net/browse/MM-51700
```release-note
NONE
```
* Revert "fix store issue take two"
This reverts commit 59f943c2c7.
* Revert "fix store override issue"
This reverts commit 29c346757a.
* Revert "Fix TestPushNotificationRace"
This reverts commit 6d62dddf86.
* Revert "fix default DSN for CI"
This reverts commit e0e69cdbb0.
* Revert "disable playbooks from more unit tests"
This reverts commit a1e97a9e96.
* Revert "disable playbooks for more tests"
This reverts commit 4d2dc74f05.
* Revert "disable playbooks for TestSAMLSettings"
This reverts commit 35c1a4312d.
* Revert "disable playbooks for more unit tests"
This reverts commit c049631a14.
* Revert "disable playbooks for mocked enterprise tests"
This reverts commit 829317fddb.
* Partially revert "disable playbooks for channel/apps mocked tests"
This reverts commit 52b4a0a6cf.
* Revert "fix TestUnitUpdateConfig"
This reverts commit 8f134f2a8a.
* Revert "add plugin mock to TestUnitUpdateConfig"
This reverts commit 3ec5419092.
* Revert "disable Boards for more test helpers"
This reverts commit 5d4d0d02d9.
* Revert "disable boards at correct place in test helpers"
This reverts commit 0c9e175f79.
* Partially revert "disable boards for slash cmd tests"
This reverts commit fad8d9de93.
* Revert "disable Boards for channels web tests"
This reverts commit 15540fdfc0.
* Revert "Adds a teardown function to playbook server tests to disable and reenable boards"
This reverts commit 9a46e3d0f4.
* Revert "Test disable boards through feature flag"
This reverts commit 787044add8.
* TestUnitUpdateConfig: restore callback check
* Revert "Revert "fix default DSN for CI""
This reverts commit 01b879d55a.
Spawn a goroutine from the server layer to have better
control for compliance tests.
https://mattermost.atlassian.net/browse/MM-51504
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-45956: Optimize FileInfo stats query
We Denormalize Post.ChannelId on FileInfo.ChannelId
```release-note
The file info stats query is now optimized by denormalizing the channelID column into the table itself. This will speed up the query to get the file count for a channel on clicking the RHS.
Migration times:
On a MySQL 8.0.31 DB with
1405 rows in FileInfo and 11M posts, it took around 0.3s
On a Postgres 12.14 DB with
1731 rows in FileInfo and 11M posts, it took around 0.27s
```
https://mattermost.atlassian.net/browse/MM-45956
There is a standard IDNA2008 which defines how unicode characters
in domain names can be compared using UTS (Unicode Technical Standard) 46.
We use that to convert any links in messages to ASCII and
then compare with the domain list in config.
https://mattermost.atlassian.net/browse/MM-51486
```release-note
NONE
```
