* Limit parallelism for full E2E tests
* Add temporary changes for testing
* Better concurrency spec
* Do not consider github.ref for concurrency grouping
* Further improve the concurrency grouping
* Remove testing lines
* Move license setting from env var to mmctl upload
* Extract common E2E steps
* Add E2E fulltests
* Fix plugin_startup_fail_spec.js, timeout runners, fix local dashboard run
* Implement reporting script and job
* Bump artifact related actions
* Fix E2E variable generator script
* Skip reporting steps if not required
* Get rid of deprecation warnings
* Fix inbucket hostname parameter
* Support arbitrary refs in test template
* Fix cycle BRANCH var for non-PR commits
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
After moving only contributor PRs to free tier it was
observed that the tests actually finish faster in free tier
than with larger runners. We have let the change soak for about
a month now without any issues.
Therefore we move to free tier runners for all PRs.
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
MySQL 5.7 is at end of life.
https://mattermost.atlassian.net/browse/MM-55589
```release-note
We bump up minimum MySQL version to be 8.0.0
```
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Simplify workflow failure check
Consolidate the workflow failure check directly into *-master.yml. This results in some code duplication, but makes it much clearer as to what's going on. While we're in here, remove the legacy reference to the `monorepo` branch when triggering.
* just remove the workflow in favour of the Argo version
* fix openApi vetting
The underlying mattermost-govet tool effectively hasn't been called for some time, as we weren't checking out and building the spec files to pass. Now that hte API is in the monorepo, build it locally and check against it.
Unfortunately, our API documentation isn't up-to-date, and this PR isn't fixing that. For now, add a discrete `make vet-api` and workflow that won't block the build until the API documentation is back in sync and can be merged into the existing `make vet` directive.
* mattermost-govet: use upstream@new
* fix missing /api/v4 prefix for commands autocomplete suggestion
* document /api/v4/ldap/users/{user_id}/group_sync_memberships
* document /api/v4/groups/{group_id}/restore
* fix /files/{file_id}/public actually at root
* document /api/v4/users/invalid_emails
* fix SetThreadUnreadByPostId
* Revert "fix SetThreadUnreadByPostId"
This reverts commit b16bcc8044.
* Revert "Revert "fix SetThreadUnreadByPostId""
This reverts commit 8bda05dc8a.
* workaround undocumented API endpoints
* remove unnecessary whiteline
* ignore go tool output
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Only the -race tests in master branch takes
more memory. So it will take more time, but there's no
need to optimize run time in master as nobody
is waiting for it to complete.
```release-note
NONE
```
Exporting a variable doesn't really pass that
down to the docker container. We pass the `--env`
variable manually.
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* Optimize mmctl CI run
- Remove coverage mode. They increase run time by 3x.
- Remove -race mode in PR builds. They increase run time by 2x and memory consumption as well.
```release-note
NONE
```
* caching go.sum path
```release-note
NONE
```
* Improve e2etests readme, remove unused apt install
* Add testing E2E Tests MultiOS
* Get rid of deprecation warning
* Smart selection of docker network driver for different OSes
* Unify makefile invocation for windows as well
* Tune JVM for ES and Keycloak
* fix: Made required changes to run locally on arm MacOS
* fix: FIxed network command and removed dashboard node user as it is not needed
* fix: make dashboard work for Mac
* Make runs without dashboard work again, make BROWSE tunable and document it
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Antonis Stamatiou <stamatiou.antonis@gmail.com>
* Constrain when we emit webhooks for retried tests
Only send webhooks (notifying all staff developers) only for retried tests on `master` and `release-*`.
* drop @devs at-mention
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
GitHub actions' YAML supports the `>` directive to a multi-line command to be merged into a single line, but requires that all segments have no leading whitespace otherwise they get silently ignored.
Since this is not nearly as obvious as one might expect, fallback to just the `|` syntax using `\` to extend the lines and keep the overall formatting, fixing the reporting of retried tests via curl webhook.
* drop public submodule replace directive
During development of the public/ submodule, a replace directive remained that was both unnecessary and harmful as discussed in https://community.mattermost.com/private-core/pl/w77sh7igwpfb9ecj5o4jjjbbyo.
Remove that, and bump the explicit dependency (even though we use go.work) to v0.0.6 so the import paths match (e.g. `mattermost` vs `mattermost-server`).
* make modules-tidy
* consistently setup-go-work
* fix build-api-spec
* fix mmctl test template
* fix workflow copy/paste typo
* disable coverage
This reduces runtime of the server test suite from ~30m to ~10m, and as far as I can see: we discarded the coverage output anyway.
* allow morph 60s to migrate when running tests
* scripts/test.sh: drop COVERMODE
Stop generating coverage data when running unit tests. It's likely we'll want this data back at some point, but for now it's unused and removing simplifies invoking tests for developers.
* scripts/test.sh: remove cleanup steps
* scripts/test.sh: drop TESTS parameter
* scripts/test.sh: drop TESTFLAGS parameter
* switch to gotestsum
* fix: Fix proper status update since job.status depicts current job status which is always success
* Rename status check for smoketests
* Allow localhost origin for local testing
* Increase instance size for smoketest job, better status check name
---------
Co-authored-by: Antonis Stamatiou <stamatiou.antonis@gmail.com>
* Includes mmctl into the mono-repo
* Update to use the new public module paths
* Adds docs check to the mmctl CI
* Fix public utils import path
* Tidy up modules
* Fix linter
* Update CI tasks to use the new file structure
* Update CI references
* Do not use the mattermost path
The /mattermost path is used by the image and it seems to conflict with
the container running the binary
* Use a regular machine in the esr-upgrade-diff job
* Add missing space
* MM-52438 Only run web app CI for PRs with web app changes
* MM-52438 Don't wait to run web app tests in CI
* MM-52438 Remove commented out caching
* MM-52438 Reuse steps to set up webapp environment
* MM-52438 Add test-ci script to root package.json
* Simplify path filter
We discovered that cancel-in-progress only controls
in-progress jobs. Which means that pending jobs will _always_
be cancelled regardless. There is an open discussion:
https://github.com/orgs/community/discussions/5435
which was closed saying this is how the feature is designed.
We try to work around this by refactoring into separate reusable
workflows and having concurrency only for PR workflows.
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
Mattermost build was a confusingly named workflow. Changed it
to Server CI to make it consistent with Webapp CI.
Also changed the filenames.
```release-note
NONE
```
* Run golangci in server CI
Also reordered the checks in check-style target to run vet check
first which has far better error messages in case of unsynced branch failures.
```release-note
NONE
```
We are disabling uploading e2e artifacts to mattermost dockerhub repository.
This was temporary, to help with the migration to the new repo, of all sourounding services.
Ticket: https://mattermost.atlassian.net/browse/CLD-5614
* Add ESR upgrade migration and CI job to verify it
The script was generated as a simple concatenation of migrations in the
interval [54, 101] through:
files=`for i in $(seq 54 101); do ls mysql/$(printf "%06d" $i)*up.sql; done`
tail -n +1 $files > ../esrupgrades/esr.5.37-7.8.mysql.up.sql
The CI job runs the migration both through the server and the script,
and for now uploads the dumps generated for manual inspection. An
automatic check for differences is still needed.
* Remove debug print in script
* Fix idx_uploadsessions_type creation
* Ignore tables db_lock and db_migration on dump
* Split workflow in two parallel jobs
* Diff dumps and upload the result
* Add cleanup script
* Use DELIMITER in the script to use mysql CLI
This allows us to remove the complexity of using a different Go script
inside a Docker image.
* Standardize Roles between migrations
Document and cleanup code.
* Upload diff only if it is not empty
* Trigger action only when related files change
* Add a global timeout to the job
* Generalize ESR to ESR upgrade action (#22573)
* Generalize action
* Use logs to ensure migrations are finished
* Add migrations from 5.37 to 6.3
* Remove tables in cleanup script, not through dump
* Add initial-version input to common action
* Add migration from 6.3 to 7.8
* Remove action debug line
* ESR Upgrade: One procedure per table in the v5.37 > v7.8 upgrade script (#22590)
* Squash Users-related migrations in one query
* Squash Drafts-related migrations in one query
* Squash UploadSessions-related migrations in one query
* Squash Threads-related migrations in one query
* Squash Channels-related migrations in one query
* Squash ChannelMembers-related migrations in one query
* Squash Jobs-related migrations in one query
* Squash Sessions-related migrations in one query
* Squash Status-related migrations in one query
* Squash Posts-related migrations in one query
* Squash TeamMembers-related migrations in one query
* Squash Schemes-related migrations in one query
* Squash CommandWebhooks-related migrations in one query
* Squash OAuthApps-related migrations in one query
* Squash Teams-related migrations in one query
* Squash Reactions-related migrations in one query
* Squash PostReminders-related migrations in one query
* Adapt ThreadMemberships migration to unified style
* Adapt LinkMetadata migrations to unified style
* Adapt GroupChannels migration to unified style
* Adapt PluginKVStore migration to unified style
* Adapt UserGroups migration to unified style
* Adapt FileInfo migration to unified style
* Adapt SidebarCategories migration to unified style
* Remove blank line
* Use tabs everywhere
* Wrap every procedure with log statements
* Remove space before parentheses in procedure call
* Remove spurious extra line
* Merge two equal consecutive conditionals
* Avoid the double list of conditions/queries
* Fix variable name
* Remove outdated comment
* Add a preprocess phase with corresponding scripts
* Join all preprocess scripts setting ExpiresAt to 0
This preprocessing is something we should always do, no matter the input
DB, so we can use a common script for all cases instead of repeating the
same code in multiple files.
* Add system-bot if it does not exist
* Cleanup the ProductNoticeViewState table
* Fix SQL
* Move esrupgrades directory under server/
* Update paths in Github action
* Fix trigger path for CI
We are defining the workflow directive to for `Artifacts generation and upload`/cd/Download artifacts step.
To be the workflow_id from the calling workflow
Signed-off-by: Akis Maziotis <akis.maziotis@mattermost.com>
* Run tests for all web app packages in CI
* Test specific packages instead of all of them
* Revert back to using test-ci scripts and rely on Jest to count the number of CPU cores
* Revert more changes made to the web app CI tests
* Fix path to client package tests
* Fix incorrect usage of types pacakge
* Make platform packages properly expose their type definitions
* Have Boards, Channels, and Playbooks get types from platform packages properly
* Run type check for all web app packages in CI
* Fix incorrect job name
* Running CodeQL everyday twice
* Updated scorecard-check to run everyday
* Running Scorecard job once a day
* Disabling Scorecard Job on every PR
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
