certbot/tools/venv3.py

31 lines
724 B
Python
Raw Permalink Normal View History

#!/usr/bin/env python3
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
# Developer virtualenv setup for Certbot client
import sys
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
import _venv_common
def create_venv(venv_path):
"""Create a Python 3 virtual environment at venv_path.
:param str venv_path: path where the venv should be created
"""
python3 = _venv_common.find_python_executable(3)
command = [python3, '-m', 'venv', venv_path]
_venv_common.subprocess_with_print(command)
def main(pip_args=None):
venv_path = _venv_common.prepare_venv_path('venv3')
create_venv(venv_path)
if not pip_args:
Refactor certbot/ and certbot/tests/ to use the same structure as the other packages (#7544) Summary of changes in this PR: - Refactor files involved in the `certbot` module to be of a similar structure to every other package; that is, inside a directory inside the main repo root (see below). - Make repo root README symlink to `certbot` README. - Pull tests outside of the distributed module. - Make `certbot/tests` not be a module so that `certbot` isn't added to Python's path for module discovery. - Remove `--pyargs` from test calls, and make sure to call tests from repo root since without `--pyargs`, `pytest` takes directory names rather than package names as arguments. - Replace mentions of `.` with `certbot` when referring to packages to install, usually editably. - Clean up some unused code around executing tests in a different directory. - Create public shim around main and make that the entry point. New directory structure summary: ``` repo root ("certbot", probably, but for clarity all files I mention are relative to here) ├── certbot │   ├── setup.py │   ├── certbot │   │   ├── __init__.py │   │   ├── achallenges.py │   │   ├── _internal │   │   │   ├── __init__.py │   │   │   ├── account.py │   │   │   ├── ... │   │   ├── ... │   ├── tests │   │   ├── account_test.py │   │   ├── display │   │   │   ├── __init__.py │   │   │   ├── ... │   │   ├── ... # note no __init__.py at this level │   ├── ... ├── acme │   ├── ... ├── certbot-apache │   ├── ... ├── ... ``` * refactor certbot/ and certbot/tests/ to use the same structure as the other packages * git grep -lE "\-e(\s+)\." | xargs sed -i -E "s/\-e(\s+)\./-e certbot/g" * git grep -lE "\.\[dev\]" | xargs sed -i -E "s/\.\[dev\]/certbot[dev]/g" * git grep -lE "\.\[dev3\]" | xargs sed -i -E "s/\.\[dev3\]/certbot[dev3]/g" * Remove replacement of certbot into . in install_and_test.py * copy license back out to main folder * remove linter_plugin.py and CONTRIBUTING.md from certbot/MANIFEST.in because these files are not under certbot/ * Move README back into main folder, and make the version inside certbot/ a symlink * symlink certbot READMEs the other way around * move testdata into the public api certbot zone * update source_paths in tox.ini to certbot/certbot to find the right subfolder for tests * certbot version has been bumped down a directory level * make certbot tests directory not a package and import sibling as module * Remove unused script cruft * change . to certbot in test_sdists * remove outdated comment referencing a command that doesn't work * Install instructions should reference an existing file * update file paths in Dockerfile * some package named in tox.ini were manually specified, change those to certbot * new directory format doesn't work easily with pyargs according to http://doc.pytest.org/en/latest/goodpractices.html#tests-as-part-of-application-code * remove other instance of pyargs * fix up some references in _release.sh by searching for ' . ' and manual check * another stray . in tox.ini * fix paths in tools/_release.sh * Remove final --pyargs call, and now-unnecessary call to modules instead of local files, since that's fixed by certbot's code being one layer deeper * Create public shim around main and make that the entry point * without pyargs, tests cannot be run from an empty directory * Remove cruft for running certbot directly from main * Have main shim take real arg * add docs/api file for main, and fix up main comment * Update certbot/docs/install.rst Co-Authored-By: Brad Warren <bmw@users.noreply.github.com> * Fix comments in readthedocs requirements files to refer to current package * Update .[docs] reference in contributing.rst * Move plugins tests to certbot tests directory * add certbot tests to MANIFEST.in so packagers can run python setup.py test * move examples directory inside certbot/ * Move CHANGELOG into certbot, and create a top-level symlink * Remove unused sys and logging from main shim * nginx http01 test no longer relies on certbot plugins common test
2019-11-25 17:28:06 -05:00
pip_args = _venv_common.REQUIREMENTS + ['-e certbot[dev3]']
_venv_common.install_packages(venv_path, pip_args)
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
Update tools/venv3.py to support py launcher on Windows (#6493) Following some inconsistencies occurred during by developments, and in the light of #6508, it decided to wrote a PR that will take fully advantage of the conversion from bash to python to the development setup tools. This PR adresses several issues when trying to use the development setup tools (`tools/venv.py` and `tools/venv3.py`: * on Windows, `python` executable is not always in PATH (default behavior) * even if the option is checked, the `python` executable is not associated to the usually symlink `python3` on Windows * on Windows again, really powerful introspection of the available Python environments can be done with `py`, the Windows Python launcher * in general for all systems, `tools/venv.py` and `tools/venv3.py` ensures that the respective Python major version will be used to setup the virtual environment if available. * finally, the best and first candidate to test should be the Python executable used to launch the `tools/venv*.py` script. It was not relevant before because it was shell scripts, but do it is. The logic is shared in `_venv_common.py`, and will be called appropriately for both scripts. In priority decreasing order, python executable will be search and tested: * from the current Python executable, as exposed by `sys.executable` * from any python or pythonX (X as a python version like 2, 3 or 2.7 or 3.4) executable available in PATH * from the Windows Python launched `py` if available Individual changes were: * Update tools/venv3.py to support py launcher on Windows * Fix typo in help message * More explicit calls with space protection * Complete refactoring to take advantage of the python runtime, and control of the compatible version to use.
2018-11-15 18:17:36 -05:00
[Windows|Unix] Rewrite bash scripts for tests into python (#6435) Certbot relies heavily on bash scripts to deploy a development environment and to execute tests. This is fine for Linux systems, including Travis, but problematic for Windows machines. This PR converts all theses scripts into Python, to make them platform independant. As a consequence, tox-win.ini is not needed anymore, and tox can be run indifferently on Windows or on Linux using a common tox.ini. AppVeyor is updated accordingly to execute tests for acme, certbot and all dns plugins. Other tests are not executed as they are for Docker, unsupported Apache/Nginx/Postfix plugins (for now) or not relevant for Windows (explicit Linux distribution tests or pylint). Another PR will be done on certbot website to update how a dev environment can be set up. * Replace several shell scripts by python equivalent. * Correction on tox coverage * Extend usage of new python scripts * Various corrections * Replace venv construction bash scripts by python equivalents * Update tox.ini * Unicode lines to compare files * Put modifications on letsencrypt-auto-source instead of generated scripts * Add executable permissions for Linux. * Merge tox win tests into main tox * Skip lock_test on Windows * Correct appveyor config * Update appveyor.yml * Explicit coverage py27 or py37 * Avoid to cover non supported certbot plugins on Windows * Update tox.ini * Remove specific warnings during CI * No cover on a debug code for tests only. * Update documentation and help script on venv/venv3.py * Customize help message for Windows * Quote correctly executable path with potential spaces in it. * Copy pipstrap from upstream
2018-11-07 20:16:16 -05:00
if __name__ == '__main__':
main(sys.argv[1:])