From 168787c58d3c6bcd8b313b12498de1a420d750f4 Mon Sep 17 00:00:00 2001 From: Peter Conrad Date: Wed, 12 Oct 2016 15:30:21 -0700 Subject: [PATCH 01/88] Fixing a weird out-of-place paragraph in the Getting Certbot section --- docs/using.rst | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/docs/using.rst b/docs/using.rst index d18d118cf..41e99f716 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -29,7 +29,12 @@ modern OSes based on Debian, Fedora, SUSE, Gentoo and Darwin. Getting Certbot =============== +Certbot is packaged for many common operating systems and web servers. Check whether +``certbot`` (or ``letsencrypt``) is packaged for your web server's OS by visiting +certbot.eff.org_, where you will also find the correct installation instructions for +your system. +.. Note:: Unless you have very specific requirements, we kindly suggest that you use the Certbot packages provided by your package manager (see certbot.eff.org_). If such packages are not available, we recommend using ``certbot-auto``, which automates the process of installing Certbot on your system. .. _certbot.eff.org: https://certbot.eff.org .. _certbot-auto: https://certbot.eff.org/docs/using.html#certbot-auto @@ -42,13 +47,6 @@ to, equivalently, as "subcommands") to request specific actions such as obtaining, renewing, or revoking certificates. Some of the most important and most commonly-used commands will be discussed throughout this document; an exhaustive list also appears near the end of the document. -======= -Certbot is packaged for many common operating systems and web servers. Check whether -``certbot`` (or ``letsencrypt``) is packaged for your web server's OS by visiting -certbot.eff.org_, where you will also find the correct installation instructions for -your system. - -.. Note:: Unless you have very specific requirements, we kindly suggest that you use the Certbot packages provided by your package manager (see certbot.eff.org_). If such packages are not available, we recommend using ``certbot-auto``, which automates the process of installing Certbot on your system. The ``certbot`` script on your web server might be named ``letsencrypt`` if your system uses an older package, or ``certbot-auto`` if you used an alternate installation method. Throughout the docs, whenever you see ``certbot``, swap in the correct name as needed. From 5345195e0c443efe850306a34e1cdfd32f1d4279 Mon Sep 17 00:00:00 2001 From: Peter Conrad Date: Fri, 21 Oct 2016 16:24:43 -0700 Subject: [PATCH 02/88] De-duping and clarifying installation information, separating it from Using. --- README.rst | 2 +- docs/install.rst | 209 ++++++++++++++++++++++++++++++++++++++++++---- docs/using.rst | 213 +---------------------------------------------- 3 files changed, 199 insertions(+), 225 deletions(-) diff --git a/README.rst b/README.rst index 244b6b510..f986703ac 100644 --- a/README.rst +++ b/README.rst @@ -32,7 +32,7 @@ Installation The easiest way to install Certbot is by visiting `certbot.eff.org`_, where you can find the correct installation instructions for many web server and OS combinations. -For more information, see the `User Guide `_. +For more information, see `Get Certbot `_. .. _certbot.eff.org: https://certbot.eff.org/ diff --git a/docs/install.rst b/docs/install.rst index e79a3b596..410a617d6 100644 --- a/docs/install.rst +++ b/docs/install.rst @@ -1,13 +1,53 @@ ===================== -Quick Installation +Get Certbot ===================== -If ``certbot`` (or ``letsencrypt``) is packaged for your Unix OS (visit -certbot.eff.org_ to find out), you can install it -from there, and run it by typing ``certbot`` (or ``letsencrypt``). Because -not all operating systems have packages yet, we provide a temporary solution -via the ``certbot-auto`` wrapper script, which obtains some dependencies from -your OS and puts others in a python virtual environment:: +.. _installation: + +Certbot is packaged for many common operating systems and web servers. Check whether +``certbot`` (or ``letsencrypt``) is packaged for your web server's OS by visiting +certbot.eff.org_, where you will also find the correct installation instructions for +your system. + +.. Note:: Unless you have very specific requirements, we kindly suggest that you use the Certbot packages provided by your package manager (see certbot.eff.org_). If such packages are not available, we recommend using ``certbot-auto``, which automates the process of installing Certbot on your system. + +.. _certbot.eff.org: https://certbot.eff.org + +.. _certbot-auto: https://certbot.eff.org/docs/install.html#certbot-auto + +System Requirements +=================== + +The Let's Encrypt Client presently only runs on Unix-ish OSes that include +Python 2.6 or 2.7; Python 3.x support will hopefully be added in the future. The +client requires root access in order to write to ``/etc/letsencrypt``, +``/var/log/letsencrypt``, ``/var/lib/letsencrypt``; to bind to ports 80 and 443 +(if you use the ``standalone`` plugin) and to read and modify webserver +configurations (if you use the ``apache`` or ``nginx`` plugins). If none of +these apply to you, it is theoretically possible to run without root privileges, +but for most users who want to avoid running an ACME client as root, either +`letsencrypt-nosudo `_ or +`simp_le `_ are more appropriate choices. + +The Apache plugin currently requires OS with augeas version 1.0; currently `it +supports +`_ +modern OSes based on Debian, Fedora, SUSE, Gentoo and Darwin. + +Alternate installation methods +================================ + +If you are offline or your operating system doesn't provide a package, you can use +an alternate method for installing ``certbot``. + +.. _certbot-auto: + +Certbot-Auto +------------ + +The ``certbot-auto`` wrapper script installs Certbot, obtaining some dependencies +from your web server OS and putting others in a python virtual environment. You can +download and run it as follows:: user@webserver:~$ wget https://dl.eff.org/certbot-auto user@webserver:~$ chmod a+x ./certbot-auto @@ -20,14 +60,155 @@ your OS and puts others in a python virtual environment:: user@server:~$ gpg2 --recv-key A2CFB51FA275A7286234E7B24D17C995CD9775F2 user@server:~$ gpg2 --trusted-key 4D17C995CD9775F2 --verify certbot-auto.asc certbot-auto -And for full command line help, you can type:: +The ``certbot-auto`` command updates to the latest client release automatically. +Since ``certbot-auto`` is a wrapper to ``certbot``, it accepts exactly +the same command line flags and arguments. For more information, see +`Certbot command-line options `_. + +For full command line help, you can type:: ./certbot-auto --help all -``certbot-auto`` updates to the latest client release automatically. And -since ``certbot-auto`` is a wrapper to ``certbot``, it accepts exactly -the same command line flags and arguments. More details about this script and -other installation methods can be found `in the User Guide -`_. +Running with Docker +------------------- + +Docker_ is an amazingly simple and quick way to obtain a +certificate. However, this mode of operation is unable to install +certificates or configure your webserver, because our installer +plugins cannot reach your webserver from inside the Docker container. + +Most users should use the operating system packages (see instructions at +certbot.eff.org_) or, as a fallback, ``certbot-auto``. You should only +use Docker if you are sure you know what you are doing and have a +good reason to do so. + +You should definitely read the :ref:`where-certs` section, in order to +know how to manage the certs +manually. `Our ciphersuites page `__ +provides some information about recommended ciphersuites. If none of +these make much sense to you, you should definitely use the +certbot-auto_ method, which enables you to use installer plugins +that cover both of those hard topics. + +If you're still not convinced and have decided to use this method, +from the server that the domain you're requesting a cert for resolves +to, `install Docker`_, then issue the following command: + +.. code-block:: shell + + sudo docker run -it --rm -p 443:443 -p 80:80 --name certbot \ + -v "/etc/letsencrypt:/etc/letsencrypt" \ + -v "/var/lib/letsencrypt:/var/lib/letsencrypt" \ + quay.io/letsencrypt/letsencrypt:latest certonly + +Running Certbot with the ``certonly`` command will obtain a certificate and place it in the directory +``/etc/letsencrypt/live`` on your system. Because Certonly cannot install the certificate from +within Docker, you must install the certificate manually according to the procedure +recommended by the provider of your webserver. + +For more information about the layout +of the ``/etc/letsencrypt`` directory, see :ref:`where-certs`. + +.. _Docker: https://docker.com +.. _`install Docker`: https://docs.docker.com/userguide/ + +Operating System Packages +------------------------- + +**FreeBSD** + + * Port: ``cd /usr/ports/security/py-certbot && make install clean`` + * Package: ``pkg install py27-certbot`` + +**OpenBSD** + + * Port: ``cd /usr/ports/security/letsencrypt/client && make install clean`` + * Package: ``pkg_add letsencrypt`` + +**Arch Linux** + +.. code-block:: shell + + sudo pacman -S certbot + +**Debian** + +If you run Debian Stretch or Debian Sid, you can install certbot packages. + +.. code-block:: shell + + sudo apt-get update + sudo apt-get install certbot python-certbot-apache + +If you don't want to use the Apache plugin, you can omit the +``python-certbot-apache`` package. + +Packages exist for Debian Jessie via backports. First you'll have to follow the +instructions at http://backports.debian.org/Instructions/ to enable the Jessie backports +repo, if you have not already done so. Then run: + +.. code-block:: shell + + sudo apt-get install letsencrypt python-letsencrypt-apache -t jessie-backports + +**Fedora** + +.. code-block:: shell + + sudo dnf install letsencrypt + +**Gentoo** + +The official Certbot client is available in Gentoo Portage. If you +want to use the Apache plugin, it has to be installed separately: + +.. code-block:: shell + + emerge -av app-crypt/letsencrypt + emerge -av app-crypt/letsencrypt-apache + +When using the Apache plugin, you will run into a "cannot find a cert or key +directive" error if you're sporting the default Gentoo ``httpd.conf``. +You can fix this by commenting out two lines in ``/etc/apache2/httpd.conf`` +as follows: + +Change + +.. code-block:: shell + + + LoadModule ssl_module modules/mod_ssl.so + + +to + +.. code-block:: shell + + # + LoadModule ssl_module modules/mod_ssl.so + # + +For the time being, this is the only way for the Apache plugin to recognise +the appropriate directives when installing the certificate. +Note: this change is not required for the other plugins. + +**Other Operating Systems** + +OS packaging is an ongoing effort. If you'd like to package +Certbot for your distribution of choice please have a +look at the :doc:`packaging`. + +Installing from source +---------------------- + +Installation from source is only supported for developers and the +whole process is described in the :doc:`contributing`. + +.. warning:: Please do **not** use ``python setup.py install`` or + ``python pip install .``. Please do **not** attempt the + installation commands as superuser/root and/or without virtual + environment, e.g. ``sudo python setup.py install``, ``sudo pip + install``, ``sudo ./venv/bin/...``. These modes of operation might + corrupt your operating system and are **not supported** by the + Certbot team! -.. _certbot.eff.org: https://certbot.eff.org/ diff --git a/docs/using.rst b/docs/using.rst index 57589349b..c7415969f 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -5,42 +5,8 @@ User Guide .. contents:: Table of Contents :local: -.. _installation: - -System Requirements -=================== - -The Let's Encrypt Client presently only runs on Unix-ish OSes that include -Python 2.6 or 2.7; Python 3.x support will hopefully be added in the future. The -client requires root access in order to write to ``/etc/letsencrypt``, -``/var/log/letsencrypt``, ``/var/lib/letsencrypt``; to bind to ports 80 and 443 -(if you use the ``standalone`` plugin) and to read and modify webserver -configurations (if you use the ``apache`` or ``nginx`` plugins). If none of -these apply to you, it is theoretically possible to run without root privileges, -but for most users who want to avoid running an ACME client as root, either -`letsencrypt-nosudo `_ or -`simp_le `_ are more appropriate choices. - -The Apache plugin currently requires OS with augeas version 1.0; currently `it -supports -`_ -modern OSes based on Debian, Fedora, SUSE, Gentoo and Darwin. - - -Getting Certbot -=============== -Certbot is packaged for many common operating systems and web servers. Check whether -``certbot`` (or ``letsencrypt``) is packaged for your web server's OS by visiting -certbot.eff.org_, where you will also find the correct installation instructions for -your system. - -.. Note:: Unless you have very specific requirements, we kindly suggest that you use the Certbot packages provided by your package manager (see certbot.eff.org_). If such packages are not available, we recommend using ``certbot-auto``, which automates the process of installing Certbot on your system. -.. _certbot.eff.org: https://certbot.eff.org - -.. _certbot-auto: https://certbot.eff.org/docs/using.html#certbot-auto - -Commands -======== +The Certbot Client +================== The Certbot client uses a number of different "commands" (also referred to, equivalently, as "subcommands") to request specific actions such as @@ -50,182 +16,9 @@ document; an exhaustive list also appears near the end of the document. The ``certbot`` script on your web server might be named ``letsencrypt`` if your system uses an older package, or ``certbot-auto`` if you used an alternate installation method. Throughout the docs, whenever you see ``certbot``, swap in the correct name as needed. - -Other installation methods --------------------------- -If you are offline or your operating system doesn't provide a package, you can use -an alternate method for installing ``certbot``. - -Certbot-Auto -^^^^^^^^^^^^ -The ``certbot-auto`` wrapper script installs Certbot, obtaining some dependencies -from your web server OS and putting others in a python virtual environment. You can -download and run it as follows:: - - user@webserver:~$ wget https://dl.eff.org/certbot-auto - user@webserver:~$ chmod a+x ./certbot-auto - user@webserver:~$ ./certbot-auto --help - -.. hint:: The certbot-auto download is protected by HTTPS, which is pretty good, but if you'd like to - double check the integrity of the ``certbot-auto`` script, you can use these steps for verification before running it:: - - user@server:~$ wget -N https://dl.eff.org/certbot-auto.asc - user@server:~$ gpg2 --recv-key A2CFB51FA275A7286234E7B24D17C995CD9775F2 - user@server:~$ gpg2 --trusted-key 4D17C995CD9775F2 --verify certbot-auto.asc certbot-auto - -The ``certbot-auto`` command updates to the latest client release automatically. -Since ``certbot-auto`` is a wrapper to ``certbot``, it accepts exactly -the same command line flags and arguments. For more information, see -`Certbot command-line options `_. - -Running with Docker -^^^^^^^^^^^^^^^^^^^ - -Docker_ is an amazingly simple and quick way to obtain a -certificate. However, this mode of operation is unable to install -certificates or configure your webserver, because our installer -plugins cannot reach your webserver from inside the Docker container. - -Most users should use the operating system packages (see instructions at -certbot.eff.org_) or, as a fallback, ``certbot-auto``. You should only -use Docker if you are sure you know what you are doing and have a -good reason to do so. - -You should definitely read the :ref:`where-certs` section, in order to -know how to manage the certs -manually. `Our ciphersuites page `__ -provides some information about recommended ciphersuites. If none of -these make much sense to you, you should definitely use the -certbot-auto_ method, which enables you to use installer plugins -that cover both of those hard topics. - -If you're still not convinced and have decided to use this method, -from the server that the domain you're requesting a cert for resolves -to, `install Docker`_, then issue the following command: - -.. code-block:: shell - - sudo docker run -it --rm -p 443:443 -p 80:80 --name certbot \ - -v "/etc/letsencrypt:/etc/letsencrypt" \ - -v "/var/lib/letsencrypt:/var/lib/letsencrypt" \ - quay.io/letsencrypt/letsencrypt:latest certonly - -Running Certbot with the ``certonly`` command will obtain a certificate and place it in the directory -``/etc/letsencrypt/live`` on your system. Because Certonly cannot install the certificate from -within Docker, you must install the certificate manually according to the procedure -recommended by the provider of your webserver. - -For more information about the layout -of the ``/etc/letsencrypt`` directory, see :ref:`where-certs`. - -.. _Docker: https://docker.com -.. _`install Docker`: https://docs.docker.com/userguide/ - - -Operating System Packages -^^^^^^^^^^^^^^^^^^^^^^^^^ - -**FreeBSD** - - * Port: ``cd /usr/ports/security/py-certbot && make install clean`` - * Package: ``pkg install py27-certbot`` - -**OpenBSD** - - * Port: ``cd /usr/ports/security/letsencrypt/client && make install clean`` - * Package: ``pkg_add letsencrypt`` - -**Arch Linux** - -.. code-block:: shell - - sudo pacman -S certbot - -**Debian** - -If you run Debian Stretch or Debian Sid, you can install certbot packages. - -.. code-block:: shell - - sudo apt-get update - sudo apt-get install certbot python-certbot-apache - -If you don't want to use the Apache plugin, you can omit the -``python-certbot-apache`` package. - -Packages exist for Debian Jessie via backports. First you'll have to follow the -instructions at http://backports.debian.org/Instructions/ to enable the Jessie backports -repo, if you have not already done so. Then run: - -.. code-block:: shell - - sudo apt-get install letsencrypt python-letsencrypt-apache -t jessie-backports - -**Fedora** - -.. code-block:: shell - - sudo dnf install letsencrypt - -**Gentoo** - -The official Certbot client is available in Gentoo Portage. If you -want to use the Apache plugin, it has to be installed separately: - -.. code-block:: shell - - emerge -av app-crypt/letsencrypt - emerge -av app-crypt/letsencrypt-apache - -When using the Apache plugin, you will run into a "cannot find a cert or key -directive" error if you're sporting the default Gentoo ``httpd.conf``. -You can fix this by commenting out two lines in ``/etc/apache2/httpd.conf`` -as follows: - -Change - -.. code-block:: shell - - - LoadModule ssl_module modules/mod_ssl.so - - -to - -.. code-block:: shell - - # - LoadModule ssl_module modules/mod_ssl.so - # - -For the time being, this is the only way for the Apache plugin to recognise -the appropriate directives when installing the certificate. -Note: this change is not required for the other plugins. - -**Other Operating Systems** - -OS packaging is an ongoing effort. If you'd like to package -Certbot for your distribution of choice please have a -look at the :doc:`packaging`. - - -Installing from source -^^^^^^^^^^^^^^^^^^^^^^ - -Installation from source is only supported for developers and the -whole process is described in the :doc:`contributing`. - -.. warning:: Please do **not** use ``python setup.py install`` or - ``python pip install .``. Please do **not** attempt the - installation commands as superuser/root and/or without virtual - environment, e.g. ``sudo python setup.py install``, ``sudo pip - install``, ``sudo ./venv/bin/...``. These modes of operation might - corrupt your operating system and are **not supported** by the - Certbot team! - .. _plugins: -Getting certificates (and chosing plugins) +Getting certificates (and choosing plugins) ========================================== The Certbot client supports a number of different "plugins" that can be From 8f3f166186264a6a962a16258c15b4b3254f592e Mon Sep 17 00:00:00 2001 From: Peter Conrad Date: Thu, 27 Oct 2016 11:34:35 -0700 Subject: [PATCH 03/88] Responding to feedback at https://github.com/certbot/certbot/pull/3675#pullrequestreview-5757007 --- docs/install.rst | 8 ++++++-- docs/using.rst | 8 ++++---- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/install.rst b/docs/install.rst index 410a617d6..56f6c1189 100644 --- a/docs/install.rst +++ b/docs/install.rst @@ -2,7 +2,12 @@ Get Certbot ===================== -.. _installation: +.. contents:: Table of Contents + :local: + + +About Certbot +============= Certbot is packaged for many common operating systems and web servers. Check whether ``certbot`` (or ``letsencrypt``) is packaged for your web server's OS by visiting @@ -13,7 +18,6 @@ your system. .. _certbot.eff.org: https://certbot.eff.org -.. _certbot-auto: https://certbot.eff.org/docs/install.html#certbot-auto System Requirements =================== diff --git a/docs/using.rst b/docs/using.rst index c7415969f..1becea8ea 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -5,10 +5,10 @@ User Guide .. contents:: Table of Contents :local: -The Certbot Client -================== +Certbot Commands +================ -The Certbot client uses a number of different "commands" (also referred +Certbot uses a number of different "commands" (also referred to, equivalently, as "subcommands") to request specific actions such as obtaining, renewing, or revoking certificates. Some of the most important and most commonly-used commands will be discussed throughout this @@ -19,7 +19,7 @@ The ``certbot`` script on your web server might be named ``letsencrypt`` if your .. _plugins: Getting certificates (and choosing plugins) -========================================== +=========================================== The Certbot client supports a number of different "plugins" that can be used to obtain and/or install certificates. From 364a6d8a2d16535c22a5a8e973212401e3967fea Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 11 Jan 2017 12:01:33 -0800 Subject: [PATCH 04/88] Release 0.10.0 (#4022) * Release 0.10.0 * Bump version to 0.11.0 --- acme/setup.py | 2 +- certbot-apache/setup.py | 2 +- certbot-auto | 261 +++++---- certbot-compatibility-test/setup.py | 2 +- certbot-nginx/setup.py | 2 +- certbot/__init__.py | 2 +- docs/cli-help.txt | 512 ++++++++++-------- letsencrypt-auto | 261 +++++---- letsencrypt-auto-source/certbot-auto.asc | 14 +- letsencrypt-auto-source/letsencrypt-auto | 26 +- letsencrypt-auto-source/letsencrypt-auto.sig | Bin 256 -> 256 bytes .../pieces/letsencrypt-auto-requirements.txt | 24 +- 12 files changed, 647 insertions(+), 461 deletions(-) diff --git a/acme/setup.py b/acme/setup.py index 5524a6734..f325b47a2 100644 --- a/acme/setup.py +++ b/acme/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.10.0.dev0' +version = '0.11.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-apache/setup.py b/certbot-apache/setup.py index 2b4ac8563..d50a414a1 100644 --- a/certbot-apache/setup.py +++ b/certbot-apache/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.10.0.dev0' +version = '0.11.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-auto b/certbot-auto index cba185eae..a2ddf76ac 100755 --- a/certbot-auto +++ b/certbot-auto @@ -15,11 +15,15 @@ set -e # Work even if somebody does "sh thisscript.sh". # Note: you can set XDG_DATA_HOME or VENV_PATH before running this script, # if you want to change where the virtual environment will be installed -XDG_DATA_HOME=${XDG_DATA_HOME:-~/.local/share} +if [ -z "$XDG_DATA_HOME" ]; then + XDG_DATA_HOME=~/.local/share +fi VENV_NAME="letsencrypt" -VENV_PATH=${VENV_PATH:-"$XDG_DATA_HOME/$VENV_NAME"} +if [ -z "$VENV_PATH" ]; then + VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" +fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.9.3" +LE_AUTO_VERSION="0.10.0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -58,6 +62,7 @@ for arg in "$@" ; do --verbose) VERBOSE=1;; -[!-]*) + OPTIND=1 while getopts ":hnvq" short_arg $arg; do case "$short_arg" in h) @@ -79,43 +84,74 @@ if [ $BASENAME = "letsencrypt-auto" ]; then HELP=0 fi +# Support for busybox and others where there is no "command", +# but "which" instead +if command -v command > /dev/null 2>&1 ; then + export EXISTS="command -v" +elif which which > /dev/null 2>&1 ; then + export EXISTS="which" +else + echo "Cannot find command nor which... please install one!" + exit 1 +fi + # certbot-auto needs root access to bootstrap OS dependencies, and # certbot itself needs root access for almost all modes of operation # The "normal" case is that sudo is used for the steps that need root, but # this script *can* be run as root (not recommended), or fall back to using -# `su` +# `su`. Auto-detection can be overrided by explicitly setting the +# environment variable LE_AUTO_SUDO to 'sudo', 'sudo_su' or '' as used below. + +# Because the parameters in `su -c` has to be a string, +# we need to properly escape it. +su_sudo() { + args="" + # This `while` loop iterates over all parameters given to this function. + # For each parameter, all `'` will be replace by `'"'"'`, and the escaped string + # will be wrapped in a pair of `'`, then appended to `$args` string + # For example, `echo "It's only 1\$\!"` will be escaped to: + # 'echo' 'It'"'"'s only 1$!' + # │ │└┼┘│ + # │ │ │ └── `'s only 1$!'` the literal string + # │ │ └── `\"'\"` is a single quote (as a string) + # │ └── `'It'`, to be concatenated with the strings following it + # └── `echo` wrapped in a pair of `'`, it's totally fine for the shell command itself + while [ $# -ne 0 ]; do + args="$args'$(printf "%s" "$1" | sed -e "s/'/'\"'\"'/g")' " + shift + done + su root -c "$args" +} + SUDO_ENV="" export CERTBOT_AUTO="$0" -if test "`id -u`" -ne "0" ; then - if command -v sudo 1>/dev/null 2>&1; then - SUDO=sudo - SUDO_ENV="CERTBOT_AUTO=$0" - else - echo \"sudo\" is not available, will use \"su\" for installation steps... - # Because the parameters in `su -c` has to be a string, - # we need properly escape it - su_sudo() { - args="" - # This `while` loop iterates over all parameters given to this function. - # For each parameter, all `'` will be replace by `'"'"'`, and the escaped string - # will be wrapped in a pair of `'`, then appended to `$args` string - # For example, `echo "It's only 1\$\!"` will be escaped to: - # 'echo' 'It'"'"'s only 1$!' - # │ │└┼┘│ - # │ │ │ └── `'s only 1$!'` the literal string - # │ │ └── `\"'\"` is a single quote (as a string) - # │ └── `'It'`, to be concatenated with the strings following it - # └── `echo` wrapped in a pair of `'`, it's totally fine for the shell command itself - while [ $# -ne 0 ]; do - args="$args'$(printf "%s" "$1" | sed -e "s/'/'\"'\"'/g")' " - shift - done - su root -c "$args" - } - SUDO=su_sudo - fi +if [ -n "${LE_AUTO_SUDO+x}" ]; then + case "$LE_AUTO_SUDO" in + su_sudo|su) + SUDO=su_sudo + ;; + sudo) + SUDO=sudo + SUDO_ENV="CERTBOT_AUTO=$0" + ;; + '') ;; # Nothing to do for plain root method. + *) + echo "Error: unknown root authorization mechanism '$LE_AUTO_SUDO'." + exit 1 + esac + echo "Using preset root authorization mechanism '$LE_AUTO_SUDO'." else - SUDO= + if test "`id -u`" -ne "0" ; then + if $EXISTS sudo 1>/dev/null 2>&1; then + SUDO=sudo + SUDO_ENV="CERTBOT_AUTO=$0" + else + echo \"sudo\" is not available, will use \"su\" for installation steps... + SUDO=su_sudo + fi + else + SUDO= + fi fi ExperimentalBootstrap() { @@ -136,7 +172,7 @@ ExperimentalBootstrap() { DeterminePythonVersion() { for LE_PYTHON in "$LE_PYTHON" python2.7 python27 python2 python; do # Break (while keeping the LE_PYTHON value) if found. - command -v "$LE_PYTHON" > /dev/null && break + $EXISTS "$LE_PYTHON" > /dev/null && break done if [ "$?" != "0" ]; then echo "Cannot find any Pythons; please install one!" @@ -177,19 +213,22 @@ BootstrapDebCommon() { # distro version (#346) virtualenv= - if apt-cache show virtualenv > /dev/null 2>&1 && ! apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then - virtualenv="virtualenv" + # virtual env is known to apt and is installable + if apt-cache show virtualenv > /dev/null 2>&1 ; then + if ! LC_ALL=C apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then + virtualenv="virtualenv" + fi fi if apt-cache show python-virtualenv > /dev/null 2>&1; then - virtualenv="$virtualenv python-virtualenv" + virtualenv="$virtualenv python-virtualenv" fi augeas_pkg="libaugeas0 augeas-lenses" - AUGVERSION=`apt-cache show --no-all-versions libaugeas0 | grep ^Version: | cut -d" " -f2` + AUGVERSION=`LC_ALL=C apt-cache show --no-all-versions libaugeas0 | grep ^Version: | cut -d" " -f2` if [ "$ASSUME_YES" = 1 ]; then - YES_FLAG="-y" + YES_FLAG="-y" fi AddBackportRepo() { @@ -248,15 +287,15 @@ BootstrapDebCommon() { python-dev \ $virtualenv \ gcc \ - dialog \ $augeas_pkg \ libssl-dev \ + openssl \ libffi-dev \ ca-certificates \ - if ! command -v virtualenv > /dev/null ; then + if ! $EXISTS virtualenv > /dev/null ; then echo Failed to install a working \"virtualenv\" command, exiting exit 1 fi @@ -307,7 +346,6 @@ BootstrapRpmCommon() { pkgs=" gcc - dialog augeas-libs openssl openssl-devel @@ -361,7 +399,6 @@ BootstrapSuseCommon() { python-devel \ python-virtualenv \ gcc \ - dialog \ augeas-lenses \ libopenssl-devel \ libffi-devel \ @@ -380,7 +417,6 @@ BootstrapArchCommon() { python2 python-virtualenv gcc - dialog augeas openssl libffi @@ -404,22 +440,26 @@ BootstrapGentooCommon() { PACKAGES=" dev-lang/python:2.7 dev-python/virtualenv - dev-util/dialog app-admin/augeas dev-libs/openssl dev-libs/libffi app-misc/ca-certificates virtual/pkgconfig" + ASK_OPTION="--ask" + if [ "$ASSUME_YES" = 1 ]; then + ASK_OPTION="" + fi + case "$PACKAGE_MANAGER" in (paludis) $SUDO cave resolve --preserve-world --keep-targets if-possible $PACKAGES -x ;; (pkgcore) - $SUDO pmerge --noreplace --oneshot $PACKAGES + $SUDO pmerge --noreplace --oneshot $ASK_OPTION $PACKAGES ;; (portage|*) - $SUDO emerge --noreplace --oneshot $PACKAGES + $SUDO emerge --noreplace --oneshot $ASK_OPTION $PACKAGES ;; esac } @@ -449,7 +489,6 @@ BootstrapMac() { fi $pkgcmd augeas - $pkgcmd dialog if [ "$(which python)" = "/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python" \ -o "$(which python)" = "/usr/bin/python" ]; then # We want to avoid using the system Python because it requires root to use pip. @@ -458,7 +497,7 @@ BootstrapMac() { $pkgcmd python fi - # Workaround for _dlopen not finding augeas on OS X + # Workaround for _dlopen not finding augeas on macOS if [ "$pkgman" = "port" ] && ! [ -e "/usr/local/lib/libaugeas.dylib" ] && [ -e "/opt/local/lib/libaugeas.dylib" ]; then echo "Applying augeas workaround" $SUDO mkdir -p /usr/local/lib/ @@ -496,8 +535,8 @@ BootstrapMageiaCommon() { if ! $SUDO urpmi --force \ git \ gcc \ - cdialog \ python-augeas \ + openssl \ libopenssl-devel \ libffi-devel \ rootcerts @@ -541,7 +580,7 @@ Bootstrap() { elif uname | grep -iq FreeBSD ; then ExperimentalBootstrap "FreeBSD" BootstrapFreeBsd elif uname | grep -iq Darwin ; then - ExperimentalBootstrap "Mac OS X" BootstrapMac + ExperimentalBootstrap "macOS" BootstrapMac elif [ -f /etc/issue ] && grep -iq "Amazon Linux" /etc/issue ; then ExperimentalBootstrap "Amazon Linux" BootstrapRpmCommon elif [ -f /etc/product ] && grep -q "Joyent Instance" /etc/product ; then @@ -557,7 +596,7 @@ Bootstrap() { } TempDir() { - mktemp -d 2>/dev/null || mktemp -d -t 'le' # Linux || OS X + mktemp -d 2>/dev/null || mktemp -d -t 'le' # Linux || macOS } @@ -594,6 +633,11 @@ if [ "$1" = "--le-auto-phase2" ]; then # `pip install --no-cache-dir -e acme -e . -e certbot-apache -e certbot-nginx`, # and then use `hashin` or a more secure method to gather the hashes. +# Hashin example: +# pip install hashin +# hashin -r letsencrypt-auto-requirements.txt cryptography==1.5.2 +# sets the new certbot-auto pinned version of cryptography to 1.5.2 + argparse==1.4.0 \ --hash=sha256:c31647edb69fd3d465a847ea3157d37bed1f95f19760b11a47aa91c04b666314 \ --hash=sha256:62b089a55be1d8949cd2bc7e0df0bddb9e028faefc8c32038cc84862aefdd6e4 @@ -601,7 +645,8 @@ argparse==1.4.0 \ # This comes before cffi because cffi will otherwise install an unchecked # version via setup_requires. pycparser==2.14 \ - --hash=sha256:7959b4a74abdc27b312fed1c21e6caf9309ce0b29ea86b591fd2e99ecdf27f73 + --hash=sha256:7959b4a74abdc27b312fed1c21e6caf9309ce0b29ea86b591fd2e99ecdf27f73 \ + --no-binary pycparser cffi==1.4.2 \ --hash=sha256:53c1c9ddb30431513eb7f3cdef0a3e06b0f1252188aaa7744af0f5a4cd45dbaf \ @@ -624,29 +669,29 @@ ConfigArgParse==0.10.0 \ --hash=sha256:3b50a83dd58149dfcee98cb6565265d10b53e9c0a2bca7eeef7fb5f5524890a7 configobj==5.0.6 \ --hash=sha256:a2f5650770e1c87fb335af19a9b7eb73fc05ccf22144eb68db7d00cd2bcb0902 -cryptography==1.3.4 \ - --hash=sha256:bede00edd11a2a62c8c98c271cc103fa3a3d72acf64f6e5e4eaf251128897b17 \ - --hash=sha256:53b39e687b744bb548a98f40736cc529d9f60959b4e6cc551322cf9505d35eb3 \ - --hash=sha256:474b73ad1139b4e423e46bbd818efd0d5c0df1c65d9f7c957d64c9215d77afde \ - --hash=sha256:aaddf9592d5b99e32dd518bb4a25b147c124f9d6b4ad64b94f01b15d1666b8c8 \ - --hash=sha256:6dcad2f407db8c3cd6ecd78361439c449a4f94786b46c54507e7e68f51e1709d \ - --hash=sha256:475c153fc622e656f1f10a9c9941d0ac7ab18df7c38d35d563a437c1c0e34f24 \ - --hash=sha256:86dd61df581cba04e89e45081efbc531faff1c9d99c77b1ce97f87216c356353 \ - --hash=sha256:75cc697e4ef5fdd0102ca749114c6370dbd11db0c9132a18834858c2566247e3 \ - --hash=sha256:ea03ad5b9df6d79fc9fc1ab23729e01e1c920d2974c5e3c634ccf45a5c378452 \ - --hash=sha256:c8872b8fe4f3416d6338ab99612f49ab314f7856cb43bffab2a32d28a6267be8 \ - --hash=sha256:468fc6e16eaec6ceaa6bc341273e6e9912d01b42b740f8cf896ace7fcd6a321d \ - --hash=sha256:d6fea3c6502735011c5d61a62aef1c1d770fc6a2def45d9e6c0d94c9651e3317 \ - --hash=sha256:3cf95f179f4bead3d5649b91860ef4cf60ad4244209190fc405908272576d961 \ - --hash=sha256:141f77e60a5b9158309b2b60288c7f81d37faa15c22a69b94c190ceefaaa6236 \ - --hash=sha256:87b7a1fe703c6424451f3372d1879dae91c7fe5e13375441a72833db76fee30e \ - --hash=sha256:f5ee3cb0cf1a6550bf483ccffa6608db267a377b45f7e3a8201a86d1d8feb19f \ - --hash=sha256:4e097286651ea318300af3251375d48b71b8228481c56cd617ddd4459a1ff261 \ - --hash=sha256:1e3d3ae3f22f22d50d340f47f25227511326f3f1396c6d2446a5b45b516c4313 \ - --hash=sha256:6a057941cb64d79834ea3cf99093fcc4787c2a5d44f686c4f297361ddc419bcd \ - --hash=sha256:68b3d5390b92559ddd3353c73ab2dfcff758f9c4ec4f5d5226ccede0e5d779f4 \ - --hash=sha256:545dc003b4b6081f9c3e452da15d819b04b696f49484aff64c0a2aedf766bef8 \ - --hash=sha256:423ff890c01be7c70dbfeaa967eeef5146f1a43a5f810ffdc07b178e48a105a9 +cryptography==1.5.3 \ + --hash=sha256:e514d92086246b53ae9b048df652cf3036b462e50a6ce9fac6b6253502679991 \ + --hash=sha256:10ee414f4b5af403a0d8f20dfa80f7dad1fc7ae5452ec5af03712d5b6e78c664 \ + --hash=sha256:7234456d1f4345a144ed07af2416c7c0659d4bb599dd1a963103dc8c183b370e \ + --hash=sha256:d3b9587406f94642bd70b3d666b813f446e95f84220c9e416ad94cbfb6be2eaa \ + --hash=sha256:b15fc6b59f1474eef62207c85888afada8acc47fae8198ba2b0197d54538961a \ + --hash=sha256:3b62d65d342704fc07ed171598db2a2775bdf587b1b6abd2cba2261bfe3ccde3 \ + --hash=sha256:059343022ec904c867a13bc55d2573e36c8cfb2c250e30d8a2e9825f253b07ba \ + --hash=sha256:c7897cf13bc8b4ee0215d83cbd51766d87c06b277fcca1f9108595508e5bcfb4 \ + --hash=sha256:9b69e983e5bf83039ddd52e52a28c7faedb2b22bdfb5876377b95aac7d3be63e \ + --hash=sha256:61e40905c426d02b3fae38088dc66ce4ef84830f7eb223dec6b3ac3ccdc676fb \ + --hash=sha256:00783a32bcd91a12177230d35bfcf70a2333ade4a6b607fac94a633a7971c671 \ + --hash=sha256:d11973f49b648cde1ea1a30e496d7557dbfeccd08b3cd9ba58d286a9c274ff8e \ + --hash=sha256:f24bedf28b81932ba6063aec9a826669f5237ea3b755efe04d98b072faa053a5 \ + --hash=sha256:3ab5725367239e3deb9b92e917aa965af3fef008f25b96a3000821869e208181 \ + --hash=sha256:8a53209de822e22b5f73bf4b99e68ac4ccc91051fd6751c8252982983e86a77d \ + --hash=sha256:5a07439d4b1e4197ac202b7eea45e26a6fd65757652dc50f1a63367f711df933 \ + --hash=sha256:26b1c4b40aec7b0074bceabe6e06565aa28176eca7323a31df66ebf89fe916d3 \ + --hash=sha256:eaa4a7b5a6682adcf8d6ebb2a08a008802657643655bb527c95c8a3860253d8e \ + --hash=sha256:8156927dcf8da274ff205ad0612f75c380df45385bacf98531a5b3348c88d135 \ + --hash=sha256:61ec0d792749d0e91e84b1d58b6dfd204806b10b5811f846c2ceca0de028c53a \ + --hash=sha256:26330c88041569ca621cc42274d0ea2667a48b6deab41467272c3aba0b6e8f07 \ + --hash=sha256:cf82ddac919b587f5e44247579b433224cc2e03332d2ea4d89aa70d7e6b64ae5 enum34==1.1.2 \ --hash=sha256:2475d7fcddf5951e92ff546972758802de5260bf409319a9f1934e6bbc8b1dc7 \ --hash=sha256:35907defb0f992b75ab7788f65fedc1cf20ffa22688e0e6f6f12afc06b3ea501 @@ -662,8 +707,6 @@ ipaddress==1.0.16 \ linecache2==1.0.0 \ --hash=sha256:e78be9c0a0dfcbac712fe04fbf92b96cddae80b1b842f24248214c8496f006ef \ --hash=sha256:4b26ff4e7110db76eeb6f5a7b64a82623839d595c2038eeda662f2a2db78e97c -ndg-httpsclient==0.4.0 \ - --hash=sha256:e8c155fdebd9c4bcb0810b4ed01ae1987554b1ee034dd7532d7b8fdae38a6274 ordereddict==1.1 \ --hash=sha256:1c35b4ac206cef2d24816c89f89cf289dd3d38cf7c449bb3fab7bf6d43f01b1f parsedatetime==2.1 \ @@ -684,9 +727,9 @@ pyasn1==0.1.9 \ --hash=sha256:5191ff6b9126d2c039dd87f8ff025bed274baf07fa78afa46f556b1ad7265d6e \ --hash=sha256:8323e03637b2d072cc7041300bac6ec448c3c28950ab40376036788e9a1af629 \ --hash=sha256:853cacd96d1f701ddd67aa03ecc05f51890135b7262e922710112f12a2ed2a7f -pyopenssl==16.0.0 \ - --hash=sha256:5add70cf00273bf957ca31fdb0df9b0ae4639e081897d5f86a0ae1f104901230 \ - --hash=sha256:363d10ee43d062285facf4e465f4f5163f9f702f9134f0a5896f134cbb92d17d +pyOpenSSL==16.2.0 \ + --hash=sha256:26ca380ddf272f7556e48064bbcd5bd71f83dfc144f3583501c7ddbd9434ee17 \ + --hash=sha256:7779a3bbb74e79db234af6a08775568c6769b5821faecf6e2f4143edb227516e pyparsing==2.1.8 \ --hash=sha256:2f0f5ceb14eccd5aef809d6382e87df22ca1da583c79f6db01675ce7d7f49c18 \ --hash=sha256:03a4869b9f3493807ee1f1cb405e6d576a1a2ca4d81a982677c0c1ad6177c56b \ @@ -701,9 +744,6 @@ pyRFC3339==1.0 \ --hash=sha256:8dfbc6c458b8daba1c0f3620a8c78008b323a268b27b7359e92a4ae41325f535 python-augeas==0.5.0 \ --hash=sha256:67d59d66cdba8d624e0389b87b2a83a176f21f16a87553b50f5703b23f29bac2 -python2-pythondialog==3.3.0 \ - --hash=sha256:04e93f24995c43dd90f338d5d865ca72ce3fb5a5358d4daa4965571db35fc3ec \ - --hash=sha256:3e6f593fead98f8a526bc3e306933533236e33729f552f52896ea504f55313fa pytz==2015.7 \ --hash=sha256:3abe6a6d3fc2fbbe4c60144211f45da2edbe3182a6f6511af6bbba0598b1f992 \ --hash=sha256:939ef9c1e1224d980405689a97ffcf7828c56d1517b31d73464356c1f2b7769e \ @@ -718,9 +758,9 @@ pytz==2015.7 \ --hash=sha256:fbd26746772c24cb93c8b97cbdad5cb9e46c86bbdb1b9d8a743ee00e2fb1fc5d \ --hash=sha256:99266ef30a37e43932deec2b7ca73e83c8dbc3b9ff703ec73eca6b1dae6befea \ --hash=sha256:8b6ce1c993909783bc96e0b4f34ea223bff7a4df2c90bdb9c4e0f1ac928689e3 -requests==2.9.1 \ - --hash=sha256:113fbba5531a9e34945b7d36b33a084e8ba5d0664b703c81a7c572d91919a5b8 \ - --hash=sha256:c577815dd00f1394203fc44eb979724b098f88264a9ef898ee45b8e5e9cf587f +requests==2.12.1 \ + --hash=sha256:3f3f27a9d0f9092935efc78054ef324eb9f8166718270aefe036dfa1e4f68e1e \ + --hash=sha256:2109ecea94df90980be040490ff1d879971b024861539abb00054062388b612e six==1.10.0 \ --hash=sha256:0ff78c403d9bccf5a425a6d31a12aa6b47f1c21ca4dc2573a7e2f32a97335eb1 \ --hash=sha256:105f8d68616f8248e24bf0e9372ef04d3cc10104f1980f54d57b2ce73a5ad56a @@ -761,18 +801,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.9.3 \ - --hash=sha256:d18ce17a75ad24d27981dfaef0524aa905eab757b267e027162b56a8967ab8fb \ - --hash=sha256:a6eff1f955eb2e4316abd9aa2fedb6d9345e6b5b8a2d64ea0ad35e05d6124099 -certbot==0.9.3 \ - --hash=sha256:a87ef4c53c018df4e52ee2f2e906ad16bbb37789f29e6f284c495a2eb4d9b243 \ - --hash=sha256:68149cb8392b29f5d5246e7226d25f913f2b10482bf3bc7368e8c8821d25f3b0 -certbot-apache==0.9.3 \ - --hash=sha256:f379b1053e10709692654d7a6fcea9eaed19b66c49a753b61e31bd06a04b0aac \ - --hash=sha256:a5d98cf972072de08f984db4e6a7f20269f3f023c43f6d4e781fe43be7c10086 -certbot-nginx==0.9.3 \ - --hash=sha256:3c26f18f0b57550f069263bd9b2984ef33eab6693e7796611c1b2cc16574069c \ - --hash=sha256:7337a2e90e0b28a1ab09e31d9fb81c6d78e6453500c824c0f18bab5d31b63058 +acme==0.10.0 \ + --hash=sha256:df4299a9881d94185a1578ed97334430a90f761ce815edd300860ca47d0538f1 \ + --hash=sha256:ddebdf1fe139c8fedbcf633955ec867496d2f7d2d2e9879d538437a69ab47876 +certbot==0.10.0 \ + --hash=sha256:fb1bfa3d54ce9366758e374f7ed99667ce20484224934d3e8e57839fcf784bc5 \ + --hash=sha256:dd64ed8fb3cc3b053f05e779b934433445918668c49bcdbb2c816062815e1661 +certbot-apache==0.10.0 \ + --hash=sha256:909d59c53507093f838f7336f75d7d78563a35b16afdf6c30f45c9f47bf069da \ + --hash=sha256:6f110dae227dd0fea9572fa12dd60b041e391f5d2028cc2e1fedd2a9a0d2bc88 +certbot-nginx==0.10.0 \ + --hash=sha256:4f33a230d420cbd0431e7b707fb9a1732bfd18d3c6056019591bd7c3a13abe92 \ + --hash=sha256:c12ffd05207b0be3c765b3d3e2927e0b2cc2b7de20654b19d154a0d789e7c1d5 UNLIKELY_EOF # ------------------------------------------------------------------------- @@ -940,7 +980,28 @@ UNLIKELY_EOF # Report error. (Otherwise, be quiet.) echo "Had a problem while installing Python packages." if [ "$VERBOSE" != 1 ]; then + echo + echo "pip prints the following errors: " + echo "=====================================================" echo "$PIP_OUT" + echo "=====================================================" + echo + echo "Certbot has problem setting up the virtual environment." + + if `echo $PIP_OUT | grep -q Killed` || `echo $PIP_OUT | grep -q "allocate memory"` ; then + echo + echo "Based on your pip output, the problem can likely be fixed by " + echo "increasing the available memory." + else + echo + echo "We were not be able to guess the right solution from your pip " + echo "output." + fi + + echo + echo "Consult https://certbot.eff.org/docs/install.html#problems-with-python-virtual-environment" + echo "for possible solutions." + echo "You may also find some support resources at https://certbot.eff.org/support/ ." fi rm -rf "$VENV_PATH" exit 1 @@ -1132,7 +1193,7 @@ UNLIKELY_EOF # TODO: Deal with quotes in pathnames. echo "Replacing certbot-auto..." # Clone permissions with cp. chmod and chown don't have a --reference - # option on OS X or BSD, and stat -c on Linux is stat -f on OS X and BSD: + # option on macOS or BSD, and stat -c on Linux is stat -f on macOS and BSD: $SUDO cp -p "$0" "$TEMP_DIR/letsencrypt-auto.permission-clone" $SUDO cp "$TEMP_DIR/letsencrypt-auto" "$TEMP_DIR/letsencrypt-auto.permission-clone" # Using mv rather than cp leaves the old file descriptor pointing to the diff --git a/certbot-compatibility-test/setup.py b/certbot-compatibility-test/setup.py index 32e5935fb..fa49d9f3d 100644 --- a/certbot-compatibility-test/setup.py +++ b/certbot-compatibility-test/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.10.0.dev0' +version = '0.11.0.dev0' install_requires = [ 'certbot', diff --git a/certbot-nginx/setup.py b/certbot-nginx/setup.py index 4c39d37c2..e140c75d2 100644 --- a/certbot-nginx/setup.py +++ b/certbot-nginx/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.10.0.dev0' +version = '0.11.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot/__init__.py b/certbot/__init__.py index 45892e269..376b0504f 100644 --- a/certbot/__init__.py +++ b/certbot/__init__.py @@ -1,4 +1,4 @@ """Certbot client.""" # version number like 1.2.3a0, must have at least 2 parts, like 1.2 -__version__ = '0.10.0.dev0' +__version__ = '0.11.0.dev0' diff --git a/docs/cli-help.txt b/docs/cli-help.txt index 279b65219..a2dd61a31 100644 --- a/docs/cli-help.txt +++ b/docs/cli-help.txt @@ -1,39 +1,61 @@ -usage: - certbot [SUBCOMMAND] [options] [-d domain] [-d domain] ... +usage: + certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ... Certbot can obtain and install HTTPS/TLS/SSL certificates. By default, it will attempt to use a webserver both for obtaining and installing the -cert. Major SUBCOMMANDS are: +cert. The most common SUBCOMMANDS and flags are: - (default) run Obtain & install a cert in your current webserver - certonly Obtain cert, but do not install it (aka "auth") - install Install a previously obtained cert in a server - renew Renew previously obtained certs that are near expiry - revoke Revoke a previously obtained certificate - register Perform tasks related to registering with the CA - rollback Rollback server configuration changes made during install - config_changes Show changes made to server config during installation - plugins Display information about installed plugins +obtain, install, and renew certificates: + (default) run Obtain & install a cert in your current webserver + certonly Obtain or renew a cert, but do not install it + renew Renew all previously obtained certs that are near expiry + -d DOMAINS Comma-separated list of domains to obtain a cert for + + --apache Use the Apache plugin for authentication & installation + --standalone Run a standalone webserver for authentication + --nginx Use the Nginx plugin for authentication & installation + --webroot Place files in a server's webroot folder for authentication + --manual Obtain certs interactively, or using shell script hooks + + -n Run non-interactively + --test-cert Obtain a test cert from a staging server + --dry-run Test "renew" or "certonly" without saving any certs to disk + +manage certificates: + certificates Display information about certs you have from Certbot + revoke Revoke a certificate (supply --cert-path) + delete Delete a certificate + +manage your account with Let's Encrypt: + register Create a Let's Encrypt ACME account + --agree-tos Agree to the ACME server's Subscriber Agreement + -m EMAIL Email address for important account notifications optional arguments: -h, --help show this help message and exit -c CONFIG_FILE, --config CONFIG_FILE - config file path (default: None) + path to config file (default: /etc/letsencrypt/cli.ini + and ~/.config/letsencrypt/cli.ini) -v, --verbose This flag can be used multiple times to incrementally increase the verbosity of output, e.g. -vvv. (default: -2) - -t, --text Use the text output instead of the curses UI. - (default: False) -n, --non-interactive, --noninteractive Run without ever asking for user input. This may require additional command line flags; the client will try to explain which ones are required if it finds one missing (default: False) - --dialog Run using interactive dialog menus (default: False) + --force-interactive Force Certbot to be interactive even if it detects + it's not being run in a terminal. This flag cannot be + used with the renew subcommand. (default: False) -d DOMAIN, --domains DOMAIN, --domain DOMAIN Domain names to apply. For multiple domains you can use multiple -d flags or enter a comma separated list - of domains as a parameter. (default: []) + of domains as a parameter. (default: Ask) + --cert-name CERTNAME Certificate name to apply. Only one certificate name + can be used per Certbot run. To see certificate names, + run 'certbot certificates'. When creating a new + certificate, specifies the new certificate's name. + (default: None) --dry-run Perform a test run of the client, obtaining test (invalid) certs but not saving them to disk. This can currently only be used with the 'certonly' and 'renew' @@ -48,6 +70,219 @@ optional arguments: because they may be necessary to accurately simulate renewal. --renew-hook commands are not called. (default: False) + --preferred-challenges PREF_CHALLS + A sorted, comma delimited list of the preferred + challenge to use during authorization with the most + preferred challenge listed first (Eg, "dns" or "tls- + sni-01,http,dns"). Not all plugins support all + challenges. See + https://certbot.eff.org/docs/using.html#plugins for + details. ACME Challenges are versioned, but if you + pick "http" rather than "http-01", Certbot will select + the latest version automatically. (default: []) + --user-agent USER_AGENT + Set a custom user agent string for the client. User + agent strings allow the CA to collect high level + statistics about success rates by OS and plugin. If + you wish to hide your server OS version from the Let's + Encrypt server, set this to "". (default: + CertbotACMEClient/0.10.0 (Ubuntu 16.04.1 LTS) + Authenticator/XXX Installer/YYY) + +automation: + Arguments for automating execution & other tweaks + + --keep-until-expiring, --keep, --reinstall + If the requested cert matches an existing cert, always + keep the existing one until it is due for renewal (for + the 'run' subcommand this means reinstall the existing + cert). (default: Ask) + --expand If an existing cert covers some subset of the + requested names, always expand and replace it with the + additional names. (default: Ask) + --version show program's version number and exit + --force-renewal, --renew-by-default + If a certificate already exists for the requested + domains, renew it now, regardless of whether it is + near expiry. (Often --keep-until-expiring is more + appropriate). Also implies --expand. (default: False) + --renew-with-new-domains + If a certificate already exists for the requested + certificate name but does not match the requested + domains, renew it now, regardless of whether it is + near expiry. (default: False) + --allow-subset-of-names + When performing domain validation, do not consider it + a failure if authorizations can not be obtained for a + strict subset of the requested domains. This may be + useful for allowing renewals for multiple domains to + succeed even if some domains no longer point at this + system. This option cannot be used with --csr. + (default: False) + --agree-tos Agree to the ACME Subscriber Agreement (default: Ask) + --account ACCOUNT_ID Account ID to use (default: None) + --duplicate Allow making a certificate lineage that duplicates an + existing one (both can be renewed in parallel) + (default: False) + --os-packages-only (certbot-auto only) install OS package dependencies + and then stop (default: False) + --no-self-upgrade (certbot-auto only) prevent the certbot-auto script + from upgrading itself to newer released versions + (default: Upgrade automatically) + -q, --quiet Silence all output except errors. Useful for + automation via cron. Implies --non-interactive. + (default: False) + +security: + Security parameters & server settings + + --rsa-key-size N Size of the RSA key. (default: 2048) + --must-staple Adds the OCSP Must Staple extension to the + certificate. Autoconfigures OCSP Stapling for + supported setups (Apache version >= 2.3.3 ). (default: + False) + --redirect Automatically redirect all HTTP traffic to HTTPS for + the newly authenticated vhost. (default: Ask) + --no-redirect Do not automatically redirect all HTTP traffic to + HTTPS for the newly authenticated vhost. (default: + Ask) + --hsts Add the Strict-Transport-Security header to every HTTP + response. Forcing browser to always use SSL for the + domain. Defends against SSL Stripping. (default: + False) + --uir Add the "Content-Security-Policy: upgrade-insecure- + requests" header to every HTTP response. Forcing the + browser to use https:// for every http:// resource. + (default: None) + --staple-ocsp Enables OCSP Stapling. A valid OCSP response is + stapled to the certificate that the server offers + during TLS. (default: None) + --strict-permissions Require that all configuration files are owned by the + current user; only needed if your config is somewhere + unsafe like /tmp/ (default: False) + +testing: + The following flags are meant for testing and integration purposes only. + + --test-cert, --staging + Use the staging server to obtain or revoke test + (invalid) certs; equivalent to --server https://acme- + staging.api.letsencrypt.org/directory (default: False) + --debug Show tracebacks in case of errors, and allow certbot- + auto execution on experimental platforms (default: + False) + --no-verify-ssl Disable verification of the ACME server's certificate. + (default: False) + --tls-sni-01-port TLS_SNI_01_PORT + Port used during tls-sni-01 challenge. This only + affects the port Certbot listens on. A conforming ACME + server will still attempt to connect on port 443. + (default: 443) + --http-01-port HTTP01_PORT + Port used in the http-01 challenge. This only affects + the port Certbot listens on. A conforming ACME server + will still attempt to connect on port 80. (default: + 80) + --break-my-certs Be willing to replace or renew valid certs with + invalid (testing/staging) certs (default: False) + +paths: + Arguments changing execution paths & servers + + --cert-path CERT_PATH + Path to where cert is saved (with auth --csr), + installed from, or revoked. (default: None) + --key-path KEY_PATH Path to private key for cert installation or + revocation (if account key is missing) (default: None) + --chain-path CHAIN_PATH + Accompanying path to a certificate chain. (default: + None) + --config-dir CONFIG_DIR + Configuration directory. (default: /etc/letsencrypt) + --work-dir WORK_DIR Working directory. (default: /var/lib/letsencrypt) + --logs-dir LOGS_DIR Logs directory. (default: /var/log/letsencrypt) + --server SERVER ACME Directory Resource URI. (default: + https://acme-v01.api.letsencrypt.org/directory) + +manage: + Various subcommands and flags are available for managing your + certificates: + + certificates List certificates managed by Certbot + delete Clean up all files related to a certificate + renew Renew all certificates (or one specifed with --cert- + name) + revoke Revoke a certificate specified with --cert-path + update_symlinks Recreate symlinks in your /etc/letsencrypt/live/ + directory + +run: + Options for obtaining & installing certs + +certonly: + Options for modifying how a cert is obtained + + --csr CSR Path to a Certificate Signing Request (CSR) in DER or + PEM format. Currently --csr only works with the + 'certonly' subcommand. (default: None) + +renew: + The 'renew' subcommand will attempt to renew all certificates (or more + precisely, certificate lineages) you have previously obtained if they are + close to expiry, and print a summary of the results. By default, 'renew' + will reuse the options used to create obtain or most recently successfully + renew each certificate lineage. You can try it with `--dry-run` first. For + more fine-grained control, you can renew individual lineages with the + `certonly` subcommand. Hooks are available to run commands before and + after renewal; see https://certbot.eff.org/docs/using.html#renewal for + more information on these. + + --pre-hook PRE_HOOK Command to be run in a shell before obtaining any + certificates. Intended primarily for renewal, where it + can be used to temporarily shut down a webserver that + might conflict with the standalone plugin. This will + only be called if a certificate is actually to be + obtained/renewed. When renewing several certificates + that have identical pre-hooks, only the first will be + executed. (default: None) + --post-hook POST_HOOK + Command to be run in a shell after attempting to + obtain/renew certificates. Can be used to deploy + renewed certificates, or to restart any servers that + were stopped by --pre-hook. This is only run if an + attempt was made to obtain/renew a certificate. If + multiple renewed certificates have identical post- + hooks, only one will be run. (default: None) + --renew-hook RENEW_HOOK + Command to be run in a shell once for each + successfully renewed certificate. For this command, + the shell variable $RENEWED_LINEAGE will point to the + config live subdirectory containing the new certs and + keys; the shell variable $RENEWED_DOMAINS will contain + a space-delimited list of renewed cert domains + (default: None) + --disable-hook-validation + Ordinarily the commands specified for --pre-hook + /--post-hook/--renew-hook will be checked for + validity, to see if the programs being run are in the + $PATH, so that mistakes can be caught early, even when + the hooks aren't being run just yet. The validation is + rather simplistic and fails if you use more advanced + shell constructs, so you can use this switch to + disable it. (default: False) + +certificates: + List certificates managed by Certbot + +delete: + Options for deleting a certificate + +revoke: + Options for revocation of certs + +register: + Options for account registration & modification + --register-unsafely-without-email Specifying this flag enables registering an account with no email address. This is strongly discouraged, @@ -65,220 +300,38 @@ optional arguments: registering a new account. (default: False) -m EMAIL, --email EMAIL Email used for registration and recovery contact. - (default: None) - --preferred-challenges PREF_CHALLS - A sorted, comma delimited list of the preferred - challenge to use during authorization with the most - preferred challenge listed first (Eg, "dns" or "tls- - sni-01,http,dns"). Not all plugins support all - challenges. See - https://certbot.eff.org/docs/using.html#plugins for - details. ACME Challenges are versioned, but if you - pick "http" rather than "http-01", Certbot will select - the latest version automatically. (default: []) - --user-agent USER_AGENT - Set a custom user agent string for the client. User - agent strings allow the CA to collect high level - statistics about success rates by OS and plugin. If - you wish to hide your server OS version from the Let's - Encrypt server, set this to "". (default: None) - -automation: - Arguments for automating execution & other tweaks - - --keep-until-expiring, --keep, --reinstall - If the requested cert matches an existing cert, always - keep the existing one until it is due for renewal (for - the 'run' subcommand this means reinstall the existing - cert) (default: False) - --expand If an existing cert covers some subset of the - requested names, always expand and replace it with the - additional names. (default: False) - --version show program's version number and exit - --force-renewal, --renew-by-default - If a certificate already exists for the requested - domains, renew it now, regardless of whether it is - near expiry. (Often --keep-until-expiring is more - appropriate). Also implies --expand. (default: False) - --allow-subset-of-names - When performing domain validation, do not consider it - a failure if authorizations can not be obtained for a - strict subset of the requested domains. This may be - useful for allowing renewals for multiple domains to - succeed even if some domains no longer point at this - system. This option cannot be used with --csr. - (default: False) - --agree-tos Agree to the ACME Subscriber Agreement (default: - False) - --account ACCOUNT_ID Account ID to use (default: None) - --duplicate Allow making a certificate lineage that duplicates an - existing one (both can be renewed in parallel) - (default: False) - --os-packages-only (certbot-auto only) install OS package dependencies - and then stop (default: False) - --no-self-upgrade (certbot-auto only) prevent the certbot-auto script - from upgrading itself to newer released versions - (default: False) - -q, --quiet Silence all output except errors. Useful for - automation via cron. Implies --non-interactive. - (default: False) - -security: - Security parameters & server settings - - --rsa-key-size N Size of the RSA key. (default: 2048) - --must-staple Adds the OCSP Must Staple extension to the - certificate. Autoconfigures OCSP Stapling for - supported setups (Apache version >= 2.3.3 ). (default: - False) - --redirect Automatically redirect all HTTP traffic to HTTPS for - the newly authenticated vhost. (default: None) - --no-redirect Do not automatically redirect all HTTP traffic to - HTTPS for the newly authenticated vhost. (default: - None) - --hsts Add the Strict-Transport-Security header to every HTTP - response. Forcing browser to always use SSL for the - domain. Defends against SSL Stripping. (default: - False) - --no-hsts Do not automatically add the Strict-Transport-Security - header to every HTTP response. (default: False) - --uir Add the "Content-Security-Policy: upgrade-insecure- - requests" header to every HTTP response. Forcing the - browser to use https:// for every http:// resource. - (default: None) - --no-uir Do not automatically set the "Content-Security-Policy: - upgrade-insecure-requests" header to every HTTP - response. (default: None) - --staple-ocsp Enables OCSP Stapling. A valid OCSP response is - stapled to the certificate that the server offers - during TLS. (default: None) - --no-staple-ocsp Do not automatically enable OCSP Stapling. (default: - None) - --strict-permissions Require that all configuration files are owned by the - current user; only needed if your config is somewhere - unsafe like /tmp/ (default: False) - -testing: - The following flags are meant for testing purposes only! Do NOT change - them, unless you really know what you're doing! - - --test-cert, --staging - Use the staging server to obtain test (invalid) certs; - equivalent to --server https://acme- - staging.api.letsencrypt.org/directory (default: False) - --debug Show tracebacks in case of errors, and allow certbot- - auto execution on experimental platforms (default: - False) - --no-verify-ssl Disable verification of the ACME server's certificate. - (default: False) - --break-my-certs Be willing to replace or renew valid certs with - invalid (testing/staging) certs (default: False) - -renew: - The 'renew' subcommand will attempt to renew all certificates (or more - precisely, certificate lineages) you have previously obtained if they are - close to expiry, and print a summary of the results. By default, 'renew' - will reuse the options used to create, obtain or most recently successfully - renew each certificate lineage. You can try it with `--dry-run` first. For - more fine-grained control, you can renew individual lineages with the - `certonly` subcommand. Hooks are available to run commands before and - after renewal; see https://certbot.eff.org/docs/using.html#renewal for - more information on these. - - --pre-hook PRE_HOOK Command to be run in a shell before obtaining any - certificates. Intended primarily for renewal, where it - can be used to temporarily shut down a webserver that - might conflict with the standalone plugin. This will - only be called if a certificate is actually to be - obtained/renewed. (default: None) - --post-hook POST_HOOK - Command to be run in a shell after attempting to - obtain/renew certificates. Can be used to deploy - renewed certificates, or to restart any servers that - were stopped by --pre-hook. This is only run if an - attempt was made to obtain/renew a certificate. - (default: None) - --renew-hook RENEW_HOOK - Command to be run in a shell once for each - successfully renewed certificate. For this command, - the shell variable $RENEWED_LINEAGE will point to the - config live subdirectory containing the new certs and - keys; the shell variable $RENEWED_DOMAINS will contain - a space-delimited list of renewed cert domains - (default: None) - --disable-hook-validation - Ordinarily the commands specified for --pre-hook - /--post-hook/--renew-hook will be checked for - validity, to see if the programs being run are in the - $PATH, so that mistakes can be caught early, even when - the hooks aren't being run just yet. The validation is - rather simplistic and fails if you use more advanced - shell constructs, so you can use this switch to - disable it. (default: True) - -certonly: - Options for modifying how a cert is obtained - - --tls-sni-01-port TLS_SNI_01_PORT - Port used during tls-sni-01 challenge. This only - affects the port Certbot listens on. A conforming ACME - server will still attempt to connect on port 443. - (default: 443) - --http-01-port HTTP01_PORT - Port used in the http-01 challenge. This only affects - the port Certbot listens on. A conforming ACME server - will still attempt to connect on port 80. (default: - 80) - --csr CSR Path to a Certificate Signing Request (CSR) in DER or - PEM format. Currently --csr only works with the - 'certonly' subcommand. (default: None) + (default: Ask) install: Options for modifying how a cert is deployed -revoke: - Options for revocation of certs + --fullchain-path FULLCHAIN_PATH + Accompanying path to a full certificate chain (cert + plus chain). (default: None) + +config_changes: + Options for controlling which changes are displayed + + --num NUM How many past revisions you want to be displayed + (default: None) rollback: - Options for reverting config changes + Options for rolling back server configuration changes --checkpoints N Revert configuration N number of checkpoints. (default: 1) plugins: - Options for the "plugins" subcommand + Options for for the "plugins" subcommand --init Initialize plugins. (default: False) --prepare Initialize and prepare plugins. (default: False) --authenticators Limit to authenticator plugins only. (default: None) --installers Limit to installer plugins only. (default: None) -config_changes: - Options for showing a history of config changes - - --num NUM How many past revisions you want to be displayed - (default: None) - -paths: - Arguments changing execution paths & servers - - --cert-path CERT_PATH - Path to where cert is saved (with auth --csr), - installed from or revoked. (default: None) - --key-path KEY_PATH Path to private key for cert installation or - revocation (if account key is missing) (default: None) - --fullchain-path FULLCHAIN_PATH - Accompanying path to a full certificate chain (cert - plus chain). (default: None) - --chain-path CHAIN_PATH - Accompanying path to a certificate chain. (default: - None) - --config-dir CONFIG_DIR - Configuration directory. (default: /etc/letsencrypt) - --work-dir WORK_DIR Working directory. (default: /var/lib/letsencrypt) - --logs-dir LOGS_DIR Logs directory. (default: /var/log/letsencrypt) - --server SERVER ACME Directory Resource URI. (default: - https://acme-v01.api.letsencrypt.org/directory) +update_symlinks: + Recreates cert and key symlinks in /etc/letsencrypt/live, if you changed + them by hand or edited a renewal configuration file plugins: Plugin Selection: Certbot client supports an extensible plugins @@ -287,15 +340,15 @@ plugins: provided below. Running --help will list flags specific to that plugin. + --configurator CONFIGURATOR + Name of the plugin that is both an authenticator and + an installer. Should not be used together with + --authenticator or --installer. (default: Ask) -a AUTHENTICATOR, --authenticator AUTHENTICATOR Authenticator plugin name. (default: None) -i INSTALLER, --installer INSTALLER Installer plugin name (also used to find domains). (default: None) - --configurator CONFIGURATOR - Name of the plugin that is both an authenticator and - an installer. Should not be used together with - --authenticator or --installer. (default: None) --apache Obtain and install certs using Apache (default: False) --nginx Obtain and install certs using Nginx (default: False) --standalone Obtain certs using a "standalone" webserver. (default: @@ -318,13 +371,24 @@ standalone: Spin up a temporary webserver manual: - Manually configure an HTTP server + Authenticate through manual configuration or custom shell scripts. When + using shell scripts, an authenticator script must be provided. The + environment variables available to this script are $CERTBOT_DOMAIN which + contains the domain being authenticated, $CERTBOT_VALIDATION which is the + validation string, and $CERTBOT_TOKEN which is the filename of the + resource requested when performing an HTTP-01 challenge. An additional + cleanup script can also be provided and can use the additional variable + $CERTBOT_AUTH_OUTPUT which contains the stdout output from the auth + script. - --manual-test-mode Test mode. Executes the manual command in subprocess. - (default: False) + --manual-auth-hook MANUAL_AUTH_HOOK + Path or command to execute for the authentication + script (default: None) + --manual-cleanup-hook MANUAL_CLEANUP_HOOK + Path or command to execute for the cleanup script + (default: None) --manual-public-ip-logging-ok - Automatically allows public IP logging. (default: - False) + Automatically allows public IP logging (default: Ask) webroot: Place files in webroot directory @@ -335,7 +399,7 @@ webroot: domain will have the webroot path that preceded it. For instance: `-w /var/www/example -d example.com -d www.example.com -w /var/www/thing -d thing.net -d - m.thing.net` (default: []) + m.thing.net` (default: Ask) --webroot-map WEBROOT_MAP JSON dictionary mapping domains to webroot paths; this implies -d for each entry. You may need to escape this diff --git a/letsencrypt-auto b/letsencrypt-auto index cba185eae..a2ddf76ac 100755 --- a/letsencrypt-auto +++ b/letsencrypt-auto @@ -15,11 +15,15 @@ set -e # Work even if somebody does "sh thisscript.sh". # Note: you can set XDG_DATA_HOME or VENV_PATH before running this script, # if you want to change where the virtual environment will be installed -XDG_DATA_HOME=${XDG_DATA_HOME:-~/.local/share} +if [ -z "$XDG_DATA_HOME" ]; then + XDG_DATA_HOME=~/.local/share +fi VENV_NAME="letsencrypt" -VENV_PATH=${VENV_PATH:-"$XDG_DATA_HOME/$VENV_NAME"} +if [ -z "$VENV_PATH" ]; then + VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" +fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.9.3" +LE_AUTO_VERSION="0.10.0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -58,6 +62,7 @@ for arg in "$@" ; do --verbose) VERBOSE=1;; -[!-]*) + OPTIND=1 while getopts ":hnvq" short_arg $arg; do case "$short_arg" in h) @@ -79,43 +84,74 @@ if [ $BASENAME = "letsencrypt-auto" ]; then HELP=0 fi +# Support for busybox and others where there is no "command", +# but "which" instead +if command -v command > /dev/null 2>&1 ; then + export EXISTS="command -v" +elif which which > /dev/null 2>&1 ; then + export EXISTS="which" +else + echo "Cannot find command nor which... please install one!" + exit 1 +fi + # certbot-auto needs root access to bootstrap OS dependencies, and # certbot itself needs root access for almost all modes of operation # The "normal" case is that sudo is used for the steps that need root, but # this script *can* be run as root (not recommended), or fall back to using -# `su` +# `su`. Auto-detection can be overrided by explicitly setting the +# environment variable LE_AUTO_SUDO to 'sudo', 'sudo_su' or '' as used below. + +# Because the parameters in `su -c` has to be a string, +# we need to properly escape it. +su_sudo() { + args="" + # This `while` loop iterates over all parameters given to this function. + # For each parameter, all `'` will be replace by `'"'"'`, and the escaped string + # will be wrapped in a pair of `'`, then appended to `$args` string + # For example, `echo "It's only 1\$\!"` will be escaped to: + # 'echo' 'It'"'"'s only 1$!' + # │ │└┼┘│ + # │ │ │ └── `'s only 1$!'` the literal string + # │ │ └── `\"'\"` is a single quote (as a string) + # │ └── `'It'`, to be concatenated with the strings following it + # └── `echo` wrapped in a pair of `'`, it's totally fine for the shell command itself + while [ $# -ne 0 ]; do + args="$args'$(printf "%s" "$1" | sed -e "s/'/'\"'\"'/g")' " + shift + done + su root -c "$args" +} + SUDO_ENV="" export CERTBOT_AUTO="$0" -if test "`id -u`" -ne "0" ; then - if command -v sudo 1>/dev/null 2>&1; then - SUDO=sudo - SUDO_ENV="CERTBOT_AUTO=$0" - else - echo \"sudo\" is not available, will use \"su\" for installation steps... - # Because the parameters in `su -c` has to be a string, - # we need properly escape it - su_sudo() { - args="" - # This `while` loop iterates over all parameters given to this function. - # For each parameter, all `'` will be replace by `'"'"'`, and the escaped string - # will be wrapped in a pair of `'`, then appended to `$args` string - # For example, `echo "It's only 1\$\!"` will be escaped to: - # 'echo' 'It'"'"'s only 1$!' - # │ │└┼┘│ - # │ │ │ └── `'s only 1$!'` the literal string - # │ │ └── `\"'\"` is a single quote (as a string) - # │ └── `'It'`, to be concatenated with the strings following it - # └── `echo` wrapped in a pair of `'`, it's totally fine for the shell command itself - while [ $# -ne 0 ]; do - args="$args'$(printf "%s" "$1" | sed -e "s/'/'\"'\"'/g")' " - shift - done - su root -c "$args" - } - SUDO=su_sudo - fi +if [ -n "${LE_AUTO_SUDO+x}" ]; then + case "$LE_AUTO_SUDO" in + su_sudo|su) + SUDO=su_sudo + ;; + sudo) + SUDO=sudo + SUDO_ENV="CERTBOT_AUTO=$0" + ;; + '') ;; # Nothing to do for plain root method. + *) + echo "Error: unknown root authorization mechanism '$LE_AUTO_SUDO'." + exit 1 + esac + echo "Using preset root authorization mechanism '$LE_AUTO_SUDO'." else - SUDO= + if test "`id -u`" -ne "0" ; then + if $EXISTS sudo 1>/dev/null 2>&1; then + SUDO=sudo + SUDO_ENV="CERTBOT_AUTO=$0" + else + echo \"sudo\" is not available, will use \"su\" for installation steps... + SUDO=su_sudo + fi + else + SUDO= + fi fi ExperimentalBootstrap() { @@ -136,7 +172,7 @@ ExperimentalBootstrap() { DeterminePythonVersion() { for LE_PYTHON in "$LE_PYTHON" python2.7 python27 python2 python; do # Break (while keeping the LE_PYTHON value) if found. - command -v "$LE_PYTHON" > /dev/null && break + $EXISTS "$LE_PYTHON" > /dev/null && break done if [ "$?" != "0" ]; then echo "Cannot find any Pythons; please install one!" @@ -177,19 +213,22 @@ BootstrapDebCommon() { # distro version (#346) virtualenv= - if apt-cache show virtualenv > /dev/null 2>&1 && ! apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then - virtualenv="virtualenv" + # virtual env is known to apt and is installable + if apt-cache show virtualenv > /dev/null 2>&1 ; then + if ! LC_ALL=C apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then + virtualenv="virtualenv" + fi fi if apt-cache show python-virtualenv > /dev/null 2>&1; then - virtualenv="$virtualenv python-virtualenv" + virtualenv="$virtualenv python-virtualenv" fi augeas_pkg="libaugeas0 augeas-lenses" - AUGVERSION=`apt-cache show --no-all-versions libaugeas0 | grep ^Version: | cut -d" " -f2` + AUGVERSION=`LC_ALL=C apt-cache show --no-all-versions libaugeas0 | grep ^Version: | cut -d" " -f2` if [ "$ASSUME_YES" = 1 ]; then - YES_FLAG="-y" + YES_FLAG="-y" fi AddBackportRepo() { @@ -248,15 +287,15 @@ BootstrapDebCommon() { python-dev \ $virtualenv \ gcc \ - dialog \ $augeas_pkg \ libssl-dev \ + openssl \ libffi-dev \ ca-certificates \ - if ! command -v virtualenv > /dev/null ; then + if ! $EXISTS virtualenv > /dev/null ; then echo Failed to install a working \"virtualenv\" command, exiting exit 1 fi @@ -307,7 +346,6 @@ BootstrapRpmCommon() { pkgs=" gcc - dialog augeas-libs openssl openssl-devel @@ -361,7 +399,6 @@ BootstrapSuseCommon() { python-devel \ python-virtualenv \ gcc \ - dialog \ augeas-lenses \ libopenssl-devel \ libffi-devel \ @@ -380,7 +417,6 @@ BootstrapArchCommon() { python2 python-virtualenv gcc - dialog augeas openssl libffi @@ -404,22 +440,26 @@ BootstrapGentooCommon() { PACKAGES=" dev-lang/python:2.7 dev-python/virtualenv - dev-util/dialog app-admin/augeas dev-libs/openssl dev-libs/libffi app-misc/ca-certificates virtual/pkgconfig" + ASK_OPTION="--ask" + if [ "$ASSUME_YES" = 1 ]; then + ASK_OPTION="" + fi + case "$PACKAGE_MANAGER" in (paludis) $SUDO cave resolve --preserve-world --keep-targets if-possible $PACKAGES -x ;; (pkgcore) - $SUDO pmerge --noreplace --oneshot $PACKAGES + $SUDO pmerge --noreplace --oneshot $ASK_OPTION $PACKAGES ;; (portage|*) - $SUDO emerge --noreplace --oneshot $PACKAGES + $SUDO emerge --noreplace --oneshot $ASK_OPTION $PACKAGES ;; esac } @@ -449,7 +489,6 @@ BootstrapMac() { fi $pkgcmd augeas - $pkgcmd dialog if [ "$(which python)" = "/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python" \ -o "$(which python)" = "/usr/bin/python" ]; then # We want to avoid using the system Python because it requires root to use pip. @@ -458,7 +497,7 @@ BootstrapMac() { $pkgcmd python fi - # Workaround for _dlopen not finding augeas on OS X + # Workaround for _dlopen not finding augeas on macOS if [ "$pkgman" = "port" ] && ! [ -e "/usr/local/lib/libaugeas.dylib" ] && [ -e "/opt/local/lib/libaugeas.dylib" ]; then echo "Applying augeas workaround" $SUDO mkdir -p /usr/local/lib/ @@ -496,8 +535,8 @@ BootstrapMageiaCommon() { if ! $SUDO urpmi --force \ git \ gcc \ - cdialog \ python-augeas \ + openssl \ libopenssl-devel \ libffi-devel \ rootcerts @@ -541,7 +580,7 @@ Bootstrap() { elif uname | grep -iq FreeBSD ; then ExperimentalBootstrap "FreeBSD" BootstrapFreeBsd elif uname | grep -iq Darwin ; then - ExperimentalBootstrap "Mac OS X" BootstrapMac + ExperimentalBootstrap "macOS" BootstrapMac elif [ -f /etc/issue ] && grep -iq "Amazon Linux" /etc/issue ; then ExperimentalBootstrap "Amazon Linux" BootstrapRpmCommon elif [ -f /etc/product ] && grep -q "Joyent Instance" /etc/product ; then @@ -557,7 +596,7 @@ Bootstrap() { } TempDir() { - mktemp -d 2>/dev/null || mktemp -d -t 'le' # Linux || OS X + mktemp -d 2>/dev/null || mktemp -d -t 'le' # Linux || macOS } @@ -594,6 +633,11 @@ if [ "$1" = "--le-auto-phase2" ]; then # `pip install --no-cache-dir -e acme -e . -e certbot-apache -e certbot-nginx`, # and then use `hashin` or a more secure method to gather the hashes. +# Hashin example: +# pip install hashin +# hashin -r letsencrypt-auto-requirements.txt cryptography==1.5.2 +# sets the new certbot-auto pinned version of cryptography to 1.5.2 + argparse==1.4.0 \ --hash=sha256:c31647edb69fd3d465a847ea3157d37bed1f95f19760b11a47aa91c04b666314 \ --hash=sha256:62b089a55be1d8949cd2bc7e0df0bddb9e028faefc8c32038cc84862aefdd6e4 @@ -601,7 +645,8 @@ argparse==1.4.0 \ # This comes before cffi because cffi will otherwise install an unchecked # version via setup_requires. pycparser==2.14 \ - --hash=sha256:7959b4a74abdc27b312fed1c21e6caf9309ce0b29ea86b591fd2e99ecdf27f73 + --hash=sha256:7959b4a74abdc27b312fed1c21e6caf9309ce0b29ea86b591fd2e99ecdf27f73 \ + --no-binary pycparser cffi==1.4.2 \ --hash=sha256:53c1c9ddb30431513eb7f3cdef0a3e06b0f1252188aaa7744af0f5a4cd45dbaf \ @@ -624,29 +669,29 @@ ConfigArgParse==0.10.0 \ --hash=sha256:3b50a83dd58149dfcee98cb6565265d10b53e9c0a2bca7eeef7fb5f5524890a7 configobj==5.0.6 \ --hash=sha256:a2f5650770e1c87fb335af19a9b7eb73fc05ccf22144eb68db7d00cd2bcb0902 -cryptography==1.3.4 \ - --hash=sha256:bede00edd11a2a62c8c98c271cc103fa3a3d72acf64f6e5e4eaf251128897b17 \ - --hash=sha256:53b39e687b744bb548a98f40736cc529d9f60959b4e6cc551322cf9505d35eb3 \ - --hash=sha256:474b73ad1139b4e423e46bbd818efd0d5c0df1c65d9f7c957d64c9215d77afde \ - --hash=sha256:aaddf9592d5b99e32dd518bb4a25b147c124f9d6b4ad64b94f01b15d1666b8c8 \ - --hash=sha256:6dcad2f407db8c3cd6ecd78361439c449a4f94786b46c54507e7e68f51e1709d \ - --hash=sha256:475c153fc622e656f1f10a9c9941d0ac7ab18df7c38d35d563a437c1c0e34f24 \ - --hash=sha256:86dd61df581cba04e89e45081efbc531faff1c9d99c77b1ce97f87216c356353 \ - --hash=sha256:75cc697e4ef5fdd0102ca749114c6370dbd11db0c9132a18834858c2566247e3 \ - --hash=sha256:ea03ad5b9df6d79fc9fc1ab23729e01e1c920d2974c5e3c634ccf45a5c378452 \ - --hash=sha256:c8872b8fe4f3416d6338ab99612f49ab314f7856cb43bffab2a32d28a6267be8 \ - --hash=sha256:468fc6e16eaec6ceaa6bc341273e6e9912d01b42b740f8cf896ace7fcd6a321d \ - --hash=sha256:d6fea3c6502735011c5d61a62aef1c1d770fc6a2def45d9e6c0d94c9651e3317 \ - --hash=sha256:3cf95f179f4bead3d5649b91860ef4cf60ad4244209190fc405908272576d961 \ - --hash=sha256:141f77e60a5b9158309b2b60288c7f81d37faa15c22a69b94c190ceefaaa6236 \ - --hash=sha256:87b7a1fe703c6424451f3372d1879dae91c7fe5e13375441a72833db76fee30e \ - --hash=sha256:f5ee3cb0cf1a6550bf483ccffa6608db267a377b45f7e3a8201a86d1d8feb19f \ - --hash=sha256:4e097286651ea318300af3251375d48b71b8228481c56cd617ddd4459a1ff261 \ - --hash=sha256:1e3d3ae3f22f22d50d340f47f25227511326f3f1396c6d2446a5b45b516c4313 \ - --hash=sha256:6a057941cb64d79834ea3cf99093fcc4787c2a5d44f686c4f297361ddc419bcd \ - --hash=sha256:68b3d5390b92559ddd3353c73ab2dfcff758f9c4ec4f5d5226ccede0e5d779f4 \ - --hash=sha256:545dc003b4b6081f9c3e452da15d819b04b696f49484aff64c0a2aedf766bef8 \ - --hash=sha256:423ff890c01be7c70dbfeaa967eeef5146f1a43a5f810ffdc07b178e48a105a9 +cryptography==1.5.3 \ + --hash=sha256:e514d92086246b53ae9b048df652cf3036b462e50a6ce9fac6b6253502679991 \ + --hash=sha256:10ee414f4b5af403a0d8f20dfa80f7dad1fc7ae5452ec5af03712d5b6e78c664 \ + --hash=sha256:7234456d1f4345a144ed07af2416c7c0659d4bb599dd1a963103dc8c183b370e \ + --hash=sha256:d3b9587406f94642bd70b3d666b813f446e95f84220c9e416ad94cbfb6be2eaa \ + --hash=sha256:b15fc6b59f1474eef62207c85888afada8acc47fae8198ba2b0197d54538961a \ + --hash=sha256:3b62d65d342704fc07ed171598db2a2775bdf587b1b6abd2cba2261bfe3ccde3 \ + --hash=sha256:059343022ec904c867a13bc55d2573e36c8cfb2c250e30d8a2e9825f253b07ba \ + --hash=sha256:c7897cf13bc8b4ee0215d83cbd51766d87c06b277fcca1f9108595508e5bcfb4 \ + --hash=sha256:9b69e983e5bf83039ddd52e52a28c7faedb2b22bdfb5876377b95aac7d3be63e \ + --hash=sha256:61e40905c426d02b3fae38088dc66ce4ef84830f7eb223dec6b3ac3ccdc676fb \ + --hash=sha256:00783a32bcd91a12177230d35bfcf70a2333ade4a6b607fac94a633a7971c671 \ + --hash=sha256:d11973f49b648cde1ea1a30e496d7557dbfeccd08b3cd9ba58d286a9c274ff8e \ + --hash=sha256:f24bedf28b81932ba6063aec9a826669f5237ea3b755efe04d98b072faa053a5 \ + --hash=sha256:3ab5725367239e3deb9b92e917aa965af3fef008f25b96a3000821869e208181 \ + --hash=sha256:8a53209de822e22b5f73bf4b99e68ac4ccc91051fd6751c8252982983e86a77d \ + --hash=sha256:5a07439d4b1e4197ac202b7eea45e26a6fd65757652dc50f1a63367f711df933 \ + --hash=sha256:26b1c4b40aec7b0074bceabe6e06565aa28176eca7323a31df66ebf89fe916d3 \ + --hash=sha256:eaa4a7b5a6682adcf8d6ebb2a08a008802657643655bb527c95c8a3860253d8e \ + --hash=sha256:8156927dcf8da274ff205ad0612f75c380df45385bacf98531a5b3348c88d135 \ + --hash=sha256:61ec0d792749d0e91e84b1d58b6dfd204806b10b5811f846c2ceca0de028c53a \ + --hash=sha256:26330c88041569ca621cc42274d0ea2667a48b6deab41467272c3aba0b6e8f07 \ + --hash=sha256:cf82ddac919b587f5e44247579b433224cc2e03332d2ea4d89aa70d7e6b64ae5 enum34==1.1.2 \ --hash=sha256:2475d7fcddf5951e92ff546972758802de5260bf409319a9f1934e6bbc8b1dc7 \ --hash=sha256:35907defb0f992b75ab7788f65fedc1cf20ffa22688e0e6f6f12afc06b3ea501 @@ -662,8 +707,6 @@ ipaddress==1.0.16 \ linecache2==1.0.0 \ --hash=sha256:e78be9c0a0dfcbac712fe04fbf92b96cddae80b1b842f24248214c8496f006ef \ --hash=sha256:4b26ff4e7110db76eeb6f5a7b64a82623839d595c2038eeda662f2a2db78e97c -ndg-httpsclient==0.4.0 \ - --hash=sha256:e8c155fdebd9c4bcb0810b4ed01ae1987554b1ee034dd7532d7b8fdae38a6274 ordereddict==1.1 \ --hash=sha256:1c35b4ac206cef2d24816c89f89cf289dd3d38cf7c449bb3fab7bf6d43f01b1f parsedatetime==2.1 \ @@ -684,9 +727,9 @@ pyasn1==0.1.9 \ --hash=sha256:5191ff6b9126d2c039dd87f8ff025bed274baf07fa78afa46f556b1ad7265d6e \ --hash=sha256:8323e03637b2d072cc7041300bac6ec448c3c28950ab40376036788e9a1af629 \ --hash=sha256:853cacd96d1f701ddd67aa03ecc05f51890135b7262e922710112f12a2ed2a7f -pyopenssl==16.0.0 \ - --hash=sha256:5add70cf00273bf957ca31fdb0df9b0ae4639e081897d5f86a0ae1f104901230 \ - --hash=sha256:363d10ee43d062285facf4e465f4f5163f9f702f9134f0a5896f134cbb92d17d +pyOpenSSL==16.2.0 \ + --hash=sha256:26ca380ddf272f7556e48064bbcd5bd71f83dfc144f3583501c7ddbd9434ee17 \ + --hash=sha256:7779a3bbb74e79db234af6a08775568c6769b5821faecf6e2f4143edb227516e pyparsing==2.1.8 \ --hash=sha256:2f0f5ceb14eccd5aef809d6382e87df22ca1da583c79f6db01675ce7d7f49c18 \ --hash=sha256:03a4869b9f3493807ee1f1cb405e6d576a1a2ca4d81a982677c0c1ad6177c56b \ @@ -701,9 +744,6 @@ pyRFC3339==1.0 \ --hash=sha256:8dfbc6c458b8daba1c0f3620a8c78008b323a268b27b7359e92a4ae41325f535 python-augeas==0.5.0 \ --hash=sha256:67d59d66cdba8d624e0389b87b2a83a176f21f16a87553b50f5703b23f29bac2 -python2-pythondialog==3.3.0 \ - --hash=sha256:04e93f24995c43dd90f338d5d865ca72ce3fb5a5358d4daa4965571db35fc3ec \ - --hash=sha256:3e6f593fead98f8a526bc3e306933533236e33729f552f52896ea504f55313fa pytz==2015.7 \ --hash=sha256:3abe6a6d3fc2fbbe4c60144211f45da2edbe3182a6f6511af6bbba0598b1f992 \ --hash=sha256:939ef9c1e1224d980405689a97ffcf7828c56d1517b31d73464356c1f2b7769e \ @@ -718,9 +758,9 @@ pytz==2015.7 \ --hash=sha256:fbd26746772c24cb93c8b97cbdad5cb9e46c86bbdb1b9d8a743ee00e2fb1fc5d \ --hash=sha256:99266ef30a37e43932deec2b7ca73e83c8dbc3b9ff703ec73eca6b1dae6befea \ --hash=sha256:8b6ce1c993909783bc96e0b4f34ea223bff7a4df2c90bdb9c4e0f1ac928689e3 -requests==2.9.1 \ - --hash=sha256:113fbba5531a9e34945b7d36b33a084e8ba5d0664b703c81a7c572d91919a5b8 \ - --hash=sha256:c577815dd00f1394203fc44eb979724b098f88264a9ef898ee45b8e5e9cf587f +requests==2.12.1 \ + --hash=sha256:3f3f27a9d0f9092935efc78054ef324eb9f8166718270aefe036dfa1e4f68e1e \ + --hash=sha256:2109ecea94df90980be040490ff1d879971b024861539abb00054062388b612e six==1.10.0 \ --hash=sha256:0ff78c403d9bccf5a425a6d31a12aa6b47f1c21ca4dc2573a7e2f32a97335eb1 \ --hash=sha256:105f8d68616f8248e24bf0e9372ef04d3cc10104f1980f54d57b2ce73a5ad56a @@ -761,18 +801,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.9.3 \ - --hash=sha256:d18ce17a75ad24d27981dfaef0524aa905eab757b267e027162b56a8967ab8fb \ - --hash=sha256:a6eff1f955eb2e4316abd9aa2fedb6d9345e6b5b8a2d64ea0ad35e05d6124099 -certbot==0.9.3 \ - --hash=sha256:a87ef4c53c018df4e52ee2f2e906ad16bbb37789f29e6f284c495a2eb4d9b243 \ - --hash=sha256:68149cb8392b29f5d5246e7226d25f913f2b10482bf3bc7368e8c8821d25f3b0 -certbot-apache==0.9.3 \ - --hash=sha256:f379b1053e10709692654d7a6fcea9eaed19b66c49a753b61e31bd06a04b0aac \ - --hash=sha256:a5d98cf972072de08f984db4e6a7f20269f3f023c43f6d4e781fe43be7c10086 -certbot-nginx==0.9.3 \ - --hash=sha256:3c26f18f0b57550f069263bd9b2984ef33eab6693e7796611c1b2cc16574069c \ - --hash=sha256:7337a2e90e0b28a1ab09e31d9fb81c6d78e6453500c824c0f18bab5d31b63058 +acme==0.10.0 \ + --hash=sha256:df4299a9881d94185a1578ed97334430a90f761ce815edd300860ca47d0538f1 \ + --hash=sha256:ddebdf1fe139c8fedbcf633955ec867496d2f7d2d2e9879d538437a69ab47876 +certbot==0.10.0 \ + --hash=sha256:fb1bfa3d54ce9366758e374f7ed99667ce20484224934d3e8e57839fcf784bc5 \ + --hash=sha256:dd64ed8fb3cc3b053f05e779b934433445918668c49bcdbb2c816062815e1661 +certbot-apache==0.10.0 \ + --hash=sha256:909d59c53507093f838f7336f75d7d78563a35b16afdf6c30f45c9f47bf069da \ + --hash=sha256:6f110dae227dd0fea9572fa12dd60b041e391f5d2028cc2e1fedd2a9a0d2bc88 +certbot-nginx==0.10.0 \ + --hash=sha256:4f33a230d420cbd0431e7b707fb9a1732bfd18d3c6056019591bd7c3a13abe92 \ + --hash=sha256:c12ffd05207b0be3c765b3d3e2927e0b2cc2b7de20654b19d154a0d789e7c1d5 UNLIKELY_EOF # ------------------------------------------------------------------------- @@ -940,7 +980,28 @@ UNLIKELY_EOF # Report error. (Otherwise, be quiet.) echo "Had a problem while installing Python packages." if [ "$VERBOSE" != 1 ]; then + echo + echo "pip prints the following errors: " + echo "=====================================================" echo "$PIP_OUT" + echo "=====================================================" + echo + echo "Certbot has problem setting up the virtual environment." + + if `echo $PIP_OUT | grep -q Killed` || `echo $PIP_OUT | grep -q "allocate memory"` ; then + echo + echo "Based on your pip output, the problem can likely be fixed by " + echo "increasing the available memory." + else + echo + echo "We were not be able to guess the right solution from your pip " + echo "output." + fi + + echo + echo "Consult https://certbot.eff.org/docs/install.html#problems-with-python-virtual-environment" + echo "for possible solutions." + echo "You may also find some support resources at https://certbot.eff.org/support/ ." fi rm -rf "$VENV_PATH" exit 1 @@ -1132,7 +1193,7 @@ UNLIKELY_EOF # TODO: Deal with quotes in pathnames. echo "Replacing certbot-auto..." # Clone permissions with cp. chmod and chown don't have a --reference - # option on OS X or BSD, and stat -c on Linux is stat -f on OS X and BSD: + # option on macOS or BSD, and stat -c on Linux is stat -f on macOS and BSD: $SUDO cp -p "$0" "$TEMP_DIR/letsencrypt-auto.permission-clone" $SUDO cp "$TEMP_DIR/letsencrypt-auto" "$TEMP_DIR/letsencrypt-auto.permission-clone" # Using mv rather than cp leaves the old file descriptor pointing to the diff --git a/letsencrypt-auto-source/certbot-auto.asc b/letsencrypt-auto-source/certbot-auto.asc index db40cfb84..2dfa27621 100644 --- a/letsencrypt-auto-source/certbot-auto.asc +++ b/letsencrypt-auto-source/certbot-auto.asc @@ -1,11 +1,11 @@ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 -iQEcBAABAgAGBQJYADL6AAoJEE0XyZXNl3XyZW8H/RgPxga4SZ8VoMGGOpzYGzaD -C/VW6IZeHjD7urkAjfSiMMStkYKlZMGcT/3Pw1L39wIX/37jqQTTh01JL+TcqRMJ -AUHmSgrErjUU42YV68u2c/wT9Dsid+OxpP/WSbJn5MomWtvGpFxffc/FK/W8ccFR -r6ZhAt2rgkBmYjrC6w8V9KTzhp4+n7ZpQPxuMFxpJhyTmMzgj9K+aI2OuKDKT7iO -nke74Lgx/xPatLDgygw5bRiFyZ+X65p/awalEXBcFW0zmlN2Fqp8om8UjtUtkVw9 -ixr9/kq9VhcHjho9cmKWl14IShbcxZZc60xL2y6gmkgoBpzVlHfvRNnxapodTsc= -=jULW +iQEcBAABAgAGBQJYdmhCAAoJEE0XyZXNl3XyuSMH/i6+2GqLh00I+VQRUUHmY/CE +PeUmrkN2N6DEFZK6Y6r7vR1QoY8xYEbmMZNmCYU+YRiO/TO3mLLycd48vbQoyttL +Bi4JalkfkLgfNZNLYvlrDE5K7LaHIiPxQfHN2RIZS4ez6eMREyQXhTPq5HGqQuQH +KkiC9CCKrLvmZXOZA+8ayvoo3U3SI1bZNu7d7c4pEDtkGRMZhNSs8Eejo+knDlny +KmEVrvakkcYTeGwz+SckY9Z7rQGyYoFr2+N3owMT40/g9ZnzkaTS/y+G2z1EnWkN +lapwugl9Pnl6Hog+SBH+osONdg04tIiNayPq11NgWNmMvbG6Lbi4p+RVg+16E1M= +=BXeZ -----END PGP SIGNATURE----- diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index efcabcb0f..409d68322 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.10.0.dev0" +LE_AUTO_VERSION="0.11.0.dev0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -801,18 +801,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.9.3 \ - --hash=sha256:d18ce17a75ad24d27981dfaef0524aa905eab757b267e027162b56a8967ab8fb \ - --hash=sha256:a6eff1f955eb2e4316abd9aa2fedb6d9345e6b5b8a2d64ea0ad35e05d6124099 -certbot==0.9.3 \ - --hash=sha256:a87ef4c53c018df4e52ee2f2e906ad16bbb37789f29e6f284c495a2eb4d9b243 \ - --hash=sha256:68149cb8392b29f5d5246e7226d25f913f2b10482bf3bc7368e8c8821d25f3b0 -certbot-apache==0.9.3 \ - --hash=sha256:f379b1053e10709692654d7a6fcea9eaed19b66c49a753b61e31bd06a04b0aac \ - --hash=sha256:a5d98cf972072de08f984db4e6a7f20269f3f023c43f6d4e781fe43be7c10086 -certbot-nginx==0.9.3 \ - --hash=sha256:3c26f18f0b57550f069263bd9b2984ef33eab6693e7796611c1b2cc16574069c \ - --hash=sha256:7337a2e90e0b28a1ab09e31d9fb81c6d78e6453500c824c0f18bab5d31b63058 +acme==0.10.0 \ + --hash=sha256:df4299a9881d94185a1578ed97334430a90f761ce815edd300860ca47d0538f1 \ + --hash=sha256:ddebdf1fe139c8fedbcf633955ec867496d2f7d2d2e9879d538437a69ab47876 +certbot==0.10.0 \ + --hash=sha256:fb1bfa3d54ce9366758e374f7ed99667ce20484224934d3e8e57839fcf784bc5 \ + --hash=sha256:dd64ed8fb3cc3b053f05e779b934433445918668c49bcdbb2c816062815e1661 +certbot-apache==0.10.0 \ + --hash=sha256:909d59c53507093f838f7336f75d7d78563a35b16afdf6c30f45c9f47bf069da \ + --hash=sha256:6f110dae227dd0fea9572fa12dd60b041e391f5d2028cc2e1fedd2a9a0d2bc88 +certbot-nginx==0.10.0 \ + --hash=sha256:4f33a230d420cbd0431e7b707fb9a1732bfd18d3c6056019591bd7c3a13abe92 \ + --hash=sha256:c12ffd05207b0be3c765b3d3e2927e0b2cc2b7de20654b19d154a0d789e7c1d5 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/letsencrypt-auto.sig b/letsencrypt-auto-source/letsencrypt-auto.sig index f3950b7d64ea779f03f935b957936a5273e20767..1657814ae2918ce8eaff699089bfc37952719d2d 100644 GIT binary patch literal 256 zcmV+b0ssEO!rDvu|K!QtZYXWV%TFVV8gb?=Y6getf$}2|kYb$M>TY5%I*K*2 zkHrc@xHXJ!g;;N&JO! z^7;_i41^#RU3P8@Q+H1QFg_TYuVYh~D#VG$8+3TA%r!Nf~d5_u|cJ-!-Tl)0JeLgmWm!v7R9B@Feh697L+tz>IPwTZ3M%f~epL zR+(>@Y)&;4>Vk4%1FfP;(Q=HW((jgT?+#k#=Vi>HxNB;^G}!a4jBTrKn~;d%r*p{V G#$5MT9fqO+ literal 256 zcmV+b0ssDRCvaD}1PJYxJ7X8ROr?`s(?BE!MRwbUu@HJmfG_K(FD%Jlo=!*v6>VHs zStR_UX!Ns;l*829-?y0LzMsj5ag3>P^-a+az`q84VnGD0L*{==#Vow+-<#g7CkT)A zW+lbL0F>^blSiI(QZDV4yYWBo_c^d8Sas}l}uktDA-T_ Date: Wed, 11 Jan 2017 19:57:46 -0500 Subject: [PATCH 05/88] make config-dir/work-dir/logs-dir output match help (#4017) --- certbot/main.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/certbot/main.py b/certbot/main.py index 91b860dbb..d076fe4bc 100644 --- a/certbot/main.py +++ b/certbot/main.py @@ -39,7 +39,7 @@ from certbot.plugins import selection as plug_sel _PERM_ERR_FMT = os.linesep.join(( "The following error was encountered:", "{0}", "If running as non-root, set --config-dir, " - "--logs-dir, and --work-dir to writeable paths.")) + "--work-dir, and --logs-dir to writeable paths.")) USER_CANCELLED = ("User chose to cancel the operation and may " "reinvoke the client.") From 94c23479e21d0387d3718639f1fb755c8895ca7b Mon Sep 17 00:00:00 2001 From: Craig Smith Date: Thu, 12 Jan 2017 12:56:55 +1030 Subject: [PATCH 06/88] Add option to specify revocation reason (#3242) (#3988) This includes two new tests in the integration test script to check that boulder gets the correct code. The encoding is specified in RFC5280 5.3.1. The codes that boulder will accept are a subset of that, specified in `boulder.revocation.reasons.go`. --- acme/acme/client.py | 8 ++++++-- acme/acme/client_test.py | 16 ++++++++++++++-- acme/acme/messages.py | 1 + certbot/cli.py | 24 ++++++++++++++++++++++++ certbot/constants.py | 11 +++++++++++ certbot/main.py | 4 +++- certbot/tests/cli_test.py | 9 +++++++++ certbot/tests/main_test.py | 26 +++++++++++++++++++++----- tests/boulder-integration.sh | 9 ++++++++- 9 files changed, 97 insertions(+), 11 deletions(-) diff --git a/acme/acme/client.py b/acme/acme/client.py index b5db57235..26109352b 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -481,17 +481,21 @@ class Client(object): # pylint: disable=too-many-instance-attributes "Recursion limit reached. Didn't get {0}".format(uri)) return chain - def revoke(self, cert): + def revoke(self, cert, rsn): """Revoke certificate. :param .ComparableX509 cert: `OpenSSL.crypto.X509` wrapped in `.ComparableX509` + :param int rsn: Reason code for certificate revocation. + :raises .ClientError: If revocation is unsuccessful. """ response = self.net.post(self.directory[messages.Revocation], - messages.Revocation(certificate=cert), + messages.Revocation( + certificate=cert, + reason=rsn), content_type=None) if response.status_code != http_client.OK: raise errors.ClientError( diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index e0403ef28..4822a1ae6 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -81,6 +81,9 @@ class ClientTest(unittest.TestCase): uri='https://www.letsencrypt-demo.org/acme/cert/1', cert_chain_uri='https://www.letsencrypt-demo.org/ca') + # Reason code for revocation + self.rsn = 1 + def test_init_downloads_directory(self): uri = 'http://www.letsencrypt-demo.org/directory' from acme.client import Client @@ -427,13 +430,22 @@ class ClientTest(unittest.TestCase): self.assertRaises(errors.Error, self.client.fetch_chain, self.certr) def test_revoke(self): - self.client.revoke(self.certr.body) + self.client.revoke(self.certr.body, self.rsn) self.net.post.assert_called_once_with( self.directory[messages.Revocation], mock.ANY, content_type=None) + def test_revocation_payload(self): + obj = messages.Revocation(certificate=self.certr.body, reason=self.rsn) + self.assertTrue('reason' in obj.to_partial_json().keys()) + self.assertEquals(self.rsn, obj.to_partial_json()['reason']) + def test_revoke_bad_status_raises_error(self): self.response.status_code = http_client.METHOD_NOT_ALLOWED - self.assertRaises(errors.ClientError, self.client.revoke, self.certr) + self.assertRaises( + errors.ClientError, + self.client.revoke, + self.certr, + self.rsn) class ClientNetworkTest(unittest.TestCase): diff --git a/acme/acme/messages.py b/acme/acme/messages.py index a7c86a10c..29d719684 100644 --- a/acme/acme/messages.py +++ b/acme/acme/messages.py @@ -469,3 +469,4 @@ class Revocation(jose.JSONObjectWithFields): resource = fields.Resource(resource_type) certificate = jose.Field( 'certificate', decoder=jose.decode_cert, encoder=jose.encode_cert) + reason = jose.Field('reason') diff --git a/certbot/cli.py b/certbot/cli.py index 31b5bafb5..d51fd58e0 100644 --- a/certbot/cli.py +++ b/certbot/cli.py @@ -1072,6 +1072,11 @@ def _create_subparsers(helpful): "--csr", type=read_file, help="Path to a Certificate Signing Request (CSR) in DER or PEM format." " Currently --csr only works with the 'certonly' subcommand.") + helpful.add("revoke", + "--reason", dest="reason", + choices=CaseInsensitiveList(constants.REVOCATION_REASONS.keys()), + action=_EncodeReasonAction, default=0, + help="Specify reason for revoking certificate.") helpful.add("rollback", "--checkpoints", type=int, metavar="N", default=flag_default("rollback_checkpoints"), @@ -1088,6 +1093,16 @@ def _create_subparsers(helpful): const=interfaces.IInstaller, help="Limit to installer plugins only.") +class CaseInsensitiveList(list): + """A list that will ignore case when searching. + + This class is passed to the `choices` argument of `argparse.add_arguments` + through the `helpful` wrapper. It is necessary due to special handling of + command line arguments by `set_by_cli` in which the `type_func` is not applied.""" + def __contains__(self, element): + return super(CaseInsensitiveList, self).__contains__(element.lower()) + + def _paths_parser(helpful): add = helpful.add verb = helpful.verb @@ -1166,6 +1181,15 @@ def _plugins_parsing(helpful, plugins): helpful.add_plugin_args(plugins) +class _EncodeReasonAction(argparse.Action): + """Action class for parsing revocation reason.""" + + def __call__(self, parser, namespace, reason, option_string=None): + """Encodes the reason for certificate revocation.""" + code = constants.REVOCATION_REASONS[reason.lower()] + setattr(namespace, self.dest, code) + + class _DomainsAction(argparse.Action): """Action class for parsing domains.""" diff --git a/certbot/constants.py b/certbot/constants.py index 7d713d29f..f64ff7e1e 100644 --- a/certbot/constants.py +++ b/certbot/constants.py @@ -35,6 +35,17 @@ CLI_DEFAULTS = dict( ) STAGING_URI = "https://acme-staging.api.letsencrypt.org/directory" +# The set of reasons for revoking a certificate is defined in RFC 5280 in +# section 5.3.1. The reasons that users are allowed to submit are restricted to +# those accepted by the ACME server implementation. They are listed in +# `letsencrypt.boulder.revocation.reasons.go`. +REVOCATION_REASONS = { + "unspecified": 0, + "keycompromise": 1, + "affiliationchanged": 3, + "superseded": 4, + "cessationofoperation": 5} + """Defaults for CLI flags and `.IConfig` attributes.""" QUIET_LOGGING_LEVEL = logging.WARNING diff --git a/certbot/main.py b/certbot/main.py index d076fe4bc..ec901c501 100644 --- a/certbot/main.py +++ b/certbot/main.py @@ -550,8 +550,10 @@ def revoke(config, unused_plugins): # TODO: coop with renewal config key = acc.key acme = client.acme_from_config_key(config, key) cert = crypto_util.pyopenssl_load_certificate(config.cert_path[1])[0] + logger.debug("Reason code for revocation: %s", config.reason) + try: - acme.revoke(jose.ComparableX509(cert)) + acme.revoke(jose.ComparableX509(cert), config.reason) except acme_errors.ClientError as e: return e.message diff --git a/certbot/tests/cli_test.py b/certbot/tests/cli_test.py index 9404a8385..32aada811 100644 --- a/certbot/tests/cli_test.py +++ b/certbot/tests/cli_test.py @@ -121,6 +121,7 @@ class ParseTest(unittest.TestCase): out = self._help_output(['--help', 'revoke']) self.assertTrue("--cert-path" in out) self.assertTrue("--key-path" in out) + self.assertTrue("--reason" in out) out = self._help_output(['-h', 'config_changes']) self.assertTrue("--cert-path" not in out) @@ -262,6 +263,14 @@ class ParseTest(unittest.TestCase): self.assertFalse(cli.option_was_set( config_dir_option, cli.flag_default(config_dir_option))) + def test_encode_revocation_reason(self): + for reason, code in constants.REVOCATION_REASONS.items(): + namespace = self.parse(['--reason', reason]) + self.assertEqual(namespace.reason, code) + for reason, code in constants.REVOCATION_REASONS.items(): + namespace = self.parse(['--reason', reason.upper()]) + self.assertEqual(namespace.reason, code) + def test_force_interactive(self): self.assertRaises( errors.Error, self.parse, "renew --force-interactive".split()) diff --git a/certbot/tests/main_test.py b/certbot/tests/main_test.py index 99aa0bdda..01ec2f061 100644 --- a/certbot/tests/main_test.py +++ b/certbot/tests/main_test.py @@ -215,7 +215,7 @@ class RevokeTest(unittest.TestCase): 'cert.pem')) self.patches = [ - mock.patch('acme.client.Client'), + mock.patch('acme.client.Client', autospec=True), mock.patch('certbot.client.Client'), mock.patch('certbot.main._determine_account'), mock.patch('certbot.main.display_ops.success_revocation') @@ -241,8 +241,9 @@ class RevokeTest(unittest.TestCase): for patch in self.patches: patch.stop() - def _call(self): - args = 'revoke --cert-path={0}'.format(self.tmp_cert_path).split() + def _call(self, extra_args=""): + args = 'revoke --cert-path={0} ' + extra_args + args = args.format(self.tmp_cert_path).split() plugins = disco.PluginsRegistry.find_all() config = configuration.NamespaceConfig( cli.prepare_and_parse_args(plugins, args)) @@ -250,6 +251,17 @@ class RevokeTest(unittest.TestCase): from certbot.main import revoke revoke(config, plugins) + @mock.patch('certbot.main.client.acme_client') + def test_revoke_with_reason(self, mock_acme_client): + mock_revoke = mock_acme_client.Client().revoke + expected = [] + for reason, code in constants.REVOCATION_REASONS.items(): + self._call("--reason " + reason) + expected.append(mock.call(mock.ANY, code)) + self._call("--reason " + reason.upper()) + expected.append(mock.call(mock.ANY, code)) + self.assertEqual(expected, mock_revoke.call_args_list) + def test_revocation_success(self): self._call() self.mock_success_revoke.assert_called_once_with(self.tmp_cert_path) @@ -1062,7 +1074,9 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods with open(CERT, 'rb') as f: cert = crypto_util.pyopenssl_load_certificate(f.read())[0] mock_revoke = mock_acme_client.Client().revoke - mock_revoke.assert_called_once_with(jose.ComparableX509(cert)) + mock_revoke.assert_called_once_with( + jose.ComparableX509(cert), + mock.ANY) @mock.patch('certbot.main._determine_account') def test_revoke_without_key(self, mock_determine_account): @@ -1071,7 +1085,9 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods with open(CERT) as f: cert = crypto_util.pyopenssl_load_certificate(f.read())[0] mock_revoke = client.acme_from_config_key().revoke - mock_revoke.assert_called_once_with(jose.ComparableX509(cert)) + mock_revoke.assert_called_once_with( + jose.ComparableX509(cert), + mock.ANY) def test_agree_dev_preview_config(self): with mock.patch('certbot.main.run') as mocked_run: diff --git a/tests/boulder-integration.sh b/tests/boulder-integration.sh index fe2f223ef..30fc17f81 100755 --- a/tests/boulder-integration.sh +++ b/tests/boulder-integration.sh @@ -162,7 +162,14 @@ common revoke --cert-path "$root/conf/live/le1.wtf/cert.pem" # revoke by cert key common revoke --cert-path "$root/conf/live/le2.wtf/cert.pem" \ --key-path "$root/conf/live/le2.wtf/privkey.pem" - +# Get new certs to test revoke with a reason, by account and by cert key +common --domains le1.wtf +common revoke --cert-path "$root/conf/live/le1.wtf/cert.pem" \ + --reason cessationOfOperation +common --domains le2.wtf +common revoke --cert-path "$root/conf/live/le2.wtf/cert.pem" \ + --key-path "$root/conf/live/le2.wtf/privkey.pem" \ + --reason keyCompromise if type nginx; then . ./certbot-nginx/tests/boulder-integration.sh From ad65b6317ddc4bb5fd374daabdea6b0ca96cd61f Mon Sep 17 00:00:00 2001 From: Peter Conrad Date: Thu, 12 Jan 2017 17:27:59 -0800 Subject: [PATCH 07/88] Edits to using.rst --- docs/using.rst | 59 ++++++++++++++++++++------------------------------ 1 file changed, 24 insertions(+), 35 deletions(-) diff --git a/docs/using.rst b/docs/using.rst index a5493b145..a1881852e 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -8,10 +8,10 @@ User Guide Certbot Commands ================ -Certbot uses a number of different "commands" (also referred -to, equivalently, as "subcommands") to request specific actions such as -obtaining, renewing, or revoking certificates. Some of the most important -and most commonly-used commands will be discussed throughout this +Certbot uses a number of different commands (also referred +to as "subcommands") to request specific actions such as +obtaining, renewing, or revoking certificates. The most important +and commonly-used commands will be discussed throughout this document; an exhaustive list also appears near the end of the document. The ``certbot`` script on your web server might be named ``letsencrypt`` if your system uses an older package, or ``certbot-auto`` if you used an alternate installation method. Throughout the docs, whenever you see ``certbot``, swap in the correct name as needed. @@ -21,24 +21,24 @@ The ``certbot`` script on your web server might be named ``letsencrypt`` if your Getting certificates (and choosing plugins) =========================================== -The Certbot client supports a number of different "plugins" that can be -used to obtain and/or install certificates. +The Certbot client supports two types of plugins for +obtaining and installing certificates. -Plugins that can obtain a cert are called "authenticators" and can be used with -the "certonly" command. This will carry out the steps needed to validate that you -control the domain(s) you are requesting a cert for, obtain a cert for the specified -domain(s), and place it in the ``/etc/letsencrypt`` directory on your -machine - without editing any of your server's configuration files to serve the -obtained certificate. If you specify multiple domains to authenticate, they will +Authenticators are plugins used with the ``certonly`` command to obtain a cert. +The authenticator validates that you +control the domain(s) you are requesting a cert for, obtains a cert for the specified +domain(s), and places the cert in the ``/etc/letsencrypt`` directory on your +machine. The authenticator does not install the cert (it does not edit any of your server's configuration files to serve the +obtained certificate). If you specify multiple domains to authenticate, they will all be listed in a single certificate. To obtain multiple seperate certificates you will need to run Certbot multiple times. -Plugins that can install a cert are called "installers" and can be used with the -"install" command. These plugins can modify your webserver's configuration to +Installers are Plugins used with the ``install`` command to install a cert. +These plugins can modify your webserver's configuration to serve your website over HTTPS using certificates obtained by certbot. -Plugins that do both can be used with the "certbot run" command, which is the default -when no command is specified. The "run" subcommand can also be used to specify +Plugins that do both can be used with the ``certbot run`` command, which is the default +when no command is specified. The ``run`` subcommand can also be used to specify a combination of distinct authenticator and installer plugins. =========== ==== ==== =============================================================== ============================= @@ -78,7 +78,7 @@ the circumstances in which each plugin can be used, and how to use it. Apache ------ -The Apache plugin currently requires OS with augeas version 1.0; currently `it +The Apache plugin currently requires an OS with augeas version 1.0; currently `it supports `_ modern OSes based on Debian, Fedora, SUSE, Gentoo and Darwin. @@ -253,22 +253,6 @@ certificate counts against several rate limits that are intended to prevent abuse of the ACME protocol, as described `here `__. -Certbot also provides a ``renew`` command. This command examines *all* existing -certificates to determine whether or not each is near expiry. For any existing -certificate that is near expiry, ``certbot renew`` will attempt to obtain a -new certificate for the same domains. Unlike ``certonly``, ``renew`` acts on -multiple certificates and always takes into account whether each one is near -expiry. Because of this, ``renew`` is suitable (and designed) for automated use, -to allow your system to automatically renew each certificate when appropriate. -Since ``renew`` will only renew certificates that are near expiry it can be -run as frequently as you want - since it will usually take no action. - -Typically, ``certbot renew`` runs a reduced risk of rate-limit problems -because it renews certificates only when necessary, and because some of -the Let's Encrypt CA's rate limit policies treat the issuance of a new -certificate under these circumstances more generously. More details about -the use of ``certbot renew`` are provided below. - .. _renewal: Renewing certificates @@ -287,7 +271,12 @@ them. The simplest form is simply This will attempt to renew any previously-obtained certificates that expire in less than 30 days. The same plugin and options that were used at the time the certificate was originally issued will be used for the -renewal attempt, unless you specify other plugins or options. +renewal attempt, unless you specify other plugins or options. Unlike ``certonly``, ``renew`` acts on +multiple certificates and always takes into account whether each one is near +expiry. Because of this, ``renew`` is suitable (and designed) for automated use, +to allow your system to automatically renew each certificate when appropriate. +Since ``renew`` will only renew certificates that are near expiry it can be +run as frequently as you want - since it will usually take no action. You can also specify hooks to be run before or after a certificate is renewed. For example, if you have only a single cert and you obtained it using @@ -470,7 +459,7 @@ Example usage for HTTP-01: #!/bin/bash rm -f /var/www/htdocs/.well-known/acme-challenge/$CERTBOT_TOKEN -Example usage for DNS-01 (Cloudflare API v4) (for example purposes only, do not use) +Example usage for DNS-01 (Cloudflare API v4) (for example purposes only, do not use as-is) :: From 50cf1e9d1ad2f81d2b84e23307c0ad77d3aa126f Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Thu, 12 Jan 2017 17:45:15 -0800 Subject: [PATCH 08/88] Remove 'called_once_with' call (#4041) * Remove 'called_once_with' call * Migrate z_util callers to patch_get_utility --- certbot/plugins/selection_test.py | 7 ++++--- certbot/tests/display/ops_test.py | 30 +++++++++++++++++------------- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/certbot/plugins/selection_test.py b/certbot/plugins/selection_test.py index c0494e565..eb4db2081 100644 --- a/certbot/plugins/selection_test.py +++ b/certbot/plugins/selection_test.py @@ -6,6 +6,7 @@ import mock import zope.component from certbot.display import util as display_util +from certbot.tests import util as test_util from certbot import interfaces @@ -126,14 +127,14 @@ class ChoosePluginTest(unittest.TestCase): from certbot.plugins.selection import choose_plugin return choose_plugin(self.plugins, "Question?") - @mock.patch("certbot.plugins.selection.z_util") + @test_util.patch_get_utility("certbot.plugins.selection.z_util") def test_selection(self, mock_util): mock_util().menu.side_effect = [(display_util.OK, 0), (display_util.OK, 1)] self.assertEqual(self.mock_stand, self._call()) self.assertEqual(mock_util().notification.call_count, 1) - @mock.patch("certbot.plugins.selection.z_util") + @test_util.patch_get_utility("certbot.plugins.selection.z_util") def test_more_info(self, mock_util): mock_util().menu.side_effect = [ (display_util.HELP, 0), @@ -144,7 +145,7 @@ class ChoosePluginTest(unittest.TestCase): self.assertEqual(self.mock_stand, self._call()) self.assertEqual(mock_util().notification.call_count, 2) - @mock.patch("certbot.plugins.selection.z_util") + @test_util.patch_get_utility("certbot.plugins.selection.z_util") def test_no_choice(self, mock_util): mock_util().menu.return_value = (display_util.CANCEL, 0) self.assertTrue(self._call() is None) diff --git a/certbot/tests/display/ops_test.py b/certbot/tests/display/ops_test.py index c2f5d302f..f6de33a92 100644 --- a/certbot/tests/display/ops_test.py +++ b/certbot/tests/display/ops_test.py @@ -115,17 +115,17 @@ class ChooseAccountTest(unittest.TestCase): from certbot.display import ops return ops.choose_account(accounts) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_one(self, mock_util): mock_util().menu.return_value = (display_util.OK, 0) self.assertEqual(self._call([self.acc1]), self.acc1) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_two(self, mock_util): mock_util().menu.return_value = (display_util.OK, 1) self.assertEqual(self._call([self.acc1, self.acc2]), self.acc2) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_cancel(self, mock_util): mock_util().menu.return_value = (display_util.CANCEL, 1) self.assertTrue(self._call([self.acc1, self.acc2]) is None) @@ -216,12 +216,12 @@ class ChooseNamesTest(unittest.TestCase): self._call(None) self.assertEqual(mock_manual.call_count, 1) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_no_installer_cancel(self, mock_util): mock_util().input.return_value = (display_util.CANCEL, []) self.assertEqual(self._call(None), []) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_no_names_choose(self, mock_util): self.mock_install().get_all_names.return_value = set() domain = "example.com" @@ -272,7 +272,7 @@ class ChooseNamesTest(unittest.TestCase): self.assertEqual(_sort_names(to_sort), sortd) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_filter_names_valid_return(self, mock_util): self.mock_install.get_all_names.return_value = set(["example.com"]) mock_util().checklist.return_value = (display_util.OK, ["example.com"]) @@ -281,14 +281,14 @@ class ChooseNamesTest(unittest.TestCase): self.assertEqual(names, ["example.com"]) self.assertEqual(mock_util().checklist.call_count, 1) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_filter_names_nothing_selected(self, mock_util): self.mock_install.get_all_names.return_value = set(["example.com"]) mock_util().checklist.return_value = (display_util.OK, []) self.assertEqual(self._call(self.mock_install), []) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_filter_names_cancel(self, mock_util): self.mock_install.get_all_names.return_value = set(["example.com"]) mock_util().checklist.return_value = ( @@ -307,7 +307,7 @@ class ChooseNamesTest(unittest.TestCase): self.assertEqual(get_valid_domains(all_invalid), []) self.assertEqual(len(get_valid_domains(two_valid)), 2) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_choose_manually(self, mock_util): from certbot.display.ops import _choose_names_manually # No retry @@ -350,7 +350,7 @@ class SuccessInstallationTest(unittest.TestCase): from certbot.display.ops import success_installation success_installation(names) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_success_installation(self, mock_util): mock_util().notification.return_value = None names = ["example.com", "abc.com"] @@ -372,7 +372,7 @@ class SuccessRenewalTest(unittest.TestCase): from certbot.display.ops import success_renewal success_renewal(names) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_success_renewal(self, mock_util): mock_util().notification.return_value = None names = ["example.com", "abc.com"] @@ -393,12 +393,16 @@ class SuccessRevocationTest(unittest.TestCase): from certbot.display.ops import success_revocation success_revocation(path) - @mock.patch("certbot.display.ops.z_util") + @test_util.patch_get_utility("certbot.display.ops.z_util") def test_success_revocation(self, mock_util): mock_util().notification.return_value = None path = "/path/to/cert.pem" self._call(path) - mock_util().notification.assert_called_once() + mock_util().notification.assert_called_once_with( + "Congratulations! You have successfully revoked the certificate " + "that was located at {0}{1}{1}".format( + path, + os.linesep), pause=False) self.assertTrue(path in mock_util().notification.call_args[0][0]) if __name__ == "__main__": From aaa732d8f3af9cf56f3008d896fb999a479fb2cd Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Fri, 13 Jan 2017 12:16:08 -0800 Subject: [PATCH 09/88] Fix problems with different test ordering (#4043) * fixes #4030 * Properly restore set_by_cli after using it * mock out post_hook so it isn't stored * fixes #4044 --- certbot/tests/cli_test.py | 24 +++++++++++------------- certbot/tests/hook_test.py | 7 ++----- certbot/tests/main_test.py | 11 ++++++----- 3 files changed, 19 insertions(+), 23 deletions(-) diff --git a/certbot/tests/cli_test.py b/certbot/tests/cli_test.py index 32aada811..b0eb96542 100644 --- a/certbot/tests/cli_test.py +++ b/certbot/tests/cli_test.py @@ -1,12 +1,11 @@ """Tests for certbot.cli.""" import argparse -import functools import unittest import os import tempfile -import six import mock +import six from six.moves import reload_module # pylint: disable=import-error from certbot import cli @@ -14,9 +13,8 @@ from certbot import constants from certbot import errors from certbot.plugins import disco -def reset_set_by_cli(): - '''Reset the state of the `set_by_cli` function''' - cli.set_by_cli.detector = None +PLUGINS = disco.PluginsRegistry.find_all() + class TestReadFile(unittest.TestCase): '''Test cli.read_file''' @@ -43,13 +41,13 @@ class ParseTest(unittest.TestCase): _multiprocess_can_split_ = True - @classmethod - def setUpClass(cls): - cls.plugins = disco.PluginsRegistry.find_all() - cls.parse = functools.partial(cli.prepare_and_parse_args, cls.plugins) - def setUp(self): - reset_set_by_cli() + reload_module(cli) + + @staticmethod + def parse(*args, **kwargs): + """Get result of cli.prepare_and_parse_args.""" + return cli.prepare_and_parse_args(PLUGINS, *args, **kwargs) def _help_output(self, args): "Run a command, and return the ouput string for scrutiny" @@ -88,7 +86,7 @@ class ParseTest(unittest.TestCase): self.assertTrue("{0}" not in out) out = self._help_output(['-h', 'nginx']) - if "nginx" in self.plugins: + if "nginx" in PLUGINS: # may be false while building distributions without plugins self.assertTrue("--nginx-ctl" in out) self.assertTrue("--webroot-path" not in out) @@ -96,7 +94,7 @@ class ParseTest(unittest.TestCase): out = self._help_output(['-h']) self.assertTrue("letsencrypt-auto" not in out) # test cli.cli_command - if "nginx" in self.plugins: + if "nginx" in PLUGINS: self.assertTrue("Use the Nginx plugin" in out) else: self.assertTrue("(the certbot nginx plugin is not" in out) diff --git a/certbot/tests/hook_test.py b/certbot/tests/hook_test.py index 87c86ad5c..0fbb91492 100644 --- a/certbot/tests/hook_test.py +++ b/certbot/tests/hook_test.py @@ -5,16 +5,14 @@ import os import unittest import mock +from six.moves import reload_module # pylint: disable=import-error from certbot import errors from certbot import hooks class HookTest(unittest.TestCase): def setUp(self): - pass - - def tearDown(self): - pass + reload_module(hooks) @mock.patch('certbot.hooks._prog') def test_validate_hooks(self, mock_prog): @@ -47,7 +45,6 @@ class HookTest(unittest.TestCase): return mock_logger.warning def test_pre_hook(self): - hooks.pre_hook.already = set() config = mock.MagicMock(pre_hook="true") self._test_a_hook(config, hooks.pre_hook, 1) self._test_a_hook(config, hooks.pre_hook, 0) diff --git a/certbot/tests/main_test.py b/certbot/tests/main_test.py index 01ec2f061..087a3615d 100644 --- a/certbot/tests/main_test.py +++ b/certbot/tests/main_test.py @@ -12,6 +12,7 @@ import datetime import pytz import six +from six.moves import reload_module # pylint: disable=import-error from acme import jose @@ -434,10 +435,9 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods '--logs-dir', self.logs_dir, '--text'] def tearDown(self): - shutil.rmtree(self.tmp_dir) # Reset globals in cli - # pylint: disable=protected-access - cli._parser = cli.set_by_cli.detector = None + reload_module(cli) + shutil.rmtree(self.tmp_dir) def _call(self, args, stdout=None): "Run the cli with output streams and actual client mocked out" @@ -874,8 +874,9 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods test_util.make_lineage(self, 'sample-renewal.conf') args = ["renew", "--dry-run", "--post-hook=no-such-command", "--disable-hook-validation"] - self._test_renewal_common(True, [], args=args, should_renew=True, - error_expected=False) + with mock.patch("certbot.hooks.post_hook"): + self._test_renewal_common(True, [], args=args, should_renew=True, + error_expected=False) @mock.patch("certbot.cli.set_by_cli") def test_ancient_webroot_renewal_conf(self, mock_set_by_cli): From e0d112f5fb879e14dbd08829d83a838b6c9989c4 Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Fri, 13 Jan 2017 15:23:09 -0800 Subject: [PATCH 10/88] Fix expand certs regression (#4053) * Fix expand certs regression * also pass new domains to renew_hook --- certbot/main.py | 2 +- certbot/renewal.py | 8 +++++--- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/certbot/main.py b/certbot/main.py index ec901c501..b95049284 100644 --- a/certbot/main.py +++ b/certbot/main.py @@ -100,7 +100,7 @@ def _auth_from_available(le_client, config, domains=None, certname=None, lineage try: if action == "renew": logger.info("Renewing an existing certificate") - renewal.renew_cert(config, le_client, lineage) + renewal.renew_cert(config, domains, le_client, lineage) elif action == "newcert": # TREAT AS NEW REQUEST logger.info("Obtaining a new certificate") diff --git a/certbot/renewal.py b/certbot/renewal.py index d65cd4904..bd07cfd07 100644 --- a/certbot/renewal.py +++ b/certbot/renewal.py @@ -263,12 +263,14 @@ def _avoid_invalidating_lineage(config, lineage, original_server): "unless you use the --break-my-certs flag!".format(names)) -def renew_cert(config, le_client, lineage): +def renew_cert(config, domains, le_client, lineage): "Renew a certificate lineage." renewal_params = lineage.configuration["renewalparams"] original_server = renewal_params.get("server", cli.flag_default("server")) _avoid_invalidating_lineage(config, lineage, original_server) - new_certr, new_chain, new_key, _ = le_client.obtain_certificate(lineage.names()) + if not domains: + domains = lineage.names() + new_certr, new_chain, new_key, _ = le_client.obtain_certificate(domains) if config.dry_run: logger.debug("Dry run: skipping updating lineage at %s", os.path.dirname(lineage.cert)) @@ -281,7 +283,7 @@ def renew_cert(config, le_client, lineage): lineage.save_successor(prior_version, new_cert, new_key.pem, new_chain, config) lineage.update_all_links_to(lineage.latest_common_version()) - hooks.renew_hook(config, lineage.names(), lineage.live_dir) + hooks.renew_hook(config, domains, lineage.live_dir) def report(msgs, category): From 7f3109f185704ff0547c39b38b256a50f90527c0 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Fri, 13 Jan 2017 17:15:50 -0800 Subject: [PATCH 11/88] Candidate 0.10.1 to master (#4057) * Release 0.10.1 (cherry picked from commit 0ead1106584f10a4c6b967e89a90f5bc9a207eca) * Bump version to 0.11.0 (cherry picked from commit 5052c64dc9d790b8bd5ef3cf19b9ed3727cfc21f) --- certbot-auto | 26 +++++++++--------- docs/cli-help.txt | 2 +- letsencrypt-auto | 26 +++++++++--------- letsencrypt-auto-source/certbot-auto.asc | 14 +++++----- letsencrypt-auto-source/letsencrypt-auto | 24 ++++++++-------- letsencrypt-auto-source/letsencrypt-auto.sig | Bin 256 -> 256 bytes .../pieces/letsencrypt-auto-requirements.txt | 24 ++++++++-------- 7 files changed, 58 insertions(+), 58 deletions(-) diff --git a/certbot-auto b/certbot-auto index a2ddf76ac..b22916997 100755 --- a/certbot-auto +++ b/certbot-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.10.0" +LE_AUTO_VERSION="0.10.1" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -801,18 +801,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.0 \ - --hash=sha256:df4299a9881d94185a1578ed97334430a90f761ce815edd300860ca47d0538f1 \ - --hash=sha256:ddebdf1fe139c8fedbcf633955ec867496d2f7d2d2e9879d538437a69ab47876 -certbot==0.10.0 \ - --hash=sha256:fb1bfa3d54ce9366758e374f7ed99667ce20484224934d3e8e57839fcf784bc5 \ - --hash=sha256:dd64ed8fb3cc3b053f05e779b934433445918668c49bcdbb2c816062815e1661 -certbot-apache==0.10.0 \ - --hash=sha256:909d59c53507093f838f7336f75d7d78563a35b16afdf6c30f45c9f47bf069da \ - --hash=sha256:6f110dae227dd0fea9572fa12dd60b041e391f5d2028cc2e1fedd2a9a0d2bc88 -certbot-nginx==0.10.0 \ - --hash=sha256:4f33a230d420cbd0431e7b707fb9a1732bfd18d3c6056019591bd7c3a13abe92 \ - --hash=sha256:c12ffd05207b0be3c765b3d3e2927e0b2cc2b7de20654b19d154a0d789e7c1d5 +acme==0.10.1 \ + --hash=sha256:1dd5124078bc44739065409f3be51765608a90994c83460578c2680c582c1026 \ + --hash=sha256:f51c2fb0a31646364abeb7fdd8cfc7c8a4e63b0641b14ab3ce1b2e3a8921a211 +certbot==0.10.1 \ + --hash=sha256:9a56fc76f726beeed2f5a08d690088377cd430907f8a38c50e2aa9a258ee1253 \ + --hash=sha256:e0d699adb3f8ca3e077a4db339de29ebb3f790fbc5f3f02e446e227ed40aa743 +certbot-apache==0.10.1 \ + --hash=sha256:1252fd7e435ba48484b0bd9b72535a9755b03d8f0440f164b9c1c560d96cadb8 \ + --hash=sha256:134f46690da55262125defa58aa74472eb4a1555c9ed83edb3c8667df5a561b5 +certbot-nginx==0.10.1 \ + --hash=sha256:afd15ed9e4f3076056b63916f272b7287084d871cb8136477d16b08f64d514f0 \ + --hash=sha256:da1b7ea4831ead3f9eb526ee11bf1bf197da0fea4defeeb7b1ce24c5d3f45b51 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/docs/cli-help.txt b/docs/cli-help.txt index a2dd61a31..672704ebc 100644 --- a/docs/cli-help.txt +++ b/docs/cli-help.txt @@ -86,7 +86,7 @@ optional arguments: statistics about success rates by OS and plugin. If you wish to hide your server OS version from the Let's Encrypt server, set this to "". (default: - CertbotACMEClient/0.10.0 (Ubuntu 16.04.1 LTS) + CertbotACMEClient/0.10.1 (Ubuntu 16.04.1 LTS) Authenticator/XXX Installer/YYY) automation: diff --git a/letsencrypt-auto b/letsencrypt-auto index a2ddf76ac..b22916997 100755 --- a/letsencrypt-auto +++ b/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.10.0" +LE_AUTO_VERSION="0.10.1" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -801,18 +801,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.0 \ - --hash=sha256:df4299a9881d94185a1578ed97334430a90f761ce815edd300860ca47d0538f1 \ - --hash=sha256:ddebdf1fe139c8fedbcf633955ec867496d2f7d2d2e9879d538437a69ab47876 -certbot==0.10.0 \ - --hash=sha256:fb1bfa3d54ce9366758e374f7ed99667ce20484224934d3e8e57839fcf784bc5 \ - --hash=sha256:dd64ed8fb3cc3b053f05e779b934433445918668c49bcdbb2c816062815e1661 -certbot-apache==0.10.0 \ - --hash=sha256:909d59c53507093f838f7336f75d7d78563a35b16afdf6c30f45c9f47bf069da \ - --hash=sha256:6f110dae227dd0fea9572fa12dd60b041e391f5d2028cc2e1fedd2a9a0d2bc88 -certbot-nginx==0.10.0 \ - --hash=sha256:4f33a230d420cbd0431e7b707fb9a1732bfd18d3c6056019591bd7c3a13abe92 \ - --hash=sha256:c12ffd05207b0be3c765b3d3e2927e0b2cc2b7de20654b19d154a0d789e7c1d5 +acme==0.10.1 \ + --hash=sha256:1dd5124078bc44739065409f3be51765608a90994c83460578c2680c582c1026 \ + --hash=sha256:f51c2fb0a31646364abeb7fdd8cfc7c8a4e63b0641b14ab3ce1b2e3a8921a211 +certbot==0.10.1 \ + --hash=sha256:9a56fc76f726beeed2f5a08d690088377cd430907f8a38c50e2aa9a258ee1253 \ + --hash=sha256:e0d699adb3f8ca3e077a4db339de29ebb3f790fbc5f3f02e446e227ed40aa743 +certbot-apache==0.10.1 \ + --hash=sha256:1252fd7e435ba48484b0bd9b72535a9755b03d8f0440f164b9c1c560d96cadb8 \ + --hash=sha256:134f46690da55262125defa58aa74472eb4a1555c9ed83edb3c8667df5a561b5 +certbot-nginx==0.10.1 \ + --hash=sha256:afd15ed9e4f3076056b63916f272b7287084d871cb8136477d16b08f64d514f0 \ + --hash=sha256:da1b7ea4831ead3f9eb526ee11bf1bf197da0fea4defeeb7b1ce24c5d3f45b51 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/certbot-auto.asc b/letsencrypt-auto-source/certbot-auto.asc index 2dfa27621..4f3794505 100644 --- a/letsencrypt-auto-source/certbot-auto.asc +++ b/letsencrypt-auto-source/certbot-auto.asc @@ -1,11 +1,11 @@ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 -iQEcBAABAgAGBQJYdmhCAAoJEE0XyZXNl3XyuSMH/i6+2GqLh00I+VQRUUHmY/CE -PeUmrkN2N6DEFZK6Y6r7vR1QoY8xYEbmMZNmCYU+YRiO/TO3mLLycd48vbQoyttL -Bi4JalkfkLgfNZNLYvlrDE5K7LaHIiPxQfHN2RIZS4ez6eMREyQXhTPq5HGqQuQH -KkiC9CCKrLvmZXOZA+8ayvoo3U3SI1bZNu7d7c4pEDtkGRMZhNSs8Eejo+knDlny -KmEVrvakkcYTeGwz+SckY9Z7rQGyYoFr2+N3owMT40/g9ZnzkaTS/y+G2z1EnWkN -lapwugl9Pnl6Hog+SBH+osONdg04tIiNayPq11NgWNmMvbG6Lbi4p+RVg+16E1M= -=BXeZ +iQEcBAABAgAGBQJYeWiuAAoJEE0XyZXNl3Xy4DsIALpCrYVDK9g1nQtdNzBJuFq7 +WlMEk3ofMrh+3sTEit1jr9+zJ2hV3POa3RtCRfRZsP0hsiNx7XbuR8t6yM6d4j3S +pAO0L5brrNoJKvKPx5v8AVGKm1EIDH/lWx/hH+HiE7OE3z/w0ppwoIy0/xIqMXt9 +O7Q70qV5Rvzh0K3KwSS9pb82ybV/mQ35uugravSOTMo2hZ82Ifh2ZEkJIsS4si2j +Oc26ruLi6ru4vwHJU2DkHZzl0oncyQZFolMZJmB47vNOCDtC303cR/poeQS9LSmF +vq3Lr0FAunCBoCjuyIGMk2SfmIhtJMS1v5dxOpZppVexedBYkoqU0FoUp/10/rM= +=7Xh1 -----END PGP SIGNATURE----- diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index 409d68322..464781a49 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -801,18 +801,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.0 \ - --hash=sha256:df4299a9881d94185a1578ed97334430a90f761ce815edd300860ca47d0538f1 \ - --hash=sha256:ddebdf1fe139c8fedbcf633955ec867496d2f7d2d2e9879d538437a69ab47876 -certbot==0.10.0 \ - --hash=sha256:fb1bfa3d54ce9366758e374f7ed99667ce20484224934d3e8e57839fcf784bc5 \ - --hash=sha256:dd64ed8fb3cc3b053f05e779b934433445918668c49bcdbb2c816062815e1661 -certbot-apache==0.10.0 \ - --hash=sha256:909d59c53507093f838f7336f75d7d78563a35b16afdf6c30f45c9f47bf069da \ - --hash=sha256:6f110dae227dd0fea9572fa12dd60b041e391f5d2028cc2e1fedd2a9a0d2bc88 -certbot-nginx==0.10.0 \ - --hash=sha256:4f33a230d420cbd0431e7b707fb9a1732bfd18d3c6056019591bd7c3a13abe92 \ - --hash=sha256:c12ffd05207b0be3c765b3d3e2927e0b2cc2b7de20654b19d154a0d789e7c1d5 +acme==0.10.1 \ + --hash=sha256:1dd5124078bc44739065409f3be51765608a90994c83460578c2680c582c1026 \ + --hash=sha256:f51c2fb0a31646364abeb7fdd8cfc7c8a4e63b0641b14ab3ce1b2e3a8921a211 +certbot==0.10.1 \ + --hash=sha256:9a56fc76f726beeed2f5a08d690088377cd430907f8a38c50e2aa9a258ee1253 \ + --hash=sha256:e0d699adb3f8ca3e077a4db339de29ebb3f790fbc5f3f02e446e227ed40aa743 +certbot-apache==0.10.1 \ + --hash=sha256:1252fd7e435ba48484b0bd9b72535a9755b03d8f0440f164b9c1c560d96cadb8 \ + --hash=sha256:134f46690da55262125defa58aa74472eb4a1555c9ed83edb3c8667df5a561b5 +certbot-nginx==0.10.1 \ + --hash=sha256:afd15ed9e4f3076056b63916f272b7287084d871cb8136477d16b08f64d514f0 \ + --hash=sha256:da1b7ea4831ead3f9eb526ee11bf1bf197da0fea4defeeb7b1ce24c5d3f45b51 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/letsencrypt-auto.sig b/letsencrypt-auto-source/letsencrypt-auto.sig index 1657814ae2918ce8eaff699089bfc37952719d2d..b6ae7f614ab4dc2a690d79615714bad351b4d11d 100644 GIT binary patch literal 256 zcmV+b0ssDm(KG>Xw6mz{JB&(j^_BG3!xds`roOPc?uil419D%7I;HWrTdP(2?gGyB zjG#Bt15g4+=GurfU8c;+Cd)41rc+>iVsh)w(Q2F8%;v4;cdep{`XpT`o%4OJ?`;VK zop-OvLXUIwE||rjxJY3^(;55MngGGlfJr5g@;B~6*J`~+O5#W<^kAFVvGBK^iM$mF zuIbNxTp?ieypH;OxYSMtN2&3PurD!$6pw#B6$G^-InE`mQ(KN&xNc{}$2k@L+FWxp z!yloUcag}@F*LZd#(uYev#n14TVxp+u?-`@ GmU%|o+lJu) literal 256 zcmV+b0ssEO!rDvu|K!QtZYXWV%TFVV8gb?=Y6getf$}2|kYb$M>TY5%I*K*2 zkHrc@xHXJ!g;;N&JO! z^7;_i41^#RU3P8@Q+H1QFg_TYuVYh~D#VG$8+3TA%r!Nf~d5_u|cJ-!-Tl)0JeLgmWm!v7R9B@Feh697L+tz>IPwTZ3M%f~epL zR+(>@Y)&;4>Vk4%1FfP;(Q=HW((jgT?+#k#=Vi>HxNB;^G}!a4jBTrKn~;d%r*p{V G#$5MT9fqO+ diff --git a/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt b/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt index 59769df85..dc199bb4a 100644 --- a/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt +++ b/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt @@ -171,15 +171,15 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.0 \ - --hash=sha256:df4299a9881d94185a1578ed97334430a90f761ce815edd300860ca47d0538f1 \ - --hash=sha256:ddebdf1fe139c8fedbcf633955ec867496d2f7d2d2e9879d538437a69ab47876 -certbot==0.10.0 \ - --hash=sha256:fb1bfa3d54ce9366758e374f7ed99667ce20484224934d3e8e57839fcf784bc5 \ - --hash=sha256:dd64ed8fb3cc3b053f05e779b934433445918668c49bcdbb2c816062815e1661 -certbot-apache==0.10.0 \ - --hash=sha256:909d59c53507093f838f7336f75d7d78563a35b16afdf6c30f45c9f47bf069da \ - --hash=sha256:6f110dae227dd0fea9572fa12dd60b041e391f5d2028cc2e1fedd2a9a0d2bc88 -certbot-nginx==0.10.0 \ - --hash=sha256:4f33a230d420cbd0431e7b707fb9a1732bfd18d3c6056019591bd7c3a13abe92 \ - --hash=sha256:c12ffd05207b0be3c765b3d3e2927e0b2cc2b7de20654b19d154a0d789e7c1d5 +acme==0.10.1 \ + --hash=sha256:1dd5124078bc44739065409f3be51765608a90994c83460578c2680c582c1026 \ + --hash=sha256:f51c2fb0a31646364abeb7fdd8cfc7c8a4e63b0641b14ab3ce1b2e3a8921a211 +certbot==0.10.1 \ + --hash=sha256:9a56fc76f726beeed2f5a08d690088377cd430907f8a38c50e2aa9a258ee1253 \ + --hash=sha256:e0d699adb3f8ca3e077a4db339de29ebb3f790fbc5f3f02e446e227ed40aa743 +certbot-apache==0.10.1 \ + --hash=sha256:1252fd7e435ba48484b0bd9b72535a9755b03d8f0440f164b9c1c560d96cadb8 \ + --hash=sha256:134f46690da55262125defa58aa74472eb4a1555c9ed83edb3c8667df5a561b5 +certbot-nginx==0.10.1 \ + --hash=sha256:afd15ed9e4f3076056b63916f272b7287084d871cb8136477d16b08f64d514f0 \ + --hash=sha256:da1b7ea4831ead3f9eb526ee11bf1bf197da0fea4defeeb7b1ce24c5d3f45b51 From 16ed5bdd47ddfa581d24a063bab400af91f65c40 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Tue, 17 Jan 2017 12:13:10 -0800 Subject: [PATCH 12/88] encode to bytes as necessary in Validator.certificate (#4026) --- .../certbot_compatibility_test/validator.py | 11 ++++++++++- certbot-compatibility-test/setup.py | 1 + 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/certbot-compatibility-test/certbot_compatibility_test/validator.py b/certbot-compatibility-test/certbot_compatibility_test/validator.py index 333b47296..62dd466a1 100644 --- a/certbot-compatibility-test/certbot_compatibility_test/validator.py +++ b/certbot-compatibility-test/certbot_compatibility_test/validator.py @@ -4,6 +4,8 @@ import socket import requests import zope.interface +import six + from acme import crypto_util from acme import errors as acme_errors from certbot import interfaces @@ -19,7 +21,14 @@ class Validator(object): def certificate(self, cert, name, alt_host=None, port=443): """Verifies the certificate presented at name is cert""" - host = alt_host if alt_host else socket.gethostbyname(name) + if alt_host is None: + host = socket.gethostbyname(name) + elif isinstance(alt_host, six.binary_type): + host = alt_host + else: + host = alt_host.encode() + name = name if isinstance(name, six.binary_type) else name.encode() + try: presented_cert = crypto_util.probe_sni(name, host, port) except acme_errors.Error as error: diff --git a/certbot-compatibility-test/setup.py b/certbot-compatibility-test/setup.py index fa49d9f3d..e515ede7b 100644 --- a/certbot-compatibility-test/setup.py +++ b/certbot-compatibility-test/setup.py @@ -9,6 +9,7 @@ version = '0.11.0.dev0' install_requires = [ 'certbot', 'certbot-apache', + 'six', 'requests', 'zope.interface', ] From 796220f6f13d48bdc6f7f3e574701adf1f508039 Mon Sep 17 00:00:00 2001 From: Peter Eckersley Date: Tue, 17 Jan 2017 12:53:13 -0800 Subject: [PATCH 13/88] Adopt consistent linting practices for the entire tree (#3843) * Use the certbot pylintrc for the ACME module * Further parallelise lint, and don't run PEP8 checks --- .pylintrc | 3 + acme/.pylintrc | 377 ------------------------------------------------- tox.ini | 4 +- 3 files changed, 4 insertions(+), 380 deletions(-) delete mode 100644 acme/.pylintrc diff --git a/.pylintrc b/.pylintrc index 49d0f29ea..d510fddfd 100644 --- a/.pylintrc +++ b/.pylintrc @@ -1,5 +1,8 @@ [MASTER] +# use as many jobs as there are cores +jobs=0 + # Specify a configuration file. #rcfile= diff --git a/acme/.pylintrc b/acme/.pylintrc deleted file mode 100644 index 06bb2a01f..000000000 --- a/acme/.pylintrc +++ /dev/null @@ -1,377 +0,0 @@ -[MASTER] - -# Specify a configuration file. -#rcfile= - -# Python code to execute, usually for sys.path manipulation such as -# pygtk.require(). -#init-hook= - -# Profiled execution. -profile=no - -# Add files or directories to the blacklist. They should be base names, not -# paths. -ignore=CVS - -# Pickle collected data for later comparisons. -persistent=yes - -# List of plugins (as comma separated values of python modules names) to load, -# usually to register additional checkers. -load-plugins=linter_plugin - -# Use multiple processes to speed up Pylint. -jobs=1 - -# Allow loading of arbitrary C extensions. Extensions are imported into the -# active Python interpreter and may run arbitrary code. -unsafe-load-any-extension=no - -# A comma-separated list of package or module names from where C extensions may -# be loaded. Extensions are loading into the active Python interpreter and may -# run arbitrary code -extension-pkg-whitelist= - - -[MESSAGES CONTROL] - -# Only show warnings with the listed confidence levels. Leave empty to show -# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED -confidence= - -# Enable the message, report, category or checker with the given id(s). You can -# either give multiple identifier separated by comma (,) or put this option -# multiple time. See also the "--disable" option for examples. -#enable= - -# Disable the message, report, category or checker with the given id(s). You -# can either give multiple identifiers separated by comma (,) or put this -# option multiple times (only on the command line, not in the configuration -# file where it should appear only once).You can also use "--disable=all" to -# disable everything first and then reenable specific checks. For example, if -# you want to run only the similarities checker, you can use "--disable=all -# --enable=similarities". If you want to run only the classes checker, but have -# no Warning level messages displayed, use"--disable=all --enable=classes -# --disable=W" -disable=fixme,locally-disabled,abstract-class-not-used -# bstract-class-not-used cannot be disabled locally (at least in -# pylint 1.4.1/2) - - - -[REPORTS] - -# Set the output format. Available formats are text, parseable, colorized, msvs -# (visual studio) and html. You can also give a reporter class, eg -# mypackage.mymodule.MyReporterClass. -output-format=text - -# Put messages in a separate file for each module / package specified on the -# command line instead of printing them on stdout. Reports (if any) will be -# written in a file name "pylint_global.[txt|html]". -files-output=no - -# Tells whether to display a full report or only the messages -reports=yes - -# Python expression which should return a note less than 10 (10 is the highest -# note). You have access to the variables errors warning, statement which -# respectively contain the number of errors / warnings messages and the total -# number of statements analyzed. This is used by the global evaluation report -# (RP0004). -evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10) - -# Add a comment according to your evaluation note. This is used by the global -# evaluation report (RP0004). -comment=no - -# Template used to display messages. This is a python new-style format string -# used to format the message information. See doc for all details -#msg-template= - - -[FORMAT] - -# Maximum number of characters on a single line. -max-line-length=80 - -# Regexp for a line that is allowed to be longer than the limit. -ignore-long-lines=^\s*(# )??$ - -# Allow the body of an if to be on the same line as the test if there is no -# else. -single-line-if-stmt=no - -# List of optional constructs for which whitespace checking is disabled -no-space-check=trailing-comma,dict-separator - -# Maximum number of lines in a module -max-module-lines=1000 - -# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1 -# tab). -indent-string=' ' - -# Number of spaces of indent required inside a hanging or continued line. -indent-after-paren=4 - -# Expected format of line ending, e.g. empty (any line ending), LF or CRLF. -expected-line-ending-format= - - -[MISCELLANEOUS] - -# List of note tags to take in consideration, separated by a comma. -notes=FIXME,XXX,TODO - - -[LOGGING] - -# Logging modules to check that the string format arguments are in logging -# function parameter format -logging-modules=logging,logger - - -[SPELLING] - -# Spelling dictionary name. Available dictionaries: none. To make it working -# install python-enchant package. -spelling-dict= - -# List of comma separated words that should not be checked. -spelling-ignore-words= - -# A path to a file that contains private dictionary; one word per line. -spelling-private-dict-file= - -# Tells whether to store unknown words to indicated private dictionary in -# --spelling-private-dict-file option instead of raising a message. -spelling-store-unknown-words=no - - -[TYPECHECK] - -# Tells whether missing members accessed in mixin class should be ignored. A -# mixin class is detected if its name ends with "mixin" (case insensitive). -ignore-mixin-members=yes - -# List of module names for which member attributes should not be checked -# (useful for modules/projects where namespaces are manipulated during runtime -# and thus existing member attributes cannot be deduced by static analysis -ignored-modules= - -# List of classes names for which member attributes should not be checked -# (useful for classes with attributes dynamically set). -ignored-classes=SQLObject - -# When zope mode is activated, add a predefined set of Zope acquired attributes -# to generated-members. -zope=no - -# List of members which are set dynamically and missed by pylint inference -# system, and so shouldn't trigger E0201 when accessed. Python regular -# expressions are accepted. -generated-members=REQUEST,acl_users,aq_parent - - -[SIMILARITIES] - -# Minimum lines number of a similarity. -min-similarity-lines=4 - -# Ignore comments when computing similarities. -ignore-comments=yes - -# Ignore docstrings when computing similarities. -ignore-docstrings=yes - -# Ignore imports when computing similarities. -ignore-imports=no - - -[VARIABLES] - -# Tells whether we should check for unused import in __init__ files. -init-import=no - -# A regular expression matching the name of dummy variables (i.e. expectedly -# not used). -dummy-variables-rgx=_$|dummy|unused - -# List of additional names supposed to be defined in builtins. Remember that -# you should avoid to define new builtins when possible. -additional-builtins= - -# List of strings which can identify a callback function by name. A callback -# name must start or end with one of those strings. -callbacks=cb_,_cb - - -[BASIC] - -# Required attributes for module, separated by a comma -required-attributes= - -# List of builtins function names that should not be used, separated by a comma -bad-functions=map,filter,input - -# Good variable names which should always be accepted, separated by a comma -good-names=i,j,k,ex,Run,_,logger - -# Bad variable names which should always be refused, separated by a comma -bad-names=foo,bar,baz,toto,tutu,tata - -# Colon-delimited sets of names that determine each other's naming style when -# the name regexes allow several styles. -name-group= - -# Include a hint for the correct naming format with invalid-name -include-naming-hint=no - -# Regular expression matching correct function names -function-rgx=[a-z_][a-z0-9_]{2,40}$ - -# Naming hint for function names -function-name-hint=[a-z_][a-z0-9_]{2,30}$ - -# Regular expression matching correct variable names -variable-rgx=[a-z_][a-z0-9_]{2,30}$ - -# Naming hint for variable names -variable-name-hint=[a-z_][a-z0-9_]{2,30}$ - -# Regular expression matching correct constant names -const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__))$ - -# Naming hint for constant names -const-name-hint=(([A-Z_][A-Z0-9_]*)|(__.*__))$ - -# Regular expression matching correct attribute names -attr-rgx=[a-z_][a-z0-9_]{2,30}$ - -# Naming hint for attribute names -attr-name-hint=[a-z_][a-z0-9_]{2,30}$ - -# Regular expression matching correct argument names -argument-rgx=[a-z_][a-z0-9_]{2,30}$ - -# Naming hint for argument names -argument-name-hint=[a-z_][a-z0-9_]{2,30}$ - -# Regular expression matching correct class attribute names -class-attribute-rgx=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$ - -# Naming hint for class attribute names -class-attribute-name-hint=([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$ - -# Regular expression matching correct inline iteration names -inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$ - -# Naming hint for inline iteration names -inlinevar-name-hint=[A-Za-z_][A-Za-z0-9_]*$ - -# Regular expression matching correct class names -class-rgx=[A-Z_][a-zA-Z0-9]+$ - -# Naming hint for class names -class-name-hint=[A-Z_][a-zA-Z0-9]+$ - -# Regular expression matching correct module names -module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$ - -# Naming hint for module names -module-name-hint=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$ - -# Regular expression matching correct method names -method-rgx=[a-z_][a-z0-9_]{2,49}$ - -# Naming hint for method names -method-name-hint=[a-z_][a-z0-9_]{2,30}$ - -# Regular expression which should only match function or class names that do -# not require a docstring. -no-docstring-rgx=__.*__|test_[A-Za-z0-9_]*|_.*|.*Test - -# Minimum line length for functions/classes that require docstrings, shorter -# ones are exempt. -docstring-min-length=-1 - - -[CLASSES] - -# List of interface methods to ignore, separated by a comma. This is used for -# instance to not check methods defines in Zope's Interface base class. -ignore-iface-methods=isImplementedBy,deferred,extends,names,namesAndDescriptions,queryDescriptionFor,getBases,getDescriptionFor,getDoc,getName,getTaggedValue,getTaggedValueTags,isEqualOrExtendedBy,setTaggedValue,isImplementedByInstancesOf,adaptWith,is_implemented_by - -# List of method names used to declare (i.e. assign) instance attributes. -defining-attr-methods=__init__,__new__,setUp - -# List of valid names for the first argument in a class method. -valid-classmethod-first-arg=cls - -# List of valid names for the first argument in a metaclass class method. -valid-metaclass-classmethod-first-arg=mcs - -# List of member names, which should be excluded from the protected access -# warning. -exclude-protected=_asdict,_fields,_replace,_source,_make - - -[DESIGN] - -# Maximum number of arguments for function / method -max-args=6 - -# Argument names that match this expression will be ignored. Default to name -# with leading underscore -ignored-argument-names=_.* - -# Maximum number of locals for function / method body -max-locals=15 - -# Maximum number of return / yield for function / method body -max-returns=6 - -# Maximum number of branch for function / method body -max-branches=12 - -# Maximum number of statements in function / method body -max-statements=50 - -# Maximum number of parents for a class (see R0901). -max-parents=12 - -# Maximum number of attributes for a class (see R0902). -max-attributes=7 - -# Minimum number of public methods for a class (see R0903). -min-public-methods=2 - -# Maximum number of public methods for a class (see R0904). -max-public-methods=20 - - -[IMPORTS] - -# Deprecated modules which should not be used, separated by a comma -deprecated-modules=regsub,TERMIOS,Bastion,rexec - -# Create a graph of every (i.e. internal and external) dependencies in the -# given file (report RP0402 must not be disabled) -import-graph= - -# Create a graph of external dependencies in the given file (report RP0402 must -# not be disabled) -ext-import-graph= - -# Create a graph of internal dependencies in the given file (report RP0402 must -# not be disabled) -int-import-graph= - - -[EXCEPTIONS] - -# Exceptions that will emit a warning when being caught. Defaults to -# "Exception" -overgeneral-exceptions=Exception diff --git a/tox.ini b/tox.ini index 959f44a8d..c5d832e58 100644 --- a/tox.ini +++ b/tox.ini @@ -82,9 +82,7 @@ basepython = python2.7 # continue, but tox return code will reflect previous error commands = pip install -q -e acme[dns,dev] -e .[dev] -e certbot-apache -e certbot-nginx -e certbot-compatibility-test -e letshelp-certbot - ./pep8.travis.sh - pylint --reports=n --rcfile=acme/.pylintrc acme/acme - pylint -j 0 --reports=n --rcfile=.pylintrc certbot certbot-apache/certbot_apache certbot-nginx/certbot_nginx certbot-compatibility-test/certbot_compatibility_test letshelp-certbot/letshelp_certbot + pylint --reports=n --rcfile=.pylintrc acme/acme certbot certbot-apache/certbot_apache certbot-nginx/certbot_nginx certbot-compatibility-test/certbot_compatibility_test letshelp-certbot/letshelp_certbot [testenv:apacheconftest] #basepython = python2.7 From 2797a0377d0382b2c8aad0f611537c3f89f06c64 Mon Sep 17 00:00:00 2001 From: Juho Juopperi Date: Tue, 17 Jan 2017 23:43:43 +0200 Subject: [PATCH 14/88] Fix misspelling "recieved" (#4059) --- certbot-apache/certbot_apache/configurator.py | 2 +- certbot/error_handler.py | 4 ++-- certbot/errors.py | 2 +- docs/ciphers.rst | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/certbot-apache/certbot_apache/configurator.py b/certbot-apache/certbot_apache/configurator.py index 27e214362..523f690b4 100644 --- a/certbot-apache/certbot_apache/configurator.py +++ b/certbot-apache/certbot_apache/configurator.py @@ -1807,7 +1807,7 @@ def get_file_path(vhost_path): else: return None except AttributeError: - # If we recieved a None path + # If we received a None path return None last_good = "" diff --git a/certbot/error_handler.py b/certbot/error_handler.py index 819d32405..b188b276a 100644 --- a/certbot/error_handler.py +++ b/certbot/error_handler.py @@ -118,9 +118,9 @@ class ErrorHandler(object): self.prev_handlers.clear() def _signal_handler(self, signum, unused_frame): - """Replacement function for handling recieved signals. + """Replacement function for handling received signals. - Store the recieved signal. If we are executing the code block in + Store the received signal. If we are executing the code block in the body of the context manager, stop by raising signal exit. :param int signum: number of current signal diff --git a/certbot/errors.py b/certbot/errors.py index 738b7536b..6d191404c 100644 --- a/certbot/errors.py +++ b/certbot/errors.py @@ -30,7 +30,7 @@ class HookCommandNotFound(Error): class SignalExit(Error): - """A Unix signal was recieved while in the ErrorHandler context manager.""" + """A Unix signal was received while in the ErrorHandler context manager.""" # Auth Handler Errors diff --git a/docs/ciphers.rst b/docs/ciphers.rst index c429c185d..cf9ffdb99 100644 --- a/docs/ciphers.rst +++ b/docs/ciphers.rst @@ -206,7 +206,7 @@ or a family of enhancements, one per selectable ciphersuite configuration. Feedback ======== -We recieve lots of feedback on the type of ciphersuites that Let's Encrypt supports and list some coallated feedback below. This section aims to track suggestions and references that people have offered or identified to improve the ciphersuites that Let's Encrypt enables when configuring TLS on servers. +We receive lots of feedback on the type of ciphersuites that Let's Encrypt supports and list some coallated feedback below. This section aims to track suggestions and references that people have offered or identified to improve the ciphersuites that Let's Encrypt enables when configuring TLS on servers. Because of the Chatham House Rule applicable to some of the discussions, people are *not* individually credited for their suggestions, but most suggestions here were made or found by other people, and I thank them for their contributions. From a5101d73a0a6e6fb12215d7efed3a4db666260df Mon Sep 17 00:00:00 2001 From: Guillaume Boudreau Date: Tue, 17 Jan 2017 17:54:14 -0500 Subject: [PATCH 15/88] certbot-auto re-installs virtualenv when plugin is causing issues (#4035) When certbot-auto cannot find the currently installed version, output the error to the end-user, instead of not showing anything, and re-installing the virtualenv. Fixes #4034 --- letsencrypt-auto-source/letsencrypt-auto | 5 +++++ letsencrypt-auto-source/letsencrypt-auto.template | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index 464781a49..7210f6a3d 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -609,6 +609,11 @@ if [ "$1" = "--le-auto-phase2" ]; then # --version output ran through grep due to python-cryptography DeprecationWarnings # grep for both certbot and letsencrypt until certbot and shim packages have been released INSTALLED_VERSION=$("$VENV_BIN/letsencrypt" --version 2>&1 | grep "^certbot\|^letsencrypt" | cut -d " " -f 2) + if [ -z "$INSTALLED_VERSION" ]; then + echo "Error: couldn't get currently installed version for $VENV_BIN/letsencrypt: " 1>&2 + "$VENV_BIN/letsencrypt" --version + exit 1 + fi else INSTALLED_VERSION="none" fi diff --git a/letsencrypt-auto-source/letsencrypt-auto.template b/letsencrypt-auto-source/letsencrypt-auto.template index b602540a0..a3ab7eaa1 100755 --- a/letsencrypt-auto-source/letsencrypt-auto.template +++ b/letsencrypt-auto-source/letsencrypt-auto.template @@ -260,6 +260,11 @@ if [ "$1" = "--le-auto-phase2" ]; then # --version output ran through grep due to python-cryptography DeprecationWarnings # grep for both certbot and letsencrypt until certbot and shim packages have been released INSTALLED_VERSION=$("$VENV_BIN/letsencrypt" --version 2>&1 | grep "^certbot\|^letsencrypt" | cut -d " " -f 2) + if [ -z "$INSTALLED_VERSION" ]; then + echo "Error: couldn't get currently installed version for $VENV_BIN/letsencrypt: " 1>&2 + "$VENV_BIN/letsencrypt" --version + exit 1 + fi else INSTALLED_VERSION="none" fi From 0fa307806e1c0220314dcbc378895c49bcb5d64d Mon Sep 17 00:00:00 2001 From: yomna Date: Tue, 17 Jan 2017 15:19:33 -0800 Subject: [PATCH 16/88] Alternate help syntax - issue 3371 (#4068) * [#3371] support for new help syntax + tests * [#3371] splitting up test to satisfy linter --- certbot/cli.py | 4 ++++ certbot/tests/cli_test.py | 20 ++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/certbot/cli.py b/certbot/cli.py index d51fd58e0..4aeac6d34 100644 --- a/certbot/cli.py +++ b/certbot/cli.py @@ -432,6 +432,10 @@ class HelpfulArgumentParser(object): self.detect_defaults = detect_defaults self.args = args + + if self.args[0] == 'help': + self.args[0] = '--help' + self.determine_verb() help1 = self.prescan_for_flag("-h", self.help_topics) help2 = self.prescan_for_flag("--help", self.help_topics) diff --git a/certbot/tests/cli_test.py b/certbot/tests/cli_test.py index b0eb96542..1fa2004d3 100644 --- a/certbot/tests/cli_test.py +++ b/certbot/tests/cli_test.py @@ -131,6 +131,26 @@ class ParseTest(unittest.TestCase): self.assertTrue("%s" not in out) self.assertTrue("{0}" not in out) + def test_help_no_dashes(self): + self._help_output(['help']) # assert SystemExit is raised here + + out = self._help_output(['help', 'all']) + self.assertTrue("--configurator" in out) + self.assertTrue("how a cert is deployed" in out) + self.assertTrue("--webroot-path" in out) + self.assertTrue("--text" not in out) + self.assertTrue("--dialog" not in out) + self.assertTrue("%s" not in out) + self.assertTrue("{0}" not in out) + + out = self._help_output(['help', 'install']) + self.assertTrue("--cert-path" in out) + self.assertTrue("--key-path" in out) + + out = self._help_output(['help', 'revoke']) + self.assertTrue("--cert-path" in out) + self.assertTrue("--key-path" in out) + def test_parse_domains(self): short_args = ['-d', 'example.com'] namespace = self.parse(short_args) From 49d46ef99a69bd4a68771074c4f4e8facc4f148e Mon Sep 17 00:00:00 2001 From: Blake Griffith Date: Tue, 17 Jan 2017 16:00:07 -0800 Subject: [PATCH 17/88] Impelment account deactivation [revision requested] (#3571) Impelment account deactivation --- .gitignore | 2 ++ acme/acme/client.py | 16 +++++++-- acme/acme/client_test.py | 14 ++++++++ acme/acme/messages.py | 1 + certbot/account.py | 13 ++++++++ certbot/cli.py | 11 ++++-- certbot/client.py | 2 +- certbot/main.py | 35 +++++++++++++++++-- certbot/tests/account_test.py | 8 +++++ certbot/tests/main_test.py | 63 +++++++++++++++++++++++++++++++++++ tests/boulder-integration.sh | 8 +++-- 11 files changed, 163 insertions(+), 10 deletions(-) diff --git a/.gitignore b/.gitignore index ac2842e27..b63e40d1c 100644 --- a/.gitignore +++ b/.gitignore @@ -33,3 +33,5 @@ tags tests/letstest/letest-*/ tests/letstest/*.pem tests/letstest/venv/ + +.venv diff --git a/acme/acme/client.py b/acme/acme/client.py index 26109352b..19f566b0d 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -132,12 +132,24 @@ class Client(object): # pylint: disable=too-many-instance-attributes """ update = regr.body if update is None else update - updated_regr = self._send_recv_regr( - regr, body=messages.UpdateRegistration(**dict(update))) + body = messages.UpdateRegistration(**dict(update)) + updated_regr = self._send_recv_regr(regr, body=body) if updated_regr != regr: raise errors.UnexpectedUpdate(regr) return updated_regr + def deactivate_registration(self, regr): + """Deactivate registration. + + :param messages.RegistrationResource regr: The Registration Resource + to be deactivated. + + :returns: The Registration resource that was deactivated. + :rtype: `.RegistrationResource` + + """ + return self.update_registration(regr, update={'status': 'deactivated'}) + def query_registration(self, regr): """Query server about registration. diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index 4822a1ae6..62555939c 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -124,6 +124,20 @@ class ClientTest(unittest.TestCase): self.assertRaises( errors.UnexpectedUpdate, self.client.update_registration, self.regr) + def test_deactivate_account(self): + self.response.headers['Location'] = self.regr.uri + self.response.json.return_value = self.regr.body.to_json() + self.assertEqual(self.regr, + self.client.deactivate_registration(self.regr)) + + def test_deactivate_account_bad_registration_returned(self): + self.response.headers['Location'] = self.regr.uri + self.response.json.return_value = "some wrong registration thing" + self.assertRaises( + errors.UnexpectedUpdate, + self.client.deactivate_registration, + self.regr) + def test_query_registration(self): self.response.json.return_value = self.regr.body.to_json() self.assertEqual(self.regr, self.client.query_registration(self.regr)) diff --git a/acme/acme/messages.py b/acme/acme/messages.py index 29d719684..54cd25c94 100644 --- a/acme/acme/messages.py +++ b/acme/acme/messages.py @@ -250,6 +250,7 @@ class Registration(ResourceBody): agreement = jose.Field('agreement', omitempty=True) authorizations = jose.Field('authorizations', omitempty=True) certificates = jose.Field('certificates', omitempty=True) + status = jose.Field('status', omitempty=True) class Authorizations(jose.JSONObjectWithFields): """Authorizations granted to Account in the process of registration. diff --git a/certbot/account.py b/certbot/account.py index 71951d8e0..877b87015 100644 --- a/certbot/account.py +++ b/certbot/account.py @@ -3,6 +3,7 @@ import datetime import hashlib import logging import os +import shutil import socket from cryptography.hazmat.primitives import serialization @@ -197,6 +198,18 @@ class AccountFileStorage(interfaces.AccountStorage): """ self._save(account, regr_only=True) + def delete(self, account_id): + """Delete registration info from disk + + :param account_id: id of account which should be deleted + + """ + account_dir_path = self._account_dir_path(account_id) + if not os.path.isdir(account_dir_path): + raise errors.AccountNotFound( + "Account at %s does not exist" % account_dir_path) + shutil.rmtree(account_dir_path) + def _save(self, account, regr_only): account_dir_path = self._account_dir_path(account.id) util.make_or_verify_dir(account_dir_path, 0o700, os.geteuid(), diff --git a/certbot/cli.py b/certbot/cli.py index 4aeac6d34..7d146e33b 100644 --- a/certbot/cli.py +++ b/certbot/cli.py @@ -366,6 +366,10 @@ VERB_HELP = [ "short": "Register for account with Let's Encrypt / other ACME server", "opts": "Options for account registration & modification" }), + ("unregister", { + "short": "Irrevocably deactivate your account", + "opts": "Options for account deactivation." + }), ("install", { "short": "Install an arbitrary cert in a server", "opts": "Options for modifying how a cert is deployed" @@ -414,6 +418,7 @@ class HelpfulArgumentParser(object): "install": main.install, "plugins": main.plugins_cmd, "register": main.register, + "unregister": main.unregister, "renew": main.renew, "revoke": main.revoke, "rollback": main.rollback, @@ -871,7 +876,9 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis help="With the register verb, indicates that details associated " "with an existing registration, such as the e-mail address, " "should be updated, rather than registering a new account.") - helpful.add(["register", "automation"], "-m", "--email", help=config_help("email")) + helpful.add( + ["register", "unregister", "automation"], "-m", "--email", + help=config_help("email")) helpful.add( ["automation", "certonly", "run"], "--keep-until-expiring", "--keep", "--reinstall", @@ -913,7 +920,7 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis "automation", "--agree-tos", dest="tos", action="store_true", help="Agree to the ACME Subscriber Agreement (default: Ask)") helpful.add( - "automation", "--account", metavar="ACCOUNT_ID", + ["unregister", "automation"], "--account", metavar="ACCOUNT_ID", help="Account ID to use") helpful.add( "automation", "--duplicate", dest="duplicate", action="store_true", diff --git a/certbot/client.py b/certbot/client.py index b40a169e6..f76688b70 100644 --- a/certbot/client.py +++ b/certbot/client.py @@ -172,7 +172,7 @@ def perform_registration(acme, config): class Client(object): - """ACME protocol client. + """Certbot's client. :ivar .IConfig config: Client configuration. :ivar .Account account: Account registered with `register`. diff --git a/certbot/main.py b/certbot/main.py index b95049284..f8cb411b5 100644 --- a/certbot/main.py +++ b/certbot/main.py @@ -406,6 +406,35 @@ def _init_le_client(config, authenticator, installer): return client.Client(config, acc, authenticator, installer, acme=acme) +def unregister(config, unused_plugins): + """Deactivate account on server""" + account_storage = account.AccountFileStorage(config) + accounts = account_storage.find_all() + reporter_util = zope.component.getUtility(interfaces.IReporter) + + if not accounts: + return "Could not find existing account to deactivate." + yesno = zope.component.getUtility(interfaces.IDisplay).yesno + prompt = ("Are you sure you would like to irrevocably deactivate " + "your account?") + wants_deactivate = yesno(prompt, yes_label='Deactivate', no_label='Abort', + default=True) + + if not wants_deactivate: + return "Deactivation aborted." + + acc, acme = _determine_account(config) + acme_client = client.Client(config, acc, None, None, acme=acme) + + # delete on boulder + acme_client.acme.deactivate_registration(acc.regr) + account_files = account.AccountFileStorage(config) + # delete local account files + account_files.delete(config.account) + + reporter_util.add_message("Account deactivated.", reporter_util.MEDIUM_PRIORITY) + + def register(config, unused_plugins): """Create or modify accounts on the server.""" @@ -413,6 +442,8 @@ def register(config, unused_plugins): # exist or not. account_storage = account.AccountFileStorage(config) accounts = account_storage.find_all() + reporter_util = zope.component.getUtility(interfaces.IReporter) + add_msg = lambda m: reporter_util.add_message(m, reporter_util.MEDIUM_PRIORITY) # registering a new account if not config.update_registration: @@ -443,9 +474,7 @@ def register(config, unused_plugins): acc.regr = acme_client.acme.update_registration(acc.regr.update( body=acc.regr.body.update(contact=('mailto:' + config.email,)))) account_storage.save_regr(acc) - reporter_util = zope.component.getUtility(interfaces.IReporter) - msg = "Your e-mail address was updated to {0}.".format(config.email) - reporter_util.add_message(msg, reporter_util.MEDIUM_PRIORITY) + add_msg("Your e-mail address was updated to {0}.".format(config.email)) def install(config, plugins): diff --git a/certbot/tests/account_test.py b/certbot/tests/account_test.py index 1c50025d7..d045afe23 100644 --- a/certbot/tests/account_test.py +++ b/certbot/tests/account_test.py @@ -190,6 +190,14 @@ class AccountFileStorageTest(unittest.TestCase): self.assertRaises( errors.AccountStorageError, self.storage.save, self.acc) + def test_delete(self): + self.storage.save(self.acc) + self.storage.delete(self.acc.id) + self.assertRaises(errors.AccountNotFound, self.storage.load, self.acc.id) + + def test_delete_no_account(self): + self.assertRaises(errors.AccountNotFound, self.storage.delete, self.acc.id) + if __name__ == "__main__": unittest.main() # pragma: no cover diff --git a/certbot/tests/main_test.py b/certbot/tests/main_test.py index 087a3615d..3348a7574 100644 --- a/certbot/tests/main_test.py +++ b/certbot/tests/main_test.py @@ -1,4 +1,5 @@ """Tests for certbot.main.""" +# pylint: disable=too-many-lines from __future__ import print_function import itertools @@ -1164,6 +1165,68 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods email in mock_utility().add_message.call_args[0][0]) +class UnregisterTest(unittest.TestCase): + def setUp(self): + self.patchers = { + '_determine_account': mock.patch('certbot.main._determine_account'), + 'account': mock.patch('certbot.main.account'), + 'client': mock.patch('certbot.main.client'), + 'get_utility': test_util.patch_get_utility()} + self.mocks = dict((k, v.start()) for k, v in self.patchers.items()) + + def tearDown(self): + for patch in self.patchers.values(): + patch.stop() + + def test_abort_unregister(self): + self.mocks['account'].AccountFileStorage.return_value = mock.Mock() + + util_mock = self.mocks['get_utility'].return_value + util_mock.yesno.return_value = False + + config = mock.Mock() + unused_plugins = mock.Mock() + + res = main.unregister(config, unused_plugins) + self.assertEqual(res, "Deactivation aborted.") + + def test_unregister(self): + mocked_storage = mock.MagicMock() + mocked_storage.find_all.return_value = ["an account"] + + self.mocks['account'].AccountFileStorage.return_value = mocked_storage + self.mocks['_determine_account'].return_value = (mock.MagicMock(), "foo") + + acme_client = mock.MagicMock() + self.mocks['client'].Client.return_value = acme_client + + config = mock.MagicMock() + unused_plugins = mock.MagicMock() + + res = main.unregister(config, unused_plugins) + + self.assertTrue(res is None) + self.assertTrue(acme_client.acme.deactivate_registration.called) + m = "Account deactivated." + self.assertTrue(m in self.mocks['get_utility']().add_message.call_args[0][0]) + + def test_unregister_no_account(self): + mocked_storage = mock.MagicMock() + mocked_storage.find_all.return_value = [] + self.mocks['account'].AccountFileStorage.return_value = mocked_storage + + acme_client = mock.MagicMock() + self.mocks['client'].Client.return_value = acme_client + + config = mock.MagicMock() + unused_plugins = mock.MagicMock() + + res = main.unregister(config, unused_plugins) + m = "Could not find existing account to deactivate." + self.assertEqual(res, m) + self.assertFalse(acme_client.acme.deactivate_registration.called) + + class TestHandleException(unittest.TestCase): """Test main._handle_exception""" @mock.patch('certbot.main.sys') diff --git a/tests/boulder-integration.sh b/tests/boulder-integration.sh index 30fc17f81..790e5a263 100755 --- a/tests/boulder-integration.sh +++ b/tests/boulder-integration.sh @@ -9,7 +9,7 @@ # Note: this script is called by Boulder integration test suite! . ./tests/integration/_common.sh -export PATH="/usr/sbin:$PATH" # /usr/sbin/nginx +export PATH="$PATH:/usr/sbin" # /usr/sbin/nginx export GOPATH="${GOPATH:-/tmp/go}" export PATH="$GOPATH/bin:$PATH" @@ -161,7 +161,8 @@ common revoke --cert-path "$root/conf/live/le.wtf/cert.pem" common revoke --cert-path "$root/conf/live/le1.wtf/cert.pem" # revoke by cert key common revoke --cert-path "$root/conf/live/le2.wtf/cert.pem" \ - --key-path "$root/conf/live/le2.wtf/privkey.pem" + --key-path "$root/conf/live/le2.wtf/privkey.pem" + # Get new certs to test revoke with a reason, by account and by cert key common --domains le1.wtf common revoke --cert-path "$root/conf/live/le1.wtf/cert.pem" \ @@ -170,6 +171,9 @@ common --domains le2.wtf common revoke --cert-path "$root/conf/live/le2.wtf/cert.pem" \ --key-path "$root/conf/live/le2.wtf/privkey.pem" \ --reason keyCompromise + +common unregister + if type nginx; then . ./certbot-nginx/tests/boulder-integration.sh From ea951150a45e0058544f9d2ea467117f4105bf88 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 18 Jan 2017 10:41:11 -0800 Subject: [PATCH 18/88] Enable Py36 Tests (#3972) * add py36 to tox * Add Python 3.6 tests to Travis * Provide real path to python stdlib during tests * set logs_dir in config_test * set *_dirs in DetermineAccountTest * Fix TLSSNI01Test * Fix RenewalTest * fix test_ancient_webroot_renewal_conf --- .travis.yml | 2 ++ certbot/plugins/common_test.py | 14 +++++++++++++- certbot/tests/configuration_test.py | 3 ++- certbot/tests/main_test.py | 7 ++++++- certbot/tests/renewal_test.py | 8 +++++++- tox.ini | 9 ++++++++- 6 files changed, 38 insertions(+), 5 deletions(-) diff --git a/.travis.yml b/.travis.yml index 3a9a994a9..b35f0ebbf 100644 --- a/.travis.yml +++ b/.travis.yml @@ -87,6 +87,8 @@ matrix: env: TOXENV=py34 - python: "3.5" env: TOXENV=py35 + - python: "3.6" + env: TOXENV=py36 - python: "2.7" env: TOXENV=nginxroundtrip diff --git a/certbot/plugins/common_test.py b/certbot/plugins/common_test.py index eee768e18..8154b255a 100644 --- a/certbot/plugins/common_test.py +++ b/certbot/plugins/common_test.py @@ -1,4 +1,7 @@ """Tests for certbot.plugins.common.""" +import os +import shutil +import tempfile import unittest import mock @@ -170,8 +173,16 @@ class TLSSNI01Test(unittest.TestCase): ] def setUp(self): + self.tempdir = tempfile.mkdtemp() + configurator = mock.MagicMock() + configurator.config.config_dir = os.path.join(self.tempdir, "config") + configurator.config.work_dir = os.path.join(self.tempdir, "work") + from certbot.plugins.common import TLSSNI01 - self.sni = TLSSNI01(configurator=mock.MagicMock()) + self.sni = TLSSNI01(configurator=configurator) + + def tearDown(self): + shutil.rmtree(self.tempdir) def test_add_chall(self): self.sni.add_chall(self.achalls[0], 0) @@ -187,6 +198,7 @@ class TLSSNI01Test(unittest.TestCase): response = challenges.TLSSNI01Response() achall = mock.MagicMock() + achall.chall.encode.return_value = "token" key = test_util.load_pyopenssl_private_key("rsa512_key.pem") achall.response_and_validation.return_value = ( response, (test_util.load_cert("cert.pem"), key)) diff --git a/certbot/tests/configuration_test.py b/certbot/tests/configuration_test.py index 183d6a95c..3a2f7d291 100644 --- a/certbot/tests/configuration_test.py +++ b/certbot/tests/configuration_test.py @@ -12,7 +12,8 @@ class NamespaceConfigTest(unittest.TestCase): def setUp(self): self.namespace = mock.MagicMock( - config_dir='/tmp/config', work_dir='/tmp/foo', foo='bar', + config_dir='/tmp/config', work_dir='/tmp/foo', + logs_dir="/tmp/bar", foo='bar', server='https://acme-server.org:443/new', tls_sni_01_port=1234, http01_port=4321) from certbot.configuration import NamespaceConfig diff --git a/certbot/tests/main_test.py b/certbot/tests/main_test.py index 3348a7574..a4f3a7805 100644 --- a/certbot/tests/main_test.py +++ b/certbot/tests/main_test.py @@ -363,6 +363,9 @@ class DetermineAccountTest(unittest.TestCase): def setUp(self): self.args = mock.MagicMock(account=None, email=None, + config_dir="unused_config", + logs_dir="unused_logs", + work_dir="unused_work", register_unsafely_without_email=False) self.config = configuration.NamespaceConfig(self.args) self.accs = [mock.MagicMock(id='x'), mock.MagicMock(id='y')] @@ -883,7 +886,9 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods def test_ancient_webroot_renewal_conf(self, mock_set_by_cli): mock_set_by_cli.return_value = False rc_path = test_util.make_lineage(self, 'sample-renewal-ancient.conf') - args = mock.MagicMock(account=None, email=None, webroot_path=None) + args = mock.MagicMock(account=None, config_dir=self.config_dir, + logs_dir=self.logs_dir, work_dir=self.work_dir, + email=None, webroot_path=None) config = configuration.NamespaceConfig(args) lineage = storage.RenewableCert(rc_path, config) renewalparams = lineage.configuration["renewalparams"] diff --git a/certbot/tests/renewal_test.py b/certbot/tests/renewal_test.py index 07c4eac00..2fe4eab68 100644 --- a/certbot/tests/renewal_test.py +++ b/certbot/tests/renewal_test.py @@ -2,6 +2,7 @@ import os import mock import unittest +import shutil import tempfile from certbot import configuration @@ -16,11 +17,16 @@ class RenewalTest(unittest.TestCase): self.tmp_dir = tempfile.mkdtemp() self.config_dir = os.path.join(self.tmp_dir, 'config') + def tearDown(self): + shutil.rmtree(self.tmp_dir) + @mock.patch('certbot.cli.set_by_cli') def test_ancient_webroot_renewal_conf(self, mock_set_by_cli): mock_set_by_cli.return_value = False rc_path = util.make_lineage(self, 'sample-renewal-ancient.conf') - args = mock.MagicMock(account=None, email=None, webroot_path=None) + args = mock.MagicMock(account=None, config_dir=self.config_dir, + logs_dir="logs", work_dir="work", + email=None, webroot_path=None) config = configuration.NamespaceConfig(args) lineage = storage.RenewableCert(rc_path, config) renewalparams = lineage.configuration['renewalparams'] diff --git a/tox.ini b/tox.ini index c5d832e58..6f6903948 100644 --- a/tox.ini +++ b/tox.ini @@ -4,7 +4,7 @@ [tox] skipsdist = true -envlist = modification,py{26,33,34,35},cover,lint +envlist = modification,py{26,33,34,35,36},cover,lint # nosetest -v => more verbose output, allows to detect busy waiting # loops, especially on Travis @@ -63,6 +63,13 @@ commands = pip install -e .[dev] nosetests -v certbot --processes=-1 --process-timeout=100 +[testenv:py36] +commands = + pip install -e acme[dns,dev] + nosetests -v acme --processes=-1 + pip install -e .[dev] + nosetests -v certbot --processes=-1 --process-timeout=100 + [testenv:py27_install] basepython = python2.7 commands = From fe358600abc1bf3d109f6610011f3f3a84a9ef55 Mon Sep 17 00:00:00 2001 From: Craig Smith Date: Thu, 19 Jan 2017 05:11:53 +1030 Subject: [PATCH 19/88] Add cleanup trap to integration test script (#4075) The integration test script spins up Python webservers. This trap will ensure that those webservers are shutdown at exit. --- tests/boulder-integration.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/tests/boulder-integration.sh b/tests/boulder-integration.sh index 790e5a263..9c6bc7708 100755 --- a/tests/boulder-integration.sh +++ b/tests/boulder-integration.sh @@ -20,6 +20,18 @@ else readlink="readlink" fi +cleanup_and_exit() { + EXIT_STATUS=$? + if SERVER_STILL_RUNNING=`ps -p $python_server_pid -o pid=` + then + echo Kill server subprocess, left running by abnormal exit + kill $SERVER_STILL_RUNNING + fi + exit $EXIT_STATUS +} + +trap cleanup_and_exit EXIT + common_no_force_renew() { certbot_test_no_force_renew \ --authenticator standalone \ From bb669528b3bd38473dcbb92d15f96b702a1605cc Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Thu, 19 Jan 2017 11:21:25 -0800 Subject: [PATCH 20/88] Remove old references to pep8 in our code (#4073) --- .pep8 | 4 ---- Dockerfile-dev | 2 +- acme/.pep8 | 4 ---- acme/setup.py | 1 - certbot-compatibility-test/Dockerfile | 2 +- docs/contributing.rst | 12 +++++------- pep8.travis.sh | 5 ----- setup.py | 1 - 8 files changed, 7 insertions(+), 24 deletions(-) delete mode 100644 .pep8 delete mode 100644 acme/.pep8 delete mode 100755 pep8.travis.sh diff --git a/.pep8 b/.pep8 deleted file mode 100644 index 22045d3d3..000000000 --- a/.pep8 +++ /dev/null @@ -1,4 +0,0 @@ -[pep8] -# E265 block comment should start with '# ' -# E501 line too long (X > 79 characters) -ignore = E265,E501 diff --git a/Dockerfile-dev b/Dockerfile-dev index c7e1d7b2e..098fae523 100644 --- a/Dockerfile-dev +++ b/Dockerfile-dev @@ -32,7 +32,7 @@ RUN /opt/certbot/src/letsencrypt-auto-source/letsencrypt-auto --os-packages-only # the above is not likely to change, so by putting it further up the # Dockerfile we make sure we cache as much as possible -COPY setup.py README.rst CHANGES.rst MANIFEST.in linter_plugin.py tox.cover.sh tox.ini pep8.travis.sh .pep8 .pylintrc /opt/certbot/src/ +COPY setup.py README.rst CHANGES.rst MANIFEST.in linter_plugin.py tox.cover.sh tox.ini .pylintrc /opt/certbot/src/ # all above files are necessary for setup.py, however, package source # code directory has to be copied separately to a subdirectory... diff --git a/acme/.pep8 b/acme/.pep8 deleted file mode 100644 index 22045d3d3..000000000 --- a/acme/.pep8 +++ /dev/null @@ -1,4 +0,0 @@ -[pep8] -# E265 block comment should start with '# ' -# E501 line too long (X > 79 characters) -ignore = E265,E501 diff --git a/acme/setup.py b/acme/setup.py index f325b47a2..37b71368c 100644 --- a/acme/setup.py +++ b/acme/setup.py @@ -40,7 +40,6 @@ dns_extras = [ dev_extras = [ 'nose', - 'pep8', 'tox', ] diff --git a/certbot-compatibility-test/Dockerfile b/certbot-compatibility-test/Dockerfile index c8ef62696..bb9359ce8 100644 --- a/certbot-compatibility-test/Dockerfile +++ b/certbot-compatibility-test/Dockerfile @@ -14,7 +14,7 @@ RUN /opt/certbot/src/certbot-auto -n --os-packages-only # the above is not likely to change, so by putting it further up the # Dockerfile we make sure we cache as much as possible -COPY setup.py README.rst CHANGES.rst MANIFEST.in linter_plugin.py tox.cover.sh tox.ini pep8.travis.sh .pep8 .pylintrc /opt/certbot/src/ +COPY setup.py README.rst CHANGES.rst MANIFEST.in linter_plugin.py tox.cover.sh tox.ini .pylintrc /opt/certbot/src/ # all above files are necessary for setup.py, however, package source # code directory has to be copied separately to a subdirectory... diff --git a/docs/contributing.rst b/docs/contributing.rst index f2129ed28..fd7caa6af 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -312,17 +312,15 @@ Steps: 2. Make sure your environment is set up properly and that you're in your virtualenv. You can do this by running ``./tools/venv.sh``. (this is a **very important** step) -3. Run ``./pep8.travis.sh`` to do a cursory check of your code style. - Fix any errors. -4. Run ``tox -e lint`` to check for pylint errors. Fix any errors. -5. Run ``tox --skip-missing-interpreters`` to run the entire test suite +3. Run ``tox -e lint`` to check for pylint errors. Fix any errors. +4. Run ``tox --skip-missing-interpreters`` to run the entire test suite including coverage. The ``--skip-missing-interpreters`` argument ignores missing versions of Python needed for running the tests. Fix any errors. -6. If your code touches communication with an ACME server/Boulder, you +5. If your code touches communication with an ACME server/Boulder, you should run the integration tests, see `integration`_. See `Known Issues`_ for some common failures that have nothing to do with your code. -7. Submit the PR. -8. Did your tests pass on Travis? If they didn't, fix any errors. +6. Submit the PR. +7. Did your tests pass on Travis? If they didn't, fix any errors. Updating certbot-auto and letsencrypt-auto diff --git a/pep8.travis.sh b/pep8.travis.sh deleted file mode 100755 index cadea8489..000000000 --- a/pep8.travis.sh +++ /dev/null @@ -1,5 +0,0 @@ -#!/bin/sh - -set -e # Fail fast - -pep8 --config=acme/.pep8 acme diff --git a/setup.py b/setup.py index 4227d5d92..bc208f693 100644 --- a/setup.py +++ b/setup.py @@ -67,7 +67,6 @@ dev_extras = [ 'astroid==1.3.5', 'coverage', 'nose', - 'pep8', 'psutil>=2.2.1', # for tests, optional 'pylint==1.4.2', # upstream #248 'tox', From e8c8ada91d4ec8f660194ee368fb397540019826 Mon Sep 17 00:00:00 2001 From: Nick Fong Date: Thu, 19 Jan 2017 11:34:10 -0800 Subject: [PATCH 21/88] Make letsencrypt-auto indentation consistent (#3986) * Make certbot-auto indentation consistent Since a majority of certbot-auto uses 2 spaces per indentation level, made indentation in letsencrypt-auto and platform-specific shell scripts a consistent 2 spaces Fixes #3902 * Fix last `if` statement body in rpm_common.sh --- letsencrypt-auto-source/letsencrypt-auto | 150 +++++++++--------- .../pieces/bootstrappers/deb_common.sh | 97 ++++++----- .../pieces/bootstrappers/mac.sh | 12 +- .../pieces/bootstrappers/mageia_common.sh | 29 ++-- .../pieces/bootstrappers/rpm_common.sh | 16 +- 5 files changed, 150 insertions(+), 154 deletions(-) diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index 7210f6a3d..2c23741ca 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -215,71 +215,71 @@ BootstrapDebCommon() { virtualenv= # virtual env is known to apt and is installable if apt-cache show virtualenv > /dev/null 2>&1 ; then - if ! LC_ALL=C apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then - virtualenv="virtualenv" - fi + if ! LC_ALL=C apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then + virtualenv="virtualenv" + fi fi if apt-cache show python-virtualenv > /dev/null 2>&1; then - virtualenv="$virtualenv python-virtualenv" + virtualenv="$virtualenv python-virtualenv" fi augeas_pkg="libaugeas0 augeas-lenses" AUGVERSION=`LC_ALL=C apt-cache show --no-all-versions libaugeas0 | grep ^Version: | cut -d" " -f2` if [ "$ASSUME_YES" = 1 ]; then - YES_FLAG="-y" + YES_FLAG="-y" fi AddBackportRepo() { - # ARGS: - BACKPORT_NAME="$1" - BACKPORT_SOURCELINE="$2" - echo "To use the Apache Certbot plugin, augeas needs to be installed from $BACKPORT_NAME." - if ! grep -v -e ' *#' /etc/apt/sources.list | grep -q "$BACKPORT_NAME" ; then - # This can theoretically error if sources.list.d is empty, but in that case we don't care. - if ! grep -v -e ' *#' /etc/apt/sources.list.d/* 2>/dev/null | grep -q "$BACKPORT_NAME"; then - if [ "$ASSUME_YES" = 1 ]; then - /bin/echo -n "Installing augeas from $BACKPORT_NAME in 3 seconds..." - sleep 1s - /bin/echo -ne "\e[0K\rInstalling augeas from $BACKPORT_NAME in 2 seconds..." - sleep 1s - /bin/echo -e "\e[0K\rInstalling augeas from $BACKPORT_NAME in 1 second ..." - sleep 1s - add_backports=1 - else - read -p "Would you like to enable the $BACKPORT_NAME repository [Y/n]? " response - case $response in - [yY][eE][sS]|[yY]|"") - add_backports=1;; - *) - add_backports=0;; - esac - fi - if [ "$add_backports" = 1 ]; then - $SUDO sh -c "echo $BACKPORT_SOURCELINE >> /etc/apt/sources.list.d/$BACKPORT_NAME.list" - $SUDO apt-get update - fi - fi - fi - if [ "$add_backports" != 0 ]; then - $SUDO apt-get install $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg - augeas_pkg= + # ARGS: + BACKPORT_NAME="$1" + BACKPORT_SOURCELINE="$2" + echo "To use the Apache Certbot plugin, augeas needs to be installed from $BACKPORT_NAME." + if ! grep -v -e ' *#' /etc/apt/sources.list | grep -q "$BACKPORT_NAME" ; then + # This can theoretically error if sources.list.d is empty, but in that case we don't care. + if ! grep -v -e ' *#' /etc/apt/sources.list.d/* 2>/dev/null | grep -q "$BACKPORT_NAME"; then + if [ "$ASSUME_YES" = 1 ]; then + /bin/echo -n "Installing augeas from $BACKPORT_NAME in 3 seconds..." + sleep 1s + /bin/echo -ne "\e[0K\rInstalling augeas from $BACKPORT_NAME in 2 seconds..." + sleep 1s + /bin/echo -e "\e[0K\rInstalling augeas from $BACKPORT_NAME in 1 second ..." + sleep 1s + add_backports=1 + else + read -p "Would you like to enable the $BACKPORT_NAME repository [Y/n]? " response + case $response in + [yY][eE][sS]|[yY]|"") + add_backports=1;; + *) + add_backports=0;; + esac + fi + if [ "$add_backports" = 1 ]; then + $SUDO sh -c "echo $BACKPORT_SOURCELINE >> /etc/apt/sources.list.d/$BACKPORT_NAME.list" + $SUDO apt-get update + fi fi + fi + if [ "$add_backports" != 0 ]; then + $SUDO apt-get install $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg + augeas_pkg= + fi } if dpkg --compare-versions 1.0 gt "$AUGVERSION" ; then - if lsb_release -a | grep -q wheezy ; then - AddBackportRepo wheezy-backports "deb http://http.debian.net/debian wheezy-backports main" - elif lsb_release -a | grep -q precise ; then - # XXX add ARM case - AddBackportRepo precise-backports "deb http://archive.ubuntu.com/ubuntu precise-backports main restricted universe multiverse" - else - echo "No libaugeas0 version is available that's new enough to run the" - echo "Certbot apache plugin..." - fi - # XXX add a case for ubuntu PPAs + if lsb_release -a | grep -q wheezy ; then + AddBackportRepo wheezy-backports "deb http://http.debian.net/debian wheezy-backports main" + elif lsb_release -a | grep -q precise ; then + # XXX add ARM case + AddBackportRepo precise-backports "deb http://archive.ubuntu.com/ubuntu precise-backports main restricted universe multiverse" + else + echo "No libaugeas0 version is available that's new enough to run the" + echo "Certbot apache plugin..." + fi + # XXX add a case for ubuntu PPAs fi $SUDO apt-get install $YES_FLAG --no-install-recommends \ @@ -294,7 +294,6 @@ BootstrapDebCommon() { ca-certificates \ - if ! $EXISTS virtualenv > /dev/null ; then echo Failed to install a working \"virtualenv\" command, exiting exit 1 @@ -331,12 +330,12 @@ BootstrapRpmCommon() { exit 1 fi if [ "$ASSUME_YES" = 1 ]; then - /bin/echo -n "Enabling the EPEL repository in 3 seconds..." - sleep 1s - /bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..." - sleep 1s - /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 seconds..." - sleep 1s + /bin/echo -n "Enabling the EPEL repository in 3 seconds..." + sleep 1s + /bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..." + sleep 1s + /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 seconds..." + sleep 1s fi if ! $SUDO $tool install $yes_flag epel-release; then echo "Could not enable EPEL. Aborting bootstrap!" @@ -505,15 +504,15 @@ BootstrapMac() { fi if ! hash pip 2>/dev/null; then - echo "pip not installed" - echo "Installing pip..." - curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python + echo "pip not installed" + echo "Installing pip..." + curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python fi if ! hash virtualenv 2>/dev/null; then - echo "virtualenv not installed." - echo "Installing with pip..." - pip install virtualenv + echo "virtualenv not installed." + echo "Installing with pip..." + pip install virtualenv fi } @@ -523,26 +522,25 @@ BootstrapSmartOS() { } BootstrapMageiaCommon() { - if ! $SUDO urpmi --force \ - python \ - libpython-devel \ - python-virtualenv + if ! $SUDO urpmi --force \ + python \ + libpython-devel \ + python-virtualenv then echo "Could not install Python dependencies. Aborting bootstrap!" exit 1 - fi + fi - if ! $SUDO urpmi --force \ - git \ - gcc \ - python-augeas \ - openssl \ - libopenssl-devel \ - libffi-devel \ - rootcerts + if ! $SUDO urpmi --force \ + git \ + gcc \ + python-augeas \ + libopenssl-devel \ + libffi-devel \ + rootcerts then - echo "Could not install additional dependencies. Aborting bootstrap!" - exit 1 + echo "Could not install additional dependencies. Aborting bootstrap!" + exit 1 fi } diff --git a/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh index ec60b7525..223280f87 100644 --- a/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh @@ -25,71 +25,71 @@ BootstrapDebCommon() { virtualenv= # virtual env is known to apt and is installable if apt-cache show virtualenv > /dev/null 2>&1 ; then - if ! LC_ALL=C apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then - virtualenv="virtualenv" - fi + if ! LC_ALL=C apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then + virtualenv="virtualenv" + fi fi if apt-cache show python-virtualenv > /dev/null 2>&1; then - virtualenv="$virtualenv python-virtualenv" + virtualenv="$virtualenv python-virtualenv" fi augeas_pkg="libaugeas0 augeas-lenses" AUGVERSION=`LC_ALL=C apt-cache show --no-all-versions libaugeas0 | grep ^Version: | cut -d" " -f2` if [ "$ASSUME_YES" = 1 ]; then - YES_FLAG="-y" + YES_FLAG="-y" fi AddBackportRepo() { - # ARGS: - BACKPORT_NAME="$1" - BACKPORT_SOURCELINE="$2" - echo "To use the Apache Certbot plugin, augeas needs to be installed from $BACKPORT_NAME." - if ! grep -v -e ' *#' /etc/apt/sources.list | grep -q "$BACKPORT_NAME" ; then - # This can theoretically error if sources.list.d is empty, but in that case we don't care. - if ! grep -v -e ' *#' /etc/apt/sources.list.d/* 2>/dev/null | grep -q "$BACKPORT_NAME"; then - if [ "$ASSUME_YES" = 1 ]; then - /bin/echo -n "Installing augeas from $BACKPORT_NAME in 3 seconds..." - sleep 1s - /bin/echo -ne "\e[0K\rInstalling augeas from $BACKPORT_NAME in 2 seconds..." - sleep 1s - /bin/echo -e "\e[0K\rInstalling augeas from $BACKPORT_NAME in 1 second ..." - sleep 1s - add_backports=1 - else - read -p "Would you like to enable the $BACKPORT_NAME repository [Y/n]? " response - case $response in - [yY][eE][sS]|[yY]|"") - add_backports=1;; - *) - add_backports=0;; - esac - fi - if [ "$add_backports" = 1 ]; then - $SUDO sh -c "echo $BACKPORT_SOURCELINE >> /etc/apt/sources.list.d/$BACKPORT_NAME.list" - $SUDO apt-get update - fi - fi - fi - if [ "$add_backports" != 0 ]; then - $SUDO apt-get install $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg - augeas_pkg= + # ARGS: + BACKPORT_NAME="$1" + BACKPORT_SOURCELINE="$2" + echo "To use the Apache Certbot plugin, augeas needs to be installed from $BACKPORT_NAME." + if ! grep -v -e ' *#' /etc/apt/sources.list | grep -q "$BACKPORT_NAME" ; then + # This can theoretically error if sources.list.d is empty, but in that case we don't care. + if ! grep -v -e ' *#' /etc/apt/sources.list.d/* 2>/dev/null | grep -q "$BACKPORT_NAME"; then + if [ "$ASSUME_YES" = 1 ]; then + /bin/echo -n "Installing augeas from $BACKPORT_NAME in 3 seconds..." + sleep 1s + /bin/echo -ne "\e[0K\rInstalling augeas from $BACKPORT_NAME in 2 seconds..." + sleep 1s + /bin/echo -e "\e[0K\rInstalling augeas from $BACKPORT_NAME in 1 second ..." + sleep 1s + add_backports=1 + else + read -p "Would you like to enable the $BACKPORT_NAME repository [Y/n]? " response + case $response in + [yY][eE][sS]|[yY]|"") + add_backports=1;; + *) + add_backports=0;; + esac + fi + if [ "$add_backports" = 1 ]; then + $SUDO sh -c "echo $BACKPORT_SOURCELINE >> /etc/apt/sources.list.d/$BACKPORT_NAME.list" + $SUDO apt-get update + fi fi + fi + if [ "$add_backports" != 0 ]; then + $SUDO apt-get install $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg + augeas_pkg= + fi } if dpkg --compare-versions 1.0 gt "$AUGVERSION" ; then - if lsb_release -a | grep -q wheezy ; then - AddBackportRepo wheezy-backports "deb http://http.debian.net/debian wheezy-backports main" - elif lsb_release -a | grep -q precise ; then - # XXX add ARM case - AddBackportRepo precise-backports "deb http://archive.ubuntu.com/ubuntu precise-backports main restricted universe multiverse" - else - echo "No libaugeas0 version is available that's new enough to run the" - echo "Certbot apache plugin..." - fi - # XXX add a case for ubuntu PPAs + if lsb_release -a | grep -q wheezy ; then + AddBackportRepo wheezy-backports "deb http://http.debian.net/debian wheezy-backports main" + elif lsb_release -a | grep -q precise ; then + # XXX add ARM case + AddBackportRepo precise-backports "deb http://archive.ubuntu.com/ubuntu precise-backports main restricted universe multiverse" + else + echo "No libaugeas0 version is available that's new enough to run the" + echo "Certbot apache plugin..." + fi + # XXX add a case for ubuntu PPAs fi $SUDO apt-get install $YES_FLAG --no-install-recommends \ @@ -104,7 +104,6 @@ BootstrapDebCommon() { ca-certificates \ - if ! $EXISTS virtualenv > /dev/null ; then echo Failed to install a working \"virtualenv\" command, exiting exit 1 diff --git a/letsencrypt-auto-source/pieces/bootstrappers/mac.sh b/letsencrypt-auto-source/pieces/bootstrappers/mac.sh index cafce037a..c101be7d7 100755 --- a/letsencrypt-auto-source/pieces/bootstrappers/mac.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/mac.sh @@ -31,14 +31,14 @@ BootstrapMac() { fi if ! hash pip 2>/dev/null; then - echo "pip not installed" - echo "Installing pip..." - curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python + echo "pip not installed" + echo "Installing pip..." + curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python fi if ! hash virtualenv 2>/dev/null; then - echo "virtualenv not installed." - echo "Installing with pip..." - pip install virtualenv + echo "virtualenv not installed." + echo "Installing with pip..." + pip install virtualenv fi } diff --git a/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh index fb417fd17..3aa0acb1c 100644 --- a/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh @@ -1,23 +1,22 @@ BootstrapMageiaCommon() { - if ! $SUDO urpmi --force \ - python \ - libpython-devel \ - python-virtualenv + if ! $SUDO urpmi --force \ + python \ + libpython-devel \ + python-virtualenv then echo "Could not install Python dependencies. Aborting bootstrap!" exit 1 - fi + fi - if ! $SUDO urpmi --force \ - git \ - gcc \ - python-augeas \ - openssl \ - libopenssl-devel \ - libffi-devel \ - rootcerts + if ! $SUDO urpmi --force \ + git \ + gcc \ + python-augeas \ + libopenssl-devel \ + libffi-devel \ + rootcerts then - echo "Could not install additional dependencies. Aborting bootstrap!" - exit 1 + echo "Could not install additional dependencies. Aborting bootstrap!" + exit 1 fi } diff --git a/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh index 26d717ea1..f86d74a90 100755 --- a/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh @@ -28,12 +28,12 @@ BootstrapRpmCommon() { exit 1 fi if [ "$ASSUME_YES" = 1 ]; then - /bin/echo -n "Enabling the EPEL repository in 3 seconds..." - sleep 1s - /bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..." - sleep 1s - /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 seconds..." - sleep 1s + /bin/echo -n "Enabling the EPEL repository in 3 seconds..." + sleep 1s + /bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..." + sleep 1s + /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 seconds..." + sleep 1s fi if ! $SUDO $tool install $yes_flag epel-release; then echo "Could not enable EPEL. Aborting bootstrap!" @@ -78,7 +78,7 @@ BootstrapRpmCommon() { fi if ! $SUDO $tool install $yes_flag $pkgs; then - echo "Could not install OS dependencies. Aborting bootstrap!" - exit 1 + echo "Could not install OS dependencies. Aborting bootstrap!" + exit 1 fi } From 169b30aa90c3a1231ff18b9dfec179c8b5f51ae2 Mon Sep 17 00:00:00 2001 From: Peter Eckersley Date: Thu, 19 Jan 2017 13:50:07 -0800 Subject: [PATCH 22/88] Fix reference to letsencrypt-auto (#4085) --- docs/contributing.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/contributing.rst b/docs/contributing.rst index fd7caa6af..c51e493bc 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -15,7 +15,7 @@ Running a local copy of the client ---------------------------------- Running the client in developer mode from your local tree is a little -different than running ``letsencrypt-auto``. To get set up, do these things +different than running ``certbot-auto``. To get set up, do these things once: .. code-block:: shell From 02615c2ac67570330ec8faf2ded9421c4efa4bc6 Mon Sep 17 00:00:00 2001 From: Nick Fong Date: Fri, 20 Jan 2017 09:40:36 -0800 Subject: [PATCH 23/88] Silence Package Manager Output when certbot-auto invoked with --quiet (#3776) * Add quiet flags to package manager invocations Add the following flags when 'certbot-auto --quiet' is invoked: - Add '-qq' to calls to 'apt-get' in Debian - Add '--quiet' to calls to 'yum' or 'dnf' in CentOS or Fedora - Add '--quiet' to calls to 'urpmi' in Mageia - Add '--quiet' to calls to 'pkg install' in FreeBSD * Fix $QUIET flag in bootstrappers - Set the value of $QUIET properly (i.e. s/$QUIET/QUIET when setting the variable) in - deb_common.sh - mageia_common.sh - rpm_common.sh - Actually use $QUIET when running $tool in rpm_common.sh * Add handling of $QUIET to Arch and Open Suse * Add logic to set --non-interactive if --quiet * Add missing $QUIET_FLAG to rpm_common.sh * Run build.py * Limit --help to 80 cols * Update indentation within bootstrappers * Add $QUIET_FLAG to second call to `urpmi` (redux) --- letsencrypt-auto-source/letsencrypt-auto | 55 ++++++++++++++----- .../letsencrypt-auto.template | 8 ++- .../pieces/bootstrappers/arch_common.sh | 6 +- .../pieces/bootstrappers/deb_common.sh | 12 ++-- .../pieces/bootstrappers/free_bsd.sh | 6 +- .../pieces/bootstrappers/mageia_common.sh | 8 ++- .../pieces/bootstrappers/rpm_common.sh | 7 ++- .../pieces/bootstrappers/suse_common.sh | 6 +- 8 files changed, 83 insertions(+), 25 deletions(-) diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index 2c23741ca..7e3af8247 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -38,8 +38,9 @@ Help for certbot itself cannot be provided until it is installed. -n, --non-interactive, --noninteractive run without asking for user input --no-self-upgrade do not download updates --os-packages-only install OS dependencies and exit - -q, --quiet provide only update/error output -v, --verbose provide more output + -q, --quiet provide only update/error output; + implies --non-interactive All arguments are accepted and forwarded to the Certbot client when run." @@ -84,6 +85,11 @@ if [ $BASENAME = "letsencrypt-auto" ]; then HELP=0 fi +# Set ASSUME_YES to 1 if QUIET (i.e. --quiet implies --non-interactive) +if [ $QUIET == 1 ]; then + ASSUME_YES=1 +fi + # Support for busybox and others where there is no "command", # but "which" instead if command -v command > /dev/null 2>&1 ; then @@ -207,7 +213,11 @@ BootstrapDebCommon() { # # - Debian 6.0.10 "squeeze" (x64) - $SUDO apt-get update || echo apt-get update hit problems but continuing anyway... + if [ "$QUIET" == 1 ]; then + QUIET_FLAG='-qq' + fi + + $SUDO apt-get $QUIET_FLAG update || echo apt-get update hit problems but continuing anyway... # virtualenv binary can be found in different packages depending on # distro version (#346) @@ -258,12 +268,12 @@ BootstrapDebCommon() { fi if [ "$add_backports" = 1 ]; then $SUDO sh -c "echo $BACKPORT_SOURCELINE >> /etc/apt/sources.list.d/$BACKPORT_NAME.list" - $SUDO apt-get update + $SUDO apt-get $QUIET_FLAG update fi fi fi if [ "$add_backports" != 0 ]; then - $SUDO apt-get install $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg + $SUDO apt-get install $QUIET_FLAG $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg augeas_pkg= fi } @@ -282,7 +292,7 @@ BootstrapDebCommon() { # XXX add a case for ubuntu PPAs fi - $SUDO apt-get install $YES_FLAG --no-install-recommends \ + $SUDO apt-get install $QUIET_FLAG $YES_FLAG --no-install-recommends \ python \ python-dev \ $virtualenv \ @@ -322,6 +332,9 @@ BootstrapRpmCommon() { if [ "$ASSUME_YES" = 1 ]; then yes_flag="-y" fi + if [ "$QUIET" == 1 ]; then + QUIET_FLAG='--quiet' + fi if ! $SUDO $tool list *virtualenv >/dev/null 2>&1; then echo "To use Certbot, packages from the EPEL repository need to be installed." @@ -337,7 +350,7 @@ BootstrapRpmCommon() { /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 seconds..." sleep 1s fi - if ! $SUDO $tool install $yes_flag epel-release; then + if ! $SUDO $tool install $yes_flag $QUIET_FLAG epel-release; then echo "Could not enable EPEL. Aborting bootstrap!" exit 1 fi @@ -379,9 +392,9 @@ BootstrapRpmCommon() { " fi - if ! $SUDO $tool install $yes_flag $pkgs; then - echo "Could not install OS dependencies. Aborting bootstrap!" - exit 1 + if ! $SUDO $tool install $yes_flag $QUIET_FLAG $pkgs; then + echo "Could not install OS dependencies. Aborting bootstrap!" + exit 1 fi } @@ -393,7 +406,11 @@ BootstrapSuseCommon() { install_flags="-l" fi - $SUDO zypper $zypper_flags in $install_flags \ + if [ "$QUIET" == 1 ]; then + QUIET_FLAG='-qq' + fi + + $SUDO zypper $QUIET_FLAG $zypper_flags in $install_flags \ python \ python-devel \ python-virtualenv \ @@ -431,7 +448,11 @@ BootstrapArchCommon() { fi if [ "$missing" ]; then - $SUDO pacman -S --needed $missing $noconfirm + if [ "$QUIET" == 1]; then + $SUDO pacman -S --needed $missing $noconfirm > /dev/null + else + $SUDO pacman -S --needed $missing $noconfirm + fi fi } @@ -464,7 +485,11 @@ BootstrapGentooCommon() { } BootstrapFreeBsd() { - $SUDO pkg install -Ay \ + if [ "$QUIET" == 1 ]; then + QUIET_FLAG="--quiet" + fi + + $SUDO pkg install -Ay $QUIET_FLAG \ python \ py27-virtualenv \ augeas \ @@ -522,7 +547,11 @@ BootstrapSmartOS() { } BootstrapMageiaCommon() { - if ! $SUDO urpmi --force \ + if [ "$QUIET" == 1 ]; then + QUIET_FLAG='--quiet' + fi + + if ! $SUDO urpmi --force $QUIET_FLAG \ python \ libpython-devel \ python-virtualenv diff --git a/letsencrypt-auto-source/letsencrypt-auto.template b/letsencrypt-auto-source/letsencrypt-auto.template index a3ab7eaa1..f0699ef77 100755 --- a/letsencrypt-auto-source/letsencrypt-auto.template +++ b/letsencrypt-auto-source/letsencrypt-auto.template @@ -38,8 +38,9 @@ Help for certbot itself cannot be provided until it is installed. -n, --non-interactive, --noninteractive run without asking for user input --no-self-upgrade do not download updates --os-packages-only install OS dependencies and exit - -q, --quiet provide only update/error output -v, --verbose provide more output + -q, --quiet provide only update/error output; + implies --non-interactive All arguments are accepted and forwarded to the Certbot client when run." @@ -84,6 +85,11 @@ if [ $BASENAME = "letsencrypt-auto" ]; then HELP=0 fi +# Set ASSUME_YES to 1 if QUIET (i.e. --quiet implies --non-interactive) +if [ $QUIET == 1 ]; then + ASSUME_YES=1 +fi + # Support for busybox and others where there is no "command", # but "which" instead if command -v command > /dev/null 2>&1 ; then diff --git a/letsencrypt-auto-source/pieces/bootstrappers/arch_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/arch_common.sh index 333f56ff7..3c8a7e736 100755 --- a/letsencrypt-auto-source/pieces/bootstrappers/arch_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/arch_common.sh @@ -25,6 +25,10 @@ BootstrapArchCommon() { fi if [ "$missing" ]; then - $SUDO pacman -S --needed $missing $noconfirm + if [ "$QUIET" == 1]; then + $SUDO pacman -S --needed $missing $noconfirm > /dev/null + else + $SUDO pacman -S --needed $missing $noconfirm + fi fi } diff --git a/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh index 223280f87..c9158555b 100644 --- a/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh @@ -17,7 +17,11 @@ BootstrapDebCommon() { # # - Debian 6.0.10 "squeeze" (x64) - $SUDO apt-get update || echo apt-get update hit problems but continuing anyway... + if [ "$QUIET" == 1 ]; then + QUIET_FLAG='-qq' + fi + + $SUDO apt-get $QUIET_FLAG update || echo apt-get update hit problems but continuing anyway... # virtualenv binary can be found in different packages depending on # distro version (#346) @@ -68,12 +72,12 @@ BootstrapDebCommon() { fi if [ "$add_backports" = 1 ]; then $SUDO sh -c "echo $BACKPORT_SOURCELINE >> /etc/apt/sources.list.d/$BACKPORT_NAME.list" - $SUDO apt-get update + $SUDO apt-get $QUIET_FLAG update fi fi fi if [ "$add_backports" != 0 ]; then - $SUDO apt-get install $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg + $SUDO apt-get install $QUIET_FLAG $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg augeas_pkg= fi } @@ -92,7 +96,7 @@ BootstrapDebCommon() { # XXX add a case for ubuntu PPAs fi - $SUDO apt-get install $YES_FLAG --no-install-recommends \ + $SUDO apt-get install $QUIET_FLAG $YES_FLAG --no-install-recommends \ python \ python-dev \ $virtualenv \ diff --git a/letsencrypt-auto-source/pieces/bootstrappers/free_bsd.sh b/letsencrypt-auto-source/pieces/bootstrappers/free_bsd.sh index deb2e2115..7d44e740a 100755 --- a/letsencrypt-auto-source/pieces/bootstrappers/free_bsd.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/free_bsd.sh @@ -1,5 +1,9 @@ BootstrapFreeBsd() { - $SUDO pkg install -Ay \ + if [ "$QUIET" == 1 ]; then + QUIET_FLAG="--quiet" + fi + + $SUDO pkg install -Ay $QUIET_FLAG \ python \ py27-virtualenv \ augeas \ diff --git a/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh index 3aa0acb1c..ba234f03c 100644 --- a/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh @@ -1,5 +1,9 @@ BootstrapMageiaCommon() { - if ! $SUDO urpmi --force \ + if [ "$QUIET" == 1 ]; then + QUIET_FLAG='--quiet' + fi + + if ! $SUDO urpmi --force $QUIET_FLAG \ python \ libpython-devel \ python-virtualenv @@ -8,7 +12,7 @@ BootstrapMageiaCommon() { exit 1 fi - if ! $SUDO urpmi --force \ + if ! $SUDO urpmi --force $QUIET_FLAG \ git \ gcc \ python-augeas \ diff --git a/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh index f86d74a90..a157257c6 100755 --- a/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh @@ -20,6 +20,9 @@ BootstrapRpmCommon() { if [ "$ASSUME_YES" = 1 ]; then yes_flag="-y" fi + if [ "$QUIET" == 1 ]; then + QUIET_FLAG='--quiet' + fi if ! $SUDO $tool list *virtualenv >/dev/null 2>&1; then echo "To use Certbot, packages from the EPEL repository need to be installed." @@ -35,7 +38,7 @@ BootstrapRpmCommon() { /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 seconds..." sleep 1s fi - if ! $SUDO $tool install $yes_flag epel-release; then + if ! $SUDO $tool install $yes_flag $QUIET_FLAG epel-release; then echo "Could not enable EPEL. Aborting bootstrap!" exit 1 fi @@ -77,7 +80,7 @@ BootstrapRpmCommon() { " fi - if ! $SUDO $tool install $yes_flag $pkgs; then + if ! $SUDO $tool install $yes_flag $QUIET_FLAG $pkgs; then echo "Could not install OS dependencies. Aborting bootstrap!" exit 1 fi diff --git a/letsencrypt-auto-source/pieces/bootstrappers/suse_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/suse_common.sh index bd4d9c68d..a9520904f 100755 --- a/letsencrypt-auto-source/pieces/bootstrappers/suse_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/suse_common.sh @@ -6,7 +6,11 @@ BootstrapSuseCommon() { install_flags="-l" fi - $SUDO zypper $zypper_flags in $install_flags \ + if [ "$QUIET" == 1 ]; then + QUIET_FLAG='-qq' + fi + + $SUDO zypper $QUIET_FLAG $zypper_flags in $install_flags \ python \ python-devel \ python-virtualenv \ From a1702e766d4ecda952e614cd482b79fb5bc912d4 Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Fri, 20 Jan 2017 11:35:40 -0800 Subject: [PATCH 24/88] Add information about cert management to the docs --- docs/using.rst | 84 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 83 insertions(+), 1 deletion(-) diff --git a/docs/using.rst b/docs/using.rst index a1881852e..be8799858 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -534,8 +534,90 @@ Example usage for DNS-01 (Cloudflare API v4) (for example purposes only, do not fi +.. _managing-certs: - +Managing certificates +===================== + +To view a list of the certificates Certbot knows about, run +the ``certificates`` subcommand: + +``certbot certifices`` + +This will return information in the following format:: + + Found the following certs: + Certificate Name: example.com + Domains: example.com, www.example.com + Expiry Date: 2017-02-19 19:53:00+00:00 (VALID: 30 days) + Certificate Path: /etc/letsencrypt/live/example.com/fullchain.pem + Private Key Path: /etc/letsencrypt/live/example.com/privkey.pem + +``Certificate Name`` gives the name Certbot knows the certificate by. Pass this name +to the ``--cert-name`` flag to specify a particular certificate for the ``run``, +``certonly``, ``certificates``, ``renew`` and ``delete`` commands:: + + certbot certonly --cert-name example.com + +The ``--cert-name`` flag can also be used to modify the domains a certificate contains, +by specifying new domains using the ``-d/--domains`` flag. If certificate ``example.com`` +previously contained ``example.com`` and ``www.example.com``, it can be modified to only +contain ``example.com`` by specifying only ``example.com`` with the ``-d/--domains`` flag:: + + certbot certonly --cert-name example.com -d example.com + +The same format can be used to expand the set of domains a certificate contains, or to +replace that set entirely:: + + certbot certonly --cert-name example.com -d example.org,www.example.org + +If a certificate is requested with ``run`` or ``certonly`` with a name that does not already +exist, the new certificate created will be assigned the name specified. + +If your account key has been compromised or you otherwise need to revoke a certificate, +use the revoke command to do so. Note that the revoke command is passed the certificate path +(ending in ``cert.pem``), not a certificate name or domain. Additionally, if a certificate +is a test cert obtained via the ``--staging/--test-cert`` flag, that flag must be passed to the +``revoke`` subcommand:: + + certbot revoke --cert-path /etc/letsencrypt/live/CERTNAME/cert.pem + +Once a certificate is revoked (or for other cert management tasks), all of a certificate's +relevant files can be removed from the system with the ``delete`` subcommand:: + + certbot delete --cert-name example.com + +For advanced certificate management tasks, it is possible to manually modify the certificate's +renewal configuration file, located at ``/etc/letsencrypt/renewal/CERTNAME``. + +.. warning:: Modifying any files in ``/etc/letsencrypt`` can make it so Certbot can no longer + properly manage its certificates, and we do not recommend doing so for most users. + +If the contents of ``/etc/letsencrypt/archive/CERTNAME`` are moved to a new folder, first specify +the new folder's name in the renewal configuration file, then run ``certbot update_symlinks`` to +point the symlinks in ``/etc/letsencrypt/live/CERTNAME`` to the new folder. + +If you would like the live certificate files whose symlink location Certbot updates on each run to +reside in a different location, first move them to that location, then specify the full path of +each of the four files in the renewal configuration file. Since the symlinks are relative links, +you must follow this with an invocation of ``certbot update_symlinks``. + +For example, say that a certificate's renewal configuration file previously contained the following +directives:: + + archive_dir = /etc/letsencrypt/archive/example.com + cert = /etc/letsencrypt/live/example.com/cert.pem + privkey = /etc/letsencrypt/live/example.com/privkey.pem + chain = /etc/letsencrypt/live/example.com/chain.pem + fullchain = /etc/letsencrypt/live/example.com/fullchain.pem + +The following commands could be used to specify where these files are located:: + + mv /etc/letsencrypt/archive/example.com /home/user/me/certbot/example_archive + sed -i 's,/etc/letsencrypt/archive/example.com,/home/user/me/certbot/example_archive,' /etc/letsencrypt/renewal/example.com.conf + mv /etc/letsencrypt/live/example.com/*.pem /home/user/me/certbot/ + sed -i 's,/etc/letsencrypt/live/example.com,/home/user/me/certbot,g' /etc/letsencrypt/renewal/example.com.conf + certbot update_symlinks .. _config-file: From 6a55de45ba95d9d7f6c21ad78f67229857ab48a7 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Mon, 23 Jan 2017 17:55:59 -0800 Subject: [PATCH 25/88] Fix letsencrypt auto source (#4093) * Some shells don't like == * run build.py --- letsencrypt-auto-source/letsencrypt-auto | 16 ++++++++-------- .../letsencrypt-auto.template | 2 +- .../pieces/bootstrappers/arch_common.sh | 2 +- .../pieces/bootstrappers/deb_common.sh | 2 +- .../pieces/bootstrappers/free_bsd.sh | 2 +- .../pieces/bootstrappers/mageia_common.sh | 2 +- .../pieces/bootstrappers/rpm_common.sh | 2 +- .../pieces/bootstrappers/suse_common.sh | 2 +- 8 files changed, 15 insertions(+), 15 deletions(-) diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index 7e3af8247..453903ab2 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -86,7 +86,7 @@ if [ $BASENAME = "letsencrypt-auto" ]; then fi # Set ASSUME_YES to 1 if QUIET (i.e. --quiet implies --non-interactive) -if [ $QUIET == 1 ]; then +if [ "$QUIET" = 1 ]; then ASSUME_YES=1 fi @@ -213,7 +213,7 @@ BootstrapDebCommon() { # # - Debian 6.0.10 "squeeze" (x64) - if [ "$QUIET" == 1 ]; then + if [ "$QUIET" = 1 ]; then QUIET_FLAG='-qq' fi @@ -332,7 +332,7 @@ BootstrapRpmCommon() { if [ "$ASSUME_YES" = 1 ]; then yes_flag="-y" fi - if [ "$QUIET" == 1 ]; then + if [ "$QUIET" = 1 ]; then QUIET_FLAG='--quiet' fi @@ -406,7 +406,7 @@ BootstrapSuseCommon() { install_flags="-l" fi - if [ "$QUIET" == 1 ]; then + if [ "$QUIET" = 1 ]; then QUIET_FLAG='-qq' fi @@ -448,7 +448,7 @@ BootstrapArchCommon() { fi if [ "$missing" ]; then - if [ "$QUIET" == 1]; then + if [ "$QUIET" = 1]; then $SUDO pacman -S --needed $missing $noconfirm > /dev/null else $SUDO pacman -S --needed $missing $noconfirm @@ -485,7 +485,7 @@ BootstrapGentooCommon() { } BootstrapFreeBsd() { - if [ "$QUIET" == 1 ]; then + if [ "$QUIET" = 1 ]; then QUIET_FLAG="--quiet" fi @@ -547,7 +547,7 @@ BootstrapSmartOS() { } BootstrapMageiaCommon() { - if [ "$QUIET" == 1 ]; then + if [ "$QUIET" = 1 ]; then QUIET_FLAG='--quiet' fi @@ -560,7 +560,7 @@ BootstrapMageiaCommon() { exit 1 fi - if ! $SUDO urpmi --force \ + if ! $SUDO urpmi --force $QUIET_FLAG \ git \ gcc \ python-augeas \ diff --git a/letsencrypt-auto-source/letsencrypt-auto.template b/letsencrypt-auto-source/letsencrypt-auto.template index f0699ef77..a8a9f04d5 100755 --- a/letsencrypt-auto-source/letsencrypt-auto.template +++ b/letsencrypt-auto-source/letsencrypt-auto.template @@ -86,7 +86,7 @@ if [ $BASENAME = "letsencrypt-auto" ]; then fi # Set ASSUME_YES to 1 if QUIET (i.e. --quiet implies --non-interactive) -if [ $QUIET == 1 ]; then +if [ "$QUIET" = 1 ]; then ASSUME_YES=1 fi diff --git a/letsencrypt-auto-source/pieces/bootstrappers/arch_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/arch_common.sh index 3c8a7e736..c3959484b 100755 --- a/letsencrypt-auto-source/pieces/bootstrappers/arch_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/arch_common.sh @@ -25,7 +25,7 @@ BootstrapArchCommon() { fi if [ "$missing" ]; then - if [ "$QUIET" == 1]; then + if [ "$QUIET" = 1]; then $SUDO pacman -S --needed $missing $noconfirm > /dev/null else $SUDO pacman -S --needed $missing $noconfirm diff --git a/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh index c9158555b..7735933c4 100644 --- a/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/deb_common.sh @@ -17,7 +17,7 @@ BootstrapDebCommon() { # # - Debian 6.0.10 "squeeze" (x64) - if [ "$QUIET" == 1 ]; then + if [ "$QUIET" = 1 ]; then QUIET_FLAG='-qq' fi diff --git a/letsencrypt-auto-source/pieces/bootstrappers/free_bsd.sh b/letsencrypt-auto-source/pieces/bootstrappers/free_bsd.sh index 7d44e740a..f1bc00f3b 100755 --- a/letsencrypt-auto-source/pieces/bootstrappers/free_bsd.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/free_bsd.sh @@ -1,5 +1,5 @@ BootstrapFreeBsd() { - if [ "$QUIET" == 1 ]; then + if [ "$QUIET" = 1 ]; then QUIET_FLAG="--quiet" fi diff --git a/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh index ba234f03c..dd1213a4c 100644 --- a/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/mageia_common.sh @@ -1,5 +1,5 @@ BootstrapMageiaCommon() { - if [ "$QUIET" == 1 ]; then + if [ "$QUIET" = 1 ]; then QUIET_FLAG='--quiet' fi diff --git a/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh index a157257c6..44c4625d9 100755 --- a/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/rpm_common.sh @@ -20,7 +20,7 @@ BootstrapRpmCommon() { if [ "$ASSUME_YES" = 1 ]; then yes_flag="-y" fi - if [ "$QUIET" == 1 ]; then + if [ "$QUIET" = 1 ]; then QUIET_FLAG='--quiet' fi diff --git a/letsencrypt-auto-source/pieces/bootstrappers/suse_common.sh b/letsencrypt-auto-source/pieces/bootstrappers/suse_common.sh index a9520904f..e60ca8628 100755 --- a/letsencrypt-auto-source/pieces/bootstrappers/suse_common.sh +++ b/letsencrypt-auto-source/pieces/bootstrappers/suse_common.sh @@ -6,7 +6,7 @@ BootstrapSuseCommon() { install_flags="-l" fi - if [ "$QUIET" == 1 ]; then + if [ "$QUIET" = 1 ]; then QUIET_FLAG='-qq' fi From 44f2d4aa206283bd1a9124504245fc4f7f230f11 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Tue, 24 Jan 2017 14:37:41 -0800 Subject: [PATCH 26/88] Fix modification-check.sh (#4091) * Run build.py * Restore letsencrypt-auto after running build * Remove temp_dir before exiting * add missing $ --- tests/modification-check.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/modification-check.sh b/tests/modification-check.sh index 73cdb0c09..5168f5ab5 100755 --- a/tests/modification-check.sh +++ b/tests/modification-check.sh @@ -46,6 +46,7 @@ cd ${SCRIPT_PATH}/../ cp letsencrypt-auto-source/letsencrypt-auto ${temp_dir}/original-lea python letsencrypt-auto-source/build.py cp letsencrypt-auto-source/letsencrypt-auto ${temp_dir}/build-lea +cp ${temp_dir}/original-lea letsencrypt-auto-source/letsencrypt-auto cd $temp_dir @@ -60,8 +61,8 @@ else build.py." fi +rm -rf $temp_dir + if $FLAG ; then exit 1 fi - -rm -rf temp_dir From 578815a20a7ac8804e8052145c373d65ad690971 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Tue, 24 Jan 2017 14:39:27 -0800 Subject: [PATCH 27/88] uncomment assertion (#4072) --- certbot/display/util.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/certbot/display/util.py b/certbot/display/util.py index 87c75739b..f7b2566a3 100644 --- a/certbot/display/util.py +++ b/certbot/display/util.py @@ -253,7 +253,7 @@ class FileDisplay(object): :rtype: bool """ - # assert_valid_call(prompt, default, cli_flag, force_interactive) + assert_valid_call(prompt, default, cli_flag, force_interactive) if self._can_interact(force_interactive): return False elif default is None: From b6fecca7ba17c7cf8dbfa5206c5e2469c0ccd1f6 Mon Sep 17 00:00:00 2001 From: Frederic BLANC Date: Wed, 25 Jan 2017 13:38:00 -0800 Subject: [PATCH 28/88] fixes #2244 --- acme/acme/client.py | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/acme/acme/client.py b/acme/acme/client.py index 19f566b0d..8fc859c58 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -676,8 +676,16 @@ class ClientNetwork(object): # pylint: disable=too-many-instance-attributes def post(self, url, obj, content_type=JOSE_CONTENT_TYPE, **kwargs): """POST object wrapped in `.JWS` and check response.""" - data = self._wrap_in_jws(obj, self._get_nonce(url)) - kwargs.setdefault('headers', {'Content-Type': content_type}) - response = self._send_request('POST', url, data=data, **kwargs) - self._add_nonce(response) - return self._check_response(response, content_type=content_type) + MAX_ATTEMPTS = 3 + for attempt in range(MAX_ATTEMPTS+1): + try: + data = self._wrap_in_jws(obj, self._get_nonce(url)) + kwargs.setdefault('headers', {'Content-Type': content_type}) + response = self._send_request('POST', url, data=data, **kwargs) + self._add_nonce(response) + return self._check_response(response, content_type=content_type) + except messages.Error as e: + if attempt < MAX_ATTEMPTS and e.typ == 'urn:ietf:params:acme:error:badNonce': + logger.debug('Got badNonce error (%i times)', attempt+1) + else: + raise From a5da551965bf78cd5394c5cafb10e1893bbd132b Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 25 Jan 2017 14:10:19 -0800 Subject: [PATCH 29/88] fix stylistic nits with POST retry --- acme/acme/client.py | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/acme/acme/client.py b/acme/acme/client.py index 8fc859c58..7555a1cc7 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -674,18 +674,27 @@ class ClientNetwork(object): # pylint: disable=too-many-instance-attributes self._add_nonce(self.head(url)) return self._nonces.pop() - def post(self, url, obj, content_type=JOSE_CONTENT_TYPE, **kwargs): - """POST object wrapped in `.JWS` and check response.""" - MAX_ATTEMPTS = 3 - for attempt in range(MAX_ATTEMPTS+1): + def post(self, *args, **kwargs): + """POST object wrapped in `.JWS` and check response. + + If the server responded with a badNonce error, the request will + be retried once. + + """ + should_retry = True + while True: try: - data = self._wrap_in_jws(obj, self._get_nonce(url)) - kwargs.setdefault('headers', {'Content-Type': content_type}) - response = self._send_request('POST', url, data=data, **kwargs) - self._add_nonce(response) - return self._check_response(response, content_type=content_type) - except messages.Error as e: - if attempt < MAX_ATTEMPTS and e.typ == 'urn:ietf:params:acme:error:badNonce': - logger.debug('Got badNonce error (%i times)', attempt+1) + return self._post_once(*args, **kwargs) + except messages.Error as error: + if should_retry and error.code == 'badNonce': + logger.debug('Retrying request after error:\n%s', error) + should_retry = False else: raise + + def _post_once(self, url, obj, content_type=JOSE_CONTENT_TYPE, **kwargs): + data = self._wrap_in_jws(obj, self._get_nonce(url)) + kwargs.setdefault('headers', {'Content-Type': content_type}) + response = self._send_request('POST', url, data=data, **kwargs) + self._add_nonce(response) + return self._check_response(response, content_type=content_type) From 46d9809fa1a99caf2027e181dc39c70ebc5944a0 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 25 Jan 2017 15:08:01 -0800 Subject: [PATCH 30/88] add test_post_failed_retry --- acme/acme/client_test.py | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index 62555939c..5c9e44a2f 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -642,7 +642,9 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase): self.wrapped_obj = mock.MagicMock() self.content_type = mock.sentinel.content_type - self.all_nonces = [jose.b64encode(b'Nonce'), jose.b64encode(b'Nonce2')] + self.all_nonces = [ + jose.b64encode(b'Nonce'), + jose.b64encode(b'Nonce2'), jose.b64encode(b'Nonce3')] self.available_nonces = self.all_nonces[:] def send_request(*args, **kwargs): @@ -690,7 +692,7 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase): self.net._wrap_in_jws.assert_called_once_with( self.obj, jose.b64decode(self.all_nonces.pop())) - assert not self.available_nonces + self.available_nonces = [] self.assertRaises(errors.MissingNonce, self.net.post, 'uri', self.obj, content_type=self.content_type) self.net._wrap_in_jws.assert_called_with( @@ -706,6 +708,15 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase): self.assertRaises(errors.BadNonce, self.net.post, 'uri', self.obj, content_type=self.content_type) + def test_post_failed_retry(self): + check_response = mock.MagicMock() + check_response.side_effect = messages.Error.with_code('badNonce') + + # pylint: disable=protected-access + self.net._check_response = check_response + self.assertRaises(messages.Error, self.net.post, 'uri', + self.obj, content_type=self.content_type) + def test_head_get_post_error_passthrough(self): self.send_request.side_effect = requests.exceptions.RequestException for method in self.net.head, self.net.get: From c650c9a7099d2d918d3a7dcb3fde4ef07e2829f4 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 25 Jan 2017 15:10:24 -0800 Subject: [PATCH 31/88] add test_post_successful_retry --- acme/acme/client_test.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index 5c9e44a2f..e88baa340 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -717,6 +717,16 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase): self.assertRaises(messages.Error, self.net.post, 'uri', self.obj, content_type=self.content_type) + def test_post_successful_retry(self): + check_response = mock.MagicMock() + check_response.side_effect = [messages.Error.with_code('badNonce'), + self.checked_response] + + # pylint: disable=protected-access + self.net._check_response = check_response + self.assertEqual(self.checked_response, self.net.post( + 'uri', self.obj, content_type=self.content_type)) + def test_head_get_post_error_passthrough(self): self.send_request.side_effect = requests.exceptions.RequestException for method in self.net.head, self.net.get: From 4d860b37b0cb106d8a1b85dad18db2bc9cd4fd89 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 25 Jan 2017 18:40:22 -0800 Subject: [PATCH 32/88] Preserve preferred-challenges on renewal (#4112) * use challenge type strings, not objectS * Factor out parse_preferred_challenges * restore pref_challs * save pref_challs * Make CheckCertCount more flexible * improve integration tests * Make pref_challs more flexible --- certbot/auth_handler.py | 9 ++++--- certbot/cli.py | 39 ++++++++++++++++++++++-------- certbot/renewal.py | 25 ++++++++++++++++++- certbot/tests/auth_handler_test.py | 5 ++-- certbot/tests/cli_test.py | 8 +++--- certbot/tests/renewal_test.py | 25 +++++++++++++++++++ tests/boulder-integration.sh | 31 +++++++++++++++--------- 7 files changed, 110 insertions(+), 32 deletions(-) diff --git a/certbot/auth_handler.py b/certbot/auth_handler.py index aad971eb6..6e9ab25a7 100644 --- a/certbot/auth_handler.py +++ b/certbot/auth_handler.py @@ -34,8 +34,7 @@ class AuthHandler(object): :ivar list achalls: DV challenges in the form of :class:`certbot.achallenges.AnnotatedChallenge` :ivar list pref_challs: sorted user specified preferred challenges - in the form of subclasses of :class:`acme.challenges.Challenge` - with the most preferred challenge listed first + type strings with the most preferred challenge listed first """ def __init__(self, auth, acme, account, pref_challs): @@ -252,8 +251,10 @@ class AuthHandler(object): # Make sure to make a copy... plugin_pref = self.auth.get_chall_pref(domain) if self.pref_challs: - chall_prefs.extend(pref for pref in self.pref_challs - if pref in plugin_pref) + plugin_pref_types = set(chall.typ for chall in plugin_pref) + for typ in self.pref_challs: + if typ in plugin_pref_types: + chall_prefs.append(challenges.Challenge.TYPES[typ]) if chall_prefs: return chall_prefs raise errors.AuthorizationError( diff --git a/certbot/cli.py b/certbot/cli.py index 7d146e33b..88d6d39db 100644 --- a/certbot/cli.py +++ b/certbot/cli.py @@ -1,4 +1,5 @@ """Certbot command line argument & config processing.""" +# pylint: disable=too-many-lines from __future__ import print_function import argparse import copy @@ -1236,13 +1237,31 @@ class _PrefChallAction(argparse.Action): """Action class for parsing preferred challenges.""" def __call__(self, parser, namespace, pref_challs, option_string=None): - aliases = {"dns": "dns-01", "http": "http-01", "tls-sni": "tls-sni-01"} - challs = [c.strip() for c in pref_challs.split(",")] - challs = [aliases[c] if c in aliases else c for c in challs] - unrecognized = ", ".join(name for name in challs - if name not in challenges.Challenge.TYPES) - if unrecognized: - raise argparse.ArgumentTypeError( - "Unrecognized challenges: {0}".format(unrecognized)) - namespace.pref_challs.extend(challenges.Challenge.TYPES[name] - for name in challs) + try: + challs = parse_preferred_challenges(pref_challs.split(",")) + except errors.Error as error: + raise argparse.ArgumentTypeError(str(error)) + namespace.pref_challs.extend(challs) + + +def parse_preferred_challenges(pref_challs): + """Translate and validate preferred challenges. + + :param pref_challs: list of preferred challenge types + :type pref_challs: `list` of `str` + + :returns: validated list of preferred challenge types + :rtype: `list` of `str` + + :raises errors.Error: if pref_challs is invalid + + """ + aliases = {"dns": "dns-01", "http": "http-01", "tls-sni": "tls-sni-01"} + challs = [c.strip() for c in pref_challs] + challs = [aliases.get(c, c) for c in challs] + unrecognized = ", ".join(name for name in challs + if name not in challenges.Challenge.TYPES) + if unrecognized: + raise errors.Error( + "Unrecognized challenges: {0}".format(unrecognized)) + return challs diff --git a/certbot/renewal.py b/certbot/renewal.py index bd07cfd07..6b61b0841 100644 --- a/certbot/renewal.py +++ b/certbot/renewal.py @@ -35,7 +35,7 @@ INT_CONFIG_ITEMS = ["rsa_key_size", "tls_sni_01_port", "http01_port"] BOOL_CONFIG_ITEMS = ["must_staple", "allow_subset_of_names"] CONFIG_ITEMS = set(itertools.chain( - BOOL_CONFIG_ITEMS, INT_CONFIG_ITEMS, STR_CONFIG_ITEMS)) + BOOL_CONFIG_ITEMS, INT_CONFIG_ITEMS, STR_CONFIG_ITEMS, ('pref_challs',))) def _reconstitute(config, full_path): @@ -165,6 +165,7 @@ def restore_required_config_elements(config, renewalparams): """ required_items = itertools.chain( + (("pref_challs", _restore_pref_challs),), six.moves.zip(BOOL_CONFIG_ITEMS, itertools.repeat(_restore_bool)), six.moves.zip(INT_CONFIG_ITEMS, itertools.repeat(_restore_int)), six.moves.zip(STR_CONFIG_ITEMS, itertools.repeat(_restore_str))) @@ -174,6 +175,28 @@ def restore_required_config_elements(config, renewalparams): setattr(config.namespace, item_name, value) +def _restore_pref_challs(unused_name, value): + """Restores preferred challenges from a renewal config file. + + If value is a `str`, it should be a single challenge type. + + :param str unused_name: option name + :param value: option value + :type value: `list` of `str` or `str` + + :returns: converted option value to be stored in the runtime config + :rtype: `list` of `str` + + :raises errors.Error: if value can't be converted to an bool + + """ + # If pref_challs has only one element, configobj saves the value + # with a trailing comma so it's parsed as a list. If this comma is + # removed by the user, the value is parsed as a str. + value = [value] if isinstance(value, str) else value + return cli.parse_preferred_challenges(value) + + def _restore_bool(name, value): """Restores an boolean key-value pair from a renewal config file. diff --git a/certbot/tests/auth_handler_test.py b/certbot/tests/auth_handler_test.py index 441550fc8..046eb5ef1 100644 --- a/certbot/tests/auth_handler_test.py +++ b/certbot/tests/auth_handler_test.py @@ -176,7 +176,8 @@ class GetAuthorizationsTest(unittest.TestCase): mock_poll.side_effect = self._validate_all self.mock_auth.get_chall_pref.return_value.append(challenges.HTTP01) - self.handler.pref_challs.extend((challenges.HTTP01, challenges.DNS01,)) + self.handler.pref_challs.extend((challenges.HTTP01.typ, + challenges.DNS01.typ,)) self.handler.get_authorizations(["0"]) @@ -187,7 +188,7 @@ class GetAuthorizationsTest(unittest.TestCase): def test_preferred_challenges_not_supported(self): self.mock_net.request_domain_challenges.side_effect = functools.partial( gen_dom_authzr, challs=acme_util.CHALLENGES) - self.handler.pref_challs.append(challenges.HTTP01) + self.handler.pref_challs.append(challenges.HTTP01.typ) self.assertRaises( errors.AuthorizationError, self.handler.get_authorizations, ["0"]) diff --git a/certbot/tests/cli_test.py b/certbot/tests/cli_test.py index 1fa2004d3..aff1bcc99 100644 --- a/certbot/tests/cli_test.py +++ b/certbot/tests/cli_test.py @@ -8,6 +8,8 @@ import mock import six from six.moves import reload_module # pylint: disable=import-error +from acme import challenges + from certbot import cli from certbot import constants from certbot import errors @@ -178,12 +180,12 @@ class ParseTest(unittest.TestCase): self.assertEqual(namespace.domains, ['example.com', 'another.net']) def test_preferred_challenges(self): - from acme.challenges import HTTP01, TLSSNI01, DNS01 - short_args = ['--preferred-challenges', 'http, tls-sni-01, dns'] namespace = self.parse(short_args) - self.assertEqual(namespace.pref_challs, [HTTP01, TLSSNI01, DNS01]) + expected = [challenges.HTTP01.typ, + challenges.TLSSNI01.typ, challenges.DNS01.typ] + self.assertEqual(namespace.pref_challs, expected) short_args = ['--preferred-challenges', 'jumping-over-the-moon'] self.assertRaises(argparse.ArgumentTypeError, self.parse, short_args) diff --git a/certbot/tests/renewal_test.py b/certbot/tests/renewal_test.py index 2fe4eab68..cd53aa91c 100644 --- a/certbot/tests/renewal_test.py +++ b/certbot/tests/renewal_test.py @@ -5,6 +5,8 @@ import unittest import shutil import tempfile +from acme import challenges + from certbot import configuration from certbot import errors from certbot import storage @@ -59,6 +61,29 @@ class RestoreRequiredConfigElementsTest(unittest.TestCase): self.assertRaises( errors.Error, self._call, self.config, renewalparams) + @mock.patch('certbot.renewal.cli.set_by_cli') + def test_pref_challs_list(self, mock_set_by_cli): + mock_set_by_cli.return_value = False + renewalparams = {'pref_challs': 'tls-sni, http-01, dns'.split(',')} + self._call(self.config, renewalparams) + expected = [challenges.TLSSNI01.typ, + challenges.HTTP01.typ, challenges.DNS01.typ] + self.assertEqual(self.config.namespace.pref_challs, expected) + + @mock.patch('certbot.renewal.cli.set_by_cli') + def test_pref_challs_str(self, mock_set_by_cli): + mock_set_by_cli.return_value = False + renewalparams = {'pref_challs': 'dns'} + self._call(self.config, renewalparams) + expected = [challenges.DNS01.typ] + self.assertEqual(self.config.namespace.pref_challs, expected) + + @mock.patch('certbot.renewal.cli.set_by_cli') + def test_pref_challs_failure(self, mock_set_by_cli): + mock_set_by_cli.return_value = False + renewalparams = {'pref_challs': 'finding-a-shrubbery'} + self.assertRaises(errors.Error, self._call, self.config, renewalparams) + @mock.patch('certbot.renewal.cli.set_by_cli') def test_must_staple_success(self, mock_set_by_cli): mock_set_by_cli.return_value = False diff --git a/tests/boulder-integration.sh b/tests/boulder-integration.sh index 9c6bc7708..930cf1c0e 100755 --- a/tests/boulder-integration.sh +++ b/tests/boulder-integration.sh @@ -96,7 +96,7 @@ common certonly -a manual -d le.wtf --rsa-key-size 4096 \ --pre-hook 'echo wtf2.pre >> "$HOOK_TEST"' \ --post-hook 'echo wtf2.post >> "$HOOK_TEST"' -common certonly -a manual -d dns.le.wtf --preferred-challenges dns-01 \ +common certonly -a manual -d dns.le.wtf --preferred-challenges dns,tls-sni \ --manual-auth-hook ./tests/manual-dns-auth.sh export CSR_PATH="${root}/csr.der" KEY_PATH="${root}/key.pem" \ @@ -113,29 +113,30 @@ common --domains le3.wtf install \ --key-path "${root}/csr/key.pem" CheckCertCount() { - CERTCOUNT=`ls "${root}/conf/archive/le.wtf/cert"* | wc -l` - if [ "$CERTCOUNT" -ne "$1" ] ; then - echo Wrong cert count, not "$1" `ls "${root}/conf/archive/le.wtf/"*` + CERTCOUNT=`ls "${root}/conf/archive/$1/cert"* | wc -l` + if [ "$CERTCOUNT" -ne "$2" ] ; then + echo Wrong cert count, not "$2" `ls "${root}/conf/archive/$1/"*` exit 1 fi } -CheckCertCount 1 +CheckCertCount "le.wtf" 1 # This won't renew (because it's not time yet) common_no_force_renew renew -CheckCertCount 1 +CheckCertCount "le.wtf" 1 -# --renew-by-default is used, so renewal should occur -[ -f "$HOOK_TEST" ] && rm -f "$HOOK_TEST" -common renew -CheckCertCount 2 -CheckHooks +# renew using HTTP manual auth hooks +common renew --cert-name le.wtf --authenticator manual +CheckCertCount "le.wtf" 2 +# renew using DNS manual auth hooks +common renew --cert-name dns.le.wtf --authenticator manual +CheckCertCount "dns.le.wtf" 2 # This will renew because the expiry is less than 10 years from now sed -i "4arenew_before_expiry = 4 years" "$root/conf/renewal/le.wtf.conf" common_no_force_renew renew --rsa-key-size 2048 -CheckCertCount 3 +CheckCertCount "le.wtf" 3 # The 4096 bit setting should persist to the first renewal, but be overriden in the second @@ -149,6 +150,12 @@ if [ "$size1" -lt 3000 ] || [ "$size2" -lt 3000 ] || [ "$size3" -gt 1800 ] ; the exit 1 fi +# --renew-by-default is used, so renewal should occur +[ -f "$HOOK_TEST" ] && rm -f "$HOOK_TEST" +common renew +CheckCertCount "le.wtf" 4 +CheckHooks + # ECDSA openssl ecparam -genkey -name secp384r1 -out "${root}/privkey-p384.pem" SAN="DNS:ecdsa.le.wtf" openssl req -new -sha256 \ From 4e297b02482d9df22a4bc4808d7a901dddfa0412 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Thu, 26 Jan 2017 12:03:12 -0800 Subject: [PATCH 33/88] Release 0.10.2 (#4120) * Release 0.10.2 (cherry picked from commit 535b04be238bdfeb16341451a936b0bb801389ab) * Bump version to 0.11.0 (cherry picked from commit 01f84df0641a98bb57b4b7be7980c3c1afe375d5) --- certbot-auto | 26 +++++++++--------- docs/cli-help.txt | 2 +- letsencrypt-auto | 26 +++++++++--------- letsencrypt-auto-source/certbot-auto.asc | 14 +++++----- letsencrypt-auto-source/letsencrypt-auto | 24 ++++++++-------- letsencrypt-auto-source/letsencrypt-auto.sig | Bin 256 -> 256 bytes .../pieces/letsencrypt-auto-requirements.txt | 24 ++++++++-------- 7 files changed, 58 insertions(+), 58 deletions(-) diff --git a/certbot-auto b/certbot-auto index b22916997..4e27bbd52 100755 --- a/certbot-auto +++ b/certbot-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.10.1" +LE_AUTO_VERSION="0.10.2" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -801,18 +801,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.1 \ - --hash=sha256:1dd5124078bc44739065409f3be51765608a90994c83460578c2680c582c1026 \ - --hash=sha256:f51c2fb0a31646364abeb7fdd8cfc7c8a4e63b0641b14ab3ce1b2e3a8921a211 -certbot==0.10.1 \ - --hash=sha256:9a56fc76f726beeed2f5a08d690088377cd430907f8a38c50e2aa9a258ee1253 \ - --hash=sha256:e0d699adb3f8ca3e077a4db339de29ebb3f790fbc5f3f02e446e227ed40aa743 -certbot-apache==0.10.1 \ - --hash=sha256:1252fd7e435ba48484b0bd9b72535a9755b03d8f0440f164b9c1c560d96cadb8 \ - --hash=sha256:134f46690da55262125defa58aa74472eb4a1555c9ed83edb3c8667df5a561b5 -certbot-nginx==0.10.1 \ - --hash=sha256:afd15ed9e4f3076056b63916f272b7287084d871cb8136477d16b08f64d514f0 \ - --hash=sha256:da1b7ea4831ead3f9eb526ee11bf1bf197da0fea4defeeb7b1ce24c5d3f45b51 +acme==0.10.2 \ + --hash=sha256:053bbf8d36be298e411e0fa397289307e1a888f419de1205931236114f8f825e \ + --hash=sha256:6465b66256849f8cd4269a43037d404b1ccf31a60d3c93fc628d6f6f4c43de78 +certbot==0.10.2 \ + --hash=sha256:1e680cab3cda6c3079ee64066903f93e421df46a8d4db63264a6359f3d5ba63e \ + --hash=sha256:33729ce100fc9d00d9820c81fc3ef9b95802b8d261f0cca0cce422bc568b1631 +certbot-apache==0.10.2 \ + --hash=sha256:c1f196506bf961e2171fb2af9feca8926100564b644acda1895acdf294c46e71 \ + --hash=sha256:8573275e74c8b74a9c7d7824636a37a4bc83327a7cc97a43b334fe8956ba8cb1 +certbot-nginx==0.10.2 \ + --hash=sha256:74c32bdaf2f3873aa811c3e2cdc205d3137b59c64dbcdca6f3982ed2a89b4f2a \ + --hash=sha256:c8265ab4b4bc0a6a3a81d7f0b8ff2606dd28ebbb3cd045635862286b97af2c36 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/docs/cli-help.txt b/docs/cli-help.txt index 672704ebc..349092630 100644 --- a/docs/cli-help.txt +++ b/docs/cli-help.txt @@ -86,7 +86,7 @@ optional arguments: statistics about success rates by OS and plugin. If you wish to hide your server OS version from the Let's Encrypt server, set this to "". (default: - CertbotACMEClient/0.10.1 (Ubuntu 16.04.1 LTS) + CertbotACMEClient/0.10.2 (Ubuntu 16.04.1 LTS) Authenticator/XXX Installer/YYY) automation: diff --git a/letsencrypt-auto b/letsencrypt-auto index b22916997..4e27bbd52 100755 --- a/letsencrypt-auto +++ b/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.10.1" +LE_AUTO_VERSION="0.10.2" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -801,18 +801,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.1 \ - --hash=sha256:1dd5124078bc44739065409f3be51765608a90994c83460578c2680c582c1026 \ - --hash=sha256:f51c2fb0a31646364abeb7fdd8cfc7c8a4e63b0641b14ab3ce1b2e3a8921a211 -certbot==0.10.1 \ - --hash=sha256:9a56fc76f726beeed2f5a08d690088377cd430907f8a38c50e2aa9a258ee1253 \ - --hash=sha256:e0d699adb3f8ca3e077a4db339de29ebb3f790fbc5f3f02e446e227ed40aa743 -certbot-apache==0.10.1 \ - --hash=sha256:1252fd7e435ba48484b0bd9b72535a9755b03d8f0440f164b9c1c560d96cadb8 \ - --hash=sha256:134f46690da55262125defa58aa74472eb4a1555c9ed83edb3c8667df5a561b5 -certbot-nginx==0.10.1 \ - --hash=sha256:afd15ed9e4f3076056b63916f272b7287084d871cb8136477d16b08f64d514f0 \ - --hash=sha256:da1b7ea4831ead3f9eb526ee11bf1bf197da0fea4defeeb7b1ce24c5d3f45b51 +acme==0.10.2 \ + --hash=sha256:053bbf8d36be298e411e0fa397289307e1a888f419de1205931236114f8f825e \ + --hash=sha256:6465b66256849f8cd4269a43037d404b1ccf31a60d3c93fc628d6f6f4c43de78 +certbot==0.10.2 \ + --hash=sha256:1e680cab3cda6c3079ee64066903f93e421df46a8d4db63264a6359f3d5ba63e \ + --hash=sha256:33729ce100fc9d00d9820c81fc3ef9b95802b8d261f0cca0cce422bc568b1631 +certbot-apache==0.10.2 \ + --hash=sha256:c1f196506bf961e2171fb2af9feca8926100564b644acda1895acdf294c46e71 \ + --hash=sha256:8573275e74c8b74a9c7d7824636a37a4bc83327a7cc97a43b334fe8956ba8cb1 +certbot-nginx==0.10.2 \ + --hash=sha256:74c32bdaf2f3873aa811c3e2cdc205d3137b59c64dbcdca6f3982ed2a89b4f2a \ + --hash=sha256:c8265ab4b4bc0a6a3a81d7f0b8ff2606dd28ebbb3cd045635862286b97af2c36 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/certbot-auto.asc b/letsencrypt-auto-source/certbot-auto.asc index 4f3794505..a848695f6 100644 --- a/letsencrypt-auto-source/certbot-auto.asc +++ b/letsencrypt-auto-source/certbot-auto.asc @@ -1,11 +1,11 @@ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 -iQEcBAABAgAGBQJYeWiuAAoJEE0XyZXNl3Xy4DsIALpCrYVDK9g1nQtdNzBJuFq7 -WlMEk3ofMrh+3sTEit1jr9+zJ2hV3POa3RtCRfRZsP0hsiNx7XbuR8t6yM6d4j3S -pAO0L5brrNoJKvKPx5v8AVGKm1EIDH/lWx/hH+HiE7OE3z/w0ppwoIy0/xIqMXt9 -O7Q70qV5Rvzh0K3KwSS9pb82ybV/mQ35uugravSOTMo2hZ82Ifh2ZEkJIsS4si2j -Oc26ruLi6ru4vwHJU2DkHZzl0oncyQZFolMZJmB47vNOCDtC303cR/poeQS9LSmF -vq3Lr0FAunCBoCjuyIGMk2SfmIhtJMS1v5dxOpZppVexedBYkoqU0FoUp/10/rM= -=7Xh1 +iQEcBAABAgAGBQJYiWfOAAoJEE0XyZXNl3XyES8H/A77QJnVn4mNBv5+WDvOuxii +OVch92SHT9FSrUpM0QNAB981SzjSt6vAwjdFjqyMYmfrITYl5Kfo9UjHJY3gfYAR +S+oiSnBBJ1+9O1+s9kRt0/zwXS6iqIyT7gKM6ZQ5PPaKeB0Iw820m16QgOdMN6d3 +jBQlUIqg6Bl5y79znx8yAYXpIrbOj/Q1xcYpKjRcnyzgR7sWIGvJvVygtXcDN+QV +HUjtumjnqFON4+638N6xPeiJobTGgTwOLr91I0IhNWKCZ5hBgvm8Oq8r+cfZejCX +XIdtlvmcw08GiKsFY8WW7k9rK1/D8uy4Xx26KAAXmdY/xdyG9rOBHOui2N921nw= +=B8NE -----END PGP SIGNATURE----- diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index 453903ab2..e5e8a431b 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.1 \ - --hash=sha256:1dd5124078bc44739065409f3be51765608a90994c83460578c2680c582c1026 \ - --hash=sha256:f51c2fb0a31646364abeb7fdd8cfc7c8a4e63b0641b14ab3ce1b2e3a8921a211 -certbot==0.10.1 \ - --hash=sha256:9a56fc76f726beeed2f5a08d690088377cd430907f8a38c50e2aa9a258ee1253 \ - --hash=sha256:e0d699adb3f8ca3e077a4db339de29ebb3f790fbc5f3f02e446e227ed40aa743 -certbot-apache==0.10.1 \ - --hash=sha256:1252fd7e435ba48484b0bd9b72535a9755b03d8f0440f164b9c1c560d96cadb8 \ - --hash=sha256:134f46690da55262125defa58aa74472eb4a1555c9ed83edb3c8667df5a561b5 -certbot-nginx==0.10.1 \ - --hash=sha256:afd15ed9e4f3076056b63916f272b7287084d871cb8136477d16b08f64d514f0 \ - --hash=sha256:da1b7ea4831ead3f9eb526ee11bf1bf197da0fea4defeeb7b1ce24c5d3f45b51 +acme==0.10.2 \ + --hash=sha256:053bbf8d36be298e411e0fa397289307e1a888f419de1205931236114f8f825e \ + --hash=sha256:6465b66256849f8cd4269a43037d404b1ccf31a60d3c93fc628d6f6f4c43de78 +certbot==0.10.2 \ + --hash=sha256:1e680cab3cda6c3079ee64066903f93e421df46a8d4db63264a6359f3d5ba63e \ + --hash=sha256:33729ce100fc9d00d9820c81fc3ef9b95802b8d261f0cca0cce422bc568b1631 +certbot-apache==0.10.2 \ + --hash=sha256:c1f196506bf961e2171fb2af9feca8926100564b644acda1895acdf294c46e71 \ + --hash=sha256:8573275e74c8b74a9c7d7824636a37a4bc83327a7cc97a43b334fe8956ba8cb1 +certbot-nginx==0.10.2 \ + --hash=sha256:74c32bdaf2f3873aa811c3e2cdc205d3137b59c64dbcdca6f3982ed2a89b4f2a \ + --hash=sha256:c8265ab4b4bc0a6a3a81d7f0b8ff2606dd28ebbb3cd045635862286b97af2c36 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/letsencrypt-auto.sig b/letsencrypt-auto-source/letsencrypt-auto.sig index b6ae7f614ab4dc2a690d79615714bad351b4d11d..cbddf8c9bccde1ecc8c53254313d39060522be1e 100644 GIT binary patch literal 256 zcmV+b0ssDd$`Yq2MInxMh51elUnIKs*wdUV$0#9hQ=wglU`j)LD$A}{ioX7lwSwZ> z_!77WQ6IlFQ18g-*={1409g#f#CD+@O`HD|kne0eV&|SxIo}L&k*7%lck{g{A>6f!_?;cW3 z2fgTd_zjY$=J$iz&I7+pEYyX4qZrNz!b_X=IpIIEEPf>iEXtFqb9s% GZWaOsyMF8d literal 256 zcmV+b0ssDm(KG>Xw6mz{JB&(j^_BG3!xds`roOPc?uil419D%7I;HWrTdP(2?gGyB zjG#Bt15g4+=GurfU8c;+Cd)41rc+>iVsh)w(Q2F8%;v4;cdep{`XpT`o%4OJ?`;VK zop-OvLXUIwE||rjxJY3^(;55MngGGlfJr5g@;B~6*J`~+O5#W<^kAFVvGBK^iM$mF zuIbNxTp?ieypH;OxYSMtN2&3PurD!$6pw#B6$G^-InE`mQ(KN&xNc{}$2k@L+FWxp z!yloUcag}@F*LZd#(uYev#n14TVxp+u?-`@ GmU%|o+lJu) diff --git a/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt b/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt index dc199bb4a..3f448889d 100644 --- a/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt +++ b/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt @@ -171,15 +171,15 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.1 \ - --hash=sha256:1dd5124078bc44739065409f3be51765608a90994c83460578c2680c582c1026 \ - --hash=sha256:f51c2fb0a31646364abeb7fdd8cfc7c8a4e63b0641b14ab3ce1b2e3a8921a211 -certbot==0.10.1 \ - --hash=sha256:9a56fc76f726beeed2f5a08d690088377cd430907f8a38c50e2aa9a258ee1253 \ - --hash=sha256:e0d699adb3f8ca3e077a4db339de29ebb3f790fbc5f3f02e446e227ed40aa743 -certbot-apache==0.10.1 \ - --hash=sha256:1252fd7e435ba48484b0bd9b72535a9755b03d8f0440f164b9c1c560d96cadb8 \ - --hash=sha256:134f46690da55262125defa58aa74472eb4a1555c9ed83edb3c8667df5a561b5 -certbot-nginx==0.10.1 \ - --hash=sha256:afd15ed9e4f3076056b63916f272b7287084d871cb8136477d16b08f64d514f0 \ - --hash=sha256:da1b7ea4831ead3f9eb526ee11bf1bf197da0fea4defeeb7b1ce24c5d3f45b51 +acme==0.10.2 \ + --hash=sha256:053bbf8d36be298e411e0fa397289307e1a888f419de1205931236114f8f825e \ + --hash=sha256:6465b66256849f8cd4269a43037d404b1ccf31a60d3c93fc628d6f6f4c43de78 +certbot==0.10.2 \ + --hash=sha256:1e680cab3cda6c3079ee64066903f93e421df46a8d4db63264a6359f3d5ba63e \ + --hash=sha256:33729ce100fc9d00d9820c81fc3ef9b95802b8d261f0cca0cce422bc568b1631 +certbot-apache==0.10.2 \ + --hash=sha256:c1f196506bf961e2171fb2af9feca8926100564b644acda1895acdf294c46e71 \ + --hash=sha256:8573275e74c8b74a9c7d7824636a37a4bc83327a7cc97a43b334fe8956ba8cb1 +certbot-nginx==0.10.2 \ + --hash=sha256:74c32bdaf2f3873aa811c3e2cdc205d3137b59c64dbcdca6f3982ed2a89b4f2a \ + --hash=sha256:c8265ab4b4bc0a6a3a81d7f0b8ff2606dd28ebbb3cd045635862286b97af2c36 From fe03c5a14c472f117eff039bd066e3019e131d53 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Thu, 26 Jan 2017 14:08:22 -0800 Subject: [PATCH 34/88] fix integration tests (#4111) --- tests/boulder-integration.sh | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/tests/boulder-integration.sh b/tests/boulder-integration.sh index 930cf1c0e..5a1f88c8a 100755 --- a/tests/boulder-integration.sh +++ b/tests/boulder-integration.sh @@ -47,23 +47,24 @@ common() { export HOOK_TEST="/tmp/hook$$" CheckHooks() { - COMMON="wtf2.auth\nwtf2.cleanup\nrenew\nrenew" EXPECTED="/tmp/expected$$" if [ $(head -n1 $HOOK_TEST) = "wtf.pre" ]; then echo "wtf.pre" > "$EXPECTED" echo "wtf2.pre" >> "$EXPECTED" - echo $COMMON >> "$EXPECTED" + echo "renew" >> "$EXPECTED" + echo "renew" >> "$EXPECTED" echo "wtf.post" >> "$EXPECTED" echo "wtf2.post" >> "$EXPECTED" else echo "wtf2.pre" > "$EXPECTED" echo "wtf.pre" >> "$EXPECTED" - echo $COMMON >> "$EXPECTED" + echo "renew" >> "$EXPECTED" + echo "renew" >> "$EXPECTED" echo "wtf2.post" >> "$EXPECTED" echo "wtf.post" >> "$EXPECTED" fi - if cmp --quiet "$EXPECTED" "$HOOK_TEST" ; then + if ! cmp --quiet "$EXPECTED" "$HOOK_TEST" ; then echo Hooks did not run as expected\; got cat "$HOOK_TEST" echo Expected @@ -91,8 +92,8 @@ common --domains le2.wtf --preferred-challenges http-01 run \ kill $python_server_pid common certonly -a manual -d le.wtf --rsa-key-size 4096 \ - --manual-auth-hook 'echo wtf2.auth >> "$HOOK_TEST" && ./tests/manual-http-auth.sh' \ - --manual-cleanup-hook 'echo wtf2.cleanup >> "$HOOK_TEST" && ./tests/manual-http-cleanup.sh' \ + --manual-auth-hook ./tests/manual-http-auth.sh \ + --manual-cleanup-hook ./tests/manual-http-cleanup.sh \ --pre-hook 'echo wtf2.pre >> "$HOOK_TEST"' \ --post-hook 'echo wtf2.post >> "$HOOK_TEST"' From bf7ce3130298016bdeb970e7c8e04cf319099969 Mon Sep 17 00:00:00 2001 From: Peter Conrad Date: Thu, 26 Jan 2017 15:53:48 -0800 Subject: [PATCH 35/88] Restructuring and editing using.rst to incorporate new command docs better. --- docs/using.rst | 283 +++++++++++++++++++++++++++---------------------- 1 file changed, 154 insertions(+), 129 deletions(-) diff --git a/docs/using.rst b/docs/using.rst index be8799858..a4d41dff3 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -22,7 +22,7 @@ Getting certificates (and choosing plugins) =========================================== The Certbot client supports two types of plugins for -obtaining and installing certificates. +obtaining and installing certificates: authenticators and installers. Authenticators are plugins used with the ``certonly`` command to obtain a cert. The authenticator validates that you @@ -60,7 +60,7 @@ manual_ Y N | Helps you obtain a cert by giving you instructions to pe | customized way. =========== ==== ==== =============================================================== ============================= -Under the hood, plugins use one of several ACME protocol "Challenges_" to +Under the hood, plugins use one of several ACME protocol challenges_ to prove you control a domain. The options are http-01_ (which uses port 80), tls-sni-01_ (port 443) and dns-01_ (requring configuration of a DNS server on port 53, though that's often not the same machine as your webserver). A few @@ -70,7 +70,7 @@ with ``--preferred-challenges``. There are also many third-party-plugins_ available. Below we describe in more detail the circumstances in which each plugin can be used, and how to use it. -.. _Challenges: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7 +.. _challenges: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7 .. _tls-sni-01: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7.3 .. _http-01: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7.2 .. _dns-01: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7.4 @@ -215,78 +215,148 @@ heroku_ Y Y Integration with Heroku SSL If you're interested, you can also :ref:`write your own plugin `. -Re-running Certbot -================== +.. _managing-certs: -Running Certbot with the ``certonly`` or ``run`` commands always requests -the creation of a single new certificate, even if you already have an -existing certificate with some of the same domain names. The ``--force-renewal``, -``--duplicate``, and ``--expand`` options control Certbot's behavior in this case. +Managing certificates +===================== + +To view a list of the certificates Certbot knows about, run +the ``certificates`` subcommand: + +``certbot certificates`` + +This returns information in the following format:: + + Found the following certs: + Certificate Name: example.com + Domains: example.com, www.example.com + Expiry Date: 2017-02-19 19:53:00+00:00 (VALID: 30 days) + Certificate Path: /etc/letsencrypt/live/example.com/fullchain.pem + Private Key Path: /etc/letsencrypt/live/example.com/privkey.pem + +``Certificate Name`` shows the name of the certificate. Pass this name +using the ``--cert-name`` flag to specify a particular certificate for the ``run``, +``certonly``, ``certificates``, ``renew`` and ``delete`` commands. Example:: + + certbot certonly --cert-name example.com + + +Re-creating and Updating Existing Certificates +---------------------------------------------- + +You can use ``certonly`` or ``run`` subcommands to request +the creation of a single new certificate even if you already have an +existing certificate with some of the same domain names. + +If a certificate is requested with ``run`` or ``certonly`` specifying a +``Certificate name`` that already exists, the new certificate replaces +the existing certificate. Otherwise a new certificate +is created and assigned the specified name. + +The ``--force-renewal``, ``--duplicate``, and ``--expand`` options +(introduced in version 0.10.0) control Certbot's behavior when re-creating +a certificate with the same name as an existing certificate. If you don't specify a requested behavior, Certbot may ask you what you intended. + ``--force-renewal`` tells Certbot to request a new certificate -with the same domains as an existing certificate. (Each and every domain -must be explicitly specified via ``-d``.) If successful, this certificate -will be saved alongside the earlier one and symbolic links (the "``live``" +with the same domains as an existing certificate. Each domain +must be explicitly specified via ``-d``. If successful, this certificate +is saved alongside the earlier one and symbolic links (the "``live``" reference) will be updated to point to the new certificate. This is a -valid method of explicitly requesting the renewal of a specific individual +valid method of renewing a specific individual certificate. ``--duplicate`` tells Certbot to create a separate, unrelated certificate -with the same domains as an existing certificate. This certificate will -be saved completely separately from the prior one. Most users probably -do not want this behavior. +with the same domains as an existing certificate. This certificate is +saved completely separately from the prior one. Most users will not +need to issue this command in normal circumstances. ``--expand`` tells Certbot to update an existing certificate with a new certificate that contains all of the old domains and one or more additional new domains. -``--allow-subset-of-names`` tells Certbot to continue with cert generation if +``--allow-subset-of-names`` tells Certbot to continue with certificate generation if only some of the specified domain authorizations can be obtained. This may be useful if some domains specified in a certificate no longer point at this system. Whenever you obtain a new certificate in any of these ways, the new -certificate exists alongside any previously-obtained certificates, whether +certificate exists alongside any previously obtained certificates, whether or not the previous certificates have expired. The generation of a new certificate counts against several rate limits that are intended to prevent abuse of the ACME protocol, as described `here `__. +Changing a Certificate’s Domains +```````````````````````````````` + +The ``--cert-name`` flag can also be used to modify the domains a certificate contains, +by specifying new domains using the ``-d`` or ``—domains`` flag. If certificate ``example.com`` +previously contained ``example.com`` and ``www.example.com``, it can be modified to only +contain ``example.com`` by specifying only ``example.com`` with the ``-d`` or ``—domains`` flag. Example:: + + certbot certonly --cert-name example.com -d example.com + +The same format can be used to expand the set of domains a certificate contains, or to +replace that set entirely:: + + certbot certonly --cert-name example.com -d example.org,www.example.org + + +Revoking certificates +--------------------- + +If your account key has been compromised or you otherwise need to revoke a certificate, +use the ``revoke`` command to do so. Note that the ``revoke`` command takes the certificate path +(ending in ``cert.pem``), not a certificate name or domain. Example:: + + certbot revoke --cert-path /etc/letsencrypt/live/CERTNAME/cert.pem + +Additionally, if a certificate +is a test cert obtained via the ``--staging`` or ``—test-cert`` flag, that flag must be passed to the +``revoke`` subcommand. +Once a certificate is revoked (or for other cert management tasks), all of a certificate's +relevant files can be removed from the system with the ``delete`` subcommand:: + + certbot delete --cert-name example.com + +.. note:: If you don’t use ``delete`` to remove the certificate completely, it will be renewed automatically at the next renewal event. + .. _renewal: Renewing certificates -===================== +--------------------- .. note:: Let's Encrypt CA issues short-lived certificates (90 days). Make sure you renew the certificates at least once in 3 months. -The ``certbot`` client now supports a ``renew`` action to check +As of version 0.10.0, ``certbot`` client supports a ``renew`` action to check all installed certificates for impending expiry and attempt to renew them. The simplest form is simply ``certbot renew`` -This will attempt to renew any previously-obtained certificates that +This command attempts to renew any previously-obtained certificates that expire in less than 30 days. The same plugin and options that were used at the time the certificate was originally issued will be used for the renewal attempt, unless you specify other plugins or options. Unlike ``certonly``, ``renew`` acts on multiple certificates and always takes into account whether each one is near expiry. Because of this, ``renew`` is suitable (and designed) for automated use, to allow your system to automatically renew each certificate when appropriate. -Since ``renew`` will only renew certificates that are near expiry it can be +Since ``renew`` only renews certificates that are near expiry it can be run as frequently as you want - since it will usually take no action. -You can also specify hooks to be run before or after a certificate is -renewed. For example, if you have only a single cert and you obtained it using -the standalone_ plugin, it will be used by default when renewing. In that case -you may want to use a command like this to renew your certificate. +The ``renew`` command includes hooks for running commands or scripts before or after a certificate is +renewed. For example, if you have a single cert obtained using +the standalone_ plugin, you might need to stop the webserver +before renewing so standalone can bind to the necessary ports, and +then restart it after the plugin is finished. Example:: -``certbot renew --pre-hook "service nginx stop" --post-hook "service nginx start"`` + certbot renew --pre-hook "service nginx stop" --post-hook "service nginx start" -This will stop Nginx so standalone can bind to the necessary ports and -then restart Nginx after the plugin is finished. The hooks will only be +The hooks will only be run if a certificate is due for renewal, so you can run this command frequently without unnecessarily stopping your webserver. More information about renewal hooks can be found by running @@ -324,30 +394,61 @@ user input (which is useful when running the command from cron). ``certbot certonly -n -d example.com -d www.example.com`` -(All of the domains covered by the certificate must be specified in +All of the domains covered by the certificate must be specified in this case in order to renew and replace the old certificate rather than obtaining a new one; don't forget any `www.` domains! Specifying a subset of the domains creates a new, separate certificate containing -only those domains, rather than replacing the original certificate.) +only those domains, rather than replacing the original certificate. When run with a set of domains corresponding to an existing certificate, -the ``certonly`` command attempts to renew that one individual certificate. +the ``certonly`` command attempts to renew that specific certificate. Please note that the CA will send notification emails to the address you provide if you do not renew certificates that are about to expire. -Certbot is working hard on improving the renewal process, and we -apologize for any inconveniences you encounter in integrating these +Certbot is working hard to improve the renewal process, and we +apologize for any inconvenience you encounter in integrating these commands into your individual environment. -.. _command-line: +Modifying the Renewal Configuration File +```````````````````````````````````````` -Certbot command-line options -============================ +For advanced certificate management tasks, it is possible to manually modify the certificate's +renewal configuration file, located at ``/etc/letsencrypt/renewal/CERTNAME``. + +.. warning:: Modifying any files in ``/etc/letsencrypt`` can damage it so Certbot can no longer properly manage its certificates, and we do not recommend doing so. + +For most tasks, it is safest to limit yourself to pointing symlinks at the files there, or using +``—renew-hook`` to copy / make new files based upon those files, if your operational situation requires it +(for instance, combining certs and keys in different way, or having copies of things with different +specific permissions that are demanded by other programs). + +If the contents of ``/etc/letsencrypt/archive/CERTNAME`` are moved to a new folder, first specify +the new folder's name in the renewal configuration file, then run ``certbot update_symlinks`` to +point the symlinks in ``/etc/letsencrypt/live/CERTNAME`` to the new folder. + +If you would like the live certificate files whose symlink location Certbot updates on each run to +reside in a different location, first move them to that location, then specify the full path of +each of the four files in the renewal configuration file. Since the symlinks are relative links, +you must follow this with an invocation of ``certbot update_symlinks``. + +For example, say that a certificate's renewal configuration file previously contained the following +directives:: + + archive_dir = /etc/letsencrypt/archive/example.com + cert = /etc/letsencrypt/live/example.com/cert.pem + privkey = /etc/letsencrypt/live/example.com/privkey.pem + chain = /etc/letsencrypt/live/example.com/chain.pem + fullchain = /etc/letsencrypt/live/example.com/fullchain.pem + +The following commands could be used to specify where these files are located:: + + mv /etc/letsencrypt/archive/example.com /home/user/me/certbot/example_archive + sed -i 's,/etc/letsencrypt/archive/example.com,/home/user/me/certbot/example_archive,' /etc/letsencrypt/renewal/example.com.conf + mv /etc/letsencrypt/live/example.com/*.pem /home/user/me/certbot/ + sed -i 's,/etc/letsencrypt/live/example.com,/home/user/me/certbot,g' /etc/letsencrypt/renewal/example.com.conf + certbot update_symlinks -Certbot supports a lot of command line options. Here's the full list, from -``certbot --help all``: -.. literalinclude:: cli-help.txt .. _where-certs: @@ -419,16 +520,16 @@ The following files are available: Pre and Post Validation Hooks ============================= -Certbot allows for the specification fo pre and post validation hooks when run +Certbot allows for the specification of pre and post validation hooks when run in manual mode. The flags to specify these scripts are ``--manual-auth-hook`` -and ``--manual-cleanup-hook`` respectively and can be used as such: +and ``--manual-cleanup-hook`` respectively and can be used as follows: :: certbot certonly --manual --manual-auth-hook /path/to/http/authenticator.sh --manual-cleanup-hook /path/to/http/cleanup.sh -d secure.example.com -This will run the authenticator.sh script, attempt the validation, and then run -the cleanup.sh script. Additionally certbot will pass three environment +This will run the ``authenticator.sh`` script, attempt the validation, and then run +the ``cleanup.sh`` script. Additionally certbot will pass three environment variables to these scripts: - ``CERTBOT_DOMAIN``: The domain being authenticated @@ -534,91 +635,6 @@ Example usage for DNS-01 (Cloudflare API v4) (for example purposes only, do not fi -.. _managing-certs: - -Managing certificates -===================== - -To view a list of the certificates Certbot knows about, run -the ``certificates`` subcommand: - -``certbot certifices`` - -This will return information in the following format:: - - Found the following certs: - Certificate Name: example.com - Domains: example.com, www.example.com - Expiry Date: 2017-02-19 19:53:00+00:00 (VALID: 30 days) - Certificate Path: /etc/letsencrypt/live/example.com/fullchain.pem - Private Key Path: /etc/letsencrypt/live/example.com/privkey.pem - -``Certificate Name`` gives the name Certbot knows the certificate by. Pass this name -to the ``--cert-name`` flag to specify a particular certificate for the ``run``, -``certonly``, ``certificates``, ``renew`` and ``delete`` commands:: - - certbot certonly --cert-name example.com - -The ``--cert-name`` flag can also be used to modify the domains a certificate contains, -by specifying new domains using the ``-d/--domains`` flag. If certificate ``example.com`` -previously contained ``example.com`` and ``www.example.com``, it can be modified to only -contain ``example.com`` by specifying only ``example.com`` with the ``-d/--domains`` flag:: - - certbot certonly --cert-name example.com -d example.com - -The same format can be used to expand the set of domains a certificate contains, or to -replace that set entirely:: - - certbot certonly --cert-name example.com -d example.org,www.example.org - -If a certificate is requested with ``run`` or ``certonly`` with a name that does not already -exist, the new certificate created will be assigned the name specified. - -If your account key has been compromised or you otherwise need to revoke a certificate, -use the revoke command to do so. Note that the revoke command is passed the certificate path -(ending in ``cert.pem``), not a certificate name or domain. Additionally, if a certificate -is a test cert obtained via the ``--staging/--test-cert`` flag, that flag must be passed to the -``revoke`` subcommand:: - - certbot revoke --cert-path /etc/letsencrypt/live/CERTNAME/cert.pem - -Once a certificate is revoked (or for other cert management tasks), all of a certificate's -relevant files can be removed from the system with the ``delete`` subcommand:: - - certbot delete --cert-name example.com - -For advanced certificate management tasks, it is possible to manually modify the certificate's -renewal configuration file, located at ``/etc/letsencrypt/renewal/CERTNAME``. - -.. warning:: Modifying any files in ``/etc/letsencrypt`` can make it so Certbot can no longer - properly manage its certificates, and we do not recommend doing so for most users. - -If the contents of ``/etc/letsencrypt/archive/CERTNAME`` are moved to a new folder, first specify -the new folder's name in the renewal configuration file, then run ``certbot update_symlinks`` to -point the symlinks in ``/etc/letsencrypt/live/CERTNAME`` to the new folder. - -If you would like the live certificate files whose symlink location Certbot updates on each run to -reside in a different location, first move them to that location, then specify the full path of -each of the four files in the renewal configuration file. Since the symlinks are relative links, -you must follow this with an invocation of ``certbot update_symlinks``. - -For example, say that a certificate's renewal configuration file previously contained the following -directives:: - - archive_dir = /etc/letsencrypt/archive/example.com - cert = /etc/letsencrypt/live/example.com/cert.pem - privkey = /etc/letsencrypt/live/example.com/privkey.pem - chain = /etc/letsencrypt/live/example.com/chain.pem - fullchain = /etc/letsencrypt/live/example.com/fullchain.pem - -The following commands could be used to specify where these files are located:: - - mv /etc/letsencrypt/archive/example.com /home/user/me/certbot/example_archive - sed -i 's,/etc/letsencrypt/archive/example.com,/home/user/me/certbot/example_archive,' /etc/letsencrypt/renewal/example.com.conf - mv /etc/letsencrypt/live/example.com/*.pem /home/user/me/certbot/ - sed -i 's,/etc/letsencrypt/live/example.com,/home/user/me/certbot,g' /etc/letsencrypt/renewal/example.com.conf - certbot update_symlinks - .. _config-file: @@ -641,6 +657,15 @@ By default, the following locations are searched: .. keep it up to date with constants.py +.. _command-line: + +Certbot command-line options +============================ + +Certbot supports a lot of command line options. Here's the full list, from +``certbot --help all``: + +.. literalinclude:: cli-help.txt Getting help ============ From 6a39a42f45f3b5c7e497b5f4296ed33d10f5c4d2 Mon Sep 17 00:00:00 2001 From: Josh Soref Date: Thu, 26 Jan 2017 19:21:54 -0500 Subject: [PATCH 36/88] Spelling (#4101) * spelling: action * spelling: artifacts * spelling: asymmetric * spelling: attempts * spelling: bizarre * spelling: certbot * spelling: certificate * spelling: certainly * spelling: challenge * spelling: client * spelling: collated * spelling: correct * spelling: considered * spelling: custom * spelling: distinguish * spelling: encoding * spelling: enhancement * spelling: equivalent * spelling: execution * spelling: existence * spelling: failed * spelling: handshake * spelling: hyphen * spelling: initialized * spelling: initialization * spelling: interpretation * spelling: letsencrypt * spelling: multiline * spelling: multipart * spelling: necessary * spelling: otherwise * spelling: output * spelling: overridden * spelling: positives * spelling: preferable * spelling: progress * spelling: recommended * spelling: referring * spelling: relativity * spelling: request * spelling: requiring * spelling: separate * spelling: source * spelling: specified * spelling: standard * spelling: successfully * spelling: unparseable * spelling: useful --- acme/acme/challenges.py | 2 +- acme/acme/client_test.py | 4 ++-- acme/acme/crypto_util_test.py | 2 +- acme/acme/jose/__init__.py | 2 +- acme/acme/jose/json_util.py | 2 +- acme/acme/jose/jwk.py | 2 +- acme/acme/messages_test.py | 2 +- certbot-apache/certbot_apache/obj.py | 2 +- .../passing/comment-continuations-2050.conf | 4 ++-- certbot-apache/certbot_apache/tests/parser_test.py | 2 +- certbot-apache/certbot_apache/tls_sni_01.py | 2 +- .../certbot_compatibility_test/test_driver.py | 4 ++-- .../chive/chive-nginx-master/nginx.conf | 2 +- .../chive/chive-nginx-master/win-utf | 2 +- .../guide-to-nginx-ssl-spdy-hsts/nginx.conf | 6 +++--- .../nginx/nginx-roundtrip-testdata/phplist/nginx.conf | 2 +- certbot-nginx/certbot_nginx/nginxparser.py | 6 +++--- .../default_vhost/nginx/naxsi_core.rules | 2 +- .../ubuntu_nginx_1_4_6/default_vhost/nginx/win-utf | 2 +- certbot/achallenges.py | 2 +- certbot/cli.py | 8 ++++---- certbot/client.py | 6 +++--- certbot/main.py | 2 +- certbot/plugins/disco.py | 4 ++-- certbot/plugins/disco_test.py | 2 +- certbot/plugins/selection_test.py | 2 +- certbot/storage.py | 2 +- certbot/tests/cli_test.py | 2 +- certbot/tests/error_handler_test.py | 2 +- certbot/tests/errors_test.py | 2 +- certbot/tests/reverter_test.py | 2 +- docs/ciphers.rst | 2 +- docs/cli-help.txt | 2 +- docs/contributing.rst | 2 +- docs/using.rst | 6 +++--- letsencrypt-auto-source/letsencrypt-auto | 4 ++-- letsencrypt-auto-source/letsencrypt-auto.template | 4 ++-- tests/boulder-integration.sh | 2 +- .../scripts/test_letsencrypt_auto_certonly_standalone.sh | 2 +- tests/letstest/scripts/test_ocsp_experimental.sh | 2 +- tools/_venv_common.sh | 2 +- tools/release.sh | 2 +- 42 files changed, 59 insertions(+), 59 deletions(-) diff --git a/acme/acme/challenges.py b/acme/acme/challenges.py index 9f9cc05b8..3bfc1e62f 100644 --- a/acme/acme/challenges.py +++ b/acme/acme/challenges.py @@ -183,7 +183,7 @@ class KeyAuthorizationChallenge(_TokenChallenge): Subclasses must implement this method, but they are likely to return completely different data structures, depending on what's - necessary to complete the challenge. Interepretation of that + necessary to complete the challenge. Interpretation of that return value must be known to the caller. :param JWK account_key: diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index e88baa340..d67c425bd 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -170,7 +170,7 @@ class ClientTest(unittest.TestCase): self.directory.new_authz, messages.NewAuthorization(identifier=self.identifier)) - def test_requets_challenges_custom_uri(self): + def test_request_challenges_custom_uri(self): self._prepare_response_for_request_challenges() self.client.request_challenges(self.identifier, 'URI') self.net.post.assert_called_once_with('URI', mock.ANY) @@ -388,7 +388,7 @@ class ClientTest(unittest.TestCase): errors.PollError, self.client.poll_and_request_issuance, csr, authzrs=(invalid_authzr,), mintime=mintime) - # exceeded max_attemps | TODO: move to a separate test + # exceeded max_attempts | TODO: move to a separate test self.assertRaises( errors.PollError, self.client.poll_and_request_issuance, csr, authzrs, mintime=mintime, max_attempts=2) diff --git a/acme/acme/crypto_util_test.py b/acme/acme/crypto_util_test.py index bd93ae0e1..ebb4010a6 100644 --- a/acme/acme/crypto_util_test.py +++ b/acme/acme/crypto_util_test.py @@ -59,7 +59,7 @@ class SSLSocketAndProbeSNITest(unittest.TestCase): def test_probe_not_recognized_name(self): self.assertRaises(errors.Error, self._probe, b'bar') - # TODO: py33/py34 tox hangs forever on do_hendshake in second probe + # TODO: py33/py34 tox hangs forever on do_handshake in second probe #def probe_connection_error(self): # self._probe(b'foo') # #time.sleep(1) # TODO: avoid race conditions in other way diff --git a/acme/acme/jose/__init__.py b/acme/acme/jose/__init__.py index f39c3beab..9116bc433 100644 --- a/acme/acme/jose/__init__.py +++ b/acme/acme/jose/__init__.py @@ -1,6 +1,6 @@ """Javascript Object Signing and Encryption (jose). -This package is a Python implementation of the stadards developed by +This package is a Python implementation of the standards developed by IETF `Javascript Object Signing and Encryption (Active WG)`_, in particular the following RFCs: diff --git a/acme/acme/jose/json_util.py b/acme/acme/jose/json_util.py index cc66d77ff..d474f4aac 100644 --- a/acme/acme/jose/json_util.py +++ b/acme/acme/jose/json_util.py @@ -60,7 +60,7 @@ class Field(object): @classmethod def _empty(cls, value): - """Is the provided value cosidered "empty" for this field? + """Is the provided value considered "empty" for this field? This is useful for subclasses that might want to override the definition of being empty, e.g. for some more exotic data types. diff --git a/acme/acme/jose/jwk.py b/acme/acme/jose/jwk.py index 4d07229b3..5b6965c4d 100644 --- a/acme/acme/jose/jwk.py +++ b/acme/acme/jose/jwk.py @@ -111,7 +111,7 @@ class JWK(json_util.TypedJSONObjectWithFields): try: key = cls._load_cryptography_key(data, password, backend) except errors.Error as error: - logger.debug('Loading symmetric key, assymentric failed: %s', error) + logger.debug('Loading symmetric key, asymmetric failed: %s', error) return JWKOct(key=data) if cls.typ is not NotImplemented and not isinstance( diff --git a/acme/acme/messages_test.py b/acme/acme/messages_test.py index a0322968c..b3454f25b 100644 --- a/acme/acme/messages_test.py +++ b/acme/acme/messages_test.py @@ -26,7 +26,7 @@ class ErrorTest(unittest.TestCase): 'type': ERROR_PREFIX + 'malformed', } self.error_custom = Error(typ='custom', detail='bar') - self.jobj_cusom = {'type': 'custom', 'detail': 'bar'} + self.jobj_custom = {'type': 'custom', 'detail': 'bar'} def test_default_typ(self): from acme.messages import Error diff --git a/certbot-apache/certbot_apache/obj.py b/certbot-apache/certbot_apache/obj.py index c71443e92..b29b0e0ee 100644 --- a/certbot-apache/certbot_apache/obj.py +++ b/certbot-apache/certbot_apache/obj.py @@ -8,7 +8,7 @@ class Addr(common.Addr): """Represents an Apache address.""" def __eq__(self, other): - """This is defined as equalivalent within Apache. + """This is defined as equivalent within Apache. ip_addr:* == ip_addr diff --git a/certbot-apache/certbot_apache/tests/apache-conf-files/passing/comment-continuations-2050.conf b/certbot-apache/certbot_apache/tests/apache-conf-files/passing/comment-continuations-2050.conf index 48b344d8a..4c3fa2af1 100644 --- a/certbot-apache/certbot_apache/tests/apache-conf-files/passing/comment-continuations-2050.conf +++ b/certbot-apache/certbot_apache/tests/apache-conf-files/passing/comment-continuations-2050.conf @@ -263,7 +263,7 @@ # # Set the following policy settings here and they will be propagated to the 30 rules # file (modsecurity_crs_30_http_policy.conf) by using macro expansion. -# If you run into false positves, you can adjust the settings here. +# If you run into false positives, you can adjust the settings here. # #SecAction \ "id:'900012', \ @@ -349,7 +349,7 @@ # -# -- [[ Check UTF enconding ]] ----------------------------------------------------------- +# -- [[ Check UTF encoding ]] ----------------------------------------------------------- # # We only want to apply this check if UTF-8 encoding is actually used by the site, otherwise # it will result in false positives. diff --git a/certbot-apache/certbot_apache/tests/parser_test.py b/certbot-apache/certbot_apache/tests/parser_test.py index 759ae1265..513f16c4e 100644 --- a/certbot-apache/certbot_apache/tests/parser_test.py +++ b/certbot-apache/certbot_apache/tests/parser_test.py @@ -178,7 +178,7 @@ class ParserInitTest(util.ApacheTest): shutil.rmtree(self.work_dir) @mock.patch("certbot_apache.parser.ApacheParser._get_runtime_cfg") - def test_unparsable(self, mock_cfg): + def test_unparseable(self, mock_cfg): from certbot_apache.parser import ApacheParser mock_cfg.return_value = ('Define: TEST') self.assertRaises( diff --git a/certbot-apache/certbot_apache/tls_sni_01.py b/certbot-apache/certbot_apache/tls_sni_01.py index 18179d080..d9e294119 100644 --- a/certbot-apache/certbot_apache/tls_sni_01.py +++ b/certbot-apache/certbot_apache/tls_sni_01.py @@ -177,7 +177,7 @@ class ApacheTlsSni01(common.TLSSNI01): ips = " ".join(str(i) for i in ip_addrs) document_root = os.path.join( self.configurator.config.work_dir, "tls_sni_01_page/") - # TODO: Python docs is not clear how mutliline string literal + # TODO: Python docs is not clear how multiline string literal # newlines are parsed on different platforms. At least on # Linux (Debian sid), when source file uses CRLF, Python still # parses it as "\n"... c.f.: diff --git a/certbot-compatibility-test/certbot_compatibility_test/test_driver.py b/certbot-compatibility-test/certbot_compatibility_test/test_driver.py index b5e023f36..71100bb27 100644 --- a/certbot-compatibility-test/certbot_compatibility_test/test_driver.py +++ b/certbot-compatibility-test/certbot_compatibility_test/test_driver.py @@ -147,7 +147,7 @@ def test_deploy_cert(plugin, temp_dir, domains): plugin.deploy_cert(domain, cert_path, util.KEY_PATH, cert_path, cert_path) plugin.save() # Needed by the Apache plugin except le_errors.Error as error: - logger.error("Plugin failed to deploy ceritificate for %s:", domain) + logger.error("Plugin failed to deploy certificate for %s:", domain) logger.exception(error) return False @@ -202,7 +202,7 @@ def test_enhancements(plugin, domains): success = False if success: - logger.info("Enhancments test succeeded") + logger.info("Enhancements test succeeded") return success diff --git a/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/chive/chive-nginx-master/nginx.conf b/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/chive/chive-nginx-master/nginx.conf index 22ad4c317..bc708783f 100644 --- a/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/chive/chive-nginx-master/nginx.conf +++ b/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/chive/chive-nginx-master/nginx.conf @@ -38,7 +38,7 @@ http { ## Define a zone for limiting the number of simultaneous ## connections nginx accepts. 1m means 32000 simultaneous ## sessions. We need to define for each server the limit_conn - ## value refering to this or other zones. + ## value referring to this or other zones. ## ** This syntax requires nginx version >= ## ** 1.1.8. Cf. http://nginx.org/en/CHANGES. If using an older ## ** version then use the limit_zone directive below diff --git a/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/chive/chive-nginx-master/win-utf b/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/chive/chive-nginx-master/win-utf index ed8bc007a..d0b7116c8 100644 --- a/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/chive/chive-nginx-master/win-utf +++ b/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/chive/chive-nginx-master/win-utf @@ -37,7 +37,7 @@ charset_map windows-1251 utf-8 { AA D084 ; # capital Ukrainian YE AB C2AB ; # left-pointing double angle quotation mark AC C2AC ; # not sign - AD C2AD ; # soft hypen + AD C2AD ; # soft hyphen AE C2AE ; # (R) AF D087 ; # capital Ukrainian YI diff --git a/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/guide-to-nginx-ssl-spdy-hsts/nginx.conf b/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/guide-to-nginx-ssl-spdy-hsts/nginx.conf index f195b4d21..55e2dab0d 100644 --- a/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/guide-to-nginx-ssl-spdy-hsts/nginx.conf +++ b/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/guide-to-nginx-ssl-spdy-hsts/nginx.conf @@ -8,7 +8,7 @@ http { keepalive_timeout 60; # http-redirects to https; even if using of hsts; - # usefull if users are typing your server-name w/out https:// + # useful if users are typing your server-name w/out https:// # logjam and a good idea anyway @@ -98,7 +98,7 @@ http { #ssl_ciphers ALL:!ADH:!EXP:!LOW:!RC2:!3DES:!SEED:!RC4:+HIGH:+MEDIUM; # - # suggestions by mozilla-server-team - good compatibility, pfs, preferrable ciphers + # suggestions by mozilla-server-team - good compatibility, pfs, preferable ciphers # # modern ciphers ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK'; @@ -106,7 +106,7 @@ http { # intermediate ciphers #ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA'; - # old ciphers (would need SSLv3, but is not recommende as of oct 2014 + # old ciphers (would need SSLv3, but is not recommended as of oct 2014 #ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA'; # logjam / cipher suggested from weakdh.org diff --git a/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/phplist/nginx.conf b/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/phplist/nginx.conf index 9cf532809..357100558 100644 --- a/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/phplist/nginx.conf +++ b/certbot-compatibility-test/nginx/nginx-roundtrip-testdata/phplist/nginx.conf @@ -29,7 +29,7 @@ server { location ~* (index\.php|upload\.php|connector\.php|dl\.php|ut\.php|lt\.php|download\.php)$ { fastcgi_split_path_info ^(.|\.php)(/.+)$; - include /etc/nginx/fastcgi_params.conf; #standar fastcgi config file + include /etc/nginx/fastcgi_params.conf; #standard fastcgi config file fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_intercept_errors on; fastcgi_pass 127.0.0.1:9000; diff --git a/certbot-nginx/certbot_nginx/nginxparser.py b/certbot-nginx/certbot_nginx/nginxparser.py index 2cbd0b86a..f6437c589 100644 --- a/certbot-nginx/certbot_nginx/nginxparser.py +++ b/certbot-nginx/certbot_nginx/nginxparser.py @@ -52,10 +52,10 @@ class RawNginxParser(object): map_statement = space + Literal("map") + space + nonspace + space + dollar_var + space # This is NOT an accurate way to parse nginx map entries; it's almost - # certianly too permissive and may be wrong in other ways, but it should + # certainly too permissive and may be wrong in other ways, but it should # preserve things correctly in mmmmost or all cases. # - # - I can neither prove nor disprove that it is corect wrt all escaped + # - I can neither prove nor disprove that it is correct wrt all escaped # semicolon situations # Addresses https://github.com/fatiherikli/nginxparser/issues/19 map_pattern = Regex(r'".*"') | Regex(r"'.*'") | nonspace @@ -143,7 +143,7 @@ class RawNginxDumper(object): def loads(source): """Parses from a string. - :param str souce: The string to parse + :param str source: The string to parse :returns: The parsed tree :rtype: list diff --git a/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/ubuntu_nginx_1_4_6/default_vhost/nginx/naxsi_core.rules b/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/ubuntu_nginx_1_4_6/default_vhost/nginx/naxsi_core.rules index c9220209f..9826e02cb 100644 --- a/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/ubuntu_nginx_1_4_6/default_vhost/nginx/naxsi_core.rules +++ b/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/ubuntu_nginx_1_4_6/default_vhost/nginx/naxsi_core.rules @@ -67,7 +67,7 @@ MainRule "rx:%[2|3]." "msg:double encoding !" "mz:ARGS|URL|BODY|$HEADERS_VAR:Co #################################### MainRule "str:&#" "msg: utf7/8 encoding" "mz:ARGS|BODY|URL|$HEADERS_VAR:Cookie" "s:$EVADE:4" id:1400; MainRule "str:%U" "msg: M$ encoding" "mz:ARGS|BODY|URL|$HEADERS_VAR:Cookie" "s:$EVADE:4" id:1401; -MainRule negative "rx:multipart/form-data|application/x-www-form-urlencoded" "msg:Content is neither mulipart/x-www-form.." "mz:$HEADERS_VAR:Content-type" "s:$EVADE:4" id:1402; +MainRule negative "rx:multipart/form-data|application/x-www-form-urlencoded" "msg:Content is neither multipart/x-www-form.." "mz:$HEADERS_VAR:Content-type" "s:$EVADE:4" id:1402; ############################# ## File uploads: 1500-1600 ## diff --git a/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/ubuntu_nginx_1_4_6/default_vhost/nginx/win-utf b/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/ubuntu_nginx_1_4_6/default_vhost/nginx/win-utf index 774fd9fc9..cd2885292 100644 --- a/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/ubuntu_nginx_1_4_6/default_vhost/nginx/win-utf +++ b/certbot-nginx/certbot_nginx/tests/testdata/etc_nginx/ubuntu_nginx_1_4_6/default_vhost/nginx/win-utf @@ -36,7 +36,7 @@ charset_map windows-1251 utf-8 { AA D084; # capital Ukrainian YE AB C2AB; # left-pointing double angle quotation mark AC C2AC; # not sign - AD C2AD; # soft hypen + AD C2AD; # soft hyphen AE C2AE; # (R) AF D087; # capital Ukrainian YI diff --git a/certbot/achallenges.py b/certbot/achallenges.py index 5ee6d2945..c2af45fdb 100644 --- a/certbot/achallenges.py +++ b/certbot/achallenges.py @@ -1,6 +1,6 @@ """Client annotated ACME challenges. -Please use names such as ``achall`` to distiguish from variables "of type" +Please use names such as ``achall`` to distinguish from variables "of type" :class:`acme.challenges.Challenge` (denoted by ``chall``) and :class:`.ChallengeBody` (denoted by ``challb``):: diff --git a/certbot/cli.py b/certbot/cli.py index 88d6d39db..f3b07d925 100644 --- a/certbot/cli.py +++ b/certbot/cli.py @@ -150,7 +150,7 @@ def possible_deprecation_warning(config): if cli_command != LEAUTO: return if config.no_self_upgrade: - # users setting --no-self-upgrade might be hanging on a clent version like 0.3.0 + # users setting --no-self-upgrade might be hanging on a client version like 0.3.0 # or 0.5.0 which is the new script, but doesn't set CERTBOT_AUTO; they don't # need warnings return @@ -318,7 +318,7 @@ class CustomHelpFormatter(argparse.HelpFormatter): # The attributes here are: # short: a string that will be displayed by "certbot -h commands" # opts: a string that heads the section of flags with which this command is documented, -# both for "cerbot -h SUBCOMMAND" and "certbot -h all" +# both for "certbot -h SUBCOMMAND" and "certbot -h all" # usage: an optional string that overrides the header of "certbot -h SUBCOMMAND" VERB_HELP = [ ("run (default)", { @@ -334,7 +334,7 @@ VERB_HELP = [ "This command obtains a TLS/SSL certificate without installing it anywhere.") }), ("renew", { - "short": "Renew all certificates (or one specifed with --cert-name)", + "short": "Renew all certificates (or one specified with --cert-name)", "opts": ("The 'renew' subcommand will attempt to renew all" " certificates (or more precisely, certificate lineages) you have" " previously obtained if they are close to expiry, and print a" @@ -496,7 +496,7 @@ class HelpfulArgumentParser(object): if "apache" in plugins: apache_doc = "--apache Use the Apache plugin for authentication & installation" else: - apache_doc = "(the cerbot apache plugin is not installed)" + apache_doc = "(the certbot apache plugin is not installed)" usage = SHORT_USAGE if help_arg == True: diff --git a/certbot/client.py b/certbot/client.py index f76688b70..8d53a29d3 100644 --- a/certbot/client.py +++ b/certbot/client.py @@ -93,7 +93,7 @@ def register(config, account_storage, tos_cb=None): Terms of Service present in the contained `.Registration.terms_of_service` is accepted by the client, and ``False`` otherwise. ``tos_cb`` will be called only if the - client acction is necessary, i.e. when ``terms_of_service is not + client action is necessary, i.e. when ``terms_of_service is not None``. This argument is optional, if not supplied it will default to automatic acceptance! @@ -438,7 +438,7 @@ class Client(object): self.installer.restart() def apply_enhancement(self, domains, enhancement, options=None): - """Applies an enhacement on all domains. + """Applies an enhancement on all domains. :param domains: list of ssl_vhosts :type list of str @@ -494,7 +494,7 @@ class Client(object): self.installer.rollback_checkpoints() self.installer.restart() except: - # TODO: suggest letshelp-letsencypt here + # TODO: suggest letshelp-letsencrypt here reporter.add_message( "An error occurred and we failed to restore your config and " "restart your server. Please submit a bug report to " diff --git a/certbot/main.py b/certbot/main.py index f8cb411b5..b52e605be 100644 --- a/certbot/main.py +++ b/certbot/main.py @@ -812,7 +812,7 @@ def make_or_verify_core_dir(directory, mode, uid, strict): raise errors.Error(_PERM_ERR_FMT.format(error)) def make_or_verify_needed_dirs(config): - """Create or verify existance of config, work, or logs directories""" + """Create or verify existence of config, work, or logs directories""" make_or_verify_core_dir(config.config_dir, constants.CONFIG_DIRS_MODE, os.geteuid(), config.strict_permissions) make_or_verify_core_dir(config.work_dir, constants.CONFIG_DIRS_MODE, diff --git a/certbot/plugins/disco.py b/certbot/plugins/disco.py index ba532eb1b..e567422e2 100644 --- a/certbot/plugins/disco.py +++ b/certbot/plugins/disco.py @@ -79,7 +79,7 @@ class PluginEntryPoint(object): return self._initialized is not None def init(self, config=None): - """Memoized plugin inititialization.""" + """Memoized plugin initialization.""" if not self.initialized: self.entry_point.require() # fetch extras! self._initialized = self.plugin_cls(config, self.name) @@ -230,7 +230,7 @@ class PluginsRegistry(collections.Mapping): def available(self): """Filter plugins based on availability.""" return self.filter(lambda p_ep: p_ep.available) - # succefully prepared + misconfigured + # successfully prepared + misconfigured def find_init(self, plugin): """Find an initialized plugin. diff --git a/certbot/plugins/disco_test.py b/certbot/plugins/disco_test.py index 7282c9ec8..6c3c39dca 100644 --- a/certbot/plugins/disco_test.py +++ b/certbot/plugins/disco_test.py @@ -255,7 +255,7 @@ class PluginsRegistryTest(unittest.TestCase): def test_find_init(self): self.assertTrue(self.reg.find_init(mock.Mock()) is None) - self.plugin_ep.initalized = True + self.plugin_ep.initialized = True self.assertTrue( self.reg.find_init(self.plugin_ep.init()) is self.plugin_ep) diff --git a/certbot/plugins/selection_test.py b/certbot/plugins/selection_test.py index eb4db2081..41c2b55c9 100644 --- a/certbot/plugins/selection_test.py +++ b/certbot/plugins/selection_test.py @@ -1,4 +1,4 @@ -"""Tests for letsenecrypt.plugins.selection""" +"""Tests for letsencrypt.plugins.selection""" import sys import unittest diff --git a/certbot/storage.py b/certbot/storage.py index af0e9d701..4f1cd0a9d 100644 --- a/certbot/storage.py +++ b/certbot/storage.py @@ -716,7 +716,7 @@ class RenewableCert(object): :returns: ``True`` if there is a complete version of this lineage with a larger version number than the current - version, and ``False`` otherwis + version, and ``False`` otherwise :rtype: bool """ diff --git a/certbot/tests/cli_test.py b/certbot/tests/cli_test.py index aff1bcc99..d157901e4 100644 --- a/certbot/tests/cli_test.py +++ b/certbot/tests/cli_test.py @@ -52,7 +52,7 @@ class ParseTest(unittest.TestCase): return cli.prepare_and_parse_args(PLUGINS, *args, **kwargs) def _help_output(self, args): - "Run a command, and return the ouput string for scrutiny" + "Run a command, and return the output string for scrutiny" output = six.StringIO() with mock.patch('certbot.main.sys.stdout', new=output): diff --git a/certbot/tests/error_handler_test.py b/certbot/tests/error_handler_test.py index a548377bd..60dcf5e99 100644 --- a/certbot/tests/error_handler_test.py +++ b/certbot/tests/error_handler_test.py @@ -70,7 +70,7 @@ class ErrorHandlerTest(unittest.TestCase): send_signal(self.signals[0]) should_be_42 *= 10 - # check exectuion stoped when the signal was sent + # check execution stoped when the signal was sent self.assertEqual(42, should_be_42) # assert signals were caught self.assertEqual([self.signals[0]], signals_received) diff --git a/certbot/tests/errors_test.py b/certbot/tests/errors_test.py index f35a5ea08..aee1857a6 100644 --- a/certbot/tests/errors_test.py +++ b/certbot/tests/errors_test.py @@ -9,7 +9,7 @@ from certbot import achallenges from certbot.tests import acme_util -class FaiiledChallengesTest(unittest.TestCase): +class FailedChallengesTest(unittest.TestCase): """Tests for certbot.errors.FailedChallenges.""" def setUp(self): diff --git a/certbot/tests/reverter_test.py b/certbot/tests/reverter_test.py index 2eeabe116..d430f8292 100644 --- a/certbot/tests/reverter_test.py +++ b/certbot/tests/reverter_test.py @@ -394,7 +394,7 @@ class TestFullCheckpointsReverter(unittest.TestCase): self.assertTrue(mock_logger.info.call_count > 0) def test_view_config_changes_bad_backups_dir(self): - # There shouldn't be any "in progess directories when this is called + # There shouldn't be any "in progress directories when this is called # It must just be clean checkpoints os.makedirs(os.path.join(self.config.backup_dir, "in_progress")) diff --git a/docs/ciphers.rst b/docs/ciphers.rst index cf9ffdb99..31ce45963 100644 --- a/docs/ciphers.rst +++ b/docs/ciphers.rst @@ -206,7 +206,7 @@ or a family of enhancements, one per selectable ciphersuite configuration. Feedback ======== -We receive lots of feedback on the type of ciphersuites that Let's Encrypt supports and list some coallated feedback below. This section aims to track suggestions and references that people have offered or identified to improve the ciphersuites that Let's Encrypt enables when configuring TLS on servers. +We receive lots of feedback on the type of ciphersuites that Let's Encrypt supports and list some collated feedback below. This section aims to track suggestions and references that people have offered or identified to improve the ciphersuites that Let's Encrypt enables when configuring TLS on servers. Because of the Chatham House Rule applicable to some of the discussions, people are *not* individually credited for their suggestions, but most suggestions here were made or found by other people, and I thank them for their contributions. diff --git a/docs/cli-help.txt b/docs/cli-help.txt index 349092630..9e1dbc268 100644 --- a/docs/cli-help.txt +++ b/docs/cli-help.txt @@ -210,7 +210,7 @@ manage: certificates List certificates managed by Certbot delete Clean up all files related to a certificate - renew Renew all certificates (or one specifed with --cert- + renew Renew all certificates (or one specified with --cert- name) revoke Revoke a certificate specified with --cert-path update_symlinks Recreate symlinks in your /etc/letsencrypt/live/ diff --git a/docs/contributing.rst b/docs/contributing.rst index c51e493bc..040c22864 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -336,7 +336,7 @@ the ``letsencrypt-auto-source`` and Building letsencrypt-auto-source/letsencrypt-auto ------------------------------------------------- Once changes to any of the aforementioned files have been made, the -``letesncrypt-auto-source/letsencrypt-auto`` script should be updated. In lieu of +``letsencrypt-auto-source/letsencrypt-auto`` script should be updated. In lieu of manually updating this script, run the build script, which lives at ``letsencrypt-auto-source/build.py``: diff --git a/docs/using.rst b/docs/using.rst index a1881852e..0d74e4f8b 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -30,7 +30,7 @@ control the domain(s) you are requesting a cert for, obtains a cert for the spec domain(s), and places the cert in the ``/etc/letsencrypt`` directory on your machine. The authenticator does not install the cert (it does not edit any of your server's configuration files to serve the obtained certificate). If you specify multiple domains to authenticate, they will -all be listed in a single certificate. To obtain multiple seperate certificates +all be listed in a single certificate. To obtain multiple separate certificates you will need to run Certbot multiple times. Installers are Plugins used with the ``install`` command to install a cert. @@ -62,7 +62,7 @@ manual_ Y N | Helps you obtain a cert by giving you instructions to pe Under the hood, plugins use one of several ACME protocol "Challenges_" to prove you control a domain. The options are http-01_ (which uses port 80), -tls-sni-01_ (port 443) and dns-01_ (requring configuration of a DNS server on +tls-sni-01_ (port 443) and dns-01_ (requiring configuration of a DNS server on port 53, though that's often not the same machine as your webserver). A few plugins support more than one challenge type, in which case you can choose one with ``--preferred-challenges``. @@ -433,7 +433,7 @@ variables to these scripts: - ``CERTBOT_DOMAIN``: The domain being authenticated - ``CERTBOT_VALIDATION``: The validation string -- ``CERTBOT_TOKEN``: Resource name part of the HTTP-01 challenege (HTTP-01 only) +- ``CERTBOT_TOKEN``: Resource name part of the HTTP-01 challenge (HTTP-01 only) Additionally for cleanup: diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index e5e8a431b..68f135bf8 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -105,7 +105,7 @@ fi # certbot itself needs root access for almost all modes of operation # The "normal" case is that sudo is used for the steps that need root, but # this script *can* be run as root (not recommended), or fall back to using -# `su`. Auto-detection can be overrided by explicitly setting the +# `su`. Auto-detection can be overridden by explicitly setting the # environment variable LE_AUTO_SUDO to 'sudo', 'sudo_su' or '' as used below. # Because the parameters in `su -c` has to be a string, @@ -1056,7 +1056,7 @@ UNLIKELY_EOF fi else - # Phase 1: Upgrade certbot-auto if neceesary, then self-invoke. + # Phase 1: Upgrade certbot-auto if necessary, then self-invoke. # # Each phase checks the version of only the thing it is responsible for # upgrading. Phase 1 checks the version of the latest release of diff --git a/letsencrypt-auto-source/letsencrypt-auto.template b/letsencrypt-auto-source/letsencrypt-auto.template index a8a9f04d5..327799210 100755 --- a/letsencrypt-auto-source/letsencrypt-auto.template +++ b/letsencrypt-auto-source/letsencrypt-auto.template @@ -105,7 +105,7 @@ fi # certbot itself needs root access for almost all modes of operation # The "normal" case is that sudo is used for the steps that need root, but # this script *can* be run as root (not recommended), or fall back to using -# `su`. Auto-detection can be overrided by explicitly setting the +# `su`. Auto-detection can be overridden by explicitly setting the # environment variable LE_AUTO_SUDO to 'sudo', 'sudo_su' or '' as used below. # Because the parameters in `su -c` has to be a string, @@ -355,7 +355,7 @@ UNLIKELY_EOF fi else - # Phase 1: Upgrade certbot-auto if neceesary, then self-invoke. + # Phase 1: Upgrade certbot-auto if necessary, then self-invoke. # # Each phase checks the version of only the thing it is responsible for # upgrading. Phase 1 checks the version of the latest release of diff --git a/tests/boulder-integration.sh b/tests/boulder-integration.sh index 5a1f88c8a..4a2131006 100755 --- a/tests/boulder-integration.sh +++ b/tests/boulder-integration.sh @@ -139,7 +139,7 @@ sed -i "4arenew_before_expiry = 4 years" "$root/conf/renewal/le.wtf.conf" common_no_force_renew renew --rsa-key-size 2048 CheckCertCount "le.wtf" 3 -# The 4096 bit setting should persist to the first renewal, but be overriden in the second +# The 4096 bit setting should persist to the first renewal, but be overridden in the second size1=`wc -c ${root}/conf/archive/le.wtf/privkey1.pem | cut -d" " -f1` size2=`wc -c ${root}/conf/archive/le.wtf/privkey2.pem | cut -d" " -f1` diff --git a/tests/letstest/scripts/test_letsencrypt_auto_certonly_standalone.sh b/tests/letstest/scripts/test_letsencrypt_auto_certonly_standalone.sh index 9b5ff88a2..815522d38 100755 --- a/tests/letstest/scripts/test_letsencrypt_auto_certonly_standalone.sh +++ b/tests/letstest/scripts/test_letsencrypt_auto_certonly_standalone.sh @@ -21,7 +21,7 @@ letsencrypt-auto certonly --no-self-upgrade -v --standalone --debug \ # 1. be in the right directory cd tests/letstest/testdata/ -# 2. refer to the config with the same level of relativitity that it itself +# 2. refer to the config with the same level of relativity that it itself # contains :/ OUT=`letsencrypt-auto certificates --config-dir sample-config -v` TEST_CERTS=`echo "$OUT" | grep TEST_CERT | wc -l` diff --git a/tests/letstest/scripts/test_ocsp_experimental.sh b/tests/letstest/scripts/test_ocsp_experimental.sh index cc787653c..91298385c 100755 --- a/tests/letstest/scripts/test_ocsp_experimental.sh +++ b/tests/letstest/scripts/test_ocsp_experimental.sh @@ -22,7 +22,7 @@ sudo venv/bin/certbot certonly --no-self-upgrade -v --standalone --debug \ # 1. be in the right directory cd tests/letstest/testdata/ -# 2. refer to the config with the same level of relativitity that it itself +# 2. refer to the config with the same level of relativity that it itself # contains :/ OUT=`sudo ../../../venv/bin/certbot certificates -v --config-dir sample-config` TEST_CERTS=`echo "$OUT" | grep TEST_CERT | wc -l` diff --git a/tools/_venv_common.sh b/tools/_venv_common.sh index a121af82d..720dcaeab 100755 --- a/tools/_venv_common.sh +++ b/tools/_venv_common.sh @@ -2,7 +2,7 @@ VENV_NAME=${VENV_NAME:-venv} -# .egg-info directories tend to cause bizzaire problems (e.g. `pip -e +# .egg-info directories tend to cause bizarre problems (e.g. `pip -e # .` might unexpectedly install letshelp-certbot only, in case # `python letshelp-certbot/setup.py build` has been called # earlier) diff --git a/tools/release.sh b/tools/release.sh index be306d8e0..75a4af29c 100755 --- a/tools/release.sh +++ b/tools/release.sh @@ -72,7 +72,7 @@ pip install -U virtualenv root_without_le="$version.$$" root="./releases/le.$root_without_le" -echo "Cloning into fresh copy at $root" # clean repo = no artificats +echo "Cloning into fresh copy at $root" # clean repo = no artifacts git clone . $root git rev-parse HEAD cd $root From caa7e4e3f0d3ab252eb00ad8ea7d24331d107568 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Fri, 27 Jan 2017 14:06:40 -0800 Subject: [PATCH 37/88] fix tools/venv.sh (#4126) --- tools/_venv_common.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/_venv_common.sh b/tools/_venv_common.sh index 720dcaeab..ddbb02c62 100755 --- a/tools/_venv_common.sh +++ b/tools/_venv_common.sh @@ -12,13 +12,13 @@ rm -rf *.egg-info # `/home/jakub/dev/letsencrypt/letsencrypt/venv/bin/python2` and # `venv/bin/python2` are the same file mv $VENV_NAME "$VENV_NAME.$(date +%s).bak" || true -virtualenv --no-site-packages $VENV_NAME $VENV_ARGS +virtualenv --no-site-packages --setuptools $VENV_NAME $VENV_ARGS . ./$VENV_NAME/bin/activate # Separately install setuptools and pip to make sure following # invocations use latest -pip install -U setuptools pip install -U pip +pip install -U setuptools pip install "$@" set +x From a1b1ae25ae21239d1eeb4e88c949e97137d28cec Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Fri, 27 Jan 2017 15:16:19 -0800 Subject: [PATCH 38/88] Don't be unnecessarily inefficient when finding a cert by name. (#4128) * Make lineage_for_certname and domains_for_certname O(1) * update tests --- certbot/cert_manager.py | 30 ++++++++++++---------------- certbot/tests/cert_manager_test.py | 32 +++++++++++------------------- 2 files changed, 25 insertions(+), 37 deletions(-) diff --git a/certbot/cert_manager.py b/certbot/cert_manager.py index ae4c5a722..71a5fe6fa 100644 --- a/certbot/cert_manager.py +++ b/certbot/cert_manager.py @@ -95,27 +95,23 @@ def delete(config): # Public Helpers ################### -def lineage_for_certname(config, certname): +def lineage_for_certname(cli_config, certname): """Find a lineage object with name certname.""" - def update_cert_for_name_match(candidate_lineage, rv): - """Return cert if it has name certname, else return rv - """ - matching_lineage_name_cert = rv - if candidate_lineage.lineagename == certname: - matching_lineage_name_cert = candidate_lineage - return matching_lineage_name_cert - return _search_lineages(config, update_cert_for_name_match, None) + configs_dir = cli_config.renewal_configs_dir + # Verify the directory is there + util.make_or_verify_dir(configs_dir, mode=0o755, uid=os.geteuid()) + renewal_file = storage.renewal_file_for_certname(cli_config, certname) + try: + return storage.RenewableCert(renewal_file, cli_config) + except (errors.CertStorageError, IOError): + logger.debug("Renewal conf file %s is broken.", renewal_file) + logger.debug("Traceback was:\n%s", traceback.format_exc()) + return None def domains_for_certname(config, certname): """Find the domains in the cert with name certname.""" - def update_domains_for_name_match(candidate_lineage, rv): - """Return domains if certname matches, else return rv - """ - matching_domains = rv - if candidate_lineage.lineagename == certname: - matching_domains = candidate_lineage.names() - return matching_domains - return _search_lineages(config, update_domains_for_name_match, None) + lineage = lineage_for_certname(config, certname) + return lineage.names() if lineage else None def find_duplicative_certs(config, domains): """Find existing certs that duplicate the request.""" diff --git a/certbot/tests/cert_manager_test.py b/certbot/tests/cert_manager_test.py index d7d1a3aff..1fa68d195 100644 --- a/certbot/tests/cert_manager_test.py +++ b/certbot/tests/cert_manager_test.py @@ -268,11 +268,11 @@ class LineageForCertnameTest(BaseCertManagerTest): """Tests for certbot.cert_manager.lineage_for_certname""" @mock.patch('certbot.util.make_or_verify_dir') - @mock.patch('certbot.storage.renewal_conf_files') + @mock.patch('certbot.storage.renewal_file_for_certname') @mock.patch('certbot.storage.RenewableCert') - def test_found_match(self, mock_renewable_cert, mock_renewal_conf_files, + def test_found_match(self, mock_renewable_cert, mock_renewal_conf_file, mock_make_or_verify_dir): - mock_renewal_conf_files.return_value = ["somefile.conf"] + mock_renewal_conf_file.return_value = "somefile.conf" mock_match = mock.Mock(lineagename="example.com") mock_renewable_cert.return_value = mock_match from certbot import cert_manager @@ -281,13 +281,10 @@ class LineageForCertnameTest(BaseCertManagerTest): self.assertTrue(mock_make_or_verify_dir.called) @mock.patch('certbot.util.make_or_verify_dir') - @mock.patch('certbot.storage.renewal_conf_files') - @mock.patch('certbot.storage.RenewableCert') - def test_no_match(self, mock_renewable_cert, mock_renewal_conf_files, + @mock.patch('certbot.storage.renewal_file_for_certname') + def test_no_match(self, mock_renewal_conf_file, mock_make_or_verify_dir): - mock_renewal_conf_files.return_value = ["somefile.conf"] - mock_match = mock.Mock(lineagename="other.com") - mock_renewable_cert.return_value = mock_match + mock_renewal_conf_file.return_value = "other.com.conf" from certbot import cert_manager self.assertEqual(cert_manager.lineage_for_certname(self.cli_config, "example.com"), None) @@ -298,11 +295,11 @@ class DomainsForCertnameTest(BaseCertManagerTest): """Tests for certbot.cert_manager.domains_for_certname""" @mock.patch('certbot.util.make_or_verify_dir') - @mock.patch('certbot.storage.renewal_conf_files') + @mock.patch('certbot.storage.renewal_file_for_certname') @mock.patch('certbot.storage.RenewableCert') - def test_found_match(self, mock_renewable_cert, mock_renewal_conf_files, + def test_found_match(self, mock_renewable_cert, mock_renewal_conf_file, mock_make_or_verify_dir): - mock_renewal_conf_files.return_value = ["somefile.conf"] + mock_renewal_conf_file.return_value = "somefile.conf" mock_match = mock.Mock(lineagename="example.com") domains = ["example.com", "example.org"] mock_match.names.return_value = domains @@ -313,15 +310,10 @@ class DomainsForCertnameTest(BaseCertManagerTest): self.assertTrue(mock_make_or_verify_dir.called) @mock.patch('certbot.util.make_or_verify_dir') - @mock.patch('certbot.storage.renewal_conf_files') - @mock.patch('certbot.storage.RenewableCert') - def test_no_match(self, mock_renewable_cert, mock_renewal_conf_files, + @mock.patch('certbot.storage.renewal_file_for_certname') + def test_no_match(self, mock_renewal_conf_file, mock_make_or_verify_dir): - mock_renewal_conf_files.return_value = ["somefile.conf"] - mock_match = mock.Mock(lineagename="example.com") - domains = ["example.com", "example.org"] - mock_match.names.return_value = domains - mock_renewable_cert.return_value = mock_match + mock_renewal_conf_file.return_value = "somefile.conf" from certbot import cert_manager self.assertEqual(cert_manager.domains_for_certname(self.cli_config, "other.com"), None) From 240438eec73ad20c50db3bc242a0a229cabf8f66 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Mon, 30 Jan 2017 09:44:55 -0800 Subject: [PATCH 39/88] Incorporate feedback from 4113 (#4115) --- acme/acme/client.py | 16 +++++++--------- acme/acme/client_test.py | 10 ++++++++++ 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/acme/acme/client.py b/acme/acme/client.py index 7555a1cc7..0324967cf 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -681,16 +681,14 @@ class ClientNetwork(object): # pylint: disable=too-many-instance-attributes be retried once. """ - should_retry = True - while True: - try: + try: + return self._post_once(*args, **kwargs) + except messages.Error as error: + if error.code == 'badNonce': + logger.debug('Retrying request after error:\n%s', error) return self._post_once(*args, **kwargs) - except messages.Error as error: - if should_retry and error.code == 'badNonce': - logger.debug('Retrying request after error:\n%s', error) - should_retry = False - else: - raise + else: + raise def _post_once(self, url, obj, content_type=JOSE_CONTENT_TYPE, **kwargs): data = self._wrap_in_jws(obj, self._get_nonce(url)) diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index d67c425bd..179a8a08c 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -717,6 +717,16 @@ class ClientNetworkWithMockedResponseTest(unittest.TestCase): self.assertRaises(messages.Error, self.net.post, 'uri', self.obj, content_type=self.content_type) + def test_post_not_retried(self): + check_response = mock.MagicMock() + check_response.side_effect = [messages.Error.with_code('malformed'), + self.checked_response] + + # pylint: disable=protected-access + self.net._check_response = check_response + self.assertRaises(messages.Error, self.net.post, 'uri', + self.obj, content_type=self.content_type) + def test_post_successful_retry(self): check_response = mock.MagicMock() check_response.side_effect = [messages.Error.with_code('badNonce'), From be5bcfe463d9649c350181fe1a84e9d513db1ff3 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Mon, 30 Jan 2017 16:55:54 -0800 Subject: [PATCH 40/88] Remove optional dependencies (#4088) * Stop using already_listening in standalone * remove already_listening * remove psutil entirely * fix #595 * Add basic perform test * make pep8 happy * Add test_perform_eacces * add _setup_perform_error * Add test_perform_unexpected_socket_error * add test_perform_eaddrinuse_no_retry * add test_perform_eaddrinuse_retry * cleanup tests * stop using dnspython * don't install dns extras in tox * remove dns extras from setup.py * Add simple_verify back to DNS response * remove dnspython from oldest tests --- acme/acme/challenges.py | 31 ++----- acme/acme/challenges_test.py | 47 ++-------- acme/acme/dns_resolver.py | 45 ---------- acme/acme/dns_resolver_test.py | 77 ---------------- acme/acme/test_util.py | 16 ---- acme/acme/util.py | 18 ---- acme/acme/util_test.py | 18 ---- acme/setup.py | 6 -- certbot/plugins/standalone.py | 113 ++++++++++------------- certbot/plugins/standalone_test.py | 127 ++++++++++---------------- certbot/plugins/util.py | 125 -------------------------- certbot/plugins/util_test.py | 139 ----------------------------- certbot/tests/util.py | 16 ---- setup.py | 1 - tox.ini | 18 ++-- 15 files changed, 122 insertions(+), 675 deletions(-) delete mode 100644 acme/acme/dns_resolver.py delete mode 100644 acme/acme/dns_resolver_test.py diff --git a/acme/acme/challenges.py b/acme/acme/challenges.py index 3bfc1e62f..83b9b9edd 100644 --- a/acme/acme/challenges.py +++ b/acme/acme/challenges.py @@ -9,7 +9,6 @@ from cryptography.hazmat.primitives import hashes import OpenSSL import requests -from acme import dns_resolver from acme import errors from acme import crypto_util from acme import fields @@ -214,36 +213,24 @@ class DNS01Response(KeyAuthorizationChallengeResponse): def simple_verify(self, chall, domain, account_public_key): """Simple verify. + This method no longer checks DNS records and is a simple wrapper + around `KeyAuthorizationChallengeResponse.verify`. + :param challenges.DNS01 chall: Corresponding challenge. :param unicode domain: Domain name being verified. :param JWK account_public_key: Public key for the key pair being authorized. - :returns: ``True`` iff validation with the TXT records resolved from a - DNS server is successful. + :return: ``True`` iff verification of the key authorization was + successful. :rtype: bool """ - if not self.verify(chall, account_public_key): + # pylint: disable=unused-argument + verified = self.verify(chall, account_public_key) + if not verified: logger.debug("Verification of key authorization in response failed") - return False - - validation_domain_name = chall.validation_domain_name(domain) - validation = chall.validation(account_public_key) - logger.debug("Verifying %s at %s...", chall.typ, validation_domain_name) - - try: - txt_records = dns_resolver.txt_records_for_name( - validation_domain_name) - except errors.DependencyError: - raise errors.DependencyError("Local validation for 'dns-01' " - "challenges requires 'dnspython'") - exists = validation in txt_records - if not exists: - logger.debug("Key authorization from response (%r) doesn't match " - "any DNS response in %r", self.key_authorization, - txt_records) - return exists + return verified @Challenge.register # pylint: disable=too-many-ancestors diff --git a/acme/acme/challenges_test.py b/acme/acme/challenges_test.py index 5ac07abdd..49e790102 100644 --- a/acme/acme/challenges_test.py +++ b/acme/acme/challenges_test.py @@ -10,7 +10,6 @@ from six.moves.urllib import parse as urllib_parse # pylint: disable=import-err from acme import errors from acme import jose from acme import test_util -from acme.dns_resolver import DNS_REQUIREMENT CERT = test_util.load_comparable_cert('cert.pem') KEY = jose.JWKRSA(key=test_util.load_rsa_private_key('rsa512_key.pem')) @@ -92,7 +91,6 @@ class DNS01ResponseTest(unittest.TestCase): from acme.challenges import DNS01 self.chall = DNS01(token=(b'x' * 16)) self.response = self.chall.response(KEY) - self.records_for_name_path = "acme.dns_resolver.txt_records_for_name" def test_to_partial_json(self): self.assertEqual(self.jmsg, self.msg.to_partial_json()) @@ -105,45 +103,16 @@ class DNS01ResponseTest(unittest.TestCase): from acme.challenges import DNS01Response hash(DNS01Response.from_json(self.jmsg)) - def test_simple_verify_bad_key_authorization(self): + def test_simple_verify_failure(self): key2 = jose.JWKRSA.load(test_util.load_vector('rsa256_key.pem')) - self.response.simple_verify(self.chall, "local", key2.public_key()) + public_key = key2.public_key() + verified = self.response.simple_verify(self.chall, "local", public_key) + self.assertFalse(verified) - @mock.patch('acme.dns_resolver.DNS_AVAILABLE', False) - def test_simple_verify_without_dns(self): - self.assertRaises( - errors.DependencyError, self.response.simple_verify, - self.chall, 'local', KEY.public_key()) - - @test_util.skip_unless(test_util.requirement_available(DNS_REQUIREMENT), - "optional dependency dnspython is not available") - def test_simple_verify_good_validation(self): # pragma: no cover - with mock.patch(self.records_for_name_path) as mock_resolver: - mock_resolver.return_value = [ - self.chall.validation(KEY.public_key())] - self.assertTrue(self.response.simple_verify( - self.chall, "local", KEY.public_key())) - mock_resolver.assert_called_once_with( - self.chall.validation_domain_name("local")) - - @test_util.skip_unless(test_util.requirement_available(DNS_REQUIREMENT), - "optional dependency dnspython is not available") - def test_simple_verify_good_validation_multitxts(self): # pragma: no cover - with mock.patch(self.records_for_name_path) as mock_resolver: - mock_resolver.return_value = [ - "!", self.chall.validation(KEY.public_key())] - self.assertTrue(self.response.simple_verify( - self.chall, "local", KEY.public_key())) - mock_resolver.assert_called_once_with( - self.chall.validation_domain_name("local")) - - @test_util.skip_unless(test_util.requirement_available(DNS_REQUIREMENT), - "optional dependency dnspython is not available") - def test_simple_verify_bad_validation(self): # pragma: no cover - with mock.patch(self.records_for_name_path) as mock_resolver: - mock_resolver.return_value = ["!"] - self.assertFalse(self.response.simple_verify( - self.chall, "local", KEY.public_key())) + def test_simple_verify_success(self): + public_key = KEY.public_key() + verified = self.response.simple_verify(self.chall, "local", public_key) + self.assertTrue(verified) class DNS01Test(unittest.TestCase): diff --git a/acme/acme/dns_resolver.py b/acme/acme/dns_resolver.py deleted file mode 100644 index 2677d92ad..000000000 --- a/acme/acme/dns_resolver.py +++ /dev/null @@ -1,45 +0,0 @@ -"""DNS Resolver for ACME client. -Required only for local validation of 'dns-01' challenges. -""" -import logging - -from acme import errors -from acme import util - -DNS_REQUIREMENT = 'dnspython>=1.12' - -try: - util.activate(DNS_REQUIREMENT) - # pragma: no cover - import dns.exception - import dns.resolver - DNS_AVAILABLE = True -except errors.DependencyError: # pragma: no cover - DNS_AVAILABLE = False - - -logger = logging.getLogger(__name__) - - -def txt_records_for_name(name): - """Resolve the name and return the TXT records. - - :param unicode name: Domain name being verified. - - :returns: A list of txt records, if empty the name could not be resolved - :rtype: list of unicode - - """ - if not DNS_AVAILABLE: - raise errors.DependencyError( - '{0} is required to use this function'.format(DNS_REQUIREMENT)) - try: - dns_response = dns.resolver.query(name, 'TXT') - except dns.resolver.NXDOMAIN as error: - return [] - except dns.exception.DNSException as error: - logger.error("Error resolving %s: %s", name, str(error)) - return [] - - return [txt_rec.decode("utf-8") for rdata in dns_response - for txt_rec in rdata.strings] diff --git a/acme/acme/dns_resolver_test.py b/acme/acme/dns_resolver_test.py deleted file mode 100644 index 2e2edd0e7..000000000 --- a/acme/acme/dns_resolver_test.py +++ /dev/null @@ -1,77 +0,0 @@ -"""Tests for acme.dns_resolver.""" -import unittest - -import mock -from six.moves import reload_module # pylint: disable=import-error - -from acme import errors -from acme import test_util -from acme.dns_resolver import DNS_REQUIREMENT - - -if test_util.requirement_available(DNS_REQUIREMENT): - import dns - - -def create_txt_response(name, txt_records): - """ - Returns an RRSet containing the 'txt_records' as the result of a DNS - query for 'name'. - - This takes advantage of the fact that an Answer object mostly behaves - like an RRset. - """ - return dns.rrset.from_text_list(name, 60, "IN", "TXT", txt_records) - - -class TxtRecordsForNameTest(unittest.TestCase): - """Tests for acme.dns_resolver.txt_records_for_name.""" - @classmethod - def _call(cls, *args, **kwargs): - from acme.dns_resolver import txt_records_for_name - return txt_records_for_name(*args, **kwargs) - - -@test_util.skip_unless(test_util.requirement_available(DNS_REQUIREMENT), - "optional dependency dnspython is not available") -class TxtRecordsForNameWithDnsTest(TxtRecordsForNameTest): - """Tests for acme.dns_resolver.txt_records_for_name with dns.""" - @mock.patch("acme.dns_resolver.dns.resolver.query") - def test_txt_records_for_name_with_single_response(self, mock_dns): - mock_dns.return_value = create_txt_response('name', ['response']) - self.assertEqual(['response'], self._call('name')) - - @mock.patch("acme.dns_resolver.dns.resolver.query") - def test_txt_records_for_name_with_multiple_responses(self, mock_dns): - mock_dns.return_value = create_txt_response( - 'name', ['response1', 'response2']) - self.assertEqual(['response1', 'response2'], self._call('name')) - - @mock.patch("acme.dns_resolver.dns.resolver.query") - def test_txt_records_for_name_domain_not_found(self, mock_dns): - mock_dns.side_effect = dns.resolver.NXDOMAIN - self.assertEquals([], self._call('name')) - - @mock.patch("acme.dns_resolver.dns.resolver.query") - def test_txt_records_for_name_domain_other_error(self, mock_dns): - mock_dns.side_effect = dns.exception.DNSException - self.assertEquals([], self._call('name')) - - -class TxtRecordsForNameWithoutDnsTest(TxtRecordsForNameTest): - """Tests for acme.dns_resolver.txt_records_for_name without dns.""" - def setUp(self): - from acme import dns_resolver - dns_resolver.DNS_AVAILABLE = False - - def tearDown(self): - from acme import dns_resolver - reload_module(dns_resolver) - - def test_exception_raised(self): - self.assertRaises( - errors.DependencyError, self._call, "example.org") - - -if __name__ == '__main__': - unittest.main() # pragma: no cover diff --git a/acme/acme/test_util.py b/acme/acme/test_util.py index ba968511f..0f5763682 100644 --- a/acme/acme/test_util.py +++ b/acme/acme/test_util.py @@ -11,9 +11,7 @@ from cryptography.hazmat.backends import default_backend from cryptography.hazmat.primitives import serialization import OpenSSL -from acme import errors from acme import jose -from acme import util def vector_path(*names): @@ -78,20 +76,6 @@ def load_pyopenssl_private_key(*names): return OpenSSL.crypto.load_privatekey(loader, load_vector(*names)) -def requirement_available(requirement): - """Checks if requirement can be imported. - - :rtype: bool - :returns: ``True`` iff requirement can be imported - - """ - try: - util.activate(requirement) - except errors.DependencyError: # pragma: no cover - return False - return True # pragma: no cover - - def skip_unless(condition, reason): # pragma: no cover """Skip tests unless a condition holds. diff --git a/acme/acme/util.py b/acme/acme/util.py index ac445b271..1fff89a9e 100644 --- a/acme/acme/util.py +++ b/acme/acme/util.py @@ -1,25 +1,7 @@ """ACME utilities.""" -import pkg_resources import six -from acme import errors - def map_keys(dikt, func): """Map dictionary keys.""" return dict((func(key), value) for key, value in six.iteritems(dikt)) - - -def activate(requirement): - """Make requirement importable. - - :param str requirement: the distribution and version to activate - - :raises acme.errors.DependencyError: if cannot activate requirement - - """ - try: - for distro in pkg_resources.require(requirement): # pylint: disable=not-callable - distro.activate() - except (pkg_resources.DistributionNotFound, pkg_resources.VersionConflict): - raise errors.DependencyError('{0} is unavailable'.format(requirement)) diff --git a/acme/acme/util_test.py b/acme/acme/util_test.py index ba6465409..00aa8b02d 100644 --- a/acme/acme/util_test.py +++ b/acme/acme/util_test.py @@ -1,8 +1,6 @@ """Tests for acme.util.""" import unittest -from acme import errors - class MapKeysTest(unittest.TestCase): """Tests for acme.util.map_keys.""" @@ -14,21 +12,5 @@ class MapKeysTest(unittest.TestCase): self.assertEqual({2: 2, 4: 4}, map_keys({1: 2, 3: 4}, lambda x: x + 1)) -class ActivateTest(unittest.TestCase): - """Tests for acme.util.activate.""" - - @classmethod - def _call(cls, *args, **kwargs): - from acme.util import activate - return activate(*args, **kwargs) - - def test_failure(self): - self.assertRaises(errors.DependencyError, self._call, 'acme>99.0.0') - - def test_success(self): - self._call('acme') - import acme as unused_acme - - if __name__ == '__main__': unittest.main() # pragma: no cover diff --git a/acme/setup.py b/acme/setup.py index 37b71368c..d05d814ad 100644 --- a/acme/setup.py +++ b/acme/setup.py @@ -33,11 +33,6 @@ if sys.version_info < (2, 7): else: install_requires.append('mock') -# dnspython 1.12 is required to support both Python 2 and Python 3. -dns_extras = [ - 'dnspython>=1.12', -] - dev_extras = [ 'nose', 'tox', @@ -77,7 +72,6 @@ setup( include_package_data=True, install_requires=install_requires, extras_require={ - 'dns': dns_extras, 'dev': dev_extras, 'docs': docs_extras, }, diff --git a/certbot/plugins/standalone.py b/certbot/plugins/standalone.py index 4fc52479f..0c15930a3 100644 --- a/certbot/plugins/standalone.py +++ b/certbot/plugins/standalone.py @@ -18,7 +18,6 @@ from certbot import errors from certbot import interfaces from certbot.plugins import common -from certbot.plugins import util logger = logging.getLogger(__name__) @@ -208,74 +207,38 @@ class Authenticator(common.Plugin): # pylint: disable=unused-argument,missing-docstring return self.supported_challenges - def _verify_ports_are_available(self, achalls): - """Confirm the ports are available to solve all achalls. - - :param list achalls: list of - :class:`~certbot.achallenges.AnnotatedChallenge` - - :raises .errors.MisconfigurationError: if required port is - unavailable - - """ - ports = [] - if any(isinstance(ac.chall, challenges.HTTP01) for ac in achalls): - ports.append(self.config.http01_port) - if any(isinstance(ac.chall, challenges.TLSSNI01) for ac in achalls): - ports.append(self.config.tls_sni_01_port) - - renewer = (self.config.verb == "renew") - - if any(util.already_listening(port, renewer) for port in ports): - raise errors.MisconfigurationError( - "At least one of the required ports is already taken.") - def perform(self, achalls): # pylint: disable=missing-docstring - self._verify_ports_are_available(achalls) + return [self._try_perform_single(achall) for achall in achalls] - try: - return self.perform2(achalls) - except errors.StandaloneBindError as error: - display = zope.component.getUtility(interfaces.IDisplay) + def _try_perform_single(self, achall): + while True: + try: + return self._perform_single(achall) + except errors.StandaloneBindError as error: + _handle_perform_error(error) - if error.socket_error.errno == socket.errno.EACCES: - display.notification( - "Could not bind TCP port {0} because you don't have " - "the appropriate permissions (for example, you " - "aren't running this program as " - "root).".format(error.port), force_interactive=True) - elif error.socket_error.errno == socket.errno.EADDRINUSE: - display.notification( - "Could not bind TCP port {0} because it is already in " - "use by another process on this system (such as a web " - "server). Please stop the program in question and then " - "try again.".format(error.port), force_interactive=True) - else: - raise # XXX: How to handle unknown errors in binding? + def _perform_single(self, achall): + if isinstance(achall.chall, challenges.HTTP01): + server, response = self._perform_http_01(achall) + else: # tls-sni-01 + server, response = self._perform_tls_sni_01(achall) + self.served[server].add(achall) + return response - def perform2(self, achalls): - """Perform achallenges without IDisplay interaction.""" - responses = [] + def _perform_http_01(self, achall): + server = self.servers.run(self.config.http01_port, challenges.HTTP01) + response, validation = achall.response_and_validation() + resource = acme_standalone.HTTP01RequestHandler.HTTP01Resource( + chall=achall.chall, response=response, validation=validation) + self.http_01_resources.add(resource) + return server, response - for achall in achalls: - if isinstance(achall.chall, challenges.HTTP01): - server = self.servers.run( - self.config.http01_port, challenges.HTTP01) - response, validation = achall.response_and_validation() - self.http_01_resources.add( - acme_standalone.HTTP01RequestHandler.HTTP01Resource( - chall=achall.chall, response=response, - validation=validation)) - else: # tls-sni-01 - server = self.servers.run( - self.config.tls_sni_01_port, challenges.TLSSNI01) - response, (cert, _) = achall.response_and_validation( - cert_key=self.key) - self.certs[response.z_domain] = (self.key, cert) - self.served[server].add(achall) - responses.append(response) - - return responses + def _perform_tls_sni_01(self, achall): + port = self.config.tls_sni_01_port + server = self.servers.run(port, challenges.TLSSNI01) + response, (cert, _) = achall.response_and_validation(cert_key=self.key) + self.certs[response.z_domain] = (self.key, cert) + return server, response def cleanup(self, achalls): # pylint: disable=missing-docstring # reduce self.served and close servers if none challenges are served @@ -286,3 +249,25 @@ class Authenticator(common.Plugin): for port, server in six.iteritems(self.servers.running()): if not self.served[server]: self.servers.stop(port) + + +def _handle_perform_error(error): + if error.socket_error.errno == socket.errno.EACCES: + raise errors.PluginError( + "Could not bind TCP port {0} because you don't have " + "the appropriate permissions (for example, you " + "aren't running this program as " + "root).".format(error.port)) + elif error.socket_error.errno == socket.errno.EADDRINUSE: + display = zope.component.getUtility(interfaces.IDisplay) + msg = ( + "Could not bind TCP port {0} because it is already in " + "use by another process on this system (such as a web " + "server). Please stop the program in question and " + "then try again.".format(error.port)) + should_retry = display.yesno(msg, "Retry", + "Cancel", default=False) + if not should_retry: + raise errors.PluginError(msg) + else: + raise diff --git a/certbot/plugins/standalone_test.py b/certbot/plugins/standalone_test.py index 08e59c929..83e0fcf7f 100644 --- a/certbot/plugins/standalone_test.py +++ b/certbot/plugins/standalone_test.py @@ -8,11 +8,9 @@ import six from acme import challenges from acme import jose -from acme import standalone as acme_standalone from certbot import achallenges from certbot import errors -from certbot import interfaces from certbot.tests import acme_util from certbot.tests import util as test_util @@ -114,6 +112,7 @@ def get_open_port(): open_socket.close() return port + class AuthenticatorTest(unittest.TestCase): """Tests for certbot.plugins.standalone.Authenticator.""" @@ -124,6 +123,7 @@ class AuthenticatorTest(unittest.TestCase): tls_sni_01_port=get_open_port(), http01_port=get_open_port(), standalone_supported_challenges="tls-sni-01,http-01") self.auth = Authenticator(self.config, name="standalone") + self.auth.servers = mock.MagicMock() def test_supported_challenges(self): self.assertEqual(self.auth.supported_challenges, @@ -146,6 +146,52 @@ class AuthenticatorTest(unittest.TestCase): self.assertEqual(self.auth.get_chall_pref(domain=None), [challenges.TLSSNI01]) + def test_perform(self): + achalls = self._get_achalls() + response = self.auth.perform(achalls) + + expected = [achall.response(achall.account_key) for achall in achalls] + self.assertEqual(response, expected) + + @test_util.patch_get_utility() + def test_perform_eaddrinuse_retry(self, mock_get_utility): + errno = socket.errno.EADDRINUSE + error = errors.StandaloneBindError(mock.MagicMock(errno=errno), -1) + self.auth.servers.run.side_effect = [error] + 2 * [mock.MagicMock()] + mock_yesno = mock_get_utility.return_value.yesno + mock_yesno.return_value = True + + self.test_perform() + self._assert_correct_yesno_call(mock_yesno) + + @test_util.patch_get_utility() + def test_perform_eaddrinuse_no_retry(self, mock_get_utility): + mock_yesno = mock_get_utility.return_value.yesno + mock_yesno.return_value = False + + errno = socket.errno.EADDRINUSE + self.assertRaises(errors.PluginError, self._fail_perform, errno) + self._assert_correct_yesno_call(mock_yesno) + + def _assert_correct_yesno_call(self, mock_yesno): + yesno_args, yesno_kwargs = mock_yesno.call_args + self.assertTrue("in use" in yesno_args[0]) + self.assertFalse(yesno_kwargs.get("default", True)) + + def test_perform_eacces(self): + errno = socket.errno.EACCES + self.assertRaises(errors.PluginError, self._fail_perform, errno) + + def test_perform_unexpected_socket_error(self): + errno = socket.errno.ENOTCONN + self.assertRaises( + errors.StandaloneBindError, self._fail_perform, errno) + + def _fail_perform(self, errno): + error = errors.StandaloneBindError(mock.MagicMock(errno=errno), -1) + self.auth.servers.run.side_effect = error + self.auth.perform(self._get_achalls()) + @classmethod def _get_achalls(cls): domain = b'localhost' @@ -157,84 +203,7 @@ class AuthenticatorTest(unittest.TestCase): return [http_01, tls_sni_01] - @mock.patch("certbot.plugins.standalone.util") - def test_perform_already_listening(self, mock_util): - http_01, tls_sni_01 = self._get_achalls() - - for achall, port in ((http_01, self.config.http01_port,), - (tls_sni_01, self.config.tls_sni_01_port)): - mock_util.already_listening.return_value = True - self.assertRaises( - errors.MisconfigurationError, self.auth.perform, [achall]) - mock_util.already_listening.assert_called_once_with(port, False) - mock_util.already_listening.reset_mock() - - @test_util.patch_get_utility() - def test_perform(self, unused_mock_get_utility): - achalls = self._get_achalls() - - self.auth.perform2 = mock.Mock(return_value=mock.sentinel.responses) - self.assertEqual(mock.sentinel.responses, self.auth.perform(achalls)) - self.auth.perform2.assert_called_once_with(achalls) - - @test_util.patch_get_utility() - def _test_perform_bind_errors(self, errno, achalls, mock_get_utility): - port = get_open_port() - def _perform2(unused_achalls): - raise errors.StandaloneBindError(mock.Mock(errno=errno), port) - - self.auth.perform2 = mock.MagicMock(side_effect=_perform2) - self.auth.perform(achalls) - mock_get_utility.assert_called_once_with(interfaces.IDisplay) - notification = mock_get_utility.return_value.notification - self.assertEqual(1, notification.call_count) - self.assertTrue(str(port) in notification.call_args[0][0]) - - def test_perform_eacces(self): - # pylint: disable=no-value-for-parameter - self._test_perform_bind_errors(socket.errno.EACCES, []) - - def test_perform_eaddrinuse(self): - # pylint: disable=no-value-for-parameter - self._test_perform_bind_errors(socket.errno.EADDRINUSE, []) - - def test_perfom_unknown_bind_error(self): - self.assertRaises( - errors.StandaloneBindError, self._test_perform_bind_errors, - socket.errno.ENOTCONN, []) - - def test_perform2(self): - http_01, tls_sni_01 = self._get_achalls() - - self.auth.servers = mock.MagicMock() - - def _run(port, tls): # pylint: disable=unused-argument - return "server{0}".format(port) - - self.auth.servers.run.side_effect = _run - responses = self.auth.perform2([http_01, tls_sni_01]) - - self.assertTrue(isinstance(responses, list)) - self.assertEqual(2, len(responses)) - self.assertTrue(isinstance(responses[0], challenges.HTTP01Response)) - self.assertTrue(isinstance(responses[1], challenges.TLSSNI01Response)) - - self.assertEqual(self.auth.servers.run.mock_calls, [ - mock.call(self.config.http01_port, challenges.HTTP01), - mock.call(self.config.tls_sni_01_port, challenges.TLSSNI01), - ]) - self.assertEqual(self.auth.served, { - "server" + str(self.config.tls_sni_01_port): set([tls_sni_01]), - "server" + str(self.config.http01_port): set([http_01]), - }) - self.assertEqual(1, len(self.auth.http_01_resources)) - self.assertEqual(1, len(self.auth.certs)) - self.assertEqual(list(self.auth.http_01_resources), [ - acme_standalone.HTTP01RequestHandler.HTTP01Resource( - acme_util.HTTP01, responses[0], mock.ANY)]) - def test_cleanup(self): - self.auth.servers = mock.Mock() self.auth.servers.running.return_value = { 1: "server1", 2: "server2", diff --git a/certbot/plugins/util.py b/certbot/plugins/util.py index 20b0fdce7..e45c26735 100644 --- a/certbot/plugins/util.py +++ b/certbot/plugins/util.py @@ -1,34 +1,11 @@ """Plugin utilities.""" import logging import os -import socket -import zope.component - -from acme import errors as acme_errors -from acme import util as acme_util - -from certbot import interfaces from certbot import util -PSUTIL_REQUIREMENT = "psutil>=2.2.1" - -try: - acme_util.activate(PSUTIL_REQUIREMENT) - import psutil # pragma: no cover - USE_PSUTIL = True -except acme_errors.DependencyError: # pragma: no cover - USE_PSUTIL = False - logger = logging.getLogger(__name__) -RENEWER_EXTRA_MSG = ( - " For automated renewal, you may want to use a script that stops" - " and starts your webserver. You can find an example at" - " https://certbot.eff.org/docs/using.html#renewal ." - " Alternatively you can use the webroot plugin to renew without" - " needing to stop and start your webserver.") - def path_surgery(cmd): """Attempt to perform PATH surgery to find cmd @@ -59,105 +36,3 @@ def path_surgery(cmd): logger.warning("Failed to find %s in%s PATH: %s", cmd, expanded, path) return False - - -def already_listening(port, renewer=False): - """Check if a process is already listening on the port. - - If so, also tell the user via a display notification. - - .. warning:: - On some operating systems, this function can only usefully be - run as root. - - :param int port: The TCP port in question. - :returns: True or False. - - """ - - if USE_PSUTIL: - return already_listening_psutil(port, renewer=renewer) - else: - logger.debug("Psutil not found, using simple socket check.") - return already_listening_socket(port, renewer=renewer) - - -def already_listening_socket(port, renewer=False): - """Simple socket based check to find out if port is already in use - - :param int port: The TCP port in question. - :returns: True or False - """ - - try: - testsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) - testsocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - try: - testsocket.bind(("", port)) - except socket.error: - display = zope.component.getUtility(interfaces.IDisplay) - extra = "" - if renewer: - extra = RENEWER_EXTRA_MSG - display.notification( - "Port {0} is already in use by another process. This will " - "prevent us from binding to that port. Please stop the " - "process that is populating the port in question and try " - "again. {1}".format(port, extra), force_interactive=True) - return True - finally: - testsocket.close() - except socket.error: - pass - return False - - -def already_listening_psutil(port, renewer=False): - """Psutil variant of the open port check - - :param int port: The TCP port in question. - :returns: True or False. - - """ - try: - net_connections = psutil.net_connections() - except psutil.AccessDenied as error: - logger.info("Access denied when trying to list network " - "connections: %s. Are you root?", error) - # this function is just a pre-check that often causes false - # positives and problems in testing (c.f. #680 on Mac, #255 - # generally); we will fail later in bind() anyway - return False - - listeners = [conn.pid for conn in net_connections - if conn.status == 'LISTEN' and - conn.type == socket.SOCK_STREAM and - conn.laddr[1] == port] - try: - if listeners and listeners[0] is not None: - # conn.pid may be None if the current process doesn't have - # permission to identify the listening process! Additionally, - # listeners may have more than one element if separate - # sockets have bound the same port on separate interfaces. - # We currently only have UI to notify the user about one - # of them at a time. - pid = listeners[0] - name = psutil.Process(pid).name() - display = zope.component.getUtility(interfaces.IDisplay) - extra = "" - if renewer: - extra = RENEWER_EXTRA_MSG - display.notification( - "The program {0} (process ID {1}) is already listening " - "on TCP port {2}. This will prevent us from binding to " - "that port. Please stop the {0} program temporarily " - "and then try again.{3}".format(name, pid, port, extra), - force_interactive=True) - return True - except (psutil.NoSuchProcess, psutil.AccessDenied): - # Perhaps the result of a race where the process could have - # exited or relinquished the port (NoSuchProcess), or the result - # of an OS policy where we're not allowed to look up the process - # name (AccessDenied). - pass - return False diff --git a/certbot/plugins/util_test.py b/certbot/plugins/util_test.py index b5d188835..947f24697 100644 --- a/certbot/plugins/util_test.py +++ b/certbot/plugins/util_test.py @@ -1,13 +1,9 @@ """Tests for certbot.plugins.util.""" import os -import socket import unittest import mock -from certbot.plugins.util import PSUTIL_REQUIREMENT -from certbot.tests import util as test_util - class PathSurgeryTest(unittest.TestCase): """Tests for certbot.plugins.path_surgery.""" @@ -34,140 +30,5 @@ class PathSurgeryTest(unittest.TestCase): self.assertTrue("/tmp" in os.environ["PATH"]) -class AlreadyListeningTest(unittest.TestCase): - """Tests for certbot.plugins.already_listening.""" - @classmethod - def _call(cls, *args, **kwargs): - from certbot.plugins.util import already_listening - return already_listening(*args, **kwargs) - - -class AlreadyListeningTestNoPsutil(AlreadyListeningTest): - """Tests for certbot.plugins.already_listening when - psutil is not available""" - @classmethod - def _call(cls, *args, **kwargs): - with mock.patch("certbot.plugins.util.USE_PSUTIL", False): - return super( - AlreadyListeningTestNoPsutil, cls)._call(*args, **kwargs) - - @test_util.patch_get_utility() - def test_ports_available(self, mock_getutil): - # Ensure we don't get error - with mock.patch("socket.socket.bind"): - self.assertFalse(self._call(80)) - self.assertFalse(self._call(80, True)) - self.assertEqual(mock_getutil.call_count, 0) - - @test_util.patch_get_utility() - def test_ports_blocked(self, mock_getutil): - with mock.patch("certbot.plugins.util.socket.socket.bind") as mock_bind: - mock_bind.side_effect = socket.error - self.assertTrue(self._call(80)) - self.assertTrue(self._call(80, True)) - with mock.patch("certbot.plugins.util.socket.socket") as mock_socket: - mock_socket.side_effect = socket.error - self.assertFalse(self._call(80)) - self.assertEqual(mock_getutil.call_count, 2) - - -@test_util.skip_unless(test_util.requirement_available(PSUTIL_REQUIREMENT), - "optional dependency psutil is not available") -class AlreadyListeningTestPsutil(AlreadyListeningTest): - """Tests for certbot.plugins.already_listening.""" - @mock.patch("certbot.plugins.util.psutil.net_connections") - @mock.patch("certbot.plugins.util.psutil.Process") - @test_util.patch_get_utility() - def test_race_condition(self, mock_get_utility, mock_process, mock_net): - # This tests a race condition, or permission problem, or OS - # incompatibility in which, for some reason, no process name can be - # found to match the identified listening PID. - import psutil - from psutil._common import sconn - conns = [ - sconn(fd=-1, family=2, type=1, laddr=("0.0.0.0", 30), - raddr=(), status="LISTEN", pid=None), - sconn(fd=3, family=2, type=1, laddr=("192.168.5.10", 32783), - raddr=("20.40.60.80", 22), status="ESTABLISHED", pid=1234), - sconn(fd=-1, family=10, type=1, laddr=("::1", 54321), - raddr=("::1", 111), status="CLOSE_WAIT", pid=None), - sconn(fd=3, family=2, type=1, laddr=("0.0.0.0", 17), - raddr=(), status="LISTEN", pid=4416)] - mock_net.return_value = conns - mock_process.side_effect = psutil.NoSuchProcess("No such PID") - # We simulate being unable to find the process name of PID 4416, - # which results in returning False. - self.assertFalse(self._call(17)) - self.assertEqual(mock_get_utility.generic_notification.call_count, 0) - mock_process.assert_called_once_with(4416) - - @mock.patch("certbot.plugins.util.psutil.net_connections") - @mock.patch("certbot.plugins.util.psutil.Process") - @test_util.patch_get_utility() - def test_not_listening(self, mock_get_utility, mock_process, mock_net): - from psutil._common import sconn - conns = [ - sconn(fd=-1, family=2, type=1, laddr=("0.0.0.0", 30), - raddr=(), status="LISTEN", pid=None), - sconn(fd=3, family=2, type=1, laddr=("192.168.5.10", 32783), - raddr=("20.40.60.80", 22), status="ESTABLISHED", pid=1234), - sconn(fd=-1, family=10, type=1, laddr=("::1", 54321), - raddr=("::1", 111), status="CLOSE_WAIT", pid=None)] - mock_net.return_value = conns - mock_process.name.return_value = "inetd" - self.assertFalse(self._call(17)) - self.assertEqual(mock_get_utility.generic_notification.call_count, 0) - self.assertEqual(mock_process.call_count, 0) - - @mock.patch("certbot.plugins.util.psutil.net_connections") - @mock.patch("certbot.plugins.util.psutil.Process") - @test_util.patch_get_utility() - def test_listening_ipv4(self, mock_get_utility, mock_process, mock_net): - from psutil._common import sconn - conns = [ - sconn(fd=-1, family=2, type=1, laddr=("0.0.0.0", 30), - raddr=(), status="LISTEN", pid=None), - sconn(fd=3, family=2, type=1, laddr=("192.168.5.10", 32783), - raddr=("20.40.60.80", 22), status="ESTABLISHED", pid=1234), - sconn(fd=-1, family=10, type=1, laddr=("::1", 54321), - raddr=("::1", 111), status="CLOSE_WAIT", pid=None), - sconn(fd=3, family=2, type=1, laddr=("0.0.0.0", 17), - raddr=(), status="LISTEN", pid=4416)] - mock_net.return_value = conns - mock_process.name.return_value = "inetd" - result = self._call(17, True) - self.assertTrue(result) - self.assertEqual(mock_get_utility.call_count, 1) - mock_process.assert_called_once_with(4416) - - @mock.patch("certbot.plugins.util.psutil.net_connections") - @mock.patch("certbot.plugins.util.psutil.Process") - @test_util.patch_get_utility() - def test_listening_ipv6(self, mock_get_utility, mock_process, mock_net): - from psutil._common import sconn - conns = [ - sconn(fd=-1, family=2, type=1, laddr=("0.0.0.0", 30), - raddr=(), status="LISTEN", pid=None), - sconn(fd=3, family=2, type=1, laddr=("192.168.5.10", 32783), - raddr=("20.40.60.80", 22), status="ESTABLISHED", pid=1234), - sconn(fd=-1, family=10, type=1, laddr=("::1", 54321), - raddr=("::1", 111), status="CLOSE_WAIT", pid=None), - sconn(fd=3, family=10, type=1, laddr=("::", 12345), raddr=(), - status="LISTEN", pid=4420), - sconn(fd=3, family=2, type=1, laddr=("0.0.0.0", 17), - raddr=(), status="LISTEN", pid=4416)] - mock_net.return_value = conns - mock_process.name.return_value = "inetd" - result = self._call(12345) - self.assertTrue(result) - self.assertEqual(mock_get_utility.call_count, 1) - mock_process.assert_called_once_with(4420) - - @mock.patch("certbot.plugins.util.psutil.net_connections") - def test_access_denied_exception(self, mock_net): - import psutil - mock_net.side_effect = psutil.AccessDenied("") - self.assertFalse(self._call(12345)) - if __name__ == "__main__": unittest.main() # pragma: no cover diff --git a/certbot/tests/util.py b/certbot/tests/util.py index 7d674e171..092807b56 100644 --- a/certbot/tests/util.py +++ b/certbot/tests/util.py @@ -13,9 +13,7 @@ from cryptography.hazmat.primitives import serialization import mock import OpenSSL -from acme import errors from acme import jose -from acme import util from certbot import constants from certbot import interfaces @@ -86,20 +84,6 @@ def load_pyopenssl_private_key(*names): return OpenSSL.crypto.load_privatekey(loader, load_vector(*names)) -def requirement_available(requirement): - """Checks if requirement can be imported. - - :rtype: bool - :returns: ``True`` iff requirement can be imported - - """ - try: - util.activate(requirement) - except errors.DependencyError: # pragma: no cover - return False - return True # pragma: no cover - - def skip_unless(condition, reason): # pragma: no cover """Skip tests unless a condition holds. diff --git a/setup.py b/setup.py index bc208f693..a47aff11b 100644 --- a/setup.py +++ b/setup.py @@ -67,7 +67,6 @@ dev_extras = [ 'astroid==1.3.5', 'coverage', 'nose', - 'psutil>=2.2.1', # for tests, optional 'pylint==1.4.2', # upstream #248 'tox', 'twine', diff --git a/tox.ini b/tox.ini index 6f6903948..e6317e665 100644 --- a/tox.ini +++ b/tox.ini @@ -13,7 +13,7 @@ envlist = modification,py{26,33,34,35,36},cover,lint # packages installed separately to ensure that downstream deps problems # are detected, c.f. #1002 commands = - pip install -e acme[dns,dev] + pip install -e acme[dev] nosetests -v acme --processes=-1 pip install -e .[dev] nosetests -v certbot --processes=-1 --process-timeout=100 @@ -37,35 +37,33 @@ deps = py{26,27}-oldest: cffi<=1.7 py{26,27}-oldest: cryptography==0.8 py{26,27}-oldest: configargparse==0.10.0 - py{26,27}-oldest: dnspython>=1.12 - py{26,27}-oldest: psutil==2.1.0 py{26,27}-oldest: PyOpenSSL==0.13 py{26,27}-oldest: requests<=2.11.1 [testenv:py33] commands = - pip install -e acme[dns,dev] + pip install -e acme[dev] nosetests -v acme --processes=-1 pip install -e .[dev] nosetests -v certbot --processes=-1 --process-timeout=100 [testenv:py34] commands = - pip install -e acme[dns,dev] + pip install -e acme[dev] nosetests -v acme --processes=-1 pip install -e .[dev] nosetests -v certbot --processes=-1 --process-timeout=100 [testenv:py35] commands = - pip install -e acme[dns,dev] + pip install -e acme[dev] nosetests -v acme --processes=-1 pip install -e .[dev] nosetests -v certbot --processes=-1 --process-timeout=100 [testenv:py36] commands = - pip install -e acme[dns,dev] + pip install -e acme[dev] nosetests -v acme --processes=-1 pip install -e .[dev] nosetests -v certbot --processes=-1 --process-timeout=100 @@ -73,12 +71,12 @@ commands = [testenv:py27_install] basepython = python2.7 commands = - pip install -e acme[dns,dev] -e .[dev] -e certbot-apache -e certbot-nginx -e letshelp-certbot + pip install -e acme[dev] -e .[dev] -e certbot-apache -e certbot-nginx -e letshelp-certbot [testenv:cover] basepython = python2.7 commands = - pip install -e acme[dns,dev] -e .[dev] -e certbot-apache -e certbot-nginx -e letshelp-certbot + pip install -e acme[dev] -e .[dev] -e certbot-apache -e certbot-nginx -e letshelp-certbot ./tox.cover.sh [testenv:lint] @@ -88,7 +86,7 @@ basepython = python2.7 # duplicate code checking; if one of the commands fails, others will # continue, but tox return code will reflect previous error commands = - pip install -q -e acme[dns,dev] -e .[dev] -e certbot-apache -e certbot-nginx -e certbot-compatibility-test -e letshelp-certbot + pip install -q -e acme[dev] -e .[dev] -e certbot-apache -e certbot-nginx -e certbot-compatibility-test -e letshelp-certbot pylint --reports=n --rcfile=.pylintrc acme/acme certbot certbot-apache/certbot_apache certbot-nginx/certbot_nginx certbot-compatibility-test/certbot_compatibility_test letshelp-certbot/letshelp_certbot [testenv:apacheconftest] From 7f3c732bbfdb94ccb462ab3f9f4b3960415b698e Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Mon, 30 Jan 2017 19:37:23 -0800 Subject: [PATCH 41/88] Cleanup test farm tests and add test_sdists (#4089) * add get_certbot_version.sh * Use get_certbot_version.sh in build.py * make test_leauto_upgrades.sh more robust * auto upgrades break tests * OCSP experimental is not needed anymore * Add test_sdists.sh * Use LE_AUTO_VERSION, not repo version * install OS deps earlier * use readlink not realpath * undo changes to build.py * Factor out version code from build.py * Use version.py in test_sdists * Remove get_certbot_version * workaround setuptools breakage --- letsencrypt-auto-source/build.py | 17 +------- letsencrypt-auto-source/version.py | 28 +++++++++++++ .../letstest/scripts/test_leauto_upgrades.sh | 3 +- ...st_letsencrypt_auto_certonly_standalone.sh | 2 +- .../scripts/test_ocsp_experimental.sh | 39 ------------------- tests/letstest/scripts/test_sdists.sh | 36 +++++++++++++++++ 6 files changed, 69 insertions(+), 56 deletions(-) create mode 100755 letsencrypt-auto-source/version.py delete mode 100755 tests/letstest/scripts/test_ocsp_experimental.sh create mode 100755 tests/letstest/scripts/test_sdists.sh diff --git a/letsencrypt-auto-source/build.py b/letsencrypt-auto-source/build.py index ea74f9766..eebad61b7 100755 --- a/letsencrypt-auto-source/build.py +++ b/letsencrypt-auto-source/build.py @@ -8,26 +8,13 @@ other, special definitions. """ from os.path import abspath, dirname, join import re -from sys import argv + +from version import certbot_version, file_contents DIR = dirname(abspath(__file__)) -def certbot_version(build_script_dir): - """Return the version number stamped in certbot/__init__.py.""" - return re.search('''^__version__ = ['"](.+)['"].*''', - file_contents(join(dirname(build_script_dir), - 'certbot', - '__init__.py')), - re.M).group(1) - - -def file_contents(path): - with open(path) as file: - return file.read() - - def build(version=None, requirements=None): """Return the built contents of the letsencrypt-auto script. diff --git a/letsencrypt-auto-source/version.py b/letsencrypt-auto-source/version.py new file mode 100755 index 000000000..c49d96654 --- /dev/null +++ b/letsencrypt-auto-source/version.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python +"""Get the current Certbot version number. + +Provides simple utilities for determining the Certbot version number and +building letsencrypt-auto. + +""" +from __future__ import print_function +from os.path import abspath, dirname, join +import re + + +def certbot_version(build_script_dir): + """Return the version number stamped in certbot/__init__.py.""" + return re.search('''^__version__ = ['"](.+)['"].*''', + file_contents(join(dirname(build_script_dir), + 'certbot', + '__init__.py')), + re.M).group(1) + + +def file_contents(path): + with open(path) as file: + return file.read() + + +if __name__ == '__main__': + print(certbot_version(dirname(abspath(__file__)))) diff --git a/tests/letstest/scripts/test_leauto_upgrades.sh b/tests/letstest/scripts/test_leauto_upgrades.sh index 08247fe8d..f7a74821b 100755 --- a/tests/letstest/scripts/test_leauto_upgrades.sh +++ b/tests/letstest/scripts/test_leauto_upgrades.sh @@ -29,7 +29,8 @@ unset PIP_INDEX_URL export PIP_EXTRA_INDEX_URL="$SAVE" git checkout -f "$BRANCH" -if ! ./letsencrypt-auto -v --debug --version | grep 0.9.0 ; then +EXPECTED_VERSION=$(grep -m1 LE_AUTO_VERSION letsencrypt-auto | cut -d\" -f2) +if ! ./letsencrypt-auto -v --debug --version --no-self-upgrade | grep $EXPECTED_VERSION ; then echo upgrade appeared to fail exit 1 fi diff --git a/tests/letstest/scripts/test_letsencrypt_auto_certonly_standalone.sh b/tests/letstest/scripts/test_letsencrypt_auto_certonly_standalone.sh index 815522d38..7a86f8d9d 100755 --- a/tests/letstest/scripts/test_letsencrypt_auto_certonly_standalone.sh +++ b/tests/letstest/scripts/test_letsencrypt_auto_certonly_standalone.sh @@ -23,7 +23,7 @@ cd tests/letstest/testdata/ # 2. refer to the config with the same level of relativity that it itself # contains :/ -OUT=`letsencrypt-auto certificates --config-dir sample-config -v` +OUT=`letsencrypt-auto certificates --config-dir sample-config -v --no-self-upgrade` TEST_CERTS=`echo "$OUT" | grep TEST_CERT | wc -l` REVOKED=`echo "$OUT" | grep REVOKED | wc -l` diff --git a/tests/letstest/scripts/test_ocsp_experimental.sh b/tests/letstest/scripts/test_ocsp_experimental.sh deleted file mode 100755 index 91298385c..000000000 --- a/tests/letstest/scripts/test_ocsp_experimental.sh +++ /dev/null @@ -1,39 +0,0 @@ -#!/bin/bash -x - -# $PUBLIC_IP $PRIVATE_IP $PUBLIC_HOSTNAME $BOULDER_URL are dynamically set at execution - -# with curl, instance metadata available from EC2 metadata service: -#public_host=$(curl -s http://169.254.169.254/2014-11-05/meta-data/public-hostname) -#public_ip=$(curl -s http://169.254.169.254/2014-11-05/meta-data/public-ipv4) -#private_ip=$(curl -s http://169.254.169.254/2014-11-05/meta-data/local-ipv4) - -cd letsencrypt -export PATH="$PWD/letsencrypt-auto-source:$PATH" -letsencrypt-auto-source/letsencrypt-auto --os-packages-only --debug --version -tools/venv.sh -sudo venv/bin/certbot certonly --no-self-upgrade -v --standalone --debug \ - --text --agree-dev-preview --agree-tos \ - --renew-by-default --redirect \ - --register-unsafely-without-email \ - --domain $PUBLIC_HOSTNAME --server $BOULDER_URL - -# we have to jump through some hoops to cope with relative paths in renewal -# conf files ... -# 1. be in the right directory -cd tests/letstest/testdata/ - -# 2. refer to the config with the same level of relativity that it itself -# contains :/ -OUT=`sudo ../../../venv/bin/certbot certificates -v --config-dir sample-config` -TEST_CERTS=`echo "$OUT" | grep TEST_CERT | wc -l` -REVOKED=`echo "$OUT" | grep REVOKED | wc -l` - -if [ "$TEST_CERTS" != 2 ] ; then - echo "Did not find two test certs as expected ($TEST_CERTS)" - exit 1 -fi - -if [ "$REVOKED" != 1 ] ; then - echo "Did not find one revoked cert as expected ($REVOKED)" - exit 1 -fi diff --git a/tests/letstest/scripts/test_sdists.sh b/tests/letstest/scripts/test_sdists.sh new file mode 100755 index 000000000..e4ebd2e14 --- /dev/null +++ b/tests/letstest/scripts/test_sdists.sh @@ -0,0 +1,36 @@ +#!/bin/sh -xe + +cd letsencrypt +./certbot-auto --os-packages-only -n --debug + +PLUGINS="certbot-apache certbot-nginx" +PYTHON=$(command -v python2.7 || command -v python27 || command -v python2 || command -v python) +TEMP_DIR=$(mktemp -d) +VERSION=$(letsencrypt-auto-source/version.py) + +# setup venv +virtualenv --no-site-packages -p $PYTHON --setuptools venv +. ./venv/bin/activate +pip install -U pip +pip install -U setuptools + +# build sdists +for pkg_dir in acme . $PLUGINS; do + cd $pkg_dir + python setup.py clean + rm -rf build dist + python setup.py sdist + mv dist/* $TEMP_DIR + cd - +done + +# test sdists +cd $TEMP_DIR +for pkg in acme certbot $PLUGINS; do + tar -xvf "$pkg-$VERSION.tar.gz" + cd "$pkg-$VERSION" + python setup.py build + python setup.py test + python setup.py install + cd - +done From 20be8b327d5cb19b2ef5dedae3b771c1de952bd3 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Tue, 31 Jan 2017 17:08:21 -0800 Subject: [PATCH 42/88] Provide a way to opt-in to EFF e-mail (#4082) * Add eff email flags * add eff_sign_up * add requests dep to certbot * make pylint happy * Add EFF subscribe uri * add POST to EFF and write tests * log EFF e-mail submission * Add eff module and tests * cleanup client tests * offer subscription when changing e-mail * cleanup client.py and tests * expand e-mail prompt --- certbot/cli.py | 6 ++ certbot/client.py | 9 ++- certbot/constants.py | 3 + certbot/eff.py | 95 ++++++++++++++++++++++++ certbot/main.py | 2 + certbot/tests/client_test.py | 56 +++++++++------ certbot/tests/eff_test.py | 135 +++++++++++++++++++++++++++++++++++ certbot/tests/main_test.py | 7 +- setup.py | 3 + 9 files changed, 288 insertions(+), 28 deletions(-) create mode 100644 certbot/eff.py create mode 100644 certbot/tests/eff_test.py diff --git a/certbot/cli.py b/certbot/cli.py index f3b07d925..a24f566d1 100644 --- a/certbot/cli.py +++ b/certbot/cli.py @@ -880,6 +880,12 @@ def prepare_and_parse_args(plugins, args, detect_defaults=False): # pylint: dis helpful.add( ["register", "unregister", "automation"], "-m", "--email", help=config_help("email")) + helpful.add(["register", "automation"], "--eff-email", action="store_true", + default=None, dest="eff_email", + help="Share your e-mail address with EFF") + helpful.add(["register", "automation"], "--no-eff-email", action="store_false", + default=None, dest="eff_email", + help="Don't share your e-mail address with EFF") helpful.add( ["automation", "certonly", "run"], "--keep-until-expiring", "--keep", "--reinstall", diff --git a/certbot/client.py b/certbot/client.py index 8d53a29d3..26c5e87a9 100644 --- a/certbot/client.py +++ b/certbot/client.py @@ -15,15 +15,16 @@ import certbot from certbot import account from certbot import auth_handler +from certbot import cli from certbot import constants from certbot import crypto_util -from certbot import errors +from certbot import eff from certbot import error_handler +from certbot import errors from certbot import interfaces -from certbot import util from certbot import reverter from certbot import storage -from certbot import cli +from certbot import util from certbot.display import ops as display_ops from certbot.display import enhancements @@ -139,6 +140,8 @@ def register(config, account_storage, tos_cb=None): account.report_new_account(acc, config) account_storage.save(acc) + eff.handle_subscription(config) + return acc, acme diff --git a/certbot/constants.py b/certbot/constants.py index f64ff7e1e..b286ca26a 100644 --- a/certbot/constants.py +++ b/certbot/constants.py @@ -106,3 +106,6 @@ RENEWAL_CONFIGS_DIR = "renewal" FORCE_INTERACTIVE_FLAG = "--force-interactive" """Flag to disable TTY checking in IDisplay.""" + +EFF_SUBSCRIBE_URI = "https://supporters.eff.org/subscribe/certbot" +"""EFF URI used to submit the e-mail address of users who opt-in.""" diff --git a/certbot/eff.py b/certbot/eff.py new file mode 100644 index 000000000..0ef23cc8c --- /dev/null +++ b/certbot/eff.py @@ -0,0 +1,95 @@ +"""Subscribes users to the EFF newsletter.""" +import logging + +import requests +import zope.component + +from certbot import constants +from certbot import interfaces + + +logger = logging.getLogger(__name__) + + +def handle_subscription(config): + """High level function to take care of EFF newsletter subscriptions. + + The user may be asked if they want to sign up for the newsletter if + they have not already specified. + + :param .IConfig config: Client configuration. + + """ + if config.email is None: + if config.eff_email: + _report_failure("you didn't provide an e-mail address") + return + if config.eff_email is None: + config.eff_email = _want_subscription() + if config.eff_email: + subscribe(config.email) + + +def _want_subscription(): + """Does the user want to be subscribed to the EFF newsletter? + + :returns: True if we should subscribe the user, otherwise, False + :rtype: bool + + """ + prompt = ( + 'Would you be willing to share your email address with the ' + "Electronic Frontier Foundation, a founding partner of the Let's " + 'Encrypt project and the non-profit organization that develops ' + "Cerbot? We'd like to send you email about EFF and our work to " + 'encrypt the web, protect its users and defend digital rights.') + display = zope.component.getUtility(interfaces.IDisplay) + return display.yesno(prompt, default=False) + + +def subscribe(email): + """Subscribe the user to the EFF mailing list. + + :param str email: the e-mail address to subscribe + + """ + url = constants.EFF_SUBSCRIBE_URI + data = {'data_type': 'json', + 'email': email, + 'form_id': 'eff_supporters_library_subscribe_form'} + logger.debug('Sending POST request to %s:\n%s', url, data) + _check_response(requests.post(url, data=data)) + + +def _check_response(response): + """Check for errors in the server's response. + + If an error occurred, it will be reported to the user. + + :param requests.Response response: the server's response to the + subscription request + + """ + logger.debug('Received response:\n%s', response.content) + if response.ok: + if not response.json()['status']: + _report_failure('your e-mail address appears to be invalid') + else: + _report_failure() + + +def _report_failure(reason=None): + """Notify the user of failing to sign them up for the newsletter. + + :param reason: a phrase describing what the problem was + beginning with a lowercase letter and no closing punctuation + :type reason: `str` or `None` + + """ + msg = ['We were unable to subscribe you the EFF mailing list'] + if reason is not None: + msg.append(' because ') + msg.append(reason) + msg.append('. You can try again later by visiting https://act.eff.org.') + reporter = zope.component.getUtility(interfaces.IReporter) + reporter.add_message(''.join(msg), reporter.LOW_PRIORITY) diff --git a/certbot/main.py b/certbot/main.py index b52e605be..7ca346050 100644 --- a/certbot/main.py +++ b/certbot/main.py @@ -24,6 +24,7 @@ from certbot import crypto_util from certbot import colored_logging from certbot import configuration from certbot import constants +from certbot import eff from certbot import errors from certbot import hooks from certbot import interfaces @@ -474,6 +475,7 @@ def register(config, unused_plugins): acc.regr = acme_client.acme.update_registration(acc.regr.update( body=acc.regr.body.update(contact=('mailto:' + config.email,)))) account_storage.save_regr(acc) + eff.handle_subscription(config) add_msg("Your e-mail address was updated to {0}.".format(config.email)) diff --git a/certbot/tests/client_test.py b/certbot/tests/client_test.py index f4b86fc7c..cc3bb098d 100644 --- a/certbot/tests/client_test.py +++ b/certbot/tests/client_test.py @@ -44,20 +44,25 @@ class RegisterTest(unittest.TestCase): def test_no_tos(self): with mock.patch("certbot.client.acme_client.Client") as mock_client: mock_client.register().terms_of_service = "http://tos" - with mock.patch("certbot.account.report_new_account"): - self.tos_cb.return_value = False - self.assertRaises(errors.Error, self._call) + with mock.patch("certbot.eff.handle_subscription") as mock_handle: + with mock.patch("certbot.account.report_new_account"): + self.tos_cb.return_value = False + self.assertRaises(errors.Error, self._call) + self.assertFalse(mock_handle.called) - self.tos_cb.return_value = True - self._call() + self.tos_cb.return_value = True + self._call() + self.assertTrue(mock_handle.called) - self.tos_cb = None - self._call() + self.tos_cb = None + self._call() + self.assertEqual(mock_handle.call_count, 2) def test_it(self): with mock.patch("certbot.client.acme_client.Client"): with mock.patch("certbot.account.report_new_account"): - self._call() + with mock.patch("certbot.eff.handle_subscription"): + self._call() @mock.patch("certbot.account.report_new_account") @mock.patch("certbot.client.display_ops.get_email") @@ -67,9 +72,11 @@ class RegisterTest(unittest.TestCase): msg = "DNS problem: NXDOMAIN looking up MX for example.com" mx_err = messages.Error.with_code('invalidContact', detail=msg) with mock.patch("certbot.client.acme_client.Client") as mock_client: - mock_client().register.side_effect = [mx_err, mock.MagicMock()] - self._call() - self.assertEqual(mock_get_email.call_count, 1) + with mock.patch("certbot.eff.handle_subscription") as mock_handle: + mock_client().register.side_effect = [mx_err, mock.MagicMock()] + self._call() + self.assertEqual(mock_get_email.call_count, 1) + self.assertTrue(mock_handle.called) @mock.patch("certbot.account.report_new_account") def test_email_invalid_noninteractive(self, _rep): @@ -77,8 +84,9 @@ class RegisterTest(unittest.TestCase): msg = "DNS problem: NXDOMAIN looking up MX for example.com" mx_err = messages.Error.with_code('invalidContact', detail=msg) with mock.patch("certbot.client.acme_client.Client") as mock_client: - mock_client().register.side_effect = [mx_err, mock.MagicMock()] - self.assertRaises(errors.Error, self._call) + with mock.patch("certbot.eff.handle_subscription"): + mock_client().register.side_effect = [mx_err, mock.MagicMock()] + self.assertRaises(errors.Error, self._call) def test_needs_email(self): self.config.email = None @@ -86,21 +94,25 @@ class RegisterTest(unittest.TestCase): @mock.patch("certbot.client.logger") def test_without_email(self, mock_logger): - with mock.patch("certbot.client.acme_client.Client"): - with mock.patch("certbot.account.report_new_account"): - self.config.email = None - self.config.register_unsafely_without_email = True - self.config.dry_run = False - self._call() - mock_logger.warning.assert_called_once_with(mock.ANY) + with mock.patch("certbot.eff.handle_subscription") as mock_handle: + with mock.patch("certbot.client.acme_client.Client"): + with mock.patch("certbot.account.report_new_account"): + self.config.email = None + self.config.register_unsafely_without_email = True + self.config.dry_run = False + self._call() + mock_logger.warning.assert_called_once_with(mock.ANY) + self.assertTrue(mock_handle.called) def test_unsupported_error(self): from acme import messages msg = "Test" mx_err = messages.Error(detail=msg, typ="malformed", title="title") with mock.patch("certbot.client.acme_client.Client") as mock_client: - mock_client().register.side_effect = [mx_err, mock.MagicMock()] - self.assertRaises(messages.Error, self._call) + with mock.patch("certbot.eff.handle_subscription") as mock_handle: + mock_client().register.side_effect = [mx_err, mock.MagicMock()] + self.assertRaises(messages.Error, self._call) + self.assertFalse(mock_handle.called) class ClientTestCommon(unittest.TestCase): diff --git a/certbot/tests/eff_test.py b/certbot/tests/eff_test.py new file mode 100644 index 000000000..fd9a61181 --- /dev/null +++ b/certbot/tests/eff_test.py @@ -0,0 +1,135 @@ +"""Tests for certbot.eff.""" +import unittest + +import mock + +from certbot import constants +from certbot.tests import util + + +class HandleSubscriptionTest(unittest.TestCase): + """Tests for certbot.eff.handle_subscription.""" + def setUp(self): + self.email = 'certbot@example.org' + self.config = mock.Mock(email=self.email, eff_email=None) + + def _call(self): + from certbot.eff import handle_subscription + return handle_subscription(self.config) + + @util.patch_get_utility() + @mock.patch('certbot.eff.subscribe') + def test_failure(self, mock_subscribe, mock_get_utility): + self.config.email = None + self.config.eff_email = True + self._call() + self.assertFalse(mock_subscribe.called) + self.assertFalse(mock_get_utility().yesno.called) + actual = mock_get_utility().add_message.call_args[0][0] + expected_part = "because you didn't provide an e-mail address" + self.assertTrue(expected_part in actual) + + @mock.patch('certbot.eff.subscribe') + def test_no_subscribe_with_no_prompt(self, mock_subscribe): + self.config.eff_email = False + with util.patch_get_utility() as mock_get_utility: + self._call() + self.assertFalse(mock_subscribe.called) + self._assert_no_get_utility_calls(mock_get_utility) + + @util.patch_get_utility() + @mock.patch('certbot.eff.subscribe') + def test_subscribe_with_no_prompt(self, mock_subscribe, mock_get_utility): + self.config.eff_email = True + self._call() + self._assert_subscribed(mock_subscribe) + self._assert_no_get_utility_calls(mock_get_utility) + + def _assert_no_get_utility_calls(self, mock_get_utility): + self.assertFalse(mock_get_utility().yesno.called) + self.assertFalse(mock_get_utility().add_message.called) + + @util.patch_get_utility() + @mock.patch('certbot.eff.subscribe') + def test_subscribe_with_prompt(self, mock_subscribe, mock_get_utility): + mock_get_utility().yesno.return_value = True + self._call() + self._assert_subscribed(mock_subscribe) + self.assertFalse(mock_get_utility().add_message.called) + self._assert_correct_yesno_call(mock_get_utility) + + def _assert_subscribed(self, mock_subscribe): + self.assertTrue(mock_subscribe.called) + self.assertEqual(mock_subscribe.call_args[0][0], self.email) + + @util.patch_get_utility() + @mock.patch('certbot.eff.subscribe') + def test_no_subscribe_with_prompt(self, mock_subscribe, mock_get_utility): + mock_get_utility().yesno.return_value = False + self._call() + self.assertFalse(mock_subscribe.called) + self.assertFalse(mock_get_utility().add_message.called) + self._assert_correct_yesno_call(mock_get_utility) + + def _assert_correct_yesno_call(self, mock_get_utility): + self.assertTrue(mock_get_utility().yesno.called) + call_args, call_kwargs = mock_get_utility().yesno.call_args + actual = call_args[0] + expected_part = 'Electronic Frontier Foundation' + self.assertTrue(expected_part in actual) + self.assertFalse(call_kwargs.get('default', True)) + + +class SubscribeTest(unittest.TestCase): + """Tests for certbot.eff.subscribe.""" + def setUp(self): + self.email = 'certbot@example.org' + self.json = {'status': True} + self.response = mock.Mock(ok=True) + self.response.json.return_value = self.json + + @mock.patch('certbot.eff.requests.post') + def _call(self, mock_post): + mock_post.return_value = self.response + + from certbot.eff import subscribe + subscribe(self.email) + self._check_post_call(mock_post) + + def _check_post_call(self, mock_post): + self.assertEqual(mock_post.call_count, 1) + call_args, call_kwargs = mock_post.call_args + self.assertEqual(call_args[0], constants.EFF_SUBSCRIBE_URI) + + data = call_kwargs.get('data') + self.assertFalse(data is None) + self.assertEqual(data.get('email'), self.email) + + @util.patch_get_utility() + def test_bad_status(self, mock_get_utility): + self.json['status'] = False + self._call() # pylint: disable=no-value-for-parameter + actual = self._get_reported_message(mock_get_utility) + expected_part = 'because your e-mail address appears to be invalid.' + self.assertTrue(expected_part in actual) + + @util.patch_get_utility() + def test_not_ok(self, mock_get_utility): + self.response.ok = False + self._call() # pylint: disable=no-value-for-parameter + actual = self._get_reported_message(mock_get_utility) + unexpected_part = 'because' + self.assertFalse(unexpected_part in actual) + + def _get_reported_message(self, mock_get_utility): + self.assertTrue(mock_get_utility().add_message.called) + return mock_get_utility().add_message.call_args[0][0] + + @util.patch_get_utility() + def test_subscribe(self, mock_get_utility): + self._call() # pylint: disable=no-value-for-parameter + self.assertFalse(mock_get_utility.called) + + +if __name__ == '__main__': + unittest.main() # pragma: no cover diff --git a/certbot/tests/main_test.py b/certbot/tests/main_test.py index a4f3a7805..22b709f69 100644 --- a/certbot/tests/main_test.py +++ b/certbot/tests/main_test.py @@ -1145,9 +1145,9 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods def test_update_registration_with_email(self, mock_utility, mock_email): email = "user@example.com" mock_email.return_value = email - with mock.patch('certbot.main.client') as mocked_client: - with mock.patch('certbot.main.account') as mocked_account: - with mock.patch('certbot.main._determine_account') as mocked_det: + with mock.patch('certbot.eff.handle_subscription') as mock_handle: + with mock.patch('certbot.main._determine_account') as mocked_det: + with mock.patch('certbot.main.account') as mocked_account: with mock.patch('certbot.main.client') as mocked_client: mocked_storage = mock.MagicMock() mocked_account.AccountFileStorage.return_value = mocked_storage @@ -1168,6 +1168,7 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods self.assertTrue(mocked_storage.save_regr.called) self.assertTrue( email in mock_utility().add_message.call_args[0][0]) + self.assertTrue(mock_handle.called) class UnregisterTest(unittest.TestCase): diff --git a/setup.py b/setup.py index a47aff11b..529459cf0 100644 --- a/setup.py +++ b/setup.py @@ -31,6 +31,9 @@ changes = read_file(os.path.join(here, 'CHANGES.rst')) version = meta['version'] # Please update tox.ini when modifying dependency version requirements +# This package relies on requests, however, it isn't specified here to avoid +# masking the more specific request requirements in acme. See +# https://github.com/pypa/pip/issues/988 for more info. install_requires = [ 'acme=={0}'.format(version), # We technically need ConfigArgParse 0.10.0 for Python 2.6 support, but From 1fd847e2160ea41f0d4242aad4ff9b425c9ef28d Mon Sep 17 00:00:00 2001 From: Amjad Mashaal Date: Wed, 1 Feb 2017 17:12:51 +0200 Subject: [PATCH 43/88] Adding checking name validity to the Apache plugin (#3639) --- certbot-apache/certbot_apache/configurator.py | 2 +- .../certbot_apache/tests/configurator_test.py | 7 ++++--- .../configurators/apache/common.py | 6 +++++- certbot-nginx/certbot_nginx/configurator.py | 20 +------------------ certbot/util.py | 19 ++++++++++++++++++ 5 files changed, 30 insertions(+), 24 deletions(-) diff --git a/certbot-apache/certbot_apache/configurator.py b/certbot-apache/certbot_apache/configurator.py index 523f690b4..a956341eb 100644 --- a/certbot-apache/certbot_apache/configurator.py +++ b/certbot-apache/certbot_apache/configurator.py @@ -472,7 +472,7 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator): "\n\nUnfortunately mod_macro is not yet supported".format( "\n ".join(vhost_macro)), force_interactive=True) - return all_names + return util.get_filtered_names(all_names) def get_name_from_ip(self, addr): # pylint: disable=no-self-use """Returns a reverse dns name if available. diff --git a/certbot-apache/certbot_apache/tests/configurator_test.py b/certbot-apache/certbot_apache/tests/configurator_test.py index 065761496..01361c8f0 100644 --- a/certbot-apache/certbot_apache/tests/configurator_test.py +++ b/certbot-apache/certbot_apache/tests/configurator_test.py @@ -103,8 +103,8 @@ class MultipleVhostsTest(util.ApacheTest): mock_getutility.notification = mock.MagicMock(return_value=True) names = self.config.get_all_names() self.assertEqual(names, set( - ["certbot.demo", "ocspvhost.com", "encryption-example.demo", - "ip-172-30-0-17", "*.blue.purple.com"])) + ["certbot.demo", "ocspvhost.com", "encryption-example.demo"] + )) @certbot_util.patch_get_utility() @mock.patch("certbot_apache.configurator.socket.gethostbyaddr") @@ -123,7 +123,8 @@ class MultipleVhostsTest(util.ApacheTest): self.config.vhosts.append(vhost) names = self.config.get_all_names() - self.assertEqual(len(names), 7) + # Names get filtered, only 5 are returned + self.assertEqual(len(names), 5) self.assertTrue("zombo.com" in names) self.assertTrue("google.com" in names) self.assertTrue("certbot.demo" in names) diff --git a/certbot-compatibility-test/certbot_compatibility_test/configurators/apache/common.py b/certbot-compatibility-test/certbot_compatibility_test/configurators/apache/common.py index 64170ca72..2e9e68daf 100644 --- a/certbot-compatibility-test/certbot_compatibility_test/configurators/apache/common.py +++ b/certbot-compatibility-test/certbot_compatibility_test/configurators/apache/common.py @@ -8,6 +8,7 @@ import zope.interface from certbot import configuration from certbot import errors as le_errors +from certbot import util as certbot_util from certbot_apache import configurator from certbot_apache import constants from certbot_compatibility_test import errors @@ -106,4 +107,7 @@ def _get_names(config): not util.IP_REGEX.match(words[1]) and words[1].find(".") != -1): all_names.add(words[1]) - return all_names, non_ip_names + return ( + certbot_util.get_filtered_names(all_names), + certbot_util.get_filtered_names(non_ip_names) + ) diff --git a/certbot-nginx/certbot_nginx/configurator.py b/certbot-nginx/certbot_nginx/configurator.py index f19a07910..6d51ca641 100644 --- a/certbot-nginx/certbot_nginx/configurator.py +++ b/certbot-nginx/certbot_nginx/configurator.py @@ -403,25 +403,7 @@ class NginxConfigurator(common.Plugin): except (socket.error, socket.herror, socket.timeout): continue - return self._get_filtered_names(all_names) - - def _get_filtered_names(self, all_names): - """Removes names that aren't considered valid by Let's Encrypt. - - :param set all_names: all names found in the Nginx configuration - - :returns: all found names that are considered valid by LE - :rtype: set - - """ - filtered_names = set() - for name in all_names: - try: - filtered_names.add(util.enforce_le_validity(name)) - except errors.ConfigurationError as error: - logger.debug('Not suggesting name "%s"', name) - logger.debug(error) - return filtered_names + return util.get_filtered_names(all_names) def _get_snakeoil_paths(self): # TODO: generate only once diff --git a/certbot/util.py b/certbot/util.py index e8532fc6d..95c669d0d 100644 --- a/certbot/util.py +++ b/certbot/util.py @@ -219,6 +219,25 @@ def safely_remove(path): raise +def get_filtered_names(all_names): + """Removes names that aren't considered valid by Let's Encrypt. + + :param set all_names: all names found in the configuration + + :returns: all found names that are considered valid by LE + :rtype: set + + """ + filtered_names = set() + for name in all_names: + try: + filtered_names.add(enforce_le_validity(name)) + except errors.ConfigurationError as error: + logger.debug('Not suggesting name "%s"', name) + logger.debug(error) + return filtered_names + + def get_os_info(filepath="/etc/os-release"): """ Get OS name and version From aa389cf8aaec0bea871aee6b388c623cce3d0ac0 Mon Sep 17 00:00:00 2001 From: Peter Eckersley Date: Wed, 1 Feb 2017 09:32:24 -0800 Subject: [PATCH 44/88] Revert "uncomment assertion (#4072)" (#4110) Revert "uncomment assertion (#4072)" --- certbot/display/util.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/certbot/display/util.py b/certbot/display/util.py index f7b2566a3..87c75739b 100644 --- a/certbot/display/util.py +++ b/certbot/display/util.py @@ -253,7 +253,7 @@ class FileDisplay(object): :rtype: bool """ - assert_valid_call(prompt, default, cli_flag, force_interactive) + # assert_valid_call(prompt, default, cli_flag, force_interactive) if self._can_interact(force_interactive): return False elif default is None: From 30bc68ccdeb6c4898450438768fcd960852c64bc Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 1 Feb 2017 09:50:42 -0800 Subject: [PATCH 45/88] Release 0.11.0 --- acme/setup.py | 2 +- certbot-apache/setup.py | 2 +- certbot-auto | 234 ++++++++++-------- certbot-compatibility-test/setup.py | 2 +- certbot-nginx/setup.py | 2 +- certbot/__init__.py | 2 +- docs/cli-help.txt | 14 +- letsencrypt-auto | 234 ++++++++++-------- letsencrypt-auto-source/certbot-auto.asc | 14 +- letsencrypt-auto-source/letsencrypt-auto | 26 +- letsencrypt-auto-source/letsencrypt-auto.sig | Bin 256 -> 256 bytes .../pieces/letsencrypt-auto-requirements.txt | 24 +- 12 files changed, 315 insertions(+), 241 deletions(-) diff --git a/acme/setup.py b/acme/setup.py index d05d814ad..6b7badbd6 100644 --- a/acme/setup.py +++ b/acme/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.0.dev0' +version = '0.11.0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-apache/setup.py b/certbot-apache/setup.py index d50a414a1..957b31c52 100644 --- a/certbot-apache/setup.py +++ b/certbot-apache/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.0.dev0' +version = '0.11.0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-auto b/certbot-auto index 4e27bbd52..90b8157fe 100755 --- a/certbot-auto +++ b/certbot-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.10.2" +LE_AUTO_VERSION="0.11.0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -38,8 +38,9 @@ Help for certbot itself cannot be provided until it is installed. -n, --non-interactive, --noninteractive run without asking for user input --no-self-upgrade do not download updates --os-packages-only install OS dependencies and exit - -q, --quiet provide only update/error output -v, --verbose provide more output + -q, --quiet provide only update/error output; + implies --non-interactive All arguments are accepted and forwarded to the Certbot client when run." @@ -84,6 +85,11 @@ if [ $BASENAME = "letsencrypt-auto" ]; then HELP=0 fi +# Set ASSUME_YES to 1 if QUIET (i.e. --quiet implies --non-interactive) +if [ "$QUIET" = 1 ]; then + ASSUME_YES=1 +fi + # Support for busybox and others where there is no "command", # but "which" instead if command -v command > /dev/null 2>&1 ; then @@ -99,7 +105,7 @@ fi # certbot itself needs root access for almost all modes of operation # The "normal" case is that sudo is used for the steps that need root, but # this script *can* be run as root (not recommended), or fall back to using -# `su`. Auto-detection can be overrided by explicitly setting the +# `su`. Auto-detection can be overridden by explicitly setting the # environment variable LE_AUTO_SUDO to 'sudo', 'sudo_su' or '' as used below. # Because the parameters in `su -c` has to be a string, @@ -207,7 +213,11 @@ BootstrapDebCommon() { # # - Debian 6.0.10 "squeeze" (x64) - $SUDO apt-get update || echo apt-get update hit problems but continuing anyway... + if [ "$QUIET" = 1 ]; then + QUIET_FLAG='-qq' + fi + + $SUDO apt-get $QUIET_FLAG update || echo apt-get update hit problems but continuing anyway... # virtualenv binary can be found in different packages depending on # distro version (#346) @@ -215,74 +225,74 @@ BootstrapDebCommon() { virtualenv= # virtual env is known to apt and is installable if apt-cache show virtualenv > /dev/null 2>&1 ; then - if ! LC_ALL=C apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then - virtualenv="virtualenv" - fi + if ! LC_ALL=C apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then + virtualenv="virtualenv" + fi fi if apt-cache show python-virtualenv > /dev/null 2>&1; then - virtualenv="$virtualenv python-virtualenv" + virtualenv="$virtualenv python-virtualenv" fi augeas_pkg="libaugeas0 augeas-lenses" AUGVERSION=`LC_ALL=C apt-cache show --no-all-versions libaugeas0 | grep ^Version: | cut -d" " -f2` if [ "$ASSUME_YES" = 1 ]; then - YES_FLAG="-y" + YES_FLAG="-y" fi AddBackportRepo() { - # ARGS: - BACKPORT_NAME="$1" - BACKPORT_SOURCELINE="$2" - echo "To use the Apache Certbot plugin, augeas needs to be installed from $BACKPORT_NAME." - if ! grep -v -e ' *#' /etc/apt/sources.list | grep -q "$BACKPORT_NAME" ; then - # This can theoretically error if sources.list.d is empty, but in that case we don't care. - if ! grep -v -e ' *#' /etc/apt/sources.list.d/* 2>/dev/null | grep -q "$BACKPORT_NAME"; then - if [ "$ASSUME_YES" = 1 ]; then - /bin/echo -n "Installing augeas from $BACKPORT_NAME in 3 seconds..." - sleep 1s - /bin/echo -ne "\e[0K\rInstalling augeas from $BACKPORT_NAME in 2 seconds..." - sleep 1s - /bin/echo -e "\e[0K\rInstalling augeas from $BACKPORT_NAME in 1 second ..." - sleep 1s - add_backports=1 - else - read -p "Would you like to enable the $BACKPORT_NAME repository [Y/n]? " response - case $response in - [yY][eE][sS]|[yY]|"") - add_backports=1;; - *) - add_backports=0;; - esac - fi - if [ "$add_backports" = 1 ]; then - $SUDO sh -c "echo $BACKPORT_SOURCELINE >> /etc/apt/sources.list.d/$BACKPORT_NAME.list" - $SUDO apt-get update - fi - fi - fi - if [ "$add_backports" != 0 ]; then - $SUDO apt-get install $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg - augeas_pkg= + # ARGS: + BACKPORT_NAME="$1" + BACKPORT_SOURCELINE="$2" + echo "To use the Apache Certbot plugin, augeas needs to be installed from $BACKPORT_NAME." + if ! grep -v -e ' *#' /etc/apt/sources.list | grep -q "$BACKPORT_NAME" ; then + # This can theoretically error if sources.list.d is empty, but in that case we don't care. + if ! grep -v -e ' *#' /etc/apt/sources.list.d/* 2>/dev/null | grep -q "$BACKPORT_NAME"; then + if [ "$ASSUME_YES" = 1 ]; then + /bin/echo -n "Installing augeas from $BACKPORT_NAME in 3 seconds..." + sleep 1s + /bin/echo -ne "\e[0K\rInstalling augeas from $BACKPORT_NAME in 2 seconds..." + sleep 1s + /bin/echo -e "\e[0K\rInstalling augeas from $BACKPORT_NAME in 1 second ..." + sleep 1s + add_backports=1 + else + read -p "Would you like to enable the $BACKPORT_NAME repository [Y/n]? " response + case $response in + [yY][eE][sS]|[yY]|"") + add_backports=1;; + *) + add_backports=0;; + esac + fi + if [ "$add_backports" = 1 ]; then + $SUDO sh -c "echo $BACKPORT_SOURCELINE >> /etc/apt/sources.list.d/$BACKPORT_NAME.list" + $SUDO apt-get $QUIET_FLAG update + fi fi + fi + if [ "$add_backports" != 0 ]; then + $SUDO apt-get install $QUIET_FLAG $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg + augeas_pkg= + fi } if dpkg --compare-versions 1.0 gt "$AUGVERSION" ; then - if lsb_release -a | grep -q wheezy ; then - AddBackportRepo wheezy-backports "deb http://http.debian.net/debian wheezy-backports main" - elif lsb_release -a | grep -q precise ; then - # XXX add ARM case - AddBackportRepo precise-backports "deb http://archive.ubuntu.com/ubuntu precise-backports main restricted universe multiverse" - else - echo "No libaugeas0 version is available that's new enough to run the" - echo "Certbot apache plugin..." - fi - # XXX add a case for ubuntu PPAs + if lsb_release -a | grep -q wheezy ; then + AddBackportRepo wheezy-backports "deb http://http.debian.net/debian wheezy-backports main" + elif lsb_release -a | grep -q precise ; then + # XXX add ARM case + AddBackportRepo precise-backports "deb http://archive.ubuntu.com/ubuntu precise-backports main restricted universe multiverse" + else + echo "No libaugeas0 version is available that's new enough to run the" + echo "Certbot apache plugin..." + fi + # XXX add a case for ubuntu PPAs fi - $SUDO apt-get install $YES_FLAG --no-install-recommends \ + $SUDO apt-get install $QUIET_FLAG $YES_FLAG --no-install-recommends \ python \ python-dev \ $virtualenv \ @@ -294,7 +304,6 @@ BootstrapDebCommon() { ca-certificates \ - if ! $EXISTS virtualenv > /dev/null ; then echo Failed to install a working \"virtualenv\" command, exiting exit 1 @@ -323,6 +332,9 @@ BootstrapRpmCommon() { if [ "$ASSUME_YES" = 1 ]; then yes_flag="-y" fi + if [ "$QUIET" = 1 ]; then + QUIET_FLAG='--quiet' + fi if ! $SUDO $tool list *virtualenv >/dev/null 2>&1; then echo "To use Certbot, packages from the EPEL repository need to be installed." @@ -331,14 +343,14 @@ BootstrapRpmCommon() { exit 1 fi if [ "$ASSUME_YES" = 1 ]; then - /bin/echo -n "Enabling the EPEL repository in 3 seconds..." - sleep 1s - /bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..." - sleep 1s - /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 seconds..." - sleep 1s + /bin/echo -n "Enabling the EPEL repository in 3 seconds..." + sleep 1s + /bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..." + sleep 1s + /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 seconds..." + sleep 1s fi - if ! $SUDO $tool install $yes_flag epel-release; then + if ! $SUDO $tool install $yes_flag $QUIET_FLAG epel-release; then echo "Could not enable EPEL. Aborting bootstrap!" exit 1 fi @@ -380,9 +392,9 @@ BootstrapRpmCommon() { " fi - if ! $SUDO $tool install $yes_flag $pkgs; then - echo "Could not install OS dependencies. Aborting bootstrap!" - exit 1 + if ! $SUDO $tool install $yes_flag $QUIET_FLAG $pkgs; then + echo "Could not install OS dependencies. Aborting bootstrap!" + exit 1 fi } @@ -394,7 +406,11 @@ BootstrapSuseCommon() { install_flags="-l" fi - $SUDO zypper $zypper_flags in $install_flags \ + if [ "$QUIET" = 1 ]; then + QUIET_FLAG='-qq' + fi + + $SUDO zypper $QUIET_FLAG $zypper_flags in $install_flags \ python \ python-devel \ python-virtualenv \ @@ -432,7 +448,11 @@ BootstrapArchCommon() { fi if [ "$missing" ]; then - $SUDO pacman -S --needed $missing $noconfirm + if [ "$QUIET" = 1]; then + $SUDO pacman -S --needed $missing $noconfirm > /dev/null + else + $SUDO pacman -S --needed $missing $noconfirm + fi fi } @@ -465,7 +485,11 @@ BootstrapGentooCommon() { } BootstrapFreeBsd() { - $SUDO pkg install -Ay \ + if [ "$QUIET" = 1 ]; then + QUIET_FLAG="--quiet" + fi + + $SUDO pkg install -Ay $QUIET_FLAG \ python \ py27-virtualenv \ augeas \ @@ -505,15 +529,15 @@ BootstrapMac() { fi if ! hash pip 2>/dev/null; then - echo "pip not installed" - echo "Installing pip..." - curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python + echo "pip not installed" + echo "Installing pip..." + curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python fi if ! hash virtualenv 2>/dev/null; then - echo "virtualenv not installed." - echo "Installing with pip..." - pip install virtualenv + echo "virtualenv not installed." + echo "Installing with pip..." + pip install virtualenv fi } @@ -523,26 +547,29 @@ BootstrapSmartOS() { } BootstrapMageiaCommon() { - if ! $SUDO urpmi --force \ - python \ - libpython-devel \ - python-virtualenv + if [ "$QUIET" = 1 ]; then + QUIET_FLAG='--quiet' + fi + + if ! $SUDO urpmi --force $QUIET_FLAG \ + python \ + libpython-devel \ + python-virtualenv then echo "Could not install Python dependencies. Aborting bootstrap!" exit 1 - fi + fi - if ! $SUDO urpmi --force \ - git \ - gcc \ - python-augeas \ - openssl \ - libopenssl-devel \ - libffi-devel \ - rootcerts + if ! $SUDO urpmi --force $QUIET_FLAG \ + git \ + gcc \ + python-augeas \ + libopenssl-devel \ + libffi-devel \ + rootcerts then - echo "Could not install additional dependencies. Aborting bootstrap!" - exit 1 + echo "Could not install additional dependencies. Aborting bootstrap!" + exit 1 fi } @@ -609,6 +636,11 @@ if [ "$1" = "--le-auto-phase2" ]; then # --version output ran through grep due to python-cryptography DeprecationWarnings # grep for both certbot and letsencrypt until certbot and shim packages have been released INSTALLED_VERSION=$("$VENV_BIN/letsencrypt" --version 2>&1 | grep "^certbot\|^letsencrypt" | cut -d " " -f 2) + if [ -z "$INSTALLED_VERSION" ]; then + echo "Error: couldn't get currently installed version for $VENV_BIN/letsencrypt: " 1>&2 + "$VENV_BIN/letsencrypt" --version + exit 1 + fi else INSTALLED_VERSION="none" fi @@ -801,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.2 \ - --hash=sha256:053bbf8d36be298e411e0fa397289307e1a888f419de1205931236114f8f825e \ - --hash=sha256:6465b66256849f8cd4269a43037d404b1ccf31a60d3c93fc628d6f6f4c43de78 -certbot==0.10.2 \ - --hash=sha256:1e680cab3cda6c3079ee64066903f93e421df46a8d4db63264a6359f3d5ba63e \ - --hash=sha256:33729ce100fc9d00d9820c81fc3ef9b95802b8d261f0cca0cce422bc568b1631 -certbot-apache==0.10.2 \ - --hash=sha256:c1f196506bf961e2171fb2af9feca8926100564b644acda1895acdf294c46e71 \ - --hash=sha256:8573275e74c8b74a9c7d7824636a37a4bc83327a7cc97a43b334fe8956ba8cb1 -certbot-nginx==0.10.2 \ - --hash=sha256:74c32bdaf2f3873aa811c3e2cdc205d3137b59c64dbcdca6f3982ed2a89b4f2a \ - --hash=sha256:c8265ab4b4bc0a6a3a81d7f0b8ff2606dd28ebbb3cd045635862286b97af2c36 +acme==0.11.0 \ + --hash=sha256:9c084f9a62241a11231af63266f2f12ad696be590393a4ab4974276a47d63404 \ + --hash=sha256:1513ae74ee8424c739a953a552890830315669d95d105469d1cff5b7db9ae888 +certbot==0.11.0 \ + --hash=sha256:cc94d890a8697b3bdbdc555bcf4ba93955f64324bc256b2ea710fd053fc03b8a \ + --hash=sha256:0f91fee360f9ce5e0584d0954fa3123832435f77f465915389032a90ac0248b1 +certbot-apache==0.11.0 \ + --hash=sha256:9a01883ca7e1159cff2a6e36bf97b83793c899c62944335f6ac2f59f9b3e8b5d \ + --hash=sha256:2b67871e7ae8bbfa7a2779fcd6444f28847fbb7a347ef4bfdb19fb55a0d75673 +certbot-nginx==0.11.0 \ + --hash=sha256:cfae45a42560e39889eebd287437556084c421a9e07a2deb3cbc0aeef92d2dab \ + --hash=sha256:dbddffe47c8e8b2d1cf47fe393b434003270a45aec896f133492c855c77e6f08 UNLIKELY_EOF # ------------------------------------------------------------------------- @@ -1024,7 +1056,7 @@ UNLIKELY_EOF fi else - # Phase 1: Upgrade certbot-auto if neceesary, then self-invoke. + # Phase 1: Upgrade certbot-auto if necessary, then self-invoke. # # Each phase checks the version of only the thing it is responsible for # upgrading. Phase 1 checks the version of the latest release of diff --git a/certbot-compatibility-test/setup.py b/certbot-compatibility-test/setup.py index e515ede7b..d7c7f2531 100644 --- a/certbot-compatibility-test/setup.py +++ b/certbot-compatibility-test/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.0.dev0' +version = '0.11.0' install_requires = [ 'certbot', diff --git a/certbot-nginx/setup.py b/certbot-nginx/setup.py index e140c75d2..48c7080df 100644 --- a/certbot-nginx/setup.py +++ b/certbot-nginx/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.0.dev0' +version = '0.11.0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot/__init__.py b/certbot/__init__.py index 376b0504f..de4097a9b 100644 --- a/certbot/__init__.py +++ b/certbot/__init__.py @@ -1,4 +1,4 @@ """Certbot client.""" # version number like 1.2.3a0, must have at least 2 parts, like 1.2 -__version__ = '0.11.0.dev0' +__version__ = '0.11.0' diff --git a/docs/cli-help.txt b/docs/cli-help.txt index 9e1dbc268..5cc6c50a9 100644 --- a/docs/cli-help.txt +++ b/docs/cli-help.txt @@ -86,7 +86,7 @@ optional arguments: statistics about success rates by OS and plugin. If you wish to hide your server OS version from the Let's Encrypt server, set this to "". (default: - CertbotACMEClient/0.10.2 (Ubuntu 16.04.1 LTS) + CertbotACMEClient/0.11.0 (Ubuntu 16.04.1 LTS) Authenticator/XXX Installer/YYY) automation: @@ -120,7 +120,6 @@ automation: system. This option cannot be used with --csr. (default: False) --agree-tos Agree to the ACME Subscriber Agreement (default: Ask) - --account ACCOUNT_ID Account ID to use (default: None) --duplicate Allow making a certificate lineage that duplicates an existing one (both can be renewed in parallel) (default: False) @@ -280,6 +279,9 @@ delete: revoke: Options for revocation of certs + --reason {keycompromise,affiliationchanged,superseded,unspecified,cessationofoperation} + Specify reason for revoking certificate. (default: 0) + register: Options for account registration & modification @@ -301,6 +303,14 @@ register: -m EMAIL, --email EMAIL Email used for registration and recovery contact. (default: Ask) + --eff-email Share your e-mail address with EFF (default: None) + --no-eff-email Don't share your e-mail address with EFF (default: + None) + +unregister: + Options for account deactivation. + + --account ACCOUNT_ID Account ID to use (default: None) install: Options for modifying how a cert is deployed diff --git a/letsencrypt-auto b/letsencrypt-auto index 4e27bbd52..90b8157fe 100755 --- a/letsencrypt-auto +++ b/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.10.2" +LE_AUTO_VERSION="0.11.0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -38,8 +38,9 @@ Help for certbot itself cannot be provided until it is installed. -n, --non-interactive, --noninteractive run without asking for user input --no-self-upgrade do not download updates --os-packages-only install OS dependencies and exit - -q, --quiet provide only update/error output -v, --verbose provide more output + -q, --quiet provide only update/error output; + implies --non-interactive All arguments are accepted and forwarded to the Certbot client when run." @@ -84,6 +85,11 @@ if [ $BASENAME = "letsencrypt-auto" ]; then HELP=0 fi +# Set ASSUME_YES to 1 if QUIET (i.e. --quiet implies --non-interactive) +if [ "$QUIET" = 1 ]; then + ASSUME_YES=1 +fi + # Support for busybox and others where there is no "command", # but "which" instead if command -v command > /dev/null 2>&1 ; then @@ -99,7 +105,7 @@ fi # certbot itself needs root access for almost all modes of operation # The "normal" case is that sudo is used for the steps that need root, but # this script *can* be run as root (not recommended), or fall back to using -# `su`. Auto-detection can be overrided by explicitly setting the +# `su`. Auto-detection can be overridden by explicitly setting the # environment variable LE_AUTO_SUDO to 'sudo', 'sudo_su' or '' as used below. # Because the parameters in `su -c` has to be a string, @@ -207,7 +213,11 @@ BootstrapDebCommon() { # # - Debian 6.0.10 "squeeze" (x64) - $SUDO apt-get update || echo apt-get update hit problems but continuing anyway... + if [ "$QUIET" = 1 ]; then + QUIET_FLAG='-qq' + fi + + $SUDO apt-get $QUIET_FLAG update || echo apt-get update hit problems but continuing anyway... # virtualenv binary can be found in different packages depending on # distro version (#346) @@ -215,74 +225,74 @@ BootstrapDebCommon() { virtualenv= # virtual env is known to apt and is installable if apt-cache show virtualenv > /dev/null 2>&1 ; then - if ! LC_ALL=C apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then - virtualenv="virtualenv" - fi + if ! LC_ALL=C apt-cache --quiet=0 show virtualenv 2>&1 | grep -q 'No packages found'; then + virtualenv="virtualenv" + fi fi if apt-cache show python-virtualenv > /dev/null 2>&1; then - virtualenv="$virtualenv python-virtualenv" + virtualenv="$virtualenv python-virtualenv" fi augeas_pkg="libaugeas0 augeas-lenses" AUGVERSION=`LC_ALL=C apt-cache show --no-all-versions libaugeas0 | grep ^Version: | cut -d" " -f2` if [ "$ASSUME_YES" = 1 ]; then - YES_FLAG="-y" + YES_FLAG="-y" fi AddBackportRepo() { - # ARGS: - BACKPORT_NAME="$1" - BACKPORT_SOURCELINE="$2" - echo "To use the Apache Certbot plugin, augeas needs to be installed from $BACKPORT_NAME." - if ! grep -v -e ' *#' /etc/apt/sources.list | grep -q "$BACKPORT_NAME" ; then - # This can theoretically error if sources.list.d is empty, but in that case we don't care. - if ! grep -v -e ' *#' /etc/apt/sources.list.d/* 2>/dev/null | grep -q "$BACKPORT_NAME"; then - if [ "$ASSUME_YES" = 1 ]; then - /bin/echo -n "Installing augeas from $BACKPORT_NAME in 3 seconds..." - sleep 1s - /bin/echo -ne "\e[0K\rInstalling augeas from $BACKPORT_NAME in 2 seconds..." - sleep 1s - /bin/echo -e "\e[0K\rInstalling augeas from $BACKPORT_NAME in 1 second ..." - sleep 1s - add_backports=1 - else - read -p "Would you like to enable the $BACKPORT_NAME repository [Y/n]? " response - case $response in - [yY][eE][sS]|[yY]|"") - add_backports=1;; - *) - add_backports=0;; - esac - fi - if [ "$add_backports" = 1 ]; then - $SUDO sh -c "echo $BACKPORT_SOURCELINE >> /etc/apt/sources.list.d/$BACKPORT_NAME.list" - $SUDO apt-get update - fi - fi - fi - if [ "$add_backports" != 0 ]; then - $SUDO apt-get install $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg - augeas_pkg= + # ARGS: + BACKPORT_NAME="$1" + BACKPORT_SOURCELINE="$2" + echo "To use the Apache Certbot plugin, augeas needs to be installed from $BACKPORT_NAME." + if ! grep -v -e ' *#' /etc/apt/sources.list | grep -q "$BACKPORT_NAME" ; then + # This can theoretically error if sources.list.d is empty, but in that case we don't care. + if ! grep -v -e ' *#' /etc/apt/sources.list.d/* 2>/dev/null | grep -q "$BACKPORT_NAME"; then + if [ "$ASSUME_YES" = 1 ]; then + /bin/echo -n "Installing augeas from $BACKPORT_NAME in 3 seconds..." + sleep 1s + /bin/echo -ne "\e[0K\rInstalling augeas from $BACKPORT_NAME in 2 seconds..." + sleep 1s + /bin/echo -e "\e[0K\rInstalling augeas from $BACKPORT_NAME in 1 second ..." + sleep 1s + add_backports=1 + else + read -p "Would you like to enable the $BACKPORT_NAME repository [Y/n]? " response + case $response in + [yY][eE][sS]|[yY]|"") + add_backports=1;; + *) + add_backports=0;; + esac + fi + if [ "$add_backports" = 1 ]; then + $SUDO sh -c "echo $BACKPORT_SOURCELINE >> /etc/apt/sources.list.d/$BACKPORT_NAME.list" + $SUDO apt-get $QUIET_FLAG update + fi fi + fi + if [ "$add_backports" != 0 ]; then + $SUDO apt-get install $QUIET_FLAG $YES_FLAG --no-install-recommends -t "$BACKPORT_NAME" $augeas_pkg + augeas_pkg= + fi } if dpkg --compare-versions 1.0 gt "$AUGVERSION" ; then - if lsb_release -a | grep -q wheezy ; then - AddBackportRepo wheezy-backports "deb http://http.debian.net/debian wheezy-backports main" - elif lsb_release -a | grep -q precise ; then - # XXX add ARM case - AddBackportRepo precise-backports "deb http://archive.ubuntu.com/ubuntu precise-backports main restricted universe multiverse" - else - echo "No libaugeas0 version is available that's new enough to run the" - echo "Certbot apache plugin..." - fi - # XXX add a case for ubuntu PPAs + if lsb_release -a | grep -q wheezy ; then + AddBackportRepo wheezy-backports "deb http://http.debian.net/debian wheezy-backports main" + elif lsb_release -a | grep -q precise ; then + # XXX add ARM case + AddBackportRepo precise-backports "deb http://archive.ubuntu.com/ubuntu precise-backports main restricted universe multiverse" + else + echo "No libaugeas0 version is available that's new enough to run the" + echo "Certbot apache plugin..." + fi + # XXX add a case for ubuntu PPAs fi - $SUDO apt-get install $YES_FLAG --no-install-recommends \ + $SUDO apt-get install $QUIET_FLAG $YES_FLAG --no-install-recommends \ python \ python-dev \ $virtualenv \ @@ -294,7 +304,6 @@ BootstrapDebCommon() { ca-certificates \ - if ! $EXISTS virtualenv > /dev/null ; then echo Failed to install a working \"virtualenv\" command, exiting exit 1 @@ -323,6 +332,9 @@ BootstrapRpmCommon() { if [ "$ASSUME_YES" = 1 ]; then yes_flag="-y" fi + if [ "$QUIET" = 1 ]; then + QUIET_FLAG='--quiet' + fi if ! $SUDO $tool list *virtualenv >/dev/null 2>&1; then echo "To use Certbot, packages from the EPEL repository need to be installed." @@ -331,14 +343,14 @@ BootstrapRpmCommon() { exit 1 fi if [ "$ASSUME_YES" = 1 ]; then - /bin/echo -n "Enabling the EPEL repository in 3 seconds..." - sleep 1s - /bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..." - sleep 1s - /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 seconds..." - sleep 1s + /bin/echo -n "Enabling the EPEL repository in 3 seconds..." + sleep 1s + /bin/echo -ne "\e[0K\rEnabling the EPEL repository in 2 seconds..." + sleep 1s + /bin/echo -e "\e[0K\rEnabling the EPEL repository in 1 seconds..." + sleep 1s fi - if ! $SUDO $tool install $yes_flag epel-release; then + if ! $SUDO $tool install $yes_flag $QUIET_FLAG epel-release; then echo "Could not enable EPEL. Aborting bootstrap!" exit 1 fi @@ -380,9 +392,9 @@ BootstrapRpmCommon() { " fi - if ! $SUDO $tool install $yes_flag $pkgs; then - echo "Could not install OS dependencies. Aborting bootstrap!" - exit 1 + if ! $SUDO $tool install $yes_flag $QUIET_FLAG $pkgs; then + echo "Could not install OS dependencies. Aborting bootstrap!" + exit 1 fi } @@ -394,7 +406,11 @@ BootstrapSuseCommon() { install_flags="-l" fi - $SUDO zypper $zypper_flags in $install_flags \ + if [ "$QUIET" = 1 ]; then + QUIET_FLAG='-qq' + fi + + $SUDO zypper $QUIET_FLAG $zypper_flags in $install_flags \ python \ python-devel \ python-virtualenv \ @@ -432,7 +448,11 @@ BootstrapArchCommon() { fi if [ "$missing" ]; then - $SUDO pacman -S --needed $missing $noconfirm + if [ "$QUIET" = 1]; then + $SUDO pacman -S --needed $missing $noconfirm > /dev/null + else + $SUDO pacman -S --needed $missing $noconfirm + fi fi } @@ -465,7 +485,11 @@ BootstrapGentooCommon() { } BootstrapFreeBsd() { - $SUDO pkg install -Ay \ + if [ "$QUIET" = 1 ]; then + QUIET_FLAG="--quiet" + fi + + $SUDO pkg install -Ay $QUIET_FLAG \ python \ py27-virtualenv \ augeas \ @@ -505,15 +529,15 @@ BootstrapMac() { fi if ! hash pip 2>/dev/null; then - echo "pip not installed" - echo "Installing pip..." - curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python + echo "pip not installed" + echo "Installing pip..." + curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python fi if ! hash virtualenv 2>/dev/null; then - echo "virtualenv not installed." - echo "Installing with pip..." - pip install virtualenv + echo "virtualenv not installed." + echo "Installing with pip..." + pip install virtualenv fi } @@ -523,26 +547,29 @@ BootstrapSmartOS() { } BootstrapMageiaCommon() { - if ! $SUDO urpmi --force \ - python \ - libpython-devel \ - python-virtualenv + if [ "$QUIET" = 1 ]; then + QUIET_FLAG='--quiet' + fi + + if ! $SUDO urpmi --force $QUIET_FLAG \ + python \ + libpython-devel \ + python-virtualenv then echo "Could not install Python dependencies. Aborting bootstrap!" exit 1 - fi + fi - if ! $SUDO urpmi --force \ - git \ - gcc \ - python-augeas \ - openssl \ - libopenssl-devel \ - libffi-devel \ - rootcerts + if ! $SUDO urpmi --force $QUIET_FLAG \ + git \ + gcc \ + python-augeas \ + libopenssl-devel \ + libffi-devel \ + rootcerts then - echo "Could not install additional dependencies. Aborting bootstrap!" - exit 1 + echo "Could not install additional dependencies. Aborting bootstrap!" + exit 1 fi } @@ -609,6 +636,11 @@ if [ "$1" = "--le-auto-phase2" ]; then # --version output ran through grep due to python-cryptography DeprecationWarnings # grep for both certbot and letsencrypt until certbot and shim packages have been released INSTALLED_VERSION=$("$VENV_BIN/letsencrypt" --version 2>&1 | grep "^certbot\|^letsencrypt" | cut -d " " -f 2) + if [ -z "$INSTALLED_VERSION" ]; then + echo "Error: couldn't get currently installed version for $VENV_BIN/letsencrypt: " 1>&2 + "$VENV_BIN/letsencrypt" --version + exit 1 + fi else INSTALLED_VERSION="none" fi @@ -801,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.2 \ - --hash=sha256:053bbf8d36be298e411e0fa397289307e1a888f419de1205931236114f8f825e \ - --hash=sha256:6465b66256849f8cd4269a43037d404b1ccf31a60d3c93fc628d6f6f4c43de78 -certbot==0.10.2 \ - --hash=sha256:1e680cab3cda6c3079ee64066903f93e421df46a8d4db63264a6359f3d5ba63e \ - --hash=sha256:33729ce100fc9d00d9820c81fc3ef9b95802b8d261f0cca0cce422bc568b1631 -certbot-apache==0.10.2 \ - --hash=sha256:c1f196506bf961e2171fb2af9feca8926100564b644acda1895acdf294c46e71 \ - --hash=sha256:8573275e74c8b74a9c7d7824636a37a4bc83327a7cc97a43b334fe8956ba8cb1 -certbot-nginx==0.10.2 \ - --hash=sha256:74c32bdaf2f3873aa811c3e2cdc205d3137b59c64dbcdca6f3982ed2a89b4f2a \ - --hash=sha256:c8265ab4b4bc0a6a3a81d7f0b8ff2606dd28ebbb3cd045635862286b97af2c36 +acme==0.11.0 \ + --hash=sha256:9c084f9a62241a11231af63266f2f12ad696be590393a4ab4974276a47d63404 \ + --hash=sha256:1513ae74ee8424c739a953a552890830315669d95d105469d1cff5b7db9ae888 +certbot==0.11.0 \ + --hash=sha256:cc94d890a8697b3bdbdc555bcf4ba93955f64324bc256b2ea710fd053fc03b8a \ + --hash=sha256:0f91fee360f9ce5e0584d0954fa3123832435f77f465915389032a90ac0248b1 +certbot-apache==0.11.0 \ + --hash=sha256:9a01883ca7e1159cff2a6e36bf97b83793c899c62944335f6ac2f59f9b3e8b5d \ + --hash=sha256:2b67871e7ae8bbfa7a2779fcd6444f28847fbb7a347ef4bfdb19fb55a0d75673 +certbot-nginx==0.11.0 \ + --hash=sha256:cfae45a42560e39889eebd287437556084c421a9e07a2deb3cbc0aeef92d2dab \ + --hash=sha256:dbddffe47c8e8b2d1cf47fe393b434003270a45aec896f133492c855c77e6f08 UNLIKELY_EOF # ------------------------------------------------------------------------- @@ -1024,7 +1056,7 @@ UNLIKELY_EOF fi else - # Phase 1: Upgrade certbot-auto if neceesary, then self-invoke. + # Phase 1: Upgrade certbot-auto if necessary, then self-invoke. # # Each phase checks the version of only the thing it is responsible for # upgrading. Phase 1 checks the version of the latest release of diff --git a/letsencrypt-auto-source/certbot-auto.asc b/letsencrypt-auto-source/certbot-auto.asc index a848695f6..baa5a7a9a 100644 --- a/letsencrypt-auto-source/certbot-auto.asc +++ b/letsencrypt-auto-source/certbot-auto.asc @@ -1,11 +1,11 @@ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 -iQEcBAABAgAGBQJYiWfOAAoJEE0XyZXNl3XyES8H/A77QJnVn4mNBv5+WDvOuxii -OVch92SHT9FSrUpM0QNAB981SzjSt6vAwjdFjqyMYmfrITYl5Kfo9UjHJY3gfYAR -S+oiSnBBJ1+9O1+s9kRt0/zwXS6iqIyT7gKM6ZQ5PPaKeB0Iw820m16QgOdMN6d3 -jBQlUIqg6Bl5y79znx8yAYXpIrbOj/Q1xcYpKjRcnyzgR7sWIGvJvVygtXcDN+QV -HUjtumjnqFON4+638N6xPeiJobTGgTwOLr91I0IhNWKCZ5hBgvm8Oq8r+cfZejCX -XIdtlvmcw08GiKsFY8WW7k9rK1/D8uy4Xx26KAAXmdY/xdyG9rOBHOui2N921nw= -=B8NE +iQEcBAABAgAGBQJYkh+9AAoJEE0XyZXNl3XyDFcH/RvYdrkHaLNfc7HX6cRgZvkM +9XgP5j+Fsb44NP/U7FwVNHfgqe7OSSSHzNuXPt4/wZTRtXgQZjKU4fWlg8OTgNe8 +MvTx7FL78GXOhMPC0DX3ZkYYOllApiAFHQuwOXroGb099PTr7msnatLunLk1yCUN +wx/i+z0PHkJp+VDDb71sNOIwDtSzRx8w8/dtnnlODkDQWbbijkMUfslmEZnd5bRH +7vd2miuDkSR2dMdYLi+Zx0tkSib06kR8ahakPEIcDuZ5CI3fRESkmRUjOxFZOt/z +KqMVLMeEq0P81anlK3QCslwibhC88BlbafEma/FNPBZdzHz4UcQALWDGtYDn2tg= +=+aXk -----END PGP SIGNATURE----- diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index 68f135bf8..90b8157fe 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.11.0.dev0" +LE_AUTO_VERSION="0.11.0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.2 \ - --hash=sha256:053bbf8d36be298e411e0fa397289307e1a888f419de1205931236114f8f825e \ - --hash=sha256:6465b66256849f8cd4269a43037d404b1ccf31a60d3c93fc628d6f6f4c43de78 -certbot==0.10.2 \ - --hash=sha256:1e680cab3cda6c3079ee64066903f93e421df46a8d4db63264a6359f3d5ba63e \ - --hash=sha256:33729ce100fc9d00d9820c81fc3ef9b95802b8d261f0cca0cce422bc568b1631 -certbot-apache==0.10.2 \ - --hash=sha256:c1f196506bf961e2171fb2af9feca8926100564b644acda1895acdf294c46e71 \ - --hash=sha256:8573275e74c8b74a9c7d7824636a37a4bc83327a7cc97a43b334fe8956ba8cb1 -certbot-nginx==0.10.2 \ - --hash=sha256:74c32bdaf2f3873aa811c3e2cdc205d3137b59c64dbcdca6f3982ed2a89b4f2a \ - --hash=sha256:c8265ab4b4bc0a6a3a81d7f0b8ff2606dd28ebbb3cd045635862286b97af2c36 +acme==0.11.0 \ + --hash=sha256:9c084f9a62241a11231af63266f2f12ad696be590393a4ab4974276a47d63404 \ + --hash=sha256:1513ae74ee8424c739a953a552890830315669d95d105469d1cff5b7db9ae888 +certbot==0.11.0 \ + --hash=sha256:cc94d890a8697b3bdbdc555bcf4ba93955f64324bc256b2ea710fd053fc03b8a \ + --hash=sha256:0f91fee360f9ce5e0584d0954fa3123832435f77f465915389032a90ac0248b1 +certbot-apache==0.11.0 \ + --hash=sha256:9a01883ca7e1159cff2a6e36bf97b83793c899c62944335f6ac2f59f9b3e8b5d \ + --hash=sha256:2b67871e7ae8bbfa7a2779fcd6444f28847fbb7a347ef4bfdb19fb55a0d75673 +certbot-nginx==0.11.0 \ + --hash=sha256:cfae45a42560e39889eebd287437556084c421a9e07a2deb3cbc0aeef92d2dab \ + --hash=sha256:dbddffe47c8e8b2d1cf47fe393b434003270a45aec896f133492c855c77e6f08 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/letsencrypt-auto.sig b/letsencrypt-auto-source/letsencrypt-auto.sig index cbddf8c9bccde1ecc8c53254313d39060522be1e..3516f2b67efe931571be3a97b8edca56ba35fae5 100644 GIT binary patch literal 256 zcmV+b0ssExr2Xjv$alfP!Y{uaidteky))};1r&Diatp1TfZ-ci?OPf2nYzmQ>;^W{ zmv|RyR+GR$Gj4A=LBWI6E*pxsk%^@ah6W6MVr69{O7V7^?LU{)6{=pU$=CRxLw@X8*fA+k4V#U;0_|;E%-(aB`J&6#^>I2#*ahk|>&%JbR%&Tg z{ITE|@5<6)|9@)?hkgJv6H7{%Yv24l!T|CXTG{8ZH&XdsUzAOP9z6t3E55#sMDhVg GH41W)-Fo!^ literal 256 zcmV+b0ssDd$`Yq2MInxMh51elUnIKs*wdUV$0#9hQ=wglU`j)LD$A}{ioX7lwSwZ> z_!77WQ6IlFQ18g-*={1409g#f#CD+@O`HD|kne0eV&|SxIo}L&k*7%lck{g{A>6f!_?;cW3 z2fgTd_zjY$=J$iz&I7+pEYyX4qZrNz!b_X=IpIIEEPf>iEXtFqb9s% GZWaOsyMF8d diff --git a/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt b/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt index 3f448889d..de0b56c7e 100644 --- a/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt +++ b/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt @@ -171,15 +171,15 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.10.2 \ - --hash=sha256:053bbf8d36be298e411e0fa397289307e1a888f419de1205931236114f8f825e \ - --hash=sha256:6465b66256849f8cd4269a43037d404b1ccf31a60d3c93fc628d6f6f4c43de78 -certbot==0.10.2 \ - --hash=sha256:1e680cab3cda6c3079ee64066903f93e421df46a8d4db63264a6359f3d5ba63e \ - --hash=sha256:33729ce100fc9d00d9820c81fc3ef9b95802b8d261f0cca0cce422bc568b1631 -certbot-apache==0.10.2 \ - --hash=sha256:c1f196506bf961e2171fb2af9feca8926100564b644acda1895acdf294c46e71 \ - --hash=sha256:8573275e74c8b74a9c7d7824636a37a4bc83327a7cc97a43b334fe8956ba8cb1 -certbot-nginx==0.10.2 \ - --hash=sha256:74c32bdaf2f3873aa811c3e2cdc205d3137b59c64dbcdca6f3982ed2a89b4f2a \ - --hash=sha256:c8265ab4b4bc0a6a3a81d7f0b8ff2606dd28ebbb3cd045635862286b97af2c36 +acme==0.11.0 \ + --hash=sha256:9c084f9a62241a11231af63266f2f12ad696be590393a4ab4974276a47d63404 \ + --hash=sha256:1513ae74ee8424c739a953a552890830315669d95d105469d1cff5b7db9ae888 +certbot==0.11.0 \ + --hash=sha256:cc94d890a8697b3bdbdc555bcf4ba93955f64324bc256b2ea710fd053fc03b8a \ + --hash=sha256:0f91fee360f9ce5e0584d0954fa3123832435f77f465915389032a90ac0248b1 +certbot-apache==0.11.0 \ + --hash=sha256:9a01883ca7e1159cff2a6e36bf97b83793c899c62944335f6ac2f59f9b3e8b5d \ + --hash=sha256:2b67871e7ae8bbfa7a2779fcd6444f28847fbb7a347ef4bfdb19fb55a0d75673 +certbot-nginx==0.11.0 \ + --hash=sha256:cfae45a42560e39889eebd287437556084c421a9e07a2deb3cbc0aeef92d2dab \ + --hash=sha256:dbddffe47c8e8b2d1cf47fe393b434003270a45aec896f133492c855c77e6f08 From a6b51f1edd3b9002f82950469cc9d39a154f9494 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 1 Feb 2017 09:50:57 -0800 Subject: [PATCH 46/88] Bump version to 0.12.0 --- acme/setup.py | 2 +- certbot-apache/setup.py | 2 +- certbot-compatibility-test/setup.py | 2 +- certbot-nginx/setup.py | 2 +- certbot/__init__.py | 2 +- letsencrypt-auto-source/letsencrypt-auto | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/acme/setup.py b/acme/setup.py index 6b7badbd6..d1e91e5ec 100644 --- a/acme/setup.py +++ b/acme/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.0' +version = '0.12.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-apache/setup.py b/certbot-apache/setup.py index 957b31c52..87ea1a281 100644 --- a/certbot-apache/setup.py +++ b/certbot-apache/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.0' +version = '0.12.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-compatibility-test/setup.py b/certbot-compatibility-test/setup.py index d7c7f2531..e2d226a72 100644 --- a/certbot-compatibility-test/setup.py +++ b/certbot-compatibility-test/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.0' +version = '0.12.0.dev0' install_requires = [ 'certbot', diff --git a/certbot-nginx/setup.py b/certbot-nginx/setup.py index 48c7080df..6de2dc6bd 100644 --- a/certbot-nginx/setup.py +++ b/certbot-nginx/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.0' +version = '0.12.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot/__init__.py b/certbot/__init__.py index de4097a9b..6451eb0d5 100644 --- a/certbot/__init__.py +++ b/certbot/__init__.py @@ -1,4 +1,4 @@ """Certbot client.""" # version number like 1.2.3a0, must have at least 2 parts, like 1.2 -__version__ = '0.11.0' +__version__ = '0.12.0.dev0' diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index 90b8157fe..bd519ff61 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.11.0" +LE_AUTO_VERSION="0.12.0.dev0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates From b5d4e0bf6a50872462b99f4f02b3688d1179b2f6 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 1 Feb 2017 18:02:21 -0800 Subject: [PATCH 47/88] fix bug when certbot is ran without any arguments (#4151) --- certbot/cli.py | 2 +- certbot/tests/cli_test.py | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/certbot/cli.py b/certbot/cli.py index a24f566d1..d7fda7574 100644 --- a/certbot/cli.py +++ b/certbot/cli.py @@ -439,7 +439,7 @@ class HelpfulArgumentParser(object): self.detect_defaults = detect_defaults self.args = args - if self.args[0] == 'help': + if self.args and self.args[0] == 'help': self.args[0] = '--help' self.determine_verb() diff --git a/certbot/tests/cli_test.py b/certbot/tests/cli_test.py index d157901e4..5f4a4e2c7 100644 --- a/certbot/tests/cli_test.py +++ b/certbot/tests/cli_test.py @@ -60,6 +60,11 @@ class ParseTest(unittest.TestCase): self.assertRaises(SystemExit, self.parse, args, output) return output.getvalue() + def test_no_args(self): + namespace = self.parse([]) + for d in ('config_dir', 'logs_dir', 'work_dir'): + self.assertEqual(getattr(namespace, d), cli.flag_default(d)) + def test_install_abspath(self): cert = 'cert' key = 'key' From dd8253b3d69329b6f23662eab4499a17dc4da9e7 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 1 Feb 2017 18:14:47 -0800 Subject: [PATCH 48/88] fixes #4083 (#4148) --- .travis.yml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index b35f0ebbf..ca06f07d0 100644 --- a/.travis.yml +++ b/.travis.yml @@ -94,11 +94,13 @@ matrix: # Only build pushes to the master branch, PRs, and branches beginning with -# `test-`. This reduces the number of simultaneous Travis runs, which speeds -# turnaround time on review since there is a cap of 5 simultaneous runs. +# `test-` or of the form `digit(s).digit(s).x`. This reduces the number of +# simultaneous Travis runs, which speeds turnaround time on review since there +# is a cap of on the number of simultaneous runs. branches: only: - master + - /^\d+\.\d+\.x$/ - /^test-.*$/ # container-based infrastructure From f12dc84271702aa11a4caca561f962d8f48bcac8 Mon Sep 17 00:00:00 2001 From: Ben Wolfe Date: Wed, 1 Feb 2017 21:37:59 -0500 Subject: [PATCH 49/88] Certbot spelling correction (#4157) --- certbot/eff.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/certbot/eff.py b/certbot/eff.py index 0ef23cc8c..746261faa 100644 --- a/certbot/eff.py +++ b/certbot/eff.py @@ -41,7 +41,7 @@ def _want_subscription(): 'Would you be willing to share your email address with the ' "Electronic Frontier Foundation, a founding partner of the Let's " 'Encrypt project and the non-profit organization that develops ' - "Cerbot? We'd like to send you email about EFF and our work to " + "Certbot? We'd like to send you email about EFF and our work to " 'encrypt the web, protect its users and defend digital rights.') display = zope.component.getUtility(interfaces.IDisplay) return display.yesno(prompt, default=False) From 99818bc6b7ad5d121cf67bd40119676006dbc0e1 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 1 Feb 2017 19:36:47 -0800 Subject: [PATCH 50/88] Release 0.11.1 (cherry picked from commit 426a804561f2b65d780ceccbb07f6c5f35b9bacf) --- acme/setup.py | 2 +- certbot-apache/setup.py | 2 +- certbot-auto | 26 +++++++++--------- certbot-compatibility-test/setup.py | 2 +- certbot-nginx/setup.py | 2 +- certbot/__init__.py | 2 +- docs/cli-help.txt | 2 +- letsencrypt-auto | 26 +++++++++--------- letsencrypt-auto-source/certbot-auto.asc | 14 +++++----- letsencrypt-auto-source/letsencrypt-auto | 26 +++++++++--------- letsencrypt-auto-source/letsencrypt-auto.sig | Bin 256 -> 256 bytes .../pieces/letsencrypt-auto-requirements.txt | 24 ++++++++-------- 12 files changed, 64 insertions(+), 64 deletions(-) diff --git a/acme/setup.py b/acme/setup.py index d1e91e5ec..338e6fff4 100644 --- a/acme/setup.py +++ b/acme/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.11.1' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-apache/setup.py b/certbot-apache/setup.py index 87ea1a281..01f3b9b83 100644 --- a/certbot-apache/setup.py +++ b/certbot-apache/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.11.1' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-auto b/certbot-auto index 90b8157fe..ac721638f 100755 --- a/certbot-auto +++ b/certbot-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.11.0" +LE_AUTO_VERSION="0.11.1" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.0 \ - --hash=sha256:9c084f9a62241a11231af63266f2f12ad696be590393a4ab4974276a47d63404 \ - --hash=sha256:1513ae74ee8424c739a953a552890830315669d95d105469d1cff5b7db9ae888 -certbot==0.11.0 \ - --hash=sha256:cc94d890a8697b3bdbdc555bcf4ba93955f64324bc256b2ea710fd053fc03b8a \ - --hash=sha256:0f91fee360f9ce5e0584d0954fa3123832435f77f465915389032a90ac0248b1 -certbot-apache==0.11.0 \ - --hash=sha256:9a01883ca7e1159cff2a6e36bf97b83793c899c62944335f6ac2f59f9b3e8b5d \ - --hash=sha256:2b67871e7ae8bbfa7a2779fcd6444f28847fbb7a347ef4bfdb19fb55a0d75673 -certbot-nginx==0.11.0 \ - --hash=sha256:cfae45a42560e39889eebd287437556084c421a9e07a2deb3cbc0aeef92d2dab \ - --hash=sha256:dbddffe47c8e8b2d1cf47fe393b434003270a45aec896f133492c855c77e6f08 +acme==0.11.1 \ + --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ + --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e +certbot==0.11.1 \ + --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ + --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 +certbot-apache==0.11.1 \ + --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ + --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 +certbot-nginx==0.11.1 \ + --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ + --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/certbot-compatibility-test/setup.py b/certbot-compatibility-test/setup.py index e2d226a72..822c65e88 100644 --- a/certbot-compatibility-test/setup.py +++ b/certbot-compatibility-test/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.11.1' install_requires = [ 'certbot', diff --git a/certbot-nginx/setup.py b/certbot-nginx/setup.py index 6de2dc6bd..fb3eedb57 100644 --- a/certbot-nginx/setup.py +++ b/certbot-nginx/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.11.1' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot/__init__.py b/certbot/__init__.py index 6451eb0d5..36cb9a713 100644 --- a/certbot/__init__.py +++ b/certbot/__init__.py @@ -1,4 +1,4 @@ """Certbot client.""" # version number like 1.2.3a0, must have at least 2 parts, like 1.2 -__version__ = '0.12.0.dev0' +__version__ = '0.11.1' diff --git a/docs/cli-help.txt b/docs/cli-help.txt index 5cc6c50a9..4cb408002 100644 --- a/docs/cli-help.txt +++ b/docs/cli-help.txt @@ -86,7 +86,7 @@ optional arguments: statistics about success rates by OS and plugin. If you wish to hide your server OS version from the Let's Encrypt server, set this to "". (default: - CertbotACMEClient/0.11.0 (Ubuntu 16.04.1 LTS) + CertbotACMEClient/0.11.1 (Ubuntu 16.04.1 LTS) Authenticator/XXX Installer/YYY) automation: diff --git a/letsencrypt-auto b/letsencrypt-auto index 90b8157fe..ac721638f 100755 --- a/letsencrypt-auto +++ b/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.11.0" +LE_AUTO_VERSION="0.11.1" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.0 \ - --hash=sha256:9c084f9a62241a11231af63266f2f12ad696be590393a4ab4974276a47d63404 \ - --hash=sha256:1513ae74ee8424c739a953a552890830315669d95d105469d1cff5b7db9ae888 -certbot==0.11.0 \ - --hash=sha256:cc94d890a8697b3bdbdc555bcf4ba93955f64324bc256b2ea710fd053fc03b8a \ - --hash=sha256:0f91fee360f9ce5e0584d0954fa3123832435f77f465915389032a90ac0248b1 -certbot-apache==0.11.0 \ - --hash=sha256:9a01883ca7e1159cff2a6e36bf97b83793c899c62944335f6ac2f59f9b3e8b5d \ - --hash=sha256:2b67871e7ae8bbfa7a2779fcd6444f28847fbb7a347ef4bfdb19fb55a0d75673 -certbot-nginx==0.11.0 \ - --hash=sha256:cfae45a42560e39889eebd287437556084c421a9e07a2deb3cbc0aeef92d2dab \ - --hash=sha256:dbddffe47c8e8b2d1cf47fe393b434003270a45aec896f133492c855c77e6f08 +acme==0.11.1 \ + --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ + --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e +certbot==0.11.1 \ + --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ + --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 +certbot-apache==0.11.1 \ + --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ + --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 +certbot-nginx==0.11.1 \ + --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ + --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/certbot-auto.asc b/letsencrypt-auto-source/certbot-auto.asc index baa5a7a9a..44e2246b7 100644 --- a/letsencrypt-auto-source/certbot-auto.asc +++ b/letsencrypt-auto-source/certbot-auto.asc @@ -1,11 +1,11 @@ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 -iQEcBAABAgAGBQJYkh+9AAoJEE0XyZXNl3XyDFcH/RvYdrkHaLNfc7HX6cRgZvkM -9XgP5j+Fsb44NP/U7FwVNHfgqe7OSSSHzNuXPt4/wZTRtXgQZjKU4fWlg8OTgNe8 -MvTx7FL78GXOhMPC0DX3ZkYYOllApiAFHQuwOXroGb099PTr7msnatLunLk1yCUN -wx/i+z0PHkJp+VDDb71sNOIwDtSzRx8w8/dtnnlODkDQWbbijkMUfslmEZnd5bRH -7vd2miuDkSR2dMdYLi+Zx0tkSib06kR8ahakPEIcDuZ5CI3fRESkmRUjOxFZOt/z -KqMVLMeEq0P81anlK3QCslwibhC88BlbafEma/FNPBZdzHz4UcQALWDGtYDn2tg= -=+aXk +iQEcBAABAgAGBQJYkqlBAAoJEE0XyZXNl3XyXUAH/RqwKDOpceChSdH/aIk891HX +VRDBQxIjZ6EB1iyebfihyZd4a5zGJ9ocMj1GxThMyLgSKbgkSRtjE/+ymDWsL0Us +Y8w9fw76BAImaJZEvjkrpqD2bSYdijnF479hBa/huZHKcQhb/sqxkNJO9SO1uj8z +bnF0UjJNjgn1hm2yHNMWwyEX7xCN/Vxiq/Zwqi7HdPus99sInJA7+04nwXaUtash +87MHpCjIiHh3axOCOjJbAWzIfsDUKeaBHgeYO+2ldOPWVQ0Amp7ghXjohryBkiux +dqhhAuvBTmNqPrbPAjdJ7Kd74NOGDo3HvAUiuXIckDWqxX2Q34w5pwxelZcIEnI= +=vmVS -----END PGP SIGNATURE----- diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index bd519ff61..ac721638f 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.12.0.dev0" +LE_AUTO_VERSION="0.11.1" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.0 \ - --hash=sha256:9c084f9a62241a11231af63266f2f12ad696be590393a4ab4974276a47d63404 \ - --hash=sha256:1513ae74ee8424c739a953a552890830315669d95d105469d1cff5b7db9ae888 -certbot==0.11.0 \ - --hash=sha256:cc94d890a8697b3bdbdc555bcf4ba93955f64324bc256b2ea710fd053fc03b8a \ - --hash=sha256:0f91fee360f9ce5e0584d0954fa3123832435f77f465915389032a90ac0248b1 -certbot-apache==0.11.0 \ - --hash=sha256:9a01883ca7e1159cff2a6e36bf97b83793c899c62944335f6ac2f59f9b3e8b5d \ - --hash=sha256:2b67871e7ae8bbfa7a2779fcd6444f28847fbb7a347ef4bfdb19fb55a0d75673 -certbot-nginx==0.11.0 \ - --hash=sha256:cfae45a42560e39889eebd287437556084c421a9e07a2deb3cbc0aeef92d2dab \ - --hash=sha256:dbddffe47c8e8b2d1cf47fe393b434003270a45aec896f133492c855c77e6f08 +acme==0.11.1 \ + --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ + --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e +certbot==0.11.1 \ + --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ + --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 +certbot-apache==0.11.1 \ + --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ + --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 +certbot-nginx==0.11.1 \ + --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ + --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/letsencrypt-auto.sig b/letsencrypt-auto-source/letsencrypt-auto.sig index 3516f2b67efe931571be3a97b8edca56ba35fae5..7aea42dd07338009aeb411d16b1ede2e47242c4a 100644 GIT binary patch literal 256 zcmV+b0ssE0LMkrq1<;I$4(Z|boMIIj?&t!1cp>;Yh?HTs)6}s^=SfcWN;bWgd`EXf zEhtU0e_$I^0tEloE_&&fmjX6Hni@Oow|k9sJt&yBk3)F3fvE|SyJe#y@%&-O3e7`~ z73zWb(bivi7Ot+2)kS>^Ui$C!RU<;^W{ zmv|RyR+GR$Gj4A=LBWI6E*pxsk%^@ah6W6MVr69{O7V7^?LU{)6{=pU$=CRxLw@X8*fA+k4V#U;0_|;E%-(aB`J&6#^>I2#*ahk|>&%JbR%&Tg z{ITE|@5<6)|9@)?hkgJv6H7{%Yv24l!T|CXTG{8ZH&XdsUzAOP9z6t3E55#sMDhVg GH41W)-Fo!^ diff --git a/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt b/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt index de0b56c7e..36d93dddd 100644 --- a/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt +++ b/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt @@ -171,15 +171,15 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.0 \ - --hash=sha256:9c084f9a62241a11231af63266f2f12ad696be590393a4ab4974276a47d63404 \ - --hash=sha256:1513ae74ee8424c739a953a552890830315669d95d105469d1cff5b7db9ae888 -certbot==0.11.0 \ - --hash=sha256:cc94d890a8697b3bdbdc555bcf4ba93955f64324bc256b2ea710fd053fc03b8a \ - --hash=sha256:0f91fee360f9ce5e0584d0954fa3123832435f77f465915389032a90ac0248b1 -certbot-apache==0.11.0 \ - --hash=sha256:9a01883ca7e1159cff2a6e36bf97b83793c899c62944335f6ac2f59f9b3e8b5d \ - --hash=sha256:2b67871e7ae8bbfa7a2779fcd6444f28847fbb7a347ef4bfdb19fb55a0d75673 -certbot-nginx==0.11.0 \ - --hash=sha256:cfae45a42560e39889eebd287437556084c421a9e07a2deb3cbc0aeef92d2dab \ - --hash=sha256:dbddffe47c8e8b2d1cf47fe393b434003270a45aec896f133492c855c77e6f08 +acme==0.11.1 \ + --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ + --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e +certbot==0.11.1 \ + --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ + --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 +certbot-apache==0.11.1 \ + --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ + --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 +certbot-nginx==0.11.1 \ + --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ + --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 From 245170ebe58c08d940f4b4b42dad4966227eaa76 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Wed, 1 Feb 2017 19:36:56 -0800 Subject: [PATCH 51/88] Bump version to 0.12.0 (cherry picked from commit f4eabe0908456b7ab2aef0ae03b5dc1db73d5805) --- acme/setup.py | 2 +- certbot-apache/setup.py | 2 +- certbot-compatibility-test/setup.py | 2 +- certbot-nginx/setup.py | 2 +- certbot/__init__.py | 2 +- letsencrypt-auto-source/letsencrypt-auto | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/acme/setup.py b/acme/setup.py index 338e6fff4..d1e91e5ec 100644 --- a/acme/setup.py +++ b/acme/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.1' +version = '0.12.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-apache/setup.py b/certbot-apache/setup.py index 01f3b9b83..87ea1a281 100644 --- a/certbot-apache/setup.py +++ b/certbot-apache/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.1' +version = '0.12.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-compatibility-test/setup.py b/certbot-compatibility-test/setup.py index 822c65e88..e2d226a72 100644 --- a/certbot-compatibility-test/setup.py +++ b/certbot-compatibility-test/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.1' +version = '0.12.0.dev0' install_requires = [ 'certbot', diff --git a/certbot-nginx/setup.py b/certbot-nginx/setup.py index fb3eedb57..6de2dc6bd 100644 --- a/certbot-nginx/setup.py +++ b/certbot-nginx/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.11.1' +version = '0.12.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot/__init__.py b/certbot/__init__.py index 36cb9a713..6451eb0d5 100644 --- a/certbot/__init__.py +++ b/certbot/__init__.py @@ -1,4 +1,4 @@ """Certbot client.""" # version number like 1.2.3a0, must have at least 2 parts, like 1.2 -__version__ = '0.11.1' +__version__ = '0.12.0.dev0' diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index ac721638f..81e3862c2 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.11.1" +LE_AUTO_VERSION="0.12.0.dev0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates From 92816275a646027afd71890784de5e649f48678e Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Thu, 9 Feb 2017 12:59:06 -0800 Subject: [PATCH 52/88] plaintextify --- docs/using.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/using.rst b/docs/using.rst index a4d41dff3..884cc46eb 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -288,13 +288,13 @@ certificate counts against several rate limits that are intended to prevent abuse of the ACME protocol, as described `here `__. -Changing a Certificate’s Domains -```````````````````````````````` +Changing a Certificate's Domains +-------------------------------- The ``--cert-name`` flag can also be used to modify the domains a certificate contains, -by specifying new domains using the ``-d`` or ``—domains`` flag. If certificate ``example.com`` +by specifying new domains using the ``-d`` or ``--domains`` flag. If certificate ``example.com`` previously contained ``example.com`` and ``www.example.com``, it can be modified to only -contain ``example.com`` by specifying only ``example.com`` with the ``-d`` or ``—domains`` flag. Example:: +contain ``example.com`` by specifying only ``example.com`` with the ``-d`` or ``--domains`` flag. Example:: certbot certonly --cert-name example.com -d example.com @@ -314,14 +314,14 @@ use the ``revoke`` command to do so. Note that the ``revoke`` command takes the certbot revoke --cert-path /etc/letsencrypt/live/CERTNAME/cert.pem Additionally, if a certificate -is a test cert obtained via the ``--staging`` or ``—test-cert`` flag, that flag must be passed to the +is a test cert obtained via the ``--staging`` or ``--test-cert`` flag, that flag must be passed to the ``revoke`` subcommand. Once a certificate is revoked (or for other cert management tasks), all of a certificate's relevant files can be removed from the system with the ``delete`` subcommand:: certbot delete --cert-name example.com -.. note:: If you don’t use ``delete`` to remove the certificate completely, it will be renewed automatically at the next renewal event. +.. note:: If you don't use ``delete`` to remove the certificate completely, it will be renewed automatically at the next renewal event. .. _renewal: @@ -410,15 +410,15 @@ apologize for any inconvenience you encounter in integrating these commands into your individual environment. Modifying the Renewal Configuration File -```````````````````````````````````````` +---------------------------------------- For advanced certificate management tasks, it is possible to manually modify the certificate's renewal configuration file, located at ``/etc/letsencrypt/renewal/CERTNAME``. -.. warning:: Modifying any files in ``/etc/letsencrypt`` can damage it so Certbot can no longer properly manage its certificates, and we do not recommend doing so. +.. warning:: Modifying any files in ``/etc/letsencrypt`` can damage them so Certbot can no longer properly manage its certificates, and we do not recommend doing so. For most tasks, it is safest to limit yourself to pointing symlinks at the files there, or using -``—renew-hook`` to copy / make new files based upon those files, if your operational situation requires it +``--renew-hook`` to copy / make new files based upon those files, if your operational situation requires it (for instance, combining certs and keys in different way, or having copies of things with different specific permissions that are demanded by other programs). From 8b5c87f983fbd2e73cab0a815ed499e4567c7d31 Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Thu, 9 Feb 2017 13:13:33 -0800 Subject: [PATCH 53/88] revert factual errors --- docs/using.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/using.rst b/docs/using.rst index 884cc46eb..91c0627b6 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -244,17 +244,17 @@ using the ``--cert-name`` flag to specify a particular certificate for the ``run Re-creating and Updating Existing Certificates ---------------------------------------------- -You can use ``certonly`` or ``run`` subcommands to request +You can use ``certonly`` or ``run`` subcommands to request the creation of a single new certificate even if you already have an -existing certificate with some of the same domain names. +existing certificate with some of the same domain names. -If a certificate is requested with ``run`` or ``certonly`` specifying a -``Certificate name`` that already exists, the new certificate replaces +If a certificate is requested with ``run`` or ``certonly`` specifying a +certificate name that already exists, Certbot updates the existing certificate. Otherwise a new certificate is created and assigned the specified name. -The ``--force-renewal``, ``--duplicate``, and ``--expand`` options -(introduced in version 0.10.0) control Certbot's behavior when re-creating +The ``--force-renewal``, ``--duplicate``, and ``--expand`` options +control Certbot's behavior when re-creating a certificate with the same name as an existing certificate. If you don't specify a requested behavior, Certbot may ask you what you intended. @@ -332,7 +332,7 @@ Renewing certificates days). Make sure you renew the certificates at least once in 3 months. -As of version 0.10.0, ``certbot`` client supports a ``renew`` action to check +As of version 0.10.0, Certbot supports a ``renew`` action to check all installed certificates for impending expiry and attempt to renew them. The simplest form is simply From 08a546fa8da7f886455a06236b14b2e04f1af947 Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Thu, 9 Feb 2017 14:38:38 -0800 Subject: [PATCH 54/88] remove trailing whitespace --- docs/using.rst | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/using.rst b/docs/using.rst index 91c0627b6..39b1c8fd7 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -268,7 +268,7 @@ valid method of renewing a specific individual certificate. ``--duplicate`` tells Certbot to create a separate, unrelated certificate -with the same domains as an existing certificate. This certificate is +with the same domains as an existing certificate. This certificate is saved completely separately from the prior one. Most users will not need to issue this command in normal circumstances. @@ -350,7 +350,7 @@ run as frequently as you want - since it will usually take no action. The ``renew`` command includes hooks for running commands or scripts before or after a certificate is renewed. For example, if you have a single cert obtained using -the standalone_ plugin, you might need to stop the webserver +the standalone_ plugin, you might need to stop the webserver before renewing so standalone can bind to the necessary ports, and then restart it after the plugin is finished. Example:: @@ -417,8 +417,8 @@ renewal configuration file, located at ``/etc/letsencrypt/renewal/CERTNAME``. .. warning:: Modifying any files in ``/etc/letsencrypt`` can damage them so Certbot can no longer properly manage its certificates, and we do not recommend doing so. -For most tasks, it is safest to limit yourself to pointing symlinks at the files there, or using -``--renew-hook`` to copy / make new files based upon those files, if your operational situation requires it +For most tasks, it is safest to limit yourself to pointing symlinks at the files there, or using +``--renew-hook`` to copy / make new files based upon those files, if your operational situation requires it (for instance, combining certs and keys in different way, or having copies of things with different specific permissions that are demanded by other programs). @@ -446,8 +446,7 @@ The following commands could be used to specify where these files are located:: sed -i 's,/etc/letsencrypt/archive/example.com,/home/user/me/certbot/example_archive,' /etc/letsencrypt/renewal/example.com.conf mv /etc/letsencrypt/live/example.com/*.pem /home/user/me/certbot/ sed -i 's,/etc/letsencrypt/live/example.com,/home/user/me/certbot,g' /etc/letsencrypt/renewal/example.com.conf - certbot update_symlinks - + certbot update_symlinks .. _where-certs: From b316a13f221245cccdf0373e6c1c1a8bd9b70ca4 Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Thu, 9 Feb 2017 14:40:02 -0800 Subject: [PATCH 55/88] oxford comma --- docs/using.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using.rst b/docs/using.rst index 39b1c8fd7..3d1ac6c85 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -236,7 +236,7 @@ This returns information in the following format:: ``Certificate Name`` shows the name of the certificate. Pass this name using the ``--cert-name`` flag to specify a particular certificate for the ``run``, -``certonly``, ``certificates``, ``renew`` and ``delete`` commands. Example:: +``certonly``, ``certificates``, ``renew``, and ``delete`` commands. Example:: certbot certonly --cert-name example.com From 14ce872b6b3cdeeeb5b012f410e5408f28783043 Mon Sep 17 00:00:00 2001 From: Chad Whitacre Date: Fri, 10 Feb 2017 17:44:11 -0500 Subject: [PATCH 56/88] Remove erroneous advice Per @pfg at https://community.letsencrypt.org/t/-/21318/2: > This is not really true anymore - account recovery via email was planned to be added at some point, but has been removed from the spec since. --- certbot/account.py | 6 ------ 1 file changed, 6 deletions(-) diff --git a/certbot/account.py b/certbot/account.py index 877b87015..669fd5e80 100644 --- a/certbot/account.py +++ b/certbot/account.py @@ -96,12 +96,6 @@ def report_new_account(acc, config): config.config_dir), reporter.MEDIUM_PRIORITY) - if acc.regr.body.emails: - recovery_msg = ("If you lose your account credentials, you can " - "recover through e-mails sent to {0}.".format( - ", ".join(acc.regr.body.emails))) - reporter.add_message(recovery_msg, reporter.MEDIUM_PRIORITY) - class AccountMemoryStorage(interfaces.AccountStorage): """In-memory account strage.""" From 90bbd93b0563585b3563df62cc9b8d8dcfdc1078 Mon Sep 17 00:00:00 2001 From: Chad Whitacre Date: Mon, 13 Feb 2017 08:49:35 -0500 Subject: [PATCH 57/88] Fix test --- certbot/tests/account_test.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/certbot/tests/account_test.py b/certbot/tests/account_test.py index d045afe23..e45fbe6ea 100644 --- a/certbot/tests/account_test.py +++ b/certbot/tests/account_test.py @@ -80,8 +80,6 @@ class ReportNewAccountTest(unittest.TestCase): self._call() call_list = mock_zope().add_message.call_args_list self.assertTrue(self.config.config_dir in call_list[0][0][0]) - self.assertTrue( - ", ".join(self.acc.regr.body.emails) in call_list[1][0][0]) class AccountMemoryStorageTest(unittest.TestCase): From b0ee98a5c370de12c2f56d634213167e5061d019 Mon Sep 17 00:00:00 2001 From: Chad Whitacre Date: Mon, 13 Feb 2017 09:04:15 -0500 Subject: [PATCH 58/88] Satisfy linter --- certbot/account.py | 2 +- certbot/client.py | 2 +- certbot/tests/account_test.py | 5 +---- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/certbot/account.py b/certbot/account.py index 669fd5e80..4b5f48411 100644 --- a/certbot/account.py +++ b/certbot/account.py @@ -82,7 +82,7 @@ class Account(object): # pylint: disable=too-few-public-methods self.meta == other.meta) -def report_new_account(acc, config): +def report_new_account(config): """Informs the user about their new ACME account.""" reporter = zope.component.queryUtility(interfaces.IReporter) if reporter is None: diff --git a/certbot/client.py b/certbot/client.py index 26c5e87a9..95882a9fc 100644 --- a/certbot/client.py +++ b/certbot/client.py @@ -137,7 +137,7 @@ def register(config, account_storage, tos_cb=None): regr = acme.agree_to_tos(regr) acc = account.Account(regr, key) - account.report_new_account(acc, config) + account.report_new_account(config) account_storage.save(acc) eff.handle_subscription(config) diff --git a/certbot/tests/account_test.py b/certbot/tests/account_test.py index e45fbe6ea..8ed591c98 100644 --- a/certbot/tests/account_test.py +++ b/certbot/tests/account_test.py @@ -62,13 +62,10 @@ class ReportNewAccountTest(unittest.TestCase): def setUp(self): self.config = mock.MagicMock(config_dir="/etc/letsencrypt") - reg = messages.Registration.from_data(email="rhino@jungle.io") - self.acc = mock.MagicMock(regr=messages.RegistrationResource( - uri=None, new_authzr_uri=None, body=reg)) def _call(self): from certbot.account import report_new_account - report_new_account(self.acc, self.config) + report_new_account(self.config) @mock.patch("certbot.account.zope.component.queryUtility") def test_no_reporter(self, mock_zope): From 1380e59f56447c34799340aecb380553e9fa2fe1 Mon Sep 17 00:00:00 2001 From: Jacob Hoffman-Andrews Date: Mon, 13 Feb 2017 19:50:33 -0800 Subject: [PATCH 59/88] Remove Link rel=next for authzs and new-certs. (#4194) An early version of the spec indicated that clients should process issuance sequentially, following Link rel=next from an account URL to an authz URL, to a new-cert URL. However, the spec has long since moved to putting these URLs in the directory. Certbot nominally supports either; This change consolidates on always using the directory, simplifying things and making the transition to the latest ACME spec easier. --- acme/acme/client.py | 40 ++++++++---------------------- acme/acme/client_test.py | 40 ++++++------------------------ acme/acme/jose/json_util.py | 2 +- acme/acme/messages.py | 6 +---- acme/acme/messages_test.py | 6 +---- acme/examples/example_client.py | 3 +-- certbot/auth_handler.py | 3 +-- certbot/tests/account_test.py | 2 +- certbot/tests/acme_util.py | 1 - certbot/tests/auth_handler_test.py | 3 +-- certbot/tests/display/ops_test.py | 4 +-- 11 files changed, 26 insertions(+), 84 deletions(-) diff --git a/acme/acme/client.py b/acme/acme/client.py index 0324967cf..d01555c75 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -71,20 +71,13 @@ class Client(object): # pylint: disable=too-many-instance-attributes self.directory = directory @classmethod - def _regr_from_response(cls, response, uri=None, new_authzr_uri=None, - terms_of_service=None): + def _regr_from_response(cls, response, uri=None, terms_of_service=None): if 'terms-of-service' in response.links: terms_of_service = response.links['terms-of-service']['url'] - if 'next' in response.links: - new_authzr_uri = response.links['next']['url'] - - if new_authzr_uri is None: - raise errors.ClientError('"next" link missing') return messages.RegistrationResource( body=messages.Registration.from_json(response.json()), uri=response.headers.get('Location', uri), - new_authzr_uri=new_authzr_uri, terms_of_service=terms_of_service) def register(self, new_reg=None): @@ -117,7 +110,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes # (c.f. acme-spec #94) return self._regr_from_response( - response, uri=regr.uri, new_authzr_uri=regr.new_authzr_uri, + response, uri=regr.uri, terms_of_service=regr.terms_of_service) def update_registration(self, regr, update=None): @@ -174,43 +167,30 @@ class Client(object): # pylint: disable=too-many-instance-attributes return self.update_registration( regr.update(body=regr.body.update(agreement=regr.terms_of_service))) - def _authzr_from_response(self, response, identifier, - uri=None, new_cert_uri=None): - # pylint: disable=no-self-use - if new_cert_uri is None: - try: - new_cert_uri = response.links['next']['url'] - except KeyError: - raise errors.ClientError('"next" link missing') - + def _authzr_from_response(self, response, identifier, uri=None): authzr = messages.AuthorizationResource( body=messages.Authorization.from_json(response.json()), - uri=response.headers.get('Location', uri), - new_cert_uri=new_cert_uri) + uri=response.headers.get('Location', uri)) if authzr.body.identifier != identifier: raise errors.UnexpectedUpdate(authzr) return authzr - def request_challenges(self, identifier, new_authzr_uri=None): + def request_challenges(self, identifier): """Request challenges. :param .messages.Identifier identifier: Identifier to be challenged. - :param str new_authzr_uri: ``new-authorization`` URI. If omitted, - will default to value found in ``directory``. :returns: Authorization Resource. :rtype: `.AuthorizationResource` """ new_authz = messages.NewAuthorization(identifier=identifier) - response = self.net.post(self.directory.new_authz - if new_authzr_uri is None else new_authzr_uri, - new_authz) + response = self.net.post(self.directory.new_authz, new_authz) # TODO: handle errors assert response.status_code == http_client.CREATED return self._authzr_from_response(response, identifier) - def request_domain_challenges(self, domain, new_authzr_uri=None): + def request_domain_challenges(self, domain): """Request challenges for domain names. This is simply a convenience function that wraps around @@ -225,7 +205,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes """ return self.request_challenges(messages.Identifier( - typ=messages.IDENTIFIER_FQDN, value=domain), new_authzr_uri) + typ=messages.IDENTIFIER_FQDN, value=domain)) def answer_challenge(self, challb, response): """Answer challenge. @@ -300,7 +280,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes """ response = self.net.get(authzr.uri) updated_authzr = self._authzr_from_response( - response, authzr.body.identifier, authzr.uri, authzr.new_cert_uri) + response, authzr.body.identifier, authzr.uri) # TODO: check and raise UnexpectedUpdate return updated_authzr, response @@ -324,7 +304,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes content_type = DER_CONTENT_TYPE # TODO: add 'cert_type 'argument response = self.net.post( - authzrs[0].new_cert_uri, # TODO: acme-spec #90 + self.directory.new_cert, req, content_type=content_type, headers={'Accept': content_type}) diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index 179a8a08c..0f4506203 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -40,6 +40,8 @@ class ClientTest(unittest.TestCase): 'https://www.letsencrypt-demo.org/acme/revoke-cert', messages.NewAuthorization: 'https://www.letsencrypt-demo.org/acme/new-authz', + messages.CertificateRequest: + 'https://www.letsencrypt-demo.org/acme/new-cert', }) from acme.client import Client @@ -56,7 +58,6 @@ class ClientTest(unittest.TestCase): self.new_reg = messages.NewRegistration(**dict(reg)) self.regr = messages.RegistrationResource( body=reg, uri='https://www.letsencrypt-demo.org/acme/reg/1', - new_authzr_uri='https://www.letsencrypt-demo.org/acme/new-reg', terms_of_service='https://www.letsencrypt-demo.org/tos') # Authorization @@ -72,8 +73,7 @@ class ClientTest(unittest.TestCase): typ=messages.IDENTIFIER_FQDN, value='example.com'), challenges=(challb,), combinations=None) self.authzr = messages.AuthorizationResource( - body=self.authz, uri=authzr_uri, - new_cert_uri='https://www.letsencrypt-demo.org/acme/new-cert') + body=self.authz, uri=authzr_uri) # Request issuance self.certr = messages.CertificateResource( @@ -98,18 +98,12 @@ class ClientTest(unittest.TestCase): self.response.json.return_value = self.regr.body.to_json() self.response.headers['Location'] = self.regr.uri self.response.links.update({ - 'next': {'url': self.regr.new_authzr_uri}, 'terms-of-service': {'url': self.regr.terms_of_service}, }) self.assertEqual(self.regr, self.client.register(self.new_reg)) # TODO: test POST call arguments - def test_register_missing_next(self): - self.response.status_code = http_client.CREATED - self.assertRaises( - errors.ClientError, self.client.register, self.new_reg) - def test_update_registration(self): # "Instance of 'Field' has no to_json/update member" bug: # pylint: disable=no-member @@ -142,13 +136,6 @@ class ClientTest(unittest.TestCase): self.response.json.return_value = self.regr.body.to_json() self.assertEqual(self.regr, self.client.query_registration(self.regr)) - def test_query_registration_updates_new_authzr_uri(self): - self.response.json.return_value = self.regr.body.to_json() - self.response.links = {'next': {'url': 'UPDATED'}} - self.assertEqual( - 'UPDATED', - self.client.query_registration(self.regr).new_authzr_uri) - def test_agree_to_tos(self): self.client.update_registration = mock.Mock() self.client.agree_to_tos(self.regr) @@ -159,9 +146,6 @@ class ClientTest(unittest.TestCase): self.response.status_code = http_client.CREATED self.response.headers['Location'] = self.authzr.uri self.response.json.return_value = self.authz.to_json() - self.response.links = { - 'next': {'url': self.authzr.new_cert_uri}, - } def test_request_challenges(self): self._prepare_response_for_request_challenges() @@ -172,8 +156,9 @@ class ClientTest(unittest.TestCase): def test_request_challenges_custom_uri(self): self._prepare_response_for_request_challenges() - self.client.request_challenges(self.identifier, 'URI') - self.net.post.assert_called_once_with('URI', mock.ANY) + self.client.request_challenges(self.identifier) + self.net.post.assert_called_once_with( + 'https://www.letsencrypt-demo.org/acme/new-authz', mock.ANY) def test_request_challenges_unexpected_update(self): self._prepare_response_for_request_challenges() @@ -181,12 +166,7 @@ class ClientTest(unittest.TestCase): identifier=self.identifier.update(value='foo')).to_json() self.assertRaises( errors.UnexpectedUpdate, self.client.request_challenges, - self.identifier, self.authzr.uri) - - def test_request_challenges_missing_next(self): - self.response.status_code = http_client.CREATED - self.assertRaises(errors.ClientError, self.client.request_challenges, - self.identifier) + self.identifier) def test_request_domain_challenges(self): self.client.request_challenges = mock.MagicMock() @@ -194,12 +174,6 @@ class ClientTest(unittest.TestCase): self.client.request_challenges(self.identifier), self.client.request_domain_challenges('example.com')) - def test_request_domain_challenges_custom_uri(self): - self.client.request_challenges = mock.MagicMock() - self.assertEqual( - self.client.request_challenges(self.identifier, 'URI'), - self.client.request_domain_challenges('example.com', 'URI')) - def test_answer_challenge(self): self.response.links['up'] = {'url': self.challr.authzr_uri} self.response.json.return_value = self.challr.body.to_json() diff --git a/acme/acme/jose/json_util.py b/acme/acme/jose/json_util.py index d474f4aac..4baadda5e 100644 --- a/acme/acme/jose/json_util.py +++ b/acme/acme/jose/json_util.py @@ -267,7 +267,7 @@ class JSONObjectWithFields(util.ImmutableMap, interfaces.JSONDeSerializable): if missing: raise errors.DeserializationError( - 'The following field are required: {0}'.format( + 'The following fields are required: {0}'.format( ','.join(missing))) @classmethod diff --git a/acme/acme/messages.py b/acme/acme/messages.py index 54cd25c94..c3df4998c 100644 --- a/acme/acme/messages.py +++ b/acme/acme/messages.py @@ -191,7 +191,7 @@ class Directory(jose.JSONDeSerializable): try: return self[name.replace('_', '-')] except KeyError as error: - raise AttributeError(str(error)) + raise AttributeError(str(error) + ': ' + name) def __getitem__(self, name): try: @@ -315,12 +315,10 @@ class RegistrationResource(ResourceWithURI): """Registration Resource. :ivar acme.messages.Registration body: - :ivar unicode new_authzr_uri: URI found in the 'next' ``Link`` header :ivar unicode terms_of_service: URL for the CA TOS. """ body = jose.Field('body', decoder=Registration.from_json) - new_authzr_uri = jose.Field('new_authzr_uri') terms_of_service = jose.Field('terms_of_service', omitempty=True) @@ -425,11 +423,9 @@ class AuthorizationResource(ResourceWithURI): """Authorization Resource. :ivar acme.messages.Authorization body: - :ivar unicode new_cert_uri: URI found in the 'next' ``Link`` header """ body = jose.Field('body', decoder=Authorization.from_json) - new_cert_uri = jose.Field('new_cert_uri') @Directory.register diff --git a/acme/acme/messages_test.py b/acme/acme/messages_test.py index b3454f25b..e84c3e992 100644 --- a/acme/acme/messages_test.py +++ b/acme/acme/messages_test.py @@ -225,14 +225,12 @@ class RegistrationResourceTest(unittest.TestCase): from acme.messages import RegistrationResource self.regr = RegistrationResource( body=mock.sentinel.body, uri=mock.sentinel.uri, - new_authzr_uri=mock.sentinel.new_authzr_uri, terms_of_service=mock.sentinel.terms_of_service) def test_to_partial_json(self): self.assertEqual(self.regr.to_json(), { 'body': mock.sentinel.body, 'uri': mock.sentinel.uri, - 'new_authzr_uri': mock.sentinel.new_authzr_uri, 'terms_of_service': mock.sentinel.terms_of_service, }) @@ -346,9 +344,7 @@ class AuthorizationResourceTest(unittest.TestCase): from acme.messages import AuthorizationResource authzr = AuthorizationResource( uri=mock.sentinel.uri, - body=mock.sentinel.body, - new_cert_uri=mock.sentinel.new_cert_uri, - ) + body=mock.sentinel.body) self.assertTrue(isinstance(authzr, jose.JSONDeSerializable)) diff --git a/acme/examples/example_client.py b/acme/examples/example_client.py index 261b37603..1386491b1 100644 --- a/acme/examples/example_client.py +++ b/acme/examples/example_client.py @@ -32,8 +32,7 @@ acme.agree_to_tos(regr) logging.debug(regr) authzr = acme.request_challenges( - identifier=messages.Identifier(typ=messages.IDENTIFIER_FQDN, value=DOMAIN), - new_authzr_uri=regr.new_authzr_uri) + identifier=messages.Identifier(typ=messages.IDENTIFIER_FQDN, value=DOMAIN)) logging.debug(authzr) authzr, authzr_response = acme.poll(authzr) diff --git a/certbot/auth_handler.py b/certbot/auth_handler.py index 6e9ab25a7..53346a77c 100644 --- a/certbot/auth_handler.py +++ b/certbot/auth_handler.py @@ -63,8 +63,7 @@ class AuthHandler(object): """ for domain in domains: - self.authzr[domain] = self.acme.request_domain_challenges( - domain, self.account.regr.new_authzr_uri) + self.authzr[domain] = self.acme.request_domain_challenges(domain) self._choose_challenges(domains) diff --git a/certbot/tests/account_test.py b/certbot/tests/account_test.py index 8ed591c98..7367717bf 100644 --- a/certbot/tests/account_test.py +++ b/certbot/tests/account_test.py @@ -110,7 +110,7 @@ class AccountFileStorageTest(unittest.TestCase): from certbot.account import Account self.acc = Account( regr=messages.RegistrationResource( - uri=None, new_authzr_uri=None, body=messages.Registration()), + uri=None, body=messages.Registration()), key=KEY) def tearDown(self): diff --git a/certbot/tests/acme_util.py b/certbot/tests/acme_util.py index 5e6b190a7..f0549666a 100644 --- a/certbot/tests/acme_util.py +++ b/certbot/tests/acme_util.py @@ -96,6 +96,5 @@ def gen_authzr(authz_status, domain, challs, statuses, combos=True): # pylint: disable=star-args return messages.AuthorizationResource( uri="https://trusted.ca/new-authz-resource", - new_cert_uri="https://trusted.ca/new-cert", body=messages.Authorization(**authz_kwargs) ) diff --git a/certbot/tests/auth_handler_test.py b/certbot/tests/auth_handler_test.py index 046eb5ef1..9d22843db 100644 --- a/certbot/tests/auth_handler_test.py +++ b/certbot/tests/auth_handler_test.py @@ -309,7 +309,6 @@ class PollChallengesTest(unittest.TestCase): new_authzr = messages.AuthorizationResource( uri=authzr.uri, - new_cert_uri=authzr.new_cert_uri, body=messages.Authorization( identifier=authzr.body.identifier, challenges=new_challbs, @@ -437,7 +436,7 @@ def gen_auth_resp(chall_list): for chall in chall_list] -def gen_dom_authzr(domain, unused_new_authzr_uri, challs, combos=True): +def gen_dom_authzr(domain, challs, combos=True): """Generates new authzr for domains.""" return acme_util.gen_authzr( messages.STATUS_PENDING, domain, challs, diff --git a/certbot/tests/display/ops_test.py b/certbot/tests/display/ops_test.py index f6de33a92..f2a9b3d07 100644 --- a/certbot/tests/display/ops_test.py +++ b/certbot/tests/display/ops_test.py @@ -104,10 +104,10 @@ class ChooseAccountTest(unittest.TestCase): self.key = KEY self.acc1 = account.Account(messages.RegistrationResource( - uri=None, new_authzr_uri=None, body=messages.Registration.from_data( + uri=None, body=messages.Registration.from_data( email="email1@g.com")), self.key) self.acc2 = account.Account(messages.RegistrationResource( - uri=None, new_authzr_uri=None, body=messages.Registration.from_data( + uri=None, body=messages.Registration.from_data( email="email2@g.com", phone="phone")), self.key) @classmethod From 94aa562cf2f2aca0ff2ef552d0e9d5ff06d92ae4 Mon Sep 17 00:00:00 2001 From: Jacob Hoffman-Andrews Date: Tue, 14 Feb 2017 21:19:18 -0800 Subject: [PATCH 60/88] Use latest Boulder for Travis tests. (#4180) --- .travis.yml | 4 ---- tests/boulder-fetch.sh | 14 ++++++++++---- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/.travis.yml b/.travis.yml index ca06f07d0..577dcbc40 100644 --- a/.travis.yml +++ b/.travis.yml @@ -10,10 +10,6 @@ before_install: # using separate envs with different TOXENVs creates 4x1 Travis build # matrix, which allows us to clearly distinguish which component under # test has failed -env: - global: - - BOULDERPATH=$PWD/boulder/ - matrix: include: - python: "2.7" diff --git a/tests/boulder-fetch.sh b/tests/boulder-fetch.sh index 0c0a8009b..ef61fe3f5 100755 --- a/tests/boulder-fetch.sh +++ b/tests/boulder-fetch.sh @@ -2,9 +2,15 @@ # Download and run Boulder instance for integration testing set -xe -# Check out special branch until latest docker changes land in Boulder master. -git clone -b docker-integration https://github.com/letsencrypt/boulder $BOULDERPATH -cd $BOULDERPATH +# Clone Boulder into a GOPATH-style directory structure even if Go isn't +# installed, because Boulder's docker-compose.yml file wll look for it there. +export GOPATH=${GOPATH:-$HOME/gopath} +BOULDERPATH=${BOULDERPATH:-$GOPATH/src/github.com/letsencrypt/boulder} +if [ ! -d ${BOULDERPATH} ]; then + git clone --depth=1 https://github.com/letsencrypt/boulder ${BOULDERPATH} +fi + +cd ${BOULDERPATH} FAKE_DNS=$(ifconfig docker0 | grep "inet addr:" | cut -d: -f2 | awk '{ print $1}') -sed -i "s/FAKE_DNS: .*/FAKE_DNS: $FAKE_DNS/" docker-compose.yml +sed -i "s/FAKE_DNS: .*/FAKE_DNS: ${FAKE_DNS}/" docker-compose.yml docker-compose up -d From 6a8113fa87ff8f55aa16395b9208ac0afe820a34 Mon Sep 17 00:00:00 2001 From: Jacob Hoffman-Andrews Date: Thu, 16 Feb 2017 10:24:48 -0800 Subject: [PATCH 61/88] Remove reference to #certbot on OFTC The #letsencrypt channel on Freenode is much more active, and is the defacto place for questions about Certbot. Users posting questions on #certbot on OFTC are not getting prompt answers. --- README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.rst b/README.rst index f986703ac..ab12562df 100644 --- a/README.rst +++ b/README.rst @@ -88,7 +88,7 @@ Main Website: https://certbot.eff.org Let's Encrypt Website: https://letsencrypt.org -IRC Channel: #letsencrypt on `Freenode`_ or #certbot on `OFTC`_ +IRC Channel: #letsencrypt on `Freenode`_ Community: https://community.letsencrypt.org From ebf5170d1270659d272b4a745b6e0314ad071785 Mon Sep 17 00:00:00 2001 From: Jacob Hoffman-Andrews Date: Thu, 16 Feb 2017 12:39:17 -0800 Subject: [PATCH 62/88] Update testing docs. (#4211) Recommend faster-running commands with clearer output first, building up to slow, noisy, `tox` last. Remove outdated Boulder testing advice, and outdated Boulder testing scripts (Vagrantfile, mac-bootstrap.sh, boulder-start.sh). Update Boulder testing section. Fill out "directory structure" section of contributing guide a little better. --- Vagrantfile | 41 ------------- docs/contributing.rst | 116 +++++++++++++---------------------- tests/boulder-integration.sh | 2 +- tests/boulder-start.sh | 9 --- tests/mac-bootstrap.sh | 26 -------- 5 files changed, 43 insertions(+), 151 deletions(-) delete mode 100644 Vagrantfile delete mode 100755 tests/boulder-start.sh delete mode 100755 tests/mac-bootstrap.sh diff --git a/Vagrantfile b/Vagrantfile deleted file mode 100644 index 23d3ddf13..000000000 --- a/Vagrantfile +++ /dev/null @@ -1,41 +0,0 @@ -# -*- mode: ruby -*- -# vi: set ft=ruby : - -# Vagrantfile API/syntax version. Don't touch unless you know what you're doing! -VAGRANTFILE_API_VERSION = "2" - -# Setup instructions from docs/contributing.rst -# Script installs dependencies for tox and boulder integration -$ubuntu_setup_script = <> /home/vagrant/.profile; fi -if ! grep -Fxq "export PATH=\\$GOROOT/bin:\\$PATH" /home/vagrant/.profile ; then echo "export PATH=\\$GOROOT/bin:\\$PATH" >> /home/vagrant/.profile; fi -if ! grep -Fxq "export GOPATH=\\$HOME/go" /home/vagrant/.profile ; then echo "export GOPATH=\\$HOME/go" >> /home/vagrant/.profile; fi -if ! grep -Fxq "cd /vagrant/; ./tests/boulder-start.sh &" /etc/rc.local ; then sed -i -e '$i \cd /vagrant/; ./tests/boulder-start.sh &\n' /etc/rc.local; fi -export DEBIAN_FRONTEND=noninteractive -sudo -E apt-get -q -y install git make libltdl-dev mariadb-server rabbitmq-server nginx-light -SETUP_SCRIPT - -Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| - - config.vm.define "ubuntu-trusty", primary: true do |ubuntu_trusty| - ubuntu_trusty.vm.box = "ubuntu/trusty64" - ubuntu_trusty.vm.provision "shell", inline: $ubuntu_setup_script - ubuntu_trusty.vm.provider "virtualbox" do |v| - # VM needs more memory to run test suite, got "OSError: [Errno 12] - # Cannot allocate memory" when running - # letsencrypt.client.tests.display.util_test.NcursesDisplayTest - # We may no longer need this. - v.memory = 1024 - - # Handle cases when the host is behind a private network by making the - # NAT engine use the host's resolver mechanisms to handle DNS requests. - v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"] - end - end - -end diff --git a/docs/contributing.rst b/docs/contributing.rst index 040c22864..5f939e947 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -54,104 +54,68 @@ where appropriate. Once you've got a working branch, you can open a pull request. All changes in your pull request must have thorough unit test coverage, pass our -`integration`_ tests, and be compliant with the :ref:`coding style -`. +tests, and be compliant with the :ref:`coding style `. .. _github issue tracker: https://github.com/certbot/certbot/issues .. _Good Volunteer Task: https://github.com/certbot/certbot/issues?q=is%3Aopen+is%3Aissue+label%3A%22Good+Volunteer+Task%22 +.. _testing: + Testing ------- -The following tools are there to help you: +When you are working in a file ``foo.py``, there should also be a file ``foo_test.py`` +either in the same directory as ``foo.py`` or in the ``tests`` subdirectory +(if there isn't, make one). While you are working on your code and tests, run +``python foo_test.py`` to run the relevant tests. -- ``tox`` starts a full set of tests. Please note that it includes - apacheconftest, which uses the system's Apache install to test config file - parsing, so it should only be run on systems that have an - experimental, non-production Apache2 install on them. ``tox -e - apacheconftest`` can be used to run those specific Apache conf tests. +For debugging, we recommend running ``pip install ipdb`` and putting +``import ipdb; ipdb.set_trace()`` statements inside the source +code. Alternatively, you can use Python's standard library `pdb`, +but you won't get TAB completion. -- ``tox --skip-missing-interpreters`` runs tox while ignoring missing versions - of Python needed for running the tests. +Once you are done with your code changes, and the tests in ``foo_test.py`` pass, +run all of the unittests for Certbot with ``tox -e py27`` (this uses Python +2.7). -- ``tox -e py27``, ``tox -e py26`` etc, run unit tests for specific Python - versions. +Once all the unittests pass, check for sufficient test coverage using +``tox -e cover``, and then check for code style with ``tox -e lint`` (all files) +or ``pylint --rcfile=.pylintrc path/to/file.py`` (single file at a time). -- ``tox -e cover`` checks the test coverage only. Calling the - ``./tox.cover.sh`` script directly (or even ``./tox.cover.sh $pkg1 - $pkg2 ...`` for any subpackages) might be a bit quicker, though. - -- ``tox -e lint`` checks the style of the whole project, while - ``pylint --rcfile=.pylintrc path`` will check a single file or - specific directory only. - -- For debugging, we recommend ``pip install ipdb`` and putting - ``import ipdb; ipdb.set_trace()`` statement inside the source - code. Alternatively, you can use Python's standard library `pdb`, - but you won't get TAB completion... +Once all of the above is successful, you may run the full test suite, +including integration tests, using ``tox``. We recommend running the +commands above first, because running all tests with ``tox`` is very +slow, and the large amount of ``tox`` output can make it hard to find +specific failures when they happen. Also note that the full test suite +will attempt to modify your system's Apache config if your user has sudo +permissions, so it should not be run on a production Apache server. +If you have trouble getting the full ``tox`` suite to run locally, it is +generally sufficient to open a pull request and let Github and Travis run +integration tests for you. .. _integration: -Integration testing with the boulder CA +Integration testing with the Boulder CA ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Generally it is sufficient to open a pull request and let Github and Travis run -integration tests for you. - -However, if you prefer to run tests, you can use Vagrant, using the Vagrantfile -in Certbot's repository. To execute the tests on a Vagrant box, the only -command you are required to run is:: - - ./tests/boulder-integration.sh - -Otherwise, please follow the following instructions. - -macOS users: Run ``./tests/mac-bootstrap.sh`` instead of -``boulder-start.sh`` to install dependencies, configure the -environment, and start boulder. - -Otherwise, install `Go`_ 1.5, ``libtool-ltdl``, ``mariadb-server`` and -``rabbitmq-server`` and then start Boulder_, an ACME CA server. - -If you can't get packages of Go 1.5 for your Linux system, -you can execute the following commands to install it: +To run integration tests locally, you need Docker and docker-compose installed +and working. Fetch and start Boulder using: .. code-block:: shell - wget https://storage.googleapis.com/golang/go1.5.3.linux-amd64.tar.gz -P /tmp/ - sudo tar -C /usr/local -xzf /tmp/go1.5.3.linux-amd64.tar.gz - if ! grep -Fxq "export GOROOT=/usr/local/go" ~/.profile ; then echo "export GOROOT=/usr/local/go" >> ~/.profile; fi - if ! grep -Fxq "export PATH=\\$GOROOT/bin:\\$PATH" ~/.profile ; then echo "export PATH=\\$GOROOT/bin:\\$PATH" >> ~/.profile; fi + ./tests/boulder-fetch.sh -These commands download `Go`_ 1.5.3 to ``/tmp/``, extracts to ``/usr/local``, -and then adds the export lines required to execute ``boulder-start.sh`` to -``~/.profile`` if they were not previously added +If you have problems with Docker, you may want to try `removing all containers and +volumes`_ and making sure you have at least 1GB of memory. -Make sure you execute the following command after `Go`_ finishes installing:: +Run the integration tests using: - if ! grep -Fxq "export GOPATH=\\$HOME/go" ~/.profile ; then echo "export GOPATH=\\$HOME/go" >> ~/.profile; fi +.. code-block:: shell -Afterwards, you'd be able to start Boulder_ using the following command:: - - ./tests/boulder-start.sh - -The script will download, compile and run the executable; please be -patient - it will take some time... Once its ready, you will see -``Server running, listening on 127.0.0.1:4000...``. Add ``/etc/hosts`` -entries pointing ``le.wtf``, ``le1.wtf``, ``le2.wtf``, ``le3.wtf`` -and ``nginx.wtf`` to 127.0.0.1. You may now run (in a separate terminal):: - - ./tests/boulder-integration.sh && echo OK || echo FAIL - -If you would like to test `certbot_nginx` plugin (highly -encouraged) make sure to install prerequisites as listed in -``certbot-nginx/tests/boulder-integration.sh`` and rerun -the integration tests suite. - -.. _Boulder: https://github.com/letsencrypt/boulder -.. _Go: https://golang.org + ./tests/boulder-integration.sh +.. _removing all containers and volumes: https://www.digitalocean.com/community/tutorials/how-to-remove-docker-images-containers-and-volumes Code components and layout ========================== @@ -159,7 +123,11 @@ Code components and layout acme contains all protocol specific code certbot - all client code + main client code +certbot-apache and certbot-nginx + client code to configure specific web servers +certbot.egg-info + configuration for packaging Certbot Plugin-architecture diff --git a/tests/boulder-integration.sh b/tests/boulder-integration.sh index 4a2131006..2ddd3c04b 100755 --- a/tests/boulder-integration.sh +++ b/tests/boulder-integration.sh @@ -1,7 +1,7 @@ #!/bin/sh -xe # Simple integration test. Make sure to activate virtualenv beforehand # (source venv/bin/activate) and that you are running Boulder test -# instance (see ./boulder-start.sh). +# instance (see ./boulder-fetch.sh). # # Environment variables: # SERVER: Passed as "certbot --server" argument. diff --git a/tests/boulder-start.sh b/tests/boulder-start.sh deleted file mode 100755 index acf8f0bbf..000000000 --- a/tests/boulder-start.sh +++ /dev/null @@ -1,9 +0,0 @@ -#!/bin/bash - -export GOPATH="${GOPATH:-/tmp/go}" -export PATH="$GOPATH/bin:$PATH" - -./tests/boulder-fetch.sh - -cd $GOPATH/src/github.com/letsencrypt/boulder -./start.py diff --git a/tests/mac-bootstrap.sh b/tests/mac-bootstrap.sh deleted file mode 100755 index 66036ce56..000000000 --- a/tests/mac-bootstrap.sh +++ /dev/null @@ -1,26 +0,0 @@ -#!/bin/sh - -#Check Homebrew -if ! hash brew 2>/dev/null; then - echo "Homebrew Not Installed\nDownloading..." - ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" -fi - -brew install libtool mariadb rabbitmq coreutils go - -mysql.server start - -rabbit_pid=`ps | grep rabbitmq | grep -v grep | awk '{ print $1}'` -if [ -n "$rabbit_pid" ]; then - echo "RabbitMQ already running" -else - rabbitmq-server & -fi - -hosts_entry=`cat /etc/hosts | grep "127.0.0.1 le.wtf"` -if [ -z "$hosts_entry" ]; then - echo "Adding hosts entry for le.wtf..." - sudo sh -c "echo 127.0.0.1 le.wtf >> /etc/hosts" -fi - -./tests/boulder-start.sh From 5bab6b512f56707632682328be2c8c060516e212 Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Wed, 22 Feb 2017 13:08:56 -0800 Subject: [PATCH 63/88] Refactor main (#4127) * Refactor main to simplify logic * Update tests and comments * Correct main test * increase timeout limit * reset timeout limit * call renew_cert in appropriate main test * Update docstrings and revert signatures of _report_new_cert and _suggest_donation_of_appropriate * replace renew_cert logic * update tests * rename _csr_obtain_cert and add a check to _report_new_cert --- certbot/cli.py | 4 +- certbot/main.py | 202 ++++++++++++++++++++----------------- certbot/renewal.py | 7 +- certbot/storage.py | 20 ++++ certbot/tests/main_test.py | 84 +++++++++------ 5 files changed, 187 insertions(+), 130 deletions(-) diff --git a/certbot/cli.py b/certbot/cli.py index d7fda7574..c0af490d2 100644 --- a/certbot/cli.py +++ b/certbot/cli.py @@ -412,8 +412,8 @@ class HelpfulArgumentParser(object): def __init__(self, args, plugins, detect_defaults=False): from certbot import main self.VERBS = { - "auth": main.obtain_cert, - "certonly": main.obtain_cert, + "auth": main.certonly, + "certonly": main.certonly, "config_changes": main.config_changes, "run": main.run, "install": main.install, diff --git a/certbot/main.py b/certbot/main.py index 7ca346050..118c0f958 100644 --- a/certbot/main.py +++ b/certbot/main.py @@ -49,61 +49,45 @@ USER_CANCELLED = ("User chose to cancel the operation and may " logger = logging.getLogger(__name__) -def _suggest_donation_if_appropriate(config, action): +def _suggest_donation_if_appropriate(config): """Potentially suggest a donation to support Certbot.""" - if config.staging or config.verb == "renew": + assert config.verb != "renew" + if config.staging: # --dry-run implies --staging return - if action not in ["renew", "newcert"]: - return reporter_util = zope.component.getUtility(interfaces.IReporter) msg = ("If you like Certbot, please consider supporting our work by:\n\n" "Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate\n" "Donating to EFF: https://eff.org/donate-le\n\n") reporter_util.add_message(msg, reporter_util.LOW_PRIORITY) - - def _report_successful_dry_run(config): reporter_util = zope.component.getUtility(interfaces.IReporter) - if config.verb != "renew": - reporter_util.add_message("The dry run was successful.", - reporter_util.HIGH_PRIORITY, on_crash=False) + assert config.verb != "renew" + reporter_util.add_message("The dry run was successful.", + reporter_util.HIGH_PRIORITY, on_crash=False) -def _auth_from_available(le_client, config, domains=None, certname=None, lineage=None): +def _get_and_save_cert(le_client, config, domains=None, certname=None, lineage=None): """Authenticate and enroll certificate. This method finds the relevant lineage, figures out what to do with it, then performs that action. Includes calls to hooks, various reports, checks, and requests for user input. - :returns: Tuple of (str action, cert_or_None) as per _find_lineage_for_domains_and_certname - action can be: "newcert" | "renew" | "reinstall" + :returns: the issued certificate or `None` if doing a dry run + :rtype: `storage.RenewableCert` or `None` """ - # If lineage is specified, use that one instead of looking around for - # a matching one. - if lineage is None: - # This will find a relevant matching lineage that exists - action, lineage = _find_lineage_for_domains_and_certname(config, domains, certname) - else: - # Renewal, where we already know the specific lineage we're - # interested in - action = "renew" - - if action == "reinstall": - # The lineage already exists; allow the caller to try installing - # it without getting a new certificate at all. - logger.info("Keeping the existing certificate") - return "reinstall", lineage - hooks.pre_hook(config) try: - if action == "renew": + if lineage is not None: + # Renewal, where we already know the specific lineage we're + # interested in logger.info("Renewing an existing certificate") renewal.renew_cert(config, domains, le_client, lineage) - elif action == "newcert": + else: # TREAT AS NEW REQUEST + assert domains is not None logger.info("Obtaining a new certificate") lineage = le_client.obtain_and_enroll_certificate(domains, certname) if lineage is False: @@ -111,10 +95,7 @@ def _auth_from_available(le_client, config, domains=None, certname=None, lineage finally: hooks.post_hook(config) - if not config.dry_run and not config.verb == "renew": - _report_new_cert(config, lineage.cert, lineage.fullchain) - - return action, lineage + return lineage def _handle_subset_cert_request(config, domains, cert): @@ -236,6 +217,18 @@ def _find_lineage_for_domains(config, domains): elif subset_names_cert is not None: return _handle_subset_cert_request(config, domains, subset_names_cert) +def _find_cert(config, domains, certname): + """Finds an existing certificate object given domains and/or a certificate name. + + :returns: Two-element tuple of a boolean that indicates if this function should be + followed by a call to fetch a certificate from the server, and either a + RenewableCert instance or None. + """ + action, lineage = _find_lineage_for_domains_and_certname(config, domains, certname) + if action == "reinstall": + logger.info("Keeping the existing certificate") + return (action != "reinstall"), lineage + def _find_lineage_for_domains_and_certname(config, domains, certname): """Find appropriate lineage based on given domains and/or certname. @@ -314,26 +307,25 @@ def _report_new_cert(config, cert_path, fullchain_path): :param str fullchain_path: path to full chain """ + if config.dry_run: + _report_successful_dry_run(config) + return + + assert cert_path and fullchain_path, "No certificates saved to report." + expiry = crypto_util.notAfter(cert_path).date() reporter_util = zope.component.getUtility(interfaces.IReporter) - if fullchain_path: - # Print the path to fullchain.pem because that's what modern webservers - # (Nginx and Apache2.4) will want. - and_chain = "and chain have" - path = fullchain_path - else: - # Unless we're in .csr mode and there really isn't one - and_chain = "has " - path = cert_path + # Print the path to fullchain.pem because that's what modern webservers + # (Nginx and Apache2.4) will want. verbswitch = ' with the "certonly" option' if config.verb == "run" else "" # XXX Perhaps one day we could detect the presence of known old webservers # and say something more informative here. - msg = ('Congratulations! Your certificate {0} been saved at {1}.' - ' Your cert will expire on {2}. To obtain a new or tweaked version of this ' - 'certificate in the future, simply run {3} again{4}. ' - 'To non-interactively renew *all* of your certificates, run "{3} renew"' - .format(and_chain, path, expiry, cli.cli_command, verbswitch)) + msg = ('Congratulations! Your certificate and chain have been saved at {0}.' + ' Your cert will expire on {1}. To obtain a new or tweaked version of this ' + 'certificate in the future, simply run {2} again{3}. ' + 'To non-interactively renew *all* of your certificates, run "{2} renew"' + .format(fullchain_path, expiry, cli.cli_command, verbswitch)) reporter_util.add_message(msg, reporter_util.MEDIUM_PRIORITY) @@ -478,6 +470,13 @@ def register(config, unused_plugins): eff.handle_subscription(config) add_msg("Your e-mail address was updated to {0}.".format(config.email)) +def _install_cert(config, le_client, domains, lineage=None): + path_provider = lineage if lineage else config + assert path_provider.cert_path is not None + + le_client.deploy_certificate(domains, path_provider.key_path, + path_provider.cert_path, path_provider.chain_path, path_provider.fullchain_path) + le_client.enhance_config(domains, path_provider.chain_path) def install(config, plugins): """Install a previously obtained cert in a server.""" @@ -492,11 +491,7 @@ def install(config, plugins): domains, _ = _find_domains_or_certname(config, installer) le_client = _init_le_client(config, authenticator=None, installer=installer) - assert config.cert_path is not None # required=True in the subparser - le_client.deploy_certificate( - domains, config.key_path, config.cert_path, config.chain_path, - config.fullchain_path) - le_client.enhance_config(domains, config.chain_path) + _install_cert(config, le_client, domains) def plugins_cmd(config, plugins): # TODO: Use IDisplay rather than print @@ -600,28 +595,32 @@ def run(config, plugins): # pylint: disable=too-many-branches,too-many-locals except errors.PluginSelectionError as e: return e.message - domains, certname = _find_domains_or_certname(config, installer) - # TODO: Handle errors from _init_le_client? le_client = _init_le_client(config, authenticator, installer) - action, lineage = _auth_from_available(le_client, config, domains, certname) + domains, certname = _find_domains_or_certname(config, installer) + should_get_cert, lineage = _find_cert(config, domains, certname) - le_client.deploy_certificate( - domains, lineage.privkey, lineage.cert, - lineage.chain, lineage.fullchain) + new_lineage = lineage + if should_get_cert: + new_lineage = _get_and_save_cert(le_client, config, domains, + certname, lineage) - le_client.enhance_config(domains, lineage.chain) + cert_path = new_lineage.cert_path if new_lineage else None + fullchain_path = new_lineage.fullchain_path if new_lineage else None + _report_new_cert(config, cert_path, fullchain_path) - if action in ("newcert", "reinstall",): + _install_cert(config, le_client, domains, new_lineage) + + if lineage is None or not should_get_cert: display_ops.success_installation(domains) else: display_ops.success_renewal(domains) - _suggest_donation_if_appropriate(config, action) + _suggest_donation_if_appropriate(config) -def _csr_obtain_cert(config, le_client): +def _csr_get_and_save_cert(config, le_client): """Obtain a cert using a user-supplied CSR This works differently in the CSR case (for now) because we don't @@ -633,16 +632,39 @@ def _csr_obtain_cert(config, le_client): if config.dry_run: logger.debug( "Dry run: skipping saving certificate to %s", config.cert_path) - else: - cert_path, _, cert_fullchain = le_client.save_certificate( + return None, None + cert_path, _, fullchain_path = le_client.save_certificate( certr, chain, config.cert_path, config.chain_path, config.fullchain_path) - _report_new_cert(config, cert_path, cert_fullchain) + return cert_path, fullchain_path -def obtain_cert(config, plugins, lineage=None): +def renew_cert(config, plugins, lineage): + """Renew & save an existing cert. Do not install it.""" + try: + # installers are used in auth mode to determine domain names + installer, auth = plug_sel.choose_configurator_plugins(config, plugins, "certonly") + except errors.PluginSelectionError as e: + logger.info("Could not choose appropriate plugin: %s", e) + raise + le_client = _init_le_client(config, auth, installer) + + _get_and_save_cert(le_client, config, lineage=lineage) + + notify = zope.component.getUtility(interfaces.IDisplay).notification + if installer is None: + notify("new certificate deployed without reload, fullchain is {0}".format( + lineage.fullchain), pause=False) + else: + # In case of a renewal, reload server to pick up new certificate. + # In principle we could have a configuration option to inhibit this + # from happening. + installer.restart() + notify("new certificate deployed with reload of {0} server; fullchain is {1}".format( + config.installer, lineage.fullchain), pause=False) + +def certonly(config, plugins): """Authenticate & obtain cert, but do not install it. - This implements the 'certonly' subcommand, and is also called from within the - 'renew' command.""" + This implements the 'certonly' subcommand.""" # SETUP: Select plugins and construct a client instance try: @@ -653,34 +675,26 @@ def obtain_cert(config, plugins, lineage=None): raise le_client = _init_le_client(config, auth, installer) - # SHOWTIME: Possibly obtain/renew a cert, and set action to renew | newcert | reinstall - if config.csr is None: # the common case - domains, certname = _find_domains_or_certname(config, installer) - action, _ = _auth_from_available(le_client, config, domains, certname, lineage) - else: - assert lineage is None, "Did not expect a CSR with a RenewableCert" - _csr_obtain_cert(config, le_client) - action = "newcert" + if config.csr: + cert_path, fullchain_path = _csr_get_and_save_cert(config, le_client) + _report_new_cert(config, cert_path, fullchain_path) + _suggest_donation_if_appropriate(config) + return - # POSTPRODUCTION: Cleanup, deployment & reporting - notify = zope.component.getUtility(interfaces.IDisplay).notification - if config.dry_run: - _report_successful_dry_run(config) - elif config.verb == "renew": - if installer is None: - notify("new certificate deployed without reload, fullchain is {0}".format( - lineage.fullchain), pause=False) - else: - # In case of a renewal, reload server to pick up new certificate. - # In principle we could have a configuration option to inhibit this - # from happening. - installer.restart() - notify("new certificate deployed with reload of {0} server; fullchain is {1}".format( - config.installer, lineage.fullchain), pause=False) - elif action == "reinstall" and config.verb == "certonly": + domains, certname = _find_domains_or_certname(config, installer) + should_get_cert, lineage = _find_cert(config, domains, certname) + + if not should_get_cert: + notify = zope.component.getUtility(interfaces.IDisplay).notification notify("Certificate not yet due for renewal; no action taken.", pause=False) - _suggest_donation_if_appropriate(config, action) + return + lineage = _get_and_save_cert(le_client, config, domains, certname, lineage) + + cert_path = lineage.cert_path if lineage else None + fullchain_path = lineage.fullchain_path if lineage else None + _report_new_cert(config, cert_path, fullchain_path) + _suggest_donation_if_appropriate(config) def renew(config, unused_plugins): """Renew previously-obtained certificates.""" diff --git a/certbot/renewal.py b/certbot/renewal.py index 6b61b0841..a0cc872a0 100644 --- a/certbot/renewal.py +++ b/certbot/renewal.py @@ -410,7 +410,12 @@ def handle_renewal_request(config): if should_renew(lineage_config, renewal_candidate): plugins = plugins_disco.PluginsRegistry.find_all() from certbot import main - main.obtain_cert(lineage_config, plugins, renewal_candidate) + # domains have been restored into lineage_config by reconstitute + # but they're unnecessary anyway because renew_cert here + # will just grab them from the certificate + # we already know it's time to renew based on should_renew + # and we have a lineage in renewal_candidate + main.renew_cert(lineage_config, plugins, renewal_candidate) renew_successes.append(renewal_candidate.fullchain) else: renew_skipped.append(renewal_candidate.fullchain) diff --git a/certbot/storage.py b/certbot/storage.py index 4f1cd0a9d..dacc73c4c 100644 --- a/certbot/storage.py +++ b/certbot/storage.py @@ -391,6 +391,26 @@ class RenewableCert(object): self._update_symlinks() self._check_symlinks() + @property + def key_path(self): + """Duck type for self.privkey""" + return self.privkey + + @property + def cert_path(self): + """Duck type for self.cert""" + return self.cert + + @property + def chain_path(self): + """Duck type for self.chain""" + return self.chain + + @property + def fullchain_path(self): + """Duck type for self.fullchain""" + return self.fullchain + @property def target_expiry(self): """The current target certificate's expiration datetime diff --git a/certbot/tests/main_test.py b/certbot/tests/main_test.py index 22b709f69..3520eb063 100644 --- a/certbot/tests/main_test.py +++ b/certbot/tests/main_test.py @@ -57,17 +57,21 @@ class RunTest(unittest.TestCase): def setUp(self): self.domain = 'example.org' self.patches = [ - mock.patch('certbot.main._auth_from_available'), + mock.patch('certbot.main._get_and_save_cert'), mock.patch('certbot.main.display_ops.success_installation'), mock.patch('certbot.main.display_ops.success_renewal'), mock.patch('certbot.main._init_le_client'), - mock.patch('certbot.main._suggest_donation_if_appropriate')] + mock.patch('certbot.main._suggest_donation_if_appropriate'), + mock.patch('certbot.main._report_new_cert'), + mock.patch('certbot.main._find_cert')] self.mock_auth = self.patches[0].start() self.mock_success_installation = self.patches[1].start() self.mock_success_renewal = self.patches[2].start() self.mock_init = self.patches[3].start() self.mock_suggest_donation = self.patches[4].start() + self.mock_report_cert = self.patches[5].start() + self.mock_find_cert = self.patches[6].start() def tearDown(self): for patch in self.patches: @@ -83,23 +87,26 @@ class RunTest(unittest.TestCase): run(config, plugins) def test_newcert_success(self): - self.mock_auth.return_value = ('newcert', mock.Mock()) + self.mock_auth.return_value = mock.Mock() + self.mock_find_cert.return_value = True, None self._call() self.mock_success_installation.assert_called_once_with([self.domain]) def test_reinstall_success(self): - self.mock_auth.return_value = ('reinstall', mock.Mock()) + self.mock_auth.return_value = mock.Mock() + self.mock_find_cert.return_value = False, mock.Mock() self._call() self.mock_success_installation.assert_called_once_with([self.domain]) def test_renewal_success(self): - self.mock_auth.return_value = ('renewal', mock.Mock()) + self.mock_auth.return_value = mock.Mock() + self.mock_find_cert.return_value = True, mock.Mock() self._call() self.mock_success_renewal.assert_called_once_with([self.domain]) -class ObtainCertTest(unittest.TestCase): - """Tests for certbot.main.obtain_cert.""" +class CertonlyTest(unittest.TestCase): + """Tests for certbot.main.certonly.""" def setUp(self): self.get_utility_patch = test_util.patch_get_utility() @@ -114,15 +121,20 @@ class ObtainCertTest(unittest.TestCase): cli.prepare_and_parse_args(plugins, args)) with mock.patch('certbot.main._init_le_client') as mock_init: - main.obtain_cert(config, plugins) + with mock.patch('certbot.main._suggest_donation_if_appropriate'): + main.certonly(config, plugins) return mock_init() # returns the client - @mock.patch('certbot.main._auth_from_available') - def test_no_reinstall_text_pause(self, mock_auth): + @mock.patch('certbot.main._find_cert') + @mock.patch('certbot.main._get_and_save_cert') + @mock.patch('certbot.main._report_new_cert') + def test_no_reinstall_text_pause(self, unused_report, mock_auth, + mock_find_cert): mock_notification = self.mock_get_utility().notification mock_notification.side_effect = self._assert_no_pause - mock_auth.return_value = ('reinstall', mock.ANY) + mock_auth.return_value = mock.Mock() + mock_find_cert.return_value = False, None self._call('certonly --webroot -d example.com'.split()) def _assert_no_pause(self, message, pause=True): @@ -495,22 +507,23 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods self._cli_missing_flag(args, "specify a plugin") args.extend(['--standalone', '-d', 'eg.is']) self._cli_missing_flag(args, "register before running") - with mock.patch('certbot.main._auth_from_available'): + with mock.patch('certbot.main._get_and_save_cert'): with mock.patch('certbot.main.client.acme_from_config_key'): args.extend(['--email', 'io@io.is']) self._cli_missing_flag(args, "--agree-tos") + @mock.patch('certbot.main._report_new_cert') @mock.patch('certbot.main.client.acme_client.Client') @mock.patch('certbot.main._determine_account') @mock.patch('certbot.main.client.Client.obtain_and_enroll_certificate') - @mock.patch('certbot.main._auth_from_available') - def test_user_agent(self, afa, _obt, det, _client): + @mock.patch('certbot.main._get_and_save_cert') + def test_user_agent(self, gsc, _obt, det, _client, unused_report): # Normally the client is totally mocked out, but here we need more # arguments to automate it... args = ["--standalone", "certonly", "-m", "none@none.com", "-d", "example.com", '--agree-tos'] + self.standard_args det.return_value = mock.MagicMock(), None - afa.return_value = "newcert", mock.MagicMock() + gsc.return_value = mock.MagicMock() with mock.patch('certbot.main.client.acme_client.ClientNetwork') as acme_net: self._call_no_clientmock(args) @@ -535,8 +548,9 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods '--key-path', 'key', '--chain-path', 'chain']) self.assertEqual(mock_pick_installer.call_count, 1) + @mock.patch('certbot.main._report_new_cert') @mock.patch('certbot.util.exe_exists') - def test_configurator_selection(self, mock_exe_exists): + def test_configurator_selection(self, mock_exe_exists, unused_report): mock_exe_exists.return_value = True real_plugins = disco.PluginsRegistry.find_all() args = ['--apache', '--authenticator', 'standalone'] @@ -562,13 +576,13 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods self._cli_missing_flag(["--standalone"], "With the standalone plugin, you probably") with mock.patch("certbot.main._init_le_client") as mock_init: - with mock.patch("certbot.main._auth_from_available") as mock_afa: - mock_afa.return_value = (mock.MagicMock(), mock.MagicMock()) + with mock.patch("certbot.main._get_and_save_cert") as mock_gsc: + mock_gsc.return_value = mock.MagicMock() self._call(["certonly", "--manual", "-d", "foo.bar"]) unused_config, auth, unused_installer = mock_init.call_args[0] self.assertTrue(isinstance(auth, manual.Authenticator)) - with mock.patch('certbot.main.obtain_cert') as mock_certonly: + with mock.patch('certbot.main.certonly') as mock_certonly: self._call(["auth", "--standalone"]) self.assertEqual(1, mock_certonly.call_count) @@ -656,12 +670,12 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods chain = 'chain' fullchain = 'fullchain' - with mock.patch('certbot.main.obtain_cert') as mock_obtaincert: + with mock.patch('certbot.main.certonly') as mock_certonly: self._call(['certonly', '--cert-path', cert, '--key-path', 'key', '--chain-path', 'chain', '--fullchain-path', 'fullchain']) - config, unused_plugins = mock_obtaincert.call_args[0] + config, unused_plugins = mock_certonly.call_args[0] self.assertEqual(config.cert_path, os.path.abspath(cert)) self.assertEqual(config.key_path, os.path.abspath(key)) self.assertEqual(config.chain_path, os.path.abspath(chain)) @@ -747,7 +761,8 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods date = '1970-01-01' mock_notAfter().date.return_value = date - mock_lineage = mock.MagicMock(cert=cert_path, fullchain=cert_path) + mock_lineage = mock.MagicMock(cert=cert_path, fullchain=cert_path, + fullchain_path=cert_path) mock_client = mock.MagicMock() mock_client.obtain_and_enroll_certificate.return_value = mock_lineage self._certonly_new_request_common(mock_client) @@ -770,7 +785,8 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods # pylint: disable=too-many-locals,too-many-arguments cert_path = test_util.vector_path('cert.pem') chain_path = '/etc/letsencrypt/live/foo.bar/fullchain.pem' - mock_lineage = mock.MagicMock(cert=cert_path, fullchain=chain_path) + mock_lineage = mock.MagicMock(cert=cert_path, fullchain=chain_path, + cert_path=cert_path, fullchain_path=chain_path) mock_lineage.should_autorenew.return_value = due_for_renewal mock_lineage.has_pending_deployment.return_value = False mock_lineage.names.return_value = ['isnot.org'] @@ -821,7 +837,8 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods return mock_lineage, mock_get_utility, stdout - def test_certonly_renewal(self): + @mock.patch('certbot.crypto_util.notAfter') + def test_certonly_renewal(self, unused_notafter): lineage, get_utility, _ = self._test_renewal_common(True, []) self.assertEqual(lineage.save_successor.call_count, 1) lineage.update_all_links_to.assert_called_once_with( @@ -830,7 +847,8 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods self.assertTrue('fullchain.pem' in cert_msg) self.assertTrue('donate' in get_utility().add_message.call_args[0][0]) - def test_certonly_renewal_triggers(self): + @mock.patch('certbot.crypto_util.notAfter') + def test_certonly_renewal_triggers(self, unused_notafter): # --dry-run should force renewal _, get_utility, _ = self._test_renewal_common(False, ['--dry-run', '--keep'], log_out="simulating renewal") @@ -932,15 +950,15 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods if names is not None: mock_lineage.names.return_value = names mock_rc.return_value = mock_lineage - with mock.patch('certbot.main.obtain_cert') as mock_obtain_cert: + with mock.patch('certbot.main.renew_cert') as mock_renew_cert: kwargs.setdefault('args', ['renew']) self._test_renewal_common(True, None, should_renew=False, **kwargs) if assert_oc_called is not None: if assert_oc_called: - self.assertTrue(mock_obtain_cert.called) + self.assertTrue(mock_renew_cert.called) else: - self.assertFalse(mock_obtain_cert.called) + self.assertFalse(mock_renew_cert.called) def test_renew_no_renewalparams(self): self._test_renew_common(assert_oc_called=False, error_expected=True) @@ -1000,8 +1018,8 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods mock_rc.return_value = mock_lineage mock_lineage.configuration = { 'renewalparams': {'authenticator': 'webroot'}} - with mock.patch('certbot.main.obtain_cert') as mock_obtain_cert: - mock_obtain_cert.side_effect = Exception + with mock.patch('certbot.main.renew_cert') as mock_renew_cert: + mock_renew_cert.side_effect = Exception self._test_renewal_common(True, None, error_expected=True, args=['renew'], should_renew=False) @@ -1035,12 +1053,12 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods mock_client = mock.MagicMock() mock_client.obtain_certificate_from_csr.return_value = (certr, chain) cert_path = '/etc/letsencrypt/live/example.com/cert.pem' - mock_client.save_certificate.return_value = cert_path, None, None + full_path = '/etc/letsencrypt/live/example.com/fullchain.pem' + mock_client.save_certificate.return_value = cert_path, None, full_path with mock.patch('certbot.main._init_le_client') as mock_init: mock_init.return_value = mock_client with test_util.patch_get_utility() as mock_get_utility: chain_path = '/etc/letsencrypt/live/example.com/chain.pem' - full_path = '/etc/letsencrypt/live/example.com/fullchain.pem' args = ('-a standalone certonly --csr {0} --cert-path {1} ' '--chain-path {2} --fullchain-path {3}').format( CSR, cert_path, chain_path, full_path).split() @@ -1060,7 +1078,7 @@ class MainTest(unittest.TestCase): # pylint: disable=too-many-public-methods def test_certonly_csr(self): mock_get_utility = self._test_certonly_csr_common() cert_msg = mock_get_utility().add_message.call_args_list[0][0][0] - self.assertTrue('cert.pem' in cert_msg) + self.assertTrue('fullchain.pem' in cert_msg) self.assertTrue( 'donate' in mock_get_utility().add_message.call_args[0][0]) From a92ca8e97c38a8adbdf5088af058ec50e55b2602 Mon Sep 17 00:00:00 2001 From: Jacob Hoffman-Andrews Date: Wed, 22 Feb 2017 18:48:01 -0800 Subject: [PATCH 64/88] Add default timeout to ClientNetwork. (#4217) In https://community.letsencrypt.org/t/letsencrypt-cli-hangs-on-certificate-request/27211, a community member pointed out that Certbot seems to hang when there are routing problems. --- acme/acme/client.py | 1 + acme/acme/client_test.py | 22 +++++++++++++++++----- 2 files changed, 18 insertions(+), 5 deletions(-) diff --git a/acme/acme/client.py b/acme/acme/client.py index d01555c75..168574d58 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -607,6 +607,7 @@ class ClientNetwork(object): # pylint: disable=too-many-instance-attributes kwargs['verify'] = self.verify_ssl kwargs.setdefault('headers', {}) kwargs['headers'].setdefault('User-Agent', self.user_agent) + kwargs.setdefault('timeout', 45) # timeout after 45 seconds response = self.session.request(method, url, *args, **kwargs) # If content is DER, log the base64 of it instead of raw bytes, to keep # binary data out of the logs. diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index 0f4506203..b7bd0740c 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -532,7 +532,7 @@ class ClientNetworkTest(unittest.TestCase): 'HEAD', 'http://example.com/', 'foo', bar='baz')) self.net.session.request.assert_called_once_with( 'HEAD', 'http://example.com/', 'foo', - headers=mock.ANY, verify=mock.ANY, bar='baz') + headers=mock.ANY, verify=mock.ANY, timeout=mock.ANY, bar='baz') @mock.patch('acme.client.logger') def test_send_request_get_der(self, mock_logger): @@ -542,7 +542,8 @@ class ClientNetworkTest(unittest.TestCase): headers={"Content-Type": "application/pkix-cert"}, content=b"hi") # pylint: disable=protected-access - self.net._send_request('HEAD', 'http://example.com/', 'foo', bar='baz') + self.net._send_request('HEAD', 'http://example.com/', 'foo', + timeout=mock.ANY, bar='baz') mock_logger.debug.assert_called_once_with( 'Received response:\nHTTP %d\n%s\n\n%s', 200, 'Content-Type: application/pkix-cert', b'aGk=') @@ -555,7 +556,7 @@ class ClientNetworkTest(unittest.TestCase): 'POST', 'http://example.com/', 'foo', data='qux', bar='baz')) self.net.session.request.assert_called_once_with( 'POST', 'http://example.com/', 'foo', - headers=mock.ANY, verify=mock.ANY, data='qux', bar='baz') + headers=mock.ANY, verify=mock.ANY, timeout=mock.ANY, data='qux', bar='baz') def test_send_request_verify_ssl(self): # pylint: disable=protected-access @@ -568,7 +569,8 @@ class ClientNetworkTest(unittest.TestCase): self.response, self.net._send_request('GET', 'http://example.com/')) self.net.session.request.assert_called_once_with( - 'GET', 'http://example.com/', verify=verify, headers=mock.ANY) + 'GET', 'http://example.com/', verify=verify, + timeout=mock.ANY, headers=mock.ANY) def test_send_request_user_agent(self): self.net.session = mock.MagicMock() @@ -577,13 +579,23 @@ class ClientNetworkTest(unittest.TestCase): headers={'bar': 'baz'}) self.net.session.request.assert_called_once_with( 'GET', 'http://example.com/', verify=mock.ANY, + timeout=mock.ANY, headers={'User-Agent': 'acme-python-test', 'bar': 'baz'}) self.net._send_request('GET', 'http://example.com/', headers={'User-Agent': 'foo2'}) self.net.session.request.assert_called_with( 'GET', 'http://example.com/', - verify=mock.ANY, headers={'User-Agent': 'foo2'}) + verify=mock.ANY, timeout=mock.ANY, headers={'User-Agent': 'foo2'}) + + def test_send_request_timeout(self): + self.net.session = mock.MagicMock() + # pylint: disable=protected-access + self.net._send_request('GET', 'http://example.com/', + headers={'bar': 'baz'}) + self.net.session.request.assert_called_once_with( + mock.ANY, mock.ANY, verify=mock.ANY, headers=mock.ANY, + timeout=45) def test_del(self): sess = mock.MagicMock() From b1a4280519b1a2d271e0e770a87a2a37915a381a Mon Sep 17 00:00:00 2001 From: silverwind Date: Thu, 23 Feb 2017 03:50:56 +0100 Subject: [PATCH 65/88] Show error details for the nginx config parser (#4221) Nginx parser errors now include helpful details on where parsing has actually failed. Related: https://github.com/certbot/certbot/issues/3798 --- certbot-nginx/certbot_nginx/parser.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/certbot-nginx/certbot_nginx/parser.py b/certbot-nginx/certbot_nginx/parser.py index 1a2c85c2c..c586aa459 100644 --- a/certbot-nginx/certbot_nginx/parser.py +++ b/certbot-nginx/certbot_nginx/parser.py @@ -205,8 +205,8 @@ class NginxParser(object): trees.append(parsed) except IOError: logger.warning("Could not open file: %s", item) - except pyparsing.ParseException: - logger.debug("Could not parse file: %s", item) + except pyparsing.ParseException as err: + logger.debug("Could not parse file: %s due to %s", item, err) return trees def _parse_ssl_options(self, ssl_options): @@ -216,8 +216,8 @@ class NginxParser(object): return nginxparser.load(_file).spaced except IOError: logger.warn("Missing NGINX TLS options file: %s", ssl_options) - except pyparsing.ParseBaseException: - logger.debug("Could not parse file: %s", ssl_options) + except pyparsing.ParseBaseException as err: + logger.debug("Could not parse file: %s due to %s", ssl_options, err) return [] def _set_locations(self, ssl_options): From 52ce335ff095c8bfde7ee624c57fafef673c19af Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Thu, 23 Feb 2017 18:31:23 -0800 Subject: [PATCH 66/88] lineage_for_certname should return None if there is no existing renewal file (#4243) * lineage_for_certname should return None if there is no existing renewal file * add unit test * add regression test to integration test * revent boulder-start to boulder-fetch --- certbot/cert_manager.py | 5 ++++- certbot/tests/cert_manager_test.py | 10 ++++++++++ tests/boulder-integration.sh | 2 ++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/certbot/cert_manager.py b/certbot/cert_manager.py index 71a5fe6fa..35d539a16 100644 --- a/certbot/cert_manager.py +++ b/certbot/cert_manager.py @@ -100,7 +100,10 @@ def lineage_for_certname(cli_config, certname): configs_dir = cli_config.renewal_configs_dir # Verify the directory is there util.make_or_verify_dir(configs_dir, mode=0o755, uid=os.geteuid()) - renewal_file = storage.renewal_file_for_certname(cli_config, certname) + try: + renewal_file = storage.renewal_file_for_certname(cli_config, certname) + except errors.CertStorageError: + return None try: return storage.RenewableCert(renewal_file, cli_config) except (errors.CertStorageError, IOError): diff --git a/certbot/tests/cert_manager_test.py b/certbot/tests/cert_manager_test.py index 1fa68d195..473970870 100644 --- a/certbot/tests/cert_manager_test.py +++ b/certbot/tests/cert_manager_test.py @@ -290,6 +290,16 @@ class LineageForCertnameTest(BaseCertManagerTest): None) self.assertTrue(mock_make_or_verify_dir.called) + @mock.patch('certbot.util.make_or_verify_dir') + @mock.patch('certbot.storage.renewal_file_for_certname') + def test_no_renewal_file(self, mock_renewal_conf_file, + mock_make_or_verify_dir): + mock_renewal_conf_file.side_effect = errors.CertStorageError() + from certbot import cert_manager + self.assertEqual(cert_manager.lineage_for_certname(self.cli_config, "example.com"), + None) + self.assertTrue(mock_make_or_verify_dir.called) + class DomainsForCertnameTest(BaseCertManagerTest): """Tests for certbot.cert_manager.domains_for_certname""" diff --git a/tests/boulder-integration.sh b/tests/boulder-integration.sh index 2ddd3c04b..ca6f48e60 100755 --- a/tests/boulder-integration.sh +++ b/tests/boulder-integration.sh @@ -100,6 +100,8 @@ common certonly -a manual -d le.wtf --rsa-key-size 4096 \ common certonly -a manual -d dns.le.wtf --preferred-challenges dns,tls-sni \ --manual-auth-hook ./tests/manual-dns-auth.sh +common certonly --cert-name newname -d newname.le.wtf + export CSR_PATH="${root}/csr.der" KEY_PATH="${root}/key.pem" \ OPENSSL_CNF=examples/openssl.cnf ./examples/generate-csr.sh le3.wtf From 28cbd6e7d385ce0bd4c28e49e422d6eaf51f9cdb Mon Sep 17 00:00:00 2001 From: Joona Hoikkala Date: Fri, 24 Feb 2017 06:28:36 +0200 Subject: [PATCH 67/88] Fix for case sensitivity when looking for vhosts (#4193) --- certbot-apache/certbot_apache/configurator.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/certbot-apache/certbot_apache/configurator.py b/certbot-apache/certbot_apache/configurator.py index a956341eb..5639dae83 100644 --- a/certbot-apache/certbot_apache/configurator.py +++ b/certbot-apache/certbot_apache/configurator.py @@ -580,7 +580,7 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator): ("/files%s//*[label()=~regexp('%s')]" % (vhost_path, parser.case_i("VirtualHost")))) paths = [path for path in paths if - os.path.basename(path) == "VirtualHost"] + os.path.basename(path.lower()) == "virtualhost"] for path in paths: new_vhost = self._create_vhost(path) if not new_vhost: From e02d641490181fb7c7722ec85b52052fe07f3534 Mon Sep 17 00:00:00 2001 From: Joona Hoikkala Date: Fri, 24 Feb 2017 21:40:03 +0200 Subject: [PATCH 68/88] Docker changes for easier testing (#4249) These changes allow developers to run tests directly from the host machine using Docker, and to enable ipdb inside the container. docker-compose.yml is upgraded to version 2 format. This means that you need docker-engine version >= 1.10.0 instead of previous requirement of version >= 1.9.1. The reason for this is to be able to use custom Dockerfile (Dockerfile-dev in this case) in build context. ipdb has been added to dev dependencies to be able to be able to debug the code without installing it on every docker run. This is also what we recommend for debugging in the developer documentation, so there really is no reason not to install it with the dev dependencies. setuptools is being upgraded to a newer version to be able to run coverage tests. This was using the older version of setuptools for some reason, and without the upgrade, coverage tests would fail horribly. Upgrading remedies the situation. Few examples: Run unit tests for certbot-apache `docker-compose run --rm --service-ports development bash -c 'cd src;nosetests -v certbot-apache'` Run coverage tests `docker-compose run --rm --service-ports development bash -c 'cd src;./tox.cover.sh'` Run linter `docker-compose run --rm --service-ports development bash -c 'cd src;tox -e lint'` --- Dockerfile-dev | 3 ++- docker-compose.yml | 26 +++++++++++++++----------- setup.py | 1 + 3 files changed, 18 insertions(+), 12 deletions(-) diff --git a/Dockerfile-dev b/Dockerfile-dev index 098fae523..dbb45f75e 100644 --- a/Dockerfile-dev +++ b/Dockerfile-dev @@ -58,7 +58,8 @@ RUN virtualenv --no-site-packages -p python2 /opt/certbot/venv && \ -e /opt/certbot/src/certbot-nginx \ -e /opt/certbot/src/letshelp-certbot \ -e /opt/certbot/src/certbot-compatibility-test \ - -e /opt/certbot/src[dev,docs] + -e /opt/certbot/src[dev,docs] && \ + /opt/certbot/venv/bin/pip install -U setuptools # install in editable mode (-e) to save space: it's not possible to # "rm -rf /opt/certbot/src" (it's stays in the underlaying image); diff --git a/docker-compose.yml b/docker-compose.yml index 8b2a8e9a3..00d3d4c72 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,14 +1,18 @@ -production: - build: . - ports: - - "443:443" +version: '2' +services: + production: + build: . + ports: + - "443:443" # For development, mount git root to /opt/certbot/src in order to # make the dev workflow more vagrant-like. -development: - build: . - ports: - - "443:443" - volumes: - - .:/opt/certbot/src - - /opt/certbot/venv + development: + build: + context: . + dockerfile: Dockerfile-dev + ports: + - "443:443" + volumes: + - .:/opt/certbot/src + - /opt/certbot/venv diff --git a/setup.py b/setup.py index 529459cf0..0c47b973f 100644 --- a/setup.py +++ b/setup.py @@ -69,6 +69,7 @@ dev_extras = [ # Pin astroid==1.3.5, pylint==1.4.2 as a workaround for #289 'astroid==1.3.5', 'coverage', + 'ipdb', 'nose', 'pylint==1.4.2', # upstream #248 'tox', From d066f8b38b432000d4217c1cd39bc292938796ce Mon Sep 17 00:00:00 2001 From: Noah Swartz Date: Fri, 24 Feb 2017 13:08:25 -0800 Subject: [PATCH 69/88] created an issue template (#4201) * created an issue template * bmw changes --- ISSUE_TEMPLATE.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 ISSUE_TEMPLATE.md diff --git a/ISSUE_TEMPLATE.md b/ISSUE_TEMPLATE.md new file mode 100644 index 000000000..e4e56f93d --- /dev/null +++ b/ISSUE_TEMPLATE.md @@ -0,0 +1,16 @@ +## My operating system is (include version): + + +## My web server is (include version): + + +## How did you install Certbot: + + +## What command did you run and what output did it produce? + + +## Can you provide a Certbot error log showing the issue? +###### It is stored by default in `/var/log/letsencrypt` - feel free to redact domain names, e-mail and IP addresses as you see fit + + From 7d02e129f9e0fd9abed850621d78e91dd7684ff8 Mon Sep 17 00:00:00 2001 From: Yen Chi Hsuan Date: Sat, 25 Feb 2017 10:21:21 +0800 Subject: [PATCH 70/88] Fix certbot-apache tests on Python 3 (#4172) --- certbot-apache/certbot_apache/obj.py | 10 ++++++++++ certbot-apache/certbot_apache/parser.py | 14 +++++++++---- .../certbot_apache/tests/configurator_test.py | 20 ++++++++++--------- .../certbot_apache/tests/display_ops_test.py | 2 +- .../certbot_apache/tests/tls_sni_01_test.py | 2 +- certbot-apache/certbot_apache/tls_sni_01.py | 2 +- tox.ini | 8 ++++++++ 7 files changed, 42 insertions(+), 16 deletions(-) diff --git a/certbot-apache/certbot_apache/obj.py b/certbot-apache/certbot_apache/obj.py index b29b0e0ee..30cb24844 100644 --- a/certbot-apache/certbot_apache/obj.py +++ b/certbot-apache/certbot_apache/obj.py @@ -25,6 +25,11 @@ class Addr(common.Addr): def __repr__(self): return "certbot_apache.obj.Addr(" + repr(self.tup) + ")" + def __hash__(self): + # Python 3 requires explicit overridden for __hash__ if __eq__ or + # __cmp__ is overridden. See https://bugs.python.org/issue2235 + return super(Addr, self).__hash__() + def _addr_less_specific(self, addr): """Returns if addr.get_addr() is more specific than self.get_addr().""" # pylint: disable=protected-access @@ -174,6 +179,11 @@ class VirtualHost(object): # pylint: disable=too-few-public-methods def __ne__(self, other): return not self.__eq__(other) + def __hash__(self): + return hash((self.filep, self.path, + tuple(self.addrs), tuple(self.get_names()), + self.ssl, self.enabled, self.modmacro)) + def conflicts(self, addrs): """See if vhost conflicts with any of the addrs. diff --git a/certbot-apache/certbot_apache/parser.py b/certbot-apache/certbot_apache/parser.py index 6bb6ff170..275a01e7f 100644 --- a/certbot-apache/certbot_apache/parser.py +++ b/certbot-apache/certbot_apache/parser.py @@ -1,10 +1,12 @@ """ApacheParser is a member object of the ApacheConfigurator class.""" import fnmatch -import itertools import logging import os import re import subprocess +import sys + +import six from certbot import errors @@ -87,7 +89,7 @@ class ApacheParser(object): while len(self.modules) != prev_size: prev_size = len(self.modules) - for match_name, match_filename in itertools.izip( + for match_name, match_filename in six.moves.zip( iterator, iterator): self.modules.add(self.get_arg(match_name)) self.modules.add( @@ -460,8 +462,12 @@ class ApacheParser(object): :rtype: str """ - # This strips off final /Z(?ms) - return fnmatch.translate(clean_fn_match)[:-7] + if sys.version_info < (3, 6): + # This strips off final /Z(?ms) + return fnmatch.translate(clean_fn_match)[:-7] + else: # pragma: no cover + # Since Python 3.6, it returns a different pattern like (?s:.*\.load)\Z + return fnmatch.translate(clean_fn_match)[4:-3] def _parse_file(self, filepath): """Parse file with Augeas diff --git a/certbot-apache/certbot_apache/tests/configurator_test.py b/certbot-apache/certbot_apache/tests/configurator_test.py index 01361c8f0..937694267 100644 --- a/certbot-apache/certbot_apache/tests/configurator_test.py +++ b/certbot-apache/certbot_apache/tests/configurator_test.py @@ -6,6 +6,8 @@ import socket import unittest import mock +# six is used in mock.patch() +import six # pylint: disable=unused-import from acme import challenges @@ -517,12 +519,12 @@ class MultipleVhostsTest(util.ApacheTest): # Test self.config.prepare_server_https("8080", temp=True) self.assertEqual(mock_add_dir.call_count, 3) - self.assertEqual(mock_add_dir.call_args_list[0][0][2], - ["1.2.3.4:8080", "https"]) - self.assertEqual(mock_add_dir.call_args_list[1][0][2], - ["[::1]:8080", "https"]) - self.assertEqual(mock_add_dir.call_args_list[2][0][2], - ["1.1.1.1:8080", "https"]) + call_args_list = [mock_add_dir.call_args_list[i][0][2] for i in range(3)] + self.assertEqual( + sorted(call_args_list), + sorted([["1.2.3.4:8080", "https"], + ["[::1]:8080", "https"], + ["1.1.1.1:8080", "https"]])) # mock_get.side_effect = ["1.2.3.4:80", "[::1]:80"] # mock_find.return_value = ["test1", "test2", "test3"] @@ -662,7 +664,7 @@ class MultipleVhostsTest(util.ApacheTest): # This calls open self.config.reverter.register_file_creation = mock.Mock() mock_open.side_effect = IOError - with mock.patch("__builtin__.open", mock_open): + with mock.patch("six.moves.builtins.open", mock_open): self.assertRaises( errors.PluginError, self.config.make_vhost_ssl, self.vh_truth[0]) @@ -1208,13 +1210,13 @@ class MultipleVhostsTest(util.ApacheTest): achall1 = achallenges.KeyAuthorizationAnnotatedChallenge( challb=acme_util.chall_to_challb( challenges.TLSSNI01( - token="jIq_Xy1mXGN37tb4L6Xj_es58fW571ZNyXekdZzhh7Q"), + token=b"jIq_Xy1mXGN37tb4L6Xj_es58fW571ZNyXekdZzhh7Q"), "pending"), domain="encryption-example.demo", account_key=account_key) achall2 = achallenges.KeyAuthorizationAnnotatedChallenge( challb=acme_util.chall_to_challb( challenges.TLSSNI01( - token="uqnaPzxtrndteOqtrXb0Asl5gOJfWAnnx6QJyvcmlDU"), + token=b"uqnaPzxtrndteOqtrXb0Asl5gOJfWAnnx6QJyvcmlDU"), "pending"), domain="certbot.demo", account_key=account_key) diff --git a/certbot-apache/certbot_apache/tests/display_ops_test.py b/certbot-apache/certbot_apache/tests/display_ops_test.py index ec6eee3f2..f8b75022e 100644 --- a/certbot-apache/certbot_apache/tests/display_ops_test.py +++ b/certbot-apache/certbot_apache/tests/display_ops_test.py @@ -38,7 +38,7 @@ class SelectVhostTest(unittest.TestCase): try: self._call(self.vhosts) except errors.MissingCommandlineFlag as e: - self.assertTrue("vhost ambiguity" in e.message) + self.assertTrue("vhost ambiguity" in str(e)) @certbot_util.patch_get_utility() def test_more_info_cancel(self, mock_util): diff --git a/certbot-apache/certbot_apache/tests/tls_sni_01_test.py b/certbot-apache/certbot_apache/tests/tls_sni_01_test.py index 5e369e3db..62464d5d0 100644 --- a/certbot-apache/certbot_apache/tests/tls_sni_01_test.py +++ b/certbot-apache/certbot_apache/tests/tls_sni_01_test.py @@ -105,7 +105,7 @@ class TlsSniPerformTest(util.ApacheTest): for achall in self.achalls: self.sni.add_chall(achall) z_domain = achall.response(self.auth_key).z_domain - z_domains.append(set([z_domain])) + z_domains.append(set([z_domain.decode('ascii')])) self.sni._mod_config() # pylint: disable=protected-access self.sni.configurator.save() diff --git a/certbot-apache/certbot_apache/tls_sni_01.py b/certbot-apache/certbot_apache/tls_sni_01.py index d9e294119..65a66d2fd 100644 --- a/certbot-apache/certbot_apache/tls_sni_01.py +++ b/certbot-apache/certbot_apache/tls_sni_01.py @@ -184,7 +184,7 @@ class ApacheTlsSni01(common.TLSSNI01): # https://docs.python.org/2.7/reference/lexical_analysis.html return self.VHOST_TEMPLATE.format( vhost=ips, - server_name=achall.response(achall.account_key).z_domain, + server_name=achall.response(achall.account_key).z_domain.decode('ascii'), ssl_options_conf_path=self.configurator.mod_ssl_conf, cert_path=self.get_cert_path(achall), key_path=self.get_key_path(achall), diff --git a/tox.ini b/tox.ini index e6317e665..232010d40 100644 --- a/tox.ini +++ b/tox.ini @@ -46,6 +46,8 @@ commands = nosetests -v acme --processes=-1 pip install -e .[dev] nosetests -v certbot --processes=-1 --process-timeout=100 + pip install -e certbot-apache + nosetests -v certbot_apache --processes=-1 --process-timeout=80 [testenv:py34] commands = @@ -53,6 +55,8 @@ commands = nosetests -v acme --processes=-1 pip install -e .[dev] nosetests -v certbot --processes=-1 --process-timeout=100 + pip install -e certbot-apache + nosetests -v certbot_apache --processes=-1 --process-timeout=80 [testenv:py35] commands = @@ -60,6 +64,8 @@ commands = nosetests -v acme --processes=-1 pip install -e .[dev] nosetests -v certbot --processes=-1 --process-timeout=100 + pip install -e certbot-apache + nosetests -v certbot_apache --processes=-1 --process-timeout=80 [testenv:py36] commands = @@ -67,6 +73,8 @@ commands = nosetests -v acme --processes=-1 pip install -e .[dev] nosetests -v certbot --processes=-1 --process-timeout=100 + pip install -e certbot-apache + nosetests -v certbot_apache --processes=-1 --process-timeout=80 [testenv:py27_install] basepython = python2.7 From e5909d379c5e0144481c7c848ccf1c13b71c2a73 Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Mon, 27 Feb 2017 13:35:29 -0800 Subject: [PATCH 71/88] Don't crash on listen unix: (#4259) Fixes #4225. * don't crash on listen unix: * correctly merge #4221 --- certbot-nginx/certbot_nginx/parser.py | 7 ++++--- certbot-nginx/certbot_nginx/tests/parser_test.py | 6 ++++++ 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/certbot-nginx/certbot_nginx/parser.py b/certbot-nginx/certbot_nginx/parser.py index c586aa459..eddc7b9b0 100644 --- a/certbot-nginx/certbot_nginx/parser.py +++ b/certbot-nginx/certbot_nginx/parser.py @@ -586,9 +586,10 @@ def _parse_server_raw(server): continue if directive[0] == 'listen': addr = obj.Addr.fromstring(directive[1]) - parsed_server['addrs'].add(addr) - if addr.ssl: - parsed_server['ssl'] = True + if addr: + parsed_server['addrs'].add(addr) + if addr.ssl: + parsed_server['ssl'] = True elif directive[0] == 'server_name': parsed_server['names'].update( _get_servernames(directive[1])) diff --git a/certbot-nginx/certbot_nginx/tests/parser_test.py b/certbot-nginx/certbot_nginx/tests/parser_test.py index 921cc3c5a..6a3f2f1de 100644 --- a/certbot-nginx/certbot_nginx/tests/parser_test.py +++ b/certbot-nginx/certbot_nginx/tests/parser_test.py @@ -323,6 +323,12 @@ class NginxParserTest(util.NginxTest): ]) self.assertTrue(server['ssl']) + def test_parse_server_raw_unix(self): + server = parser._parse_server_raw([ #pylint: disable=protected-access + ['listen', 'unix:/var/run/nginx.sock'] + ]) + self.assertEqual(len(server['addrs']), 0) + def test_parse_server_global_ssl_applied(self): nparser = parser.NginxParser(self.config_path, self.ssl_options) server = nparser.parse_server([ From 80055ec770ee0e896a52b19c6420c4f2cf6a59fb Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Mon, 27 Feb 2017 15:15:19 -0800 Subject: [PATCH 72/88] Cleanup issue template (#4256) --- ISSUE_TEMPLATE.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/ISSUE_TEMPLATE.md b/ISSUE_TEMPLATE.md index e4e56f93d..e648a21b4 100644 --- a/ISSUE_TEMPLATE.md +++ b/ISSUE_TEMPLATE.md @@ -1,16 +1,14 @@ ## My operating system is (include version): -## My web server is (include version): +## I installed Certbot with (certbot-auto, OS package manager, pip, etc): -## How did you install Certbot: +## I ran this command and it produced this output: -## What command did you run and what output did it produce? - - -## Can you provide a Certbot error log showing the issue? -###### It is stored by default in `/var/log/letsencrypt` - feel free to redact domain names, e-mail and IP addresses as you see fit +## Certbot's behavior differed from what I expected because: +## Here is a Certbot log showing the issue (if available): +###### Logs are stored in `/var/log/letsencrypt` by default. Feel free to redact domains, e-mail and IP addresses as you see fit. From 402ad8b35311460babb7195095b10d02a0b14e48 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Mon, 27 Feb 2017 17:17:08 -0800 Subject: [PATCH 73/88] bump requests requirement to >=2.10 (#4248) --- acme/setup.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/acme/setup.py b/acme/setup.py index d1e91e5ec..9b43278af 100644 --- a/acme/setup.py +++ b/acme/setup.py @@ -15,7 +15,11 @@ install_requires = [ 'PyOpenSSL>=0.13', 'pyrfc3339', 'pytz', - 'requests[security]>=2.4.1', # security extras added in 2.4.1 + # requests>=2.10 is required to fix + # https://github.com/shazow/urllib3/issues/556. This requirement can be + # relaxed to 'requests[security]>=2.4.1', however, less useful errors + # will be raised for some network/SSL errors. + 'requests[security]>=2.10', # For pkg_resources. >=1.0 so pip resolves it to a version cryptography # will tolerate; see #2599: 'setuptools>=1.0', From 44a6ec29c58e6bcceb05543ea8c5503fc8c5b772 Mon Sep 17 00:00:00 2001 From: Damien Tournoud Date: Tue, 28 Feb 2017 03:13:06 +0100 Subject: [PATCH 74/88] Fix direct usages of the root logger (#4236) Some code uses `logging.debug` and `logging.info` instead of the file-specific logger in `logger.debug` and `logger.info`. --- acme/acme/challenges.py | 4 ++-- acme/acme/client.py | 6 +++--- acme/acme/client_test.py | 2 +- certbot-nginx/certbot_nginx/configurator.py | 2 +- certbot/ocsp.py | 6 +++--- certbot/tests/ocsp_test.py | 2 +- 6 files changed, 11 insertions(+), 11 deletions(-) diff --git a/acme/acme/challenges.py b/acme/acme/challenges.py index 83b9b9edd..3b1e90166 100644 --- a/acme/acme/challenges.py +++ b/acme/acme/challenges.py @@ -425,7 +425,7 @@ class TLSSNI01Response(KeyAuthorizationChallengeResponse): # TODO: domain is not necessary if host is provided if "host" not in kwargs: host = socket.gethostbyname(domain) - logging.debug('%s resolved to %s', domain, host) + logger.debug('%s resolved to %s', domain, host) kwargs["host"] = host kwargs.setdefault("port", self.PORT) @@ -445,7 +445,7 @@ class TLSSNI01Response(KeyAuthorizationChallengeResponse): """ # pylint: disable=protected-access sans = crypto_util._pyopenssl_cert_or_req_san(cert) - logging.debug('Certificate %s. SANs: %s', cert.digest('sha1'), sans) + logger.debug('Certificate %s. SANs: %s', cert.digest('sha1'), sans) return self.z_domain.decode() in sans def simple_verify(self, chall, domain, account_public_key, diff --git a/acme/acme/client.py b/acme/acme/client.py index 168574d58..23eabe4b9 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -600,10 +600,10 @@ class ClientNetwork(object): # pylint: disable=too-many-instance-attributes """ if method == "POST": - logging.debug('Sending POST request to %s:\n%s', + logger.debug('Sending POST request to %s:\n%s', url, kwargs['data']) else: - logging.debug('Sending %s request to %s.', method, url) + logger.debug('Sending %s request to %s.', method, url) kwargs['verify'] = self.verify_ssl kwargs.setdefault('headers', {}) kwargs['headers'].setdefault('User-Agent', self.user_agent) @@ -651,7 +651,7 @@ class ClientNetwork(object): # pylint: disable=too-many-instance-attributes def _get_nonce(self, url): if not self._nonces: - logging.debug('Requesting fresh nonce') + logger.debug('Requesting fresh nonce') self._add_nonce(self.head(url)) return self._nonces.pop() diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index b7bd0740c..3621a0824 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -544,7 +544,7 @@ class ClientNetworkTest(unittest.TestCase): # pylint: disable=protected-access self.net._send_request('HEAD', 'http://example.com/', 'foo', timeout=mock.ANY, bar='baz') - mock_logger.debug.assert_called_once_with( + mock_logger.debug.assert_called_with( 'Received response:\nHTTP %d\n%s\n\n%s', 200, 'Content-Type: application/pkix-cert', b'aGk=') diff --git a/certbot-nginx/certbot_nginx/configurator.py b/certbot-nginx/certbot_nginx/configurator.py index 6d51ca641..7348def2f 100644 --- a/certbot-nginx/certbot_nginx/configurator.py +++ b/certbot-nginx/certbot_nginx/configurator.py @@ -630,7 +630,7 @@ class NginxConfigurator(common.Plugin): stderr=subprocess.PIPE) text = proc.communicate()[1] # nginx prints output to stderr except (OSError, ValueError) as error: - logging.debug(error, exc_info=True) + logger.debug(error, exc_info=True) raise errors.PluginError( "Unable to run %s -V" % self.conf('ctl')) diff --git a/certbot/ocsp.py b/certbot/ocsp.py index 8921dbb88..d34110f88 100644 --- a/certbot/ocsp.py +++ b/certbot/ocsp.py @@ -16,7 +16,7 @@ class RevocationChecker(object): self.broken = False if not util.exe_exists("openssl"): - logging.info("openssl not installed, can't check revocation") + logger.info("openssl not installed, can't check revocation") self.broken = True return @@ -61,7 +61,7 @@ class RevocationChecker(object): logger.debug("Querying OCSP for %s", cert_path) logger.debug(" ".join(cmd)) try: - output, err = util.run_script(cmd, log=logging.debug) + output, err = util.run_script(cmd, log=logger.debug) except errors.SubprocessError: logger.info("OCSP check failed for %s (are we offline?)", cert_path) return False @@ -80,7 +80,7 @@ class RevocationChecker(object): try: url, _err = util.run_script( ["openssl", "x509", "-in", cert_path, "-noout", "-ocsp_uri"], - log=logging.debug) + log=logger.debug) except errors.SubprocessError: logger.info("Cannot extract OCSP URI from %s", cert_path) return None, None diff --git a/certbot/tests/ocsp_test.py b/certbot/tests/ocsp_test.py index 549e83ca8..91dd6f8d6 100644 --- a/certbot/tests/ocsp_test.py +++ b/certbot/tests/ocsp_test.py @@ -28,7 +28,7 @@ class OCSPTest(unittest.TestCase): def tearDown(self): pass - @mock.patch('certbot.ocsp.logging.info') + @mock.patch('certbot.ocsp.logger.info') @mock.patch('certbot.ocsp.Popen') @mock.patch('certbot.util.exe_exists') def test_init(self, mock_exists, mock_popen, mock_log): From 0d8a4b4ebdf12d6664550cd52a40c9f70e864c03 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Tue, 28 Feb 2017 15:17:07 -0800 Subject: [PATCH 75/88] Make mod-check more flexible (#4268) * fixes #4166 * Run mod-check from anywhere * pass TRAVIS_BRANCH through in tox --- tests/modification-check.sh | 61 ++++++++++++++++--------------------- tox.ini | 4 ++- 2 files changed, 30 insertions(+), 35 deletions(-) diff --git a/tests/modification-check.sh b/tests/modification-check.sh index 5168f5ab5..6f412ba47 100755 --- a/tests/modification-check.sh +++ b/tests/modification-check.sh @@ -1,46 +1,39 @@ #!/bin/bash temp_dir=`mktemp -d` +trap "rm -rf $temp_dir" EXIT -# Script should be run from Certbot's root directory - -SCRIPT_PATH=`dirname $0` -SCRIPT_PATH=`readlink -f $SCRIPT_PATH` +# cd to repo root +cd $(dirname $(dirname $(readlink -f $0))) FLAG=false -# Compare root letsencrypt-auto and certbot-auto with published versions - -cp letsencrypt-auto ${temp_dir}/letsencrypt-to-be-checked -cp certbot-auto ${temp_dir}/certbot-to-be-checked - -cp letsencrypt-auto-source/pieces/fetch.py ${temp_dir}/fetch.py -cd ${temp_dir} - -LATEST_VERSION=`python fetch.py --latest-version` -python fetch.py --le-auto-script v${LATEST_VERSION} - -cmp -s letsencrypt-auto letsencrypt-to-be-checked - -if [ $? != 0 ]; then - echo "Root letsencrypt-auto has changed." - FLAG=true +if ! cmp -s certbot-auto letsencrypt-auto; then + echo "Root certbot-auto and letsencrypt-auto differ." + FLAG=true else - echo "Root letsencrypt-auto is unchanged." + cp certbot-auto "$temp_dir/local-auto" + cp letsencrypt-auto-source/pieces/fetch.py "$temp_dir/fetch.py" + cd $temp_dir + + # Compare file against current version in the target branch + BRANCH=${TRAVIS_BRANCH:-master} + URL="https://raw.githubusercontent.com/certbot/certbot/$BRANCH/certbot-auto" + curl -sS $URL > certbot-auto + if cmp -s certbot-auto local-auto; then + echo "Root *-auto were unchanged." + else + # Compare file against the latest released version + python fetch.py --le-auto-script "v$(python fetch.py --latest-version)" + if cmp -s letsencrypt-auto local-auto; then + echo "Root *-auto were updated to the latest version." + else + echo "Root *-auto have unexpected changes." + FLAG=true + fi + fi + cd ~- fi -cmp -s letsencrypt-auto certbot-to-be-checked - -if [ $? != 0 ]; then - echo "Root certbot-auto has changed." - FLAG=true -else - echo "Root certbot-auto is unchanged." -fi - -# Cleanup -rm ${temp_dir}/* -cd ${SCRIPT_PATH}/../ - # Compare letsencrypt-auto-source/letsencrypt-auto with output of build.py cp letsencrypt-auto-source/letsencrypt-auto ${temp_dir}/original-lea diff --git a/tox.ini b/tox.ini index 232010d40..ea1423415 100644 --- a/tox.ini +++ b/tox.ini @@ -151,7 +151,9 @@ commands = docker run --rm -t -i lea whitelist_externals = docker -passenv = DOCKER_* +passenv = + DOCKER_* + TRAVIS_BRANCH [testenv:le_auto_wheezy] # At the moment, this tests under Python 2.7 only, as only that version is From 11ec1eb91148f6420e10d2296b8a507c90337792 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Thu, 2 Mar 2017 10:31:15 -0800 Subject: [PATCH 76/88] Revert "Remove Link rel=next for authzs and new-certs." (#4277) --- acme/acme/client.py | 40 ++++++++++++++++++++++-------- acme/acme/client_test.py | 40 ++++++++++++++++++++++++------ acme/acme/jose/json_util.py | 2 +- acme/acme/messages.py | 6 ++++- acme/acme/messages_test.py | 6 ++++- acme/examples/example_client.py | 3 ++- certbot/auth_handler.py | 3 ++- certbot/tests/account_test.py | 2 +- certbot/tests/acme_util.py | 1 + certbot/tests/auth_handler_test.py | 3 ++- certbot/tests/display/ops_test.py | 4 +-- 11 files changed, 84 insertions(+), 26 deletions(-) diff --git a/acme/acme/client.py b/acme/acme/client.py index 23eabe4b9..ddcba7635 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -71,13 +71,20 @@ class Client(object): # pylint: disable=too-many-instance-attributes self.directory = directory @classmethod - def _regr_from_response(cls, response, uri=None, terms_of_service=None): + def _regr_from_response(cls, response, uri=None, new_authzr_uri=None, + terms_of_service=None): if 'terms-of-service' in response.links: terms_of_service = response.links['terms-of-service']['url'] + if 'next' in response.links: + new_authzr_uri = response.links['next']['url'] + + if new_authzr_uri is None: + raise errors.ClientError('"next" link missing') return messages.RegistrationResource( body=messages.Registration.from_json(response.json()), uri=response.headers.get('Location', uri), + new_authzr_uri=new_authzr_uri, terms_of_service=terms_of_service) def register(self, new_reg=None): @@ -110,7 +117,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes # (c.f. acme-spec #94) return self._regr_from_response( - response, uri=regr.uri, + response, uri=regr.uri, new_authzr_uri=regr.new_authzr_uri, terms_of_service=regr.terms_of_service) def update_registration(self, regr, update=None): @@ -167,30 +174,43 @@ class Client(object): # pylint: disable=too-many-instance-attributes return self.update_registration( regr.update(body=regr.body.update(agreement=regr.terms_of_service))) - def _authzr_from_response(self, response, identifier, uri=None): + def _authzr_from_response(self, response, identifier, + uri=None, new_cert_uri=None): + # pylint: disable=no-self-use + if new_cert_uri is None: + try: + new_cert_uri = response.links['next']['url'] + except KeyError: + raise errors.ClientError('"next" link missing') + authzr = messages.AuthorizationResource( body=messages.Authorization.from_json(response.json()), - uri=response.headers.get('Location', uri)) + uri=response.headers.get('Location', uri), + new_cert_uri=new_cert_uri) if authzr.body.identifier != identifier: raise errors.UnexpectedUpdate(authzr) return authzr - def request_challenges(self, identifier): + def request_challenges(self, identifier, new_authzr_uri=None): """Request challenges. :param .messages.Identifier identifier: Identifier to be challenged. + :param str new_authzr_uri: ``new-authorization`` URI. If omitted, + will default to value found in ``directory``. :returns: Authorization Resource. :rtype: `.AuthorizationResource` """ new_authz = messages.NewAuthorization(identifier=identifier) - response = self.net.post(self.directory.new_authz, new_authz) + response = self.net.post(self.directory.new_authz + if new_authzr_uri is None else new_authzr_uri, + new_authz) # TODO: handle errors assert response.status_code == http_client.CREATED return self._authzr_from_response(response, identifier) - def request_domain_challenges(self, domain): + def request_domain_challenges(self, domain, new_authzr_uri=None): """Request challenges for domain names. This is simply a convenience function that wraps around @@ -205,7 +225,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes """ return self.request_challenges(messages.Identifier( - typ=messages.IDENTIFIER_FQDN, value=domain)) + typ=messages.IDENTIFIER_FQDN, value=domain), new_authzr_uri) def answer_challenge(self, challb, response): """Answer challenge. @@ -280,7 +300,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes """ response = self.net.get(authzr.uri) updated_authzr = self._authzr_from_response( - response, authzr.body.identifier, authzr.uri) + response, authzr.body.identifier, authzr.uri, authzr.new_cert_uri) # TODO: check and raise UnexpectedUpdate return updated_authzr, response @@ -304,7 +324,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes content_type = DER_CONTENT_TYPE # TODO: add 'cert_type 'argument response = self.net.post( - self.directory.new_cert, + authzrs[0].new_cert_uri, # TODO: acme-spec #90 req, content_type=content_type, headers={'Accept': content_type}) diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index 3621a0824..b3db21ac9 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -40,8 +40,6 @@ class ClientTest(unittest.TestCase): 'https://www.letsencrypt-demo.org/acme/revoke-cert', messages.NewAuthorization: 'https://www.letsencrypt-demo.org/acme/new-authz', - messages.CertificateRequest: - 'https://www.letsencrypt-demo.org/acme/new-cert', }) from acme.client import Client @@ -58,6 +56,7 @@ class ClientTest(unittest.TestCase): self.new_reg = messages.NewRegistration(**dict(reg)) self.regr = messages.RegistrationResource( body=reg, uri='https://www.letsencrypt-demo.org/acme/reg/1', + new_authzr_uri='https://www.letsencrypt-demo.org/acme/new-reg', terms_of_service='https://www.letsencrypt-demo.org/tos') # Authorization @@ -73,7 +72,8 @@ class ClientTest(unittest.TestCase): typ=messages.IDENTIFIER_FQDN, value='example.com'), challenges=(challb,), combinations=None) self.authzr = messages.AuthorizationResource( - body=self.authz, uri=authzr_uri) + body=self.authz, uri=authzr_uri, + new_cert_uri='https://www.letsencrypt-demo.org/acme/new-cert') # Request issuance self.certr = messages.CertificateResource( @@ -98,12 +98,18 @@ class ClientTest(unittest.TestCase): self.response.json.return_value = self.regr.body.to_json() self.response.headers['Location'] = self.regr.uri self.response.links.update({ + 'next': {'url': self.regr.new_authzr_uri}, 'terms-of-service': {'url': self.regr.terms_of_service}, }) self.assertEqual(self.regr, self.client.register(self.new_reg)) # TODO: test POST call arguments + def test_register_missing_next(self): + self.response.status_code = http_client.CREATED + self.assertRaises( + errors.ClientError, self.client.register, self.new_reg) + def test_update_registration(self): # "Instance of 'Field' has no to_json/update member" bug: # pylint: disable=no-member @@ -136,6 +142,13 @@ class ClientTest(unittest.TestCase): self.response.json.return_value = self.regr.body.to_json() self.assertEqual(self.regr, self.client.query_registration(self.regr)) + def test_query_registration_updates_new_authzr_uri(self): + self.response.json.return_value = self.regr.body.to_json() + self.response.links = {'next': {'url': 'UPDATED'}} + self.assertEqual( + 'UPDATED', + self.client.query_registration(self.regr).new_authzr_uri) + def test_agree_to_tos(self): self.client.update_registration = mock.Mock() self.client.agree_to_tos(self.regr) @@ -146,6 +159,9 @@ class ClientTest(unittest.TestCase): self.response.status_code = http_client.CREATED self.response.headers['Location'] = self.authzr.uri self.response.json.return_value = self.authz.to_json() + self.response.links = { + 'next': {'url': self.authzr.new_cert_uri}, + } def test_request_challenges(self): self._prepare_response_for_request_challenges() @@ -156,9 +172,8 @@ class ClientTest(unittest.TestCase): def test_request_challenges_custom_uri(self): self._prepare_response_for_request_challenges() - self.client.request_challenges(self.identifier) - self.net.post.assert_called_once_with( - 'https://www.letsencrypt-demo.org/acme/new-authz', mock.ANY) + self.client.request_challenges(self.identifier, 'URI') + self.net.post.assert_called_once_with('URI', mock.ANY) def test_request_challenges_unexpected_update(self): self._prepare_response_for_request_challenges() @@ -166,7 +181,12 @@ class ClientTest(unittest.TestCase): identifier=self.identifier.update(value='foo')).to_json() self.assertRaises( errors.UnexpectedUpdate, self.client.request_challenges, - self.identifier) + self.identifier, self.authzr.uri) + + def test_request_challenges_missing_next(self): + self.response.status_code = http_client.CREATED + self.assertRaises(errors.ClientError, self.client.request_challenges, + self.identifier) def test_request_domain_challenges(self): self.client.request_challenges = mock.MagicMock() @@ -174,6 +194,12 @@ class ClientTest(unittest.TestCase): self.client.request_challenges(self.identifier), self.client.request_domain_challenges('example.com')) + def test_request_domain_challenges_custom_uri(self): + self.client.request_challenges = mock.MagicMock() + self.assertEqual( + self.client.request_challenges(self.identifier, 'URI'), + self.client.request_domain_challenges('example.com', 'URI')) + def test_answer_challenge(self): self.response.links['up'] = {'url': self.challr.authzr_uri} self.response.json.return_value = self.challr.body.to_json() diff --git a/acme/acme/jose/json_util.py b/acme/acme/jose/json_util.py index 4baadda5e..d474f4aac 100644 --- a/acme/acme/jose/json_util.py +++ b/acme/acme/jose/json_util.py @@ -267,7 +267,7 @@ class JSONObjectWithFields(util.ImmutableMap, interfaces.JSONDeSerializable): if missing: raise errors.DeserializationError( - 'The following fields are required: {0}'.format( + 'The following field are required: {0}'.format( ','.join(missing))) @classmethod diff --git a/acme/acme/messages.py b/acme/acme/messages.py index c3df4998c..54cd25c94 100644 --- a/acme/acme/messages.py +++ b/acme/acme/messages.py @@ -191,7 +191,7 @@ class Directory(jose.JSONDeSerializable): try: return self[name.replace('_', '-')] except KeyError as error: - raise AttributeError(str(error) + ': ' + name) + raise AttributeError(str(error)) def __getitem__(self, name): try: @@ -315,10 +315,12 @@ class RegistrationResource(ResourceWithURI): """Registration Resource. :ivar acme.messages.Registration body: + :ivar unicode new_authzr_uri: URI found in the 'next' ``Link`` header :ivar unicode terms_of_service: URL for the CA TOS. """ body = jose.Field('body', decoder=Registration.from_json) + new_authzr_uri = jose.Field('new_authzr_uri') terms_of_service = jose.Field('terms_of_service', omitempty=True) @@ -423,9 +425,11 @@ class AuthorizationResource(ResourceWithURI): """Authorization Resource. :ivar acme.messages.Authorization body: + :ivar unicode new_cert_uri: URI found in the 'next' ``Link`` header """ body = jose.Field('body', decoder=Authorization.from_json) + new_cert_uri = jose.Field('new_cert_uri') @Directory.register diff --git a/acme/acme/messages_test.py b/acme/acme/messages_test.py index e84c3e992..b3454f25b 100644 --- a/acme/acme/messages_test.py +++ b/acme/acme/messages_test.py @@ -225,12 +225,14 @@ class RegistrationResourceTest(unittest.TestCase): from acme.messages import RegistrationResource self.regr = RegistrationResource( body=mock.sentinel.body, uri=mock.sentinel.uri, + new_authzr_uri=mock.sentinel.new_authzr_uri, terms_of_service=mock.sentinel.terms_of_service) def test_to_partial_json(self): self.assertEqual(self.regr.to_json(), { 'body': mock.sentinel.body, 'uri': mock.sentinel.uri, + 'new_authzr_uri': mock.sentinel.new_authzr_uri, 'terms_of_service': mock.sentinel.terms_of_service, }) @@ -344,7 +346,9 @@ class AuthorizationResourceTest(unittest.TestCase): from acme.messages import AuthorizationResource authzr = AuthorizationResource( uri=mock.sentinel.uri, - body=mock.sentinel.body) + body=mock.sentinel.body, + new_cert_uri=mock.sentinel.new_cert_uri, + ) self.assertTrue(isinstance(authzr, jose.JSONDeSerializable)) diff --git a/acme/examples/example_client.py b/acme/examples/example_client.py index 1386491b1..261b37603 100644 --- a/acme/examples/example_client.py +++ b/acme/examples/example_client.py @@ -32,7 +32,8 @@ acme.agree_to_tos(regr) logging.debug(regr) authzr = acme.request_challenges( - identifier=messages.Identifier(typ=messages.IDENTIFIER_FQDN, value=DOMAIN)) + identifier=messages.Identifier(typ=messages.IDENTIFIER_FQDN, value=DOMAIN), + new_authzr_uri=regr.new_authzr_uri) logging.debug(authzr) authzr, authzr_response = acme.poll(authzr) diff --git a/certbot/auth_handler.py b/certbot/auth_handler.py index 53346a77c..6e9ab25a7 100644 --- a/certbot/auth_handler.py +++ b/certbot/auth_handler.py @@ -63,7 +63,8 @@ class AuthHandler(object): """ for domain in domains: - self.authzr[domain] = self.acme.request_domain_challenges(domain) + self.authzr[domain] = self.acme.request_domain_challenges( + domain, self.account.regr.new_authzr_uri) self._choose_challenges(domains) diff --git a/certbot/tests/account_test.py b/certbot/tests/account_test.py index 7367717bf..8ed591c98 100644 --- a/certbot/tests/account_test.py +++ b/certbot/tests/account_test.py @@ -110,7 +110,7 @@ class AccountFileStorageTest(unittest.TestCase): from certbot.account import Account self.acc = Account( regr=messages.RegistrationResource( - uri=None, body=messages.Registration()), + uri=None, new_authzr_uri=None, body=messages.Registration()), key=KEY) def tearDown(self): diff --git a/certbot/tests/acme_util.py b/certbot/tests/acme_util.py index f0549666a..5e6b190a7 100644 --- a/certbot/tests/acme_util.py +++ b/certbot/tests/acme_util.py @@ -96,5 +96,6 @@ def gen_authzr(authz_status, domain, challs, statuses, combos=True): # pylint: disable=star-args return messages.AuthorizationResource( uri="https://trusted.ca/new-authz-resource", + new_cert_uri="https://trusted.ca/new-cert", body=messages.Authorization(**authz_kwargs) ) diff --git a/certbot/tests/auth_handler_test.py b/certbot/tests/auth_handler_test.py index 9d22843db..046eb5ef1 100644 --- a/certbot/tests/auth_handler_test.py +++ b/certbot/tests/auth_handler_test.py @@ -309,6 +309,7 @@ class PollChallengesTest(unittest.TestCase): new_authzr = messages.AuthorizationResource( uri=authzr.uri, + new_cert_uri=authzr.new_cert_uri, body=messages.Authorization( identifier=authzr.body.identifier, challenges=new_challbs, @@ -436,7 +437,7 @@ def gen_auth_resp(chall_list): for chall in chall_list] -def gen_dom_authzr(domain, challs, combos=True): +def gen_dom_authzr(domain, unused_new_authzr_uri, challs, combos=True): """Generates new authzr for domains.""" return acme_util.gen_authzr( messages.STATUS_PENDING, domain, challs, diff --git a/certbot/tests/display/ops_test.py b/certbot/tests/display/ops_test.py index f2a9b3d07..f6de33a92 100644 --- a/certbot/tests/display/ops_test.py +++ b/certbot/tests/display/ops_test.py @@ -104,10 +104,10 @@ class ChooseAccountTest(unittest.TestCase): self.key = KEY self.acc1 = account.Account(messages.RegistrationResource( - uri=None, body=messages.Registration.from_data( + uri=None, new_authzr_uri=None, body=messages.Registration.from_data( email="email1@g.com")), self.key) self.acc2 = account.Account(messages.RegistrationResource( - uri=None, body=messages.Registration.from_data( + uri=None, new_authzr_uri=None, body=messages.Registration.from_data( email="email2@g.com", phone="phone")), self.key) @classmethod From 5e6a6f51d3627d2ec297d5e1bdb4c2ddfb113654 Mon Sep 17 00:00:00 2001 From: Brad Warren Date: Thu, 2 Mar 2017 10:31:37 -0800 Subject: [PATCH 77/88] Fix test_leauto_upgrades.sh (#4278) * fix-test-leauto-upgrades * redirect stderr * redirect stderr part 2 --- .../letstest/scripts/test_leauto_upgrades.sh | 22 ++++++++----------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/tests/letstest/scripts/test_leauto_upgrades.sh b/tests/letstest/scripts/test_leauto_upgrades.sh index f7a74821b..b46080eff 100755 --- a/tests/letstest/scripts/test_leauto_upgrades.sh +++ b/tests/letstest/scripts/test_leauto_upgrades.sh @@ -4,13 +4,6 @@ # are dynamically set at execution cd letsencrypt -#git checkout v0.1.0 use --branch instead -SAVE="$PIP_EXTRA_INDEX_URL" -unset PIP_EXTRA_INDEX_URL -export PIP_INDEX_URL="https://isnot.org/pip/0.1.0/" - -#OLD_LEAUTO="https://raw.githubusercontent.com/letsencrypt/letsencrypt/5747ab7fd9641986833bad474d71b46a8c589247/letsencrypt-auto" - if ! command -v git ; then if [ "$OS_TYPE" = "ubuntu" ] ; then @@ -22,15 +15,18 @@ if ! command -v git ; then fi fi BRANCH=`git rev-parse --abbrev-ref HEAD` -git checkout -f v0.1.0 -./letsencrypt-auto -v --debug --version -unset PIP_INDEX_URL - -export PIP_EXTRA_INDEX_URL="$SAVE" +# 0.4.1 is the oldest version of letsencrypt-auto that can be used because +# it's the first version that both pins package versions and properly supports +# --no-self-upgrade. +git checkout -f v0.4.1 +if ! ./letsencrypt-auto -v --debug --version --no-self-upgrade 2>&1 | grep 0.4.1 ; then + echo initial installation appeared to fail + exit 1 +fi git checkout -f "$BRANCH" EXPECTED_VERSION=$(grep -m1 LE_AUTO_VERSION letsencrypt-auto | cut -d\" -f2) -if ! ./letsencrypt-auto -v --debug --version --no-self-upgrade | grep $EXPECTED_VERSION ; then +if ! ./letsencrypt-auto -v --debug --version --no-self-upgrade 2>&1 | grep $EXPECTED_VERSION ; then echo upgrade appeared to fail exit 1 fi From b040717e4d6841342991067bc722c5cb5b990656 Mon Sep 17 00:00:00 2001 From: Noah Swartz Date: Thu, 2 Mar 2017 10:31:55 -0800 Subject: [PATCH 78/88] Changelog (#4252) * made a changelog * fix date for 0.6.0 * fix brad nits * fix typo --- CHANGELOG.md | 326 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 326 insertions(+) create mode 100644 CHANGELOG.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 000000000..4cf02e954 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,326 @@ +# 0.11.1 +## 02/01/2017 + +* Resolve a problem where Certbot would crash while parsing command line +arguments in some cases. +* Fix a typo. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/pulls?q=is%3Apr%20milestone%3A0.11.1%20is%3Aclosed + +# 0.11.0 +## 02/01/2017 + +* Providing `--quiet` to `certbot-auto` now silences package manager output. +* The UI has been improved in the standalone plugin. When using the +plugin while running Certbot interactively and a required port is bound +by another process, Certbot will give you the option to retry to grab +the port rather than immediately exiting. +* You are now able to deactivate your account with the Let's Encrypt +server using the `unregister` subcommand. +* When revoking a certificate using the `revoke` subcommand, you now +have the option to provide the reason the certificate is being revoked +to Let's Encrypt with `--reason`. +* Removal of the optional `dnspython` dependency in our `acme` package. +Now the library does not support client side verification of the DNS +challenge. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.11.0+is%3Aclosed + +# 0.10.2 +## 01/25/2017 + +* We now save `--preferred-challenges` values for renewal. Previously +these values were discarded causing a different challenge type to be +used when renewing certs in some cases. +* If Certbot receives a request with a `badNonce` error, we +automatically retry the request. Since nonces from Let's Encrypt expire, +this helps people performing the DNS challenge with the `manual` plugin +who may have to wait an extended period of time for their DNS changes to +propagate. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.2+is%3Aclosed + +# 0.10.1 +## 01/13/2017 + +* Resolve problems where when asking Certbot to update a certificate at +an existing path to include different domain names, the old names would +continue to be used. +* Fix issues successfully running our unit test suite on some systems. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.1+is%3Aclosed + +# 0.10.0 +## 01/11/2017 + +* The ability to customize and automatically complete DNS and HTTP +domain validation challenges with the manual plugin. The flags +`--manual-auth-hook` and `--manual-cleanup-hook` can now be provided +when using the manual plugin to execute commands provided by the user to +perform and clean up challenges provided by the CA. This is best used in +complicated setups where the DNS challenge must be used or Certbot's +existing plugins cannot be used to perform HTTP challenges. For more +information on how this works, see `certbot --help manual`. +* A `--cert-name` flag for specifying the name to use for the +certificate in Certbot's configuration directory. Using this flag in +combination with `-d/--domains`, a user can easily request a new +certificate with different domains and save it with the name provided by +`--cert-name`. Additionally, `--cert-name` can be used to select a +certificate with the `certonly` and `run` subcommands so a full list of +domains in the certificate does not have to be provided. +* The subcommand `certificates` for listing the certificates managed by +Certbot and their properties. +* A `delete` subcommand for removing certificates managed by Certbot +from the configuration directory. +* Support for requesting internationalized domain names (IDNs). +* Removal of the ncurses interface. This change solves problems people +were having on many systems, reduces the number of Certbot dependencies, +and simplifies our code. Certbot's only interface now is the text +interface which was available by providing `-t/--text` to earlier +versions of Certbot. +* Hooks provided to Certbot are now saved to be reused during renewal. +If you run Certbot with `--pre-hook`, `--renew-hook`, or `--post-hook` +flags when obtaining a certificate, the provided commands will +automatically be saved and executed again when renewing the certificate. +A pre-hook and/or post-hook can also be given to the `certbot renew` +command either on the command line or in a [configuration +file](https://certbot.eff.org/docs/using.html#configuration-file) to run +an additional command before/after any certificate is renewed. Hooks +will only be run if a certificate is renewed. +* Recategorized `-h/--help` output to improve documentation and +discoverability. +* Busybox support in certbot-auto. +* Many small bug fixes. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.0is%3Aclosed + +# 0.9.3 +## 10/13/2016 + +* Adopt more conservative behavior about reporting a needed port as +unavailable when using the standalone plugin. +* The Apache plugin uses information about your OS to help determine the +layout of your Apache configuration directory. We added a patch to +ensure this code behaves the same way when testing on different systems +as the tests were failing in some cases. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/milestone/27?closed=1 + +# 0.9.2 +## 10/12/2016 + +* Ensuring we properly copy `ssl on;` directives as necessary when +performing domain validation in the Nginx plugin. +* Verifying that our optional dependencies version matches what is +required by Certbot. +* A fix for problems where symlinks were becoming files when they were +packaged, causing errors during testing and OS packaging. +* Stop requiring that all possibly required ports are available when +using the standalone plugin. Only verify the ports are available when +you know they are necessary. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/milestone/26?closed=1 + +# 0.9.1 +## 10/06/2016 + +* This version of Certbot simply fixes a bug that was introduced in version +0.9.0 where the command line flag -q/--quiet wasn't respected in some cases. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/milestone/25?closed=1 + +# 0.9.0 +## 10/05/2016 + +* An alpha version of the Nginx plugin. This plugin fully automates the +process of obtaining and installing certificates with Nginx. +Additionally, it is able to automatically configure security +enhancements such as an HTTP to HTTPS redirect and OCSP stapling. To use +this plugin, you must have the `certbot-nginx` package installed (which +is installed automatically when using `certbot-auto`) and provide +`--nginx` on the command line. This plugin is still in its early stages +so we recommend you use it with some caution and make sure you have a +backup of your Nginx configuration. +* Support for the `DNS` challenge in the `acme` library as well as `DNS` +support in Certbot's `manual` plugin. This allows you to create DNS +records to prove to Let's Encrypt you control the requested the domain +name. To use this feature, include `--manual --preferred-challenges dns` +on the command line. +* Help with enabling Extra Packages for Enterprise Linux (EPEL) on +CentOS 6 when using `certbot-auto`. To use `certbot-auto` on CentOS 6, +the EPEL repository has to be enabled. `certbot-auto` will now prompt +users asking them if they would like the script to enable this for them +automatically. This is done without prompting users when using +`letsencrypt-auto` or if `-n/--non-interactive/--noninteractive` is +included on the command line. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.9.0+is%3Aclosed + +# 0.8.1 +## 06/14/2016 + +* Preserving a certificate's common name when using `renew` +* Save webroot values for renewal when they are entered interactively +* Problems with an invalid user-agent string on OS X +* Gracefully reporting the Apache plugin isn't usable when Augeas is not installed +* Experimental support for Mageia has been added to `certbot-auto` + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.8.1+ + +# 0.8.0 +## 06/02/2016 + +* The main new feature in this release is the `register` subcommand which +can be used to register an account with the Let's Encrypt CA. +* Additionally, you can run `certbot register --update-registration` to +change the e-mail address associated with your registration. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.8.0+ + +# 0.7.0 +## 05/27/2016 + +* `--must-staple` to request certificates from Let's Encrypt with the +OCSP must staple extension +* automatic configuration of OSCP stapling for Apache +* requesting certificates for domains found in the common name of a +custom CSR +* a number of bug fixes + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=milestone%3A0.7.0+is%3Aissue + +# 0.6.0 +## 05/12/2016 + +* Renamed the client from `letsencrypt` to `certbot` +* Fixed a small json deserialization error +* Versioned the datetime dependency in setup.py +* Preserve domain order in generated CSRs +* Some minor bug fixes + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue%20milestone%3A0.6.0%20is%3Aclosed%20 + +# 0.5.0 +## 04/05/2016 + +* The ability to use the webroot plugin interactively. +* The flags --pre-hook, --post-hook, and --renew-hook which can be used +with the renew subcommand to register shell commands to run in +response to renewal events. Pre-hook commands will be run before +any certs are renewed, post-hook commands will be run after any +certs are renewed, and renew-hook commands will be run after each +cert is renewed. If no certs are due for renewal, no command is run. +* Cleaner renewal configuration files. In /etc/letsencrypt/renewal by +default, these files can be used to control what parameters are used +when renewing a specific certificate. +* A -q/--quiet flag which silences all output except errors. +* An --allow-subset-of-domains flag which can be used with the renew +command to prevent renewal failures for a subset of the requested +domains from causing the client to exit. + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=milestone%3A0.5.0+is%3Aissue + +# 0.4.2 +## 03/03/2016 + +* Resolves problems encountered when compiling letsencrypt +against the new OpenSSL release. +* A patch fixing problems of using letsencrypt renew with configuration files +from private beta has been added. + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.4.2 + +# 0.4.1 +## 02/29/2016 + +* Fixes Apache parsing errors with some configurations +* Fixes Werkzeug dependency problems on some Red Hat systems +* Fixes bootstrapping failures when using letsencrypt-auto with --no-self-upgrade +* Fixes problems with parsing renewal config files from private beta + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=is:issue+milestone:0.4.1 + +# 0.4.0 +## 02/10/2016 + +* The new verb/subcommand `renew` can be used to renew your existing +certificates as they approach expiration. Running `letsencrypt renew` +will examine all existing certificate lineages and determine if any are +less than 30 days from expiration. If so, the client will use the +settings provided when you previously obtained the certificate to renew +it. The subcommand finishes by printing a summary of which renewals were +successful, failed, or not yet due. +* A `--dry-run` flag has been added to help with testing configuration +without affecting production rate limits. Currently supported by the +`renew` and `certonly` subcommands, providing `--dry-run` on the command +line will obtain certificates from the staging server without saving the +resulting certificates to disk. +* Major improvements have been added to letsencrypt-auto. This script +has been rewritten to include full support for Python 2.6, the ability +for letsencrypt-auto to update itself, and improvements to the +stability, security, and performance of the script. +* Support for Apache 2.2 has been added to the Apache plugin. + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.4.0 + +# 0.3.0 +## 01/27/2016 + +* A non-interactive mode which can be enabled by including `-n` or +`--non-interactive` on the command line. This can be used to +guarantee the client will not prompt when run automatically using +cron/systemd. +* Preparation for the new letsencrypt-auto script. Over the past +couple months, we've been working on increasing the reliability and +security of letsencrypt-auto. A number of changes landed in this +release to prepare for the new version of this script. + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.3.0 + +# 0.2.0 +## 01/14/2016 + +* Apache plugin support for non-Debian based systems. Support has been +added for modern Red Hat based systems such as Fedora 23, Red Hat 7, +and CentOS 7 running Apache 2.4. In theory, this plugin should be +able to be configured to run on any Unix-like OS running Apache 2.4. +* Relaxed PyOpenSSL version requirements. This adds support for systems +with PyOpenSSL versions 0.13 or 0.14. +* Resolves issues with the Apache plugin enabling an HTTP to HTTPS +redirect on some systems. +* Improved error messages from the client. + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.2.0 + +# 0.1.1 +## 12/15/2015 + +* Fix a confusing UI path that caused some users to repeatedly renew +their certs while experimenting with the client, in some cases +hitting issuance rate limits +* Fixes numerous Apache configuration parser fixes +* Avoids attempting to issue for unqualified domain names like +"localhost" +* Fixes --webroot permission handling for non-root users + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=milestone%3A0.1.1 From 5e671682caac181072141747747a97b9ca2f48cb Mon Sep 17 00:00:00 2001 From: Erica Portnoy Date: Thu, 2 Mar 2017 15:26:24 -0800 Subject: [PATCH 79/88] Candidate 0.12.0 (#4286) * Release 0.12.0 * Bump version to 0.13.0 --- acme/setup.py | 2 +- certbot-apache/setup.py | 2 +- certbot-auto | 26 +++++++++--------- certbot-compatibility-test/setup.py | 2 +- certbot-nginx/setup.py | 2 +- certbot/__init__.py | 2 +- docs/cli-help.txt | 2 +- letsencrypt-auto | 26 +++++++++--------- letsencrypt-auto-source/certbot-auto.asc | 14 +++++----- letsencrypt-auto-source/letsencrypt-auto | 26 +++++++++--------- letsencrypt-auto-source/letsencrypt-auto.sig | Bin 256 -> 256 bytes .../pieces/letsencrypt-auto-requirements.txt | 24 ++++++++-------- 12 files changed, 64 insertions(+), 64 deletions(-) diff --git a/acme/setup.py b/acme/setup.py index 9b43278af..f169f59a7 100644 --- a/acme/setup.py +++ b/acme/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.13.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-apache/setup.py b/certbot-apache/setup.py index 87ea1a281..56a48abc6 100644 --- a/certbot-apache/setup.py +++ b/certbot-apache/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.13.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot-auto b/certbot-auto index ac721638f..54cc429cf 100755 --- a/certbot-auto +++ b/certbot-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.11.1" +LE_AUTO_VERSION="0.12.0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.1 \ - --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ - --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e -certbot==0.11.1 \ - --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ - --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 -certbot-apache==0.11.1 \ - --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ - --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 -certbot-nginx==0.11.1 \ - --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ - --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 +acme==0.12.0 \ + --hash=sha256:a6050619b3e07b41d197992bb15b32c755dfa0665cfa1c20faa82806a798265b \ + --hash=sha256:a05cba6b5b0fffdfa246b32492a44769011d45205f3ee8efde1f37ee9843fbdf +certbot==0.12.0 \ + --hash=sha256:d018d13665eb4cfe7038c2df636e3f4928742b83769b95edfdb0311277f0eb48 \ + --hash=sha256:4a71925c035b62dfb7c3343c619ee090add76188b47225272b57798ad63388b7 +certbot-apache==0.12.0 \ + --hash=sha256:de86907ea60e7bc35d252b87dec04eab3c7f3a1ea768774876e7ff582d89d640 \ + --hash=sha256:77dde63cf97292b09da8ae09ef8a7a6d83a3b1ee0f8d1fefe513fc77a6292509 +certbot-nginx==0.12.0 \ + --hash=sha256:c66d848c4577f1f91a06a8119b40f1ab90af1546addea27905434bd070f3924d \ + --hash=sha256:4dab2c93304c80d8d0d2e5214939f016804fd46859dd7a39b892d8b7195ab5ec UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/certbot-compatibility-test/setup.py b/certbot-compatibility-test/setup.py index e2d226a72..73d3b704b 100644 --- a/certbot-compatibility-test/setup.py +++ b/certbot-compatibility-test/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.13.0.dev0' install_requires = [ 'certbot', diff --git a/certbot-nginx/setup.py b/certbot-nginx/setup.py index 6de2dc6bd..24c2564b9 100644 --- a/certbot-nginx/setup.py +++ b/certbot-nginx/setup.py @@ -4,7 +4,7 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.13.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ diff --git a/certbot/__init__.py b/certbot/__init__.py index 6451eb0d5..0c667378d 100644 --- a/certbot/__init__.py +++ b/certbot/__init__.py @@ -1,4 +1,4 @@ """Certbot client.""" # version number like 1.2.3a0, must have at least 2 parts, like 1.2 -__version__ = '0.12.0.dev0' +__version__ = '0.13.0.dev0' diff --git a/docs/cli-help.txt b/docs/cli-help.txt index 4cb408002..9ef9d9e6c 100644 --- a/docs/cli-help.txt +++ b/docs/cli-help.txt @@ -86,7 +86,7 @@ optional arguments: statistics about success rates by OS and plugin. If you wish to hide your server OS version from the Let's Encrypt server, set this to "". (default: - CertbotACMEClient/0.11.1 (Ubuntu 16.04.1 LTS) + CertbotACMEClient/0.12.0 (Ubuntu 16.04.2 LTS) Authenticator/XXX Installer/YYY) automation: diff --git a/letsencrypt-auto b/letsencrypt-auto index ac721638f..54cc429cf 100755 --- a/letsencrypt-auto +++ b/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.11.1" +LE_AUTO_VERSION="0.12.0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.1 \ - --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ - --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e -certbot==0.11.1 \ - --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ - --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 -certbot-apache==0.11.1 \ - --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ - --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 -certbot-nginx==0.11.1 \ - --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ - --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 +acme==0.12.0 \ + --hash=sha256:a6050619b3e07b41d197992bb15b32c755dfa0665cfa1c20faa82806a798265b \ + --hash=sha256:a05cba6b5b0fffdfa246b32492a44769011d45205f3ee8efde1f37ee9843fbdf +certbot==0.12.0 \ + --hash=sha256:d018d13665eb4cfe7038c2df636e3f4928742b83769b95edfdb0311277f0eb48 \ + --hash=sha256:4a71925c035b62dfb7c3343c619ee090add76188b47225272b57798ad63388b7 +certbot-apache==0.12.0 \ + --hash=sha256:de86907ea60e7bc35d252b87dec04eab3c7f3a1ea768774876e7ff582d89d640 \ + --hash=sha256:77dde63cf97292b09da8ae09ef8a7a6d83a3b1ee0f8d1fefe513fc77a6292509 +certbot-nginx==0.12.0 \ + --hash=sha256:c66d848c4577f1f91a06a8119b40f1ab90af1546addea27905434bd070f3924d \ + --hash=sha256:4dab2c93304c80d8d0d2e5214939f016804fd46859dd7a39b892d8b7195ab5ec UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/certbot-auto.asc b/letsencrypt-auto-source/certbot-auto.asc index 44e2246b7..417d43387 100644 --- a/letsencrypt-auto-source/certbot-auto.asc +++ b/letsencrypt-auto-source/certbot-auto.asc @@ -1,11 +1,11 @@ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 -iQEcBAABAgAGBQJYkqlBAAoJEE0XyZXNl3XyXUAH/RqwKDOpceChSdH/aIk891HX -VRDBQxIjZ6EB1iyebfihyZd4a5zGJ9ocMj1GxThMyLgSKbgkSRtjE/+ymDWsL0Us -Y8w9fw76BAImaJZEvjkrpqD2bSYdijnF479hBa/huZHKcQhb/sqxkNJO9SO1uj8z -bnF0UjJNjgn1hm2yHNMWwyEX7xCN/Vxiq/Zwqi7HdPus99sInJA7+04nwXaUtash -87MHpCjIiHh3axOCOjJbAWzIfsDUKeaBHgeYO+2ldOPWVQ0Amp7ghXjohryBkiux -dqhhAuvBTmNqPrbPAjdJ7Kd74NOGDo3HvAUiuXIckDWqxX2Q34w5pwxelZcIEnI= -=vmVS +iQEcBAABAgAGBQJYuJdQAAoJEE0XyZXNl3Xyw+oH/1AQ90P3397rKB0jP+5MchtR +Nz4ScKL86x9s+o/OzAN76gLhJNj/gOVWoyeK8wVkJ07MpbGyLBiYFsXPZWYUcJ77 +LRj4sGAxJatptHG+PnzIquAf+swynqVu0QdBv8ImKwYrqOlULR+Kr8QZE95Ena51 +JPkbm5o0ipSbByIpraAYabCOHj7SrsFQtMx+tPTd7xaliO8VkguzLQt93QQC7CNj +JIO/yURnfKzutTOe3OPzBzbb6e2yhHcHZcSyv8S0DCIAoB08N9Bs8aAbVwmD89Fq +fwYxLZherXRZ2VtJ2Sf/hUP2ZrEH/mvCkKjzznZokFGJXLvTEc8fC/O6c/q/nLw= +=YiSx -----END PGP SIGNATURE----- diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index 81e3862c2..cc248a36a 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.12.0.dev0" +LE_AUTO_VERSION="0.13.0.dev0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.1 \ - --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ - --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e -certbot==0.11.1 \ - --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ - --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 -certbot-apache==0.11.1 \ - --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ - --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 -certbot-nginx==0.11.1 \ - --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ - --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 +acme==0.12.0 \ + --hash=sha256:a6050619b3e07b41d197992bb15b32c755dfa0665cfa1c20faa82806a798265b \ + --hash=sha256:a05cba6b5b0fffdfa246b32492a44769011d45205f3ee8efde1f37ee9843fbdf +certbot==0.12.0 \ + --hash=sha256:d018d13665eb4cfe7038c2df636e3f4928742b83769b95edfdb0311277f0eb48 \ + --hash=sha256:4a71925c035b62dfb7c3343c619ee090add76188b47225272b57798ad63388b7 +certbot-apache==0.12.0 \ + --hash=sha256:de86907ea60e7bc35d252b87dec04eab3c7f3a1ea768774876e7ff582d89d640 \ + --hash=sha256:77dde63cf97292b09da8ae09ef8a7a6d83a3b1ee0f8d1fefe513fc77a6292509 +certbot-nginx==0.12.0 \ + --hash=sha256:c66d848c4577f1f91a06a8119b40f1ab90af1546addea27905434bd070f3924d \ + --hash=sha256:4dab2c93304c80d8d0d2e5214939f016804fd46859dd7a39b892d8b7195ab5ec UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/letsencrypt-auto.sig b/letsencrypt-auto-source/letsencrypt-auto.sig index 7aea42dd07338009aeb411d16b1ede2e47242c4a..711300dda497b4f0de469e0d5f9fdeffcc0ed791 100644 GIT binary patch literal 256 zcmV+b0ssCPKrmxhSd~Gl5(vW|<6opm^3z*2@_s;^(O#c8A(BZ&bT<6@KKsFX>*P^+ z5OMKIYIwR>5CX?W?*GvZf6nD9_+<;knP+ME`_3x{Odk_e&tozlwSx$&=Gd}j4{T;d zN=0Y**I$U*TkAErI2%6WFOxeoL@LtX^L?L>(^-}P0vSB0ac_x}l1H9mca`Z>jI7HK ze*M3p)+_=!N|Mdxjmt%{u8Za)+;9^WpG(BR|)b z`MuJ4TFdF^IQv0a>mH6LZ6Uwm>Ao3CY`8B~Piue4_sWH{fV;aQFnC4iPVzu|)T9g< GmtWvEc7D?U literal 256 zcmV+b0ssE0LMkrq1<;I$4(Z|boMIIj?&t!1cp>;Yh?HTs)6}s^=SfcWN;bWgd`EXf zEhtU0e_$I^0tEloE_&&fmjX6Hni@Oow|k9sJt&yBk3)F3fvE|SyJe#y@%&-O3e7`~ z73zWb(bivi7Ot+2)kS>^Ui$C!RU< Date: Thu, 2 Mar 2017 15:46:16 -0800 Subject: [PATCH 80/88] update changelog (#4287) --- CHANGELOG.md | 360 ++++++++++++++++++++++++++------------------------- 1 file changed, 185 insertions(+), 175 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4cf02e954..cc1ad82ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,13 @@ +# 0.12.0 +## 03/02/2017 + +* Allow non-camelcase Apache VirtualHost names +* Allow more log messages to be silenced +* Fix a regression around using `--cert-name` when getting new certificates + +More information about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue%20milestone%3A0.12.0 + # 0.11.1 ## 02/01/2017 @@ -31,70 +41,70 @@ https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.11.0+is%3Ac # 0.10.2 ## 01/25/2017 -* We now save `--preferred-challenges` values for renewal. Previously -these values were discarded causing a different challenge type to be -used when renewing certs in some cases. -* If Certbot receives a request with a `badNonce` error, we -automatically retry the request. Since nonces from Let's Encrypt expire, -this helps people performing the DNS challenge with the `manual` plugin -who may have to wait an extended period of time for their DNS changes to -propagate. +* We now save `--preferred-challenges` values for renewal. Previously +these values were discarded causing a different challenge type to be +used when renewing certs in some cases. +* If Certbot receives a request with a `badNonce` error, we +automatically retry the request. Since nonces from Let's Encrypt expire, +this helps people performing the DNS challenge with the `manual` plugin +who may have to wait an extended period of time for their DNS changes to +propagate. -More details about these changes can be found on our GitHub repo: -https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.2+is%3Aclosed +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.2+is%3Aclosed # 0.10.1 ## 01/13/2017 -* Resolve problems where when asking Certbot to update a certificate at -an existing path to include different domain names, the old names would -continue to be used. -* Fix issues successfully running our unit test suite on some systems. +* Resolve problems where when asking Certbot to update a certificate at +an existing path to include different domain names, the old names would +continue to be used. +* Fix issues successfully running our unit test suite on some systems. More details about these changes can be found on our GitHub repo: -https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.1+is%3Aclosed +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.1+is%3Aclosed # 0.10.0 ## 01/11/2017 -* The ability to customize and automatically complete DNS and HTTP -domain validation challenges with the manual plugin. The flags -`--manual-auth-hook` and `--manual-cleanup-hook` can now be provided -when using the manual plugin to execute commands provided by the user to -perform and clean up challenges provided by the CA. This is best used in -complicated setups where the DNS challenge must be used or Certbot's -existing plugins cannot be used to perform HTTP challenges. For more -information on how this works, see `certbot --help manual`. -* A `--cert-name` flag for specifying the name to use for the -certificate in Certbot's configuration directory. Using this flag in -combination with `-d/--domains`, a user can easily request a new -certificate with different domains and save it with the name provided by -`--cert-name`. Additionally, `--cert-name` can be used to select a -certificate with the `certonly` and `run` subcommands so a full list of -domains in the certificate does not have to be provided. -* The subcommand `certificates` for listing the certificates managed by -Certbot and their properties. -* A `delete` subcommand for removing certificates managed by Certbot -from the configuration directory. -* Support for requesting internationalized domain names (IDNs). -* Removal of the ncurses interface. This change solves problems people -were having on many systems, reduces the number of Certbot dependencies, -and simplifies our code. Certbot's only interface now is the text -interface which was available by providing `-t/--text` to earlier -versions of Certbot. -* Hooks provided to Certbot are now saved to be reused during renewal. -If you run Certbot with `--pre-hook`, `--renew-hook`, or `--post-hook` -flags when obtaining a certificate, the provided commands will -automatically be saved and executed again when renewing the certificate. -A pre-hook and/or post-hook can also be given to the `certbot renew` -command either on the command line or in a [configuration -file](https://certbot.eff.org/docs/using.html#configuration-file) to run -an additional command before/after any certificate is renewed. Hooks -will only be run if a certificate is renewed. -* Recategorized `-h/--help` output to improve documentation and -discoverability. -* Busybox support in certbot-auto. -* Many small bug fixes. +* The ability to customize and automatically complete DNS and HTTP +domain validation challenges with the manual plugin. The flags +`--manual-auth-hook` and `--manual-cleanup-hook` can now be provided +when using the manual plugin to execute commands provided by the user to +perform and clean up challenges provided by the CA. This is best used in +complicated setups where the DNS challenge must be used or Certbot's +existing plugins cannot be used to perform HTTP challenges. For more +information on how this works, see `certbot --help manual`. +* A `--cert-name` flag for specifying the name to use for the +certificate in Certbot's configuration directory. Using this flag in +combination with `-d/--domains`, a user can easily request a new +certificate with different domains and save it with the name provided by +`--cert-name`. Additionally, `--cert-name` can be used to select a +certificate with the `certonly` and `run` subcommands so a full list of +domains in the certificate does not have to be provided. +* The subcommand `certificates` for listing the certificates managed by +Certbot and their properties. +* A `delete` subcommand for removing certificates managed by Certbot +from the configuration directory. +* Support for requesting internationalized domain names (IDNs). +* Removal of the ncurses interface. This change solves problems people +were having on many systems, reduces the number of Certbot dependencies, +and simplifies our code. Certbot's only interface now is the text +interface which was available by providing `-t/--text` to earlier +versions of Certbot. +* Hooks provided to Certbot are now saved to be reused during renewal. +If you run Certbot with `--pre-hook`, `--renew-hook`, or `--post-hook` +flags when obtaining a certificate, the provided commands will +automatically be saved and executed again when renewing the certificate. +A pre-hook and/or post-hook can also be given to the `certbot renew` +command either on the command line or in a [configuration +file](https://certbot.eff.org/docs/using.html#configuration-file) to run +an additional command before/after any certificate is renewed. Hooks +will only be run if a certificate is renewed. +* Recategorized `-h/--help` output to improve documentation and +discoverability. +* Busybox support in certbot-auto. +* Many small bug fixes. More details about these changes can be found on our GitHub repo: https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.0is%3Aclosed @@ -102,31 +112,31 @@ https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.0is%3Acl # 0.9.3 ## 10/13/2016 -* Adopt more conservative behavior about reporting a needed port as -unavailable when using the standalone plugin. -* The Apache plugin uses information about your OS to help determine the -layout of your Apache configuration directory. We added a patch to -ensure this code behaves the same way when testing on different systems -as the tests were failing in some cases. +* Adopt more conservative behavior about reporting a needed port as +unavailable when using the standalone plugin. +* The Apache plugin uses information about your OS to help determine the +layout of your Apache configuration directory. We added a patch to +ensure this code behaves the same way when testing on different systems +as the tests were failing in some cases. More details about these changes can be found on our GitHub repo: -https://github.com/certbot/certbot/milestone/27?closed=1 +https://github.com/certbot/certbot/milestone/27?closed=1 # 0.9.2 ## 10/12/2016 -* Ensuring we properly copy `ssl on;` directives as necessary when -performing domain validation in the Nginx plugin. -* Verifying that our optional dependencies version matches what is -required by Certbot. -* A fix for problems where symlinks were becoming files when they were -packaged, causing errors during testing and OS packaging. -* Stop requiring that all possibly required ports are available when -using the standalone plugin. Only verify the ports are available when -you know they are necessary. +* Ensuring we properly copy `ssl on;` directives as necessary when +performing domain validation in the Nginx plugin. +* Verifying that our optional dependencies version matches what is +required by Certbot. +* A fix for problems where symlinks were becoming files when they were +packaged, causing errors during testing and OS packaging. +* Stop requiring that all possibly required ports are available when +using the standalone plugin. Only verify the ports are available when +you know they are necessary. More details about these changes can be found on our GitHub repo: -https://github.com/certbot/certbot/milestone/26?closed=1 +https://github.com/certbot/certbot/milestone/26?closed=1 # 0.9.1 ## 10/06/2016 @@ -135,43 +145,43 @@ https://github.com/certbot/certbot/milestone/26?closed=1 0.9.0 where the command line flag -q/--quiet wasn't respected in some cases. More details about these changes can be found on our GitHub repo: -https://github.com/certbot/certbot/milestone/25?closed=1 +https://github.com/certbot/certbot/milestone/25?closed=1 # 0.9.0 ## 10/05/2016 -* An alpha version of the Nginx plugin. This plugin fully automates the -process of obtaining and installing certificates with Nginx. -Additionally, it is able to automatically configure security -enhancements such as an HTTP to HTTPS redirect and OCSP stapling. To use -this plugin, you must have the `certbot-nginx` package installed (which -is installed automatically when using `certbot-auto`) and provide -`--nginx` on the command line. This plugin is still in its early stages -so we recommend you use it with some caution and make sure you have a -backup of your Nginx configuration. -* Support for the `DNS` challenge in the `acme` library as well as `DNS` -support in Certbot's `manual` plugin. This allows you to create DNS -records to prove to Let's Encrypt you control the requested the domain -name. To use this feature, include `--manual --preferred-challenges dns` -on the command line. -* Help with enabling Extra Packages for Enterprise Linux (EPEL) on -CentOS 6 when using `certbot-auto`. To use `certbot-auto` on CentOS 6, -the EPEL repository has to be enabled. `certbot-auto` will now prompt -users asking them if they would like the script to enable this for them -automatically. This is done without prompting users when using -`letsencrypt-auto` or if `-n/--non-interactive/--noninteractive` is -included on the command line. +* An alpha version of the Nginx plugin. This plugin fully automates the +process of obtaining and installing certificates with Nginx. +Additionally, it is able to automatically configure security +enhancements such as an HTTP to HTTPS redirect and OCSP stapling. To use +this plugin, you must have the `certbot-nginx` package installed (which +is installed automatically when using `certbot-auto`) and provide +`--nginx` on the command line. This plugin is still in its early stages +so we recommend you use it with some caution and make sure you have a +backup of your Nginx configuration. +* Support for the `DNS` challenge in the `acme` library as well as `DNS` +support in Certbot's `manual` plugin. This allows you to create DNS +records to prove to Let's Encrypt you control the requested the domain +name. To use this feature, include `--manual --preferred-challenges dns` +on the command line. +* Help with enabling Extra Packages for Enterprise Linux (EPEL) on +CentOS 6 when using `certbot-auto`. To use `certbot-auto` on CentOS 6, +the EPEL repository has to be enabled. `certbot-auto` will now prompt +users asking them if they would like the script to enable this for them +automatically. This is done without prompting users when using +`letsencrypt-auto` or if `-n/--non-interactive/--noninteractive` is +included on the command line. More details about these changes can be found on our GitHub repo: -https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.9.0+is%3Aclosed +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.9.0+is%3Aclosed # 0.8.1 ## 06/14/2016 -* Preserving a certificate's common name when using `renew` -* Save webroot values for renewal when they are entered interactively -* Problems with an invalid user-agent string on OS X -* Gracefully reporting the Apache plugin isn't usable when Augeas is not installed +* Preserving a certificate's common name when using `renew` +* Save webroot values for renewal when they are entered interactively +* Problems with an invalid user-agent string on OS X +* Gracefully reporting the Apache plugin isn't usable when Augeas is not installed * Experimental support for Mageia has been added to `certbot-auto` More details about these changes can be found on our GitHub repo: @@ -180,10 +190,10 @@ https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.8.1+ # 0.8.0 ## 06/02/2016 -* The main new feature in this release is the `register` subcommand which -can be used to register an account with the Let's Encrypt CA. -* Additionally, you can run `certbot register --update-registration` to -change the e-mail address associated with your registration. +* The main new feature in this release is the `register` subcommand which +can be used to register an account with the Let's Encrypt CA. +* Additionally, you can run `certbot register --update-registration` to +change the e-mail address associated with your registration. More details about these changes can be found on our GitHub repo: https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.8.0+ @@ -191,18 +201,18 @@ https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.8.0+ # 0.7.0 ## 05/27/2016 -* `--must-staple` to request certificates from Let's Encrypt with the -OCSP must staple extension -* automatic configuration of OSCP stapling for Apache -* requesting certificates for domains found in the common name of a -custom CSR -* a number of bug fixes +* `--must-staple` to request certificates from Let's Encrypt with the +OCSP must staple extension +* automatic configuration of OSCP stapling for Apache +* requesting certificates for domains found in the common name of a +custom CSR +* a number of bug fixes More details about these changes can be found on our GitHub repo: -https://github.com/certbot/certbot/issues?q=milestone%3A0.7.0+is%3Aissue +https://github.com/certbot/certbot/issues?q=milestone%3A0.7.0+is%3Aissue # 0.6.0 -## 05/12/2016 +## 05/12/2016 * Renamed the client from `letsencrypt` to `certbot` * Fixed a small json deserialization error @@ -216,34 +226,34 @@ https://github.com/certbot/certbot/issues?q=is%3Aissue%20milestone%3A0.6.0%20is% # 0.5.0 ## 04/05/2016 -* The ability to use the webroot plugin interactively. -* The flags --pre-hook, --post-hook, and --renew-hook which can be used -with the renew subcommand to register shell commands to run in -response to renewal events. Pre-hook commands will be run before -any certs are renewed, post-hook commands will be run after any -certs are renewed, and renew-hook commands will be run after each -cert is renewed. If no certs are due for renewal, no command is run. -* Cleaner renewal configuration files. In /etc/letsencrypt/renewal by -default, these files can be used to control what parameters are used -when renewing a specific certificate. -* A -q/--quiet flag which silences all output except errors. -* An --allow-subset-of-domains flag which can be used with the renew -command to prevent renewal failures for a subset of the requested -domains from causing the client to exit. +* The ability to use the webroot plugin interactively. +* The flags --pre-hook, --post-hook, and --renew-hook which can be used +with the renew subcommand to register shell commands to run in +response to renewal events. Pre-hook commands will be run before +any certs are renewed, post-hook commands will be run after any +certs are renewed, and renew-hook commands will be run after each +cert is renewed. If no certs are due for renewal, no command is run. +* Cleaner renewal configuration files. In /etc/letsencrypt/renewal by +default, these files can be used to control what parameters are used +when renewing a specific certificate. +* A -q/--quiet flag which silences all output except errors. +* An --allow-subset-of-domains flag which can be used with the renew +command to prevent renewal failures for a subset of the requested +domains from causing the client to exit. More details about these changes can be found on our GitHub repo: -https://github.com/letsencrypt/letsencrypt/issues?q=milestone%3A0.5.0+is%3Aissue +https://github.com/letsencrypt/letsencrypt/issues?q=milestone%3A0.5.0+is%3Aissue # 0.4.2 ## 03/03/2016 -* Resolves problems encountered when compiling letsencrypt -against the new OpenSSL release. +* Resolves problems encountered when compiling letsencrypt +against the new OpenSSL release. * A patch fixing problems of using letsencrypt renew with configuration files -from private beta has been added. +from private beta has been added. More details about these changes can be found on our GitHub repo: -https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.4.2 +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.4.2 # 0.4.1 ## 02/29/2016 @@ -254,73 +264,73 @@ https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.4.2 * Fixes problems with parsing renewal config files from private beta More details about these changes can be found on our GitHub repo: -https://github.com/letsencrypt/letsencrypt/issues?q=is:issue+milestone:0.4.1 +https://github.com/letsencrypt/letsencrypt/issues?q=is:issue+milestone:0.4.1 # 0.4.0 ## 02/10/2016 -* The new verb/subcommand `renew` can be used to renew your existing -certificates as they approach expiration. Running `letsencrypt renew` -will examine all existing certificate lineages and determine if any are -less than 30 days from expiration. If so, the client will use the -settings provided when you previously obtained the certificate to renew -it. The subcommand finishes by printing a summary of which renewals were -successful, failed, or not yet due. -* A `--dry-run` flag has been added to help with testing configuration -without affecting production rate limits. Currently supported by the -`renew` and `certonly` subcommands, providing `--dry-run` on the command -line will obtain certificates from the staging server without saving the -resulting certificates to disk. -* Major improvements have been added to letsencrypt-auto. This script -has been rewritten to include full support for Python 2.6, the ability -for letsencrypt-auto to update itself, and improvements to the -stability, security, and performance of the script. -* Support for Apache 2.2 has been added to the Apache plugin. +* The new verb/subcommand `renew` can be used to renew your existing +certificates as they approach expiration. Running `letsencrypt renew` +will examine all existing certificate lineages and determine if any are +less than 30 days from expiration. If so, the client will use the +settings provided when you previously obtained the certificate to renew +it. The subcommand finishes by printing a summary of which renewals were +successful, failed, or not yet due. +* A `--dry-run` flag has been added to help with testing configuration +without affecting production rate limits. Currently supported by the +`renew` and `certonly` subcommands, providing `--dry-run` on the command +line will obtain certificates from the staging server without saving the +resulting certificates to disk. +* Major improvements have been added to letsencrypt-auto. This script +has been rewritten to include full support for Python 2.6, the ability +for letsencrypt-auto to update itself, and improvements to the +stability, security, and performance of the script. +* Support for Apache 2.2 has been added to the Apache plugin. More details about these changes can be found on our GitHub repo: -https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.4.0 - +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.4.0 + # 0.3.0 ## 01/27/2016 -* A non-interactive mode which can be enabled by including `-n` or -`--non-interactive` on the command line. This can be used to -guarantee the client will not prompt when run automatically using -cron/systemd. -* Preparation for the new letsencrypt-auto script. Over the past -couple months, we've been working on increasing the reliability and -security of letsencrypt-auto. A number of changes landed in this -release to prepare for the new version of this script. +* A non-interactive mode which can be enabled by including `-n` or +`--non-interactive` on the command line. This can be used to +guarantee the client will not prompt when run automatically using +cron/systemd. +* Preparation for the new letsencrypt-auto script. Over the past +couple months, we've been working on increasing the reliability and +security of letsencrypt-auto. A number of changes landed in this +release to prepare for the new version of this script. More details about these changes can be found on our GitHub repo: -https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.3.0 +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.3.0 # 0.2.0 ## 01/14/2016 -* Apache plugin support for non-Debian based systems. Support has been -added for modern Red Hat based systems such as Fedora 23, Red Hat 7, -and CentOS 7 running Apache 2.4. In theory, this plugin should be -able to be configured to run on any Unix-like OS running Apache 2.4. -* Relaxed PyOpenSSL version requirements. This adds support for systems -with PyOpenSSL versions 0.13 or 0.14. -* Resolves issues with the Apache plugin enabling an HTTP to HTTPS -redirect on some systems. -* Improved error messages from the client. +* Apache plugin support for non-Debian based systems. Support has been +added for modern Red Hat based systems such as Fedora 23, Red Hat 7, +and CentOS 7 running Apache 2.4. In theory, this plugin should be +able to be configured to run on any Unix-like OS running Apache 2.4. +* Relaxed PyOpenSSL version requirements. This adds support for systems +with PyOpenSSL versions 0.13 or 0.14. +* Resolves issues with the Apache plugin enabling an HTTP to HTTPS +redirect on some systems. +* Improved error messages from the client. More details about these changes can be found on our GitHub repo: -https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.2.0 +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.2.0 # 0.1.1 ## 12/15/2015 -* Fix a confusing UI path that caused some users to repeatedly renew -their certs while experimenting with the client, in some cases -hitting issuance rate limits -* Fixes numerous Apache configuration parser fixes -* Avoids attempting to issue for unqualified domain names like -"localhost" -* Fixes --webroot permission handling for non-root users +* Fix a confusing UI path that caused some users to repeatedly renew +their certs while experimenting with the client, in some cases +hitting issuance rate limits +* Fixes numerous Apache configuration parser fixes +* Avoids attempting to issue for unqualified domain names like +"localhost" +* Fixes --webroot permission handling for non-root users More details about these changes can be found on our GitHub repo: -https://github.com/letsencrypt/letsencrypt/issues?q=milestone%3A0.1.1 +https://github.com/letsencrypt/letsencrypt/issues?q=milestone%3A0.1.1 From 26a7023b8dd985674dbf9ff2d35d4065ca7cae9d Mon Sep 17 00:00:00 2001 From: Sagi Kedmi Date: Fri, 3 Mar 2017 02:49:34 +0200 Subject: [PATCH 81/88] Change QSA to NE in HTTPS redirection (#4204) * Change QSA to NE in HTTPS redirection * Seamless transition to new HTTPS redirection RewriteRule --- certbot-apache/certbot_apache/configurator.py | 41 +++++++++++++----- certbot-apache/certbot_apache/constants.py | 8 +++- .../certbot_apache/tests/configurator_test.py | 43 ++++++++++++++++--- 3 files changed, 73 insertions(+), 19 deletions(-) diff --git a/certbot-apache/certbot_apache/configurator.py b/certbot-apache/certbot_apache/configurator.py index 5639dae83..cdfc01626 100644 --- a/certbot-apache/certbot_apache/configurator.py +++ b/certbot-apache/certbot_apache/configurator.py @@ -1315,18 +1315,15 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator): # even with save() and load() if not self._is_rewrite_engine_on(general_vh): self.parser.add_dir(general_vh.path, "RewriteEngine", "on") + names = ssl_vhost.get_names() for idx, name in enumerate(names): args = ["%{SERVER_NAME}", "={0}".format(name), "[OR]"] if idx == len(names) - 1: args.pop() self.parser.add_dir(general_vh.path, "RewriteCond", args) - if self.get_version() >= (2, 3, 9): - self.parser.add_dir(general_vh.path, "RewriteRule", - constants.REWRITE_HTTPS_ARGS_WITH_END) - else: - self.parser.add_dir(general_vh.path, "RewriteRule", - constants.REWRITE_HTTPS_ARGS) + + self._set_https_redirection_rewrite_rule(general_vh) self.save_notes += ("Redirecting host in %s to ssl vhost in %s\n" % (general_vh.filep, ssl_vhost.filep)) @@ -1336,12 +1333,24 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator): logger.info("Redirecting vhost in %s to ssl vhost in %s", general_vh.filep, ssl_vhost.filep) + def _set_https_redirection_rewrite_rule(self, vhost): + if self.get_version() >= (2, 3, 9): + self.parser.add_dir(vhost.path, "RewriteRule", + constants.REWRITE_HTTPS_ARGS_WITH_END) + else: + self.parser.add_dir(vhost.path, "RewriteRule", + constants.REWRITE_HTTPS_ARGS) + + def _verify_no_certbot_redirect(self, vhost): """Checks to see if a redirect was already installed by certbot. Checks to see if virtualhost already contains a rewrite rule that is identical to Certbot's redirection rewrite rule. + For graceful transition to new rewrite rules for HTTPS redireciton we + delete certbot's old rewrite rules and set the new one instead. + :param vhost: vhost to check :type vhost: :class:`~certbot_apache.obj.VirtualHost` @@ -1355,19 +1364,29 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator): # rewrite_args_dict keys are directive ids and the corresponding value # for each is a list of arguments to that directive. rewrite_args_dict = defaultdict(list) - pat = r'.*(directive\[\d+\]).*' + pat = r'(.*directive\[\d+\]).*' for match in rewrite_path: m = re.match(pat, match) if m: - dir_id = m.group(1) - rewrite_args_dict[dir_id].append(match) + dir_path = m.group(1) + rewrite_args_dict[dir_path].append(match) if rewrite_args_dict: redirect_args = [constants.REWRITE_HTTPS_ARGS, constants.REWRITE_HTTPS_ARGS_WITH_END] - for matches in rewrite_args_dict.values(): - if [self.aug.get(x) for x in matches] in redirect_args: + for dir_path, args_paths in rewrite_args_dict.items(): + arg_vals = [self.aug.get(x) for x in args_paths] + + # Search for past redirection rule, delete it, set the new one + if arg_vals in constants.OLD_REWRITE_HTTPS_ARGS: + self.aug.remove(dir_path) + self._set_https_redirection_rewrite_rule(vhost) + self.save() + raise errors.PluginEnhancementAlreadyPresent( + "Certbot has already enabled redirection") + + if arg_vals in redirect_args: raise errors.PluginEnhancementAlreadyPresent( "Certbot has already enabled redirection") diff --git a/certbot-apache/certbot_apache/constants.py b/certbot-apache/certbot_apache/constants.py index dcc635c4b..3cfeb4dd6 100644 --- a/certbot-apache/certbot_apache/constants.py +++ b/certbot-apache/certbot_apache/constants.py @@ -136,15 +136,19 @@ AUGEAS_LENS_DIR = pkg_resources.resource_filename( """Path to the Augeas lens directory""" REWRITE_HTTPS_ARGS = [ - "^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[L,QSA,R=permanent]"] + "^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[L,NE,R=permanent]"] """Apache version<2.3.9 rewrite rule arguments used for redirections to https vhost""" REWRITE_HTTPS_ARGS_WITH_END = [ - "^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[END,QSA,R=permanent]"] + "^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[END,NE,R=permanent]"] """Apache version >= 2.3.9 rewrite rule arguments used for redirections to https vhost""" +OLD_REWRITE_HTTPS_ARGS = [ + ["^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[L,QSA,R=permanent]"], + ["^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[END,QSA,R=permanent]"]] + HSTS_ARGS = ["always", "set", "Strict-Transport-Security", "\"max-age=31536000\""] """Apache header arguments for HSTS""" diff --git a/certbot-apache/certbot_apache/tests/configurator_test.py b/certbot-apache/certbot_apache/tests/configurator_test.py index 937694267..45e701bd5 100644 --- a/certbot-apache/certbot_apache/tests/configurator_test.py +++ b/certbot-apache/certbot_apache/tests/configurator_test.py @@ -18,6 +18,7 @@ from certbot.tests import acme_util from certbot.tests import util as certbot_util from certbot_apache import configurator +from certbot_apache import constants from certbot_apache import parser from certbot_apache import obj @@ -1047,6 +1048,36 @@ class MultipleVhostsTest(util.ApacheTest): self.assertTrue("rewrite_module" in self.config.parser.modules) + @mock.patch("certbot.util.run_script") + @mock.patch("certbot.util.exe_exists") + def test_redirect_with_old_https_redirection(self, mock_exe, _): + self.config.parser.update_runtime_variables = mock.Mock() + mock_exe.return_value = True + self.config.get_version = mock.Mock(return_value=(2, 2, 0)) + + ssl_vhost = self.config.choose_vhost("certbot.demo") + + # pylint: disable=protected-access + http_vhost = self.config._get_http_vhost(ssl_vhost) + + # Create an old (previously suppoorted) https redirectoin rewrite rule + self.config.parser.add_dir( + http_vhost.path, "RewriteRule", + ["^", + "https://%{SERVER_NAME}%{REQUEST_URI}", + "[L,QSA,R=permanent]"]) + + self.config.save() + + try: + self.config.enhance("certbot.demo", "redirect") + except errors.PluginEnhancementAlreadyPresent: + args_paths = self.config.parser.find_dir( + "RewriteRule", None, http_vhost.path, False) + arg_vals = [self.config.aug.get(x) for x in args_paths] + self.assertEqual(arg_vals, constants.REWRITE_HTTPS_ARGS) + + def test_redirect_with_conflict(self): self.config.parser.modules.add("rewrite_module") ssl_vh = obj.VirtualHost( @@ -1134,7 +1165,7 @@ class MultipleVhostsTest(util.ApacheTest): http_vhost.path, "RewriteRule", ["^", "https://%{SERVER_NAME}%{REQUEST_URI}", - "[L,QSA,R=permanent]"]) + "[L,NE,R=permanent]"]) self.config.save() ssl_vhost = self.config.make_vhost_ssl(self.vh_truth[0]) @@ -1145,7 +1176,7 @@ class MultipleVhostsTest(util.ApacheTest): conf_text = open(ssl_vhost.filep).read() commented_rewrite_rule = ("# RewriteRule ^ " "https://%{SERVER_NAME}%{REQUEST_URI} " - "[L,QSA,R=permanent]") + "[L,NE,R=permanent]") self.assertTrue(commented_rewrite_rule in conf_text) mock_get_utility().add_message.assert_called_once_with(mock.ANY, @@ -1164,7 +1195,7 @@ class MultipleVhostsTest(util.ApacheTest): "RewriteCond", ["%{DOCUMENT_ROOT}/%{REQUEST_FILENAME}", "!-f"]) self.config.parser.add_dir( http_vhost.path, "RewriteRule", - ["^(.*)$", "b://u%{REQUEST_URI}", "[P,QSA,L]"]) + ["^(.*)$", "b://u%{REQUEST_URI}", "[P,NE,L]"]) # Add a chunk that should be commented out. self.config.parser.add_dir(http_vhost.path, @@ -1175,7 +1206,7 @@ class MultipleVhostsTest(util.ApacheTest): http_vhost.path, "RewriteRule", ["^", "https://%{SERVER_NAME}%{REQUEST_URI}", - "[L,QSA,R=permanent]"]) + "[L,NE,R=permanent]"]) self.config.save() @@ -1186,13 +1217,13 @@ class MultipleVhostsTest(util.ApacheTest): not_commented_cond1 = ("RewriteCond " "%{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f") not_commented_rewrite_rule = ("RewriteRule " - "^(.*)$ b://u%{REQUEST_URI} [P,QSA,L]") + "^(.*)$ b://u%{REQUEST_URI} [P,NE,L]") commented_cond1 = "# RewriteCond %{HTTPS} !=on" commented_cond2 = "# RewriteCond %{HTTPS} !^$" commented_rewrite_rule = ("# RewriteRule ^ " "https://%{SERVER_NAME}%{REQUEST_URI} " - "[L,QSA,R=permanent]") + "[L,NE,R=permanent]") self.assertTrue(not_commented_cond1 in conf_line_set) self.assertTrue(not_commented_rewrite_rule in conf_line_set) From 53117b0ce0f9e781ae9a44570cfc7dc3891bb918 Mon Sep 17 00:00:00 2001 From: Jacob Hoffman-Andrews Date: Thu, 2 Mar 2017 17:27:29 -0800 Subject: [PATCH 82/88] Remove UnexpectedUpdate exceptions. (#4197) * Remove UnexpectedUpdate exceptions. These exceptions trigger when the server sends the client back an object with a field that doesn't exactly match what the client previously sent. This causes unnecessary breakage in various cases, doesn't prevent any problems, and isn't required by spec. * Back out all UnexpectedUpdate removals except registration update. --- acme/acme/client.py | 3 --- acme/acme/client_test.py | 10 ---------- certbot/client.py | 2 -- 3 files changed, 15 deletions(-) diff --git a/acme/acme/client.py b/acme/acme/client.py index ddcba7635..6c5ed79a2 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -134,8 +134,6 @@ class Client(object): # pylint: disable=too-many-instance-attributes update = regr.body if update is None else update body = messages.UpdateRegistration(**dict(update)) updated_regr = self._send_recv_regr(regr, body=body) - if updated_regr != regr: - raise errors.UnexpectedUpdate(regr) return updated_regr def deactivate_registration(self, regr): @@ -301,7 +299,6 @@ class Client(object): # pylint: disable=too-many-instance-attributes response = self.net.get(authzr.uri) updated_authzr = self._authzr_from_response( response, authzr.body.identifier, authzr.uri, authzr.new_cert_uri) - # TODO: check and raise UnexpectedUpdate return updated_authzr, response def request_issuance(self, csr, authzrs): diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index b3db21ac9..7e7ffe779 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -121,8 +121,6 @@ class ClientTest(unittest.TestCase): # TODO: split here and separate test self.response.json.return_value = self.regr.body.update( contact=()).to_json() - self.assertRaises( - errors.UnexpectedUpdate, self.client.update_registration, self.regr) def test_deactivate_account(self): self.response.headers['Location'] = self.regr.uri @@ -130,14 +128,6 @@ class ClientTest(unittest.TestCase): self.assertEqual(self.regr, self.client.deactivate_registration(self.regr)) - def test_deactivate_account_bad_registration_returned(self): - self.response.headers['Location'] = self.regr.uri - self.response.json.return_value = "some wrong registration thing" - self.assertRaises( - errors.UnexpectedUpdate, - self.client.deactivate_registration, - self.regr) - def test_query_registration(self): self.response.json.return_value = self.regr.body.to_json() self.assertEqual(self.regr, self.client.query_registration(self.regr)) diff --git a/certbot/client.py b/certbot/client.py index 95882a9fc..a342c1bf3 100644 --- a/certbot/client.py +++ b/certbot/client.py @@ -155,8 +155,6 @@ def perform_registration(acme, config): :returns: Registration Resource. :rtype: `acme.messages.RegistrationResource` - - :raises .UnexpectedUpdate: """ try: return acme.register(messages.NewRegistration.from_data(email=config.email)) From 93908a33bcdf79db297c38445cc31266ea066a31 Mon Sep 17 00:00:00 2001 From: yomna Date: Thu, 2 Mar 2017 17:28:45 -0800 Subject: [PATCH 83/88] [#3451] small changes to the standalone documentation (#4247) --- docs/using.rst | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/using.rst b/docs/using.rst index 628043ff9..1582c5ab7 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -144,6 +144,10 @@ the ``--nginx`` flag on the commandline. Standalone ---------- +Use standalone mode to obtain a cert if you don't want to use (or don't currently have) +existing server software. The standalone plugin does not rely on any other server +software running on the machine where you obtain the cert. + To obtain a cert using a "standalone" webserver, you can use the standalone plugin by including ``certonly`` and ``--standalone`` on the command line. This plugin needs to bind to port 80 or 443 in @@ -154,10 +158,8 @@ one of the options shown below on the command line. * ``--standalone-supported-challenges http-01`` to use port 80 * ``--standalone-supported-challenges tls-sni-01`` to use port 443 -The standalone plugin does not rely on any other server software running -on the machine where you obtain the certificate. It must still be possible -for that machine to accept inbound connections from the Internet on the -specified port using each requested domain name. +It must still be possible for your machine to accept inbound connections from +the Internet on the specified port using each requested domain name. Manual ------ From 12a6e49cf11d3b7de623c6279ffbf1387328688f Mon Sep 17 00:00:00 2001 From: Blake Griffith Date: Thu, 2 Mar 2017 21:16:19 -0800 Subject: [PATCH 84/88] Remove use of sha1 (#4271) These are not security critical uses of sha1 but they should still be removed. --- acme/acme/challenges.py | 2 +- certbot/tests/crypto_util_test.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/acme/acme/challenges.py b/acme/acme/challenges.py index 3b1e90166..ac4e3d60a 100644 --- a/acme/acme/challenges.py +++ b/acme/acme/challenges.py @@ -445,7 +445,7 @@ class TLSSNI01Response(KeyAuthorizationChallengeResponse): """ # pylint: disable=protected-access sans = crypto_util._pyopenssl_cert_or_req_san(cert) - logger.debug('Certificate %s. SANs: %s', cert.digest('sha1'), sans) + logger.debug('Certificate %s. SANs: %s', cert.digest('sha256'), sans) return self.z_domain.decode() in sans def simple_verify(self, chall, domain, account_public_key, diff --git a/certbot/tests/crypto_util_test.py b/certbot/tests/crypto_util_test.py index a580574a4..946e772c1 100644 --- a/certbot/tests/crypto_util_test.py +++ b/certbot/tests/crypto_util_test.py @@ -336,8 +336,8 @@ class CertLoaderTest(unittest.TestCase): from certbot.crypto_util import pyopenssl_load_certificate cert, file_type = pyopenssl_load_certificate(CERT) - self.assertEqual(cert.digest('sha1'), - OpenSSL.crypto.load_certificate(file_type, CERT).digest('sha1')) + self.assertEqual(cert.digest('sha256'), + OpenSSL.crypto.load_certificate(file_type, CERT).digest('sha256')) def test_load_invalid_cert(self): from certbot.crypto_util import pyopenssl_load_certificate From 1507b6b7316b0e0ba5aabc56cbaf64904ae15d59 Mon Sep 17 00:00:00 2001 From: yonjah Date: Sat, 4 Mar 2017 00:34:30 +0800 Subject: [PATCH 85/88] Added documentation about renew exit status #Fixes #4090 (#4234) * Added documentation about renew exit status #Fixes #4090 * recommend using post-hook instead of renew-hook --- docs/using.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/using.rst b/docs/using.rst index 1582c5ab7..104c688e7 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -411,6 +411,13 @@ Certbot is working hard to improve the renewal process, and we apologize for any inconvenience you encounter in integrating these commands into your individual environment. +.. note:: ``certbot renew`` exit status will only be 1 if a renewal attempt failed. + This means ``certbot renew`` exit status will be 0 if no cert needs to be updated. + If you write a custom script and expect to run a command only after a cert was actually renewed + you will need to use the ``--post-hook`` since the exit status will be 0 both on successful renewal + and when renewal is not necessary. + + Modifying the Renewal Configuration File ---------------------------------------- From 80326511bb1f0f7392f2c5bd259a74e3c6c0c195 Mon Sep 17 00:00:00 2001 From: Alex Bowers Date: Fri, 3 Mar 2017 18:28:05 +0000 Subject: [PATCH 86/88] Improve error reporting for hooks (#4235) * Improve error reporting for hooks * My bad * Whitespace. --- certbot/hooks.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/certbot/hooks.py b/certbot/hooks.py index 5cda478cc..ada3d3aaa 100644 --- a/certbot/hooks.py +++ b/certbot/hooks.py @@ -44,8 +44,12 @@ def validate_hook(shell_cmd, hook_name): cmd = shell_cmd.split(None, 1)[0] if not _prog(cmd): path = os.environ["PATH"] - msg = "Unable to find {2}-hook command {0} in the PATH.\n(PATH is {1})".format( - cmd, path, hook_name) + if os.path.exists(cmd): + msg = "{1}-hook command {0} exists, but is not executable.".format(cmd, hook_name) + else: + msg = "Unable to find {2}-hook command {0} in the PATH.\n(PATH is {1})".format( + cmd, path, hook_name) + raise errors.HookCommandNotFound(msg) def pre_hook(config): From ea578870da558c86546fb6b9b34d95ffea6f12a3 Mon Sep 17 00:00:00 2001 From: Jacob Hoffman-Andrews Date: Fri, 3 Mar 2017 10:37:31 -0800 Subject: [PATCH 87/88] ipdb can now be run without pip installing. (#4257) --- docs/contributing.rst | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/contributing.rst b/docs/contributing.rst index 5f939e947..de9904936 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -69,10 +69,8 @@ either in the same directory as ``foo.py`` or in the ``tests`` subdirectory (if there isn't, make one). While you are working on your code and tests, run ``python foo_test.py`` to run the relevant tests. -For debugging, we recommend running ``pip install ipdb`` and putting -``import ipdb; ipdb.set_trace()`` statements inside the source -code. Alternatively, you can use Python's standard library `pdb`, -but you won't get TAB completion. +For debugging, we recommend putting +``import ipdb; ipdb.set_trace()`` statements inside the source code. Once you are done with your code changes, and the tests in ``foo_test.py`` pass, run all of the unittests for Certbot with ``tox -e py27`` (this uses Python From 2862ade0b12c3970ba36c9cefa8f4b89b7a82e0f Mon Sep 17 00:00:00 2001 From: Shiloh Heurich Date: Fri, 3 Mar 2017 13:44:12 -0500 Subject: [PATCH 88/88] docs(ciphers): newer keylength.com recommendations (#4266) --- docs/ciphers.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ciphers.rst b/docs/ciphers.rst index 31ce45963..1b320cdf9 100644 --- a/docs/ciphers.rst +++ b/docs/ciphers.rst @@ -255,7 +255,7 @@ I have access to an English-language summary of the recommendations. Keylength.com ~~~~~~~~~~~~~ -Damien Giry collects recommendations by academic researchers and standards organizations about keylengths for particular cryptoperiods, years, or security levels. The keylength recommendations of the various sources are summarized in a chart. This site has been updated over time and includes expert guidance from eight sources published between 2000 and 2015. +Damien Giry collects recommendations by academic researchers and standards organizations about keylengths for particular cryptoperiods, years, or security levels. The keylength recommendations of the various sources are summarized in a chart. This site has been updated over time and includes expert guidance from eight sources published between 2000 and 2017. http://www.keylength.com/