diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 000000000..cc1ad82ed --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,336 @@ +# 0.12.0 +## 03/02/2017 + +* Allow non-camelcase Apache VirtualHost names +* Allow more log messages to be silenced +* Fix a regression around using `--cert-name` when getting new certificates + +More information about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue%20milestone%3A0.12.0 + +# 0.11.1 +## 02/01/2017 + +* Resolve a problem where Certbot would crash while parsing command line +arguments in some cases. +* Fix a typo. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/pulls?q=is%3Apr%20milestone%3A0.11.1%20is%3Aclosed + +# 0.11.0 +## 02/01/2017 + +* Providing `--quiet` to `certbot-auto` now silences package manager output. +* The UI has been improved in the standalone plugin. When using the +plugin while running Certbot interactively and a required port is bound +by another process, Certbot will give you the option to retry to grab +the port rather than immediately exiting. +* You are now able to deactivate your account with the Let's Encrypt +server using the `unregister` subcommand. +* When revoking a certificate using the `revoke` subcommand, you now +have the option to provide the reason the certificate is being revoked +to Let's Encrypt with `--reason`. +* Removal of the optional `dnspython` dependency in our `acme` package. +Now the library does not support client side verification of the DNS +challenge. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.11.0+is%3Aclosed + +# 0.10.2 +## 01/25/2017 + +* We now save `--preferred-challenges` values for renewal. Previously +these values were discarded causing a different challenge type to be +used when renewing certs in some cases. +* If Certbot receives a request with a `badNonce` error, we +automatically retry the request. Since nonces from Let's Encrypt expire, +this helps people performing the DNS challenge with the `manual` plugin +who may have to wait an extended period of time for their DNS changes to +propagate. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.2+is%3Aclosed + +# 0.10.1 +## 01/13/2017 + +* Resolve problems where when asking Certbot to update a certificate at +an existing path to include different domain names, the old names would +continue to be used. +* Fix issues successfully running our unit test suite on some systems. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.1+is%3Aclosed + +# 0.10.0 +## 01/11/2017 + +* The ability to customize and automatically complete DNS and HTTP +domain validation challenges with the manual plugin. The flags +`--manual-auth-hook` and `--manual-cleanup-hook` can now be provided +when using the manual plugin to execute commands provided by the user to +perform and clean up challenges provided by the CA. This is best used in +complicated setups where the DNS challenge must be used or Certbot's +existing plugins cannot be used to perform HTTP challenges. For more +information on how this works, see `certbot --help manual`. +* A `--cert-name` flag for specifying the name to use for the +certificate in Certbot's configuration directory. Using this flag in +combination with `-d/--domains`, a user can easily request a new +certificate with different domains and save it with the name provided by +`--cert-name`. Additionally, `--cert-name` can be used to select a +certificate with the `certonly` and `run` subcommands so a full list of +domains in the certificate does not have to be provided. +* The subcommand `certificates` for listing the certificates managed by +Certbot and their properties. +* A `delete` subcommand for removing certificates managed by Certbot +from the configuration directory. +* Support for requesting internationalized domain names (IDNs). +* Removal of the ncurses interface. This change solves problems people +were having on many systems, reduces the number of Certbot dependencies, +and simplifies our code. Certbot's only interface now is the text +interface which was available by providing `-t/--text` to earlier +versions of Certbot. +* Hooks provided to Certbot are now saved to be reused during renewal. +If you run Certbot with `--pre-hook`, `--renew-hook`, or `--post-hook` +flags when obtaining a certificate, the provided commands will +automatically be saved and executed again when renewing the certificate. +A pre-hook and/or post-hook can also be given to the `certbot renew` +command either on the command line or in a [configuration +file](https://certbot.eff.org/docs/using.html#configuration-file) to run +an additional command before/after any certificate is renewed. Hooks +will only be run if a certificate is renewed. +* Recategorized `-h/--help` output to improve documentation and +discoverability. +* Busybox support in certbot-auto. +* Many small bug fixes. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.10.0is%3Aclosed + +# 0.9.3 +## 10/13/2016 + +* Adopt more conservative behavior about reporting a needed port as +unavailable when using the standalone plugin. +* The Apache plugin uses information about your OS to help determine the +layout of your Apache configuration directory. We added a patch to +ensure this code behaves the same way when testing on different systems +as the tests were failing in some cases. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/milestone/27?closed=1 + +# 0.9.2 +## 10/12/2016 + +* Ensuring we properly copy `ssl on;` directives as necessary when +performing domain validation in the Nginx plugin. +* Verifying that our optional dependencies version matches what is +required by Certbot. +* A fix for problems where symlinks were becoming files when they were +packaged, causing errors during testing and OS packaging. +* Stop requiring that all possibly required ports are available when +using the standalone plugin. Only verify the ports are available when +you know they are necessary. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/milestone/26?closed=1 + +# 0.9.1 +## 10/06/2016 + +* This version of Certbot simply fixes a bug that was introduced in version +0.9.0 where the command line flag -q/--quiet wasn't respected in some cases. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/milestone/25?closed=1 + +# 0.9.0 +## 10/05/2016 + +* An alpha version of the Nginx plugin. This plugin fully automates the +process of obtaining and installing certificates with Nginx. +Additionally, it is able to automatically configure security +enhancements such as an HTTP to HTTPS redirect and OCSP stapling. To use +this plugin, you must have the `certbot-nginx` package installed (which +is installed automatically when using `certbot-auto`) and provide +`--nginx` on the command line. This plugin is still in its early stages +so we recommend you use it with some caution and make sure you have a +backup of your Nginx configuration. +* Support for the `DNS` challenge in the `acme` library as well as `DNS` +support in Certbot's `manual` plugin. This allows you to create DNS +records to prove to Let's Encrypt you control the requested the domain +name. To use this feature, include `--manual --preferred-challenges dns` +on the command line. +* Help with enabling Extra Packages for Enterprise Linux (EPEL) on +CentOS 6 when using `certbot-auto`. To use `certbot-auto` on CentOS 6, +the EPEL repository has to be enabled. `certbot-auto` will now prompt +users asking them if they would like the script to enable this for them +automatically. This is done without prompting users when using +`letsencrypt-auto` or if `-n/--non-interactive/--noninteractive` is +included on the command line. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.9.0+is%3Aclosed + +# 0.8.1 +## 06/14/2016 + +* Preserving a certificate's common name when using `renew` +* Save webroot values for renewal when they are entered interactively +* Problems with an invalid user-agent string on OS X +* Gracefully reporting the Apache plugin isn't usable when Augeas is not installed +* Experimental support for Mageia has been added to `certbot-auto` + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.8.1+ + +# 0.8.0 +## 06/02/2016 + +* The main new feature in this release is the `register` subcommand which +can be used to register an account with the Let's Encrypt CA. +* Additionally, you can run `certbot register --update-registration` to +change the e-mail address associated with your registration. + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue+milestone%3A0.8.0+ + +# 0.7.0 +## 05/27/2016 + +* `--must-staple` to request certificates from Let's Encrypt with the +OCSP must staple extension +* automatic configuration of OSCP stapling for Apache +* requesting certificates for domains found in the common name of a +custom CSR +* a number of bug fixes + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=milestone%3A0.7.0+is%3Aissue + +# 0.6.0 +## 05/12/2016 + +* Renamed the client from `letsencrypt` to `certbot` +* Fixed a small json deserialization error +* Versioned the datetime dependency in setup.py +* Preserve domain order in generated CSRs +* Some minor bug fixes + +More details about these changes can be found on our GitHub repo: +https://github.com/certbot/certbot/issues?q=is%3Aissue%20milestone%3A0.6.0%20is%3Aclosed%20 + +# 0.5.0 +## 04/05/2016 + +* The ability to use the webroot plugin interactively. +* The flags --pre-hook, --post-hook, and --renew-hook which can be used +with the renew subcommand to register shell commands to run in +response to renewal events. Pre-hook commands will be run before +any certs are renewed, post-hook commands will be run after any +certs are renewed, and renew-hook commands will be run after each +cert is renewed. If no certs are due for renewal, no command is run. +* Cleaner renewal configuration files. In /etc/letsencrypt/renewal by +default, these files can be used to control what parameters are used +when renewing a specific certificate. +* A -q/--quiet flag which silences all output except errors. +* An --allow-subset-of-domains flag which can be used with the renew +command to prevent renewal failures for a subset of the requested +domains from causing the client to exit. + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=milestone%3A0.5.0+is%3Aissue + +# 0.4.2 +## 03/03/2016 + +* Resolves problems encountered when compiling letsencrypt +against the new OpenSSL release. +* A patch fixing problems of using letsencrypt renew with configuration files +from private beta has been added. + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.4.2 + +# 0.4.1 +## 02/29/2016 + +* Fixes Apache parsing errors with some configurations +* Fixes Werkzeug dependency problems on some Red Hat systems +* Fixes bootstrapping failures when using letsencrypt-auto with --no-self-upgrade +* Fixes problems with parsing renewal config files from private beta + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=is:issue+milestone:0.4.1 + +# 0.4.0 +## 02/10/2016 + +* The new verb/subcommand `renew` can be used to renew your existing +certificates as they approach expiration. Running `letsencrypt renew` +will examine all existing certificate lineages and determine if any are +less than 30 days from expiration. If so, the client will use the +settings provided when you previously obtained the certificate to renew +it. The subcommand finishes by printing a summary of which renewals were +successful, failed, or not yet due. +* A `--dry-run` flag has been added to help with testing configuration +without affecting production rate limits. Currently supported by the +`renew` and `certonly` subcommands, providing `--dry-run` on the command +line will obtain certificates from the staging server without saving the +resulting certificates to disk. +* Major improvements have been added to letsencrypt-auto. This script +has been rewritten to include full support for Python 2.6, the ability +for letsencrypt-auto to update itself, and improvements to the +stability, security, and performance of the script. +* Support for Apache 2.2 has been added to the Apache plugin. + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.4.0 + +# 0.3.0 +## 01/27/2016 + +* A non-interactive mode which can be enabled by including `-n` or +`--non-interactive` on the command line. This can be used to +guarantee the client will not prompt when run automatically using +cron/systemd. +* Preparation for the new letsencrypt-auto script. Over the past +couple months, we've been working on increasing the reliability and +security of letsencrypt-auto. A number of changes landed in this +release to prepare for the new version of this script. + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.3.0 + +# 0.2.0 +## 01/14/2016 + +* Apache plugin support for non-Debian based systems. Support has been +added for modern Red Hat based systems such as Fedora 23, Red Hat 7, +and CentOS 7 running Apache 2.4. In theory, this plugin should be +able to be configured to run on any Unix-like OS running Apache 2.4. +* Relaxed PyOpenSSL version requirements. This adds support for systems +with PyOpenSSL versions 0.13 or 0.14. +* Resolves issues with the Apache plugin enabling an HTTP to HTTPS +redirect on some systems. +* Improved error messages from the client. + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=is%3Aissue+milestone%3A0.2.0 + +# 0.1.1 +## 12/15/2015 + +* Fix a confusing UI path that caused some users to repeatedly renew +their certs while experimenting with the client, in some cases +hitting issuance rate limits +* Fixes numerous Apache configuration parser fixes +* Avoids attempting to issue for unqualified domain names like +"localhost" +* Fixes --webroot permission handling for non-root users + +More details about these changes can be found on our GitHub repo: +https://github.com/letsencrypt/letsencrypt/issues?q=milestone%3A0.1.1 diff --git a/Dockerfile-dev b/Dockerfile-dev index dbb45f75e..2a89b2ff5 100644 --- a/Dockerfile-dev +++ b/Dockerfile-dev @@ -13,7 +13,7 @@ EXPOSE 443 # authenticator and text mode only?) VOLUME /etc/letsencrypt /var/lib/letsencrypt -WORKDIR /opt/certbot +WORKDIR /opt/certbot/src # no need to mkdir anything: # https://docs.docker.com/reference/builder/#copy diff --git a/ISSUE_TEMPLATE.md b/ISSUE_TEMPLATE.md index e4e56f93d..e648a21b4 100644 --- a/ISSUE_TEMPLATE.md +++ b/ISSUE_TEMPLATE.md @@ -1,16 +1,14 @@ ## My operating system is (include version): -## My web server is (include version): +## I installed Certbot with (certbot-auto, OS package manager, pip, etc): -## How did you install Certbot: +## I ran this command and it produced this output: -## What command did you run and what output did it produce? - - -## Can you provide a Certbot error log showing the issue? -###### It is stored by default in `/var/log/letsencrypt` - feel free to redact domain names, e-mail and IP addresses as you see fit +## Certbot's behavior differed from what I expected because: +## Here is a Certbot log showing the issue (if available): +###### Logs are stored in `/var/log/letsencrypt` by default. Feel free to redact domains, e-mail and IP addresses as you see fit. diff --git a/README.rst b/README.rst index f986703ac..ab12562df 100644 --- a/README.rst +++ b/README.rst @@ -88,7 +88,7 @@ Main Website: https://certbot.eff.org Let's Encrypt Website: https://letsencrypt.org -IRC Channel: #letsencrypt on `Freenode`_ or #certbot on `OFTC`_ +IRC Channel: #letsencrypt on `Freenode`_ Community: https://community.letsencrypt.org diff --git a/acme/acme/challenges.py b/acme/acme/challenges.py index 83b9b9edd..ac4e3d60a 100644 --- a/acme/acme/challenges.py +++ b/acme/acme/challenges.py @@ -425,7 +425,7 @@ class TLSSNI01Response(KeyAuthorizationChallengeResponse): # TODO: domain is not necessary if host is provided if "host" not in kwargs: host = socket.gethostbyname(domain) - logging.debug('%s resolved to %s', domain, host) + logger.debug('%s resolved to %s', domain, host) kwargs["host"] = host kwargs.setdefault("port", self.PORT) @@ -445,7 +445,7 @@ class TLSSNI01Response(KeyAuthorizationChallengeResponse): """ # pylint: disable=protected-access sans = crypto_util._pyopenssl_cert_or_req_san(cert) - logging.debug('Certificate %s. SANs: %s', cert.digest('sha1'), sans) + logger.debug('Certificate %s. SANs: %s', cert.digest('sha256'), sans) return self.z_domain.decode() in sans def simple_verify(self, chall, domain, account_public_key, diff --git a/acme/acme/client.py b/acme/acme/client.py index 168574d58..6c5ed79a2 100644 --- a/acme/acme/client.py +++ b/acme/acme/client.py @@ -71,13 +71,20 @@ class Client(object): # pylint: disable=too-many-instance-attributes self.directory = directory @classmethod - def _regr_from_response(cls, response, uri=None, terms_of_service=None): + def _regr_from_response(cls, response, uri=None, new_authzr_uri=None, + terms_of_service=None): if 'terms-of-service' in response.links: terms_of_service = response.links['terms-of-service']['url'] + if 'next' in response.links: + new_authzr_uri = response.links['next']['url'] + + if new_authzr_uri is None: + raise errors.ClientError('"next" link missing') return messages.RegistrationResource( body=messages.Registration.from_json(response.json()), uri=response.headers.get('Location', uri), + new_authzr_uri=new_authzr_uri, terms_of_service=terms_of_service) def register(self, new_reg=None): @@ -110,7 +117,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes # (c.f. acme-spec #94) return self._regr_from_response( - response, uri=regr.uri, + response, uri=regr.uri, new_authzr_uri=regr.new_authzr_uri, terms_of_service=regr.terms_of_service) def update_registration(self, regr, update=None): @@ -127,8 +134,6 @@ class Client(object): # pylint: disable=too-many-instance-attributes update = regr.body if update is None else update body = messages.UpdateRegistration(**dict(update)) updated_regr = self._send_recv_regr(regr, body=body) - if updated_regr != regr: - raise errors.UnexpectedUpdate(regr) return updated_regr def deactivate_registration(self, regr): @@ -167,30 +172,43 @@ class Client(object): # pylint: disable=too-many-instance-attributes return self.update_registration( regr.update(body=regr.body.update(agreement=regr.terms_of_service))) - def _authzr_from_response(self, response, identifier, uri=None): + def _authzr_from_response(self, response, identifier, + uri=None, new_cert_uri=None): + # pylint: disable=no-self-use + if new_cert_uri is None: + try: + new_cert_uri = response.links['next']['url'] + except KeyError: + raise errors.ClientError('"next" link missing') + authzr = messages.AuthorizationResource( body=messages.Authorization.from_json(response.json()), - uri=response.headers.get('Location', uri)) + uri=response.headers.get('Location', uri), + new_cert_uri=new_cert_uri) if authzr.body.identifier != identifier: raise errors.UnexpectedUpdate(authzr) return authzr - def request_challenges(self, identifier): + def request_challenges(self, identifier, new_authzr_uri=None): """Request challenges. :param .messages.Identifier identifier: Identifier to be challenged. + :param str new_authzr_uri: ``new-authorization`` URI. If omitted, + will default to value found in ``directory``. :returns: Authorization Resource. :rtype: `.AuthorizationResource` """ new_authz = messages.NewAuthorization(identifier=identifier) - response = self.net.post(self.directory.new_authz, new_authz) + response = self.net.post(self.directory.new_authz + if new_authzr_uri is None else new_authzr_uri, + new_authz) # TODO: handle errors assert response.status_code == http_client.CREATED return self._authzr_from_response(response, identifier) - def request_domain_challenges(self, domain): + def request_domain_challenges(self, domain, new_authzr_uri=None): """Request challenges for domain names. This is simply a convenience function that wraps around @@ -205,7 +223,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes """ return self.request_challenges(messages.Identifier( - typ=messages.IDENTIFIER_FQDN, value=domain)) + typ=messages.IDENTIFIER_FQDN, value=domain), new_authzr_uri) def answer_challenge(self, challb, response): """Answer challenge. @@ -280,8 +298,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes """ response = self.net.get(authzr.uri) updated_authzr = self._authzr_from_response( - response, authzr.body.identifier, authzr.uri) - # TODO: check and raise UnexpectedUpdate + response, authzr.body.identifier, authzr.uri, authzr.new_cert_uri) return updated_authzr, response def request_issuance(self, csr, authzrs): @@ -304,7 +321,7 @@ class Client(object): # pylint: disable=too-many-instance-attributes content_type = DER_CONTENT_TYPE # TODO: add 'cert_type 'argument response = self.net.post( - self.directory.new_cert, + authzrs[0].new_cert_uri, # TODO: acme-spec #90 req, content_type=content_type, headers={'Accept': content_type}) @@ -600,10 +617,10 @@ class ClientNetwork(object): # pylint: disable=too-many-instance-attributes """ if method == "POST": - logging.debug('Sending POST request to %s:\n%s', + logger.debug('Sending POST request to %s:\n%s', url, kwargs['data']) else: - logging.debug('Sending %s request to %s.', method, url) + logger.debug('Sending %s request to %s.', method, url) kwargs['verify'] = self.verify_ssl kwargs.setdefault('headers', {}) kwargs['headers'].setdefault('User-Agent', self.user_agent) @@ -651,7 +668,7 @@ class ClientNetwork(object): # pylint: disable=too-many-instance-attributes def _get_nonce(self, url): if not self._nonces: - logging.debug('Requesting fresh nonce') + logger.debug('Requesting fresh nonce') self._add_nonce(self.head(url)) return self._nonces.pop() diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py index b7bd0740c..7e7ffe779 100644 --- a/acme/acme/client_test.py +++ b/acme/acme/client_test.py @@ -40,8 +40,6 @@ class ClientTest(unittest.TestCase): 'https://www.letsencrypt-demo.org/acme/revoke-cert', messages.NewAuthorization: 'https://www.letsencrypt-demo.org/acme/new-authz', - messages.CertificateRequest: - 'https://www.letsencrypt-demo.org/acme/new-cert', }) from acme.client import Client @@ -58,6 +56,7 @@ class ClientTest(unittest.TestCase): self.new_reg = messages.NewRegistration(**dict(reg)) self.regr = messages.RegistrationResource( body=reg, uri='https://www.letsencrypt-demo.org/acme/reg/1', + new_authzr_uri='https://www.letsencrypt-demo.org/acme/new-reg', terms_of_service='https://www.letsencrypt-demo.org/tos') # Authorization @@ -73,7 +72,8 @@ class ClientTest(unittest.TestCase): typ=messages.IDENTIFIER_FQDN, value='example.com'), challenges=(challb,), combinations=None) self.authzr = messages.AuthorizationResource( - body=self.authz, uri=authzr_uri) + body=self.authz, uri=authzr_uri, + new_cert_uri='https://www.letsencrypt-demo.org/acme/new-cert') # Request issuance self.certr = messages.CertificateResource( @@ -98,12 +98,18 @@ class ClientTest(unittest.TestCase): self.response.json.return_value = self.regr.body.to_json() self.response.headers['Location'] = self.regr.uri self.response.links.update({ + 'next': {'url': self.regr.new_authzr_uri}, 'terms-of-service': {'url': self.regr.terms_of_service}, }) self.assertEqual(self.regr, self.client.register(self.new_reg)) # TODO: test POST call arguments + def test_register_missing_next(self): + self.response.status_code = http_client.CREATED + self.assertRaises( + errors.ClientError, self.client.register, self.new_reg) + def test_update_registration(self): # "Instance of 'Field' has no to_json/update member" bug: # pylint: disable=no-member @@ -115,8 +121,6 @@ class ClientTest(unittest.TestCase): # TODO: split here and separate test self.response.json.return_value = self.regr.body.update( contact=()).to_json() - self.assertRaises( - errors.UnexpectedUpdate, self.client.update_registration, self.regr) def test_deactivate_account(self): self.response.headers['Location'] = self.regr.uri @@ -124,18 +128,17 @@ class ClientTest(unittest.TestCase): self.assertEqual(self.regr, self.client.deactivate_registration(self.regr)) - def test_deactivate_account_bad_registration_returned(self): - self.response.headers['Location'] = self.regr.uri - self.response.json.return_value = "some wrong registration thing" - self.assertRaises( - errors.UnexpectedUpdate, - self.client.deactivate_registration, - self.regr) - def test_query_registration(self): self.response.json.return_value = self.regr.body.to_json() self.assertEqual(self.regr, self.client.query_registration(self.regr)) + def test_query_registration_updates_new_authzr_uri(self): + self.response.json.return_value = self.regr.body.to_json() + self.response.links = {'next': {'url': 'UPDATED'}} + self.assertEqual( + 'UPDATED', + self.client.query_registration(self.regr).new_authzr_uri) + def test_agree_to_tos(self): self.client.update_registration = mock.Mock() self.client.agree_to_tos(self.regr) @@ -146,6 +149,9 @@ class ClientTest(unittest.TestCase): self.response.status_code = http_client.CREATED self.response.headers['Location'] = self.authzr.uri self.response.json.return_value = self.authz.to_json() + self.response.links = { + 'next': {'url': self.authzr.new_cert_uri}, + } def test_request_challenges(self): self._prepare_response_for_request_challenges() @@ -156,9 +162,8 @@ class ClientTest(unittest.TestCase): def test_request_challenges_custom_uri(self): self._prepare_response_for_request_challenges() - self.client.request_challenges(self.identifier) - self.net.post.assert_called_once_with( - 'https://www.letsencrypt-demo.org/acme/new-authz', mock.ANY) + self.client.request_challenges(self.identifier, 'URI') + self.net.post.assert_called_once_with('URI', mock.ANY) def test_request_challenges_unexpected_update(self): self._prepare_response_for_request_challenges() @@ -166,7 +171,12 @@ class ClientTest(unittest.TestCase): identifier=self.identifier.update(value='foo')).to_json() self.assertRaises( errors.UnexpectedUpdate, self.client.request_challenges, - self.identifier) + self.identifier, self.authzr.uri) + + def test_request_challenges_missing_next(self): + self.response.status_code = http_client.CREATED + self.assertRaises(errors.ClientError, self.client.request_challenges, + self.identifier) def test_request_domain_challenges(self): self.client.request_challenges = mock.MagicMock() @@ -174,6 +184,12 @@ class ClientTest(unittest.TestCase): self.client.request_challenges(self.identifier), self.client.request_domain_challenges('example.com')) + def test_request_domain_challenges_custom_uri(self): + self.client.request_challenges = mock.MagicMock() + self.assertEqual( + self.client.request_challenges(self.identifier, 'URI'), + self.client.request_domain_challenges('example.com', 'URI')) + def test_answer_challenge(self): self.response.links['up'] = {'url': self.challr.authzr_uri} self.response.json.return_value = self.challr.body.to_json() @@ -544,7 +560,7 @@ class ClientNetworkTest(unittest.TestCase): # pylint: disable=protected-access self.net._send_request('HEAD', 'http://example.com/', 'foo', timeout=mock.ANY, bar='baz') - mock_logger.debug.assert_called_once_with( + mock_logger.debug.assert_called_with( 'Received response:\nHTTP %d\n%s\n\n%s', 200, 'Content-Type: application/pkix-cert', b'aGk=') diff --git a/acme/acme/jose/json_util.py b/acme/acme/jose/json_util.py index 4baadda5e..d474f4aac 100644 --- a/acme/acme/jose/json_util.py +++ b/acme/acme/jose/json_util.py @@ -267,7 +267,7 @@ class JSONObjectWithFields(util.ImmutableMap, interfaces.JSONDeSerializable): if missing: raise errors.DeserializationError( - 'The following fields are required: {0}'.format( + 'The following field are required: {0}'.format( ','.join(missing))) @classmethod diff --git a/acme/acme/messages.py b/acme/acme/messages.py index c3df4998c..54cd25c94 100644 --- a/acme/acme/messages.py +++ b/acme/acme/messages.py @@ -191,7 +191,7 @@ class Directory(jose.JSONDeSerializable): try: return self[name.replace('_', '-')] except KeyError as error: - raise AttributeError(str(error) + ': ' + name) + raise AttributeError(str(error)) def __getitem__(self, name): try: @@ -315,10 +315,12 @@ class RegistrationResource(ResourceWithURI): """Registration Resource. :ivar acme.messages.Registration body: + :ivar unicode new_authzr_uri: URI found in the 'next' ``Link`` header :ivar unicode terms_of_service: URL for the CA TOS. """ body = jose.Field('body', decoder=Registration.from_json) + new_authzr_uri = jose.Field('new_authzr_uri') terms_of_service = jose.Field('terms_of_service', omitempty=True) @@ -423,9 +425,11 @@ class AuthorizationResource(ResourceWithURI): """Authorization Resource. :ivar acme.messages.Authorization body: + :ivar unicode new_cert_uri: URI found in the 'next' ``Link`` header """ body = jose.Field('body', decoder=Authorization.from_json) + new_cert_uri = jose.Field('new_cert_uri') @Directory.register diff --git a/acme/acme/messages_test.py b/acme/acme/messages_test.py index e84c3e992..b3454f25b 100644 --- a/acme/acme/messages_test.py +++ b/acme/acme/messages_test.py @@ -225,12 +225,14 @@ class RegistrationResourceTest(unittest.TestCase): from acme.messages import RegistrationResource self.regr = RegistrationResource( body=mock.sentinel.body, uri=mock.sentinel.uri, + new_authzr_uri=mock.sentinel.new_authzr_uri, terms_of_service=mock.sentinel.terms_of_service) def test_to_partial_json(self): self.assertEqual(self.regr.to_json(), { 'body': mock.sentinel.body, 'uri': mock.sentinel.uri, + 'new_authzr_uri': mock.sentinel.new_authzr_uri, 'terms_of_service': mock.sentinel.terms_of_service, }) @@ -344,7 +346,9 @@ class AuthorizationResourceTest(unittest.TestCase): from acme.messages import AuthorizationResource authzr = AuthorizationResource( uri=mock.sentinel.uri, - body=mock.sentinel.body) + body=mock.sentinel.body, + new_cert_uri=mock.sentinel.new_cert_uri, + ) self.assertTrue(isinstance(authzr, jose.JSONDeSerializable)) diff --git a/acme/examples/example_client.py b/acme/examples/example_client.py index 1386491b1..261b37603 100644 --- a/acme/examples/example_client.py +++ b/acme/examples/example_client.py @@ -32,7 +32,8 @@ acme.agree_to_tos(regr) logging.debug(regr) authzr = acme.request_challenges( - identifier=messages.Identifier(typ=messages.IDENTIFIER_FQDN, value=DOMAIN)) + identifier=messages.Identifier(typ=messages.IDENTIFIER_FQDN, value=DOMAIN), + new_authzr_uri=regr.new_authzr_uri) logging.debug(authzr) authzr, authzr_response = acme.poll(authzr) diff --git a/acme/setup.py b/acme/setup.py index d1e91e5ec..df9937032 100644 --- a/acme/setup.py +++ b/acme/setup.py @@ -4,35 +4,30 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.13.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ + 'argparse', # load_pem_private/public_key (>=0.6) # rsa_recover_prime_factors (>=0.8) 'cryptography>=0.8', # Connection.set_tlsext_host_name (>=0.13) + 'mock', 'PyOpenSSL>=0.13', 'pyrfc3339', 'pytz', - 'requests[security]>=2.4.1', # security extras added in 2.4.1 + # requests>=2.10 is required to fix + # https://github.com/shazow/urllib3/issues/556. This requirement can be + # relaxed to 'requests[security]>=2.4.1', however, less useful errors + # will be raised for some network/SSL errors. + 'requests[security]>=2.10', # For pkg_resources. >=1.0 so pip resolves it to a version cryptography # will tolerate; see #2599: 'setuptools>=1.0', 'six', ] -# env markers in extras_require cause problems with older pip: #517 -# Keep in sync with conditional_requirements.py. -if sys.version_info < (2, 7): - install_requires.extend([ - # only some distros recognize stdlib argparse as already satisfying - 'argparse', - 'mock<1.1.0', - ]) -else: - install_requires.append('mock') - dev_extras = [ 'nose', 'tox', diff --git a/certbot-apache/certbot_apache/configurator.py b/certbot-apache/certbot_apache/configurator.py index 5639dae83..cdfc01626 100644 --- a/certbot-apache/certbot_apache/configurator.py +++ b/certbot-apache/certbot_apache/configurator.py @@ -1315,18 +1315,15 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator): # even with save() and load() if not self._is_rewrite_engine_on(general_vh): self.parser.add_dir(general_vh.path, "RewriteEngine", "on") + names = ssl_vhost.get_names() for idx, name in enumerate(names): args = ["%{SERVER_NAME}", "={0}".format(name), "[OR]"] if idx == len(names) - 1: args.pop() self.parser.add_dir(general_vh.path, "RewriteCond", args) - if self.get_version() >= (2, 3, 9): - self.parser.add_dir(general_vh.path, "RewriteRule", - constants.REWRITE_HTTPS_ARGS_WITH_END) - else: - self.parser.add_dir(general_vh.path, "RewriteRule", - constants.REWRITE_HTTPS_ARGS) + + self._set_https_redirection_rewrite_rule(general_vh) self.save_notes += ("Redirecting host in %s to ssl vhost in %s\n" % (general_vh.filep, ssl_vhost.filep)) @@ -1336,12 +1333,24 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator): logger.info("Redirecting vhost in %s to ssl vhost in %s", general_vh.filep, ssl_vhost.filep) + def _set_https_redirection_rewrite_rule(self, vhost): + if self.get_version() >= (2, 3, 9): + self.parser.add_dir(vhost.path, "RewriteRule", + constants.REWRITE_HTTPS_ARGS_WITH_END) + else: + self.parser.add_dir(vhost.path, "RewriteRule", + constants.REWRITE_HTTPS_ARGS) + + def _verify_no_certbot_redirect(self, vhost): """Checks to see if a redirect was already installed by certbot. Checks to see if virtualhost already contains a rewrite rule that is identical to Certbot's redirection rewrite rule. + For graceful transition to new rewrite rules for HTTPS redireciton we + delete certbot's old rewrite rules and set the new one instead. + :param vhost: vhost to check :type vhost: :class:`~certbot_apache.obj.VirtualHost` @@ -1355,19 +1364,29 @@ class ApacheConfigurator(augeas_configurator.AugeasConfigurator): # rewrite_args_dict keys are directive ids and the corresponding value # for each is a list of arguments to that directive. rewrite_args_dict = defaultdict(list) - pat = r'.*(directive\[\d+\]).*' + pat = r'(.*directive\[\d+\]).*' for match in rewrite_path: m = re.match(pat, match) if m: - dir_id = m.group(1) - rewrite_args_dict[dir_id].append(match) + dir_path = m.group(1) + rewrite_args_dict[dir_path].append(match) if rewrite_args_dict: redirect_args = [constants.REWRITE_HTTPS_ARGS, constants.REWRITE_HTTPS_ARGS_WITH_END] - for matches in rewrite_args_dict.values(): - if [self.aug.get(x) for x in matches] in redirect_args: + for dir_path, args_paths in rewrite_args_dict.items(): + arg_vals = [self.aug.get(x) for x in args_paths] + + # Search for past redirection rule, delete it, set the new one + if arg_vals in constants.OLD_REWRITE_HTTPS_ARGS: + self.aug.remove(dir_path) + self._set_https_redirection_rewrite_rule(vhost) + self.save() + raise errors.PluginEnhancementAlreadyPresent( + "Certbot has already enabled redirection") + + if arg_vals in redirect_args: raise errors.PluginEnhancementAlreadyPresent( "Certbot has already enabled redirection") diff --git a/certbot-apache/certbot_apache/constants.py b/certbot-apache/certbot_apache/constants.py index dcc635c4b..3cfeb4dd6 100644 --- a/certbot-apache/certbot_apache/constants.py +++ b/certbot-apache/certbot_apache/constants.py @@ -136,15 +136,19 @@ AUGEAS_LENS_DIR = pkg_resources.resource_filename( """Path to the Augeas lens directory""" REWRITE_HTTPS_ARGS = [ - "^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[L,QSA,R=permanent]"] + "^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[L,NE,R=permanent]"] """Apache version<2.3.9 rewrite rule arguments used for redirections to https vhost""" REWRITE_HTTPS_ARGS_WITH_END = [ - "^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[END,QSA,R=permanent]"] + "^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[END,NE,R=permanent]"] """Apache version >= 2.3.9 rewrite rule arguments used for redirections to https vhost""" +OLD_REWRITE_HTTPS_ARGS = [ + ["^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[L,QSA,R=permanent]"], + ["^", "https://%{SERVER_NAME}%{REQUEST_URI}", "[END,QSA,R=permanent]"]] + HSTS_ARGS = ["always", "set", "Strict-Transport-Security", "\"max-age=31536000\""] """Apache header arguments for HSTS""" diff --git a/certbot-apache/certbot_apache/tests/configurator_test.py b/certbot-apache/certbot_apache/tests/configurator_test.py index 937694267..45e701bd5 100644 --- a/certbot-apache/certbot_apache/tests/configurator_test.py +++ b/certbot-apache/certbot_apache/tests/configurator_test.py @@ -18,6 +18,7 @@ from certbot.tests import acme_util from certbot.tests import util as certbot_util from certbot_apache import configurator +from certbot_apache import constants from certbot_apache import parser from certbot_apache import obj @@ -1047,6 +1048,36 @@ class MultipleVhostsTest(util.ApacheTest): self.assertTrue("rewrite_module" in self.config.parser.modules) + @mock.patch("certbot.util.run_script") + @mock.patch("certbot.util.exe_exists") + def test_redirect_with_old_https_redirection(self, mock_exe, _): + self.config.parser.update_runtime_variables = mock.Mock() + mock_exe.return_value = True + self.config.get_version = mock.Mock(return_value=(2, 2, 0)) + + ssl_vhost = self.config.choose_vhost("certbot.demo") + + # pylint: disable=protected-access + http_vhost = self.config._get_http_vhost(ssl_vhost) + + # Create an old (previously suppoorted) https redirectoin rewrite rule + self.config.parser.add_dir( + http_vhost.path, "RewriteRule", + ["^", + "https://%{SERVER_NAME}%{REQUEST_URI}", + "[L,QSA,R=permanent]"]) + + self.config.save() + + try: + self.config.enhance("certbot.demo", "redirect") + except errors.PluginEnhancementAlreadyPresent: + args_paths = self.config.parser.find_dir( + "RewriteRule", None, http_vhost.path, False) + arg_vals = [self.config.aug.get(x) for x in args_paths] + self.assertEqual(arg_vals, constants.REWRITE_HTTPS_ARGS) + + def test_redirect_with_conflict(self): self.config.parser.modules.add("rewrite_module") ssl_vh = obj.VirtualHost( @@ -1134,7 +1165,7 @@ class MultipleVhostsTest(util.ApacheTest): http_vhost.path, "RewriteRule", ["^", "https://%{SERVER_NAME}%{REQUEST_URI}", - "[L,QSA,R=permanent]"]) + "[L,NE,R=permanent]"]) self.config.save() ssl_vhost = self.config.make_vhost_ssl(self.vh_truth[0]) @@ -1145,7 +1176,7 @@ class MultipleVhostsTest(util.ApacheTest): conf_text = open(ssl_vhost.filep).read() commented_rewrite_rule = ("# RewriteRule ^ " "https://%{SERVER_NAME}%{REQUEST_URI} " - "[L,QSA,R=permanent]") + "[L,NE,R=permanent]") self.assertTrue(commented_rewrite_rule in conf_text) mock_get_utility().add_message.assert_called_once_with(mock.ANY, @@ -1164,7 +1195,7 @@ class MultipleVhostsTest(util.ApacheTest): "RewriteCond", ["%{DOCUMENT_ROOT}/%{REQUEST_FILENAME}", "!-f"]) self.config.parser.add_dir( http_vhost.path, "RewriteRule", - ["^(.*)$", "b://u%{REQUEST_URI}", "[P,QSA,L]"]) + ["^(.*)$", "b://u%{REQUEST_URI}", "[P,NE,L]"]) # Add a chunk that should be commented out. self.config.parser.add_dir(http_vhost.path, @@ -1175,7 +1206,7 @@ class MultipleVhostsTest(util.ApacheTest): http_vhost.path, "RewriteRule", ["^", "https://%{SERVER_NAME}%{REQUEST_URI}", - "[L,QSA,R=permanent]"]) + "[L,NE,R=permanent]"]) self.config.save() @@ -1186,13 +1217,13 @@ class MultipleVhostsTest(util.ApacheTest): not_commented_cond1 = ("RewriteCond " "%{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f") not_commented_rewrite_rule = ("RewriteRule " - "^(.*)$ b://u%{REQUEST_URI} [P,QSA,L]") + "^(.*)$ b://u%{REQUEST_URI} [P,NE,L]") commented_cond1 = "# RewriteCond %{HTTPS} !=on" commented_cond2 = "# RewriteCond %{HTTPS} !^$" commented_rewrite_rule = ("# RewriteRule ^ " "https://%{SERVER_NAME}%{REQUEST_URI} " - "[L,QSA,R=permanent]") + "[L,NE,R=permanent]") self.assertTrue(not_commented_cond1 in conf_line_set) self.assertTrue(not_commented_rewrite_rule in conf_line_set) diff --git a/certbot-apache/setup.py b/certbot-apache/setup.py index 87ea1a281..db8cb11db 100644 --- a/certbot-apache/setup.py +++ b/certbot-apache/setup.py @@ -4,12 +4,13 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.13.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ 'acme=={0}'.format(version), 'certbot=={0}'.format(version), + 'mock', 'python-augeas', # For pkg_resources. >=1.0 so pip resolves it to a version cryptography # will tolerate; see #2599: @@ -18,11 +19,6 @@ install_requires = [ 'zope.interface', ] -if sys.version_info < (2, 7): - install_requires.append('mock<1.1.0') -else: - install_requires.append('mock') - docs_extras = [ 'Sphinx>=1.0', # autodoc_member_order = 'bysource', autodoc_default_flags 'sphinx_rtd_theme', diff --git a/certbot-auto b/certbot-auto index ac721638f..54cc429cf 100755 --- a/certbot-auto +++ b/certbot-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.11.1" +LE_AUTO_VERSION="0.12.0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.1 \ - --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ - --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e -certbot==0.11.1 \ - --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ - --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 -certbot-apache==0.11.1 \ - --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ - --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 -certbot-nginx==0.11.1 \ - --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ - --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 +acme==0.12.0 \ + --hash=sha256:a6050619b3e07b41d197992bb15b32c755dfa0665cfa1c20faa82806a798265b \ + --hash=sha256:a05cba6b5b0fffdfa246b32492a44769011d45205f3ee8efde1f37ee9843fbdf +certbot==0.12.0 \ + --hash=sha256:d018d13665eb4cfe7038c2df636e3f4928742b83769b95edfdb0311277f0eb48 \ + --hash=sha256:4a71925c035b62dfb7c3343c619ee090add76188b47225272b57798ad63388b7 +certbot-apache==0.12.0 \ + --hash=sha256:de86907ea60e7bc35d252b87dec04eab3c7f3a1ea768774876e7ff582d89d640 \ + --hash=sha256:77dde63cf97292b09da8ae09ef8a7a6d83a3b1ee0f8d1fefe513fc77a6292509 +certbot-nginx==0.12.0 \ + --hash=sha256:c66d848c4577f1f91a06a8119b40f1ab90af1546addea27905434bd070f3924d \ + --hash=sha256:4dab2c93304c80d8d0d2e5214939f016804fd46859dd7a39b892d8b7195ab5ec UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/certbot-compatibility-test/setup.py b/certbot-compatibility-test/setup.py index e2d226a72..238d7a2d5 100644 --- a/certbot-compatibility-test/setup.py +++ b/certbot-compatibility-test/setup.py @@ -4,21 +4,17 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.13.0.dev0' install_requires = [ 'certbot', 'certbot-apache', + 'mock', 'six', 'requests', 'zope.interface', ] -if sys.version_info < (2, 7): - install_requires.append('mock<1.1.0') -else: - install_requires.append('mock') - if sys.version_info < (2, 7, 9): # For secure SSL connexion with Python 2.7 (InsecurePlatformWarning) install_requires.append('ndg-httpsclient') diff --git a/certbot-nginx/certbot_nginx/configurator.py b/certbot-nginx/certbot_nginx/configurator.py index 6d51ca641..7348def2f 100644 --- a/certbot-nginx/certbot_nginx/configurator.py +++ b/certbot-nginx/certbot_nginx/configurator.py @@ -630,7 +630,7 @@ class NginxConfigurator(common.Plugin): stderr=subprocess.PIPE) text = proc.communicate()[1] # nginx prints output to stderr except (OSError, ValueError) as error: - logging.debug(error, exc_info=True) + logger.debug(error, exc_info=True) raise errors.PluginError( "Unable to run %s -V" % self.conf('ctl')) diff --git a/certbot-nginx/certbot_nginx/parser.py b/certbot-nginx/certbot_nginx/parser.py index c586aa459..eddc7b9b0 100644 --- a/certbot-nginx/certbot_nginx/parser.py +++ b/certbot-nginx/certbot_nginx/parser.py @@ -586,9 +586,10 @@ def _parse_server_raw(server): continue if directive[0] == 'listen': addr = obj.Addr.fromstring(directive[1]) - parsed_server['addrs'].add(addr) - if addr.ssl: - parsed_server['ssl'] = True + if addr: + parsed_server['addrs'].add(addr) + if addr.ssl: + parsed_server['ssl'] = True elif directive[0] == 'server_name': parsed_server['names'].update( _get_servernames(directive[1])) diff --git a/certbot-nginx/certbot_nginx/tests/parser_test.py b/certbot-nginx/certbot_nginx/tests/parser_test.py index 921cc3c5a..6a3f2f1de 100644 --- a/certbot-nginx/certbot_nginx/tests/parser_test.py +++ b/certbot-nginx/certbot_nginx/tests/parser_test.py @@ -323,6 +323,12 @@ class NginxParserTest(util.NginxTest): ]) self.assertTrue(server['ssl']) + def test_parse_server_raw_unix(self): + server = parser._parse_server_raw([ #pylint: disable=protected-access + ['listen', 'unix:/var/run/nginx.sock'] + ]) + self.assertEqual(len(server['addrs']), 0) + def test_parse_server_global_ssl_applied(self): nparser = parser.NginxParser(self.config_path, self.ssl_options) server = nparser.parse_server([ diff --git a/certbot-nginx/setup.py b/certbot-nginx/setup.py index 6de2dc6bd..bdc45b9b4 100644 --- a/certbot-nginx/setup.py +++ b/certbot-nginx/setup.py @@ -4,12 +4,13 @@ from setuptools import setup from setuptools import find_packages -version = '0.12.0.dev0' +version = '0.13.0.dev0' # Please update tox.ini when modifying dependency version requirements install_requires = [ 'acme=={0}'.format(version), 'certbot=={0}'.format(version), + 'mock', 'PyOpenSSL', 'pyparsing>=1.5.5', # Python3 support; perhaps unnecessary? # For pkg_resources. >=1.0 so pip resolves it to a version cryptography @@ -18,11 +19,6 @@ install_requires = [ 'zope.interface', ] -if sys.version_info < (2, 7): - install_requires.append('mock<1.1.0') -else: - install_requires.append('mock') - docs_extras = [ 'Sphinx>=1.0', # autodoc_member_order = 'bysource', autodoc_default_flags 'sphinx_rtd_theme', diff --git a/certbot/__init__.py b/certbot/__init__.py index 6451eb0d5..0c667378d 100644 --- a/certbot/__init__.py +++ b/certbot/__init__.py @@ -1,4 +1,4 @@ """Certbot client.""" # version number like 1.2.3a0, must have at least 2 parts, like 1.2 -__version__ = '0.12.0.dev0' +__version__ = '0.13.0.dev0' diff --git a/certbot/auth_handler.py b/certbot/auth_handler.py index 53346a77c..6e9ab25a7 100644 --- a/certbot/auth_handler.py +++ b/certbot/auth_handler.py @@ -63,7 +63,8 @@ class AuthHandler(object): """ for domain in domains: - self.authzr[domain] = self.acme.request_domain_challenges(domain) + self.authzr[domain] = self.acme.request_domain_challenges( + domain, self.account.regr.new_authzr_uri) self._choose_challenges(domains) diff --git a/certbot/client.py b/certbot/client.py index 95882a9fc..a342c1bf3 100644 --- a/certbot/client.py +++ b/certbot/client.py @@ -155,8 +155,6 @@ def perform_registration(acme, config): :returns: Registration Resource. :rtype: `acme.messages.RegistrationResource` - - :raises .UnexpectedUpdate: """ try: return acme.register(messages.NewRegistration.from_data(email=config.email)) diff --git a/certbot/hooks.py b/certbot/hooks.py index 5cda478cc..ada3d3aaa 100644 --- a/certbot/hooks.py +++ b/certbot/hooks.py @@ -44,8 +44,12 @@ def validate_hook(shell_cmd, hook_name): cmd = shell_cmd.split(None, 1)[0] if not _prog(cmd): path = os.environ["PATH"] - msg = "Unable to find {2}-hook command {0} in the PATH.\n(PATH is {1})".format( - cmd, path, hook_name) + if os.path.exists(cmd): + msg = "{1}-hook command {0} exists, but is not executable.".format(cmd, hook_name) + else: + msg = "Unable to find {2}-hook command {0} in the PATH.\n(PATH is {1})".format( + cmd, path, hook_name) + raise errors.HookCommandNotFound(msg) def pre_hook(config): diff --git a/certbot/ocsp.py b/certbot/ocsp.py index 8921dbb88..d34110f88 100644 --- a/certbot/ocsp.py +++ b/certbot/ocsp.py @@ -16,7 +16,7 @@ class RevocationChecker(object): self.broken = False if not util.exe_exists("openssl"): - logging.info("openssl not installed, can't check revocation") + logger.info("openssl not installed, can't check revocation") self.broken = True return @@ -61,7 +61,7 @@ class RevocationChecker(object): logger.debug("Querying OCSP for %s", cert_path) logger.debug(" ".join(cmd)) try: - output, err = util.run_script(cmd, log=logging.debug) + output, err = util.run_script(cmd, log=logger.debug) except errors.SubprocessError: logger.info("OCSP check failed for %s (are we offline?)", cert_path) return False @@ -80,7 +80,7 @@ class RevocationChecker(object): try: url, _err = util.run_script( ["openssl", "x509", "-in", cert_path, "-noout", "-ocsp_uri"], - log=logging.debug) + log=logger.debug) except errors.SubprocessError: logger.info("Cannot extract OCSP URI from %s", cert_path) return None, None diff --git a/certbot/plugins/util.py b/certbot/plugins/util.py index e45c26735..f0e2f4c5b 100644 --- a/certbot/plugins/util.py +++ b/certbot/plugins/util.py @@ -33,6 +33,6 @@ def path_surgery(cmd): return True else: expanded = " expanded" if any(added) else "" - logger.warning("Failed to find %s in%s PATH: %s", cmd, + logger.warning("Failed to find executable %s in%s PATH: %s", cmd, expanded, path) return False diff --git a/certbot/tests/account_test.py b/certbot/tests/account_test.py index 7367717bf..8ed591c98 100644 --- a/certbot/tests/account_test.py +++ b/certbot/tests/account_test.py @@ -110,7 +110,7 @@ class AccountFileStorageTest(unittest.TestCase): from certbot.account import Account self.acc = Account( regr=messages.RegistrationResource( - uri=None, body=messages.Registration()), + uri=None, new_authzr_uri=None, body=messages.Registration()), key=KEY) def tearDown(self): diff --git a/certbot/tests/acme_util.py b/certbot/tests/acme_util.py index f0549666a..5e6b190a7 100644 --- a/certbot/tests/acme_util.py +++ b/certbot/tests/acme_util.py @@ -96,5 +96,6 @@ def gen_authzr(authz_status, domain, challs, statuses, combos=True): # pylint: disable=star-args return messages.AuthorizationResource( uri="https://trusted.ca/new-authz-resource", + new_cert_uri="https://trusted.ca/new-cert", body=messages.Authorization(**authz_kwargs) ) diff --git a/certbot/tests/auth_handler_test.py b/certbot/tests/auth_handler_test.py index 9d22843db..046eb5ef1 100644 --- a/certbot/tests/auth_handler_test.py +++ b/certbot/tests/auth_handler_test.py @@ -309,6 +309,7 @@ class PollChallengesTest(unittest.TestCase): new_authzr = messages.AuthorizationResource( uri=authzr.uri, + new_cert_uri=authzr.new_cert_uri, body=messages.Authorization( identifier=authzr.body.identifier, challenges=new_challbs, @@ -436,7 +437,7 @@ def gen_auth_resp(chall_list): for chall in chall_list] -def gen_dom_authzr(domain, challs, combos=True): +def gen_dom_authzr(domain, unused_new_authzr_uri, challs, combos=True): """Generates new authzr for domains.""" return acme_util.gen_authzr( messages.STATUS_PENDING, domain, challs, diff --git a/certbot/tests/crypto_util_test.py b/certbot/tests/crypto_util_test.py index a580574a4..946e772c1 100644 --- a/certbot/tests/crypto_util_test.py +++ b/certbot/tests/crypto_util_test.py @@ -336,8 +336,8 @@ class CertLoaderTest(unittest.TestCase): from certbot.crypto_util import pyopenssl_load_certificate cert, file_type = pyopenssl_load_certificate(CERT) - self.assertEqual(cert.digest('sha1'), - OpenSSL.crypto.load_certificate(file_type, CERT).digest('sha1')) + self.assertEqual(cert.digest('sha256'), + OpenSSL.crypto.load_certificate(file_type, CERT).digest('sha256')) def test_load_invalid_cert(self): from certbot.crypto_util import pyopenssl_load_certificate diff --git a/certbot/tests/display/ops_test.py b/certbot/tests/display/ops_test.py index f2a9b3d07..f6de33a92 100644 --- a/certbot/tests/display/ops_test.py +++ b/certbot/tests/display/ops_test.py @@ -104,10 +104,10 @@ class ChooseAccountTest(unittest.TestCase): self.key = KEY self.acc1 = account.Account(messages.RegistrationResource( - uri=None, body=messages.Registration.from_data( + uri=None, new_authzr_uri=None, body=messages.Registration.from_data( email="email1@g.com")), self.key) self.acc2 = account.Account(messages.RegistrationResource( - uri=None, body=messages.Registration.from_data( + uri=None, new_authzr_uri=None, body=messages.Registration.from_data( email="email2@g.com", phone="phone")), self.key) @classmethod diff --git a/certbot/tests/ocsp_test.py b/certbot/tests/ocsp_test.py index 549e83ca8..91dd6f8d6 100644 --- a/certbot/tests/ocsp_test.py +++ b/certbot/tests/ocsp_test.py @@ -28,7 +28,7 @@ class OCSPTest(unittest.TestCase): def tearDown(self): pass - @mock.patch('certbot.ocsp.logging.info') + @mock.patch('certbot.ocsp.logger.info') @mock.patch('certbot.ocsp.Popen') @mock.patch('certbot.util.exe_exists') def test_init(self, mock_exists, mock_popen, mock_log): diff --git a/docs/ciphers.rst b/docs/ciphers.rst index 31ce45963..1b320cdf9 100644 --- a/docs/ciphers.rst +++ b/docs/ciphers.rst @@ -255,7 +255,7 @@ I have access to an English-language summary of the recommendations. Keylength.com ~~~~~~~~~~~~~ -Damien Giry collects recommendations by academic researchers and standards organizations about keylengths for particular cryptoperiods, years, or security levels. The keylength recommendations of the various sources are summarized in a chart. This site has been updated over time and includes expert guidance from eight sources published between 2000 and 2015. +Damien Giry collects recommendations by academic researchers and standards organizations about keylengths for particular cryptoperiods, years, or security levels. The keylength recommendations of the various sources are summarized in a chart. This site has been updated over time and includes expert guidance from eight sources published between 2000 and 2017. http://www.keylength.com/ diff --git a/docs/cli-help.txt b/docs/cli-help.txt index 4cb408002..9ef9d9e6c 100644 --- a/docs/cli-help.txt +++ b/docs/cli-help.txt @@ -86,7 +86,7 @@ optional arguments: statistics about success rates by OS and plugin. If you wish to hide your server OS version from the Let's Encrypt server, set this to "". (default: - CertbotACMEClient/0.11.1 (Ubuntu 16.04.1 LTS) + CertbotACMEClient/0.12.0 (Ubuntu 16.04.2 LTS) Authenticator/XXX Installer/YYY) automation: diff --git a/docs/contributing.rst b/docs/contributing.rst index 5f939e947..5cdf86147 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -14,15 +14,24 @@ Getting Started Running a local copy of the client ---------------------------------- -Running the client in developer mode from your local tree is a little -different than running ``certbot-auto``. To get set up, do these things -once: +Running the client in developer mode from your local tree is a little different +than running Certbot as a user. To get set up, clone our git repository by +running: .. code-block:: shell git clone https://github.com/certbot/certbot + +If you're on macOS, we recommend you skip the rest of this section and instead +run Certbot in Docker. You can find instructions for how to do this :ref:`here +`. If you're running on Linux, you can run the following commands to +install dependencies and set up a virtual environment where you can run +Certbot. You only need to do this once. + +.. code-block:: shell + cd certbot - ./letsencrypt-auto-source/letsencrypt-auto --os-packages-only + ./certbot-auto --os-packages-only ./tools/venv.sh Then in each shell where you're working on the client, do: @@ -69,10 +78,8 @@ either in the same directory as ``foo.py`` or in the ``tests`` subdirectory (if there isn't, make one). While you are working on your code and tests, run ``python foo_test.py`` to run the relevant tests. -For debugging, we recommend running ``pip install ipdb`` and putting -``import ipdb; ipdb.set_trace()`` statements inside the source -code. Alternatively, you can use Python's standard library `pdb`, -but you won't get TAB completion. +For debugging, we recommend putting +``import ipdb; ipdb.set_trace()`` statements inside the source code. Once you are done with your code changes, and the tests in ``foo_test.py`` pass, run all of the unittests for Certbot with ``tox -e py27`` (this uses Python @@ -285,8 +292,7 @@ Steps: including coverage. The ``--skip-missing-interpreters`` argument ignores missing versions of Python needed for running the tests. Fix any errors. 5. If your code touches communication with an ACME server/Boulder, you - should run the integration tests, see `integration`_. See `Known Issues`_ - for some common failures that have nothing to do with your code. + should run the integration tests, see `integration`_. 6. Submit the PR. 7. Did your tests pass on Travis? If they didn't, fix any errors. @@ -346,56 +352,36 @@ This should generate documentation in the ``docs/_build/html`` directory. -Other methods for running the client -==================================== +.. _docker: -Vagrant -------- +Running the client with Docker +============================== -If you are a Vagrant user, Certbot comes with a Vagrantfile that -automates setting up a development environment in an Ubuntu 14.04 -LTS VM. To set it up, simply run ``vagrant up``. The repository is -synced to ``/vagrant``, so you can get started with: +You can use Docker Compose to quickly set up an environment for running and +testing Certbot. This is especially useful for macOS users. To install Docker +Compose, follow the instructions at https://docs.docker.com/compose/install/. -.. code-block:: shell +.. note:: Linux users can simply run ``pip install docker-compose`` to get + Docker Compose after installing Docker Engine and activating your shell as + described in the :ref:`Getting Started ` section. - vagrant ssh - cd /vagrant - sudo ./venv/bin/certbot +Now you can develop on your host machine, but run Certbot and test your changes +in Docker. When using ``docker-compose`` make sure you are inside your clone of +the Certbot repository. As an example, you can run the following command to +check for linting errors:: -Support for other Linux distributions coming soon. + docker-compose run --rm --service-ports development bash -c 'tox -e lint' -.. note:: - Unfortunately, Python distutils and, by extension, setup.py and - tox, use hard linking quite extensively. Hard linking is not - supported by the default sync filesystem in Vagrant. As a result, - all actions with these commands are *significantly slower* in - Vagrant. One potential fix is to `use NFS`_ (`related issue`_). +You can also leave a terminal open running a shell in the Docker container and +modify Certbot code in another window. The Certbot repo on your host machine is +mounted inside of the container so any changes you make immediately take +effect. To do this, run:: -.. _use NFS: http://docs.vagrantup.com/v2/synced-folders/nfs.html -.. _related issue: https://github.com/ClusterHQ/flocker/issues/516 + docker-compose run --rm --service-ports development bash +Now running the check for linting errors described above is as easy as:: -Docker ------- - -OSX users will probably find it easiest to set up a Docker container for -development. Certbot comes with a Dockerfile (``Dockerfile-dev``) -for doing so. To use Docker on OSX, install and setup docker-machine using the -instructions at https://docs.docker.com/installation/mac/. - -To build the development Docker image:: - - docker build -t certbot -f Dockerfile-dev . - -Now run tests inside the Docker image: - -.. code-block:: shell - - docker run -it certbot bash - cd src - tox -e py27 - + tox -e lint .. _prerequisites: diff --git a/docs/using.rst b/docs/using.rst index c162e2fde..7c17796e7 100644 --- a/docs/using.rst +++ b/docs/using.rst @@ -144,6 +144,10 @@ the ``--nginx`` flag on the commandline. Standalone ---------- +Use standalone mode to obtain a cert if you don't want to use (or don't currently have) +existing server software. The standalone plugin does not rely on any other server +software running on the machine where you obtain the cert. + To obtain a cert using a "standalone" webserver, you can use the standalone plugin by including ``certonly`` and ``--standalone`` on the command line. This plugin needs to bind to port 80 or 443 in @@ -154,10 +158,8 @@ one of the options shown below on the command line. * ``--preferred-challenges http`` to use port 80 * ``--preferred-challenges tls-sni`` to use port 443 -The standalone plugin does not rely on any other server software running -on the machine where you obtain the certificate. It must still be possible -for that machine to accept inbound connections from the Internet on the -specified port using each requested domain name. +It must still be possible for your machine to accept inbound connections from +the Internet on the specified port using each requested domain name. .. note:: The ``--standalone-supported-challenges`` option has been deprecated since ``certbot`` version 0.9.0. @@ -429,6 +431,13 @@ Certbot is working hard to improve the renewal process, and we apologize for any inconvenience you encounter in integrating these commands into your individual environment. +.. note:: ``certbot renew`` exit status will only be 1 if a renewal attempt failed. + This means ``certbot renew`` exit status will be 0 if no cert needs to be updated. + If you write a custom script and expect to run a command only after a cert was actually renewed + you will need to use the ``--post-hook`` since the exit status will be 0 both on successful renewal + and when renewal is not necessary. + + Modifying the Renewal Configuration File ---------------------------------------- diff --git a/letsencrypt-auto b/letsencrypt-auto index ac721638f..54cc429cf 100755 --- a/letsencrypt-auto +++ b/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.11.1" +LE_AUTO_VERSION="0.12.0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.1 \ - --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ - --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e -certbot==0.11.1 \ - --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ - --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 -certbot-apache==0.11.1 \ - --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ - --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 -certbot-nginx==0.11.1 \ - --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ - --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 +acme==0.12.0 \ + --hash=sha256:a6050619b3e07b41d197992bb15b32c755dfa0665cfa1c20faa82806a798265b \ + --hash=sha256:a05cba6b5b0fffdfa246b32492a44769011d45205f3ee8efde1f37ee9843fbdf +certbot==0.12.0 \ + --hash=sha256:d018d13665eb4cfe7038c2df636e3f4928742b83769b95edfdb0311277f0eb48 \ + --hash=sha256:4a71925c035b62dfb7c3343c619ee090add76188b47225272b57798ad63388b7 +certbot-apache==0.12.0 \ + --hash=sha256:de86907ea60e7bc35d252b87dec04eab3c7f3a1ea768774876e7ff582d89d640 \ + --hash=sha256:77dde63cf97292b09da8ae09ef8a7a6d83a3b1ee0f8d1fefe513fc77a6292509 +certbot-nginx==0.12.0 \ + --hash=sha256:c66d848c4577f1f91a06a8119b40f1ab90af1546addea27905434bd070f3924d \ + --hash=sha256:4dab2c93304c80d8d0d2e5214939f016804fd46859dd7a39b892d8b7195ab5ec UNLIKELY_EOF # ------------------------------------------------------------------------- diff --git a/letsencrypt-auto-source/certbot-auto.asc b/letsencrypt-auto-source/certbot-auto.asc index 44e2246b7..417d43387 100644 --- a/letsencrypt-auto-source/certbot-auto.asc +++ b/letsencrypt-auto-source/certbot-auto.asc @@ -1,11 +1,11 @@ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 -iQEcBAABAgAGBQJYkqlBAAoJEE0XyZXNl3XyXUAH/RqwKDOpceChSdH/aIk891HX -VRDBQxIjZ6EB1iyebfihyZd4a5zGJ9ocMj1GxThMyLgSKbgkSRtjE/+ymDWsL0Us -Y8w9fw76BAImaJZEvjkrpqD2bSYdijnF479hBa/huZHKcQhb/sqxkNJO9SO1uj8z -bnF0UjJNjgn1hm2yHNMWwyEX7xCN/Vxiq/Zwqi7HdPus99sInJA7+04nwXaUtash -87MHpCjIiHh3axOCOjJbAWzIfsDUKeaBHgeYO+2ldOPWVQ0Amp7ghXjohryBkiux -dqhhAuvBTmNqPrbPAjdJ7Kd74NOGDo3HvAUiuXIckDWqxX2Q34w5pwxelZcIEnI= -=vmVS +iQEcBAABAgAGBQJYuJdQAAoJEE0XyZXNl3Xyw+oH/1AQ90P3397rKB0jP+5MchtR +Nz4ScKL86x9s+o/OzAN76gLhJNj/gOVWoyeK8wVkJ07MpbGyLBiYFsXPZWYUcJ77 +LRj4sGAxJatptHG+PnzIquAf+swynqVu0QdBv8ImKwYrqOlULR+Kr8QZE95Ena51 +JPkbm5o0ipSbByIpraAYabCOHj7SrsFQtMx+tPTd7xaliO8VkguzLQt93QQC7CNj +JIO/yURnfKzutTOe3OPzBzbb6e2yhHcHZcSyv8S0DCIAoB08N9Bs8aAbVwmD89Fq +fwYxLZherXRZ2VtJ2Sf/hUP2ZrEH/mvCkKjzznZokFGJXLvTEc8fC/O6c/q/nLw= +=YiSx -----END PGP SIGNATURE----- diff --git a/letsencrypt-auto-source/letsencrypt-auto b/letsencrypt-auto-source/letsencrypt-auto index 81e3862c2..35c7a530c 100755 --- a/letsencrypt-auto-source/letsencrypt-auto +++ b/letsencrypt-auto-source/letsencrypt-auto @@ -23,7 +23,7 @@ if [ -z "$VENV_PATH" ]; then VENV_PATH="$XDG_DATA_HOME/$VENV_NAME" fi VENV_BIN="$VENV_PATH/bin" -LE_AUTO_VERSION="0.12.0.dev0" +LE_AUTO_VERSION="0.13.0.dev0" BASENAME=$(basename $0) USAGE="Usage: $BASENAME [OPTIONS] A self-updating wrapper script for the Certbot ACME client. When run, updates @@ -833,18 +833,18 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.1 \ - --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ - --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e -certbot==0.11.1 \ - --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ - --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 -certbot-apache==0.11.1 \ - --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ - --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 -certbot-nginx==0.11.1 \ - --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ - --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 +acme==0.12.0 \ + --hash=sha256:a6050619b3e07b41d197992bb15b32c755dfa0665cfa1c20faa82806a798265b \ + --hash=sha256:a05cba6b5b0fffdfa246b32492a44769011d45205f3ee8efde1f37ee9843fbdf +certbot==0.12.0 \ + --hash=sha256:d018d13665eb4cfe7038c2df636e3f4928742b83769b95edfdb0311277f0eb48 \ + --hash=sha256:4a71925c035b62dfb7c3343c619ee090add76188b47225272b57798ad63388b7 +certbot-apache==0.12.0 \ + --hash=sha256:de86907ea60e7bc35d252b87dec04eab3c7f3a1ea768774876e7ff582d89d640 \ + --hash=sha256:77dde63cf97292b09da8ae09ef8a7a6d83a3b1ee0f8d1fefe513fc77a6292509 +certbot-nginx==0.12.0 \ + --hash=sha256:c66d848c4577f1f91a06a8119b40f1ab90af1546addea27905434bd070f3924d \ + --hash=sha256:4dab2c93304c80d8d0d2e5214939f016804fd46859dd7a39b892d8b7195ab5ec UNLIKELY_EOF # ------------------------------------------------------------------------- @@ -1093,6 +1093,9 @@ else On failure, return non-zero. """ + +from __future__ import print_function + from distutils.version import LooseVersion from json import loads from os import devnull, environ @@ -1194,12 +1197,12 @@ def main(): flag = argv[1] try: if flag == '--latest-version': - print latest_stable_version(get) + print(latest_stable_version(get)) elif flag == '--le-auto-script': tag = argv[2] verified_new_le_auto(get, tag, dirname(argv[0])) except ExpectedError as exc: - print exc.args[0], exc.args[1] + print(exc.args[0], exc.args[1]) return 1 else: return 0 diff --git a/letsencrypt-auto-source/letsencrypt-auto.sig b/letsencrypt-auto-source/letsencrypt-auto.sig index 7aea42dd0..711300dda 100644 Binary files a/letsencrypt-auto-source/letsencrypt-auto.sig and b/letsencrypt-auto-source/letsencrypt-auto.sig differ diff --git a/letsencrypt-auto-source/pieces/fetch.py b/letsencrypt-auto-source/pieces/fetch.py index 365a5a36a..e7ebb9e0a 100644 --- a/letsencrypt-auto-source/pieces/fetch.py +++ b/letsencrypt-auto-source/pieces/fetch.py @@ -10,6 +10,9 @@ On failure, return non-zero. """ + +from __future__ import print_function + from distutils.version import LooseVersion from json import loads from os import devnull, environ @@ -111,12 +114,12 @@ def main(): flag = argv[1] try: if flag == '--latest-version': - print latest_stable_version(get) + print(latest_stable_version(get)) elif flag == '--le-auto-script': tag = argv[2] verified_new_le_auto(get, tag, dirname(argv[0])) except ExpectedError as exc: - print exc.args[0], exc.args[1] + print(exc.args[0], exc.args[1]) return 1 else: return 0 diff --git a/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt b/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt index 36d93dddd..d70d24e2a 100644 --- a/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt +++ b/letsencrypt-auto-source/pieces/letsencrypt-auto-requirements.txt @@ -171,15 +171,15 @@ letsencrypt==0.7.0 \ # THE LINES BELOW ARE EDITED BY THE RELEASE SCRIPT; ADD ALL DEPENDENCIES ABOVE. -acme==0.11.1 \ - --hash=sha256:9f4efac6dc4477a3baa7eb2392d4f7583f974e4ad336439aa1961ef805622a77 \ - --hash=sha256:db35258edfc13dfe5839215898fe2d5d3caafc9a084f631a032f3fdf712c694e -certbot==0.11.1 \ - --hash=sha256:ba80552df0f390dbc5fcd14b4ea4b1499ea866f5f78c8c1a375abc25101dedf1 \ - --hash=sha256:6c1724486d500c5163c9313d6a14af5af9f4515f79553627303a6b86df2c3af2 -certbot-apache==0.11.1 \ - --hash=sha256:70132d9013509011b9edeba64fc208961f50ef78457f58d3b80a61094102efcd \ - --hash=sha256:efe2224b531595edee366423c115e2874a3c9011890321d3ccda0367efc776c0 -certbot-nginx==0.11.1 \ - --hash=sha256:1895eea1de92ab3dfd762998a4be7868ec3ec4d42cce7772995e4e9b2e488e6a \ - --hash=sha256:e5e5ffe8930ba10139bb61c2a05a30e84d9a69a7d8fc6a7b391f707eae8bfce5 +acme==0.12.0 \ + --hash=sha256:a6050619b3e07b41d197992bb15b32c755dfa0665cfa1c20faa82806a798265b \ + --hash=sha256:a05cba6b5b0fffdfa246b32492a44769011d45205f3ee8efde1f37ee9843fbdf +certbot==0.12.0 \ + --hash=sha256:d018d13665eb4cfe7038c2df636e3f4928742b83769b95edfdb0311277f0eb48 \ + --hash=sha256:4a71925c035b62dfb7c3343c619ee090add76188b47225272b57798ad63388b7 +certbot-apache==0.12.0 \ + --hash=sha256:de86907ea60e7bc35d252b87dec04eab3c7f3a1ea768774876e7ff582d89d640 \ + --hash=sha256:77dde63cf97292b09da8ae09ef8a7a6d83a3b1ee0f8d1fefe513fc77a6292509 +certbot-nginx==0.12.0 \ + --hash=sha256:c66d848c4577f1f91a06a8119b40f1ab90af1546addea27905434bd070f3924d \ + --hash=sha256:4dab2c93304c80d8d0d2e5214939f016804fd46859dd7a39b892d8b7195ab5ec diff --git a/letshelp-certbot/setup.py b/letshelp-certbot/setup.py index b616da688..b26ab41fe 100644 --- a/letshelp-certbot/setup.py +++ b/letshelp-certbot/setup.py @@ -7,12 +7,9 @@ from setuptools import find_packages version = '0.7.0.dev0' install_requires = [ + 'mock', 'setuptools', # pkg_resources ] -if sys.version_info < (2, 7): - install_requires.append('mock<1.1.0') -else: - install_requires.append('mock') docs_extras = [ 'Sphinx>=1.0', # autodoc_member_order = 'bysource', autodoc_default_flags diff --git a/setup.py b/setup.py index 0c47b973f..0e8d19a22 100644 --- a/setup.py +++ b/setup.py @@ -36,12 +36,14 @@ version = meta['version'] # https://github.com/pypa/pip/issues/988 for more info. install_requires = [ 'acme=={0}'.format(version), + 'argparse', # We technically need ConfigArgParse 0.10.0 for Python 2.6 support, but # saying so here causes a runtime error against our temporary fork of 0.9.3 # in which we added 2.6 support (see #2243), so we relax the requirement. 'ConfigArgParse>=0.9.3', 'configobj', 'cryptography>=0.7', # load_pem_x509_certificate + 'mock', 'parsedatetime>=1.3', # Calendar.parseDT 'PyOpenSSL', 'pyrfc3339', @@ -54,17 +56,6 @@ install_requires = [ 'zope.interface', ] -# env markers in extras_require cause problems with older pip: #517 -# Keep in sync with conditional_requirements.py. -if sys.version_info < (2, 7): - install_requires.extend([ - # only some distros recognize stdlib argparse as already satisfying - 'argparse', - 'mock<1.1.0', - ]) -else: - install_requires.append('mock') - dev_extras = [ # Pin astroid==1.3.5, pylint==1.4.2 as a workaround for #289 'astroid==1.3.5', diff --git a/tests/letstest/scripts/test_leauto_upgrades.sh b/tests/letstest/scripts/test_leauto_upgrades.sh index f7a74821b..b46080eff 100755 --- a/tests/letstest/scripts/test_leauto_upgrades.sh +++ b/tests/letstest/scripts/test_leauto_upgrades.sh @@ -4,13 +4,6 @@ # are dynamically set at execution cd letsencrypt -#git checkout v0.1.0 use --branch instead -SAVE="$PIP_EXTRA_INDEX_URL" -unset PIP_EXTRA_INDEX_URL -export PIP_INDEX_URL="https://isnot.org/pip/0.1.0/" - -#OLD_LEAUTO="https://raw.githubusercontent.com/letsencrypt/letsencrypt/5747ab7fd9641986833bad474d71b46a8c589247/letsencrypt-auto" - if ! command -v git ; then if [ "$OS_TYPE" = "ubuntu" ] ; then @@ -22,15 +15,18 @@ if ! command -v git ; then fi fi BRANCH=`git rev-parse --abbrev-ref HEAD` -git checkout -f v0.1.0 -./letsencrypt-auto -v --debug --version -unset PIP_INDEX_URL - -export PIP_EXTRA_INDEX_URL="$SAVE" +# 0.4.1 is the oldest version of letsencrypt-auto that can be used because +# it's the first version that both pins package versions and properly supports +# --no-self-upgrade. +git checkout -f v0.4.1 +if ! ./letsencrypt-auto -v --debug --version --no-self-upgrade 2>&1 | grep 0.4.1 ; then + echo initial installation appeared to fail + exit 1 +fi git checkout -f "$BRANCH" EXPECTED_VERSION=$(grep -m1 LE_AUTO_VERSION letsencrypt-auto | cut -d\" -f2) -if ! ./letsencrypt-auto -v --debug --version --no-self-upgrade | grep $EXPECTED_VERSION ; then +if ! ./letsencrypt-auto -v --debug --version --no-self-upgrade 2>&1 | grep $EXPECTED_VERSION ; then echo upgrade appeared to fail exit 1 fi diff --git a/tests/modification-check.sh b/tests/modification-check.sh index 5168f5ab5..6f412ba47 100755 --- a/tests/modification-check.sh +++ b/tests/modification-check.sh @@ -1,46 +1,39 @@ #!/bin/bash temp_dir=`mktemp -d` +trap "rm -rf $temp_dir" EXIT -# Script should be run from Certbot's root directory - -SCRIPT_PATH=`dirname $0` -SCRIPT_PATH=`readlink -f $SCRIPT_PATH` +# cd to repo root +cd $(dirname $(dirname $(readlink -f $0))) FLAG=false -# Compare root letsencrypt-auto and certbot-auto with published versions - -cp letsencrypt-auto ${temp_dir}/letsencrypt-to-be-checked -cp certbot-auto ${temp_dir}/certbot-to-be-checked - -cp letsencrypt-auto-source/pieces/fetch.py ${temp_dir}/fetch.py -cd ${temp_dir} - -LATEST_VERSION=`python fetch.py --latest-version` -python fetch.py --le-auto-script v${LATEST_VERSION} - -cmp -s letsencrypt-auto letsencrypt-to-be-checked - -if [ $? != 0 ]; then - echo "Root letsencrypt-auto has changed." - FLAG=true +if ! cmp -s certbot-auto letsencrypt-auto; then + echo "Root certbot-auto and letsencrypt-auto differ." + FLAG=true else - echo "Root letsencrypt-auto is unchanged." + cp certbot-auto "$temp_dir/local-auto" + cp letsencrypt-auto-source/pieces/fetch.py "$temp_dir/fetch.py" + cd $temp_dir + + # Compare file against current version in the target branch + BRANCH=${TRAVIS_BRANCH:-master} + URL="https://raw.githubusercontent.com/certbot/certbot/$BRANCH/certbot-auto" + curl -sS $URL > certbot-auto + if cmp -s certbot-auto local-auto; then + echo "Root *-auto were unchanged." + else + # Compare file against the latest released version + python fetch.py --le-auto-script "v$(python fetch.py --latest-version)" + if cmp -s letsencrypt-auto local-auto; then + echo "Root *-auto were updated to the latest version." + else + echo "Root *-auto have unexpected changes." + FLAG=true + fi + fi + cd ~- fi -cmp -s letsencrypt-auto certbot-to-be-checked - -if [ $? != 0 ]; then - echo "Root certbot-auto has changed." - FLAG=true -else - echo "Root certbot-auto is unchanged." -fi - -# Cleanup -rm ${temp_dir}/* -cd ${SCRIPT_PATH}/../ - # Compare letsencrypt-auto-source/letsencrypt-auto with output of build.py cp letsencrypt-auto-source/letsencrypt-auto ${temp_dir}/original-lea diff --git a/tools/release.sh b/tools/release.sh index 75a4af29c..81582cef0 100755 --- a/tools/release.sh +++ b/tools/release.sh @@ -88,9 +88,9 @@ SetVersion() { sed -i "s/^version.*/version = '$ver'/" $pkg_dir/setup.py done sed -i "s/^__version.*/__version__ = '$ver'/" certbot/__init__.py - + # interactive user input - git add -p certbot $SUBPKGS certbot-compatibility-test + git add -p certbot $SUBPKGS certbot-compatibility-test } diff --git a/tox.ini b/tox.ini index 232010d40..ea1423415 100644 --- a/tox.ini +++ b/tox.ini @@ -151,7 +151,9 @@ commands = docker run --rm -t -i lea whitelist_externals = docker -passenv = DOCKER_* +passenv = + DOCKER_* + TRAVIS_BRANCH [testenv:le_auto_wheezy] # At the moment, this tests under Python 2.7 only, as only that version is