[ADD] poetry support

- Auto regenerate poetry pyproject.toml from all requirements.txt with
script poetry_update.py
- Sort dependancy in pyproject.toml
- Rename directory venv to .venv
- In bash, use USER instead of whoami
- Remove python3 from ubuntu dependancy. Ignore python from system.
- Use pyenv with poetry
- Simplify pip installation script
This commit is contained in:
Mathieu Benoit 2020-07-24 02:20:30 -04:00
parent 8a1d12adbc
commit 2888cbf890
35 changed files with 2240 additions and 210 deletions

2
.gitignore vendored
View file

@ -3,7 +3,7 @@
*.tmp
*.bak
.idea
venv
.venv
*.log
config.conf
*.DS_Store

1
.python-version Normal file
View file

@ -0,0 +1 @@
3.7.7

View file

@ -2,7 +2,7 @@
## Simplify push tag
For RELEASE.md, replace next value by a script to select all remote by manifest file.
Actually, need to push manually all different remote.
> ./venv/repo forall -pc "git push ERPLibre --tags"
> ./.venv/repo forall -pc "git push ERPLibre --tags"
## Funding
- Add funding for MathBenTech and TechnoLibre

View file

@ -106,6 +106,6 @@ To regenerate only manifest.xml.
# Coding
## Create module scaffold (run in the venv)
```bash
source ./venv/bin/activate
source ./.venv/bin/activate
python odoo/odoo-bin scaffold MODULE_NAME addons/REPO_NAME/
```

View file

@ -28,5 +28,6 @@ Execute script:
git checkout -b NEW_BRANCH
git commit -am "Add new repo"
./script/install_locally_dev.sh
./script/poetry_update.py
```
[Update your repo.](./GIT_REPO.md)

View file

@ -5,19 +5,19 @@ This is a guide to understand git-repo. Scripts in ERPLibre use git-repo automat
## Setup repo
```bash
curl https://storage.googleapis.com/git-repo-downloads/repo > ./venv/repo
curl https://storage.googleapis.com/git-repo-downloads/repo > ./.venv/repo
```
## prod
```bash
./venv/repo init -u http://git.erplibre.ca/ERPLibre -b master
./venv/repo sync
./.venv/repo init -u http://git.erplibre.ca/ERPLibre -b master
./.venv/repo sync
```
## dev
```bash
./venv/repo init -u http://git.erplibre.ca/ERPLibre -b 12.0_repo -m ./manifest/default.dev.xml
./venv/repo sync
./.venv/repo init -u http://git.erplibre.ca/ERPLibre -b 12.0_repo -m ./manifest/default.dev.xml
./.venv/repo sync
```
## dev locally
@ -25,8 +25,8 @@ curl https://storage.googleapis.com/git-repo-downloads/repo > ./venv/repo
```bash
git daemon --base-path=. --export-all --reuseaddr --informative-errors --verbose &
./venv/repo init -u git://127.0.0.1:9418/ -b $(git rev-parse --abbrev-ref HEAD) -m ./manifest/default.dev.xml
./venv/repo sync -m ./manifest/default.dev.xml
./.venv/repo init -u git://127.0.0.1:9418/ -b $(git rev-parse --abbrev-ref HEAD) -m ./manifest/default.dev.xml
./.venv/repo sync -m ./manifest/default.dev.xml
```
# Create Manifest
@ -37,7 +37,7 @@ Freezes all repo, from dev to prod.
This will add revision git hash in the Manifest.
```bash
./venv/repo manifest -r -o ./default.xml
./.venv/repo manifest -r -o ./default.xml
```
Do your commit.
```bash
@ -51,14 +51,14 @@ git commit -am "Updated manifest/default.staged.xml"
git daemon --base-path=. --export-all --reuseaddr --informative-errors --verbose &
./venv/repo init -u git://127.0.0.1:9418/ -b $(git rev-parse --abbrev-ref HEAD) -m ./manifest/default.staged.xml
./venv/repo sync -m ./manifest/default.staged.xml
./.venv/repo init -u git://127.0.0.1:9418/ -b $(git rev-parse --abbrev-ref HEAD) -m ./manifest/default.staged.xml
./.venv/repo sync -m ./manifest/default.staged.xml
./venv/repo manifest -r -o ./default.xml
./.venv/repo manifest -r -o ./default.xml
```
## Create a dev version
```bash
./venv/repo manifest -o ./manifest/default.dev.xml
./.venv/repo manifest -o ./manifest/default.dev.xml
```
Do your commit.
```bash
@ -68,17 +68,17 @@ git commit -am "[#ticket] subject: short sentence"
## Useful command
### Search all repo with specific branch name
```bash
./venv/repo forall -pc "git branch -a|grep BRANCH"
./.venv/repo forall -pc "git branch -a|grep BRANCH"
```
### Search missing branch in all repo
```bash
./venv/repo forall -pc 'git branch -a|(grep /BRANCH$||echo "no match")|grep "no match"'
./.venv/repo forall -pc 'git branch -a|(grep /BRANCH$||echo "no match")|grep "no match"'
```
### Search change file in all repo
```bash
./venv/repo forall -pc "git status -s"
./.venv/repo forall -pc "git status -s"
```
### Clean all

13
doc/POETRY.md Normal file
View file

@ -0,0 +1,13 @@
# Poetry
## Add automatically dependancies
Add your dependancies in files [requirements.txt](../requirements.txt) and run script
```bash
./script/poetry_update.py
```
This will search all requirements.txt files and update pyproject.toml, to run poetry update.
## Add manually dependancies
The automatic script will erase this dependancy, but you can add it for your locally test.
```bash
poetry add PYTHON_MODULE
```

View file

@ -3,7 +3,7 @@ A guide on how to do a release.
## Generate new prod
```bash
./venv/repo manifest -r -o ./default.xml
./.venv/repo manifest -r -o ./default.xml
```
Do your commit.
```bash
@ -21,10 +21,10 @@ git tag v#.#.#
# Push your tags
git push --tags
# Add tags for all repo
./venv/repo forall -pc "git tag ERPLibre/v#.#.#"
./venv/repo forall -pc "git push ERPLibre --tags"
./.venv/repo forall -pc "git tag ERPLibre/v#.#.#"
./.venv/repo forall -pc "git push ERPLibre --tags"
# Get all difference between a tag and HEAD, to update the CHANGELOG.md
./venv/repo forall -pc "git diff ERPLibre/v#.#.#..HEAD"
./.venv/repo forall -pc "git diff ERPLibre/v#.#.#..HEAD"
```
# TIPS

1797
poetry.lock generated

File diff suppressed because it is too large Load diff

5
poetry.toml Normal file
View file

@ -0,0 +1,5 @@
cache-dir = "./"
[virtualenvs]
create = false
in-project = false
path = "{cache-dir}.venv"

View file

@ -1,50 +1,101 @@
[tool.poetry]
[build-system]
requires = [ "poetry>=1.0.0",]
build-backend = "poetry.masonry.api"
[tool.poetry]
name = "ERPLibre"
version = "1.1.0"
description = "TODO"
description = "Easy way to configure Odoo community"
license = "AGPL-3.0-or-later"
authors = ["TechnoLibre"]
authors = [ "Mathieu Benoit <mathben@technolibre.ca>", "Alexandre Ferreira Benevides <afbenevides@technolibre.ca>",]
[tool.poetry.dependencies]
python = "^3.7"
giturlparse = "^0.9.2"
agithub = "^2.2.2"
gitpython = "^3.1.7"
PyYAML = "^5.3.1"
retrying = "^1.3.3"
xmltodict = "^0.12.0"
Babel = "^2.3.4"
Jinja2 = "^2.10.1"
Mako = "^1.0.4"
MarkupSafe = "^0.23"
Pillow = "^6.1.0"
PyPDF2 = "^1.26.0"
PyYAML = "^5.3.1"
SOAPpy = "^0.12.22"
Voicent-Python = "^1.0"
Werkzeug = "^0.11.15"
XlsxWriter = "^0.9.3"
agithub = "^2.2.2"
altair = "^4.1.0"
bokeh = "^1.1.0"
chardet = "^3.0.4"
cython = "^0.29.21"
ddt = "^1.2.0"
decorator = "^4.0.10"
docutils = "^0.12"
ebaysdk = "^2.1.5"
email_validator = "^1.1.1"
extract-msg = "^0.24.4"
factur-x = "^1.12"
feedparser = "^5.2.1"
freezegun = "^0.3.10"
gevent = "^1.3.4"
gitpython = "^3.1.7"
giturlparse = "^0.9.2"
greenlet = "^0.4.13"
html2text = "^2016.9.19"
Jinja2 = "^2.10.1"
invoice2data = "^0.3.5"
iscompatible = "^0.1.1"
lasso = "^0.0.5"
libsass = "^0.12.3"
Mako = "^1.0.4"
MarkupSafe = "^0.23"
lxml = "^4.2.3"
matplotlib = ">=2.0.0"
mock = "^2.0.0"
mpld3 = "^0.3"
num2words = "^0.5.6"
numpy = "^1.19.1"
"oca.decorators" = "^0.0.1"
ofxparse = "^0.16"
openpyxl = "^3.0.4"
openupgradelib = "^2.0.0"
passlib = "^1.6.5"
pdf2image = "^1.13.1"
phonenumbers = "^8.12.7"
plotly = "^4.1.0"
premailer = "^3.7.0"
psutil = "^4.3.1"
psycopg2 = "^2.7.3.1"
py-Asterisk = "^0.5.18"
"py3o.formats" = "^0.3"
"py3o.template" = "^0.10.0"
pycountry = "^20.7.3"
pydot = "^1.2.3"
pygount = "<1.2.0"
pyldap = "^2.4.28"
pyotp = "^2.3.0"
pyparsing = "^2.1.10"
PyPDF2 = "^1.26.0"
pyserial = "^3.1.1"
pysftp = "^0.2.9"
python-dateutil = "^2.5.3"
python-u2flib-server = "^5.0.0"
pytz = "^2016.7"
pyusb = "^1.0.0"
pyzbar = "^0.1.8"
qrcode = "^5.3"
raven = "^6.10.0"
reportlab = "^3.3.0"
requests = "^2.20.0"
retrying = "^1.3.3"
sphinx = ">=1.6.7"
sphinx-patchqueue = ">=1.0"
suds-jurko = "^0.6"
toml = "^0.10.1"
unidecode = "^1.0.22"
vatnumber = "^1.2"
vobject = "^0.9.3"
Werkzeug = "^0.11.15"
XlsxWriter = "^0.9.3"
websocket-client = "^0.57.0"
wheel = "^0.34.2"
xlrd = "^1.0.0"
xlwt = "^1.3"
xmltodict = "^0.12.0"
zxcvbn = "^4.4.28"
[tool.poetry.dev-dependencies]
websocket_client = "^0.57.0"

View file

@ -1,3 +1,4 @@
# For script
giturlparse
agithub
gitpython
@ -5,3 +6,16 @@ PyYAML
retrying
xmltodict
openupgradelib
# For OSX
cython
# For Odoo
wheel
phonenumbers
# For testing
websocket-client
# For updating poetry
toml
iscompatible

2
run.sh
View file

@ -1,3 +1,3 @@
#!/usr/bin/env bash
source ./venv/bin/activate
source ./.venv/bin/activate
python3 ./odoo/odoo-bin -c ./config.conf --limit-time-real 99999 --limit-time-cpu 99999 $@

View file

@ -1,4 +1,4 @@
#!./venv/bin/python
#!./.venv/bin/python
import os
import sys
import argparse

View file

@ -1,4 +1,4 @@
#!./venv/bin/python
#!./.venv/bin/python
import os
import sys
import argparse

View file

@ -1,4 +1,4 @@
#!./venv/bin/python
#!./.venv/bin/python
import os
import sys
import argparse

View file

@ -1,4 +1,4 @@
#!./venv/bin/python
#!./.venv/bin/python
# © 2020 TechnoLibre (http://www.technolibre.ca)
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).

View file

@ -1,4 +1,4 @@
#!./venv/bin/python
#!./.venv/bin/python
import os
import sys
import argparse

View file

@ -1,4 +1,4 @@
#!./venv/bin/python
#!./.venv/bin/python
import os
import sys
import argparse

View file

@ -1,4 +1,4 @@
#!./venv/bin/python
#!./.venv/bin/python
import os
import sys
import argparse

View file

@ -1,4 +1,4 @@
#!./venv/bin/python
#!./.venv/bin/python
# © 2020 TechnoLibre (http://www.technolibre.ca)
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
import os
@ -351,6 +351,7 @@ class GitTool:
webbrowser.open_new_tab(url)
def generate_install_locally(self, repo_path="./"):
filename_locally = f"{repo_path}script/install_locally.sh"
lst_repo = self.get_repo_info(repo_path=repo_path)
lst_result = []
for repo in lst_repo:
@ -361,14 +362,15 @@ class GitTool:
str_repo = f' printf "${{EL_HOME}}/{repo.get("path")}," >> ' \
f'${{EL_CONFIG_FILE}}\n'
lst_result.append(str_repo)
with open(f"{repo_path}script/install_locally.sh") as file:
with open(filename_locally) as file:
all_lines = file.readlines()
# search place to add/replace lines
index = 0
find_index = False
index_find = 0
for line in all_lines:
if not find_index and "if [[ $EL_MINIMAL_ADDONS = \"False\" ]]; then\n" == line:
if not find_index and \
"if [[ ${EL_MINIMAL_ADDONS} = \"False\" ]]; then\n" == line:
index_find = index + 1
for insert_line in lst_result:
all_lines.insert(index_find, insert_line)
@ -381,8 +383,12 @@ class GitTool:
break
index += 1
if not find_index:
print(f"ERROR cannot regenerate file {filename_locally}, "
f"did you change the header?")
# create file
with open(f"{repo_path}script/install_locally.sh", mode="w") as file:
with open(filename_locally, mode="w") as file:
file.writelines(all_lines)
@staticmethod

View file

@ -1,4 +1,4 @@
#!./venv/bin/python
#!./.venv/bin/python
import os
import sys
import argparse

View file

@ -7,7 +7,7 @@
#-------------------------------------------------------------------------------
################################################################################
EL_USER=$(whoami)
EL_USER=${USER}
#--------------------------------------------------
# Install PostgreSQL Server
@ -26,11 +26,17 @@ brew install git python3 wget pyenv
brew link git
brew link wget
echo "\n---- Installing nodeJS NPM and rtlcss for LTR support ----"
brew install nodejs npm
brew install nodejs npm openssl
sudo npm install -g rtlcss
sudo npm install -g lessc
pyenv install 3.7.7
yes n|pyenv install 3.7.7
pyenv local 3.7.7
echo 'export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"' >> ~/.zshrc
echo -e "\n---- Installing poetry for reliable python package ----"
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
#--------------------------------------------------
# Install Wkhtmltopdf if needed
#--------------------------------------------------
@ -40,16 +46,3 @@ if [ ! -f "wkhtmltox-0.12.5-1.macos-carbon.pkg" ]; then
sudo sudo installer -pkg wkhtmltox-0.12.5-1.macos-carbon.pkg -target /
else echo "Wkhtmltopdf already installed"
fi
# ===============================================================================================
# !!!! FOR OSX there is a problem with python 3.7.5 with PILLOW, use 3.6.9 instead and works fine
# install python 3.6.9
# brew install pyenv
# pyenv install 3.6.9
# then redoo link:
# unlink /usr/local/bin/python3
# ln -s ~/.pyenv/versions/3.6.9/bin/python3.6 /usr/local/bin/python3
# ===============================================================================================
echo "\n---- Installing venv if not already existing (rm -r venv if already exists) ----"
#~/.pyenv/versions/3.6.9/bin/python3.6 -m venv venv
/Users/"${EL_USER}"/.pyenv/versions/3.7.7/bin/python3 -m venv venv

View file

@ -30,8 +30,8 @@ User=${EL_USER}
Group=${EL_USER}
Restart=always
RestartSec=5
PIDFile=${EL_HOME_ERPLIBRE}/venv/service.pid
ExecStart=${EL_HOME_ERPLIBRE}/venv/run.sh
PIDFile=${EL_HOME_ERPLIBRE}/.venv/service.pid
ExecStart=${EL_HOME_ERPLIBRE}/.venv/run.sh
StandardOutput=journal+console
[Install]
@ -51,14 +51,14 @@ sudo su ${EL_USER} -c "sudo rm -f /tmp/${EL_USER}run.sh"
cat <<EOF > /tmp/${EL_USER}run.sh
#!/usr/bin/env bash
cd ${EL_HOME_ERPLIBRE}
source ./venv/bin/activate
source ./.venv/bin/activate
python3 ${EL_HOME_ERPLIBRE}/odoo/odoo-bin -c ${EL_HOME_ERPLIBRE}/config.conf --limit-time-real 99999 --limit-time-cpu 99999 $@
EOF
echo -e "* Security Run File"
sudo cp /tmp/${EL_USER}run.sh ${EL_HOME_ERPLIBRE}/venv/run.sh
sudo chmod 755 ${EL_HOME_ERPLIBRE}/venv/run.sh
sudo chown ${EL_USER}: ${EL_HOME_ERPLIBRE}/venv/run.sh
sudo cp /tmp/${EL_USER}run.sh ${EL_HOME_ERPLIBRE}/.venv/run.sh
sudo chmod 755 ${EL_HOME_ERPLIBRE}/.venv/run.sh
sudo chown ${EL_USER}: ${EL_HOME_ERPLIBRE}/.venv/run.sh
echo "-----------------------------------------------------------"
echo "Done! The ERPLibre server is up and running. Specifications:"

View file

@ -2,7 +2,7 @@
. ./env_var.sh
EL_USER=$(whoami)
EL_USER=${USER}
#EL_INSTALL_WKHTMLTOPDF="True"
##
@ -31,42 +31,65 @@ sudo apt-get upgrade -y
# Install PostgreSQL Server
#--------------------------------------------------
echo -e "\n---- Install PostgreSQL Server ----"
sudo apt-get install postgresql -y
sudo apt-get install postgresql libpq-dev -y
echo -e "\n---- Creating the ERPLibre PostgreSQL User ----"
sudo su - postgres -c "createuser -s $EL_USER" 2> /dev/null || true
sudo su - postgres -c "createuser -s ${EL_USER}" 2> /dev/null || true
#--------------------------------------------------
# Install Dependencies
#--------------------------------------------------
echo -e "\n--- Installing Python 3 + pip3 --"
sudo apt-get install git python3 python3-pip build-essential wget python3-dev python3-venv python3-wheel libxslt-dev libzip-dev libldap2-dev libsasl2-dev python3-setuptools node-less libpng12-0 gdebi-core -y
echo -e "\n--- Installing debian dependancy --"
sudo apt-get install git build-essential wget libxslt-dev libzip-dev libldap2-dev libsasl2-dev node-less libpng12-0 gdebi-core libffi-dev -y
sudo apt-get install libmariadbd-dev -y
echo -e "\n---- Installing nodeJS NPM and rtlcss for LTR support ----"
sudo apt-get install nodejs npm -y
sudo npm install -g rtlcss
sudo npm install -g lessc
if [ ${EL_INSTALL_NGINX} = "True" ]; then
sudo apt install nginx -y
echo -e "\n---- Installing nginx ----"
sudo apt install nginx -y
fi
echo -e "\n---- Installing python 3.7.7 with pyenv ----"
curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash
export PATH="/home/${USER}/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
yes n|pyenv install 3.7.7
pyenv local 3.7.7
python_exec="$(pyenv root)/versions/3.7.7/bin/python"
echo -e "\n---- Installing poetry for reliable python package ----"
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | ${python_exec}
#/home/"${USER}"/.poetry/bin/poetry env use ${python_exec}
#source $HOME/.poetry/env
#--------------------------------------------------
# Install Wkhtmltopdf if needed
#--------------------------------------------------
if [ ${EL_INSTALL_WKHTMLTOPDF} = "True" ]; then
echo -e "\n---- Installing wkhtml ----"
INSTALLED=$(dpkg -s wkhtmltox|grep installed)
if [ "" == "${INSTALLED}" ]; then
echo -e "\n---- Install wkhtml and place shortcuts on correct place ----"
#pick up correct one from x64 & x32 versions:
if [ "`getconf LONG_BIT`" == "64" ];then
_url=$WKHTMLTOX_X64
_url=${WKHTMLTOX_X64}
else
_url=$WKHTMLTOX_X32
_url=${WKHTMLTOX_X32}
fi
sudo wget ${_url}
sudo gdebi --n `basename ${_url}`
sudo ln -s /usr/local/bin/wkhtmltopdf /usr/bin
sudo ln -s /usr/local/bin/wkhtmltoimage /usr/bin
else echo -e "\n---- Already installed wkhtml ----"
fi
else
echo "Wkhtmltopdf isn't installed due to the choice of the user!"

View file

@ -2,7 +2,7 @@
. ./env_var.sh
EL_USER=$(whoami)
EL_USER=${USER}
EL_HOME=$PWD
EL_HOME_ODOO="${EL_HOME}/odoo"
#EL_INSTALL_WKHTMLTOPDF="True"
@ -14,17 +14,6 @@ EL_CONFIG_FILE="${EL_HOME}/config.conf"
#EL_MINIMAL_ADDONS="False"
#EL_INSTALL_NGINX="True"
if hash python3.7 2>/dev/null; then
PYTHON37="True"
PYTHON36="False"
elif hash python3.6 2>/dev/null; then
PYTHON37="False"
PYTHON36="True"
else
echo "Missing python3.7 or python3.6. Python3.8 is not compatible."
exit 1
fi
echo -e "* Create server config file"
touch ${EL_CONFIG_FILE}
@ -40,7 +29,7 @@ printf "longpolling_port = ${EL_LONGPOLLING_PORT}\n" >> ${EL_CONFIG_FILE}
printf "addons_path = ${EL_HOME_ODOO}/addons,${EL_HOME}/addons/addons," >> ${EL_CONFIG_FILE}
printf "${EL_HOME}/addons/OCA_web," >> ${EL_CONFIG_FILE}
if [[ $EL_MINIMAL_ADDONS = "False" ]]; then
if [[ ${EL_MINIMAL_ADDONS} = "False" ]]; then
printf "${EL_HOME}/addons/ERPLibre_erplibre_addons," >> ${EL_CONFIG_FILE}
printf "${EL_HOME}/addons/MathBenTech_development," >> ${EL_CONFIG_FILE}
printf "${EL_HOME}/addons/MathBenTech_odoo-business-spending-management-quebec-canada," >> ${EL_CONFIG_FILE}
@ -134,7 +123,7 @@ printf "\n" >> ${EL_CONFIG_FILE}
printf "workers = 2\n" >> ${EL_CONFIG_FILE}
printf "max_cron_threads = 2\n" >> ${EL_CONFIG_FILE}
if [ ${EL_INSTALL_NGINX} = "True" ]; then
if [[ ${EL_INSTALL_NGINX} = "True" ]]; then
printf "xmlrpc_interface = 127.0.0.1\n" >> ${EL_CONFIG_FILE}
printf "netrpc_interface = 127.0.0.1\n" >> ${EL_CONFIG_FILE}
printf "proxy_mode = True\n" >> ${EL_CONFIG_FILE}
@ -144,22 +133,34 @@ echo -e "\n---- Install Odoo with addons module ----"
git submodule update --init
# Generate empty addons if missing
if [ ! -d "./addons/addons" ]; then
if [[ ! -d "./addons/addons" ]]; then
mkdir -p ./addons/addons
fi
echo -e "\n---- Create Virtual environment Python ----"
cd ${EL_HOME}
if [[ ${PYTHON37} = "True" ]]; then
python3.7 -m venv venv
elif [[ ${PYTHON36} = "True" ]]; then
python3.6 -m venv venv
if [[ ! -f "./.venv" ]]; then
echo -e "\n---- Create Virtual environment Python ----"
if [[ -f "/home/"${USER}"/.pyenv/versions/3.7.7/bin/python3" ]]; then
/home/"${USER}"/.pyenv/versions/3.7.7/bin/python3 -m venv .venv
elif [[ -f "/Users/"${USER}"/.pyenv/versions/3.7.7/bin/python3" ]]; then
/Users/"${USER}"/.pyenv/versions/3.7.7/bin/python3 -m venv .venv
else
echo "Missing pyenv, please refer installation guide."
exit 1
fi
fi
cd -
echo -e "\n---- Installing poetry dependancy ----"
.venv/bin/pip install --upgrade pip
source $HOME/.poetry/env
poetry install
# Link for dev
echo -e "\n---- Add link dependency in site-packages of Python ----"
ln -fs ${EL_HOME_ODOO}/odoo ${EL_HOME}/.venv/lib/python3.7/site-packages/
# Install git-repo if missing
if [ ! -f "./venv/repo" ]; then
echo "Install git-repo from Google APIS"
curl https://storage.googleapis.com/git-repo-downloads/repo > ./venv/repo
chmod +x ./venv/repo
if [[ ! -f "./.venv/repo" ]]; then
echo "\n---- Install git-repo from Google APIS ----"
curl https://storage.googleapis.com/git-repo-downloads/repo > ./.venv/repo
chmod +x ./.venv/repo
fi

View file

@ -2,37 +2,7 @@
. ./env_var.sh
EL_USER=$(whoami)
EL_HOME=$PWD
EL_HOME_ODOO="${EL_HOME}/odoo"
#EL_INSTALL_WKHTMLTOPDF="True"
#EL_PORT="8069"
#EL_LONGPOLLING_PORT="8072"
#EL_SUPERADMIN="admin"
EL_CONFIG_FILE="${EL_HOME}/config.conf"
#EL_CONFIG="${EL_USER}"
#EL_MINIMAL_ADDONS="False"
#EL_INSTALL_NGINX="True"
#EL_MANIFEST_PROD="./manifest/default.xml"
#EL_MANIFEST_DEV="./manifest/default.dev.xml"
./script/install_locally.sh
# Update git-repo
./script/update_manifest_local_dev.sh
echo -e "\n---- Install python packages/requirements ----"
${EL_HOME}/venv/bin/pip3 install --upgrade pip
${EL_HOME}/venv/bin/pip3 install wheel phonenumbers
${EL_HOME}/venv/bin/pip3 install -r "${EL_HOME}/odoo/requirements.txt"
${EL_HOME}/venv/bin/pip3 install -r "${EL_HOME}/requirements.txt"
# For dev/testing
${EL_HOME}/venv/bin/pip3 install websocket-client
echo -e "\n---- Add link dependency in site-packages of Python ----"
if [[ ${PYTHON37} = "True" ]]; then
ln -fs ${EL_HOME_ODOO}/odoo ${EL_HOME}/venv/lib/python3.7/site-packages/
elif [[ ${PYTHON36} = "True" ]]; then
ln -fs ${EL_HOME_ODOO}/odoo ${EL_HOME}/venv/lib/python3.6/site-packages/
fi

View file

@ -2,37 +2,7 @@
. ./env_var.sh
EL_USER=$(whoami)
EL_HOME=$PWD
EL_HOME_ODOO="${EL_HOME}/odoo"
#EL_INSTALL_WKHTMLTOPDF="True"
#EL_PORT="8069"
#EL_LONGPOLLING_PORT="8072"
#EL_SUPERADMIN="admin"
EL_CONFIG_FILE="${EL_HOME}/config.conf"
#EL_CONFIG="${EL_USER}"
#EL_MINIMAL_ADDONS="False"
#EL_INSTALL_NGINX="True"
#EL_MANIFEST_PROD="./manifest/default.xml"
#EL_MANIFEST_DEV="./manifest/default.dev.xml"
./script/install_locally.sh
# Update git-repo
./script/update_manifest_prod.sh
echo -e "\n---- Install python packages/requirements ----"
${EL_HOME}/venv/bin/pip3 install --upgrade pip
${EL_HOME}/venv/bin/pip3 install wheel phonenumbers
${EL_HOME}/venv/bin/pip3 install -r "${EL_HOME}/odoo/requirements.txt"
${EL_HOME}/venv/bin/pip3 install -r "${EL_HOME}/requirements.txt"
# For dev/testing
${EL_HOME}/venv/bin/pip3 install websocket-client
echo -e "\n---- Add link dependency in site-packages of Python ----"
if [[ ${PYTHON37} = "True" ]]; then
ln -fs ${EL_HOME_ODOO}/odoo ${EL_HOME}/venv/lib/python3.7/site-packages/
elif [[ ${PYTHON36} = "True" ]]; then
ln -fs ${EL_HOME_ODOO}/odoo ${EL_HOME}/venv/lib/python3.6/site-packages/
fi

View file

@ -0,0 +1,3 @@
#!/usr/bin/env bash
source $HOME/.poetry/env
poetry add -vv $(grep -v ";" ./.venv/build_dependancy.txt | grep -v "*" | sed 's/==/@^/' )

240
script/poetry_update.py Executable file
View file

@ -0,0 +1,240 @@
#!./.venv/bin/python
import os
import sys
import argparse
import logging
import toml
from collections import OrderedDict, defaultdict
from pathlib import Path
import iscompatible
new_path = os.path.normpath(os.path.join(os.path.dirname(__file__), '..'))
sys.path.append(new_path)
from script import git_tool
_logger = logging.getLogger(__name__)
def get_config():
"""Parse command line arguments, extracting the config file name,
returning the union of config file and command line arguments
:return: dict of config file settings and command line arguments
"""
config = git_tool.GitTool.get_project_config()
parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter,
description='''\
Update pip dependancy in Poetry, clear pyproject.toml, search all dependancies
from requirements.txt, search conflict version and generate a new list.
Launch Poetry installation.
''',
epilog='''\
'''
)
parser.add_argument('-d', '--dir', dest="dir", default="./",
help="Path of repo to change remote, including submodule.")
parser.add_argument('-v', '--verbose', action="store_true",
help="More information in execution, show stats.")
args = parser.parse_args()
return args
def get_lst_requirements_txt():
return list(Path(".").rglob("requirements.[tT][xX][tT]"))
def combine_requirements(config):
"""
Search all module and version in all requirements.txt file in this project.
For each version, check compatibility and show warning with provenance file
Generate requirements.txt to "./.venv/build_dependancy.txt"
:param config:
:return:
"""
priority_filename_requirement = "odoo/requirements.txt"
lst_sign = ("==", "!=", ">=", "<=", "<", ">", ";")
lst_requirements = []
ignore_requirements = ["sys_platform == 'win32'", "python_version < '3.7'"]
dct_requirements = defaultdict(set)
lst_requirements_file = get_lst_requirements_txt()
lst_requirements_with_condition = set()
dct_special_condition = defaultdict(list)
dct_requirements_module_filename = defaultdict(list)
for requirements_filename in lst_requirements_file:
with open(requirements_filename, 'r') as f:
for a in f.readlines():
b = a.strip()
if not b or b[0] == "#":
continue
# Regroup requirement
for sign in lst_sign:
if sign in b:
# Exception, some time the sign ; can be first, just check
if sign != ";" and ";" in b and b.find(sign) > b.find(";"):
module_name = b[:b.find(";")]
else:
module_name = b[:b.find(sign)]
module_name = module_name.strip()
# Special condition for ";", ignore it
if ";" in b:
for ignore_string in ignore_requirements:
if ignore_string in b:
break
if ignore_string in b:
break
lst_requirements_with_condition.add(module_name)
value = b[:b.find(";")].strip()
dct_special_condition[module_name].append(b)
else:
value = b
dct_requirements[module_name].add(value)
filename = str(requirements_filename)
dct_requirements_module_filename[value].append(filename)
break
else:
dct_requirements[b].add(b)
filename = str(requirements_filename)
dct_requirements_module_filename[b].append(filename)
lst_requirements.append(b)
dct_requirements_diff_version = {k: v for k, v in dct_requirements.items() if
len(v) > 1}
# dct_requirements_same_version = {k: v for k, v in dct_requirements.items() if
# len(v) == 1}
if config.verbose:
# TODO show total repo to compare lst requirements
print(f"Total requirements.txt {len(lst_requirements_file)}, "
f"total module {len(lst_requirements)}, "
f"unique module {len(dct_requirements)}, "
f"module with different version {len(dct_requirements_diff_version)}.")
if dct_requirements_diff_version:
# Validate compatibility
for key, lst_requis in dct_requirements_diff_version.items():
result = None
lst_version_requis = []
for requis in lst_requis:
if ".*" in requis:
requis = requis.replace(".*", "")
result_number = iscompatible.parse_requirements(requis)
if not result_number:
# Ignore empty version
continue
# Exception of missing feature in iscompatible
# TODO support me in iscompatible lib
no_version = result_number[0][1]
if "b" in no_version:
result_number[0] = result_number[0][0], no_version[
:no_version.find("b")]
result_number = iscompatible.string_to_tuple(result_number[0][1])
lst_version_requis.append((requis, result_number))
# Check compatibility with all possibility
is_compatible = True
if len(lst_version_requis) > 1:
highest_value = sorted(lst_version_requis, key=lambda tup: tup[1])[-1]
for version_requis in lst_version_requis:
is_compatible &= iscompatible.iscompatible(version_requis[0],
highest_value[1])
if is_compatible:
result = highest_value[0]
else:
# Find the requirements file and print the conflict
# Take the version from Odoo by default, else take the more recent
odoo_value = None
for version_requis in lst_version_requis:
filename_1 = dct_requirements_module_filename.get(
version_requis[0])
if priority_filename_requirement in filename_1:
odoo_value = version_requis[0]
break
if odoo_value:
str_result_choose = f"Select {odoo_value} because from Odoo"
result = odoo_value
else:
result = highest_value[0]
str_result_choose = f"Select highest value {result}"
str_versions = " VS ".join(
[f"{a[0]} from {dct_requirements_module_filename.get(a[0])}" for
a in lst_version_requis])
print(f"WARNING - Not compatible {str_versions} - "
f"{str_result_choose}.")
elif len(lst_version_requis) == 1:
result = lst_version_requis[0][0]
else:
result = key
if result:
dct_requirements[key] = set((result,))
else:
print(f"Internal error, missing result for {lst_requis}.")
with open("./.venv/build_dependancy.txt", 'w') as f:
f.writelines([f"{list(a)[0]}\n" for a in dct_requirements.values()])
def sorted_dependancy_poetry(pyproject_filename):
# Open pyproject.toml
with open(pyproject_filename, 'r') as f:
dct_pyproject = toml.load(f)
# Get dependencies and update list, sorted
# [tool.poetry.dependencies]
tool = dct_pyproject.get("tool")
if tool:
poetry = tool.get("poetry")
if poetry:
dependencies = poetry.get("dependencies")
python_dependencie = ("python", dependencies.get("python", ''))
lst_dependency = [(k, v) for k, v in dependencies.items() if k != "python"]
lst_dependency = sorted(lst_dependency, key=lambda tup: tup[0])
poetry["dependencies"] = OrderedDict([python_dependencie] + lst_dependency)
# Rewrite pyproject.toml
with open(pyproject_filename, 'w') as f:
toml.dump(dct_pyproject, f)
def delete_dependancy_poetry(pyproject_filename):
# Open pyproject.toml
with open(pyproject_filename, 'r') as f:
dct_pyproject = toml.load(f)
# Get dependencies and update list, sorted
# [tool.poetry.dependencies]
tool = dct_pyproject.get("tool")
if tool:
poetry = tool.get("poetry")
if poetry:
dependencies = poetry.get("dependencies")
python_dependencie = ("python", dependencies.get("python", ''))
poetry["dependencies"] = OrderedDict([python_dependencie])
# Rewrite pyproject.toml
with open(pyproject_filename, 'w') as f:
toml.dump(dct_pyproject, f)
def call_poetry_add_build_dependancy():
os.system("./script/poetry_add_build_dependancy.sh")
def main():
# repo = Repo(root_path)
# lst_repo = get_all_repo()
config = get_config()
pyproject_toml_filename = f'{config.dir}pyproject.toml'
delete_dependancy_poetry(pyproject_toml_filename)
combine_requirements(config)
call_poetry_add_build_dependancy()
sorted_dependancy_poetry(pyproject_toml_filename)
if __name__ == '__main__':
main()

View file

@ -6,6 +6,6 @@
#EL_MANIFEST_DEV="./manifest/default.dev.xml"
# Update git-repo
./venv/repo init -u http://git.erplibre.ca/ERPLibre -b $(git rev-parse --abbrev-ref HEAD) -m ${EL_MANIFEST_DEV}
#./venv/repo sync --force-sync
./venv/repo sync --force-sync -v
./.venv/repo init -u http://git.erplibre.ca/ERPLibre -b $(git rev-parse --abbrev-ref HEAD) -m ${EL_MANIFEST_DEV}
#./.venv/repo sync --force-sync
./.venv/repo sync --force-sync -v

View file

@ -9,7 +9,7 @@
git daemon --base-path=. --export-all --reuseaddr --informative-errors --verbose &
DAEMON_PID=$!
./venv/repo init -u git://127.0.0.1:9418/ -b $(git rev-parse --abbrev-ref HEAD) -m ${EL_MANIFEST_DEV}
./venv/repo sync -v --force-sync -m ${EL_MANIFEST_DEV}
./.venv/repo init -u git://127.0.0.1:9418/ -b $(git rev-parse --abbrev-ref HEAD) -m ${EL_MANIFEST_DEV}
./.venv/repo sync -v --force-sync -m ${EL_MANIFEST_DEV}
kill ${DAEMON_PID}

View file

@ -10,7 +10,7 @@
git daemon --base-path=. --export-all --reuseaddr --informative-errors --verbose &
DAEMON_PID=$!
./venv/repo init -u git://127.0.0.1:9418/ -b $(git rev-parse --abbrev-ref HEAD) -m ${EL_MANIFEST_EXP}
./venv/repo sync -v --force-sync -m ${EL_MANIFEST_EXP}
./.venv/repo init -u git://127.0.0.1:9418/ -b $(git rev-parse --abbrev-ref HEAD) -m ${EL_MANIFEST_EXP}
./.venv/repo sync -v --force-sync -m ${EL_MANIFEST_EXP}
kill ${DAEMON_PID}

View file

@ -9,7 +9,7 @@
git daemon --base-path=. --export-all --reuseaddr --informative-errors --verbose &
DAEMON_PID=$!
./venv/repo init -u git://127.0.0.1:9418/ -b $(git rev-parse --abbrev-ref HEAD) -m ${EL_MANIFEST_PROD}
./venv/repo sync -v -m ${EL_MANIFEST_PROD}
./.venv/repo init -u git://127.0.0.1:9418/ -b $(git rev-parse --abbrev-ref HEAD) -m ${EL_MANIFEST_PROD}
./.venv/repo sync -v -m ${EL_MANIFEST_PROD}
kill ${DAEMON_PID}

View file

@ -6,6 +6,6 @@
#EL_MANIFEST_DEV="./manifest/default.dev.xml"
# Update git-repo
./venv/repo init -u http://git.erplibre.ca/ERPLibre -b $(git rev-parse --abbrev-ref HEAD)
#./venv/repo sync --force-sync
./venv/repo sync -v
./.venv/repo init -u http://git.erplibre.ca/ERPLibre -b $(git rev-parse --abbrev-ref HEAD)
#./.venv/repo sync --force-sync
./.venv/repo sync -v