From 12ad81413059b74ba591d022c16dc0b49027bff7 Mon Sep 17 00:00:00 2001 From: Richard West Date: Fri, 28 Mar 2014 16:53:47 -0400 Subject: [PATCH 1/9] Use the https not git@ protocol to get the RMG-database in .travis.yml The builds got this far then stopped for 10 minutes and were killed: $ git clone git@github.com:GreenGroup/RMG-database.git Cloning into 'RMG-database'... The authenticity of host 'github.com (192.30.252.128)' can't be established. RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48. Are you sure you want to continue connecting (yes/no)? Hopefully the https version can continue without the RSA fingerprint check... --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 6956df7af3..9e86ec15d8 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,7 +6,7 @@ before_install: - sudo apt-get update -qq - sudo apt-get install git build-essential gfortran python-dev liblapack-dev python-openbabel python-setuptools python-pip - sudo apt-get install python-rdkit librdkit-dev librdkit1 rdkit-data - - git clone git@github.com:GreenGroup/RMG-database.git + - git clone https://github.com/GreenGroup/RMG-database.git install: - pip install numpy --use-mirrors - pip install -r requirements.txt --use-mirrors From ece7b819eeca4f5bb5f24bfd699ab3e8bb3ebe7e Mon Sep 17 00:00:00 2001 From: Richard West Date: Fri, 28 Mar 2014 17:35:01 -0400 Subject: [PATCH 2/9] Use system site_packages (and numpy/scipy) in travis config. Currently it spends thousands of lines of log file (and quite some time) compiling scipy for each build. The Ubuntu apt-get install of python-scipy should save this, but you need to enable the system site packages (instead of the isolated virtual environment) to use it. [1,2] Hopefully this (using the system site packages) also fixes the apparent failure of rdkit: current build failed with: echo "Checking you have rdkit..." Checking you have rdkit... Traceback (most recent call last): File "", line 1, in ImportError: No module named rdkit [1] http://danielnouri.org/notes/2012/11/23/use-apt-get-to-install-python-dependencies-for-travis-ci/ [2] https://github.com/dnouri/nolearn/blob/master/.travis.yml --- .travis.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.travis.yml b/.travis.yml index 9e86ec15d8..c4d756871f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,10 +2,13 @@ language: python python: - "2.6" - "2.7" +virtualenv: + system_site_packages: true before_install: - sudo apt-get update -qq - sudo apt-get install git build-essential gfortran python-dev liblapack-dev python-openbabel python-setuptools python-pip - sudo apt-get install python-rdkit librdkit-dev librdkit1 rdkit-data + - sudo apt-get install -qq python-numpy python-scipy - git clone https://github.com/GreenGroup/RMG-database.git install: - pip install numpy --use-mirrors From ae8980a4cc702349fb689e6f336e70bc3b7c0020 Mon Sep 17 00:00:00 2001 From: Richard West Date: Fri, 28 Mar 2014 18:32:58 -0400 Subject: [PATCH 3/9] Simplify travis config: no QM and no Python 2.6 Trying to get the simplest possible configuration working first, then will add complexity. --- .travis.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index c4d756871f..5e0817ea70 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,6 +1,5 @@ language: python python: - - "2.6" - "2.7" virtualenv: system_site_packages: true @@ -13,6 +12,6 @@ before_install: install: - pip install numpy --use-mirrors - pip install -r requirements.txt --use-mirrors - - make + - make noQM - pip install . script: make test From 61968677932b2caab2dddf8ba31615c51344b96c Mon Sep 17 00:00:00 2001 From: Richard West Date: Tue, 1 Apr 2014 12:54:14 -0400 Subject: [PATCH 4/9] Remove redundant parts from Travis config file. --use-mirrors has been deprecated, and pip install . likely isn't needed --- .travis.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 5e0817ea70..d3bed2a99f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -10,8 +10,8 @@ before_install: - sudo apt-get install -qq python-numpy python-scipy - git clone https://github.com/GreenGroup/RMG-database.git install: - - pip install numpy --use-mirrors - - pip install -r requirements.txt --use-mirrors + - pip install numpy + - pip install -r requirements.txt - make noQM - - pip install . script: make test + From bbf134244a4ce5cafc4e5d6092e5fc2fbd098e19 Mon Sep 17 00:00:00 2001 From: Richard West Date: Fri, 28 Mar 2014 18:59:41 -0400 Subject: [PATCH 5/9] Try using the apt-get version of matplotlib too, in travis build. This should speed up and simplify the build, if it works. --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index d3bed2a99f..b8c94b4155 100644 --- a/.travis.yml +++ b/.travis.yml @@ -7,7 +7,7 @@ before_install: - sudo apt-get update -qq - sudo apt-get install git build-essential gfortran python-dev liblapack-dev python-openbabel python-setuptools python-pip - sudo apt-get install python-rdkit librdkit-dev librdkit1 rdkit-data - - sudo apt-get install -qq python-numpy python-scipy + - sudo apt-get install -qq python-numpy python-scipy python-matplotlib - git clone https://github.com/GreenGroup/RMG-database.git install: - pip install numpy From 805c36f38e960d2f37c3b41ad80568431a760445 Mon Sep 17 00:00:00 2001 From: Richard West Date: Tue, 1 Apr 2014 07:20:46 -0400 Subject: [PATCH 6/9] Try building RDKit from source during Travis-CI testing These lines in the build script are based off the RDKit travis config https://github.com/rdkit/rdkit/blob/master/.travis.yml But modified to turn on InChI support Once this is working we can worry about cacheing the finished build to speed it up. --- .travis.yml | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index b8c94b4155..55e6ee9042 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,9 +6,32 @@ virtualenv: before_install: - sudo apt-get update -qq - sudo apt-get install git build-essential gfortran python-dev liblapack-dev python-openbabel python-setuptools python-pip - - sudo apt-get install python-rdkit librdkit-dev librdkit1 rdkit-data +# - sudo apt-get install python-rdkit librdkit-dev librdkit1 rdkit-data - sudo apt-get install -qq python-numpy python-scipy python-matplotlib - git clone https://github.com/GreenGroup/RMG-database.git + + # RDKit, based on https://github.com/rdkit/rdkit/blob/master/.travis.yml + - sudo apt-get install -qq flex bison build-essential python-numpy cmake python-dev sqlite3 libsqlite3-dev libboost-dev libboost-python-dev libboost-regex-dev python-imaging openjdk-7-jdk swig junit + - export CXX=g++ + - export CC=gcc + - git clone https://github.com/rdkit/rdkit.git + - cd rdkit + - export RDBASE=`pwd` + - export PYTHONPATH=${PYTHONPATH}:${RDBASE} + - export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${RDBASE}/lib + - cd External/INCHI-API + - ./download-inchi.sh + - cd ../../ + - mkdir build + - cd build + - mkdir $RDBASE/External/java_lib + - cp /usr/share/java/junit.jar $RDBASE/External/java_lib + - ls -l $RDBASE/External/java_lib + - cmake .. -DRDK_BUILD_INCHI_SUPPORT=ON + - make -j2 + - make install + - cd .. + - cd .. install: - pip install numpy - pip install -r requirements.txt From c44ac67e9e9ddf45c19a72804c1c0b1e85ec5866 Mon Sep 17 00:00:00 2001 From: Richard West Date: Tue, 1 Apr 2014 08:40:35 -0400 Subject: [PATCH 7/9] Travis gets and builds PyDAS and PyDQED Unlike the instructions for users in our installation guide, don't have Travis sudo the make install for PyDAS and PyDQED. 'sudo' on Ubuntu makes it discover the wrong python, which then doesn't have Cython installed, and it all goes wrong. Hopefully this clears it up. I also split into several lines, to make debugging the Travis log easier. --- .travis.yml | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 55e6ee9042..2fa37c69e1 100644 --- a/.travis.yml +++ b/.travis.yml @@ -9,7 +9,8 @@ before_install: # - sudo apt-get install python-rdkit librdkit-dev librdkit1 rdkit-data - sudo apt-get install -qq python-numpy python-scipy python-matplotlib - git clone https://github.com/GreenGroup/RMG-database.git - + - git clone https://github.com/GreenGroup/PyDAS.git + - git clone https://github.com/GreenGroup/PyDQED.git # RDKit, based on https://github.com/rdkit/rdkit/blob/master/.travis.yml - sudo apt-get install -qq flex bison build-essential python-numpy cmake python-dev sqlite3 libsqlite3-dev libboost-dev libboost-python-dev libboost-regex-dev python-imaging openjdk-7-jdk swig junit - export CXX=g++ @@ -35,6 +36,14 @@ before_install: install: - pip install numpy - pip install -r requirements.txt + - cd PyDAS + - make F77=gfortran + - make install + - cd .. + - cd PyDQED + - make F77=gfortran + - make install + - cd .. - make noQM script: make test From 7b3f3b6548291c708386fd46608d3f47c590382b Mon Sep 17 00:00:00 2001 From: Richard West Date: Tue, 1 Apr 2014 17:32:43 -0400 Subject: [PATCH 8/9] Trying to get Travis things in the right directories. - RMG-database - RPyDAS - RPyDQED - Rrdkit etc. should all live alongside, not inside, the RMG-Py directory. Hopefully this works solves the database problem --- .travis.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.travis.yml b/.travis.yml index 2fa37c69e1..022858ed54 100644 --- a/.travis.yml +++ b/.travis.yml @@ -8,6 +8,7 @@ before_install: - sudo apt-get install git build-essential gfortran python-dev liblapack-dev python-openbabel python-setuptools python-pip # - sudo apt-get install python-rdkit librdkit-dev librdkit1 rdkit-data - sudo apt-get install -qq python-numpy python-scipy python-matplotlib + - cd .. - git clone https://github.com/GreenGroup/RMG-database.git - git clone https://github.com/GreenGroup/PyDAS.git - git clone https://github.com/GreenGroup/PyDQED.git @@ -34,8 +35,10 @@ before_install: - cd .. - cd .. install: + - cd $TRAVIS_BUILD_DIR - pip install numpy - pip install -r requirements.txt + - cd .. - cd PyDAS - make F77=gfortran - make install @@ -44,6 +47,7 @@ install: - make F77=gfortran - make install - cd .. + - cd $TRAVIS_BUILD_DIR - make noQM script: make test From 48b06e9cc928a46d6ea8f55f6ee76a40eb2f5909 Mon Sep 17 00:00:00 2001 From: Richard West Date: Mon, 31 Mar 2014 19:01:50 -0400 Subject: [PATCH 9/9] Try running eg1 as well as make test in travis testing. --- .travis.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 022858ed54..d2ff12da4f 100644 --- a/.travis.yml +++ b/.travis.yml @@ -49,5 +49,6 @@ install: - cd .. - cd $TRAVIS_BUILD_DIR - make noQM -script: make test - +script: + - make test + - make eg1