craneium.net

brain matters . . .

  • Increase font size
  • Default font size
  • Decrease font size
Home Python Programming Compiling PyTables, parallel iPython, and Trilinos 10 for PyTrilinos LOCALLY From Almost Nothing

Compiling PyTables, parallel iPython, and Trilinos 10 for PyTrilinos LOCALLY From Almost Nothing

E-mail Print PDF

(Update iPython parallel is now very different: See Here For More)

 

This is instructions for how to do a LOCAL install of parallel ipython, pyTables, and Trilinos 10 for use with pytrilinos on a SGI Altix HPC with only the following available:

gccgfortranblaslapackfftw, tar, wget

My parallel n-dimensional fluid dynamics solver relies on pyTables, parallel ipython, and now pyTrilinos so here is how to download/compile/build/install everything needed!  This is a rough chart of the dependencies running around:

General notes:

These instructions are designed for a local install, meaning you don't have root access to the machine in question.  I use the path of "/home/u27/mgoodman/POST" but any directory that you have write access to will do.

I assume that after wget'ing a particular package, you can untar and change into the resulting directory, so I omit this instruction in almost all steps.

The make steps can be accelerated by the addition of the "-j" flag on computers with more than one core.  This is especially key during the Trilinos make step (which can take over a couple hours and then fail)

 

Installing zlib with shared libraries:

$wget http://www.zlib.net/zlib-1.2.3.tar.gz

$./configure --prefix=/home/u27/mgoodman/POST 

$make

$make install

Now we need to make the shared libraries!

$make clean

$./configure -s --prefix=/home/u27/mgoodman/POST

$make

$make install

Installing bz2 including shared objects:

$wget http://www.bzip.org/1.0.5/bzip2-1.0.5.tar.gz

$./configure

$make

$make -f Makefile-libbz2_so

$make install PREFIX=/home/u27/mgoodman/POST

At this point I added the following lines to my ".profile" file:

PATH=/home/u27/mgoodman/POST/bin:$PATH

INCLUDE=/home/u27/mgoodman/POST/include:$INCLUDE

LD_LIBRARY_PATH=/home/u27/mgoodman/POST/lib:$LD_LIBRARY_PATH

Restart your shell session to ensure the libraries, includes and binaries we create are preferred to any (probably wrong) ones that are already installed

Compiling and Installing python2.6:

$wget http://python.org/ftp/python/2.6.4/Python-2.6.4.tar.bz2

$./configure --prefix=/home/u27/mgoodman/POST

$make

$make install

Huzzah!  Not even halfway there though!

 

Installing setuptools:

$wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz

$python setup.py config

$python setup.py build

$python setup.py install

 

Installing ipython dependancies for parallel computing:

$easy_install zope.Interface
$easy_install Twisted
$easy_install nose
$easy_install sphinx
$easy_install Foolscap
$easy_install pexpect

 

Installing OpenSSL for pyOpenSSL

$wget http://pypi.python.org/packages/source/p/pyOpenSSL/pyOpenSSL-0.10.tar.gz

$./config --prefix=/home/u27/mgoodman/POST

$make

$make install

$easy_install pyOpenSSL

 

Compiling and Installing IPython:

$wget http://ipython.scipy.org/dist/0.10/ipython-0.10.tar.gz

$python setup.py config

This should produce output that looks like:

BUILDING IPYTHON

python: 2.6.4 (r264, Jan 17 2010, 17:21:54)  [GCC 4.4.0]

platform: linux2

OPTIONAL DEPENDENCIES

Zope.Interface: yes

Twisted: 9.0.0

Foolscap: 0.4.2

OpenSSL: 0.10

sphinx: 0.6.4

pygments: 1.2.2

nose: 0.11.1

pexpect: 2.4

If any of the optional modules are missing the parallel computing functionalities will not be present!  Now build and install:

$python setup.py build

$python setup.py install

 

Installing Numpy:

$wget http://sourceforge.net/projects/numpy/files/NumPy/1.4.0/numpy-1.4.0.tar.gz/download
$python setup.py config
(make sure it finds BLAS/LAPACK)
$python setup.py build
$python setup.py install

 

Installing Swig:

$wget http://prdownloads.sourceforge.net/swig/swig-1.3.40.tar.gz

$./configure

$make

$make install

Installing SuiteSparse (contains UMFPACK)

$wget http://www.cise.ufl.edu/research/sparse/SuiteSparse/current/SuiteSparse.tar.gz

in "UFconfig/UFconfig.mk"
Replace
"CHOLMOD_CONFIG ="
with
"CHOLMOD_CONFIG = -DNPARTITION"
And replace
"CFLAGS ="
with
"CFLAGS = -O3 -fexceptions -ffast-math -funroll-loops -fPIC"

If you need to do direct solution of large matrices (1M+ DOF) consider adding the -DL64 flag (this will slow smaller matrix solutions though and at this size you should use the Trilinos iterative solvers)

In the main directory of SuiteSparse:

$make

(It failed to build the "demos" but everything else seemed to work fine)

$make install PREFIX=/home/u27/mgoodman/POST

 

Compiling and Installing Scipy:

Scipy is not strictly necessary for trilinos, but if it would be a strange situation where you are using one and not the other!

$wget http://sourceforge.net/projects/scipy/files/scipy/0.7.1/scipy-0.7.1.tar.gz/download

Untar and edit the site.cfg.example and save as site.cfg

The section regarding fftw, umfpack, and amd need to be edited.  The lines changed for me look like the following:

[amd]

library_dirs = /home/u27/mgoodman/local-install/SuiteSparse/AMD/Lib

include_dirs = /home/u27/mgoodman/local-install/SuiteSparse/AMD/Include/

amd_libs = amd

 

[umfpack]

library_dirs = /home/u27/mgoodman/local-install/SuiteSparse/UMFPACK/Lib/

include_dirs = /home/u27/mgoodman/local-install/SuiteSparse/UMFPACK/Include/

umfpack_libs = umfpack

Run:

$python setup.py config

Make sure it finds UMFPACK includes and libraries.  The output should look something like this:

lapack_info:

libraries lapack not found in /home/u27/mgoodman/POST/lib

libraries lapack not found in /usr/local/lib64

FOUND:

libraries = ['lapack']

library_dirs = ['/usr/local/lib']

language = f77

 

FOUND:

libraries = ['lapack', 'blas']

library_dirs = ['/usr/local/lib']

define_macros = [('NO_ATLAS_INFO', 1)]

language = f77

 

umfpack_info:

amd_info:

FOUND:

libraries = ['amd']

library_dirs = ['/home/u27/mgoodman/local-install/SuiteSparse/AMD/Lib']

swig_opts = ['-I/home/u27/mgoodman/local-install/SuiteSparse/AMD/Include']

define_macros = [('SCIPY_AMD_H', None)]

include_dirs = ['/home/u27/mgoodman/local-install/SuiteSparse/AMD/Include']

 

FOUND:

libraries = ['umfpack', 'amd']

library_dirs = ['/home/u27/mgoodman/local-install/SuiteSparse/UMFPACK/Lib/', '/home/u27/mgoodman/local-install/SuiteSparse/AMD/Lib']

swig_opts = ['-I/home/u27/mgoodman/local-install/SuiteSparse/UMFPACK/Include', '-I/home/u27/mgoodman/local-install/SuiteSparse/AMD/Include']

define_macros = [('SCIPY_UMFPACK_H', None), ('SCIPY_AMD_H', None)]

include_dirs = ['/home/u27/mgoodman/local-install/SuiteSparse/UMFPACK/Include', '/home/u27/mgoodman/local-install/SuiteSparse/AMD/Include']

If that looks good proceed with:

$python setup.py build

(this will take a while)

$python setup.py install

Compiling and Installing hdf5 libraries:

$wget http://www.hdfgroup.org/ftp/HDF5/current16/src/hdf5-1.6.10.tar.gz

$./configure --prefix=/home/u27/mgoodman/POST

$make

$make install

 

Compiling and Installing CMake:

$wget http://www.cmake.org/files/v2.8/cmake-2.8.0.tar.gz

untar and enter directory

$./configure --prefix=/home/u27/mgoodman/POST

$make

$make install

 

Compiling and Installing pytables:

For whatever reason this particular installer ignores the standard INCLUDE and LD_LIBRARY_PATH environment vars, so we need to do some hacking if we want a local install to work.

Open the setup.py and make some changes . . .

In the section starting with:

if os.name == 'posix':

default_header_dirs = ['/usr/include', '/usr/local/include', '/home/u27/mgoodman/POST/include']

default_library_dirs = [

os.path.join(_tree, _arch)

for _tree in ('/', '/usr', '/usr/local')

for _arch in ('lib64', 'lib') ]

default_library_dirs.append('/home/u27/mgoodman/POST/lib')

default_runtime_dirs = default_library_dirs

 

Now we can build install properly:

$python setup.py config --cflags="-I/home/u27/mgoodman/POST/include/" --lflags="-L/home/u27/mgoodman/POST/lib"

$python setup.py build --cflags="-I/home/u27/mgoodman/POST/include/" --lflags="-L/home/u27/mgoodman/POST/lib"

$python setup.py install --cflags="-I/home/u27/mgoodman/POST/include/" --lflags="-L/home/u27/mgoodman/POST/lib" --prefix=/home/u27/mgoodman/POST/

 

Compiling and Installing Trilinos 10:

$wget http://trilinos.sandia.gov/download/files/trilinos-10.0.4-Source.tar.gz

Make sure you have a good amount of disk space, this build takes on the order of 1.5Gb, so I had to make some disk space arrangements on my allocation.
Create a file named config-trilinos.sh containing:

cmake \

-D BUILD_SHARED_LIBS=ON   \

-D CMAKE_INSTALL_PREFIX=/home/u27/mgoodman/POST \

-D CMAKE_BUILD_TYPE:STRING=DEBUG \

-D TPL_ENABLE_MPI:BOOL=ON \

-D Trilinos_ENABLE_DEFAULT_PACKAGES:BOOL=OFF \

-D Trilinos_ENABLE_PyTrilinos:BOOL=ON \

-D Trilinos_ENABLE_Meros:BOOL=ON \

-D Trilinos_ENABLE_Epetra:BOOL=ON \

-D Trilinos_ENABLE_EpetraExt:BOOL=ON \

-D Trilinos_ENABLE_Belos:BOOL=ON \

-D Trilinos_ENABLE_Ifpack:BOOL=ON \

-D Trilinos_ENABLE_Teuchos:BOOL=ON \

-D Trilinos_ENABLE_Amesos:BOOL=ON \

-D Trilinos_ENABLE_Gallery:BOOL=ON \

-D Trilinos_ENABLE_ML:BOOL=ON \

-D Trilinos_ENABLE_ForTrilinos:BOOL=ON \

-D Trilinos_ENABLE_TESTS:BOOL=ON \

-D Teuchos_ENABLE_MPI:BOOL=ON \

-D Teuchos_ENABLE_ABC:BOOL=ON \

-D Teuchos_ENABLE_COMPLEX:BOOL=ON \

-D Teuchos_ENABLE_EXTENDED:BOOL=ON \

-D DART_TESTING_TIMEOUT:STRING=600 \

-D Trilinos_EXTRA_LINK_FLAGS:STRING="$FORTRAN_LIBRARIES" \

-D CMAKE_VERBOSE_MAKEFILE:BOOL=TRUE       \

-D Trilinos_VERBOSE_CONFIGURE:BOOL=TRUE   \

.

So finally

$bash config-trilinos.sh

$make

$make install

Now go get some real work done!
Last Updated on Thursday, 19 May 2011 18:09