Build & I/O Interception¶
In this section, we describe how to build UnifyFS with I/O interception.
The current version of UnifyFS adopts the mdhim key-value store, which strictly requires:
“An MPI distribution that supports MPI_THREAD_MULTIPLE and per-object locking of critical sections (this excludes OpenMPI up to version 3.0.1, the current version as of this writing)”
as specified in the project github.
How to Build UnifyFS¶
To install all dependencies and set up your build environment, we recommend using the Spack package manager. If you already have Spack, make sure you have the latest release or if using a clone of their develop branch, ensure you have pulled the latest changes.
Building with Spack¶
These instructions assume that you do not already have a module system installed such as LMod, Dotkit, or Environment Modules. If your system already has Dotkit or LMod installed then installing the environment-modules package with Spack is unnecessary (so you can safely skip that step).
If you use Dotkit then replace
spack load with
First, install Spack if you don’t already have it:
$ git clone https://github.com/spack/spack $ ./spack/bin/spack install environment-modules $ . spack/share/spack/setup-env.sh
Make use of Spack’s shell support
to automatically add Spack to your
PATH and allow the use of the
Then install UnifyFS:
$ spack install unifyfs $ spack load unifyfs
Include or remove variants with Spack when installing UnifyFS when a custom
build is desired. Type
spack info unifyfs for more info.
Build with parallel HDF5
Build with serial HDF5
||Build with gfortran|
||Build with NUMA|
||Enable PMI2 build options|
||Enable PMIx build options|
The initial install could take a while as Spack will install build
dependencies (autoconf, automake, m4, libtool, and pkg-config) as well as
any dependencies of dependencies (cmake, perl, etc.) if you don’t already
have these dependencies installed through Spack or haven’t told Spack where
they are locally installed on your system (i.e., through a custom
spack spec -I unifyfs before installing to see what Spack is going
Building with Autotools¶
Download the latest UnifyFS release from the Releases page.
Building the Dependencies¶
UnifyFS requires MPI, LevelDB, GOTCHA(version 0.0.2), FlatCC, and Margo. References to these dependencies can be found here.
Build the Dependencies with Spack¶
Once Spack is installed on your system (see above), you can install just the dependencies for an easier manual installation of UnifyFS.
If you use Dotkit then replace
spack load with
$ spack install leveldb $ spack install email@example.com $ spack install flatcc $ spack install margo
You can use
spack install --only=dependencies unifyfs to install all of
UnifyFS’s dependencies without installing UnifyFS.
Keep in mind this will also install all the build dependencies and dependencies of dependencies if you haven’t already installed them through Spack or told Spack where they are locally installed on your system.
Then to manually build UnifyFS:
$ spack load leveldb $ spack load firstname.lastname@example.org $ spack load flatcc $ spack load mercury $ spack load argobots $ spack load margo $ $ ./autogen.sh $ ./configure --prefix=/path/to/install $ make $ make install
To build with gfortran compatibility, include the
./configure --prefix=/path/to/install/ --enable-fortran
To see all available build configuration options, type
./autogen.sh has been run.
Build the Dependencies without Spack¶
For users who cannot use Spack, a bootstrap.sh script has been provided in order to make manual build and installation of dependencies easier. Simply run the script in the top level directory of the source code.
References to the UnifyFS dependencies can be found here.
After bootstrap.sh is finished building the dependencies, it will print out the commands you need to run to build UnifyFS. The commands look something like this:
$ export PKG_CONFIG_PATH=path/to/mercury/lib/pkgconfig:path/to/argobots/lib/pkgconfig:path/to/margo/lib/pkgconfig $ ./autogen.sh $ ./configure --prefix=/path/to/install --with-gotcha=/path/to/gotcha --with-leveldb=/path/to/leveldb --with-flatcc=/path/to/flatcc $ make $ make install
You may need to add the following to your configure line if it is not in your default path on a linux machine:
This is needed to enable NUMA-aware memory allocation on Linux machines. Set the
NUMA policy at runtime with
UNIFYFS_NUMA_POLICY = local | interleaved, or set
NUMA nodes explicitly with
UNIFYFS_USE_NUMA_BANK = <node no.>
POSIX calls can be intercepted via the methods described below.
Steps for static linking using –wrap:
To intercept I/O calls using a static link, you must add flags to your link line. UnifyFS installs a unifyfs-config script that returns those flags, e.g.,
$ mpicc -o test_write \ `<unifyfs>/bin/unifyfs-config --pre-ld-flags` \ test_write.c \ `<unifyfs>/bin/unifyfs-config --post-ld-flags`
Steps for dynamic linking using gotcha:
To intercept I/O calls using gotcha, use the following syntax to link an application.
$ mpicc -o test_write test_write.c \ -I<unifyfs>/include -L<unifycy>/lib -lunifyfs_gotcha \ -L<gotcha>/lib64 -lgotcha
$ mpif90 -o test_write test_write.F \ -I<unifyfs>/include -L<unifycy>/lib -lunifyfsf -lunifyfs_gotcha