UnifyFS Configuration
Here, we explain how users can customize the runtime behavior of UnifyFS. In particular, UnifyFS provides the following ways to configure:
Configuration file:
$INSTALL_PREFIX/etc/unifyfs/unifyfs.confEnvironment variables
Command line options to
unifyfsd
All configuration settings have corresponding environment variables, but only
certain settings have command line options. When defined via multiple methods,
the command line options have the highest priority, followed by environment
variables, and finally config file options from unifyfs.conf.
The system-wide configuration file is used by default when available.
However, users can specify a custom location for the configuration file using
the -f command-line option to unifyfsd (see below).
There is a sample unifyfs.conf file in the installation directory
under etc/unifyfs/. This file is also available in the extras directory
in the source repository.
The unified method for providing configuration control is adapted from CONFIGURATOR. Configuration settings are grouped within named sections, and each setting consists of a key-value pair with one of the following types:
BOOL:0|1,y|n,Y|N,yes|no,true|false,on|offFLOAT: scalars convertible to C double, or compatible expressionINT: scalars convertible to C long, or compatible expressionSTRING: quoted character string
System Configuration File (unifyfs.conf)
unifyfs.conf specifies the system-wide configuration options. The file is
written in INI language format, as supported by the inih parser.
The config file has several sections, each with a few key-value settings.
In this description, we use section.key as shorthand for the name of
a given section and key.
Key |
Type |
Description |
|---|---|---|
cleanup |
BOOL |
cleanup storage on server exit (default: off) |
configfile |
STRING |
path to custom configuration file |
daemonize |
BOOL |
enable server daemonization (default: off) |
mountpoint |
STRING |
mountpoint path prefix (default: /unifyfs) |
Key |
Type |
Description |
|---|---|---|
cwd |
STRING |
effective starting current working directory |
excl_private |
BOOL |
create node-local private files when given O_EXCL (default: on) |
fsync_persist |
BOOL |
persist data to storage on fsync() (default: on) |
local_extents |
BOOL |
service reads from local data (default: off) |
max_files |
INT |
maximum number of open files per client process (default: 128) |
node_local_extents |
BOOL |
service reads from node local data for laminated files (default: off) |
super_magic |
BOOL |
whether to return UNIFYFS (on) or TMPFS (off) statfs magic (default: on) |
unlink_usecs |
INT |
number of microseconds to sleep after initiating unlink rpc (default: 0) |
write_index_size |
INT |
maximum size (B) of memory buffer for storing write log metadata |
write_sync |
BOOL |
sync data to server after every write (default: off) |
The client.cwd setting is used to emulate the behavior one
expects when changing into a working directory before starting a job
and then using relative file names within the application.
If set, the application changes its working directory to
the value specified in client.cwd when unifyfs_mount() is called.
The value specified in client.cwd must be within the directory space
of the UnifyFS mount point.
Enabling the client.local_extents optimization may significantly improve
read performance for extents written by the same process. However, it should
not be used by applications in which different processes write to the same byte
offset within a file, nor should it be used with applications that truncate
files.
Key |
Type |
Description |
|---|---|---|
dir |
STRING |
path to directory to contain server log file |
file |
STRING |
log file base name (rank will be appended) |
on_error |
BOOL |
increase log verbosity upon encountering an error (default: off) |
verbosity |
INT |
logging verbosity level [0-5] (default: 0) |
Key |
Type |
Description |
|---|---|---|
chunk_size |
INT |
data chunk size (B) (default: 4 MiB) |
shmem_size |
INT |
maximum size (B) of data in shared memory (default: 256 MiB) |
spill_size |
INT |
maximum size (B) of data in spillover file (default: 4 GiB) |
spill_dir |
STRING |
path to spillover data directory |
Key |
Type |
Description |
|---|---|---|
tcp |
BOOL |
Use TCP for server-to-server rpcs (default: on, turn off to enable libfabric RMA) |
client_timeout |
INT |
timeout in milliseconds for rpcs between client and server (default: 5000) |
server_timeout |
INT |
timeout in milliseconds for rpcs between servers (default: 15000) |
Key |
Type |
Description |
|---|---|---|
dir |
STRING |
path to directory to contain server-local state |
Key |
Type |
Description |
|---|---|---|
hostfile |
STRING |
path to server hostfile |
init_timeout |
INT |
timeout in seconds to wait for servers to be ready for clients (default: 120) |
local_extents |
BOOL |
use server extents to service local reads without consulting file owner |
Key |
Type |
Description |
|---|---|---|
dir |
STRING |
path to directory to contain server shared files |
Environment Variables
All environment variables take the form UNIFYFS_SECTION_KEY, except for
the [unifyfs] section, which uses UNIFYFS_KEY. For example,
the setting log.verbosity has a corresponding environment variable
named UNIFYFS_LOG_VERBOSITY, while unifyfs.mountpoint corresponds to
UNIFYFS_MOUNTPOINT.
Command Line Options
For server command line options, we use getopt_long() format. Thus, all
command line options have long and short forms. The long form uses
--section-key=value, while the short form -<optchar> value, where
the short option character is given in the below table.
Note that for configuration options of type BOOL, the value is optional.
When not provided, the true value is assumed. If the short form option
is used, the value must immediately follow the option character (e.g., -Cyes).
LongOpt |
ShortOpt |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|