NYSGrid: User Documentation

Sample Scripts and Examples

Here is an example on how to test a local and remote install of the OSG:

-bash-2.05b$ uname -a
Linux u2-grid.ccr.buffalo.edu 2.4.21-47.0.1.ELsmp #1 SMP Fri Oct 13 17:56:20 EDT 2006 i686 i686 i386 GNU/Linux

-bash-2.05b$ source /opt/grid/setup.sh

-bash-2.05b$ grid-proxy-init

Your identity: /DC=org/DC=doegrids/OU=People/CN=Jon Bednasz 258078
    Enter GRID pass phrase for this identity:
    Creating proxy ....................................... Done
    Your proxy is valid until: Thu Feb 1 01:38:05 2007

-bash-2.05b$ /opt/grid/verify/site_verify.pl

============================================
Info: Site verification initiated at Wed Jan 31 18:38:43 2007 GMT.
============================================
-------------------------------------------------------------------------------
-------- Begin u2-grid.ccr.buffalo.edu at Wed Jan 31 18:38:43 2007 GMT --------
-------------------------------------------------------------------------------
Checking prerequisites needed for testing: PASS
Checking for a valid proxy for jbednasz@u2-grid.ccr.buffalo.edu: PASS
Checking if remote host is reachable: PASS
Checking for a running gatekeeper: YES; port 2119
Checking authentication: PASS
Checking 'Hello, World' application: PASS
Checking remote host uptime: PASS
13:38:44 up 19 days, 6 min, 9 users, load average: 37.95, 29.88, 26.38
Checking remote Internet network services list: PASS
Checking remote Internet servers database configuration: PASS
Checking for GLOBUS_LOCATION: /opt/grid/globus
Checking expiration date of remote host certificate: Jun 22 14:19:21 2007 GMT
Checking for gatekeeper configuration file: YES
/opt/grid/globus/etc/globus-gatekeeper.conf
Checking for a running gsiftp server: YES; port 2811
Checking gsiftp (local client, local host -> remote host): PASS
Checking gsiftp (local client, remote host -> local host): PASS
Checking that no differences exist between gsiftp'd files: PASS
Checking users in grid-mapfile, if none must be using Prima:
    des,dosar,fermilab,fmri,gadu,geant4,glow,grase,
    gridex,grow,ivdgl,ligo,mariachi,mis7,nanohub,osg,
    osgedu,sam,samgrid,sdss,star,uscms01
Checking for remote globus-sh-tools-vars.sh: YES
Checking configured grid services: PASS
jobmanager,jobmanager-fork,jobmanager-managedfork,jobmanager-pbs
Checking scheduler types associated with remote jobmanagers: PASS
jobmanager is of type fork
jobmanager-fork is of type fork
jobmanager-managedfork is of type managedfork
jobmanager-pbs is of type pbs
Checking for paths to binaries of remote schedulers: PASS
Path to managedfork binaries is .
Path to pbs binaries is /usr/bin
Checking remote scheduler status: PASS
pbs : 416 jobs running, 330 jobs idle/pending
Checking for a running MDS service: YES; port 2135
Checking for configured Generic Information Provider service: NO; GLUE attributes absent. Detailed validation information at http://grow.its.uiowa.edu/osg-gip/
Checking if Globus is deployed from the VDT: YES; version 1.3.10b
Checking for OSG osg-attributes.conf: YES
Checking for OSG grid3-user-vo-map.txt: YES
ivdgl users: ivdgl
i2u2 users: i2u2
geant4 users: geant4
grow users: grow
osgedu users: osgedu
nanohub users: nanohub
gridex users: gridex
fmri users: fmri
cdf users: cdf
osg users: osg
mariachi users: mariachi
star users: star
dosar users: dosar
uscms users: uscms01
grase users: grase
ligo users: ligo
glow users: glow
fermilab users: fermilab
dzero users: sam,samgrid
mis users: mis7
des users: des
sdss users: sdss
gadu users: gadu
Checking for OSG site name: GRASE-CCR-U2
Checking for OSG $GRID3 definition: /opt/grid
Checking for OSG $APP definition: /san/scratch/grid/grid-tmp/grid-apps
Checking for OSG $DATA definition: /san/scratch/grid/grid-tmp/grid-data
Checking for OSG $TMP definition: /san/scratch/grid/grid-tmp/grid-data
Checking for OSG $WNTMP definition: /san/scratch/grid/grid-tmp
Checking for OSG $APP existence: PASS
Checking for OSG $DATA existence: PASS
Checking for OSG $TMP existence: PASS
Checking for OSG $APP writability: PASS
Checking for OSG $DATA writability: PASS
Checking for OSG $TMP writability: PASS
Checking for OSG $APP available space: 1859.152 GB
Checking for OSG $DATA available space: 1859.152 GB
Checking for OSG $TMP available space: 1859.152 GB
Checking for OSG additional site-specific variable definitions: YES
MountPoints
SAMPLE_LOCATION default /SAMPLE-path
SAMPLE_SCRATCH devel /SAMPLE-path
VO-cms-CMSSW_1_2_0 CMSSW_1_2_0 /san/scratch/grid/grid-tmp/grid-apps/cmssoft/cms
Checking for OSG execution jobmanager(s): u2-grid.ccr.buffalo.edu/jobmanager-pbs
Checking for OSG utility jobmanager(s): u2-grid.ccr.buffalo.edu/jobmanager
Checking for OSG sponsoring VO: GRASE
Checking for OSG policy expression: NONE
Checking for OSG setup.sh: YES
Checking for OSG $Monalisa_HOME definition: /opt/grid/MonaLisa
Checking for MonALISA configuration: PASS
key ml_env vars:
FARM_NAME = GRASE-CCR-U2
FARM_HOME = /opt/grid/MonaLisa/Service/VDTFarm
FARM_CONF_FILE = /opt/grid/MonaLisa/Service/VDTFarm/vdtFarm.conf
SHOULD_UPDATE = false
URL_LIST_UPDATE = http://monalisa.cacr.caltech.edu/FARM_ML,http://monalisa.cern.ch/MONALISA/FARM_ML
key ml_properties vars:
lia.Monitor.group = OSG
lia.Monitor.useIPaddress = undef
MonaLisa.ContactEmail = smgallo@ccr.buffalo.edu
Checking for a running MonALISA: PASS
MonALISA is ALIVE (pid 2960)
MonALISA_Version = 1.4.12-200603031824
MonALISA_VDate = 2006-03-03
VoModulesDir = VoModules-v0.20
tcpServer_Port = 9002
storeType = epgsqldb
Checking for a running GANGLIA gmond daemon: NO
gmond does not appear to be running
Checking for a running GANGLIA gmetad daemon: NO
gmetad does not appear to be running
-------------------------------------------------------------------------------
--------- End u2-grid.ccr.buffalo.edu at Wed Jan 31 18:39:56 2007 GMT ---------
-------------------------------------------------------------------------------
==============================================
Info: Site verification completed at Wed Jan 31 18:39:56 2007 GMT.

Now let's test a remote site at Binghamton:

-bash-2.05b$ /opt/grid/verify/site_verify.pl --host=rommel.cs.binghamton.edu

============================================
Info: Site verification initiated at Wed Jan 31 18:41:22 2007 GMT.
============================================
-------------------------------------------------------------------------------
-------- Begin rommel.cs.binghamton.edu at Wed Jan 31 18:41:22 2007 GMT -------
-------------------------------------------------------------------------------
Checking prerequisites needed for testing: PASS
Checking for a valid proxy for jbednasz@u2-grid.ccr.buffalo.edu: PASS
Checking if remote host is reachable: PASS
Checking for a running gatekeeper: YES; port 2119
Checking authentication: PASS
Checking 'Hello, World' application: PASS
Checking remote host uptime: PASS
13:41:28 up 54 days, 23:08, 0 users, load average: 0.00, 0.00, 0.00
Checking remote Internet network services list: PASS
Checking remote Internet servers database configuration: PASS
Checking for GLOBUS_LOCATION: /opt/osg/globus
Checking expiration date of remote host certificate: Jan 26 01:19:47 2008 GMT
Checking for gatekeeper configuration file: YES
/opt/osg/globus/etc/globus-gatekeeper.conf
Checking for a running gsiftp server: YES; port 2811
Checking gsiftp (local client, local host -> remote host): PASS
Checking gsiftp (local client, remote host -> local host): PASS
Checking that no differences exist between gsiftp'd files: PASS
Checking users in grid-mapfile, if none must be using Prima: grase,mis
Checking for remote globus-sh-tools-vars.sh: YES
Checking configured grid services: PASS
jobmanager,jobmanager-fork,jobmanager-pbs
Checking scheduler types associated with remote jobmanagers: PASS
jobmanager is of type fork
jobmanager-fork is of type fork
jobmanager-pbs is of type pbs
Checking for paths to binaries of remote schedulers: PASS
Path to pbs binaries is /usr/local/bin
Checking remote scheduler status: PASS
pbs : 0 jobs running, 1 jobs idle/pending
Checking for a running MDS service: YES; port 2135
Checking for configured Generic Information Provider service: NO; GLUE attributes absent. Detailed validation information at http://grow.its.uiowa.edu/osg-gip/
Checking if Globus is deployed from the VDT: YES; version 1.3.10d
Checking for OSG osg-attributes.conf: YES
Checking for OSG grid3-user-vo-map.txt: YES
ops users: ops
ivdgl users: ivdgl
i2u2 users: i2u2
geant4 users: geant4
grow users: grow
osgedu users: osgedu
nanohub users: nanohub
gridex users: gridex
gpn users: gpn
fmri users: fmri
cdf users: cdf
nwicg users: nwicg
osg users: osg
engage users: engage
usatlas users: usatlas1,usatlas2,usatlas3,usatlas4
mariachi users: mariachi
star users: star
dosar users: dosar
uscms users: uscms01
grase users: grase
ligo users: ligo
glow users: glow
fermilab users: fermilab
dzero users: sam,samgrid
gugrid users: gugrid
compbiogrid users: compbiogrid
mis users: mis
des users: des
sdss users: sdss
gadu users: gadu
Checking for OSG site name: GRASE-BINGHAMTON
Checking for OSG $GRID3 definition: /opt/osg
Checking for OSG $APP definition: /opt/grid3_share/apps
Checking for OSG $DATA definition: /opt/grid3_share/data
Checking for OSG $TMP definition: /opt/grid3_share/data
Checking for OSG $WNTMP definition: /opt/grid3_share/data
Checking for OSG $APP existence: PASS
Checking for OSG $DATA existence: PASS
Checking for OSG $TMP existence: PASS
Checking for OSG $APP writability: PASS
Checking for OSG $DATA writability: PASS
Checking for OSG $TMP writability: PASS
Checking for OSG $APP available space: 2067.728 GB
Checking for OSG $DATA available space: 2067.728 GB
Checking for OSG $TMP available space: 2067.728 GB
Checking for OSG additional site-specific variable definitions: YES
MountPoints
ATLAS_APP prod /opt/grid3_share/apps/atlas_app
ATLAS_DATA prod /opt/grid3_share/data/atlas_data
ATLAS_LOC_GCC 3.2 /opt/grid3_share/apps/atlas_app/gcc32
ATLAS_LOC_GCE prod /opt/grid3_share/apps/atlas_app/GCE-Server/gce-server
ATLAS_LOC_KitVal prod /opt/grid3_share/apps/atlas_app/atlas_rel/kitval/KitValidation
ATLAS_LOC_Trfs prod /opt/grid3_share/apps/atlas_app/Atlas-Trfs/atlas-trfs
ATLAS_STAGE prod /opt/grid3_share/data/atlas_data
SAMPLE_LOCATION default /SAMPLE-path
SAMPLE_SCRATCH devel /SAMPLE-path
Checking for OSG execution jobmanager(s): rommel/jobmanager-pbs
Checking for OSG utility jobmanager(s): rommel/jobmanager
Checking for OSG sponsoring VO: GRASE
Checking for OSG policy expression: NONE
Checking for OSG setup.sh: YES
Checking for OSG $Monalisa_HOME definition: /opt/osg/MonaLisa
Checking for MonALISA configuration: PASS
key ml_env vars:
FARM_NAME = GRASE-BINGHAMTON
FARM_HOME = /opt/osg/MonaLisa/Service/VDTFarm
FARM_CONF_FILE = /opt/osg/MonaLisa/Service/VDTFarm/vdtFarm.conf
SHOULD_UPDATE = false
URL_LIST_UPDATE = http://monalisa.cacr.caltech.edu/FARM_ML,http://monalisa.cern.ch/MONALISA/FARM_ML
key ml_properties vars:
lia.Monitor.group = OSG
lia.Monitor.useIPaddress = undef
MonaLisa.ContactEmail = smgallo@ccr.buffalo.edu
Checking for a running MonALISA: NO
MonALISA does not appear to be running
Checking for a running GANGLIA gmond daemon: NO
gmond does not appear to be running
Checking for a running GANGLIA gmetad daemon: NO
gmetad does not appear to be running
-------------------------------------------------------------------------------
--------- End rommel.cs.binghamton.edu at Wed Jan 31 18:42:22 2007 GMT --------
-------------------------------------------------------------------------------
=================================================
Info: Site verification completed at Wed Jan 31 18:42:22 2007 GMT.

Now let's run a sample grid job:

-bash-2.05b$ mkdir grid-sample
-bash-2.05b$ cd grid-sample/
-bash-2.05b$ ls -ltr
total 24
-rw-r--r-- 1 jbednasz ccrstaff 1387 Jan 31 13:35 counter.c
-rwxr-xr-x 1 jbednasz ccrstaff 1331 Jan 31 13:35 run_grid_job.sh
-rw-r--r-- 1 jbednasz ccrstaff 20 Jan 31 13:35 sum-file.txt

-bash-2.05b$ cat counter.c

/*
* Simple program to read integers from an input file (one per line), add
* them up, and display the total and the name of the current host.
*
* Steve Gallo, Center for Computational Research, SUNY-Buffalo
*/

#include
#include
#include
#include

extern int errno;

int
main(int argc, char *argv[])
{
/* Initialize variables */

FILE *fdIn = NULL, *fdOut = NULL;
int input = 0, count = 0;
char hostname[64];

/* Verify arguments and input file */

if ( 3 != argc )
{
fprintf(stderr,
"Usage: %s ",
argv[0]);
exit(1);
}

/* Open the input file and read each line */

fdIn = fopen(argv[1], "r");
if ( NULL == fdIn )
{
fprintf(stderr,
"Error opening input file %s: %s ",
argv[0], strerror(errno));
exit(1);
}

fscanf(fdIn, "%d", &input);
while ( ! feof(fdIn) )
{
count += input;
fscanf(fdIn, "%d", &input);
}
fclose(fdIn);

/* Write the output file */

fdOut= fopen(argv[2], "w");
if ( NULL == fdOut )
{
fprintf(stderr,
"Error opening output file %s: %s ",
argv[0], strerror(errno));
exit(1);
}

gethostname(hostname, 64);
fprintf(fdOut, "Hostname = %s ", hostname);
fprintf(fdOut, "count = %d ", count);
fclose(fdOut);

exit (0);

} /* main() */

-bash-2.05b$ cat run_grid_job.sh

#!/bin/sh
#
# Example script for staging and running a job on the grid using
# globus. It is assumed that grid-proxy-init has already been run.
#
# Steve Gallo, Center for Computational Research, SUNY-Buffalo
# ----------------------------------------------------------------------

# Verify arguments and input file

if [ $# -ne 3 ]; then
echo "Usage: $0 "
exit 1
fi

if [ ! -f $2 ]; then
echo "Error reading input file: $1"
exit 1
fi

# Initialize filenames

host=$1
input_file=$2
output_file=$3
run_directory=`globus-job-run ${host} /bin/sh -c 'echo $OSG_DATA'`

echo
echo "Remote directory: ${run_directory}"

# Push executable

echo
echo "Staging the executable to the grid"

globus-url-copy file://`pwd`/counter gsiftp://${host}/${run_directory}/counter
globus-job-run ${host} -dir ${run_directory} /bin/sh -c "chmod u+x counter"

# Push data, run job, pull data back

echo
echo "Running the counter"

globus-url-copy file://`pwd`/${input_file} gsiftp://${host}/${run_directory}/${input_file}
globus-job-run ${host} -dir ${run_directory} /bin/sh -c "./counter ${input_file} ${output_file}"
globus-url-copy gsiftp://${host}/${run_directory}/${output_file} file://`pwd`/${output_file}

echo
echo "Results"
echo
cat -n ${output_file}
echo

# Cleanup

/bin/rm ${output_file}

-bash-2.05b$ cat sum-file.txt
1
2
3
4
5
65
6
6
67

-bash-2.05b$ cc counter.c -o counter

-bash-2.05b$ ./run_grid_job.sh rommel.cs.binghamton.edu sum-file.txt sum-file.out

Remote directory: /opt/grid3_share/data

Staging the executable to the grid

Running the counter

Results

1 Hostname = rommel
2
3 count = 159