Difference between revisions of "Tutorial - Users 1 - Command line interface"

From Madagascar
Jump to navigation Jump to search
Line 1: Line 1:
Madagascar is designed to be used from the command line.  Programmers create Madagascar programs (prefixed with the sf designation) that read and write Madagascar files.  These programs are designed to be as general as possible, each one operates on any dataset provided in RSF format, provided you also provide the correct parameters and have the right type of data for the program.  Thus, each Madagascar program can be used in even ways that the original developer may not have foreseen, which makes your life easier as a user!
+
Madagascar was designed initially to be used from the command line.  Programmers create Madagascar programs (prefixed with the sf designation) that read and write Madagascar files.  These programs are designed to be as general as possible, each one operates on any dataset in RSF format, provided you also supply the correct parameters and have the right type of data for the program.  Thus, each Madagascar program can be used in different ways that the original developer may not have foreseen, which makes your life easier as a user!
  
 
= Command line interface =  
 
= Command line interface =  
Madagascar programs follow the standard UNIX conventions for reading and writing RSF files to and from STDIN and STDOUT.  This is also the same convention that SeismicUnix uses, so for those of you who are SU fans, this should be an easy conversion.   
+
Madagascar programs follow the standard UNIX conventions for reading and writing RSF files to and from STDIN and STDOUT.  This is also the same convention that [http://www.cwp.mines.edu/cwpcodes/ Seismic Unix] uses, so for those of you who are SU fans, this should be an easy conversion.   
  
For example: the program sfattr allows us to get attributes about an RSF file (mean, standard deviation, min, max, etc.)  To get the attributes for a pre-computed file, we might use:
+
For example: the program [[Guide_to_madagascar_programs#sfattr|sfattr]] allows us to get attributes about an RSF file (mean, standard deviation, min, max, etc.)  To get the attributes for a pre-computed file, we might use:
 
<bash>
 
<bash>
 
sfattr < junk.rsf  
 
sfattr < junk.rsf  
Line 20: Line 20:
 
   total samples = 64000  
 
   total samples = 64000  
 
</code>
 
</code>
For an example of reading and writing an RSF file I'm going to demonstrate the use of sfwindow which is a program that allows us to select portions of an RSF file.  When sfwindow is used without any additional parameters, we are able to make a copy of a file with a different filename.  For example:
+
For an example of reading and writing an RSF file I'm going to demonstrate the use of [[Guide_to_madagascar_programs#sfwindow|sfwindow]] which is a program that allows us to select portions of an RSF file.  When <tt>sfwindow</tt> is used without any additional parameters, we are able to make a copy of a file with a different filename.  For example:
 
<bash>
 
<bash>
 
sfwindow < junk.rsf > junk2.rsf
 
sfwindow < junk.rsf > junk2.rsf

Revision as of 16:18, 9 May 2011

Madagascar was designed initially to be used from the command line. Programmers create Madagascar programs (prefixed with the sf designation) that read and write Madagascar files. These programs are designed to be as general as possible, each one operates on any dataset in RSF format, provided you also supply the correct parameters and have the right type of data for the program. Thus, each Madagascar program can be used in different ways that the original developer may not have foreseen, which makes your life easier as a user!

Command line interface

Madagascar programs follow the standard UNIX conventions for reading and writing RSF files to and from STDIN and STDOUT. This is also the same convention that Seismic Unix uses, so for those of you who are SU fans, this should be an easy conversion.

For example: the program sfattr allows us to get attributes about an RSF file (mean, standard deviation, min, max, etc.) To get the attributes for a pre-computed file, we might use: <bash> sfattr < junk.rsf </bash> which outputs:

    rms =       1.41316 
   mean =      0.999667 
 2-norm =       357.503 
 variance =      0.997693 
std dev =      0.998846 
    max =       5.05567 at 36 8 27 
    min =      -3.59936 at 18 9 6 
 nonzero samples = 64000 
 total samples = 64000 

For an example of reading and writing an RSF file I'm going to demonstrate the use of sfwindow which is a program that allows us to select portions of an RSF file. When sfwindow is used without any additional parameters, we are able to make a copy of a file with a different filename. For example: <bash> sfwindow < junk.rsf > junk2.rsf </bash> gives us two files, junk.rsf and junk2.rsf which are identical but not the same file.

In addition to specifying files to read in and out on the command line we can specify the parameters for each program that are necessary for it to run, or to produce the desired result. The general format for specifying parameters on the command line is key=val, where key is the name of the parameter that you want to set, and val is the value of the parameter. There are four (4) different types of values that are acceptable: int, float, boolean, or string. Going back to the window program, we can specify the number of traces or pieces of the file that we want to keep like:

<bash> sfwindow < junk.rsf n1=10 > junk-win.rsf </bash>

Self-documentation

Of course, we can specify as many parameters as we'd like on the command line. To figure out which parameters are needed for a specific program, just type the name of the program with no input files our output files on the command line as follows:

sfwindow

which brings up the self-documentation, which looks something like the following:

NAME

       sfwindow

DESCRIPTION

       Window a portion of a dataset. 

SYNOPSIS

       sfwindow < in.rsf > out.rsf verb=n squeeze=y j#=(1,...) d#=(d1,d2,...) f#=(0,...) min#=(o1,o2,,...) n#=(0,...) max#=(o1+(n1-1)*d1,o2+(n1-1)*d2,,...)

PARAMETERS

       float   d#=(d1,d2,...)  sampling in #-th dimension 
       largeint f#=(0,...)     window start in #-th dimension 
       int     j#=(1,...)      jump in #-th dimension 
       float   max#=(o1+(n1-1)*d1,o2+(n1-1)*d2,,...)   maximum in #-th dimension 
       float   min#=(o1,o2,,...)       minimum in #-th dimension 
       largeint n#=(0,...)     window size in #-th dimension 
       bool    squeeze=y [y/n] if y, squeeze dimensions equal to 1 to the end 
       bool    verb=n [y/n]    Verbosity flag

USED IN

       bei/dpmv/krchdmo
       bei/dpmv/matt
       bei/dwnc/sigmoid
       bei/fdm/kjartjac
       bei/fld/cube
       bei/fld/shotmovie
       bei/fld/synmarine
       bei/fld/yc
       bei/ft1/autocor

The self-documentation tells us the function of the program, as well as the parameters that are available to be specified. The parameter format is type - name=default value [options] and then a short description of the parameter. File parameters request a name of a file. For example: file=junk.rsf

Piping

Sometimes we want to chain multiple commands together without writing intermediate Madagascar RSF files in the process. Fortunately, we can do this using another standard UNIX construct, Pipes. Pipes allow us to connect the standard output from one Madagascar program to the standard input to another program without first writing to a file. For example we could do the following without pipes:

 sfwindow < junk.rsf > junk-win.rsf
 sftransp < junk-win.rsf > junk2.rsf

Or we could do the equivalent using pipes on one line:

 sfwindow < junk.rsf | sftransp > junk2.rsf

Pipes simply make these statements more compact, and allow us to reduce the number of files that we need to save to disk. If you're doing something many times over, then pipes will make your life significantly easier. I highly recommend that you use pipes! Also, you should know that you can chain as many commands together as necessary using Pipes.

Note: Madagascar uses a trick to make sure that you can send as much data as you want through the pipes because a conventional UNIX pipe has a maximum amount of data that can be sent through it. To avoid this limitation, Madagascar writes the data to temporary files before sending it through a pipe.

Interacting with files

Ultimately though, 95% of your time using Madagascar on the command line will be to inspect and view files that are output by your scripts. Some of the key commands that are used to interact with files on the command line are:

sfin - get header information

SFIN is the most used program on the command line, because most often we simply need to check the dimensionality of our files to make sure that we have them in the correct order.

<bash> sfin junk.rsf

junk.rsf:

   in="/var/tmp/junk.rsf@"
   esize=4 type=float form=native 
   n1=100         d1=0.004       o1=0          label1="Time" unit1="s" 
   n2=34          d2=0.1         o2=0          label2="Distance" unit2="km" 

3400 elements 13600 bytes </bash>

sfattr - get file attributes

SFATTR is also commonly used from the command line to check files for correct values. Most often, we use sfattr to ensure that files are not filled with zeros, or with NaN's after a long computation, or to make sure that the values are reasonable. sfattr can be used to obtain basic statistics about the files as well.

<bash> sfattr < junk.rsf

    rms =             1 
   mean =             1 
 2-norm =       58.3095 

variance = 0

std dev =             0 
    max =             1 at 1 1 
    min =             1 at 1 1 

nonzero samples = 3400

 total samples = 3400 

</bash>

sfwindow - window out specific portions of a file to be used elsewhere

SFWINDOW is one of the most versatile programs to use. Typically, it is used to select subsets of the data contained in an RSF file for computation elsewhere. Typically, you specify the data subset you want to keep using, the n, j, and f parameters which specify indices in the arrays to keep. For example if we want to keep the 15th-30th time samples for each distance in junk.rsf, we might use the following command:

<bash> sfwindow < junk.rsf f1=15 n1=15 j1=1 > junk-win.rsf </bash>

sftransp - transpose the dataset axes

SFTRANSP is also very versatile, and used to reorder RSF files for other programs to be used. For example:

<bash> sftransp < junk.rsf plane=12 > junk-transposed.rsf </bash>

swaps the first and second axes, so that now the first axis is distance and the second axis is time.

Scripting

While you could use Madagascar solely from the command line, you'll probably find that it is far too tedious to do so. Especially, when you want to change the value of a parameter and then rerun the same task. To avoid some of that nastiness, you could use shell scripting to automate portions of your processing flow, but shell scripts quickly become cumbersome and are not optimal scripts (shell language is ugly). Fortunately, Madagascar comes batteries included, and has its own scripting language available for you to use called SCons. SCons is incredibly powerful as we will see in later tutorials, so please don't rush out and start writing shell scripts. Read the next few tutorials then decide whether you still want to do so.


Back to Tutorial