Tutorial - Users 1 - Command line interface

From Madagascar
Revision as of 22:47, 25 April 2011 by Jgodwin (talk | contribs)
Jump to navigation Jump to search

Madagascar is designed to be used from the command line. Programmers create Madagascar programs (prefixed with the sf designation) that read and write Madagascar files. These programs are designed to be as general as possible, each one operates on any dataset provided in RSF format, provided you also provide the correct parameters and have the right type of data for the program. Thus, each Madagascar program can be used in even ways that the original developer may not have foreseen, which makes your life easier as a user!

Command line interface

Madagascar programs follow the standard UNIX conventions for reading and writing RSF files to and from STDIN and STDOUT. This is also the same convention that SeismicUnix uses, so for those of you who are SU fans, this should be an easy conversion.

For example: the program sfattr allows us to get attributes about an RSF file (mean, standard deviation, min, max, etc.) To get the attributes for a pre-computed file, we might use: <bash> sfattr < junk.rsf </bash> which outputs:

    rms =       1.41316 
   mean =      0.999667 
 2-norm =       357.503 
 variance =      0.997693 
std dev =      0.998846 
    max =       5.05567 at 36 8 27 
    min =      -3.59936 at 18 9 6 
 nonzero samples = 64000 
 total samples = 64000 

For an example of reading and writing an RSF file I'm going to demonstrate the use of sfwindow which is a program that allows us to select portions of an RSF file. When sfwindow is used without any additional parameters, we are able to make a copy of a file with a different filename. For example: <bash> sfwindow < junk.rsf > junk2.rsf </bash> gives us two files, junk.rsf and junk2.rsf which are identical but not the same file.

In addition to specifying files to read in and out on the command line we can specify the parameters for each program that are necessary for it to run, or to produce the desired result. The general format for specifying parameters on the command line is key=val, where key is the name of the parameter that you want to set, and val is the value of the parameter. There are four (4) different types of values that are acceptable: int, float, boolean, or string. Going back to the window program, we can specify the number of traces or pieces of the file that we want to keep like:

<bash> sfwindow < junk.rsf n1=10 > junk-win.rsf </bash>


Of course, we can specify as many parameters as we'd like on the command line. To figure out which parameters are needed for a specific program, just type the name of the program with no input files our output files on the command line as follows:


which brings up the self-documentation, which looks something like the following:




       Window a portion of a dataset. 


       sfwindow < in.rsf > out.rsf verb=n squeeze=y j#=(1,...) d#=(d1,d2,...) f#=(0,...) min#=(o1,o2,,...) n#=(0,...) max#=(o1+(n1-1)*d1,o2+(n1-1)*d2,,...)


       float   d#=(d1,d2,...)  sampling in #-th dimension 
       largeint f#=(0,...)     window start in #-th dimension 
       int     j#=(1,...)      jump in #-th dimension 
       float   max#=(o1+(n1-1)*d1,o2+(n1-1)*d2,,...)   maximum in #-th dimension 
       float   min#=(o1,o2,,...)       minimum in #-th dimension 
       largeint n#=(0,...)     window size in #-th dimension 
       bool    squeeze=y [y/n] if y, squeeze dimensions equal to 1 to the end 
       bool    verb=n [y/n]    Verbosity flag



The self-documentation tells us the function of the program, as well as the parameters that are available to be specified. The parameter format is type - name=default value [options] and then a short description of the parameter. File parameters request a name of a file. For example: file=junk.rsf


Sometimes we want to chain multiple commands together without writing intermediate Madagascar RSF files in the process. Fortunately, we can do this using another standard UNIX construct, Pipes. Pipes allow us to connect the standard output from one Madagascar program to the standard input to another program without first writing to a file. For example we could do the following without pipes:

 sfwindow < junk.rsf > junk-win.rsf
 sftransp < junk-win.rsf > junk2.rsf

Or we could do the equivalent using pipes on one line:

 sfwindow < junk.rsf | sftransp > junk2.rsf

Pipes simply make these statements more compact, and allow us to reduce the number of files that we need to save to disk. If you're doing something many times over, then pipes will make your life significantly easier. I highly recommend that you use pipes! Also, you should know that you can chain as many commands together as necessary using Pipes.

Note: Madagascar uses a trick to make sure that you can send as much data as you want through the pipes because a conventional UNIX pipe has a maximum amount of data that can be sent through it. To avoid this limitation, Madagascar writes the data to temporary files before sending it through a pipe.

Interacting with files

Ultimately though, 95% of your time using Madagascar on the command line will be to inspect and view files that are output by your scripts. Some of the key commands that are used to interact with files on the command line are:

sfin - get header information

SFIN is the most used program on the command line, because most often we simply need to check the dimensionality of our files to make sure that we have them in the correct order.

<bash> sfin junk.rsf


   esize=4 type=float form=native 
   n1=100         d1=0.004       o1=0          label1="Time" unit1="s" 
   n2=34          d2=0.1         o2=0          label2="Distance" unit2="km" 

3400 elements 13600 bytes </bash>

sfattr - get file attributes

SFATTR is also commonly used from the command line to check files for correct values. Most often, we use sfattr to ensure that files are not filled with zeros, or with NaN's after a long computation, or to make sure that the values are reasonable. sfattr can be used to obtain basic statistics about the files as well.

<bash> sfattr < junk.rsf

    rms =             1 
   mean =             1 
 2-norm =       58.3095 

variance = 0

std dev =             0 
    max =             1 at 1 1 
    min =             1 at 1 1 

nonzero samples = 3400

 total samples = 3400 


sfwindow - window out specific portions of a file to be used elsewhere

SFWINDOW is one of the most versatile programs to use. Typically, it is used to select subsets of the data contained in an RSF file for computation elsewhere. Typically, you specify the data subset you want to keep using, the n, j, and f parameters which specify indices in the arrays to keep. For example if we want to keep the 15th-30th time samples for each distance in junk.rsf, we might use the following command:

<bash> sfwindow < junk.rsf f1=15 n1=15 j1=1 > junk-win.rsf </bash>

sftransp - transpose the dataset axes

SFTRANSP is also very versatile, and used to reorder RSF files for other programs to be used. For example:

<bash> sftransp < junk.rsf plane=12 > junk-transposed.rsf </bash>

swaps the first and second axes, so that now the first axis is distance and the second axis is time.


While you could use Madagascar solely from the command line, you'll probably find that it is far too tedious to do so. Especially, when you want to change the value of a parameter and then rerun the same task. To avoid some of that nastiness, you could use shell scripting to automate portions of your processing flow, but shell scripts quickly become cumbersome and are not optimal scripts (shell language is ugly). Fortunately, Madagascar comes batteries included, and has its own scripting language available for you to use called SCons. SCons is incredibly powerful as we will see in later tutorials, so please don't rush out and start writing shell scripts. Read the next few tutorials then decide whether you still want to do so.

Back to Tutorial