Effelsberg 100m Teleskop

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Reducing spectral line data ======

The raw data (in MBFITS format) are converted to CLASS format automatically. This is done using the method described in Winkel, Kraus, & Bach, //Unbiased flux calibration methods for spectral-line radio observations//, [[http://adsabs.harvard.edu/abs/2012arXiv1203.0741W| A&A 540, A140, 2012]] (Section 3.4 and Section 4.5) - assuming a //constant system temperature// (Tsys) and //flat calibration signal// (Tcal). The latter is produced by a noise diode the power of which is in most cases directly fed into the waveguide within the receiver. //Neglecting the frequency-dependence of Tsys and Tcal introduces a bias, especially for wide-band observations!// (For more information see Winkel, Kraus, & Bach 2012 where we also show ways to do it better.)

The data files can be found on the //observer4// or //observer6// in the directory
**/daten/Class/[fw_]class_YYYY_MM_DD.100m**. There might be a short delay until the data is processed and is visible in the class file after the observation has finished

**Spectra in Class are always in units of Tcal!** To obtain Kelvins or Jy/Beam you have to multiply with the temperature of the noise diode (Tcal) and appropriate antenna sensitivities/efficiencies. Approximate values for this can be found on our receiver website [[http://www3.mpifr-bonn.mpg.de/div/effelsberg/receivers/receiver.html|receiver website]]. However, it is highly recommended to do calibration measurements during your observation for better accuracy. Furthermore, the automatic Class pipeline does not account for atmospheric opacity, elevation-gain curve, and antenna efficiency. Information about these steps can be found in Kraus 2009 (calibration memo).

Of course it is also possible to use the raw data directly, should one wish to do so.

===== Using class =====
To reduce your (spectroscopic and pointing) data with ''class'' one should use the **observer4** or **observer6** computer. Both are in the MPIfR LDAP environment, i.e., everyone with an account in the MPIfR network can login to this computer with his/her account. It is connected to the ''/homes'' server. It is also connected to ''/soft/astro/gag'', so your gildas package of choice should work. It is also possible to use the Effelsberg version of class that is used to write the spectroscopic and pointing data. To enable this (if you are using a C-Shell like tcsh), change to the bash shell by typing 
<code>bash</code> 
and source the init file:\\
<code bash>source /opt/bin/init_rtsoft.sh</code>
afterward you can run
<code bash>class</code>

The Data from the Class pipeline is stored in ''/daten/Class''. To open the file with Class use
<code bash>las90> file in "/daten/Class/class_2010_10_20.100m"
las90> find</code>

For pointings you have to switch to continuum mode first:
<code bash>las90> set type c
las90> find</code>

One can switch back to line observations with:
<code bash>las90> set type l
las90> find</code>

To look for new data in the file:
<code bash>las90> new</code>
(Note: This doesn't always work. In such cases simply reopen the file, using ''file open''.)

Printing is possible using:
<code bash>las90> hardcopy /dev eps color /plot HP-RECH</code>
In some cases, a file permission error may occur, in that case create a temporary eps file, please
<code bash>las90> hardcopy "/tmp/tmp.eps" /dev eps color /plot HP-RECH /overwrite</code>
A list of available printer names is obtained using
<code bash>sh> lpstat -p -d</code>


For more information on how to use Class have a look into the [[http://www.iram.fr/IRAMFR/GILDAS/|Gildas documentation]].

===== Frequency switching =====
Note that in frequency switching mode the Class pipeline already performs the folding (since the ''fold'' command of Class produces incorrect results for some cases). Details can be found in the [[information_for_astronomers:fsw_bug|News section]] and [[http://adslabs.org/adsabs/abs/2012A%26A...540A.140W/|Winkel, Kraus & Bach 2012]].

===== Reprocessing spectra using the Class pipeline =====
Sometimes it may be necessary to re-reduce the data with the Class pipeline. This can be done on the observer4 computer. The necessary software is installed in ''/opt/specpipeline''. There is a bash script to set all the necessary paths. "tcsh" shell users should change to "bash" by typing ''bash''. Launch the following command:

<code bash>
source /opt/bin/init_rtsoft.sh
</code>

Now you should be ready to run the offline Class calibration pipeline. There are two ways to go, an interactive tool and a command-line tool (for simple tasks).

==== Interactive mode - OfflinePipeline ====
Type

<code python>OfflinePipeline</code>

Now you are in an ipython interface to reduce the data. There are the following commands to reduce the data:

<code python>setFitsDir('/daten/Raw')</code>

which sets the source directory for the rawdata. The program will look in this directory for the raw mbfits files.

<code python>setClassName('Filename','Directory')</code>
This sets the output gildas class file. If only:

<code python>setClassName('Filename')</code>

is given, the named file is created in the current dictionary. Without this command a file name e.g. like ''class_2010_10_20.100m'' is created in the current directory. To write the files one has to know the scan number and must give the subscan number. Subscan numbers are 1 based. If the subscan is not in the file an error is plotted.

<code python>reduceSubscan(scannumber, subscannumber)</code>

If you reduce the same scan twice, it is written twice to the class file with the same scan number (which you usually don't want).

=== Batch processing ===

For projects involving lots of scan numbers, one may want to speed up things by using a small piece of python code such that one doesn't have to type in all scan numbers manually:

<code python>
setClassName('myfile.100m')
setFitsDir('/daten/Raw')

# choose the desired scan numbers, using the python range function (the second number is exclusive!)
scannums = range(6901, 6930)

# alternatively, you can also use a shell-glob pattern and do some rexexp magic to obtain associated scan numbers
import glob, re
filenames = sorted(glob.glob('/daten/Raw/*_27-13_*.FITS'))
scannums = [re.search(r'20\d{6,6}_(\d{4,4})_.*', f).group(1) for f in filenames]
 
for scannr in scannums:
    info = getScanInfo(scannr)
    print scannr
    subscanList = info.getSubscanNumList()
    for subscan in subscanList:
        try:
            reduceSubscan(scannr, subscan)
        except:
            pass

</code>
(In python it is important to keep indentation!) 

=== Options ===
There are a couple of options that can be passed to the ''reduceSubscan'' method.

^Keyword^Default^Description^
|febe|"*"|Process only data with a certain frontend-backend name (e.g., 'S60mm-XFFTS'), "*" means all|
|baseband|"*"|Process only data with specified baseband number (e.g., 1), "*" means all|
|fswFold|True|The pipeline does the frequency switch folding operation (you really want this and not let Class do it). See [[information_for_astronomers:fsw_bug|About Bandpass Ghosts]].|
|regridMethod|"fft"|Which regrid algorithm to use for frequency shifting. The default ('fft') algorithm is fast and is the only which doesn't lower spectral resolution. **However, ringing effects might occur.** This is because the FFTS spectra are not fully-sampled in the strict sense of Shannon(-Nyquist)s sampling theorem. Ringing usually occurs when strong narrowband features are present. Other methods include 'gaussian' (for a convolution based approach) and [[http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.interpolate.interp1d.html#scipy.interpolate.interp1d|scipy's]] regrid algorithms: 'linear', 'nearest', 'zero', 'slinear', 'quadratic', 'cubic'.|
|averMethod|"mean"|Allows to specifiy the averaging-method ('mean' or 'median', the latter might help with RFI)|
|skipDumps|(2, 0)|Allows to skip dumps at beginning/end of a subscan. At the beginning, the telescope sometimes is not yet fully on-source.|
|reverseArrayOrientation|False|The latter function additionally flips the data arrays for the following reduce operations, in case the IF flip was wrongly applied (usually happens only if testing new stuff).|
|shiftFFTSRefChannel|False|Before February 2011 the reference channel of the FFTS was wrong (in Fits header) and has to be shifted by +0.5 channels if the spectrum was not flipped, otherwise -0.5 channels. See also the [[fix:frequency_shift_in_ffts_observation_due_to_wrong_reference_channel|News section]].|
|autoLsrCorr|True|Per default the class pipeline attempts to correct the data for LSR Doppler shifts (due observer motion w.r.t. the LSR) if not already accounted for by hardware (LO tuning). If you really wanted a fixed-frequency setup (e.g., for system testing) you can disable automatic correction with this keyword.|
|doAatm|False|Apply AATM atmospheric model opacity correction (based on weather data).|
|gainCurve|(1., 0., 0.)|Apply gain curve correction (must be a tuple of (a0, a1, a2); see [[information_for_astronomers:rx_list|receiver details]].|
|tcal|1.|Apply tcal factor. Note, this can also be used to scale the data (e.g., to produce Jy/Beam instead of K, or to apply antenna efficiency). If you want to use different tcal factors per Baseband or FeBe you will need to use the baseband/febe arguments and run the pipeline several times (once for each baseband/febe).|


==== Shell processing - CmdLinePipeline ====
Often one just wants to reprocess a bunch of Fitsfiles. For this, we made a small shell-(python-)script.

<code bash>
python CmdLinePipeline.py -h
</code>
<code txt>
usage: CmdLinePipeline.py [-h] [-c CLASSFILE] [--baseband BASEBAND]
                          [--febe FEBE] [--no-fsw-fold]
                          [--no-auto-lsr-correction]
                          [--regrid-method {fft,gaussian,linear,nearest,zero,slinear,quadratic,cubic}]
                          [--averaging-method {mean,median}]
                          [--skip-dumps SKIPDUMPS SKIPDUMPS] [--spectral-flip]
                          [--shift-ref-chan]
                          fitsnames [fitsnames ...]

Command line interface to Class pipeline

positional arguments:
  fitsnames             list of raw-fits files

optional arguments:
  -h, --help            show this help message and exit
  -c CLASSFILE, --classfile CLASSFILE
                        name of output class file (default
                        "<date>_<project>.100m") you can choose any name,
                        <date> and <project> are meta-variables and will be
                        replaced by what is found in the fits files
                        (potentially creating more than one class file)
  --baseband BASEBAND   Filter by baseband
  --febe FEBE           Filter by febe
  --no-fsw-fold         do fswitch fold in pipeline (rather than class)
  --aatm                apply AATM opacity correction
  --gaincurve a0 a1 a2  apply gain curve correction (default: 1., 0., 0.)
                        Note, early versions of python-argparse dont support
                        scientific notation
  --tcal TCAL           apply tcal factor (default: 1.) Note, this can also be
                        used to scale the data (e.g., to produce Jy/Beam
                        instead of K, or to apply antenna efficiency).If you
                        want to use different tcal factors per baseband or
                        FeBe you will need to use the --baseband/--febe
                        arguments and run the pipeline several times (once for
                        each baseband/febe)
  --no-auto-lsr-correction
                        do not automatically correct for LSR shifts in fixed-
                        freq mode, default: False
  --regrid-method {fft,gaussian,linear,nearest,zero,slinear,quadratic,cubic}
                        which method to use for fswitch freq shifting
                        (default: fft) possible regridMethods are: fft,
                        linear, gaussian (the latter is recommended if you
                        experience ringing in the spectrum, linear and
                        gaussian decrease spectral resolution!)
  --averaging-method {mean,median}
                        averaging method (mean or median, default: mean)
  --skip-dumps SKIPDUMPS SKIPDUMPS
                        dumps to skip at beginning/end (tuple), default: 2 0
  --spectral-flip       additional spectral flip of the data arrays, default:
                        False
  --shift-ref-chan      before February 2011 the reference channel of the FFTS
                        had to beshifted by +0.5 channels if the spectrum was
                        not IF flipped, otherwise -0.5, default: False

</code>

The meaning of the parameters is like in the Table above (just slightly different argument names). The only difference is the '-c CLASSFILE' option that allows to give a certain class name. Default is "<date>_<project>.100m" where the date and project code are automatically filled-in from what is found in the MBfits files (this can lead to multiple class files obviously).

Typical usage would be:
<code bash>
# put everything into one class file
python CmdLinePipeline.py -c '/homes/user/myclass.100m' /daten/Raw/20141201*myprojectcode*XFFTS*

# put everything into separate class files (split by observing date and project code)
python CmdLinePipeline.py -c '/homes/user/<date>_<project>.100m' /daten/Raw/20141201*myprojectcode*XFFTS*

# process just one MBFITS file, and put class file(s) into current directory
python CmdLinePipeline.py /daten/Raw/20141211_2506_62-14_S13mm-XFFTS.FITS
</code>
Effelsberg 100m Teleskop

User Tools

Site Tools

Page Tools
