User Tools
information_for_astronomers:user_guide:data_storage_and_archive
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
information_for_astronomers:user_guide:data_storage_and_archive [2014/05/20 14:57] hafok |
information_for_astronomers:user_guide:data_storage_and_archive [2017/01/13 10:00] (current) hafok |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| Continuum (Mapping , Pointing, Focus etc) and Spectroscopy is acquired on the be4 computer.\\ | Continuum (Mapping , Pointing, Focus etc) and Spectroscopy is acquired on the be4 computer.\\ | ||
| - | The data is stored in **/daten/Raw**. Logging information is stored in the mySQL observation database and until??? available also as text files in **/daten/Obslogs**. Spectroscopy data is processed with the spectroscopy pipeline on the computer observer5 and stored locally on observer5 in **/Class**.\\ | + | The data is stored in **/daten/Raw**. This data obtained with the new FitsWriter/control program is stored to **/daten/mbfits**. Binary Data for the HI Survey (the array data section) is stored as subscan binary files in **/daten/AFFTS**. Logging information for the observed scans is stored in the mySQL observation database. Spectroscopy data is processed with the spectroscopy pipeline on the computer be4 and stored locally on be4 in **/daten/Class**. /daten is now actually a NAS exporting 120 TB of disk space.\\ |
| \\ | \\ | ||
| - | * observer5 rsyncs every few minutes the class data to be4:**/daten/Class**. | + | * be4 rsyncs the data directories **/daten/Raw**, **/daten/AFFTS**, **/daten/Class** every 15 minutes to the effelsberg NAS which is exported as /effelsbergData/ to the local servers (be2,be4m observer2, observer4, observer5, observer6 and observer7)in the local data network 10.200.200. |
| - | * be4 rsyncs the data directories **/daten/Raw**, **/daten/AFFTS**, **/daten/Class** every 15 minutes to be3. | + | |
| * The data is moved to monthly folders after 30 days (e.g. **/daten/Raw/Raw-2011-03**). | * The data is moved to monthly folders after 30 days (e.g. **/daten/Raw/Raw-2011-03**). | ||
| - | * The data is deleted on be3 and be4 after 1-2 years. | + | * The data is deleted on be4 after 0.5-2 years. |
| * HI survey data is copied to the University of Bonn. | * HI survey data is copied to the University of Bonn. | ||
| - | * be3 syncs every 20 min these directories to **/effbg/effdata**. This volume is accessible in the MPIfR network as a NFS3 mount. | + | * be2 syncs every 20 min these directories to **/eff/data**. This volume is accessible in the MPIfR network as a NFS3 mount. Due to the limited storage we will delete data after several years. |
| - | * After one year the data is copied into the tape library. It is accessible in the MPIfR networkat the location **/hsm/effarchive**. The data is archived to tape. This is a NFS4 mount. You need a kerberos certificate to access it, via kinit from your computer connected to the institute LDAP account system. | + | * After one year the data is copied into the tape library. It is accessible in the MPIfR network at the location **/hsm/effarchive**. The data is archived to tape. This is a NFS4 mount. You need a kerberos certificate to access it, via kinit from your computer connected to the institute LDAP account system. |
| + | |||
| + | The data is accessible via **observer2** via rsync+ssh, scp using the obseff account. This computer is also accessible from the guest network in Effelsberg via **observer2g** | ||
information_for_astronomers/user_guide/data_storage_and_archive.1400590674.txt.gz · Last modified: 2014/05/20 14:57 by hafok
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International
