____________________________
CUTLASS Tutorial
(Based on the CUTLASS Workshop, University of Leicester, 2nd - 3rd December, 1998)
Contributors : Mark Lester, Steve Milan, Julian Thornhill, Chris Thomas, Jim Wild, Tim Yeoman.
Comments, suggestions, additions to Tim Yeoman, tim.yeoman @ion.le.ac.uk
____________________________
____________________________
Contents
1. General introduction 2. System overview 3. Operations 4. Real-time access and control 5. Data analysis software
5.1 Data structures and basic routines 5.2 Fitact and rawful 5.3 Go 5.4 Merging 5.5 Map_potential
____________________________
____________________________
1. General Introduction
____________________________
General Overview
♦ ♦ ♦ ♦ ♦
What is CUTLASS? What can you do with CUTLASS? How does CUTLASS relate to SuperDARN What is the Workshop Structure? What are the Workshop Aims?
CUTLASS Workshop Introduction 98/2
Radio and Space Plasma Physics
What is CUTLASS?
♦
♦
♦ ♦ ♦
Two HF Coherent Radars with one at Hankasalmi and one at Pykkvibaer. CUTLASS is a UK National Facility run by RSPP, University of Leicester. Collaborative Project with IRF and FMI. Forms part of the SuperDARN network of radars. Group also runs an ionosonde at Longyearbyen.
CUTLASS Workshop Introduction 98/3
Radio and Space Plasma Physics
CUTLASS Staff
♦ ♦ ♦ ♦ ♦ ♦ ♦
Mark Lester - CUTLASS PI. Tim Yeoman - Scheduling Representative. Chris Thomas - CUTLASS Project Manager. Julian Thornhill - Real Time Control/Experiment S/W. Stuart Crooks - Data Archiving and Requests. Peter Chapman - CUTLASS Maintenance. Mick Parsons - CUTLASS Maintenance.
CUTLASS Workshop Introduction 98/4
Radio and Space Plasma Physics
W
J
The locations and fields-of-view of northern and southern hemisphere SuperDARN radars. Operational radars are shown in yellow and proposed radars in red.
Transmitted signal
Ground backscatter
Ground reflection
Ionospheric Ionospheric reflection backscatter
B
ray
kr
300
1 2F
ray
C
F region 11 F 2 B E region 100 2F
0 1000 Range (km) 2000 200
1 2E
ray
A 1F 1E
0
A schematic illustration presenting some of the possible modes of propagation and the regions from which backscatter can occur. The stated ranges and altitudes are approximate and depend on ionospheric conditions. Three example rays are indicated: A E-region mode; B F-region mode producing both far and near-range scatter; C a ray that penetrates the ionosphere (from Milan et al., 1997).
What does CUTLASS measure?
♦ ♦ ♦ ♦ ♦ ♦
Main parameters measured by each CUTLASS radar are: Received Power; Line of Sight Velocity; Spectral Width; Elevation Angle of Returned Signal; Two overlapping radars give 2-D velocity vectors
CUTLASS Workshop Introduction 98/5
Radio and Space Plasma Physics
Science Topics
♦ ♦ ♦ ♦ ♦ ♦ ♦
Ionospheric Flows associated with solar wind magnetosphere coupling. Magnetospheric Substorms. ULF Waves. Travelling Ionospheric Disturbances. Auroral Irregularities Artificially Stimulated Irregularities Heating Diagnostic.
CUTLASS Workshop Introduction 98/6
Radio and Space Plasma Physics
Relationship to SuperDARN
♦ ♦ ♦
♦ ♦
Currently 14 operating radars. 1 radar to be deployed. Common Operations Software (RADOPS 2000). Common Scheduling on all radars. Access to all of the radars
CUTLASS Workshop Introduction 98/7
Radio and Space Plasma Physics
Workshop Structure
♦ ♦ ♦ ♦ ♦
System Description. Radar Operations. Real Time Access. Demonstration of Data Analysis Software. Data Analysis Workshop.
CUTLASS Workshop Introduction 98/8
Radio and Space Plasma Physics
____________________________
2. System Overview
____________________________
System Overview
CUTLASS is a coherent radar
♦
♦
sensitive to field aligned irregularities Allow refraction to make the radar wave vector to achieve orthogonality, and Propagation conditions to get the signal to the area of interest Consists of 16 600W solid state transmitters each connected to a single log-periodic antenna. Both operates from 8 to 20 MHz Phasing network allows each transmitter to be phased appropriately for 1 of 16 possible beam directions To operate over the required frequency range the phasing arrangement consists of time delays implemented with delay lines - Phasor network - as a opposed to a true Butler Matrix (STARE) which operates at a single frequency The phasing network provides approximately constant pointing direction and beam width over the frequency range Beam width is approx. 3.6 degrees Azimuthal coverage is approx. 50 degrees
♦
Operates in the frequency range 8 - 20 MHz - frequency is selected to:
♦ ♦
♦
CUTLASS is a phased array radar
♦ ♦
♦
♦
♦ ♦
Radio and Space Plasma Physics
Beam forming
Tx 1
Tx 2
Tx 3
Tx 4
…….. 16 transmitter Beam selection
16 channels each with 16 delays
Signal source
Radio and Space Plasma Physics
System Overview
♦ ♦
♦ ♦ ♦
♦ ♦
Need to receive backscattered signals Receive path is provided from the individual antennas which bypasses the transmitters and is routed through the phasing network Receive beam is the same a the transmit beam Signals from the phasing network are summed sent to the receiver Receiver down-converts the signal to provide baseband in-phase and quadrature signals - to determine the sign of the frequency offset Signals from the receiver are sampled by a two channel A/D converter CUTLASS incorporates an interferometer:♦ Four receive only antennas form a second array ♦ Signals from these are phased to form beams with the same look directions as those for the main array ♦ Signals from the interferometer array are summed and routed to an additional receiver channel ♦ The down-converted interferometer signals are sampled by two further A/D channels
Radio and Space Plasma Physics
CUTLASS Interferometer
Antenna spacing = 15.24 m Interferometer spacing = 100m (Iceland) 200m (Finland)
Interferometer spacing
Equipment hut
Antenna spacing
Radio and Space Plasma Physics
CUTLASS Antenna Model
-
No amplitude taper applied -
Delay element errors ignored
Power in dB normalised to max. power (One way) Idealised case - No phase errors introduced. Amplitude taper (dB)
Beam 1 0 Beam 2
Frequency 8.0 MHz.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0 Beam 3 0 Beam 4 0 Beam 5 0 Beam 6 0
-30dB 0
-20dB
Width 11.7 Azim.-24.2 Beam 7 Beam 8 0
-10dB
0dB
-30dB
-20dB
Width 11.4 Azim.-20.8 Beam 9 0
-10dB
0dB
-30dB
-20dB
Width 11.2 Azim.-17.5 Beam 10 0
-10dB
0dB
-30dB
-20dB
Width 11.1 Azim.-14.2 Beam 11 0
-10dB
0dB
-30dB
-20dB
Width 10.9 Azim.-11.0 Beam 12 0
-10dB
0dB
-30dB
-20dB
Width Azim.-
-10dB
-30dB 0
-20dB
Width 10.7 Azim.-4.6 Beam 13 Beam 14 0
-10dB
0dB
-30dB
-20dB
Width 10.6 Azim.-1.4 Beam 15 0
-10dB
0dB
-30dB
-20dB
Width 10.6 Azim. 1.4 Beam 16 0
-10dB
0dB
-30dB
-20dB
Width 10.7 Azim. 4.6
-10dB
0dB
-30dB
-20dB
Width 10.8 Azim. 7.8 0
-10dB
0dB
-30dB
-20dB
Width Azim.
-10dB
-30dB
-20dB
Width 11.1 Azim. 14.2
-10dB
0dB
-30dB
-20dB
Width 11.2 Azim. 17.5
-10dB
0dB
-30dB
-20dB
Width 11.4 Azim. 20.8
-10dB
0dB
-30dB
-20dB
Width 11.7 Azim. 24.2
-10dB
0dB Antenna polar diagram
-30dB
-20dB
-10dB
CUTLASS Antenna Model
-
No amplitude taper applied -
Delay element errors ignored
Power in dB normalised to max. power (One way) Idealised case - No phase errors introduced. Amplitude taper (dB)
Beam 1 0 Beam 2
Frequency 10.0 MHz.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0 Beam 3 0 Beam 4 0 Beam 5 0 Beam 6 0
-30dB 0
-20dB
Width 9.4 Azim.-24.3 Beam 7 Beam 8 0
-10dB
0dB
-30dB
-20dB
Width 9.2 Azim.-20.9 Beam 9 0
-10dB
0dB
-30dB
-20dB
Width 9.0 Azim.-17.6 Beam 10 0
-10dB
0dB
-30dB
-20dB
Width 8.9 Azim.-14.3 Beam 11 0
-10dB
0dB
-30dB
-20dB
Width 8.8 Azim.-11.1 Beam 12 0
-10dB
0dB
-30dB
-20dB
Width Azim.-
-10dB
-30dB 0
-20dB
Width 8.6 Azim.-4.7 Beam 13 Beam 14 0
-10dB
0dB
-30dB
-20dB
Width 8.5 Azim.-1.5 Beam 15 0
-10dB
0dB
-30dB
-20dB
Width 8.5 Azim. 1.5 Beam 16 0
-10dB
0dB
-30dB
-20dB
Width 8.6 Azim. 4.7
-10dB
0dB
-30dB
-20dB
Width 8.6 Azim. 7.8 0
-10dB
0dB
-30dB
-20dB
Width Azim.
-10dB
-30dB
-20dB
Width 8.9 Azim. 14.3
-10dB
0dB
-30dB
-20dB
Width 9.0 Azim. 17.6
-10dB
0dB
-30dB
-20dB
Width 9.2 Azim. 20.9
-10dB
0dB
-30dB
-20dB
Width 9.4 Azim. 24.3
-10dB
0dB Antenna polar diagram
-30dB
-20dB
-10dB
CUTLASS Antenna Model
-
No amplitude taper applied -
Delay element errors ignored
Power in dB normalised to max. power (One way) Idealised case - No phase errors introduced. Amplitude taper (dB)
Beam 1 0 Beam 2
Frequency 12.0 MHz.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0 Beam 3 0 Beam 4 0 Beam 5 0 Beam 6 0
-30dB 0
-20dB
Width 7.9 Azim.-24.4 Beam 7 Beam 8 0
-10dB
0dB
-30dB
-20dB
Width 7.7 Azim.-21.0 Beam 9 0
-10dB
0dB
-30dB
-20dB
Width 7.6 Azim.-17.6 Beam 10 0
-10dB
0dB
-30dB
-20dB
Width 7.4 Azim.-14.3 Beam 11 0
-10dB
0dB
-30dB
-20dB
Width 7.3 Azim.-11.1 Beam 12 0
-10dB
0dB
-30dB
-20dB
Width Azim.-
-10dB
-30dB 0
-20dB
Width 7.2 Azim.-4.7 Beam 13 Beam 14 0
-10dB
0dB
-30dB
-20dB
Width 7.1 Azim.-1.5 Beam 15 0
-10dB
0dB
-30dB
-20dB
Width 7.1 Azim. 1.5 Beam 16 0
-10dB
0dB
-30dB
-20dB
Width 7.2 Azim. 4.7
-10dB
0dB
-30dB
-20dB
Width 7.3 Azim. 7.9 0
-10dB
0dB
-30dB
-20dB
Width Azim.
-10dB
-30dB
-20dB
Width 7.4 Azim. 14.3
-10dB
0dB
-30dB
-20dB
Width 7.6 Azim. 17.6
-10dB
0dB
-30dB
-20dB
Width 7.7 Azim. 21.0
-10dB
0dB
-30dB
-20dB
Width 7.9 Azim. 24.4
-10dB
0dB Antenna polar diagram
-30dB
-20dB
-10dB
CUTLASS Antenna Model
-
No amplitude taper applied -
Delay element errors ignored
Power in dB normalised to max. power (One way) Idealised case - No phase errors introduced. Amplitude taper (dB)
Beam 1 0 Beam 2
Frequency 16.0 MHz.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0 Beam 3 0 Beam 4 0 Beam 5 0 Beam 6 0
-30dB 0
-20dB
Width 5.9 Azim.-24.5 Beam 7 Beam 8 0
-10dB
0dB
-30dB
-20dB
Width 5.8 Azim.-21.0 Beam 9 0
-10dB
0dB
-30dB
-20dB
Width 5.7 Azim.-17.7 Beam 10 0
-10dB
0dB
-30dB
-20dB
Width 5.6 Azim.-14.4 Beam 11 0
-10dB
0dB
-30dB
-20dB
Width 5.5 Azim.-11.1 Beam 12 0
-10dB
0dB
-30dB
-20dB
Width Azim.-
-10dB
-30dB 0
-20dB
Width 5.5 Azim.-4.7 Beam 13 Beam 14 0
-10dB
0dB
-30dB
-20dB
Width 5.4 Azim.-1.6 Beam 15 0
-10dB
0dB
-30dB
-20dB
Width 5.4 Azim. 1.6 Beam 16 0
-10dB
0dB
-30dB
-20dB
Width 5.5 Azim. 4.7
-10dB
0dB
-30dB
-20dB
Width 5.5 Azim. 7.9 0
-10dB
0dB
-30dB
-20dB
Width Azim.
-10dB
-30dB
-20dB
Width 5.6 Azim. 14.4
-10dB
0dB
-30dB
-20dB
Width 5.7 Azim. 17.7
-10dB
0dB
-30dB
-20dB
Width 5.8 Azim. 21.0
-10dB
0dB
-30dB
-20dB
Width 5.9 Azim. 24.5
-10dB
0dB Antenna polar diagram
-30dB
-20dB
-10dB
CUTLASS Antenna Model
-
No amplitude taper applied -
Delay element errors ignored
Power in dB normalised to max. power (One way) Idealised case - No phase errors introduced. Amplitude taper (dB)
Beam 1 0 Beam 2
Frequency 20.0 MHz.
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0 Beam 3 0 Beam 4 0 Beam 5 0 Beam 6 0
-30dB 0
-20dB
Width 4.7 Azim.-24.5 Beam 7 Beam 8 0
-10dB
0dB
-30dB
-20dB
Width 4.6 Azim.-21.1 Beam 9 0
-10dB
0dB
-30dB
-20dB
Width 4.5 Azim.-17.7 Beam 10 0
-10dB
0dB
-30dB
-20dB
Width 4.5 Azim.-14.4 Beam 11 0
-10dB
0dB
-30dB
-20dB
Width 4.4 Azim.-11.2 Beam 12 0
-10dB
0dB
-30dB
-20dB
Width Azim.-
-10dB
-30dB 0
-20dB
Width 4.3 Azim.-4.7 Beam 13 Beam 14 0
-10dB
0dB
-30dB
-20dB
Width 4.4 Azim.-1.6 Beam 15 0
-10dB
0dB
-30dB
-20dB
Width 4.4 Azim. 1.6 Beam 16 0
-10dB
0dB
-30dB
-20dB
Width 4.3 Azim. 4.7
-10dB
0dB
-30dB
-20dB
Width 4.4 Azim. 7.9 0
-10dB
0dB
-30dB
-20dB
Width Azim.
-10dB
-30dB
-20dB
Width 4.5 Azim. 14.4
-10dB
0dB
-30dB
-20dB
Width 4.5 Azim. 17.7
-10dB
0dB
-30dB
-20dB
Width 4.6 Azim. 21.1
-10dB
0dB
-30dB
-20dB
Width 4.7 Azim. 24.5
-10dB
0dB Antenna polar diagram
-30dB
-20dB
-10dB
System control
♦ ♦
♦
♦ ♦ ♦
Two PCs control the system - use the QNX real time operating system Timing computer generates the system timing:♦ Programmable pulse sequence is stored in RAM and replayed ♦ Generates the receive sample signal Main computer samples and processes the received signal ♦ Contains the A/D converters which sample the signal ♦ Implements the scheduling programme which controls the radar ♦ Carries out the signal processing tasks ♦ Stores the data on local disk ♦ Copies data to magneto-optical (MO) disks ♦ Handles communications tasks Computers are connected via an ethernet LAN which is part of the RSPPG subnet Network connection is via ISDN and analogue dial up lines Radars may instigate communications to report problems
Radio and Space Plasma Physics
Radar control
Antennas
Transmitters
Phasing network
Beam selection
Rx 1
Rx 2
Sample timing
Storage Main computer
Timing computer
Local networking
Radio and Space Plasma Physics
CUTLASS schematic
Power supplies Antenna
Phasing Matrix
Power Supplies 8 off 2 channel delay units
LP Tx/Rx
HP Tx/Rx
HV PS
Predriver
Driver
EB 104
Filter
Power sense
Receiver
AGC module
Synthesiser
16 off Tx modules
Computer system
Remote control and monitoring
Signal Real Time processing Control
BAS Box
Data storage
Radio and Space Plasma Physics
CUTLASS frequency allocations and bands
♦
Finland Frequency Allocations
♦
♦
Iceland Frequency Allocations
♦
Band
Band Width (kHz) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Start Freq (kHz) 8305 8965 9900 11075 11550 12370 13200 15010 16210 16555 17970 18850 19415 19705 19800
Stop Freq (kHz) 8335 9040 9985 11275 11600 12415 13260 15080 16360 16615 18050 18865 19680 19755 19990 30 75 85 200 50 45 60 70 150 60 80 115 265 50 190
♦ ♦ ♦ ♦ ♦
♦
♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦
♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦
Start Freq End Freq Width (kHz) (kHz) (kHz) 20 8000 8195 195 21 8430 8850 420 22 8985 9395 410 23 10155 10655 500 24 10655 11175 520 25 11290 11450 160 26 11475 11595 120 27 12105 12235 130 28 12305 12510 205 29 12590 13280 690 30 13360 13565 205 31 13875 13995 120 32 14400 15015 615 33 15805 16365 560 34 16500 16685 185 35 16820 17475 655 36 18175 18770 595 37 18835 1888 550 38 19910 20000 90 39 10155 11175 1020 ( band 39 is bands 23 & 24 combined) Radio and Space Plasma Physics
Pulse scheme
♦ ♦
♦
♦
♦
♦
Need a modulation scheme which will generate useful information CUTLASS is a pulse radar ♦ Employs 300 microsecond pulses = 45 km range resolution, or ♦ 100 microsecond pulses - 15 km resolution Receiver bandwidth is set to the appropriate value for the pulse length ♦ Approx. 3 KHz or 10 KHz respectively Normally use a 7 pulse sequence ♦ Unit time is 2.4 milliseconds ♦ Pulse spacings are 9, 12, 20, 22, 26 and 27 ♦ Pulse sequence duration is 79.2 milliseconds ♦ This is the ‘best’ pulse sequence anyone has managed to generate Routinely sample ranges from 180 km to 3500 km ♦ Corresponds to sample times from 2.4 ms to 46.6 ms Therefore need to transmit and receive at the same time ♦ This is of course impossible!
Radio and Space Plasma Physics
CUTLASS pulse sequence
Radio and Space Plasma Physics
ACF Lag Calcualtion
Radio and Space Plasma Physics
ACF - Spectra
TL, the minimum lag separation, is the smallest sampling interval in constructing the backscatter autocorrelation function (ACF). If a power spectrum is created by an FFT of the ACF, the spectrum will have alias points at +/- (1/2TL), the Nyquist frequency.
An example of radar ACFs and spectra is shown overleaf - see rawful.pro section for more Radio and Space Plasma Physics
Signal processing
♦ ♦ ♦ ♦
♦
♦ ♦
♦ ♦ ♦
♦
Pulse sequence is transmitted Receive signal is sampled at either 100 or 300 microseconds Sampling ends after time appropriate to last pulse and maximum range For each range of interest:♦ Auto-correlation is carried out for all delays available in the pulse sequence At any range and time, receive signal from that range (wanted signal) and from other pulses and ranges (cross-range noise) Another pulse sequence is transmitted and the process repeated The wanted signals will be correlated whilst the cross range noise is uncorrelated and may be removed by averaging for a number of pulse sequences 10 pulse sequences are required to remove the noise 10 pulse sequences require 1 second to transmit 1 second is the shortest integration period available with this (and any other similar) pulse sequence The acf averaged over the integration time form the raw radar data
Radio and Space Plasma Physics
ACF phase, full ACF, ACF magnitude and derived spectrum - ground scatter
Bad Lags and fitacf
Bad lags, positions in the auto-correlation function which do not contain useful information, arise from two sources:♦ Missing receive samples because we were transmitting at that time, and ♦ Strong echoes from some ranges which contaminate the acf at other ranges We need to take account of these missing points when processing the raw data At the end of each integration time the raw data for a particular beam is passed to the fitacf routine. Taking account of bad lags this:♦ Determines the zero lag power (signal strength) by fitting the appropriate function to the power of the auto-correlation function. ♦ Exponential fitting gives the lambda power value, whilst a gaussian fit gives rise to the sigma power value ♦ The phase of the auto-correlation function, taking into account any 2π ambiguities, is fitted with a straight line. The slope of this fitted line is the Doppler frequency shift which corresponds to the irregularity velocity. ♦ The Fourier transform of the complex auto-correlation function provides the spectra of the received signal from which the spectral width can be determined ♦ These derived values for each range and beam constitute the fit data
Radio and Space Plasma Physics
♦
♦ ♦
Operation of the radar
♦
♦
For each scan ♦ For each beam in the scan ♦ Set the required beam in the phasing network ♦ Search for a quiet frequency in the selected frequency band ♦ Transmit the pulse sequence ♦ Sample the signal ♦ Carry out the acf calculation ♦ Check for strong signals and adjust the receiver gain setting if required ♦ Sum the acf values ♦ Continue until the integration time is complete ♦ Set the next beam and repeat ♦ Send the raw data to other tasks:♦ Fitacf - generates the .fit data ♦ Vlptm - generates the beam swung vector velocity values ♦ Summary process - generates the .smr files for the daily plots At the end of the scan:♦ Wait for the scan synchronisation time ♦ Store the raw data for the scan on disk (.dat files) ♦ Stores the fit data on disk (.fit files)
Radio and Space Plasma Physics
Data recording and transfer
Data files are generated for each 2 hour period e.g. ♦ Format of the file name is 98120204y.xxx ♦ Where y is a single letter indicating the radar f for finland, e for iceland, and ♦ xxx is the file type .dat .fit .inx .smr .vec .grd ♦ If the radar is restarted, e.g. after power cuts, a new file is generated. ♦ A 2 hour period may therefore consist of several files denoted by additional letters e.g. ♦ 98120204f.fit, 98120204fA.fit, 98120204fB.fit, 98120204fC.fit Shortly after midnight, files for the day are copied to MO disk Each MO disk will stores data for several days depending on the operating mode and integration time selected MO capacity is 2 GBytes per side Disks are sent to Leicester by post After data has been archived to tape at Leicester, MO disks are erased and returned to the radar sites
♦
♦ ♦
♦ ♦ ♦
Radio and Space Plasma Physics
Routine data processing
♦
♦
♦
Each day:♦ Operation of the radars is checked ♦ .vec .smr and .grd files are transferred from the radars ♦ Summary plot for each radar is produced and placed in the Web pages (.smr) ♦ CDHF key parameters are produced and sent to the GGS (.vec) ♦ Combined clock plot for both radars is placed in the Web pages ♦ .grd files are sent to APL for near real time convection mapping When the MO disks are received from the radars:♦ Data are sent to APL to form part of the SuperDARN Multiple Radar (MR) distribution ♦ Data are archived (and duplicated) and catalogued ♦ Statistical information is generated When SuperDARN MR tapes are received:♦ Data are archived ♦ Stacked summary plots are produced to produce a quick reference source
Radio and Space Plasma Physics
CUTLASS Data Processing Overview
Julian Iceland Status email (daily) List processor (cutlass-status) Deputy Others Engineering summary (.SMR) Science summary (.VEC) vlptm
Finland email ISTP key parameters to CDF (daily) DDS II DDS II
Power
MO disk
Leicester
LOS Velocity
DDS II .FIT, .INX and .DAT files on Exabyte to APL (weekly)
WWW server
GIF Automatic daily hardcopy
Clock plot
Post
PC CUTLASS archive ion On-line calalogue DDS II ion SuperDARN archive SuperDARN backup DDS II CUTLASS backup Stacked summary plots
SuperDARN Distribution Exabyte tape (weekly)
Radio and Space Plasma Physics
CUTLASS kp generation
Radar FITACF Line of sight velocities VLPTM L shell fits .vec files PSTN
Leicester
log files
incoming copied
KP process
kp database
cdf file Space based ISTP instruments
ftp
sfdu file Other ground based ISTP instruments (SuperDARN radars)
CDHF
kp CD ROMS
Radio and Space Plasma Physics
____________________________
3. Operations
____________________________
70°N
60°N
Pykkvibær Hankasalmi
GEOGRAPHIC COORDS
0°E
15°E
30°E
45°E
The fields-of-view of the CUTLASS Finland (yellow shading) and Iceland (red shading) radars.
CUTLASS Operations
q Common time 50%
(including 5 days fast CT, 1 min scans) q Discretionary time 30%
Special modes on CUTLASS q Special time 20%
Special modes on all SuperDARN radars
Radio and Space Plasma Physics
SPECIAL MODES
q q q q q q Range resolution (45, 30, 15 km) Integration time (down to 1 s) Lag to first range Scan mode Transmit frequency Plus much more in principle
Data can be displayed in a number of formats, examples of which are shown on the following pages. Initial viewing is done through summary plots (next page), which have a link to the radar schedule, which allows the radar mode to be identified.
Radio and Space Plasma Physics
800
SUPERDARN PARAMETER PLOT Goose Bay and Halley: velocity
17 Mar 1996
Beamswung vectors
600 400 200
Velocity (m s -1) ) ) ) ) ) ) ) ) )
1000 m s -1
Ionospheric scatter only
0 -200 -400 -600 -800
(a) Bz +ve, By=0
0114 00s
80˚
(b) Bz -ve, By +ve
0135 59s
70˚
22 MLT
24 MLT
0113 59s
-80˚
0135 59s
-70˚
22 MLT
24 MLT
An example of a CUTLASS discretionary scan mode
9 0 15
Scan sequence: 15, 9, 14, 9, 13, 9, ...1, 9, 0, 9 Full scan every 240 s Beam 9 data every 14 s
L L LL
Hankasalmi
SUPERDARN PARAMETER PLOT
Hankasalmi: vel
Beam 9
80
6 Aug 1995 (218)
band15-0camp9 scan mode (-6210)
1930 (218) to 2330 (218)
400
75 Magnetic Latitude
300 200 100
Velocity (ms -1)
70
0 -100 -200
65
-300 -400 Ionospheric scat only
60 1930
2000
2030
2100
2130 UT
2200
2230
2300
2330
Beam 8
80
1930 (218) to 2330 (218)
400
75 Magnetic Latitude
300 200 100
Velocity (ms -1)
70
0 -100 -200
65
-300 -400 Ionospheric scat only
60 1930
2000
2030
2100
2130 UT
2200
2230
2300
2330
World Wide Web access to Data, Operations and Software ***********************************************
CUTLASS home page: http://ion.le.ac.uk/cutlass/cutlass.html CUTLASS Operations Schedule http://ion.le.ac.uk/~jth/sched/opyears.htm Form Access to Summary and Clock Plots http://ion.le.ac.uk/cutlass/summary_plot_choose.html Form Access to Data Request http://ion.le.ac.uk/cutlass/data_request.html Form Access to CUTLASS/SuperDARN Discretionary Time Request http://ion.le.ac.uk/cutlass/time_allocation.html Search the CUTLASS Data Catalogue http://ion.le.ac.uk/cutlass/cutlass_data_search.html
Radio and Space Plasma Physics
____________________________
4. Realtime Access and Control
____________________________
____________________________
CUTLASS Realtime Communications
Both CUTLASS radars are connected to Leicester via standard telephone modems (for emergency use), ISDN digital phone links and via leased line internet connections (courtesy of the Finnish Meteorological Institute and the University of Reykyavik). This allows the radars to appear on the Leicester campus TCP/IP network, so that standard utilities such as telnet and ftp can be used. The phone lines are expensive, but provide reliable, rapid data transfer. The ISDN links are used for: Uploading new radar control programs and schedules Realtime radar monitor during campaign operations Realtime radar control during campaigns The leased lines are used for Nightly transfer of the previous days summary data Nightly automatic email detailing system status and hardware performance Automatic email notification of power cuts Automatic email and SMS pager notification of failure to sound Dissemination of realtime summary data to web clients (including Leicester realtime displays, and APL space weather convection maps Routine monitoring of operations during campaigns
Campaigns
During certain campaigns the ISDN link s are held open to view the radar returns and alter radar operations in real time. This can be done by viewing the field-of-view display that is generated on the radar's console (visible only via Leicester) and using purpose-built radar control programmes to specify radar operation modes. The radar returns may also be viewed using a sophisticated Java web display. This allows any internet Java client to connect to the Leicester web page and see a live data feed. Links to the Leicester and other SuperDARN data feeds and displays can be found at http://ion.le.ac.uk/realtime/new_frame_display.html. The data stream is transmitted from the radar to Leicester, and then duplicated as necessary depending on how many Java clients are requesting access. Access to the link may be restricted during campaign periods.
____________________________
____________________________
If the appropriate radar control program is running, certain parameters can be interactively altered without the need to load a new control program. These parameters include: Frequency band(s) being sounded Beams being sounded Distance to firat range Range separation Integration time These parameters are normally only adjusted by Leicester operations staff, or approved campaign personnel. Direct access to the radars is required to make these changes.
Related items
A SAMNET magnetometer is co-located with the Iceland CUTLASS radar. The magnetometer data is sent back nightly to Leicester via the Iceland leased line connection, and onward to York, via ftp. Leicester operate a vertical ionosonde at Longyearbyen, Svalbard. It is connected to the internet via the campus network at UNIS. A sounding is performed every 5 minutes, and data is returned immediately to Leicester. A web page graphic is then generated, and the data archived, with summary plots generated. The summary plots, and the realtime graphic can be found at: http://ion.le.ac.uk:8082/cadi/index.html
____________________________
____________________________
5. Data Analysis Software
5.1 Data structures and basic routines 5.2 Fitact and rawful 5.3 Go 5.4 Merging 5.5 Map_potential
____________________________
/cutlass/data/incoming
95022216f.fit 95022216f.dat 95022216f.inx 95022216f.rin
year month day hour station
Radio and Space Plasma Physics
Why so complicated ? BINARY, .FIT,.DAT: DIRECT ACCESS: EVOLVING DATA FORMAT: read_fit_dg.c, read_fit386_v110.c, read_fit386_v130.c. small fast, hence index files (also variable record size)
Radio and Space Plasma Physics
Reading a .fit file - Essential routines
/people/cutlass/fitread
fitropen.c find_fit_rec.c read_fit.c fit_close.c
e.g.
printpv.f
Radio and Space Plasma Physics
.fit file data structure http://ion.le.ac.uk/cutlass/general/data_structure.html http://ion.le.ac.uk/cutlass/general/fitread.html
f = {fit, rec_time: p: {parms, rev: {rev_no, nparm: st_id: year: month: day: hour: minut: sec: txpow: nave: atten: lagfr: smsep: ercod: agc_stat: lopwr_stat: radops_sys_ress: noise: radops_sys_resl: intt: txpl: mpinc: mppul: mplgs: nrang: frang: rsep: bmnum: xcf: tfreq: scan: mxpwr: lvmax: usr_resL1: usr_resL2: cp: usr_resS1: long(0), $ major: byte(0), minor: byte(0)}, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ long(0), $ lonarr(2), $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ 0, $ long(0), $ long(0), $ long(0), $ long(0), $ 0, $ 0, $
usr_resS2: usr_resS3: pulse_pattern: lag_table: combf: noise_lev: noise_lag0: noise_vel: pwr_lag0: slist: nsel: qflag: pwr_l: pwr_l_err: pwr_s: pwr_s_err: vel: vel_err: width_l: width_l_err: width_s: width_s_err: stnd_dev_l: stnd_dev_s: stnd_dev_phi: gscat: x_qflag: x_pwr_l: x_pwr_l_err: x_pwr_s: x_pwr_s_err: x_vel: x_vel_err: x_width_l: x_width_l_err: x_width_s: x_width_s_err: phi0: phi0_err: elev: elev_low: elev_high: x_stnd_dev_l: x_stnd_dev_s: x_stnd_dev_phi: num_lags:
0, $ 0}, $ intarr(16), $ intarr(2,48), $ bytarr(80), $ double(0.0), $ double(0.0), $ double(0.0), $ dblarr(75), $ intarr(75), $ 0, $ lonarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ intarr(75), $ lonarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ dblarr(75), $ intarr(75)}
CUTLASS data ASCII export format
Flat ASCII files are available for standard scans of CUTLASS fitted parameters. The data is exported in datafiles for each radar and a field-of-view file which defines the locations of the range gates. FORMATS: Data files Contents Date (yyyymmdd), station name, parameter, scatter flag Beam time and day number Data for 75 range gates of each beam (format: 75I6), beams 0-15. e.g. 20001101 Hankasalmi pwr_l Ion scatter only
1530 08s (306) 26 24 31 …. 1530 15s (306) 24 27 34
Field-of-view files Contents Date (yyyymmdd), station name, coordinate system Longitudes of edges of range gates, ranges 1-75, beams 0-15, (format: 76F8.2) Latitudes of edges of range gates, ranges 1-75, beams 0-15, (format: 76F8.2) e.g. 20001101 20.74 65.14 Hankasalmi 20.52 65.24 geog 20.07 … 19.84 …
20.29 65.35
Libraries
/people/cutlass/lib
libacf.a libmag.a libraw.a
libfit.a libmerge.a librbpos.a
libgen.a libpgm.a
/people/cutlass/lib/share libacf.so libmag.so libraw.so libfit.so libmerge.so librbpos.so libgen.so libpgm.so
Detailed SuperDARN Documentation: http://ion.le.ac.uk/cutlass/SuperDARN.html
Radio and Space Plasma Physics
Programs
Coordcnv - geographic to AACGM transformation
print_raw_file_info - basic info on raw datafile create_raw_index - does what it says Smerge - merge vector creation printpv - basic power/velocity text output Reconstruct - makes an .inx file print_raw
- basic raw data text output
Radio and Space Plasma Physics
Raw data
q Investigate details of the radar spectra and ACFs rawful.pro q Recreate .fit data from a consistent version of fitacf ufitacf.c
Radio and Space Plasma Physics
rawful.pro
q
A widget-based IDL procedure for browsing raw SuperDARN data
q q q q
Functions
Creates .rin files Displays raw parameter block information Creates fitted data blocks Plots: q Lag0 power q ACFs q ACF power and phase q Derived Spectrum q 4 X Spectra q Spectrum Range Waterfall q Will loop in time or range
Documentation: http://ion.le.ac.uk/cutlass/idl/rawful.html
Radio and Space Plasma Physics
.dat file data structure
Record length Record number "rawwrite" Parameter list (as in .fit structure) PULSE_PATTERN (lags at which pulses are transmitted) LAG_TABLE (definitions of lags used in calculating the ACFs) Comment buffer Lag-0 Powers (compressed) ACF data XCF data (if parms.xcf flag set) 2 bytes 4 bytes 8 bytes 96 bytes Variable in length: 1-16 2 byte integers, the number being determined by parms.mppul Variable in length: 1-48 pairs of 2 byte integers, the number being determined by parms.mplgs 80 bytes Variable in length: 1-75 compressed 2 byte values, the number being determined by parms.nrang Variable in length: 1-75 ACF data blocks (defined below) Variable in length: 1-75 XCF data blocks (defined below)
Lag 0 compression algorithm: 32 bit integers are converted into 16 bit pseudo-floating point numbers of format SVVVVVVVVVVVCCCC where S is the sign (1=negative) VV are the value bits and CC is the shift count. The high order bit is suppressed and is assumed to be 1. The compression algorithm takes the absolute value of the 32 bit signed number and shifts it right until the upper 16 bits are zero. The most significant bit is the replaced by the sign and the lower 4 bits by the shift count.
ACF data block
Range number Real part, lag0 Imaginary part, lag0 Real part, lag k[1] Imaginary part, lag k[1] Real part, lag k[2] Imaginary part, lag k[2] " " " Real part, lag k[N] Imaginary part, lag k[N]
2 byte integer, first range is range=1 2 compressed 2 byte values. There must always be a lag0. The real part of lag0 must be the same as the ag0 power, and the imaginary part must be 0 The number of lags is determined by parms.mplgs.The LAG_TABLE defines the pairs of samples used to calculate a given lag. Lag k[i] may be equal to i but may not be. The lags do not have to be in order, but they usually are. N=parms.mplgs
ACF power and phase
Spectra for a full radar beam
Spectra for 4 range gates
ACF phase, full ACF, ACF magnitude and derived spectrum - ionospheric scatter
Spectra with deduced bad lags
SuperDARN visualisation software:
Go
Go runs in the IDL scientific data visualisation environment Go is constantly evolving (always contains a few surprises) It is (relatively) easy to bolt on your own routines
For instance, code has been developed for plotting magnetometer, EISCAT, Polar UVI, meridian scanning photometer, etc. data from within Go Documentation:
http://ion.le.ac.uk/cutlass/idl/go.html
Radio and Space Plasma Physics
♦
Go allows you to:
Plot maps of radar backscatter in a variety of coordinate systems Plot Range-time-parameter plots Plot time-series for a particular point within the field-of-view And much much more...
♦
♦
♦
♦
Radio and Space Plasma Physics
CUTLASS workshop Loading a file list_paths cat cat, n cat, /all info resize, 24 file, ‘97112718f_’ list_parameters plot_map plot_polar Map plotting and navigating scans set_coords, ‘range’ plot_map plot_map, /beams skip, n go, n go_time, hhmm plot_map, 3, 2, /allbeams map, x0, x1, y0, y1 Time series plotting time, 1800,2400 set_beam, 15 set_coords, ’gate’ plot_rti set_range, 27 plot_graph plot_polar, /oval,/clip plot_summary, 10,3 plot_polar, /oval set_coords,’mag’ set_coords, ‘geog/stereo’ map, 0,40,60,95 plot_rti, beam=n
GO tutorial
plot_rti, 1,4, beam=[0,5,10,15] set_format, /portrait
plot_summary, 10,3,skip=10
vel set_scale, -500,500 width_l
plot_rti map, y=[y0, y1] time, hhmm, hhmm set_beam,9
print, get_time_series(/data)
1
CUTLASS workshop print, get_time_series(/time) time, 2400,2600 set_beam,9 plot_stack, ranges=[27,30], inc=200, /zeros go_time, 2000 plot_range plot_range, y=[-500,500] Postscript printing ps_open @summary_plot.go ps_close Taking control set_format, /portrait clear_page plot_title pwr_l plot_rti_panel, 1,3,0,0,/info plot_colour_bar,3,0 vel plot_rti_panel, 1,3,0,1,/info plot_colour_bar,3,1 width_l plot_rti_panel, 1,3,0,2,/info ps_preview ps_print custom_commands @summary_plot.go plot_colour_bar,3,2
GO tutorial
Plotting merged vectors merge_file, ‘/cutlass/data/cw/merge/ 971127ef_1830120.mrg’ merge_info go_time,2136 plot_polar_merge plot_polar_merge, /no_data @merge_panels.go
2
CUTLASS workshop
GO tutorial
Producing merged vectors Outside go: merge_ef –f 97112718 –s 1830 –i 120 –d 10 –c -S
3
Command line instructions
Command (short-hand) Arguments {optional} ----------------------------------------------------------------------------------------------------------------------General: print_info (info) print_scan_mode_info (scan_mode) print_hardware_info {,radar id} commands custom_commands (custom) resize, hours set_news, 'on' | 'off' print_news history add_history, command clear_history ----------------------------------------------------------------------------------------------------------------------File handling: read_new_file (file), read_data_file catalogue (cat) next_file prev_file list_paths add_path, remove_path, squash, juice, choose_filter, scan_filter, ‘filename’ {,’par’,’par’,’par’,time=[hhmm,hhmm] | /all, /append} {,time=[hhmm,hhmm] | /all} {,n | /all,/comments}
1
‘pathname’ {,/promote} ‘pathname’ ‘filename’ {,/cordial} ‘filename’ scan_id {,st_id=radar id} [n,n,..],[n,n,..] | /normal_cw | /normal_ccw | /unknown {,subset=[n,n,n],scan='scan_name'} list_scans {,n,n} list_beams {,n,n} make_comment, {‘filename’,} 'Comment string...' ----------------------------------------------------------------------------------------------------------------------Navigating scans: browse {,coloumns,rows} print_scan_info list_scans {,n,n} first_scan skip_scan (skip) {,n, /quiet} goto_scan (go), n goto_scan_time, hhmm {,ss} (go_time) ----------------------------------------------------------------------------------------------------------------------Beam and range gate: print_beam_range set_beam, beam set_range (set_gate), gate set_cell, beam,gate -----------------------------------------------------------------------------------------------------------------------
Command line instructions
----------------------------------------------------------------------------------------------------------------------Parameters: print_parameter_info list_parameters default_parameter_list set_parameter_list, ‘par’,’par’,’par’ (par_list) set_parameter (par), ‘par’ pwr_l vel width_l elev set_scatter, 0|1|2|3 ----------------------------------------------------------------------------------------------------------------------Colour scale: print_scale_info default_scale set_scale, n,n {,n_cols} set_elev_scale, n,n {,n_cols} ----------------------------------------------------------------------------------------------------------------------Coordinate system: print_coord_info print_fov_info set_coords,
2
'geog' | 'geog/stereo' | 'mag' | 'mag/stereo' | 'range' | 'gate' set_rti_limits (time), hhmm,hhmm {,date='yymmdd'} set_map_limits (map), xmin,xmax,ymin,ymax | x=[xmin,xmax] | y=[ymin,ymax] ----------------------------------------------------------------------------------------------------------------------Maps and plotting: plot_map plot_beams plot_rti plot_summary plot_polar movie plot_2_par, par,par movie_2_par, par,par plot_graph plot_stack plot_range plot_2_graphs,par,par plot_frequency overlay_magnetometers overlay_oval overlay_polar_oval {,chain=n,charsize=n} {Kp=n} {Kp=n} {,columns,rows,/beams | /allbeams,skip=n, /nodata} {,columns,rows,column,row} {,columns,rows,beam=[n,n,...] | /allbeams} {,columns,rows | /all | /overview, beams=[n,n]} {,/clip | clip=[n,n],/oval | oval=Kp} {,scan,scan,/beams,/vels} {,columns,rows,/beams | /allbeams} {,scan,scan,/beams | /polar} {,columns,rows,beams=[],ranges=[], y=[ymin,ymax]} {,columns,rows,beams=[],ranges=[],inc=n,/zeros, /nopoints,exclude=[],/bar} {,columns,rows,column,row,y=[],legend='', /nopoints,/noline,exclude=[],/bar} {,columns,rows,beams=[],ranges=[], y1=[ymin,ymax],y2=[ymin,ymax]}
Command line instructions
overlay_zenith overlay_polar_zenith overlay_mag_pole overlay_track, overlay_rti_axis, next_map prev_map plot_title plot_colour_bar print_format_info set_format, set_invert, set_time_format, set_ps_font, set_window, set_window_size, set_min_charsize, clear_page plot_map_panel plot_map_info plot_rti_panel {,columns,rows,column,row} {,charsize=n} ‘filename’ 'geog' | 'mag' | 'range' | 'gate' {,/left | /right, /integer}
3
{,’title’,main_title='title',/nodate} {,rows,row} /landscape | /portrait, /square | /free, /colourscale | /greyscale, /guppies | /sardines 'on' | 'off' 'hhmm' | 'hh:mm' | 'hh:mm:ss' 'on' | 'off' n n n
{,columns,rows,column,row,/info,/nodata,/no_x,/no_y} {,columns,rows,column,row} {,columns,rows,column,row,beam=n,/info, freq_band=[,],max_gap=n,/no_marks,/no_x,/no_y, /no_axis} plot_rti_info {,columns,rows,column,row,beam=n} plot_summary_panel {,columns,rows,column,row,/no_x,/no_y} plot_polar_panel {,columns,rows,column,row,/clip,/info, /hemi_info,/zenith} plot_graph_panel {,columns,rows,column,row,pos=[beam,range], /left,/right,y=[ymin,ymax],legend='', /nopoints,exclude=[],/bar,/info,/no_x,/no_y} plot_stack_panel {,columns,rows,column,row,beam=n,ranges=[], inc=n,/zeros,/nopoints,exclude=[], /bar,/no_x,/no_y} plot_range_panel {,columns,rows,column,row,y=[],legend='legend', /nopoints,/noline,exclude=[],/bar,/no_x,/no_y} plot_frequency_panel {,columns,rows,column,row} define_panel {,columns,rows,column,row,/square,/bar} plot_label, columns,rows,column,row,x,y,’label’, {,charthick=n,charsize=n,alignment=n, orientation=n} ----------------------------------------------------------------------------------------------------------------------Postscript printing: print_ps_info ps_print ps_open {ps_start} ps_close {ps_stop} ps_landscape ps_rmfiles ps_pause set_ps_size, set_colour_device, {,filename} {,filename,/bw | /colour}
'A4' | 'A5' | n ‘device name’
Command line instructions
set_bw_device, ‘device name’ ----------------------------------------------------------------------------------------------------------------------Miscellaneous: threshold, [passband_min,passband_max] print_scan {,compact} -----------------------------------------------------------------------------------------------------------------------
4
♦
Creating merged vectors
♦
merge_ef -f 97112718 -s 1830 -i 120 -S -d 10
file name root start time integration time (seconds) single scan merging (no integration) duration (hours) Resulting file: 971127ef_1830120.mrg
♦ ♦ Documentation: http://ion.le.ac.uk/cutlass/merge/merge.html
♦ ♦ ♦ ♦ ♦
-f -s hhmm -i n -S -d n
♦
Radio and Space Plasma Physics
SuperDARN visualisation software:
Map_potential
Map_potential runs in the IDL scientific data visualisation environment It is a widget-based utility which will ingest data from as many radars as are available It produces equipotential maps which represent the global convection pattern which best fits the line of sight data, stabilised by an IMF-driven statistical model. Documentation:
http://ion.le.ac.uk/~gp3/map_potential.html
Radio and Space Plasma Physics
True vecs 03/17/1996 01:44:00 01:46:00 UT
APL MODEL 0