HSRED Reduction Pipeline

HSRED v2.0 can be downloaded via github. Just execute the following command:

As before, HSRED requires copies of both the idlutils and idlspec2d packages. HSRED v2.0 has been tested and verified to work with idlutils v5_5_15 and idlspec2d v5_7_1, but will also work with the (now ancient) versions of these packages needed for the old version of HSRED, so there is no need to update if you already have a working version.

Once you have downloaded all the pieces of code you need, continue following the instructions for setting up your environment variables here. Any installation or usage questions can be addressed by Ben Weiner.

A new wrapper script is included with HSRED v2.0 that should streamline the reduction process. Step one is to place all the files you wish to reduce from a single night in a directory together: biases, dome and twilight flats, comparison lamp exposures, and any science exposures. You must include both the .fits files and their associated _map files. Sets of exposures using different gratings (270/600) or central wavelengths must be sorted and separated, one working directory per config (per night).

From your working directory, start idl and run the command:

for a standard reduction with cosmic-ray rejection, summed combination of exposures, red-leak removal, etc. The optional rerun keyword can be passed to make more than one reduction of the same dataset. If not specified, rerun will default to 0100 and your reduced data products will be placed in the subdirectory ‘reduction/0100’.

If you have included F-stars in your target configuration, and wish to perform flux calibration as part of the coaddition, run:

Note that, for flux calibration, you must still add your stars and their photometry to the $HSRED_DIR/etc/standardstars.dat file, as described here. However, it should no longer be necessary to generate and use “plugcat” files. All info needed should now be read from the included _map files.   If your data was taken with the 600-line grating, you must specify the /do600 keyword, like:

For data obtained with an offset sky exposure (for sky-subtraction in crowded fields), the procedure is a bit more complicated and requires editing the lists/cal.list file. First, to reduce your science frames without the normal fiber-based sky subtraction, run

Next, the sky offset frame must be reduced separately. Edit the cal.list, commenting out the line for your science frames, and adding a new line giving the sky offset frames you wish to reduce (See the old page here for more on the format of the cal.list file, and remember it is no longer necessary to specify a plugcat.) Now, run the pipeline wrapper again. NOTE it is very important to use a different “rerun” number for this reduction, to avoid confusion with file names of the reduced data products.

The final step is to manually subtract the reduced sky offset frame from the reduced science frame, using the reduced data format and method of your choice.

Compared to earlier versions of HSRED, an expanded set of data products is generated, generally thanks to an enhanced version of hs_toiraf.pro which is called automatically within hs_pipeline_wrap. If you only wish to have the spHect* files, please just edit hs_pipeline_wrap.pro to comment out the relevant lines. There have also been minor changes to what is stored in the spHect* files, so in either case, please consult the description below:

skysub_/###.*.ms.fits: Subdirectory containing individual iraf-format spectra, linearized in wavelength, one file per fiber. Format is a 4543x1x4 cube. The 4 spectra in this cube are (in order):

• Sky-subtracted, varaince-weighted, coadded spectrum
• Sky-subtracted, summed spectrum
• Sky spectrum that was subtracted
• sigma (sqrt(variance)) spectrum

If the pipeline was run with the /uberextract option, 1) and 2) will be the same, and contain the flux-calibrated, averaged spectrum (rather than one in coadded counts). This is true for all the data products described here. .ms.fits : All 300 fibers in one file, with linearized wavelengths. This is a 4543x300x4 data cube, and the 4 elements along the 3rd dimension are the same as those described above for the single-fiber files. spHect*fits: multi-extension fits file containing all coadded, sky-subtracted spectra for each hectospec configuration. Wavelengths are tabulated per pixel, rather than being described as a function in the fits header. The new, slightly altered data format is:

spObs*fits: multi-extension fits file containing sky-subtracted spectra for each individual science exposure (non-coadded). Format is similar to the coadded spHect files, but spObs files include the sky spectrum subtracted from each fiber on this exposure in HDU4.