Data Reduction Software

Remote Data Reduction with CHARA

For remote data reduction, we are running a virtual machine on the CHARA server in Atlanta. This machine currently has the redclimb and redfluor data reduction packages installed as well as the packages for the PAVO and MIRC-X beam combiners.

Arranging Access to the Remote Data Reduction Machine

Use this form to request access. After submitting this form, we will make an account on the machine for the user, start a VNC server on their account, and send their login credentials. We use VNC because the graphical elements of the reduction software packages are often prohibitively slow through ssh. Please allow 3-5 business days for a response to your request. If we have not responded within that time, email CHARA Data Scientist Jeremy Jones (This email address is being protected from spambots. You need JavaScript enabled to view it.).

Connecting to the Remote Data Reduction Machine

  • Download and install a VNC viewer. We recommend tigervnc because of the viewers we’ve tested, it runs the fastest. Others do work, though.
  • If using tigervnc's viewer, run the vncviewer command. This should open a window that looks like this:

tigervnc viewer connection window

  • Input the address you are given into the "VNC server" prompt. This should bring up a password prompt. This will connect you to the Atlanta remote observing machine.
  • If you get the error "unable to connect to host: Connection timed out (10060)", you are likely behind a firewall and should try again from another network or get in touch with your IT team to open up the 5900 range of ports (5900-5999).
  • If it is your first time accessing the system, we recommend changing the automatically generated passwords. To change your password for your VNC server, run the command “vncpasswd” and to change your account password, use the command “passwd”.

Finding the Data

Once logged on, you can find the data in the /dbstorage/ directory. From there, it is organized by beam combiner. CLIMB, Classic, JouFLU, and miscellaneous CHARA data are in the /dbstorage/Cl_CL_JF/ directory. The rest are self-explanatory. Data are proprietary within 18 months, so if you have any data from that time period, make sure to note it in your access form or email Jeremy. If you wish to access another PI’s proprietary data, we will, of course, need their permission. Once we have that, we will give you access to it. Older data are freely accessible.

Reducing your Data

Classic/JouFLU Data Reduction

We recommend that you use redfluor for Classic/JouFLU reduction. A tutorial can be found here.

Note: Make sure to use the –D flag to indicate the destination for the reduced files as you will not have write permissions in the data directory.

CLIMB Data Reduction

We recommend using John Monnier's IDL pipeline to reduce CLIMB data. A tutorial for how to reduce CLIMB data on the CHARA Remote Data Reduction Machine was written by Kathryn Lester.

Note: Run start_climb.script from a local working directory and not the raw data directory. The redclimb pipeline is also available for reducing CLIMB data. More information can be found here.

Note also: When using redclimb, make sure to use the –D flag to indicate the destination for the reduced files as you will not have write permissions in the data directory. We will be providing John Monnier’s IDL pipeline soon.

PAVO Data Reduction

A tutorial for reducing PAVO data with the CHARA Remote Data Reduction Machine can be found here. All of the IDL scripts are installed and available. To access them, run pavo_idl. We will run headstrip.pro on each night’s data as it comes in.

MIRCX/MYSTIC Data Reduction

The MIRCX/MYSTIC pipeline is installed on the Remote Data Reduction Machine. Documentation can be found here. When running the MIRCX pipeline on the Remote Data Reduction Machine, run the following command: source mircx_python This will set path variables to point to the MIRCX reduction pipeline and start a virtual environment for python with all the prerequisites installed.

Note: Please do not run mircx_reduce.py in the raw data directory (to keep the raw data free of contamination by reduction products) or your home directory (the home disk has limited space). You should run mircx_reduce.py in your directory in /mnt/disk2/ or /mnt/disk5. Starting in 2025, new users will have their home directory set up on /mnt/disk5. If you do not have a directory for this purpose, please contact Jeremy and he will set one up for you. If you are unsure if you have such a directory, run the command ls /mnt/disk2 (or /mnt/disk5) and look for your username.

For old MIRC data prior to the MIRC-X upgrades, use the IDL pipeline. A tutorial can be found here.

Additional Resources

See the 2020 CHARA Summer Seminar page to view presentations on observing and reducing data from each of the beam combiners.

CHARA Seeing Reviewer

The CHARA Seeing Reviewer is installed on the Remote Data Reduction machine. You can use this tool to review the wave-front sensor logs to see what the seeing (or any other values logged by the wave-front sensors) was like on any open night since the wave-front sensors were installed in 2019. To run the CHARA Seeing Reviewer, start its virtual environment with source srenv, then run seeing_reviewer.py.

Example of the Seeing Reviewer

Questions and Problems

Please direct any questions to CHARA Data Scientist Jeremy Jones (This email address is being protected from spambots. You need JavaScript enabled to view it.). If you encounter any errors in connecting to or using the data reduction machine, please use this form to report the problem.