User Tools

Site Tools


chara:trouble_shooting

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
chara:trouble_shooting [2021/11/04 17:13]
charaobs
chara:trouble_shooting [2021/11/04 23:53]
charaobs [Can't find fringes]
Line 8: Line 8:
 This is a list of simple things to check if you can't find fringes: This is a list of simple things to check if you can't find fringes:
  
-  * Were the clocks synced? Make sure the [SYNC CLOCKS] button on Cosmic Debris has been pushed to start the night. If the OPLE server does not display the correct CHARA time and the errors don't read (0) or (1), the clocks were not synced. As the VME runs on its own clock and does not use the NTP server, it can drift over the course of the night and cause problems with finding fringes, even if it was synced at the start of the night. Syncing the clocks multiple time during the night may help to avoid this hidden clock problem+  * Were the clocks synced? Make sure the [SYNC CLOCKS] button on Cosmic Debris has been pushed to start the night. If the OPLE server does not display the correct CHARA time and the errors don't read (0) or (1), the clocks were not synced. 
-  * Did the Astrolib update on OPLE? If the job queue is stopped too soon after slewing on Cosmic Debris, the correct calculations for the carts will not be done by OPLE and you may be searching for fringes with the wrong star data. Hit STAR ACQUIRED on CD to update ople. The proper star can also be entered by typing hd #### into the OPLE server and hitting ENTER.+  * Did the Astrolib update on OPLE? If the job queue is stopped too soon after slewing on Cosmic Debris, the correct calculations for the carts will not be done by OPLE and you may be searching for fringes with the wrong star data. Hit STAR ACQUIRED on CD to update ople. The proper star can also be entered manually by typing hd #### into the OPLE server and hitting ENTER.
   * Are the PoP's correct? After a PoP change, the PoP's are sometimes not updated in CD or ople.   * Are the PoP's correct? After a PoP change, the PoP's are sometimes not updated in CD or ople.
-  * Are the carts behaving or are there vibrations or jumps of 100 or more microns every 3-6 seconds? Restart the ople server if they persist.+  * Are the carts behaving or are there vibrations or jumps of 100 or more microns every 3-6 seconds? How are the metrology signals? Are they strong and staying white? Red signals mean one or more metrology signals may have gone too low and homing carts is necessary.  The METDATA function on the ople gui can be used to sample the metrology signal and may show noise spikes that can be disruptive to the smooth cart motion. Push the METDATA button on the configure tab and then again about 6 seconds later.  Plots will pop up to show the frequency and power of any noise.
   * Is the target a high proper motion star? Red dwarfs are close stars and can have high proper motions. Scan a wider range to see if it is outside of the usual calculated scan range. Binaries can also have very high offsets from the expected position due to mistakenly using astromod calculations from the companion star.   * Is the target a high proper motion star? Red dwarfs are close stars and can have high proper motions. Scan a wider range to see if it is outside of the usual calculated scan range. Binaries can also have very high offsets from the expected position due to mistakenly using astromod calculations from the companion star.
-  * Do you have enough flux from each telescope or on each baseline?+  * Do you have enough flux from each telescope or on each baseline?  Is the telescope tiptilt struggling with low flux?  Can the camera gain be raised or the exposure made longer to help hold the star?
   * Did you get the same star in each telescope? Sometimes a busy star field and poor pointing of the telescopes can lead to the wrong star being acquired and locked by tiptilt. View the stars in the finder window to see if all the stars match.   * Did you get the same star in each telescope? Sometimes a busy star field and poor pointing of the telescopes can lead to the wrong star being acquired and locked by tiptilt. View the stars in the finder window to see if all the stars match.
   * Check the CHARA time on the GPS server. The "Ext-CHARA," "CHARA-Sys," and "Ext-Sys" time offsets listed on the GPS server should be small (< 0.01 sec). If there are large time offsets, then a GSYNC might be needed.   * Check the CHARA time on the GPS server. The "Ext-CHARA," "CHARA-Sys," and "Ext-Sys" time offsets listed on the GPS server should be small (< 0.01 sec). If there are large time offsets, then a GSYNC might be needed.
   * Check the time on the OPLE server. If the time is off or there are any lost ticks/seconds "Lost T/S" listed, then MSYNC ople using the GPS GUI and type "syncople" into the ople server.   * Check the time on the OPLE server. If the time is off or there are any lost ticks/seconds "Lost T/S" listed, then MSYNC ople using the GPS GUI and type "syncople" into the ople server.
   * Are the [MAN] buttons pressed (gray) for the moving carts on the OPLE Control gui? (The reference cart will remain green.)   * Are the [MAN] buttons pressed (gray) for the moving carts on the OPLE Control gui? (The reference cart will remain green.)
-  * Check that the carts are within delay line range (-1 to 44 meters) and errors are small. Were the carts homed before the first slew of the night? Did any carts go to the front switch after slewing? This can cause them to lose their position. Turn off the OL button to send it back to zero. If it does not track on the home switch when displaying a zero position (0.000000), it was lost. Is the E2 cart stuck at 27.97m? How are the metrology signals? Are they strong and staying white? Red signals mean one or more metrology signals have gone too low and homing carts is necessary. See section on homing carts in __OPLE and Metrology__  section below.+  * Check that the carts are within delay line range (-1 to 44 meters) and errors are small. Were the carts homed and checked before the first slew of the night? Did any carts go to the front switch after slewing? This can cause them to lose their position. Turn off the OL button to send it back to zero. If it does not track on the home switch when displaying a zero position (0.000000), it was lost. Is the E2 cart stuck at 27.97m? See section on homing carts in __OPLE and Metrology__  section below.
   * Are the LDC's working?   * Are the LDC's working?
       * Is the glass position within allowable range (-10mm to 49mm) on all beams?       * Is the glass position within allowable range (-10mm to 49mm) on all beams?
Line 25: Line 25:
       * If LDCs are working correctly, the LDC positions (in mm) should roughly match up with the reference cart positions (in meters).       * If LDCs are working correctly, the LDC positions (in mm) should roughly match up with the reference cart positions (in meters).
  
-  * Check the instrument alignment. Is flux getting through to the detector? How long has it been since the last NIRO camera alignment? Classic and CLIMB programs can run for about an hour before the light will drift from the central pixel. Use the Classic or CLIMB gui to view the light on the pixels by clicking the PICTURE tab and then the PIXEL AREA button. Turn the camera off after a few seconds to avoid crashing NIRO. Is the right dither power turned on? CLIMB 1 and Classic use different dithers. If Classic or CLIMB fringes are found in a scan, but not when in recording mode, the dither powers are likely not on. Are the camera settings correct for the seeing conditions and flux levels?+  * Check the instrument alignment. Is flux getting through to the detector? How long has it been since the last NIRO camera alignment? Classic and CLIMB programs can run for about an hour before the light will drift from the central pixel. Use the Classic or CLIMB gui to view the light on the pixels by clicking the PICTURE tab and then the PIXEL AREA button. Turn the camera off with the STOP button. Is the right dither power turned on? CLIMB 1 and Classic use different dithers. If Classic or CLIMB fringes are found in a scan, but not when in recording mode, the dither powers are likely not on. Are the camera settings correct for the seeing conditions and flux levels? 
  
 ===== Restarting Servers ===== ===== Restarting Servers =====
Line 78: Line 79:
 ==== The Telescope won't move or stopped moving ==== ==== The Telescope won't move or stopped moving ====
  
- \\ Have the powers been turned on to the drives? Are the scopes disabled? The usual state of the telescopes is disabled until enabled. This is due to the stall function of the scopes which eventually disables the scopes when they are stowed. Enable the scopes by hitting [ENABLE] on the dome gui or in the telescope gui control tab. If a scope disables itself during a slew, it may be just an overcautious stall function. Hit ENABLE and the scope should continue to slew. If it disables again without moving, there may be something wrong at the scope, ie. a hatch left open, a ladder not put away, a tool on the floor, or something else physically impeding the motion of the scope. You will need to go to the dome to see what it is. The computer in the dome will give you control of the scope to turn it away from the problem. \\  \\ Sometimes the dome guis get hung up and can cause erratic motion or no motion of the scopes. Check them for current times and continuous updates of numbers. If they are not updating, try to REOPEN them first. If that does not work, close the gui and open a new one. If a new one does not open, the dome server may be dead or Sockman lost track of it. See **Dome Server Restart**  below. \\  \\ **Azimuth Limit Switches** \\  \\ As of 11-'17, the azimuth limit switches are enabled and can stop the motion of the scopes if they try to go beyond -90º or +450º. The scopes will not be movable with normal inputs so follow these instructions to return them from the out of range condition. \\  \\ 1. On the domegui MANUAL tab, click STOP so pulses won't be sent to the drive by the control software. \\  \\ 2. Make sure you understand why the limit was hit which may require a trip to the telescope. \\  \\ 3. Click the OVERRIDE ON button in domegui MANUAL tab. After this, the hardware doesn't care about the limits switches and you're free to move the telescope. \\  \\ 4. Click ENABLE then you can move the telescope back to its normal range of operation. \\  \\ 5. After the telescope is back in it normal range, click OVERRIDE OFF which makes the hardware aware of the limits again.+\\ 
 +Have the powers been turned on to the drives? Are the scopes disabled? The usual state of the telescopes is disabled until enabled. This is due to the stall function of the scopes which eventually disables the scopes when they are stowed. Enable the scopes by hitting [ENABLE] on the dome gui or in the telescope gui control tab. If a scope disables itself during a slew, it may be just an overcautious stall function. Hit ENABLE and the scope should continue to slew. If it disables again without moving, there may be something wrong at the scope, ie. a hatch left open, a ladder not put away, a tool on the floor, or something else physically impeding the motion of the scope. You will need to go to the dome to see what it is. The computer in the dome will give you control of the scope to turn it away from the problem.\\ 
 +\\ 
 +Sometimes the dome guis get hung up and can cause erratic motion or no motion of the scopes. Check them for current times and continuous updates of numbers. If they are not updating, try to REOPEN them first. If that does not work, close the gui and open a new one. If a new one does not open, the dome server may be dead or Sockman lost track of it. See **Dome Server Restart** below.\\ 
 +\\ 
 +**Azimuth Limit Switches** \\ 
 +\\ 
 +<font 14px/Arial,Helvetica,sans-serif;;inherit;;inherit>As of 11-'17, the azimuth limit switches are enabled and can stop the motion of the scopes if they try to go beyond -90º or +450º. The scopes will not be movable with normal inputs so follow these instructions to return them from the out of range condition.\\ 
 +\\ 
 +1. On the domegui MANUAL tab, click STOP so pulses won't be sent to the drive by the control software.\\ 
 +\\ 
 +2. Make sure you understand why the limit was hit which may require a trip to the telescope.\\ 
 +\\ 
 +3. Click the OVERRIDE ON button in domegui MANUAL tab. After this, the hardware doesn't care about the limits switches and you're free to move the telescope.\\ 
 +\\ 
 +4. Click ENABLE then you can move the telescope back to its normal range of operation.\\ 
 +\\ 
 +5. After the telescope is back in it normal range, click OVERRIDE OFF which makes the hardware aware of the limits again</font>
 + 
 +<font 16px/Calibri,sans-serif;;#333333;;white>**However, If you notice that a telescope will stop near AZ 90 or 270 with the scope still being ENABLED and refusing to move, this is often due to the new AZ limit switch being tripped at some point earlier. To get it moving again:**</font> 
 + 
 +<font 14px/Arial,Helvetica,sans-serif;;#333333;;white>1. On the domegui MANUAL tab, click STOP so pulses won't be sent to the drive by the control software.</font>  \\ 
 +\\ 
 +<font inherit/inherit;;initial;;white>2. M</font>ake sure the all the scopes’ demand positions agree – for example, sometimes bringing a scope to a configuration that’s already on sky and issuing a slew command will make the additional scope go around North the “wrong” way.\\ 
 +\\ 
 +<font inherit/inherit;;initial;;white>3. Click the OVERRIDE ON button in domegui MANUAL tab.</font> 
 + 
 +<font 14px/Arial,Helvetica,sans-serif;;#333333;;inherit>4. Move the scope a bit back toward the direction it was coming from – for example, if the scope stopped at AZ 268 while rotating clockwise, move it back to 265 or so using AZ DEC. Then press STOP.</font> 
 + 
 +<font 14px/Arial,Helvetica,sans-serif;;#333333;;inherit>5. Move the scope past AZ 270 by pressing AC INC. Usally 2-4 degrees will do.</font> 
 + 
 +<font 14px/Arial,Helvetica,sans-serif;;#333333;;inherit>6. Click OVERRIDE OFF, go to the AUTO tab and press AUTO, then NEXT (also in the obsgtk; this will restore the original star's demand position to the scope).</font> 
 + 
 +<font 14px/Arial,Helvetica,sans-serif;;#333333;;inherit>7. When you have time, go to the scope and reset the limit switch; otherwise, it will stop each time you pass AZ 270/90.</font> 
  
 ==== The Telescope won't track ==== ==== The Telescope won't track ====
Line 147: Line 182:
 {{:chara:files:e1_hut_coms_recovery.pdf|E1 HuT and Cooler Communications Recovery}}  \\ {{:chara:files:e1_hut_coms_recovery.pdf|E1 HuT and Cooler Communications Recovery}}  \\
  \\  \\
-<font 16px/inherit;;inherit;;inherit>**S2 Finder Server Restart**</font>+<font 16px/inherit;;inherit;;inherit>**S2 Acquisition Server Restart**</font>
  
-S2'finder server resides on "S2pi4" computer. Logging on to it and running bootlaunch will restart it+S2'acquisition server resides on "S2pi4" computer. Logging on to it (ssh s2pi4) and running bootlaunch will restart it
  
-====== Dome issues ======+====== Dome issue ======
  
  \\  \\
Line 268: Line 303:
  
   * Good metrology signals are important to the proper positioning of the carts. Monitor the signal strength by running [RUN MULTIPLE] on the Metrology Monitor. Place the windows above the TV windows for each scope you are using. They should show white sine waves that are around the height of the window. Erratic, fluctuating waves indicate self-interference or a weak signal. This may cause the carts to lose their place as the signal strength falls too low. Red waves indicate that some displayed signal has gone too low and the carts will all need to be homed. A careful adjustment of the MET2 mirror can sometimes bring the signal back. Do not adjust the MET1 mirror.   * Good metrology signals are important to the proper positioning of the carts. Monitor the signal strength by running [RUN MULTIPLE] on the Metrology Monitor. Place the windows above the TV windows for each scope you are using. They should show white sine waves that are around the height of the window. Erratic, fluctuating waves indicate self-interference or a weak signal. This may cause the carts to lose their place as the signal strength falls too low. Red waves indicate that some displayed signal has gone too low and the carts will all need to be homed. A careful adjustment of the MET2 mirror can sometimes bring the signal back. Do not adjust the MET1 mirror.
-  * To home the carts, turn off the [OL] and [MAN] buttons on each cart and it will automatically return to the front switch. If the cart has no issue, it will arrive at the target position of 0m and the home switch at the same time. The X in the OPLE server under the HM will indicate it has homed to the home switch. If a cart does not reach the home switch when it returns to position 0m, it was lost and likely the cause of the difficulty in finding fringes. Hit the [HOME] button and it should move forward and trigger the home switch. Hit [TRACK] to home it to the new home position.+  * To home the carts, turn off the [OL] and [MAN] buttons on each cart and it will automatically return to the front switch. If the cart has no issue, it will arrive at the target position of 0m and the home switch at the same time. The X in the OPLE server under the HM will indicate it has homed to the home switch. If a cart does not reach the home switch when it returns to position 0m, it was lost and likely the cause of the difficulty in finding fringes. You can also find the size of the error by typing the command "homechk S1" into the ople server. Hit the [HOME] button and it should move forward and trigger the home switch. Hit [TRACK] to home it to the new home position. 
  
 ===== Beam Samplers ===== ===== Beam Samplers =====
chara/trouble_shooting.txt · Last modified: 2024/06/18 00:21 by charaobs