User Tools

Site Tools


chara:trouble_shooting

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
chara:trouble_shooting [2022/02/09 19:24]
charaobs
chara:trouble_shooting [2022/10/14 13:17]
gail_stargazer
Line 18: Line 18:
   * Check the time on the OPLE server. If the time is off or there are any lost ticks/seconds "Lost T/S" listed, then MSYNC ople using the GPS GUI and type "syncople" into the ople server.   * Check the time on the OPLE server. If the time is off or there are any lost ticks/seconds "Lost T/S" listed, then MSYNC ople using the GPS GUI and type "syncople" into the ople server.
   * Are the [MAN] buttons pressed (gray) for the moving carts on the OPLE Control gui? (The reference cart will remain green.)   * Are the [MAN] buttons pressed (gray) for the moving carts on the OPLE Control gui? (The reference cart will remain green.)
-  * Check that the carts are within delay line range (-1.3 to 44.25 meters) and errors are small. Were the carts homed and checked before the first slew of the night? Did any carts go to the front switch after slewing? This can cause them to lose their position. Turn off the OL button to send it back to zero. If it does not track on the home switch when displaying a zero position (0.000000), it was lost. Is the E2 cart stuck at 27.97m? See section on homing carts in __OPLE and Metrology__  section below.+  * Check that the carts are within delay line range (-1.3 to 44.25 meters) and errors are small. Were the carts homed and checked before the first slew of the night? Did any carts go to the front switch after slewing? This can cause them to lose their position. To check the homing of the carts, turn off the OL button to send it back to zero. If it does not track on the home switch when displaying a zero position (0.000000), it was lost. There is a new "CHECK" button on OPLE gtk that will send carts to home switch and report whether the is an error in the cart position. Is the E2 cart stuck at 27.97m? See section on homing carts in __OPLE and Metrology__  section below.
   * Are the LDC's working?   * Are the LDC's working?
       * Is the glass position within allowable range (-10mm to 49mm) on all beams?       * Is the glass position within allowable range (-10mm to 49mm) on all beams?
Line 26: Line 26:
  
   * Check the instrument alignment. Is flux getting through to the detector? How long has it been since the last NIRO camera alignment? Classic and CLIMB programs can run for about an hour before the light will drift from the central pixel. Use the Classic or CLIMB gui to view the light on the pixels by clicking the PICTURE tab and then the PIXEL AREA button. Turn the camera off with the STOP button. Is the right dither power turned on? CLIMB 1 and Classic use different dithers. If Classic or CLIMB fringes are found in a scan, but not when in recording mode, the dither powers are likely not on. Are the camera settings correct for the seeing conditions and flux levels?   * Check the instrument alignment. Is flux getting through to the detector? How long has it been since the last NIRO camera alignment? Classic and CLIMB programs can run for about an hour before the light will drift from the central pixel. Use the Classic or CLIMB gui to view the light on the pixels by clicking the PICTURE tab and then the PIXEL AREA button. Turn the camera off with the STOP button. Is the right dither power turned on? CLIMB 1 and Classic use different dithers. If Classic or CLIMB fringes are found in a scan, but not when in recording mode, the dither powers are likely not on. Are the camera settings correct for the seeing conditions and flux levels?
 +  * There is a script that will display offsets for all scopes against Hour Angle to help find offsets when using MIRCX and MYSTIC. You'll need to log into the MIRCX spooler computer with the command ssh spooler@mircx, then use the command fringe_predictor. A window will pop up displaying the last five nights of offsets vs. HA.
  
 ===== The new OPLE system ===== ===== The new OPLE system =====
Line 54: Line 55:
  
  \\ The Shutters server can become unresponsive or disconnected from the Socket Manager. This server must be restarted from the lab and not from the Control Room. Follow these instructions to restart it. Note that Shutters runs on ople, not ctrscrut. \\  \\ To start the shutter server on ople: \\  \\ Log into the ople computer and kill the process labeled shutters with the PID as described in **Restarting Servers**  above. \\ Turn off the power to the Shutters with the switch on the computer rack which is to the left of the computer desk and marked "SHUTTERS". Restart the Shutters server with the commands below. After restarting the server and testing the gui to see that it works, turn the SHUTTERS power back on with the switch. There is a printed sheet of directions in the lab to help you. \\  \\ /usr/local/bin/tsockman rm shutters \\ ctrscrut/usr/local/bin/shutter_server /ctrscrut/chara/etc/shutter.cfg &  \\ The Shutters server can become unresponsive or disconnected from the Socket Manager. This server must be restarted from the lab and not from the Control Room. Follow these instructions to restart it. Note that Shutters runs on ople, not ctrscrut. \\  \\ To start the shutter server on ople: \\  \\ Log into the ople computer and kill the process labeled shutters with the PID as described in **Restarting Servers**  above. \\ Turn off the power to the Shutters with the switch on the computer rack which is to the left of the computer desk and marked "SHUTTERS". Restart the Shutters server with the commands below. After restarting the server and testing the gui to see that it works, turn the SHUTTERS power back on with the switch. There is a printed sheet of directions in the lab to help you. \\  \\ /usr/local/bin/tsockman rm shutters \\ ctrscrut/usr/local/bin/shutter_server /ctrscrut/chara/etc/shutter.cfg &
 +
 +==== Restarting Socket Manager ====
 +
 +In very rare circumstances, the socket manager might need to be restarted. This is usually needed if GUIs stop communicating (e.g., OPLE GUI stops communicating with OPLE server). Another example occurs when mircx_bootLauncher tries to remove dead server names from the socket_manager before restarting. If the socket_manager does not respond, the mircx_bootLauncher will timeout.
 +
 +Follow these instructions to restart the socket manager, but only do so if really necessary:
 +
 +ssh -X observe@ctrscrut
 +
 +killall socket_manager
 +
 +socket_manager &
  
 ==== MIRC-X CredoneImAcq Server ==== ==== MIRC-X CredoneImAcq Server ====
Line 126: Line 139:
 You may have to reinitialize the scope on a bright star. If the powers were turned off quickly when the problem was noticed, the position of the scope should be retained and slewing to a bright star will get it in the finder. If not, you may need to go out to the telescope to find the bright star to reacquire the scope's position. You may have to reinitialize the scope on a bright star. If the powers were turned off quickly when the problem was noticed, the position of the scope should be retained and slewing to a bright star will get it in the finder. If not, you may need to go out to the telescope to find the bright star to reacquire the scope's position.
  
-Be aware that when restarting a dome server, the telescopes position may not be retained and the dome gui may display Az: 0.0. El: 0.0.  The gui will scroll messages that say Az: 0.0. El: 0.0 with an error in scope position.  If the scope is known to be at STOW position, you can manually enter the scope's AZ and EL using the manual tab on the dome gui and hitting the INIT button.+Be aware that when restarting a dome server, the telescopes position may not be retained and the dome gui may display Az: 0.0. El: 0.0. The gui will scroll messages that say Az: 0.0. El: 0.0 with an error in scope position. If the scope is known to be at STOW position, you can manually enter the scope's AZ and EL using the manual tab on the dome gui and hitting the INIT button.
  
-Also be aware that the gains do not always return to the normal values of Slew 4-7 and Tracking 7-10 when a dome server has crashed and is restarted.  They may come up as Gain 22 for all axes and modes and they will need to be turned down or the scopes will oscillate badly.+Also be aware that the gains do not always return to the normal values of Slew 4-7 and Tracking 7-10 when a dome server has crashed and is restarted. They may come up as Gain 22 for all axes and modes and they will need to be turned down or the scopes will oscillate badly.
  
 ==== Telescope is not receiving the commanded position for a target. ==== ==== Telescope is not receiving the commanded position for a target. ====
chara/trouble_shooting.txt ยท Last modified: 2023/11/21 01:42 by charaobs