User Tools

Site Tools


chara:trouble_shooting

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
chara:trouble_shooting [2021/11/09 14:06]
gail_stargazer
chara:trouble_shooting [2021/11/09 14:41]
gail_stargazer
Line 69: Line 69:
  
 Then restart credone using the desktop icon or by issuing the following command on spooler@mircx: \\ credoneImAcq –no-display Then restart credone using the desktop icon or by issuing the following command on spooler@mircx: \\ credoneImAcq –no-display
 +
 +(the first "–" should be two dashes: - and -).
  
 ===== Telescopes and Dome Servers ===== ===== Telescopes and Dome Servers =====
Line 82: Line 84:
 1. On the domegui MANUAL tab, click STOP so pulses won't be sent to the drive by the control software. \\ 1. On the domegui MANUAL tab, click STOP so pulses won't be sent to the drive by the control software. \\
 \\ \\
-2. Make sure you understand why the limit was hit which may require a trip to the telescope. If the azimuth positions on all telescope servers and dome guis match, it is likely the limit switch causing the stall and not that the scope is actually in a wrong position. \\ +2. Make sure you understand why the limit was hit which may require a trip to the telescope. If the azimuth positions on all telescope servers and dome guis match, it is likely the limit switch causing the stall and not that the scope is actually in a wrong position.</font><font inherit/inherit;;initial;;white>M</font>ake sure all the scopes’ demand positions agree – for example, sometimes bringing a scope to a configuration that’s already on sky and issuing a slew command will make the additional scope go around North the “wrong” way. \\  \\ <font 14px/Arial,Helvetica,sans-serif;;inherit;;inherit>3. Click the OVERRIDE ON button in domegui MANUAL tab. After this, the hardware doesn't care about the limits switches and you're free to move the telescope. \\  \\ 
-\\ +4. Click ENABLE then you can move the telescope back to its normal range of operation. \\  \\
-3. Click the OVERRIDE ON button in domegui MANUAL tab. After this, the hardware doesn't care about the limits switches and you're free to move the telescope. \\ +
-\\ +
-4. Click ENABLE then you can move the telescope back to its normal range of operation. \\ +
-\\+
 5. After the telescope is back in it normal range, click OVERRIDE OFF which makes the hardware aware of the limits again and then hit AUTO on the AUTO tab to resume normal operation <font 16px/Calibri,sans-serif;;#333333;;white>**However, If you notice that a telescope will stop near AZ 90 or 270 with the scope still being ENABLED and refusing to move, this is often due to the new AZ limit switch being tripped at some point earlier. To get it moving again:**</font> 5. After the telescope is back in it normal range, click OVERRIDE OFF which makes the hardware aware of the limits again and then hit AUTO on the AUTO tab to resume normal operation <font 16px/Calibri,sans-serif;;#333333;;white>**However, If you notice that a telescope will stop near AZ 90 or 270 with the scope still being ENABLED and refusing to move, this is often due to the new AZ limit switch being tripped at some point earlier. To get it moving again:**</font>
  
-<font 14px/Arial,Helvetica,sans-serif;;#333333;;white>1. On the domegui MANUAL tab, click STOP so pulses won't be sent to the drive by the control software.</font>  <font inherit/inherit;;initial;;white>2. M</font>ake sure the all the scopes’ demand positions agree – for example, sometimes bringing a scope to a configuration that’s already on sky and issuing a slew command will make the additional scope go around North the “wrong” way. <font inherit/inherit;;initial;;white>3. Click the OVERRIDE ON button in domegui MANUAL tab.</font>+<font 14px/Arial,Helvetica,sans-serif;;inherit;;inherit>1. On the domegui MANUAL tab, click STOP so pulses won't be sent to the drive by the control software. \\  \\ 
 +2.</font><font inherit/inherit;;initial;;white>M</font>ake sure all the scopes’ demand positions agree – for example, sometimes bringing a scope to a configuration that’s already on sky and issuing a slew command will make the additional scope go around North the “wrong” way. \\  \\ <font 14px/Arial,Helvetica,sans-serif;;inherit;;inherit>3. Click the OVERRIDE ON button in domegui MANUAL tab. After this, the hardware doesn't care about the limits switches and you're free to move the telescope.</font> 
 + 
 +<font inherit/inherit;;initial;;white>2. M</font>ake sure the all the scopes’ demand positions agree – for example, sometimes bringing a scope to a configuration that’s already on sky and issuing a slew command will make the additional scope go around North the “wrong” way. <font inherit/inherit;;initial;;white>3. Click the OVERRIDE ON button in domegui MANUAL tab.</font>
  
 <font 14px/Arial,Helvetica,sans-serif;;#333333;;inherit>4. Move the scope a bit back toward the direction it was coming from – for example, if the scope stopped at AZ 268 while rotating clockwise, move it back to 265 or so using AZ DEC. Then press STOP.</font> <font 14px/Arial,Helvetica,sans-serif;;#333333;;inherit>4. Move the scope a bit back toward the direction it was coming from – for example, if the scope stopped at AZ 268 while rotating clockwise, move it back to 265 or so using AZ DEC. Then press STOP.</font>
Line 181: Line 182:
 S2's acquisition server resides on "S2pi4" computer. Logging on to it (ssh s2pi4) and running bootlaunch will restart it S2's acquisition server resides on "S2pi4" computer. Logging on to it (ssh s2pi4) and running bootlaunch will restart it
  
-====== Dome issue ======+===== Dome issue =====
  
 At times the domes do not rotate, open or close, or otherwise behave. Some problems are simple and others are more complex. At times the domes do not rotate, open or close, or otherwise behave. Some problems are simple and others are more complex.
Line 199: Line 200:
 If a dome will not close after observing, it may need to be closed manually at the bunker. Turn the power to the dome off at the switch, disconnect the cable from the computer control box and connect it to the manual control switches above. Turn on the power to the dome after connecting the cable and use the two controls to close the dropout first, then the slit. The dropout must be fully closed and the slit closed over it to be shut properly. Inform the staff that the dome did not function properly so someone can look at it in the morning. If a dome will not close after observing, it may need to be closed manually at the bunker. Turn the power to the dome off at the switch, disconnect the cable from the computer control box and connect it to the manual control switches above. Turn on the power to the dome after connecting the cable and use the two controls to close the dropout first, then the slit. The dropout must be fully closed and the slit closed over it to be shut properly. Inform the staff that the dome did not function properly so someone can look at it in the morning.
  
-====== HUT servers ======+===== HUT servers ===== 
 + 
 +==== I can't change the camera settings on the TV ==== 
 + 
 +**CHECK THIS SECTION - THESE ARE VERY CLOSE BUT DIFFERENT PARAGRAPHS:**
  
 The HUT servers control functions such as beacon and dichroic movements, heater and dehumidifier usage, and various AO functions. An observer may find that the obsgtk is no longer controlling the beacon LED's, beacon flat or dichroic alignments. This happens on occasion with E2 and other scopes. The HUT server may be the cause if it has quit or lost connection or the AOB may be at fault. To see if it is the server, open the HUT gui for the desired telescope from the CHARA menu. If the alignments can be changed from the gui, then the HUT server is ok. You can use the hut gui to continue observing. If the hut gui gives move error messages, cycle the power on the AOB and open a new hut server to restore the connection to the obsgtk. On the POWER gui, turn off the power to the AOB for the offending telescope and turn it back on. Stop the hut server by logging into the appropriate telescope computer and identifying the PID with the bootlaunch_master command and killing the process with the kill -9 #### command. Start the new server via the bootlaunch_master command. Hit REOPEN on the obsgtk to reopen the connection to the HUT server and hit reopen on Cosmic Debris as well. The HUT servers control functions such as beacon and dichroic movements, heater and dehumidifier usage, and various AO functions. An observer may find that the obsgtk is no longer controlling the beacon LED's, beacon flat or dichroic alignments. This happens on occasion with E2 and other scopes. The HUT server may be the cause if it has quit or lost connection or the AOB may be at fault. To see if it is the server, open the HUT gui for the desired telescope from the CHARA menu. If the alignments can be changed from the gui, then the HUT server is ok. You can use the hut gui to continue observing. If the hut gui gives move error messages, cycle the power on the AOB and open a new hut server to restore the connection to the obsgtk. On the POWER gui, turn off the power to the AOB for the offending telescope and turn it back on. Stop the hut server by logging into the appropriate telescope computer and identifying the PID with the bootlaunch_master command and killing the process with the kill -9 #### command. Start the new server via the bootlaunch_master command. Hit REOPEN on the obsgtk to reopen the connection to the HUT server and hit reopen on Cosmic Debris as well.
 +
 +The HUT servers control functions such as finder and acq exposure times and gains, heater and dehumidifier usage, and various AO functions. An observer may find that the camera settings do not display or are not adjustable. The HUT server may be the cause if it has quit. To see if it is the server, open the HUT gui for the desired telescope from the CHARA menu. If the camera settings are displayed and the settings can be changed from the gui, then the HUT server is ok. Restart the telescope server to reopen the connection to the HUT server and hit reopen on Cosmic Debris as well. If the gui is not functioning, the HUT server will need to be restarted. The HUT servers run on the computer at each telescope. Find the PID from Sockman for the server you want to shut down. Open a terminal and log into the correct telescope computer (for example, type "s1" to log on to S1). Use the command "ps aux | grep HUT" to find the process you need to shut down. The number should match the PID from Sockman. Kill the process by typing "kill -9 pid" where pid is the process number and remove the entry from Sockman with the REMOVE button on the gui or by command line. Restart the hut server using the new "bootlaunch_hut" command described in the section on Restarting Servers.
 +
 +If the server won't restart, a reboot of the power supply in the telescope bunker might be necessary. The power supply that controls the acquisition and finder cameras as well as their controllers is located on top of the computer rack in each bunker. The power supply has green readouts of volts and current. After turning the power off for 10 seconds and back on, try restarting the server from the computer in the bunker to see if it starts cleanly. If so, then restart the telescope server, reopen the connection to the telescope gui, and hit REOPEN on Cosmic Debris. Part of the HUT server also controls the AO table. If the AOB part of the HUT server doesn't work, then the power supply on the back of the AO table in the telescope dome might need recycling. This power supply controls the actuators at M2 and the AO table. The power supply box is a 6×9 inch aluminum box on the back of the AO table, behind the keyboard and monitor. Turn it off with the power button on the bottom edge of the box, wait 5 seconds and turn it back on. The HUT server should now restart cleanly. Restart the telescope server as well to make the connections to the telescope gui. Hit REOPEN in Cosmic Debris if you are observing to make all needed connections.
  
 ===== Tiptilt Server. (These sections apply to the old lab tiptilt system retired in April 2021) ===== ===== Tiptilt Server. (These sections apply to the old lab tiptilt system retired in April 2021) =====
 +
 +[[:chara:old_lab_tiptilt_server|Instructions for the old lab tiptilit system are archived here.]]
  
 The tiptilt server controls the CCD based tiptilt detection system. \\  \\ The tiptilt server controls the CCD based tiptilt detection system. \\  \\
Line 304: Line 315:
 ==== Restarting the Beam Sampler Server ==== ==== Restarting the Beam Sampler Server ====
  
-The most common cause for getting multiple copies of a server is using the socket manage RESTART button, which is flaky at best. Don't do this. If you need to restart a server you should do it manually. 1. Make sure you are logged into the right machine: ssh ctrscrut 2. If you are not sure see if the socket manager will tell you. If it doesn't have a look in the file: /ctrscrut/chara/etc/sock_manager/socket_manager.list 3. See if there are any ghosts running ps aux | grep esp_server Yes, it's hard to know which one it is with the esp servers. You can work out which ones are ghosts by typing the command tsockman |grep ctrscrut which will give you a list of the servers running on the machine you are interested in. Checking for non-matching PIDs will tell you which processes you need to stop. 4. Stop those processes: kill -9 PID1 PID2 …. 5. Make sure there is no sign of it in the socket manager tsockman rm BS 5. Restart the beam sampler servers (this starts both beam sampler servers BS1 and BS2): bootlaunch_beamsamp+The most common cause for getting multiple copies of a server is using the socket manage RESTART button, which is flaky at best. Don't do this. If you need to restart a server you should do it manually. 
 + 
 +1. Make sure you are logged into the right machine: ssh ctrscrut 
 + 
 +2. If you are not sure see if the socket manager will tell you. If it doesn't have a look in the file: /ctrscrut/chara/etc/sock_manager/socket_manager.list 
 + 
 +3. See if there are any ghosts running ps aux | grep esp_server Yes, it's hard to know which one it is with the esp servers. You can work out which ones are ghosts by typing the command tsockman |grep ctrscrut which will give you a list of the servers running on the machine you are interested in. Checking for non-matching PIDs will tell you which processes you need to stop. 
 + 
 +4. Stop those processes: kill -9 PID1 PID2 …. 5. Make sure there is no sign of it in the socket manager tsockman rm BS 
 + 
 +5. Restart the beam sampler servers (this starts both beam sampler servers BS1 and BS2): bootlaunch_beamsamp
  
 ==== "Failed to request position of S1" ==== ==== "Failed to request position of S1" ====
Line 359: Line 380:
 If an object is not in SIMBAD (such as a recently discovered nova), then you must enter the object manually using the instructions below. If an object is not in SIMBAD (such as a recently discovered nova), then you must enter the object manually using the instructions below.
  
-==== Finding a star in the CHARA database ====+**Finding a star in the CHARA database**
  
 Here is an example of how to find the CHARA number of a star, nova, or AGN that is already in the CHARA database: Here is an example of how to find the CHARA number of a star, nova, or AGN that is already in the CHARA database:
chara/trouble_shooting.txt · Last modified: 2024/06/18 00:21 by charaobs