The CHARA Array Wiki

User Tools

Site Tools

Trace:
chara:trouble_shooting

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
chara:trouble_shooting [2018/07/05 11:05]
jones [__Telescopes and Dome Servers__] 		chara:trouble_shooting [2018/07/05 11:16]
jones [GUIs are frozen]
Line 73: Line 73:
 \\  This might mean that the gain for the tracking servo is wrong. Note that changing this gain can be dangerous, especially if you set it too high as that can cause the telescope to oscillated and damage the drives. Please only do this if you are very very sure that it is necessary. Symptoms of bad gain are:\\ \\  The scope over shoots the position while slewing. The star will be seen to move out of the window and may come back after a few seconds. This means the slewing gain is too low.\\  The scope oscillates when tracking or after a slew. The star will be tracing an ellipse, figure eight or other looping shape. This means the tracking gain is too low. You can damp this out with the telescope or dome gui by disabling the scope, then re-enabling it. Adjust the gain upward and watch it on the next slew.\\ \\  In all cases if either gain is too high the scope will go into "Fog Horn" mode, which is bad. You always want to use the lowest gain that still allows the scope to work as best as possible. If the tiptilt tells you the scope is oscillating slowly, the gain may be too low. If it is oscillating quickly it may be too high.\\ \\  On 10-22-2016, the gain settings were:\\ \\  \\  This might mean that the gain for the tracking servo is wrong. Note that changing this gain can be dangerous, especially if you set it too high as that can cause the telescope to oscillated and damage the drives. Please only do this if you are very very sure that it is necessary. Symptoms of bad gain are:\\ \\  The scope over shoots the position while slewing. The star will be seen to move out of the window and may come back after a few seconds. This means the slewing gain is too low.\\  The scope oscillates when tracking or after a slew. The star will be tracing an ellipse, figure eight or other looping shape. This means the tracking gain is too low. You can damp this out with the telescope or dome gui by disabling the scope, then re-enabling it. Adjust the gain upward and watch it on the next slew.\\ \\  In all cases if either gain is too high the scope will go into "Fog Horn" mode, which is bad. You always want to use the lowest gain that still allows the scope to work as best as possible. If the tiptilt tells you the scope is oscillating slowly, the gain may be too low. If it is oscillating quickly it may be too high.\\ \\  On 10-22-2016, the gain settings were:\\ \\ 
  
-| Scope\\ | AZ Slewing\\ ||| EL Slewing\\ ||| AZ Tracking\\ ||| EL Tracking\\ ||| Date Updated\\ |+| Scope\\ | AZ Slewing\\ || EL Slewing\\ || AZ Tracking\\ || EL Tracking\\ || Date Updated\\ |
 | \\ | Gain\\ | Fn\\ | Gain\\ | Fn\\ | Gain\\ | Fn\\ | Gain\\ | Fn\\ | \\ | | \\ | Gain\\ | Fn\\ | Gain\\ | Fn\\ | Gain\\ | Fn\\ | Gain\\ | Fn\\ | \\ |
 | S1\\ | 7\\ | 4\\ | 4\\ | 3\\ | 16\\ | 4\\ | 10\\ | 7\\ | 10-26-2017\\ | | S1\\ | 7\\ | 4\\ | 4\\ | 3\\ | 16\\ | 4\\ | 10\\ | 7\\ | 10-26-2017\\ |
Line 83: Line 83:
 \\  Note that these values may change as the temperatures change rapidly. These are still being adjusted so this is by no means final. Be sure to set mode back to AUTO if changing the gains left it in Slewing or Tracking mode.\\ \\  \\  Note that these values may change as the temperatures change rapidly. These are still being adjusted so this is by no means final. Be sure to set mode back to AUTO if changing the gains left it in Slewing or Tracking mode.\\ \\ 
 ==== How to Adjust CPUMotor Gains ==== ==== How to Adjust CPUMotor Gains ====
-\\  The GAIN controls the gain of the feedback between the encoder and the drive velocity. A high gain means a "stiffer" response, but can lead to oscillations or fog-horning if it's too high.\\ \\  The Fn controls the maximum frequency of the servo response. A high Fn means higher frequencies are allowed through, which can mean correcting for faster problems but if too high can also lead to oscillations or fog-horning.\\ \\  The software will not make changes to either of these quickly as that is a dangerous thing to do. There is NO POINT to clicking the up or down buttons more than once every few seconds. Indeed it is bad to do so as you will confuse the software. The change between slewing and tracking is also slow for similar reasons. This is why sometimes the "wrong" thing seems to change. It is a sign that you are trying to do things too quickly.\\ \\  Slew and tracking mode work differently, mostly because the speeds are so different.\\ \\  IN SLEWING MODE\\ \\  - If the gain is too low you will overshoot the target.\\ \\  - If the gain is too high it will fog-horn.\\ \\  - If the Fn is too high it will also fog-horn, even at low gains.\\ \\  You need to have the lowest possible gain and Fn in slew mode that doesn't overshoot the target. Fn in slew mode should almost never be higher than 4. If it is, please turn it back down to 4. If you think this is a problem please let Theo know, along with a detailed explanation of what happened.\\ \\  IN TRACKING MODE\\ \\  - If the gain is too low it will keep moving between slewing and tracking.\\ \\  - If the gain is too high it will fog-horn.\\ \\  - The same goes for Fn.\\ \\  In tracking mode you want the highest gain and Fn that allows the telescope to track well without fog-horning. If it "oscillates", which you will see in the green dots of tiptilt oscillating, try turning up the gain, and also try turning down the Fn.\\ \\  Some final remarks:\\ \\  - The gain is temperature dependent, so when the temperature changes these things will change, but more so for tracking.\\ \\  - The tiptilt system almost never causes oscillations, it almost always shows you that the scope is oscillating. If the white dots are centered on tiptilt and the green dots are moving the tiptilt is doing it's job and correcting for scope motion.\\ \\  - If a drive gets disabled at the end of a slew, the gain is too low. At low gain and low velocity the encoder signal changes very slowly or doesn't change at all. After 5 secs the software interprets this as a stall and disables the drive. The tricky part is that increasing the gain to avoid this situation might make the telescope to oscillate during the next slew. So the gain should be low (4 or even 1) during slewing but higher 7 or 10 when the telescope is basically at the target position.\\ {{https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif|external image cleardot.gif}}\\ \\ [[/Trouble%20Shooting#page.troubleshooting|Back to Main Menu]]\\ \\ \\  +\\  The GAIN controls the gain of the feedback between the encoder and the drive velocity. A high gain means a "stiffer" response, but can lead to oscillations or fog-horning if it's too high.\\ \\  The Fn controls the maximum frequency of the servo response. A high Fn means higher frequencies are allowed through, which can mean correcting for faster problems but if too high can also lead to oscillations or fog-horning.\\ \\  The software will not make changes to either of these quickly as that is a dangerous thing to do. There is NO POINT to clicking the up or down buttons more than once every few seconds. Indeed it is bad to do so as you will confuse the software. The change between slewing and tracking is also slow for similar reasons. This is why sometimes the "wrong" thing seems to change. It is a sign that you are trying to do things too quickly.\\ \\  Slew and tracking mode work differently, mostly because the speeds are so different.\\ \\  IN SLEWING MODE\\ \\  - If the gain is too low you will overshoot the target.\\ \\  - If the gain is too high it will fog-horn.\\ \\  - If the Fn is too high it will also fog-horn, even at low gains.\\ \\  You need to have the lowest possible gain and Fn in slew mode that doesn't overshoot the target. Fn in slew mode should almost never be higher than 4. If it is, please turn it back down to 4. If you think this is a problem please let Theo know, along with a detailed explanation of what happened.\\ \\  IN TRACKING MODE\\ \\  - If the gain is too low it will keep moving between slewing and tracking.\\ \\  - If the gain is too high it will fog-horn.\\ \\  - The same goes for Fn.\\ \\  In tracking mode you want the highest gain and Fn that allows the telescope to track well without fog-horning. If it "oscillates", which you will see in the green dots of tiptilt oscillating, try turning up the gain, and also try turning down the Fn.\\ \\  Some final remarks:\\ \\  - The gain is temperature dependent, so when the temperature changes these things will change, but more so for tracking.\\ \\  - The tiptilt system almost never causes oscillations, it almost always shows you that the scope is oscillating. If the white dots are centered on tiptilt and the green dots are moving the tiptilt is doing it's job and correcting for scope motion.\\ \\  - If a drive gets disabled at the end of a slew, the gain is too low. At low gain and low velocity the encoder signal changes very slowly or doesn't change at all. After 5 secs the software interprets this as a stall and disables the drive. The tricky part is that increasing the gain to avoid this situation might make the telescope to oscillate during the next slew. So the gain should be low (4 or even 1) during slewing but higher 7 or 10 when the telescope is basically at the target position.\\ {{https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif|external image cleardot.gif}}\\ 
-====== **Dome issues** ======+======Dome issues======
 \\  At times the domes do not rotate, open or close, or otherwise behave. Some problems are simple and others are more complex.\\ \\  \\  At times the domes do not rotate, open or close, or otherwise behave. Some problems are simple and others are more complex.\\ \\ 
-==== **Dome does not rotate** ====+====Dome does not rotate====
 \\  Sometimes when observing, the dome will not follow the telescope during a slew. This can happen when the Autodome feature is not turned on. Click the ON button on the MAIN tab of the telescope gui to enable it. This may happen after a server restart so always check the dome position with the spycam during a slew after a server restart. Also make sure the target position of the dome matches the telescope's position. If not, it will insist on being in the wrong place. If it is not at the same AZ as the scope, manually move it until it is centered on the telescope in spycam 1. If the dome AZ does not read the same as the telescope AZ, enter the scope AZ in the position box of the DOME tab of appropriate dome server and hit the INIT POS button to tell it at what AZ it is.\\ \\  If the dome does not turn at all, even with the manual controls on the telescope or dome guis, the control may be set to manual on the control box. This can happen if there was work done at the dome during the day. If the dome opens, but does not turn, check to see that control of the dome rotation is in the computer position and not manual on the dome rotation controller box just inside the door of the bunker.\\ \\  \\  Sometimes when observing, the dome will not follow the telescope during a slew. This can happen when the Autodome feature is not turned on. Click the ON button on the MAIN tab of the telescope gui to enable it. This may happen after a server restart so always check the dome position with the spycam during a slew after a server restart. Also make sure the target position of the dome matches the telescope's position. If not, it will insist on being in the wrong place. If it is not at the same AZ as the scope, manually move it until it is centered on the telescope in spycam 1. If the dome AZ does not read the same as the telescope AZ, enter the scope AZ in the position box of the DOME tab of appropriate dome server and hit the INIT POS button to tell it at what AZ it is.\\ \\  If the dome does not turn at all, even with the manual controls on the telescope or dome guis, the control may be set to manual on the control box. This can happen if there was work done at the dome during the day. If the dome opens, but does not turn, check to see that control of the dome rotation is in the computer position and not manual on the dome rotation controller box just inside the door of the bunker.\\ \\ 
-==== **Dome does not open** ====+====Dome does not open====
 \\  If the dome won't open, try hitting the SLIT CLOSE button on the telescope gui first. It may be that there was a computer or server issue and the computer thinks the dome is open already and won't allow it to open again. Hit the SLIT OPEN button to see if it works now. If it still won't open, go to the bunker and look to see if the power switch is on to the dome. It will be up and red if on.\\  Also look to see if the power cable is connected to the computer box. If it's connected up high to the manual dome controls, the cable needs to be brought back to the computer box connector. __Turn the power off first__ to the dome before disconnecting the power cable. Slide it onto the connector at the box and turn the collar to lock it in place. Turn the power back on and see if you can open the dome from the computer in the bunker.\\ \\  \\  If the dome won't open, try hitting the SLIT CLOSE button on the telescope gui first. It may be that there was a computer or server issue and the computer thinks the dome is open already and won't allow it to open again. Hit the SLIT OPEN button to see if it works now. If it still won't open, go to the bunker and look to see if the power switch is on to the dome. It will be up and red if on.\\  Also look to see if the power cable is connected to the computer box. If it's connected up high to the manual dome controls, the cable needs to be brought back to the computer box connector. __Turn the power off first__ to the dome before disconnecting the power cable. Slide it onto the connector at the box and turn the collar to lock it in place. Turn the power back on and see if you can open the dome from the computer in the bunker.\\ \\ 
 ==== Dome does not close ==== ==== Dome does not close ====
 \\  If a dome will not close after observing, it may need to be closed manually at the bunker. Turn the power to the dome off at the switch, disconnect the cable from the computer control box and connect it to the manual control switches above. Turn on the power to the dome after connecting the cable and use the two controls to close the dropout first, then the slit. The dropout must be fully closed and the slit closed over it to be shut properly. Inform the staff that the dome did not function properly so someone can look at it in the morning.\\ \\ \\  \\  If a dome will not close after observing, it may need to be closed manually at the bunker. Turn the power to the dome off at the switch, disconnect the cable from the computer control box and connect it to the manual control switches above. Turn on the power to the dome after connecting the cable and use the two controls to close the dropout first, then the slit. The dropout must be fully closed and the slit closed over it to be shut properly. Inform the staff that the dome did not function properly so someone can look at it in the morning.\\ \\ \\ 
-====== **HUT servers** ======+======HUT servers======
 \\  \\ 
-==== **I can't change the camera settings on the TV** ==== +====I can't change the camera settings on the TV==== 
-\\  The HUT servers control functions such as finder and acq exposure times and gains, heater and dehumidifier usage, and various AO functions. An observer may find that the camera settings do not display or are not adjustable. The HUT server may be the cause if it has quit. To see if it is the server, open the HUT gui for the desired telescope from the CHARA menu. If the camera settings are displayed and the settings can be changed from the gui, then the HUT server is ok. Restart the telescope server to reopen the connection to the HUT server and hit reopen on Cosmic Debris as well. If the gui is not functioning, the HUT server will need to be restarted. The HUT servers run on the computer at each telescope. Find the PID from Sockman for the server you want to shut down. Open a terminal and log into the correct telescope computer (for example, type "s1" to log on to S1). Use the command "ps aux | grep HUT" to find the process you need to shut down. The number should match the PID from Sockman. Kill the process by typing "kill -9 pid" where pid is the process number and remove the entry from Sockman with the REMOVE button on the gui or by command line. Restart the hut server using the new "bootlaunch_hut" command described in the section on [[/Trouble%20Shooting#chapter.restart.servers|Restarting Servers]]. If the server won't restart, a reboot of the power supply in the telescope bunker might be necessary. The power supply that controls the acquisition and finder cameras as well as their controllers is located on top of the computer rack in each bunker. The power supply has green readouts of volts and current. After turning the power off for 10 seconds and back on, try restarting the server from the computer in the bunker to see if it starts cleanly. If so, then restart the telescope server, reopen the connection to the telescope gui, and hit REOPEN on Cosmic Debris. Part of the HUT server also controls the AO table. If the AOB part of the HUT server doesn't work, then the power supply on the back of the AO table in the telescope dome might need recycling. This power supply controls the actuators at M2 and the AO table. The power supply box is a 6x9 inch aluminum box on the back of the AO table, behind the keyboard and monitor. Turn it off with the power button on the bottom edge of the box, wait 5 seconds and turn it back on. The HUT server should now restart cleanly. Restart the telescope server as well to make the connections to the telescope gui. Hit REOPEN in Cosmic Debris if you are observing to make all needed connections.\\ \\ \\ +\\  The HUT servers control functions such as finder and acq exposure times and gains, heater and dehumidifier usage, and various AO functions. An observer may find that the camera settings do not display or are not adjustable. The HUT server may be the cause if it has quit. To see if it is the server, open the HUT gui for the desired telescope from the CHARA menu. If the camera settings are displayed and the settings can be changed from the gui, then the HUT server is ok. Restart the telescope server to reopen the connection to the HUT server and hit reopen on Cosmic Debris as well. If the gui is not functioning, the HUT server will need to be restarted. The HUT servers run on the computer at each telescope. Find the PID from Sockman for the server you want to shut down. Open a terminal and log into the correct telescope computer (for example, type "s1" to log on to S1). Use the command "ps aux | grep HUT" to find the process you need to shut down. The number should match the PID from Sockman. Kill the process by typing "kill -9 pid" where pid is the process number and remove the entry from Sockman with the REMOVE button on the gui or by command line. Restart the hut server using the new "bootlaunch_hut" command described in the section on [[chara:trouble_shooting#restarting_servers|Restarting Servers]]. If the server won't restart, a reboot of the power supply in the telescope bunker might be necessary. The power supply that controls the acquisition and finder cameras as well as their controllers is located on top of the computer rack in each bunker. The power supply has green readouts of volts and current. After turning the power off for 10 seconds and back on, try restarting the server from the computer in the bunker to see if it starts cleanly. If so, then restart the telescope server, reopen the connection to the telescope gui, and hit REOPEN on Cosmic Debris. Part of the HUT server also controls the AO table. If the AOB part of the HUT server doesn't work, then the power supply on the back of the AO table in the telescope dome might need recycling. This power supply controls the actuators at M2 and the AO table. The power supply box is a 6x9 inch aluminum box on the back of the AO table, behind the keyboard and monitor. Turn it off with the power button on the bottom edge of the box, wait 5 seconds and turn it back on. The HUT server should now restart cleanly. Restart the telescope server as well to make the connections to the telescope gui. Hit REOPEN in Cosmic Debris if you are observing to make all needed connections.\\ \\ \\ 
 ===== Tiptilt Server ===== ===== Tiptilt Server =====
 \\  The tiptilt server controls the CCD based tiptilt detection system.\\ \\  Before you start the tiptilt server, you must ensure that the power to the cooling system and the CCD iteslf is on. It is extremely important that the cooler be running before you turn on the CCD and is only turned off if you are sure the CCD is NOT running. You can start the server from the X windows menu or with the command xtiptilt.\\ \\  Note that there are background counts and read noise to deal with. Whenever you change the frame rate, please ensure that the bias frame is OK. The server will attempt to load an old bias frame that should work, but if things are not working, try making a new bias frame by ensuring that the detector is in the dark and typing "mkbias" into the tiptilt server.\\ \\  In the tiptilt GUI windows, the white dots represent the starlight while the green dots represent the motion applied to telescope's secondary mirror to keep the starlight centered. When tiptilt is locked the white dots will be brought to the center of the tiptilt window. The green dots should be mostly centered also. W2 and E2 telescopes have a small oscillation that show as back and forth plots of the green dots.\\ \\ \\  \\  The tiptilt server controls the CCD based tiptilt detection system.\\ \\  Before you start the tiptilt server, you must ensure that the power to the cooling system and the CCD iteslf is on. It is extremely important that the cooler be running before you turn on the CCD and is only turned off if you are sure the CCD is NOT running. You can start the server from the X windows menu or with the command xtiptilt.\\ \\  Note that there are background counts and read noise to deal with. Whenever you change the frame rate, please ensure that the bias frame is OK. The server will attempt to load an old bias frame that should work, but if things are not working, try making a new bias frame by ensuring that the detector is in the dark and typing "mkbias" into the tiptilt server.\\ \\  In the tiptilt GUI windows, the white dots represent the starlight while the green dots represent the motion applied to telescope's secondary mirror to keep the starlight centered. When tiptilt is locked the white dots will be brought to the center of the tiptilt window. The green dots should be mostly centered also. W2 and E2 telescopes have a small oscillation that show as back and forth plots of the green dots.\\ \\ \\ 
Line 135: Line 135:
  Try using a slower frame rate or increasing the NSUM. Also ensure that the acquistion is properly aligned with the laser. To change the frame rate, click the [EXP] button on the tiptilt GUI and enter a longer integration time. Remember to change "Tiptilt (mS)" on Cosmic Debris to keep the same exposure time when slewing to the next target.\\ \\   Try using a slower frame rate or increasing the NSUM. Also ensure that the acquistion is properly aligned with the laser. To change the frame rate, click the [EXP] button on the tiptilt GUI and enter a longer integration time. Remember to change "Tiptilt (mS)" on Cosmic Debris to keep the same exposure time when slewing to the next target.\\ \\ 
 ==== Tiptilt counts are negative ==== ==== Tiptilt counts are negative ====
- The bias frame is bad. Get a new one or turn it off.\\ \\ [[/Trouble%20Shooting#page.troubleshooting|Back to Main Menu]]\\ \\  + The bias frame is bad. Get a new one or turn it off.\\ 
-===== __OPLE and Metrology__ =====+=====OPLE and Metrology=====
 \\  \\ 
 ==== OPLE Server doesn't open - complains about not finding run_ople ==== ==== OPLE Server doesn't open - complains about not finding run_ople ====
Line 148: Line 148:
   * Type "make install"   * Type "make install"
   * This will install run_ople. Try opening the ople server again.   * This will install run_ople. Try opening the ople server again.
-==== **Metrology and Homing carts** ====+====Metrology and Homing carts====
  
   * Good metrology signals are important to the proper positioning of the carts. Monitor the signal strength by running [RUN MULTIPLE] on the Metrology Monitor. Place the windows above the TV windows for each scope you are using. They should show white sine waves that are around the height of the window. Erratic, fluctuating waves indicate self-interference or a weak signal. This may cause the carts to lose their place as the signal strength falls too low. Red waves indicate that some displayed signal has gone too low and the carts will all need to be homed. A careful adjustment of the MET2 mirror can sometimes bring the signal back. Do not adjust the MET1 mirror.   * Good metrology signals are important to the proper positioning of the carts. Monitor the signal strength by running [RUN MULTIPLE] on the Metrology Monitor. Place the windows above the TV windows for each scope you are using. They should show white sine waves that are around the height of the window. Erratic, fluctuating waves indicate self-interference or a weak signal. This may cause the carts to lose their place as the signal strength falls too low. Red waves indicate that some displayed signal has gone too low and the carts will all need to be homed. A careful adjustment of the MET2 mirror can sometimes bring the signal back. Do not adjust the MET1 mirror.
   * To home the carts, turn off the [OL] and [MAN] buttons on each cart and it will automatically return to the front switch. If the cart has no issue, it will arrive at the target position of 0m and the home switch at the same time. The X in the OPLE server under the HM will indicate it has homed to the home switch. If a cart does not reach the home switch when it returns to position 0m, it was lost and likely the cause of the difficulty in finding fringes. Hit the [HOME] button and it should move forward and trigger the home switch. Hit [TRACK] to home it to the new home position.   * To home the carts, turn off the [OL] and [MAN] buttons on each cart and it will automatically return to the front switch. If the cart has no issue, it will arrive at the target position of 0m and the home switch at the same time. The X in the OPLE server under the HM will indicate it has homed to the home switch. If a cart does not reach the home switch when it returns to position 0m, it was lost and likely the cause of the difficulty in finding fringes. Hit the [HOME] button and it should move forward and trigger the home switch. Hit [TRACK] to home it to the new home position.
-\\ [[/Trouble%20Shooting#page.troubleshooting|Back to Main Menu]]\\ \\  +=====Beam Samplers=====
-===== __Beam Samplers__ =====+
 \\  \\ 
 ==== Beam Samplers are not moving ==== ==== Beam Samplers are not moving ====
Line 168: Line 167:
  The most common cause for getting multiple copies of a server is using the socket manage RESTART button, which is flaky at best. Don't do this. If you need to restart a server you should do it manually.\\ \\  1. Make sure you are logged into the right machine:\\ \\  ssh ctrscrut\\ \\  2. If you are not sure see if the socket manager will tell you. If it doesn't have a look in the file:\\ \\  /ctrscrut/chara/etc/sock_manager/socket_manager.list\\ \\  3. See if there are any ghosts running\\ \\  ps aux | grep esp_server\\ \\  Yes, it's hard to know which one it is with the esp servers. You can work out which ones are ghosts by typing the command\\ \\  tsockman |grep ctrscrut\\ \\  which will give you a list of the servers running on the machine you are interested in. Checking for non-matching PIDs will tell you which processes you need to stop.\\ \\  4. Stop those processes:\\ \\  kill -9 PID1 PID2 ....\\ \\  5. Make sure there is no sign of it in the socket manager\\ \\  tsockman rm BS\\ \\  5. Restart the beam sampler servers (this starts both beam sampler servers BS1 and BS2):\\ \\  bootlaunch_beamsamp\\ \\ \\   The most common cause for getting multiple copies of a server is using the socket manage RESTART button, which is flaky at best. Don't do this. If you need to restart a server you should do it manually.\\ \\  1. Make sure you are logged into the right machine:\\ \\  ssh ctrscrut\\ \\  2. If you are not sure see if the socket manager will tell you. If it doesn't have a look in the file:\\ \\  /ctrscrut/chara/etc/sock_manager/socket_manager.list\\ \\  3. See if there are any ghosts running\\ \\  ps aux | grep esp_server\\ \\  Yes, it's hard to know which one it is with the esp servers. You can work out which ones are ghosts by typing the command\\ \\  tsockman |grep ctrscrut\\ \\  which will give you a list of the servers running on the machine you are interested in. Checking for non-matching PIDs will tell you which processes you need to stop.\\ \\  4. Stop those processes:\\ \\  kill -9 PID1 PID2 ....\\ \\  5. Make sure there is no sign of it in the socket manager\\ \\  tsockman rm BS\\ \\  5. Restart the beam sampler servers (this starts both beam sampler servers BS1 and BS2):\\ \\  bootlaunch_beamsamp\\ \\ \\ 
 ==== "Failed to request position of S1" ==== ==== "Failed to request position of S1" ====
-\\  Cosmic Debris reports "Failed to request position of S1" when trying to set the beam order. This indicates that ople is no longer talking to the beam samplers. If the beam sampler server has been restarted recently, then the ople server will also need to be restarted to re-establish the connection.\\ \\  The beam sample server runs on ctrscrut. The ople server tries to open a connection to the beam sampler when it starts. If the beam sampler is not there, or dies,\\  you need to restart the ople server as there is no command to reopen that connection.\\ \\ [[/Trouble%20Shooting#page.troubleshooting|Back to Main Menu]]\\ \\  +\\  Cosmic Debris reports "Failed to request position of S1" when trying to set the beam order. This indicates that ople is no longer talking to the beam samplers. If the beam sampler server has been restarted recently, then the ople server will also need to be restarted to re-establish the connection.\\ \\  The beam sample server runs on ctrscrut. The ople server tries to open a connection to the beam sampler when it starts. If the beam sampler is not there, or dies,\\  you need to restart the ople server as there is no command to reopen that connection.\\ 
-===== __Remote Observing__ =====+=====Remote Observing=====
 \\  \\ 
 ==== VPN connection is not working ==== ==== VPN connection is not working ====
Line 180: Line 179:
   * If either Phase 1 or Phase 2 states say IDLE for a given VPN, then you need to re-establish the connection. To do this, click on the "DROP" button for that VPN. Then click on the "CONNECT" button and wait for the Phase 1 and Phase 2 connections to be established.   * If either Phase 1 or Phase 2 states say IDLE for a given VPN, then you need to re-establish the connection. To do this, click on the "DROP" button for that VPN. Then click on the "CONNECT" button and wait for the Phase 1 and Phase 2 connections to be established.
   * Make sure to close Firefox by using CTRL-Q to quit the program and not just close the window.   * Make sure to close Firefox by using CTRL-Q to quit the program and not just close the window.
-\\ [[/Trouble%20Shooting#page.troubleshooting|Back to Main Menu]]\\ \\  +=====General Problems=====
-===== __General Problems__ =====+
 \\  \\ 
 ==== GUIs are frozen ==== ==== GUIs are frozen ====
- Check to see if Sockman is working. Click the [LIST] button on sockman. If the list doesn't come up, Sockman may be hung up. Follow the Sockman restart procedure in the text file on the desktop if it won't respond.\\  Hit [REOPEN] on the gui to see if it reconnects with its server. if that does not fix it, the server may be dead. From the LIST on Sockman, select the server suspected of being frozen. Ping it and if it reports back as being dead, see [[/Trouble%20Shooting#chapter.general.frozenserver|Server is frozen]] section below.\\  Telescope guis often freeze. Hit reopen on the gui to bring it back to life. Hitting [MOVE] on the telescope gui before the star has stopped moving can cause the gui to freeze. Avoid doing this. Close the dead one and open a new one to fix the problem.\\ \\ + Check to see if Sockman is working. Click the [LIST] button on sockman. If the list doesn't come up, Sockman may be hung up. Follow the Sockman restart procedure in the text file on the desktop if it won't respond.\\  Hit [REOPEN] on the gui to see if it reconnects with its server. if that does not fix it, the server may be dead. From the LIST on Sockman, select the server suspected of being frozen. Ping it and if it reports back as being dead, see [[chara:trouble_shooting#server_is_frozen|Server is frozen]] section below.\\  Telescope guis often freeze. Hit reopen on the gui to bring it back to life. Hitting [MOVE] on the telescope gui before the star has stopped moving can cause the gui to freeze. Avoid doing this. Close the dead one and open a new one to fix the problem.\\ \\ 
 ==== Server display is blank or filled with jibberish ==== ==== Server display is blank or filled with jibberish ====
  
chara/trouble_shooting.txt · Last modified: 2024/06/18 00:21 by charaobs

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki