This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
chara:trouble_shooting [2022/01/18 22:32] charaobs |
chara:trouble_shooting [2023/02/08 19:30] charaobs [The Telescope won't move or stopped moving] |
||
---|---|---|---|
Line 9: | Line 9: | ||
* Were the clocks synced? Make sure the [SYNC CLOCKS] button on Cosmic Debris has been pushed to start the night. If the OPLE server does not display the correct CHARA time and the errors don't read (0) or (1), the clocks were not synced. | * Were the clocks synced? Make sure the [SYNC CLOCKS] button on Cosmic Debris has been pushed to start the night. If the OPLE server does not display the correct CHARA time and the errors don't read (0) or (1), the clocks were not synced. | ||
- | * Did the Astrolib update on OPLE? If the job queue is stopped too soon after slewing on Cosmic Debris, the correct calculations for the carts will not be done by OPLE and you may be searching for fringes with the wrong star data. Hit STAR ACQUIRED on CD to update | + | * Did the Astrolib update on OPLE? If the job queue is stopped too soon after slewing on Cosmic Debris, the correct calculations for the carts will not be done by OPLE and you may be searching for fringes with the wrong star data. The star identifier will be displayed in the server window. If it is not correct, hit STAR ACQUIRED on CD to update |
- | * Are the PoP's correct? After a PoP change, the PoP's are sometimes not updated in CD or ople. | + | * Are the PoP's correct? After a PoP change, the PoP's are sometimes not updated in CD or ople. Compare the PoP's in the configuration tab of CD or OPLE with the PoP Overview or Popperi gtk. |
* Are the carts behaving or are there vibrations or jumps of 100 or more microns every 3-6 seconds? How are the metrology signals? Are they strong and staying white? Red signals mean one or more metrology signals may have gone too low and homing carts is necessary. The METDATA function on the ople gui can be used to sample the metrology signal and may show noise spikes that can be disruptive to the smooth cart motion. Push the METDATA button on the configure tab and then again about 6 seconds later. Plots will pop up to show the frequency and power of any noise. | * Are the carts behaving or are there vibrations or jumps of 100 or more microns every 3-6 seconds? How are the metrology signals? Are they strong and staying white? Red signals mean one or more metrology signals may have gone too low and homing carts is necessary. The METDATA function on the ople gui can be used to sample the metrology signal and may show noise spikes that can be disruptive to the smooth cart motion. Push the METDATA button on the configure tab and then again about 6 seconds later. Plots will pop up to show the frequency and power of any noise. | ||
* Is the target a high proper motion star? Red dwarfs are close stars and can have high proper motions. Scan a wider range to see if it is outside of the usual calculated scan range. Binaries can also have very high offsets from the expected position due to mistakenly using astromod calculations from the companion star. | * Is the target a high proper motion star? Red dwarfs are close stars and can have high proper motions. Scan a wider range to see if it is outside of the usual calculated scan range. Binaries can also have very high offsets from the expected position due to mistakenly using astromod calculations from the companion star. | ||
- | * Do you have enough flux from each telescope or on each baseline? Is the telescope tiptilt struggling with low flux? Can the camera gain be raised or the exposure made longer to help hold the star? | + | * Do you have enough flux from each telescope or on each baseline? Is the telescope tiptilt struggling with low flux? Can the camera gain be raised or the exposure made longer to help hold the star? Passing clouds or contrails can lower flux unexpectedly. |
* Did you get the same star in each telescope? Sometimes a busy star field and poor pointing of the telescopes can lead to the wrong star being acquired and locked by tiptilt. View the stars in the finder window to see if all the stars match. | * Did you get the same star in each telescope? Sometimes a busy star field and poor pointing of the telescopes can lead to the wrong star being acquired and locked by tiptilt. View the stars in the finder window to see if all the stars match. | ||
* Check the CHARA time on the GPS server. The " | * Check the CHARA time on the GPS server. The " | ||
* Check the time on the OPLE server. If the time is off or there are any lost ticks/ | * Check the time on the OPLE server. If the time is off or there are any lost ticks/ | ||
* Are the [MAN] buttons pressed (gray) for the moving carts on the OPLE Control gui? (The reference cart will remain green.) | * Are the [MAN] buttons pressed (gray) for the moving carts on the OPLE Control gui? (The reference cart will remain green.) | ||
- | * Check that the carts are within delay line range (-1 to 44 meters) and errors are small. Were the carts homed and checked before the first slew of the night? Did any carts go to the front switch after slewing? This can cause them to lose their position. | + | * Check that the carts are within delay line range (-1.3 to 44.25 meters) and errors are small. Were the carts homed and checked before the first slew of the night? Did any carts go to the front switch after slewing? This can cause them to lose their position. |
* Are the LDC's working? | * Are the LDC's working? | ||
* Is the glass position within allowable range (-10mm to 49mm) on all beams? | * Is the glass position within allowable range (-10mm to 49mm) on all beams? | ||
Line 26: | Line 26: | ||
* Check the instrument alignment. Is flux getting through to the detector? How long has it been since the last NIRO camera alignment? Classic and CLIMB programs can run for about an hour before the light will drift from the central pixel. Use the Classic or CLIMB gui to view the light on the pixels by clicking the PICTURE tab and then the PIXEL AREA button. Turn the camera off with the STOP button. Is the right dither power turned on? CLIMB 1 and Classic use different dithers. If Classic or CLIMB fringes are found in a scan, but not when in recording mode, the dither powers are likely not on. Are the camera settings correct for the seeing conditions and flux levels? | * Check the instrument alignment. Is flux getting through to the detector? How long has it been since the last NIRO camera alignment? Classic and CLIMB programs can run for about an hour before the light will drift from the central pixel. Use the Classic or CLIMB gui to view the light on the pixels by clicking the PICTURE tab and then the PIXEL AREA button. Turn the camera off with the STOP button. Is the right dither power turned on? CLIMB 1 and Classic use different dithers. If Classic or CLIMB fringes are found in a scan, but not when in recording mode, the dither powers are likely not on. Are the camera settings correct for the seeing conditions and flux levels? | ||
+ | * There is a script that will display offsets for all scopes against Hour Angle to help find offsets when using MIRCX and MYSTIC. You'll need to log into the MIRCX spooler computer with the command ssh spooler@mircx, | ||
+ | |||
+ | ===== The new OPLE system ===== | ||
+ | |||
+ | With the implementation of the new ople system which replaced the VME in Fall of 2021, new troubleshooting issues have arisen. Since there are now 6 new ople computers to run the carts for each scope individually, | ||
+ | |||
+ | The traditional ople server will still be used to communicate with each new ople computer, identified as OPLE 1 to OPLE 6. When the communications are good, each active cart will be displayed in the ople server or ople gui status tab as before. At times, an ople computer can lose communications or a server can crash and the server or comms needs to be restarted. | ||
+ | |||
+ | If a cart cannot be started, stopped or otherwise commanded, look to the OPLESystem gui to see if a green indicator has turned red. A message will also often pop up on the ople gui saying the command could not be sent. This may require a simple start command to restart the server or a reboot of the computer to get it back to yellow and then a start to get it back to green. Do either of these steps with a right click of the red button and then select start or reboot. After a reboot, select start to load the servers after the indicator has turned yellow. When this is done, the ople server will need to be connected to the newly restarted server. Type " | ||
+ | |||
+ | Some times a cart is stopped and cannot be commanded. If the cart has gone to the front hard or back hard switch, it will not be usable until it is moved from the switch and the Ople Controller box is reset. There are 6 silver boxes for these controllers with two green LED's for the front and back switches and two red LED's for the back hard and front hard switches. If a red LED is lit, there will be an error displayed on the message window and the cart is disabled. The cart will need to be moved off the switch and then the box can be reset with the RESET button on the front. The error display will go away and the red LED will be off. The cart is now controllable. | ||
+ | |||
===== Restarting Servers ===== | ===== Restarting Servers ===== | ||
Line 33: | Line 45: | ||
If a server is not running or Socket Manager reports that a server is dead, then look at the socket manager list to find out what computer the server runs on ([[: | If a server is not running or Socket Manager reports that a server is dead, then look at the socket manager list to find out what computer the server runs on ([[: | ||
- | To restart a server, log on to the machine that runs the server and type " | + | To restart a server, log on to the machine that runs the server and type " |
- | * bootlaunch_beamsamp – Starts | + | ==== Restarting Servers using the rc.local file ==== |
- | * bootlaunch_zaber – Starts the ZABER_2 server. | + | |
- | \\ telescope bunker computers | + | \\ |
+ | This procedure is applicable to servers that have not switched over to the bootlaunch paradigm, which may not be any at this point.\\ | ||
+ | \\ | ||
+ | If a server is not running or Socket Manager reports that a server is dead, then look at the socket manager list to find out what computer the server runs on ([[: | ||
+ | \\ | ||
+ | Log on to the relevant computer by typing the computer name (ctrscrut, ople, s1, …). If the shortcut doesn' | ||
+ | \\ | ||
+ | Find out if the server is running by typing "ps aux | grep // | ||
+ | [ctrscrut: | ||
+ | observe 9281 0.0 0.0 61156 692 pts/3 S+ 13:58 0:00 grep pico_1\\ | ||
+ | observe 12578 0.0 0.0 24524 11212 ? S Jun16 33:14 / | ||
+ | \\ | ||
+ | If the entry for the dead server shows up in the process list, then identify the process identification number (12578 for the example above) and kill the server by typing "kill -9 // | ||
+ | \\ | ||
+ | Look up the commands to restart the server by typing "more / | ||
+ | \\ | ||
+ | #Start PICO server for PICO #1\\ | ||
+ | /bin/rm -f / | ||
+ | / | ||
+ | / | ||
+ | \\ | ||
+ | The first command removes the lock to allow the server to restart. The second command removes the name from the socket manager listing. The last command restarts the server. Note that if you are restarting the servers as observe, you will need to remove the part of the command in the rc.local file that saves information in / | ||
+ | \\ | ||
+ | There are text files on the desktop with many of the restart commands. Use these files for quick access to the relevant commands. The commands are edited and can be copied exactly as written. Files include Dome servers and all servers running on ctrscrut. Many of these commands are also located on the [[: | ||
- | * bootlaunch_hut – Starts the E1_HUT, E2_HUT, S1_HUT, S2_HUT, W1_HUT, or W2_HUT server, depending on the machine it's launched from. | + | Shutters Server |
- | * bootlaunch_rpc – Starts the RPC_E1, RPC_E2, RPC_S1, RPC_S2, RPC_W1, or RPC_W2 server, depending on the machine it's launched from. | + | |
- | * bootlaunch_weather – Starts the E1_WEATHER, E2_WEATHER, S1_WEATHER, S2_WEATHER, W1_WEATHER, or W2_WEATHER server, depending on the machine it's launched from. | + | |
- | * bootlaunch_lower – Starts the E1_Lower, E2_Lower, S1_Lower, S2_Lower, W1_Lower, or W2_Lower cylinder server, depending on the machine it's launched from. | + | |
- | * bootlaunch_upper – Starts the E1_Upper, E2_Upper, S1_Upper, S2_Upper, W1_Upper, or W2_Upper server, depending on the machine it's launched from. | + | |
- | \\ Note: The bootlaunch scripts will not start a new server | + | \\ |
+ | The Shutters server can become unresponsive or disconnected from the Socket Manager. This server must be restarted from the lab and not from the Control Room. Follow these instructions to restart it. Note that Shutters runs on ople, not ctrscrut.\\ | ||
+ | \\ | ||
+ | To start the shutter | ||
+ | \\ | ||
+ | Log into the ople computer and kill the process | ||
+ | Turn off the power to the Shutters with the switch on the computer rack which is to the left of the computer desk and marked " | ||
+ | \\ | ||
+ | / | ||
+ | ctrscrut/ | ||
- | ==== Restarting Servers using the rc.local file ==== | ||
- | \\ This procedure is applicable to servers that have not switched over to the bootlaunch paradigm, which may not be any at this point. \\ \\ If a server is not running or Socket Manager | + | ==== Restarting |
- | \\ | + | |
- | Log on to the relevant computer by typing the computer name (ctrscrut, ople, s1, …). If the shortcut doesn' | + | In very rare circumstances, the socket manager |
+ | |||
+ | Follow these instructions | ||
+ | |||
+ | ssh -X observe@ctrscrut | ||
- | ==== Shutters Server ==== | + | killall socket_manager |
- | \\ The Shutters server can become unresponsive or disconnected from the Socket Manager. This server must be restarted from the lab and not from the Control Room. Follow these instructions to restart it. Note that Shutters runs on ople, not ctrscrut. \\ \\ To start the shutter server on ople: \\ \\ Log into the ople computer and kill the process labeled shutters with the PID as described in **Restarting Servers** | + | socket_manager |
==== MIRC-X CredoneImAcq Server ==== | ==== MIRC-X CredoneImAcq Server ==== | ||
Line 73: | Line 115: | ||
===== Telescopes and Dome Servers ===== | ===== Telescopes and Dome Servers ===== | ||
- | |||
- | Here we discuss things that can go wrong with the telescopes. | ||
==== The Telescope won't move or stopped moving ==== | ==== The Telescope won't move or stopped moving ==== | ||
Line 80: | Line 120: | ||
Have the powers been turned on to the drives? Are the scopes disabled? The usual state of the telescopes is disabled until enabled. This is due to the stall function of the scopes which eventually disables the scopes when they are stowed. Enable the scopes by hitting [ENABLE] on the dome gui, the telescope gui control tab, or the bottom of the obsgtk. If a scope disables itself during a slew, it may be just an overcautious stall function or the slew gains may be too high or low. Check the gains and if they are OK, Hit ENABLE and the scope should continue to slew. If it disables again without moving, there may be something wrong at the scope, ie. a hatch left open, a ladder not put away, a tool on the floor, or something else physically impeding the motion of the scope. You will need to go to the dome to see what it is. The computer in the dome will give you control of the scope to turn it away from the problem. | Have the powers been turned on to the drives? Are the scopes disabled? The usual state of the telescopes is disabled until enabled. This is due to the stall function of the scopes which eventually disables the scopes when they are stowed. Enable the scopes by hitting [ENABLE] on the dome gui, the telescope gui control tab, or the bottom of the obsgtk. If a scope disables itself during a slew, it may be just an overcautious stall function or the slew gains may be too high or low. Check the gains and if they are OK, Hit ENABLE and the scope should continue to slew. If it disables again without moving, there may be something wrong at the scope, ie. a hatch left open, a ladder not put away, a tool on the floor, or something else physically impeding the motion of the scope. You will need to go to the dome to see what it is. The computer in the dome will give you control of the scope to turn it away from the problem. | ||
- | Sometimes the dome guis get hung up and can cause erratic motion or no motion of the scopes. Check them for current times and continuous updates of numbers. If they are not updating, try to REOPEN them first. If that does not work, close the gui and open a new one. If a new one does not open, the dome server may be dead or Sockman lost track of it. See **Dome Server Restart** | + | Sometimes the dome guis get hung up and can cause erratic motion or no motion of the scopes. Check them for current times and continuous updates of numbers. If they are not updating, try to REOPEN them first. If that does not work, close the gui and open a new one. If a new one does not open, the dome server may be dead or Sockman lost track of it. See **Dome Server Restart** below.\\ |
\\ | \\ | ||
- | 1. On the domegui MANUAL tab, click STOP so pulses won't be sent to the drive by the control software. | + | **Azimuth Limit Switches** |
\\ | \\ | ||
- | 2. Make sure you understand why the limit was hit which may require a trip to the telescope. If the azimuth positions on all telescope servers and dome guis match, it is likely the limit switch causing the stall and not that the scope is actually in a wrong position.</ | + | <font 14px/ |
- | 4. Click ENABLE then you can move the telescope back to its normal range of operation. \\ \\ | + | \\ |
- | 5. After the telescope is back in it normal range, click OVERRIDE OFF which makes the hardware aware of the limits again and then hit AUTO on the AUTO tab to resume normal operation | + | 1. On the domegui MANUAL tab, click STOP so pulses won't be sent to the drive by the control software.\\ |
+ | \\ | ||
+ | 2. Make sure you understand why the limit was hit which may require a trip to the telescope. If the azimuth positions on all telescope servers and dome guis match, it is likely the limit switch causing the stall and not that the scope is actually in a wrong position.</ | ||
+ | \\ | ||
+ | <font 14px/ | ||
+ | \\ | ||
+ | 4. Click ENABLE then you can move the telescope back to its normal range of operation.\\ | ||
+ | \\ | ||
+ | 5. After the telescope is back in it normal range, click OVERRIDE OFF which makes the hardware aware of the limits again and then hit AUTO on the AUTO tab to resume normal operation.</ | ||
- | <font 14px/ | + | <font 14px/ |
- | 2.</ | + | |
+ | <font 14px/ | ||
+ | \\ | ||
+ | 2.</ | ||
+ | \\ | ||
+ | <font 14px/ | ||
<font 14px/ | <font 14px/ | ||
- | <font 14px/ | + | <font 14px/ |
<font 14px/ | <font 14px/ | ||
- | <font 14px/ | + | <font 14px/ |
==== The Telescope won't track ==== | ==== The Telescope won't track ==== | ||
Line 109: | Line 163: | ||
==== Dome Server Restart ==== | ==== Dome Server Restart ==== | ||
- | Dome servers are now started using the bootlaunch_master command. | + | Dome servers are now started using the bootlaunch_master command. The manual process that has been superceded is archived. |
- | To manually start the dome server: | + | To restart |
- | 1. Make sure the power to the drives is OFF. | + | 1. Make sure the power to the drives is OFF. Disable the scopes. |
- | 2. Login to the relevant computer as root. For example, type "s1" or " | + | 2. Login to the relevant computer as observe. For example, type " |
- | 3. Work out the process ID number (PID), either with the command | + | 3. Work out the process ID number (PID) by typing bootlaunch_master |
- | (s1:1001) tsockman get dome_S1 \\ | + | 4. Use kill -2 PID to kill the server. Entering |
- | Name : dome_S1 \\ | + | |
- | Machine : s1.chara-array.org \\ | + | |
- | PID : 29953 \\ | + | |
- | Commands : -1 \\ | + | |
- | Data : -1 \\ | + | |
- | Message : 4002 \\ | + | |
- | Restart : / | + | |
- | or with \\ | + | |
- | (s1:1003) ps aux | grep dome \\ | + | |
- | theo 4473 0.0 0.0 61188 748 pts/3 S+ 10:45 0:00 grep dome \\ | + | |
- | observe 29953 18.5 0.4 35596 9860 ? Sl Apr21 416:11 / | + | |
- | It can also be found by pulling up the LIST on SOCKMAN and selecting | + | |
- | So in this case the PID is 29953. \\ \\ | + | |
- | 4. Try and stop the server gracefully: | + | |
- | 5. You should then check that the server | + | 5. Type bootlaunch_master again to restart |
- | [s1:600] tsockman get dome_S1 \\ | + | |
- | Name : dome_S1 \\ | + | |
- | Machine : s1.chara-array.org \\ | + | |
- | PID : 15635 \\ | + | |
- | Commands : -1 \\ | + | |
- | Data : -1 \\ | + | |
- | Message : 2008 \\ | + | |
- | Restart : / | + | |
- | If the socket manager still thinks it's running you will need to stop it forcefully: kill -9 29953; tsockman rm dome_S1 | + | |
- | 6. Restart | + | 6. Turn the power to the drives back on. |
- | [s1:602] more /etc/rc.local \\ | + | 7. Hit REOPEN and ENABLE on the domegtk, and type " |
- | <<< | + | |
- | #Run the dome server | + | |
- | / | + | |
- | / | + | |
- | (Note: the part of the command that saves information to / | + | |
- | 7. Turn the power to the drives back on. | + | You may have to reinitialize the scope on a bright star. If the powers were turned off quickly when the problem was noticed, the position of the scope should be retained and slewing |
- | 8. Hit REOPEN | + | Be aware that when restarting a dome server, the telescopes position may not be retained and the dome gui may display Az: 0.0. El: 0.0. The gui will scroll messages that say Az: 0.0. El: 0.0 with an error in scope position. If the scope is known to be at STOW position, you can manually enter the scope' |
- | You may have to reinitialize the scope on a bright star. If the powers were turned off quickly when the problem was noticed, | + | Also be aware that the gains do not always return |
- | ==== Telescope | + | ==== Telescope is not receiving the commanded position for a target. |
- | The NTP server has obsoleted this problem and fix as of 2014. | + | Sometimes it happens that a telescope receives the wrong position for a target or does not receive the commanded position at all. The commanded position is listed on the telescope server in the first column under TCS Az/El; the second column lists the actual position of the telescope. Try entering the star designation directly into the telescope server, ie. hd 123456. If it does not accept the number, try closing and restarting the telescope server and hitting repoen on Cosmic Debris and the telescope gui. Try entering the star into the server again. If that does not work, it is possible that something is wrong with the dome server. To restart the dome server follow these steps above. |
- | + | ||
- | ==== Telescope is not receiving the commanded position for a target ==== | + | |
- | + | ||
- | Sometimes it happens that a telescope receives the wrong position for a target or does not receive the commanded position at all. The commanded position is listed on the telescope server in the first column under TCS Az/El; the second column lists the actual position of the telescope. Try entering the star designation directly into the telescope server, ie. hd 123456. If it does not accept the number, try closing and restarting the telescope server and hitting repoen on Cosmic Debris and the telescope gui. Try entering the star into the server again. If that does not work, it is possible that something is wrong with the dome server. To restart the dome server follow these steps: * DISABLE the telescope using either the telescope or dome GUI. | + | |
- | + | ||
- | * Turn off the power for the telescope (both AZ/EL). | + | |
- | * First shutdown the telescope server | + | |
- | * Then use SOCKMAN to select the appropriate dome server from the list (dome_E1, etc) Get the PID number for reference. | + | |
- | * The dome server will need to be restarted from the command line in a terminal. | + | |
- | * Open a terminal window and log on to the telescope computer by typing " | + | |
- | * Look for instruction in the / | + | |
- | * Locate instruction listed under "#Run the dome server" | + | |
- | * / | + | |
- | * / | + | |
- | * (Replace S1 with commands for appropriate telescope) | + | |
- | * After the dome server is running, re-open the telescope server and click [REOPEN} on the dome and telescope GUIs and on Cosmic Debris. | + | |
- | * ENABLE the telescope using the telescope or dome GUI. Check to see if the command and telescope positions are correct. If they are, then turn on the power for the telescope drives. Re-enter the star information by entering the HD number in the telescope server and click [GO NEXT] to send the telescope to the star. Make sure that the telescope is behaving as expected. The telescope might have drifted a bit, so if you can't find the star, it might be necessary to use the Telrad on a bright star to re-initialize the pointing. | + | |
- | * Restart commands for the dome servers are also listed on the desktop in a text file. | + | |
==== Telescope is tracking poorly, overshooting in slew, oscillating. ==== | ==== Telescope is tracking poorly, overshooting in slew, oscillating. ==== | ||
- | This might mean that the gain for the tracking servo is wrong. Note that changing this gain can be dangerous, especially if you set it too high as that can cause the telescope to oscillated and damage the drives. Please only do this if you are very very sure that it is necessary. Symptoms of bad gain are: The scope over shoots the position while slewing. The star will be seen to move out of the window and may come back after a few seconds. This means the slewing gain is too low. The scope oscillates when tracking or after a slew. The star will be tracing an ellipse, figure eight or other looping shape. This means the tracking gain is too low. You can damp this out with the telescope or dome gui by disabling the scope, then re-enabling it. Adjust the gain upward and watch it on the next slew. In all cases if either gain is too high the scope will go into "Fog Horn" mode, which is bad. You always want to use the lowest gain that still allows the scope to work as best as possible. If the tiptilt tells you the scope is oscillating slowly, the gain may be too low. If it is oscillating quickly it may be too high. On 10-22-2016, the gain settings were: | + | This might mean that the gain for the tracking servo is wrong. Note that changing this gain can be dangerous, especially if you set it too high as that can cause the telescope to oscillated and damage the drives. Please only do this if you are very very sure that it is necessary. Symptoms of bad gain are: The scope over shoots the position while slewing. The star will be seen to move out of the window and may come back after a few seconds. This means the slewing gain is too low. The scope oscillates when tracking or after a slew. The star will be tracing an ellipse, figure eight or other looping shape. This means the tracking gain is too low. You can damp this out with the telescope or dome gui by disabling the scope, then re-enabling it. Adjust the gain upward and watch it on the next slew. In all cases if either gain is too high the scope will go into "Fog Horn" mode, which is bad. This can be seen during slews on the twfs or labao as vibrating spots, usually in one axis. |
- | |Scope|AZ Slewing| |AZ Slewing| |AZ Tracking| |EL Tracking| |Date Updated| | + | You always want to use the lowest gain that still allows the scope to work as best as possible. If the tiptilt tells you the scope is oscillating slowly, the gain may be too low. If it is oscillating quickly it may be too high. |
- | | |Gain|Fn|Gain|Fn|Gain|Fn|Gain|Fn| | | + | |
- | |S1|7|4|4|3|16|4|10|7|10-26-2017| | + | |
- | |S2|7| |7| |10| |10| |10-22-2016| | + | |
- | |E1|7| |7| |13| |10| |10-22-2016| | + | |
- | |E2|7| |7| |13| |10| |10-22-2016| | + | |
- | |W1|13| |10| |13| |10| |10-22-2016| | + | |
- | |W2|7| |7| |13| |13| |10-22-2016| | + | |
- | \\ | + | The usual values for slewing gains are 4-7 and tracking gains are 7-10. Note that these values may change as the temperatures change rapidly. |
- | Note that these values may change as the temperatures change rapidly. | + | |
==== How to Adjust CPUMotor Gains ==== | ==== How to Adjust CPUMotor Gains ==== | ||
Line 240: | Line 240: | ||
Sometimes when observing, the dome will not follow the telescope during a slew. This can happen when the Autodome feature is not turned on. Click the ON button on the MAIN tab of the telescope gui to enable it. This may happen after a server restart so always check the dome position with the spycam during a slew after a server restart. Also make sure the target position of the dome matches the telescope' | Sometimes when observing, the dome will not follow the telescope during a slew. This can happen when the Autodome feature is not turned on. Click the ON button on the MAIN tab of the telescope gui to enable it. This may happen after a server restart so always check the dome position with the spycam during a slew after a server restart. Also make sure the target position of the dome matches the telescope' | ||
- | If the dome does not turn at all, even with the manual controls on the telescope or dome guis, the control may be set to manual on the control box. This can happen if there was work done at the dome during the day. If the dome opens, but does not turn, check to see that control of the dome rotation is in the computer position and not manual on the dome rotation controller box just inside the door of the bunker. Sometimes the drive wheel jumps in the track and cannot turn the dome, even when the motor works. | + | If the dome does not turn at all, even with the manual controls on the telescope or dome guis, the control may be set to manual on the control box. This can happen if there was work done at the dome during the day. If the dome opens, but does not turn, check to see that control of the dome rotation is in the computer position and not manual on the dome rotation controller box just inside the door of the bunker. Sometimes the drive wheel jumps in the track and cannot turn the dome, even when the motor works. This will need to be fixed during the day. |
==== Dome does not open ==== | ==== Dome does not open ==== | ||
Line 253: | Line 253: | ||
===== HUT servers ===== | ===== HUT servers ===== | ||
- | |||
- | ==== I can't change the camera settings on the TV ==== | ||
- | |||
- | **CHECK THIS SECTION - THESE ARE VERY CLOSE BUT DIFFERENT PARAGRAPHS: | ||
The HUT servers control functions such as beacon and dichroic movements, heater and dehumidifier usage, and various AO functions. An observer may find that the obsgtk is no longer controlling the beacon LED's, beacon flat or dichroic alignments. This happens on occasion with E2 and other scopes. The HUT server may be the cause if it has quit or lost connection or the AOB may be at fault. To see if it is the server, open the HUT gui for the desired telescope from the CHARA menu. If the alignments can be changed from the gui, then the HUT server is ok. You can use the hut gui to continue observing. If the hut gui gives move error messages, cycle the power on the AOB and open a new hut server to restore the connection to the obsgtk. On the POWER gui, turn off the power to the AOB for the offending telescope and turn it back on. Stop the hut server by logging into the appropriate telescope computer and identifying the PID with the bootlaunch_master command and killing the process with the kill -9 #### command. Start the new server via the bootlaunch_master command. Hit REOPEN on the obsgtk to reopen the connection to the HUT server and hit reopen on Cosmic Debris as well. | The HUT servers control functions such as beacon and dichroic movements, heater and dehumidifier usage, and various AO functions. An observer may find that the obsgtk is no longer controlling the beacon LED's, beacon flat or dichroic alignments. This happens on occasion with E2 and other scopes. The HUT server may be the cause if it has quit or lost connection or the AOB may be at fault. To see if it is the server, open the HUT gui for the desired telescope from the CHARA menu. If the alignments can be changed from the gui, then the HUT server is ok. You can use the hut gui to continue observing. If the hut gui gives move error messages, cycle the power on the AOB and open a new hut server to restore the connection to the obsgtk. On the POWER gui, turn off the power to the AOB for the offending telescope and turn it back on. Stop the hut server by logging into the appropriate telescope computer and identifying the PID with the bootlaunch_master command and killing the process with the kill -9 #### command. Start the new server via the bootlaunch_master command. Hit REOPEN on the obsgtk to reopen the connection to the HUT server and hit reopen on Cosmic Debris as well. | ||
- | |||
- | The HUT servers control functions such as finder and acq exposure times and gains, heater and dehumidifier usage, and various AO functions. An observer may find that the camera settings do not display or are not adjustable. The HUT server may be the cause if it has quit. To see if it is the server, open the HUT gui for the desired telescope from the CHARA menu. If the camera settings are displayed and the settings can be changed from the gui, then the HUT server is ok. Restart the telescope server to reopen the connection to the HUT server and hit reopen on Cosmic Debris as well. If the gui is not functioning, | ||
If the server won't restart, a reboot of the power supply in the telescope bunker might be necessary. The power supply that controls the acquisition and finder cameras as well as their controllers is located on top of the computer rack in each bunker. The power supply has green readouts of volts and current. After turning the power off for 10 seconds and back on, try restarting the server from the computer in the bunker to see if it starts cleanly. If so, then restart the telescope server, reopen the connection to the telescope gui, and hit REOPEN on Cosmic Debris. Part of the HUT server also controls the AO table. If the AOB part of the HUT server doesn' | If the server won't restart, a reboot of the power supply in the telescope bunker might be necessary. The power supply that controls the acquisition and finder cameras as well as their controllers is located on top of the computer rack in each bunker. The power supply has green readouts of volts and current. After turning the power off for 10 seconds and back on, try restarting the server from the computer in the bunker to see if it starts cleanly. If so, then restart the telescope server, reopen the connection to the telescope gui, and hit REOPEN on Cosmic Debris. Part of the HUT server also controls the AO table. If the AOB part of the HUT server doesn' | ||
Line 267: | Line 261: | ||
[[: | [[: | ||
+ | |||
+ | ===== LabAO ===== | ||
+ | |||
+ | ==== LabAO Shutters Not Working ==== | ||
+ | |||
+ | If the labao shutters are not working, the power button on each of the two red control boxes on the top of the table might need to be pressed. After that the other button needs to be pressed and held for 5 seconds so that the units can be controlled remotely. (Note: look for instructions on table.) | ||
+ | |||
+ | ==== LabAO DM Electronics ==== | ||
+ | |||
+ | If a labao server complains that it failed to find the EDAC40 device, try unplugging the power and ethernet cables for DM and then replugging in again. | ||
===== OPLE and Metrology ===== | ===== OPLE and Metrology ===== | ||
Line 285: | Line 289: | ||
* Good metrology signals are important to the proper positioning of the carts. Monitor the signal strength by running [RUN MULTIPLE] on the Metrology Monitor. Place the windows above the TV windows for each scope you are using. They should show white sine waves that are around the height of the window. Erratic, fluctuating waves indicate self-interference or a weak signal. This may cause the carts to lose their place as the signal strength falls too low. Red waves indicate that some displayed signal has gone too low and the carts will all need to be homed. A careful adjustment of the MET2 mirror can sometimes bring the signal back. Do not adjust the MET1 mirror. | * Good metrology signals are important to the proper positioning of the carts. Monitor the signal strength by running [RUN MULTIPLE] on the Metrology Monitor. Place the windows above the TV windows for each scope you are using. They should show white sine waves that are around the height of the window. Erratic, fluctuating waves indicate self-interference or a weak signal. This may cause the carts to lose their place as the signal strength falls too low. Red waves indicate that some displayed signal has gone too low and the carts will all need to be homed. A careful adjustment of the MET2 mirror can sometimes bring the signal back. Do not adjust the MET1 mirror. | ||
- | * To home the carts, turn off the [OL] and [MAN] buttons on each cart and it will automatically return to the front switch. If the cart has no issue, it will arrive at the target position of 0m and the home switch at the same time. The X in the OPLE server under the HM will indicate it has stopped on the home switch. If a cart does not reach the home switch when it returns to position 0m, it was lost and likely the cause of the difficulty in finding fringes. | + | * To home the carts, turn off the [OL] and [MAN] buttons on each cart and it will automatically return to the front switch. If the cart has no issue, it will arrive at the target position of 0m and the home switch at the same time. The X in the OPLE server under the HM will indicate it has stopped on the home switch. If a cart does not reach the home switch when it returns to position 0m, it was lost and likely the cause of the difficulty in finding fringes. The X does not guarantee the cart has retained its home position. This home position |
===== Beam Samplers ===== | ===== Beam Samplers ===== | ||
Line 348: | Line 352: | ||
* If either Phase 1 or Phase 2 states say IDLE for a given VPN, then you need to re-establish the connection. To do this, click on the " | * If either Phase 1 or Phase 2 states say IDLE for a given VPN, then you need to re-establish the connection. To do this, click on the " | ||
* Make sure to close Firefox by using CTRL-Q to quit the program and not just close the window. | * Make sure to close Firefox by using CTRL-Q to quit the program and not just close the window. | ||
+ | |||
+ | ===== Network Problems ===== | ||
+ | |||
+ | ==== User accounts not recognized ==== | ||
+ | |||
+ | The gps computer does the user authentication for the control side while ctrscrut provides the home directories. Yellowpages is gone, the new paradigm is LDAP (local directory access protocol). If you can't log on to computers on the control network, then make sure both ctscrut (michelson) and gps computers are running. If the login screen screen on ctrscrut says " | ||
+ | |||
+ | If gps computer does not start, check the power distribution unit (PDU) at the top of that rack. If any units show both " | ||
===== General Problems ===== | ===== General Problems ===== | ||
Line 362: | Line 374: | ||
* If the top of a server goes blank, try typing " | * If the top of a server goes blank, try typing " | ||
- | * If the server screen fills with jibberish, try hitting CTRL-l in the server to clear it. | + | * If the server screen fills with jibberish, try hitting CTRL-L in the server to clear it. |
+ | * If a server freezes, sometimes hitting CTRL-C and then N, for NO, will unfreeze | ||
==== Server is frozen ==== | ==== Server is frozen ==== | ||
Line 372: | Line 385: | ||
* Click [REOPEN] on Cosmic Debris and relevant GUIs to re-initialize communication with the server after it is restarted. | * Click [REOPEN] on Cosmic Debris and relevant GUIs to re-initialize communication with the server after it is restarted. | ||
* A folder is on the desktop that has the restart commands for CTRSCRUT servers and the Shutters server restart command running on OPLE. Use it to restart servers that will not reopen from the menu. | * A folder is on the desktop that has the restart commands for CTRSCRUT servers and the Shutters server restart command running on OPLE. Use it to restart servers that will not reopen from the menu. | ||
+ | * if a telescope server does not close after using CTRL-C when restarting them after UT midnight and the Send ON button is green on the obsgtk, turn the Send ON to OFF and the server will close. Turn the Send ON button back on to resume the setup for the night. | ||
==== PAVO Server - Error communicating with IFW ==== | ==== PAVO Server - Error communicating with IFW ==== | ||
Line 460: | Line 474: | ||
\\ | \\ | ||
- | Last updated | + | Last updated |