This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
chara:trouble_shooting [2018/09/25 15:32] gail_stargazer |
chara:trouble_shooting [2021/07/08 11:48] gail_stargazer |
||
---|---|---|---|
Line 2: | Line 2: | ||
\\ | \\ | ||
- | This document describes how to solve common problems that sometimes arise at the array.\\ | + | This document describes how to solve common problems that sometimes arise at the array. |
===== Can't find fringes ===== | ===== Can't find fringes ===== | ||
Line 8: | Line 9: | ||
* Were the clocks synced? Make sure the [SYNC CLOCKS] button on Cosmic Debris has been pushed to start the night. If the OPLE server does not display the correct CHARA time and the errors don't read (0) or (1), the clocks were not synced. As the VME runs on its own clock and does not use the NTP server, it can drift over the course of the night and cause problems with finding fringes, even if it was synced at the start of the night. Syncing the clocks multiple time during the night may help to avoid this hidden clock problem. | * Were the clocks synced? Make sure the [SYNC CLOCKS] button on Cosmic Debris has been pushed to start the night. If the OPLE server does not display the correct CHARA time and the errors don't read (0) or (1), the clocks were not synced. As the VME runs on its own clock and does not use the NTP server, it can drift over the course of the night and cause problems with finding fringes, even if it was synced at the start of the night. Syncing the clocks multiple time during the night may help to avoid this hidden clock problem. | ||
- | * Did the Astrolib update on OPLE? If the job queue is stopped too soon after slewing on Cosmic Debris, the correct calculations for the carts will not be done by OPLE and you may be searching for fringes with the wrong star data. The proper star can be entered by typing hd …. into the OPLE server and hitting ENTER. | + | * Did the Astrolib update on OPLE? If the job queue is stopped too soon after slewing on Cosmic Debris, the correct calculations for the carts will not be done by OPLE and you may be searching for fringes with the wrong star data. Hit STAR ACQUIRED on CD to update ople. The proper star can also be entered by typing hd #### into the OPLE server and hitting ENTER. |
+ | * Are the PoP's correct? After a PoP change, the PoP's are sometimes not updated in CD or ople. | ||
+ | * Are the carts behaving or are there vibrations or jumps of 100 or more microns every 3-6 seconds? Restart the ople server if they persist. | ||
* Is the target a high proper motion star? Red dwarfs are close stars and can have high proper motions. Scan a wider range to see if it is outside of the usual calculated scan range. Binaries can also have very high offsets from the expected position due to mistakenly using astromod calculations from the companion star. | * Is the target a high proper motion star? Red dwarfs are close stars and can have high proper motions. Scan a wider range to see if it is outside of the usual calculated scan range. Binaries can also have very high offsets from the expected position due to mistakenly using astromod calculations from the companion star. | ||
+ | * Do you have enough flux from each telescope or on each baseline? | ||
* Did you get the same star in each telescope? Sometimes a busy star field and poor pointing of the telescopes can lead to the wrong star being acquired and locked by tiptilt. View the stars in the finder window to see if all the stars match. | * Did you get the same star in each telescope? Sometimes a busy star field and poor pointing of the telescopes can lead to the wrong star being acquired and locked by tiptilt. View the stars in the finder window to see if all the stars match. | ||
* Check the CHARA time on the GPS server. The " | * Check the CHARA time on the GPS server. The " | ||
Line 27: | Line 31: | ||
==== Restarting Servers using the bootlaunch paradigm ==== | ==== Restarting Servers using the bootlaunch paradigm ==== | ||
- | \\ A number of servers use an interim bootlaunch paradigm to restart. This is confined to servers that run on ubuntu machines, namely the telescope bunker computers and gps. The basic syntax is " | + | If a server is not running or Socket Manager reports that a server is dead, then look at the socket manager list to find out what computer the server runs on ([[: |
+ | |||
+ | To restart a server, log on to the machine that runs the server and type " | ||
* bootlaunch_beamsamp – Starts the beam sampler servers, BS1 and BS2. | * bootlaunch_beamsamp – Starts the beam sampler servers, BS1 and BS2. | ||
Line 44: | Line 50: | ||
==== Restarting Servers using the rc.local file ==== | ==== Restarting Servers using the rc.local file ==== | ||
- | \\ This procedure is applicable to servers that have not switched over to the bootlaunch paradigm. \\ \\ If a server is not running or Socket Manager reports that a server is dead, then look at the socket manager list to find out what computer the server runs on ([[: | + | \\ This procedure is applicable to servers that have not switched over to the bootlaunch paradigm. \\ \\ If a server is not running or Socket Manager reports that a server is dead, then look at the socket manager list to find out what computer the server runs on ([[: |
\\ | \\ | ||
Log on to the relevant computer by typing the computer name (ctrscrut, ople, s1, …). If the shortcut doesn' | Log on to the relevant computer by typing the computer name (ctrscrut, ople, s1, …). If the shortcut doesn' | ||
Line 51: | Line 57: | ||
\\ The Shutters server can become unresponsive or disconnected from the Socket Manager. This server must be restarted from the lab and not from the Control Room. Follow these instructions to restart it. Note that Shutters runs on ople, not ctrscrut. \\ \\ To start the shutter server on ople: \\ \\ Log into the ople computer and kill the process labeled shutters with the PID as described in **Restarting Servers** | \\ The Shutters server can become unresponsive or disconnected from the Socket Manager. This server must be restarted from the lab and not from the Control Room. Follow these instructions to restart it. Note that Shutters runs on ople, not ctrscrut. \\ \\ To start the shutter server on ople: \\ \\ Log into the ople computer and kill the process labeled shutters with the PID as described in **Restarting Servers** | ||
+ | |||
+ | ==== MIRC-X CredoneImAcq Server ==== | ||
+ | |||
+ | The MIRC-X CredoneImAcq server and GUI can be opened using the " | ||
+ | |||
+ | From a terminal window, log on to mircx (old observe password): \\ ssh -X spooler@mircx | ||
+ | |||
+ | Look up credone process: \\ ps aux | grep credone | ||
+ | |||
+ | Kill all credone processes (including the ones for the error messages) using "kill -9 PID" where PID is the process number. Note that credone can't be restarted if there are any error processes in the list. Check "ps aux | grep credone" | ||
+ | |||
+ | Then restart credone using the desktop icon or by issuing the following command on spooler@mircx: | ||
===== Telescopes and Dome Servers ===== | ===== Telescopes and Dome Servers ===== | ||
Line 58: | Line 76: | ||
==== The Telescope won't move or stopped moving ==== | ==== The Telescope won't move or stopped moving ==== | ||
- | \\ Have the powers been turned on to the drives? Are the scopes disabled? The usual state of the telescopes is disabled until enabled. This is due to the stall function of the scopes which eventually disables the scopes when they are stowed. Enable the scopes by hitting [ENABLE] on the dome gui or in the telescope gui control tab. If a scope disables itself during a slew, it may be just an overcautious stall function. Hit ENABLE and the scope should continue to slew. If it disables again without moving, there may be something wrong at the scope, ie. a hatch left open, a ladder not put away, a tool on the floor, or something else physically impeding the motion of the scope. You will need to go to the dome to see what it is. The computer in the dome will give you control of the scope to turn it away from the problem. \\ \\ Sometimes the dome guis get hung up and can cause erratic motion or no motion of the scopes. Check them for current times and continuous updates of numbers. If they are not updating, try to REOPEN them first. If that does not work, close the gui and open a new one. If a new one does not open, the dome server may be dead or Sockman lost track of it. See **Dome Server Restart** | + | \\ Have the powers been turned on to the drives? Are the scopes disabled? The usual state of the telescopes is disabled until enabled. This is due to the stall function of the scopes which eventually disables the scopes when they are stowed. Enable the scopes by hitting [ENABLE] on the dome gui or in the telescope gui control tab. If a scope disables itself during a slew, it may be just an overcautious stall function. Hit ENABLE and the scope should continue to slew. If it disables again without moving, there may be something wrong at the scope, ie. a hatch left open, a ladder not put away, a tool on the floor, or something else physically impeding the motion of the scope. You will need to go to the dome to see what it is. The computer in the dome will give you control of the scope to turn it away from the problem. \\ \\ Sometimes the dome guis get hung up and can cause erratic motion or no motion of the scopes. Check them for current times and continuous updates of numbers. If they are not updating, try to REOPEN them first. If that does not work, close the gui and open a new one. If a new one does not open, the dome server may be dead or Sockman lost track of it. See **Dome Server Restart** |
==== The Telescope won't track ==== | ==== The Telescope won't track ==== | ||
Line 70: | Line 88: | ||
==== Dome Server Restart ==== | ==== Dome Server Restart ==== | ||
- | \\ To manually start the dome server: \\ \\ 1. Make sure the power to the drives is OFF. \\ 2. Login to the relevant computer as root. For example, type " | + | \\ To manually start the dome server: \\ \\ 1. Make sure the power to the drives is OFF. \\ 2. Login to the relevant computer as root. For example, type " |
==== Telescope clock is not correct ==== | ==== Telescope clock is not correct ==== | ||
Line 93: | Line 111: | ||
* ENABLE the telescope using the telescope or dome GUI. Check to see if the command and telescope positions are correct. If they are, then turn on the power for the telescope drives. Re-enter the star information by entering the HD number in the telescope server and click [GO NEXT] to send the telescope to the star. Make sure that the telescope is behaving as expected. The telescope might have drifted a bit, so if you can't find the star, it might be necessary to use the Telrad on a bright star to re-initialize the pointing. | * ENABLE the telescope using the telescope or dome GUI. Check to see if the command and telescope positions are correct. If they are, then turn on the power for the telescope drives. Re-enter the star information by entering the HD number in the telescope server and click [GO NEXT] to send the telescope to the star. Make sure that the telescope is behaving as expected. The telescope might have drifted a bit, so if you can't find the star, it might be necessary to use the Telrad on a bright star to re-initialize the pointing. | ||
* Restart commands for the dome servers are also listed on the desktop in a text file. | * Restart commands for the dome servers are also listed on the desktop in a text file. | ||
- | |||
< | < | ||
- | \\ | + | |
- | ==== Telescope is tracking poorly, overshooting in slew, oscillating. ==== | + | \ |
+ | ==== Telescope is tracking poorly, overshooting in slew, oscillating. ==== | ||
</ | </ | ||
- | | + | < |
+ | This might mean that the gain for the tracking servo is wrong. Note that changing this gain can be dangerous, especially if you set it too high as that can cause the telescope to oscillated and damage the drives. Please only do this if you are very very sure that it is necessary. Symptoms of bad gain are: The scope over shoots the position while slewing. The star will be seen to move out of the window and may come back after a few seconds. This means the slewing gain is too low. The scope oscillates when tracking or after a slew. The star will be tracing an ellipse, figure eight or other looping shape. This means the tracking gain is too low. You can damp this out with the telescope or dome gui by disabling the scope, then re-enabling it. Adjust the gain upward and watch it on the next slew. In all cases if either gain is too high the scope will go into "Fog Horn" mode, which is bad. You always want to use the lowest gain that still allows the scope to work as best as possible. If the tiptilt tells you the scope is oscillating slowly, the gain may be too low. If it is oscillating quickly it may be too high. On 10-22-2016, the gain settings were: | Scope | AZ Slewing | ||
+ | |||
+ | </ | ||
| |Gain|Fn|Gain|Fn|Gain|Fn|Gain|Fn| | | | |Gain|Fn|Gain|Fn|Gain|Fn|Gain|Fn| | | ||
Line 129: | Line 151: | ||
==== Dome does not close ==== | ==== Dome does not close ==== | ||
- | \\ If a dome will not close after observing, it may need to be closed manually at the bunker. Turn the power to the dome off at the switch, disconnect the cable from the computer control box and connect it to the manual control switches above. Turn on the power to the dome after connecting the cable and use the two controls to close the dropout first, then the slit. The dropout must be fully closed and the slit closed over it to be shut properly. Inform the staff that the dome did not function properly so someone can look at it in the morning. | + | \\ If a dome will not close after observing, it may need to be closed manually at the bunker. Turn the power to the dome off at the switch, disconnect the cable from the computer control box and connect it to the manual control switches above. Turn on the power to the dome after connecting the cable and use the two controls to close the dropout first, then the slit. The dropout must be fully closed and the slit closed over it to be shut properly. Inform the staff that the dome did not function properly so someone can look at it in the morning. |
====== HUT servers ====== | ====== HUT servers ====== | ||
Line 135: | Line 157: | ||
==== I can't change the camera settings on the TV ==== | ==== I can't change the camera settings on the TV ==== | ||
- | \\ The HUT servers control functions such as finder and acq exposure times and gains, heater and dehumidifier usage, and various AO functions. An observer may find that the camera settings do not display or are not adjustable. The HUT server may be the cause if it has quit. To see if it is the server, open the HUT gui for the desired telescope from the CHARA menu. If the camera settings are displayed and the settings can be changed from the gui, then the HUT server is ok. Restart the telescope server to reopen the connection to the HUT server and hit reopen on Cosmic Debris as well. If the gui is not functioning, | + | \\ The HUT servers control functions such as finder and acq exposure times and gains, heater and dehumidifier usage, and various AO functions. An observer may find that the camera settings do not display or are not adjustable. The HUT server may be the cause if it has quit. To see if it is the server, open the HUT gui for the desired telescope from the CHARA menu. If the camera settings are displayed and the settings can be changed from the gui, then the HUT server is ok. Restart the telescope server to reopen the connection to the HUT server and hit reopen on Cosmic Debris as well. If the gui is not functioning, |
===== Tiptilt Server ===== | ===== Tiptilt Server ===== | ||
- | \\ The tiptilt server controls the CCD based tiptilt detection system. \\ \\ Before you start the tiptilt server, you must ensure that the power to the cooling system and the CCD iteslf is on. It is extremely important that the cooler be running before you turn on the CCD and is only turned off if you are sure the CCD is NOT running. You can start the server from the X windows menu or with the command xtiptilt. \\ \\ Note that there are background counts and read noise to deal with. Whenever you change the frame rate, please ensure that the bias frame is OK. The server will attempt to load an old bias frame that should work, but if things are not working, try making a new bias frame by ensuring that the detector is in the dark and typing " | + | \\ The tiptilt server controls the CCD based tiptilt detection system. \\ \\ Before you start the tiptilt server, you must ensure that the power to the cooling system and the CCD iteslf is on. It is extremely important that the cooler be running before you turn on the CCD and is only turned off if you are sure the CCD is NOT running. You can start the server from the X windows menu or with the command xtiptilt. \\ \\ Note that there are background counts and read noise to deal with. Whenever you change the frame rate, please ensure that the bias frame is OK. The server will attempt to load an old bias frame that should work, but if things are not working, try making a new bias frame by ensuring that the detector is in the dark and typing " |
==== Tiptilt server complains about the CCD ==== | ==== Tiptilt server complains about the CCD ==== | ||
- | \\ Is the CCD turned on? When the tiptilt server starts up it tries no more than five times to communicate with the CCD. If they all fail, it will give up. If this happens, try cycling the power to the CCD and try again. If this fails, connect to the tiptilt machine and type the command rtccdAPIDemo, | + | \\ Is the CCD turned on? When the tiptilt server starts up it tries no more than five times to communicate with the CCD. If they all fail, it will give up. If this happens, try cycling the power to the CCD and try again. If this fails, connect to the tiptilt machine and type the command rtccdAPIDemo, |
==== Tiptilt doesn' | ==== Tiptilt doesn' | ||
- | \\ Sometimes the telescope server will not show that TT is running. It will show 0Hz for a signal rate for TT. Running TIPTILT COMM will not get it started while other scopes do show it starting. Close and restart any telescope servers that won't connect after two tries of TIPTILT COMM. \\ Note: There is more info in the software manual on this topic, but I wasn't sure if it was still relevant. | + | \\ Sometimes the telescope server will not show that TT is running. It will show 0Hz for a signal rate for TT. Running TIPTILT COMM will not get it started while other scopes do show it starting. Close and restart any telescope servers that won't connect after two tries of TIPTILT COMM. \\ Note: There is more info in the software manual on this topic, but I wasn't sure if it was still relevant. |
==== Tiptilt server says the clock isn't running ==== | ==== Tiptilt server says the clock isn't running ==== | ||
Line 224: | Line 246: | ||
==== Restarting the Beam Sampler Server ==== | ==== Restarting the Beam Sampler Server ==== | ||
- | The most common cause for getting multiple copies of a server is using the socket manage RESTART button, which is flaky at best. Don't do this. If you need to restart a server you should do it manually. \\ \\ 1. Make sure you are logged into the right machine: \\ \\ ssh ctrscrut \\ \\ 2. If you are not sure see if the socket manager will tell you. If it doesn' | + | The most common cause for getting multiple copies of a server is using the socket manage RESTART button, which is flaky at best. Don't do this. If you need to restart a server you should do it manually. \\ \\ 1. Make sure you are logged into the right machine: \\ \\ ssh ctrscrut \\ \\ 2. If you are not sure see if the socket manager will tell you. If it doesn' |
==== " | ==== " | ||
Line 270: | Line 292: | ||
==== Adding or Finding a Star in the CHARA Database ==== | ==== Adding or Finding a Star in the CHARA Database ==== | ||
- | DBADD is a command | + | The command |
You can also look up different identifiers for the star by using the SIMBAD database: [[http:// | You can also look up different identifiers for the star by using the SIMBAD database: [[http:// | ||
\\ | \\ | ||
- | SIMBAD will show many identifiers for the star, and often will show one of the database identifiers that Cosmic Debris recognizes. | + | DBADD uses SIMBAD to look up the coordinates of the star and then searches to see if the star is already in the CHARA database. If the star is in the database, it will return the CHARA number. If the star is not in the CHARA database, it will ask if you want to add it and assign a CHARA number if you answer yes. |
- | If an object is not in SIMBAD (such as a recently discovered nova), then you must enter the object manually | + | If an object is not in SIMBAD (such as a recently discovered nova), then you must enter the object manually |
\\ __Finding a star in the CHARA database__ | \\ __Finding a star in the CHARA database__ | ||
Line 286: | Line 308: | ||
Note: If the target information stored in the CHARA database does not match SIMBAD, then dbadd will provide the coordinates and magnitudes listed in SIMBAD, and it will also list the CHARA numbers of objects in the database located within 10 arcsec. Please check these nearby entries before adding a new star to the database. | Note: If the target information stored in the CHARA database does not match SIMBAD, then dbadd will provide the coordinates and magnitudes listed in SIMBAD, and it will also list the CHARA numbers of objects in the database located within 10 arcsec. Please check these nearby entries before adding a new star to the database. | ||
- | \\ __Targets that are not in the CHARA database__ \\ \\ If an object is not in the CHARA database, then you can use dbadd to query SIMBAD and add the star to the CHARA database. This might need to be done for faint stars, novae, and AGN. If the object is found in SIMBAD, then dbadd will return the target' | + | \\ __Targets that are not in the CHARA database__ \\ \\ If an object is not in the CHARA database, then you can use dbadd to query SIMBAD and add the star to the CHARA database. This might need to be done for faint stars, novae, and AGN. If the object is found in SIMBAD, then dbadd will return the target' |
Here is an example of how to add an object to the CHARA database: | Here is an example of how to add an object to the CHARA database: | ||
Line 292: | Line 314: | ||
[ctrscrut: | [ctrscrut: | ||
- | ID | + | ID 2MASS_J20340850+4136592 \\ CHARA 320414 \\ 2000.0 \\ CAT8 2MASS_J20340850+4136592 \\ RA 20 34 08.5148 \\ DEC +41 36 59.408 \\ PMRA -2.32 \\ PMDEC -3.87 \\ PAR 0.58 \\ RVEL -27.50 \\ VMAG 10.03 \\ HMAG 6.23 \\ KMAG 5.99 \\ SP O5.5Ifc \\ END |
Is the above the object you are looking for? [y/n] y \\ Added "BD+41 3807" to the database. | Is the above the object you are looking for? [y/n] y \\ Added "BD+41 3807" to the database. | ||
- | The object "BD+41 3807" has been added to the database as CHARA number 320414. | + | In the example above, the object "BD+41 3807" has been added to the database as CHARA number 320414. |
- | \\ __Targets that are not in the CHARA database or in SIMBAD__ \\ \\ Some objects like recently discovered novae are not in SIMBAD. Entering dbadd <star designation> | + | \\ __Targets that are not in SIMBAD__ \\ \\ Some objects like recently discovered novae are not in SIMBAD. Entering dbadd <star designation> |
+ | |||
+ | When a recognized identifier is entered, the HD number with A or B will usually display. Confirm that the coordinates, | ||
==== Adding an anchor to the wiki page ==== | ==== Adding an anchor to the wiki page ==== | ||
- | To add an anchor to the wikipage, type the following where you want the anchor inserted (without the spaces between the symbols): \\ [ [ #AnchorName ] ] \\ \\ Highlight the text you want to be a link to the anchor, click the link symbol above and select anchor. Select a page(such as Trouble Shooting) where the anchor is located and give the same name for the anchor. | + | To add an anchor to the wikipage, type the following where you want the anchor inserted (without the spaces between the symbols): \\ [ [ #AnchorName ] ] \\ \\ Highlight the text you want to be a link to the anchor, click the link symbol above and select anchor. Select a page(such as Trouble Shooting) where the anchor is located and give the same name for the anchor. |
+ | |||
+ | ==== Displays for the MIRC-X wolverine2 monitors ==== | ||
+ | |||
+ | Follow instructions in the [[: | ||
- | \\ | + | \\ Last updated 2017-04-12 |