October 27, 2020
Things Judit had to do in the lab to recover after power outage:
Shutters - cycled the power using the manual switch on the SHUTTER box in the rack
Homed Newport stage VIS beams (same as Esp BC2 gui VISB axis)
Bootlaunch_master on ctrscrut to get the BS servers running
Homed all 6 Newport BS stages (using guis Esp BS1 and BS2)
Labao DM electronics were all fine, all those UPSes kept the power
Started up labao computer
Pushed power button on the USB hub for labao cameras (on top of the shelf above W1) before starting labao servers so that the servers could find the cameras
Labao shutters were OK, on same UPSes as labao DM
Cycled ApertureWheels power and inited all 6 Aperture Wheels
Started up tiptilt computer → check was forced due to not clean shutdown
Soft rebooted tiptilt computer with entering BIOS and RE-SAVED it for proper clock start
Homed all axes of tiptilt zabers (ZABER 2) using ENG gui, because they lost their position. [Best to do this when the beams are aligned with labao and corner cubes are in, so that the movements can be seen in tiptilt gui. Laser ND=6. One click on Home button on any axis homes all axes. After homing, for each axes GO TO SETPOS takes it to the latest position in the cfg file. It is likely that the zaber positions need to be adjusted and fresh set positions saved.]
All pops were out and PERI positions lost → All 6 PERI axes had to be homed and AFTER THAT set pop5 first with POPPERIGTK and adjusted M10 as needed.
Additional Notes After Power Outage on August, 17, 2021, June 2022, Feb 2024.
Reinitialize the NIRO Filter Wheel. Open the niro_filter_gtk and click REINIT. Check to see if the indicator wheel on the NIRO camera is pointing to the correct filter after it is homed. Do not turn the wheel by hand. If it points to the wrong filter, home the filter wheel again, and compare counts for each filter (or look for metrology light in J-band) to confirm the correct filters are in position. Then loosen the screw on the indicator wheel and align so that it points to the correct filter label that is currently in.
Leave plenty of time to do a full check of the alignment on all beams.
Check alignment of MIRC-X and MYSTIC using STS. Make sure the fiber stages and internal delays are consistent with the positions recorded in the manual/headers. All of the MIRC-X and MYSTIC components are on UPS, so they should retain their positions, but it's always good to check in case one of the UPS fails.
Home the engineering beam using BC1. A list of engineering beam positions for each beam is on a post-it note near the stage. Note that CLIMB_B1 and CLIMB_B2 are usually not homed and just left in their last good positions. If they are homed, the fringe positions for the CLIMB1 stages: CL1= 7.0565 mm and CL2 = 5.3224 mm. For the CLIMB2 stages: CL4= 2.9630 mm and CL5= 7.8500 mm. (See email from Judit on Aug 8, 2021.)
Home and re-align lab tiptilt zaber stages. See the
link here for tiptilt zaber_2 positions and instructions for aligning the beams.
Check the PAVO ENG
GUI to see if the PAVO zaber stages lost position. If so, prior to the next run, follow instructions in the
pavo manual for homing the zaber stages. See special notes for homing the lenslet stage - after aligning, check to make sure you can see light in the camera in both the lenslet and image plane (the homing of the lenslet is not precise so the position might need to be fine tuned).
Restart network connection on spica-console:
The Intel I219 network interface is not natively supported by Linux (something about shipping with an invalid checksum which Windows drivers ignore, but Linux takes to heart). Nils downloaded the driver source code which ignores the checksum problem and placed it at /root/e1000e.x.x.x.x (where x.x.x.x is the version).
You should be able to access it with the following command sequence (the observe account should have sudo access):
To find out the version:
sudo ls /root
enter that directory:
sudo cd /root/e1000e.<version>/src
remove the old module (if it is still loaded):
sudo rmmod e1000e
build the new driver:
sudo make install
load the new module:
sudo modprobe e1000e
ensure the module is persistent across a reboot:
sudo dpkg-reconfigure linux-image-$(uname -r)
The STST chiller will power back on after an outage, but the cooler does not start automatically. Press the settings button to start the cooling again.
Karolina's Notes from Power outage 07/23/2024 - 1 min, scheduled because some electricians work.
After today's power outage, I followed our recover after power outage list and checked in the lab.
Shutters - were working no power cycling needed.
Homed Newport stage VIS beams (same as Esp BC2 gui VISB axis) - done
Bootlaunch_master on ctrscrut to get the BS servers running - done
Homed all 6 Newport BS stages (using guis Esp BS1 and BS2) - done
Labao DM electronics were all fine, all those UPSes kept the power - were fine
Started up labao computer - was running, labao servers started without any problem and detected all cameras
Pushed power button on the USB hub for labao cameras (on top of the shelf above W1) before starting labao servers so that the servers could find the cameras - no need
Labao shutters were OK, on same UPSes as labao DM - were ok.
Cycled ApertureWheels power and inited all 6 Aperture Wheels - done
Started up tiptilt computer→check was forced due to not clean shutdown - tiptilt computer is off since we did not use it recently
All pops were out and PERI positions lost→All 6 PERI axes had to be homed and AFTER THAT set pop5 first with POPPERIGTK and adjusted M10 as needed. The Periscopes were ok, didn't lose the positions, I checked them by changing positions.
I did not check any instruments.
Notes added Sept 20, 2024:
Check the Exhibit Hall Displays. If the slides are not displaying correctly, log on and reboot each computer:
ssh -X pi@ehpi01
Enter wifi password
sudo reboot
The ehpi's go from 01 to 10.
You can reboot all 10 of the displays in the exhibit hall using the following command sequence:
Facitilities Checks: