Apache Point Observatory
New Mexico, USA

---

 

TRIPLESPEC

• 2024-03-18: The instrument is performing nominally.
• 2024-03-11: The instrument is performing nominally.
• 2024-03-04: The instrument is performing nominally.
• 2024-02-26: The instrument is performing nominally.
• 2024-02-19: The instrument is performing nominally.
• 2024-02-12: The instrument is performing nominally. Cart rotation mechanism for rotating on the cart is getting harder and harder to operate.
• 2024-02-05: The instrument is performing nominally. Cart rotation mechanism for rotating on the cart is getting harder and harder to operate.
• 2024-01-29: The instrument is performing nominally. Cart rotation mechanism for rotating on the cart is getting harder and harder to operate.
• 2024-01-22: The instrument is performing nominally.
• 2024-01-15: The instrument is performing nominally.
• 2024-01-08: The instrument is performing nominally.
• 2024-01-02: The instrument is performing nominally.
• 2023-12-18: The instrument is performing nominally.
• 2023-12-11: The instrument is performing nominally.
• 2023-12-04: The instrument is performing nominally.
• 2023-11-27: The instrument is performing nominally.
• 2023-11-20: The instrument is performing nominally.
• 2023-11-13: The instrument is performing nominally.
• 2023-11-06: The instrument is performing nominally.
• 2023-10-30: The instrument is performing nominally.
• 2023-10-23: The instrument is performing nominally.
• 2023-10-16: The instrument is performing nominally. Rebuilt the cart rotation mechanism, tightened the forward ring and fixed the left kinematic V which was not secure to the instrument front.
• 2023-10-09: The instrument is performing nominally.
• 2023-10-02: The instrument is performing nominally.
• 2023-09-25: The instrument is performing nominally.
• 2023-09-18: The instrument is performing nominally.
• 2023-09-11: The instrument is performing nominally.
• 2023-09-04: The instrument is performing nominally.
• 2023-08-25: The instrument is performing nominally.
• 2023-07-10: The instrument is performing nominally.
• 2023-07-03: The instrument is performing nominally.
• 2023-06-26: The instrument is performing nominally.
• 2023-06-19: The instrument is performing nominally.
• 2023-06-12: The instrument is performing nominally.
• 2023-06-05: The instrument is performing nominally.
• 2023-05-30: The instrument is performing nominally.
• 2023-05-22: The instrument is performing nominally.
• 2023-05-15: The instrument is performing nominally.
• 2023-05-08: The instrument is performing nominally.
• 2023-04-24: The instrument is performing nominally.
• 2023-04-17: The instrument is performing nominally.
• 2023-04-10: The instrument is performing nominally. The ICC computer froze over the weekend and needed to be retarted.
• 2023-04-03: The instrument is performing nominally.
• 2023-03-27: The instrument is performing nominally.
• 2023-03-20: The instrument is performing nominally.
• 2023-03-13: The instrument is performing nominally.
• 2023-03-06: The instrument is performing nominally.
• 2023-02-27: The instrument is performing nominally.
• 2023-02-20: The instrument is performing nominally.
• 2023-02-13: The instrument is performing nominally.
• 2023-02-06: The instrument is performing nominally.
• 2023-01-30: The instrument is performing nominally.
• 2023-01-23: The instrument is performing nominally.
• 2023-01-17: The instrument is performing nominally.
• 2022-12-12: The instrument is performing nominally.
• 2022-12-05: The instrument is performing nominally.
• 2022-11-28: The instrument is performing nominally.
• 2022-11-21: The instrument is performing nominally.
• 2022-11-14: The instrument is performing nominally.
• 2022-11-07: The instrument is performing nominally.
• 2022-10-31: The instrument is performing nominally.
• 2022-10-24: The instrument is performing nominally.
• 2022-10-17: The instrument is performing nominally.
• 2022-10-10: The instrument is performing nominally.
• 2022-10-03: The instrument is performing nominally.
• 2022-09-26: The instrument is performing nominally.
• 2022-09-19: The instrument is performing nominally.
• 2022-09-12: The instrument is performing nominally.
• 2022-09-06: The instrument is performing nominally.
• 2022-08-29: The instrument is performing nominally.
• 2022-08-22: The instrument is performing nominally.
• 2022-08-15: The instrument is performing nominally.
• 2022-08-01: The instrument is performing nominally.
• 2022-07-25: The instrument is performing nominally.
• 2022-07-18: The instrument is performing nominally.
• 2022-07-11: The instrument is performing nominally.
• 2022-07-05: The instrument is performing nominally.
• 2022-06-27: The instrument is performing nominally.
• 2022-06-20: The instrument is performing nominally.
• 2022-06-13: The instrument is performing nominally.
• 2022-06-06: The instrument is performing nominally.
• 2022-06-01: The instrument is performing nominally.
• 2022-05-23: tcamera-icc (Triplespec slitviewer) performed well the past 2 weeks. No issues reported.
• 2022-05-16: tcamera-icc (Triplespec slitviewer) performed well the past week. No issues reported.
• 2022-05-09: tcamera-icc (Triplespec slitviewer) performed well over the weekend. No issues reported.
• 2022-05-06: tcamera-icc has been up for a day after we grounded the front cover. Then Matt Nelson restarted the server software and on the first try it connected to the Leach. Nothing different was done. PR updated. Not convinced this will not repeat itself given we do not think we fixed anything, however for now the instrument is up.
• 2022-05-02: The spectragraph array electronics is now producing 0 byte images again. Without the tcamera-icc it is not able to access the slit information. However, the raw data in the forTron directory is good. It is just missing telescope information for the headers. The instrument could therefore be used for bright point source targets using the NA2 guider temporarily.
• 2022-04-28: The spectragraph array electronics is now producing 0 byte images again. Without the tcamera-icc it is not able to access the slit information. However, the raw data in the forTron directory is good. It is just missing telescope information for the headers. The instrument could therefore be used for bright point source targets using the NA2 guider temporarily.
• 2022-04-25: The spectragraph array electronics is now producing 0 byte images again. Without the tcamera-icc it is not able to access the slit information. There has been a failure of the slitviewer (tcamera) computer. We are working on repair path forward but there are currently no spares on hand. Spare parts on order.
• 2022-04-18: The spectragraph array and electronics is operating nominally. There has been a failure of the slitviewer (tcamera) computer. We are working on repair path forward but there are currently no spares on hand. Russet is working on a path forward to using the instrument without the slitviewer. Otherwise the instrument slitviewing capablity may be down for sometime. There still is some reported low level pattern noise seen in long darks or images.
• 2022-04-14: The spectragraph array and electronics is operating nominally. There has been a failure of the slitviewer (tcamera) computer. We are working on repair path forward but there are currently no spares on hand. Russet is working on a path forward to using the instrument without the slitviewer. Otherwise the instrument slitviewing capablity may be down for sometime. There still is some reported low level pattern noise seen in long darks or images.
• 2022-04-04: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2022-03-28: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2022-03-21: There was a problem with the tcamera (slitviewer array) icc communicating with the leach controller last week. It was able to be resolved but it is unclear what caused the issue. The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2022-03-14: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2022-03-07: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2022-02-28: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2022-02-21: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2022-02-14: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2022-02-07: The instrument is operating nominally however the past week both leach controllers required resets during afternooncheckouts. Something to be aware of. There still is some reported low level pattern noise seen in long darks or images.
• 2022-01-31: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2022-01-26: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-12-20: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-12-13: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-12-06: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-11-29: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-11-22: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-11-15: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-11-08: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-11-01: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-10-25: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-10-18: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-10-11: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-10-04: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-09-27: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-09-20: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-09-18: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-09-13: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-09-06: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-08-30: The instrument is operating nominally. There still is some reported low level pattern noise seen in long darks or images.
• 2021-08-17: The instrument is operating nominally. A clock driver board was replaced (after a new one was purchased). This corrected the corrupted images. There still is some low level pattern noise seen in long darks or images.
• 2021-08-09: The observing specialists reported corrupted images on Thursday night. We tried to troubleshoot and images became all zeros. Today we attempted multiple reboots and restarts of the software as well as powercycles of the main array controller. We attempted a swap of the PCI card but the problem remained. The instument is unusable for tonight. Will try further troubleshooting tomorrow.
• 2021-08-02: The instrument is operating nominally.
• 2021-07-27: The instrument is operating nominally.
• 2021-07-12: The instrument is operating nominally.
• 2021-07-06: The instrument is operating nominally.
• 2021-06-28: The instrument is operating nominally.
• 2021-05-24: The instrument is operating nominally.
• 2021-05-17: The instrument is operating nominally.
• 2021-05-11: The instrument is operating nominally.
• 2021-05-04: The instrument is operating nominally.
• 2021-04-19: The instrument is operating nominally.
• 2021-04-12: The instrument is operating nominally.
• 2021-04-05: The instrument is operating nominally.
• 2021-03-29: The instrument is operating nominally.
• 2021-02-22: The instrument is operating nominally.
• 2021-02-15: The instrument is operating nominally.
• 2021-02-01: No change in the instrument status. There still appears to be a low level pattern noise in some images. From David Decolibus: "the low-level pattern noise can be effectively avoided just by keeping the instrument upright (between -90 and +90 mount). In the twelve nights of observing I've had (beginning in September), I have not seen the pattern noise in *any* data in which I have kept the instrument upright. A few exposures below the horizontal (like for a standard star) is fine, but keeping the instrument inverted for an extended period seems to introduce the pattern noise. Last night, I wasn't paying attention and the slit angle I chose was about 30 degrees below the horizontal (-120 mount) for about 30 minutes. I started to see the pattern noise on the last few frames of this data. As soon as I rotated the slit 180 degrees, the noise went away, which only confirms my suspicions that it is strongly correlated with instrument rotator position."
• 2021-01-25: No change in the instrument status. There still appears to be a low level pattern noise in some images. From David Decolibus: "the low-level pattern noise can be effectively avoided just by keeping the instrument upright (between -90 and +90 mount). In the twelve nights of observing I've had (beginning in September), I have not seen the pattern noise in *any* data in which I have kept the instrument upright. A few exposures below the horizontal (like for a standard star) is fine, but keeping the instrument inverted for an extended period seems to introduce the pattern noise. Last night, I wasn't paying attention and the slit angle I chose was about 30 degrees below the horizontal (-120 mount) for about 30 minutes. I started to see the pattern noise on the last few frames of this data. As soon as I rotated the slit 180 degrees, the noise went away, which only confirms my suspicions that it is strongly correlated with instrument rotator position."
• 2021-01-18: No change in the instrument status. There still appears to be a low level pattern noise in some images. From David Decolibus: "the low-level pattern noise can be effectively avoided just by keeping the instrument upright (between -90 and +90 mount). In the twelve nights of observing I've had (beginning in September), I have not seen the pattern noise in *any* data in which I have kept the instrument upright. A few exposures below the horizontal (like for a standard star) is fine, but keeping the instrument inverted for an extended period seems to introduce the pattern noise. Last night, I wasn't paying attention and the slit angle I chose was about 30 degrees below the horizontal (-120 mount) for about 30 minutes. I started to see the pattern noise on the last few frames of this data. As soon as I rotated the slit 180 degrees, the noise went away, which only confirms my suspicions that it is strongly correlated with instrument rotator position."
• 2021-01-11: No change in the instrument status. There still appears to be a low level pattern noise in some images. From David Decolibus: "the low-level pattern noise can be effectively avoided just by keeping the instrument upright (between -90 and +90 mount). In the twelve nights of observing I've had (beginning in September), I have not seen the pattern noise in *any* data in which I have kept the instrument upright. A few exposures below the horizontal (like for a standard star) is fine, but keeping the instrument inverted for an extended period seems to introduce the pattern noise. Last night, I wasn't paying attention and the slit angle I chose was about 30 degrees below the horizontal (-120 mount) for about 30 minutes. I started to see the pattern noise on the last few frames of this data. As soon as I rotated the slit 180 degrees, the noise went away, which only confirms my suspicions that it is strongly correlated with instrument rotator position."
• 2021-01-05: No change in the instrument status. There still appears to be a low level pattern noise in some images. From David Decolibus: "the low-level pattern noise can be effectively avoided just by keeping the instrument upright (between -90 and +90 mount). In the twelve nights of observing I've had (beginning in September), I have not seen the pattern noise in *any* data in which I have kept the instrument upright. A few exposures below the horizontal (like for a standard star) is fine, but keeping the instrument inverted for an extended period seems to introduce the pattern noise. Last night, I wasn't paying attention and the slit angle I chose was about 30 degrees below the horizontal (-120 mount) for about 30 minutes. I started to see the pattern noise on the last few frames of this data. As soon as I rotated the slit 180 degrees, the noise went away, which only confirms my suspicions that it is strongly correlated with instrument rotator position."
• 2020-12-07: No change in the instrument status
• 2020-11-30: No change in the instrument status
• 2020-10-26: No change in the instrument status
• 2020-10-19: No change in the instrument status
• 2020-10-12: No change in the instrument status
• 2020-10-04: No change in the instrument status
• 2020-09-28: No change in the instrument status
• 2020-09-04: No change in the instrument status
• 2020-08-28: No change in the instrument status
• 2020-08-17: No change in the instrument status
• 2020-08-11: No change in the instrument status
• 2020-08-04: No change in the instrument status
• 2020-06-22: No change in the instrument status
• 2020-06-20: No change in the instrument status
• 2020-06-01: No change in the instrument status
• 2020-04-06: No change in the instrument status
• 2020-03-30: No change in the instrument status
• 2020-03-23: No change in the instrument status
• 2020-03-09: The instrument is operating nominally. The low level pattern noise is still appearing intermittently but is at a much lower level. It appears to have a correlation with instrument rotation. It is recommended that users try to keep the instrument orientation between +/-80 degrees rotator angle. Please report instances where the noise is affecting your data reductions. The observing specialists can give more details.
• 2020-03-02: The instrument is operating nominally. The low level pattern noise is still appearing intermittently but is at a much lower level. It appears to have a correlation with instrument rotation. It is recommended that users try to keep the instrument orientation between +/-80 degrees rotator angle. Please report instances where the noise is affecting your data reductions. The observing specialists can give more details.
• 2020-02-24: The instrument is operating nominally. The low level pattern noise is still appearing intermittently but is at a much lower level. It appears to have a correlation with instrument rotation. It is recommended that users try to keep the instrument orientation between +/-80 degrees rotator angle. The observing specialists can give more details.
• 2020-02-17: The instrument is operating nominally. We swapped electronic readout powersupplies last week. The low level pattern noise is still appearing intermittently but is at a much lower level.
• 2020-02-17: The instrument is operating nominally. The low level pattern noise is still appearing intermittently.
• 2020-02-10: The instrument is operating nominally. The low level pattern noise is still appearing intermittently.
• 2020-02-03: The instrument is operating nominally. The low level pattern noise is still appearing intermittently.
• 2020-01-27: The instrument is operating nominally. The low level pattern noise is still appearing intermittently.
• 2020-01-13: The instrument is operating nominally.
• 2020-01-06: The instrument is operating nominally.
• 2020-01-03: The instrument is operating nominally.
• 2020-01-03: Instrument is operating nominally
• 2019-12-16: Instrument is operating nominally
• 2019-12-09: Instrument is operating nominally
• 2019-11-04: Instrument is operating nominally
• 2019-10-28: Instrument is operating nominally, however a pickup noise has been identified in the data by a recent observer. It is intermittent and varies in stength. It appears to have a 120Hz component to it.
• 2019-10-21: Instrument is operating nominally
• 2019-10-14: Instrument is operating nominally
• 2019-10-07: Instrument is operating nominally
• 2019-09-23: Instrument is operating nominally
• 2019-09-09: Instrument is operating nominally
• 2019-08-26: Instrument is operating nominally
• 2019-08-12: Instrument is operating nominally
• 2019-08-05: Instrument is operating nominally
• 2019-07-08: Instrument is operating nominally
• 2019-05-16: Instrument is operating nominally
• 2019-01-02: Instrument is operating nominally
• 2019-01-02: Instrument is operating nominally
• 2018-10-15: Instrument is operating nominally
• 2018-12-03: Instrument is operating nominally
• 2018-04-02: Instrument is operating nominally
• 2018-03-19: Instrument is operating nominally
• 2017-10-30: Instrument is operating nominally
• 2017-08-22: Instrument is operating nominally
• 2017-07-10: On sky engineering this weekend was brief but was enough to release the instrument for science use. Focus had changed a small amount but possibly for the better. It will be able to return to science for scheduled observations this Wednesday.
• 2017-07-08: The instrument was cooled down this past week and is taking good images. On sky engineering tonight to determine if array focus and tip-tilt need adjustments before returning to science.
• 2017-06-22: circuit boards installed in the aux box. Instrument telemetry verified however it is still not displayable in TUI. Instrument will remain on the vacuum pump while a baseline of temperature tlemetry is maintained.
• 2017-06-21: Some delay in checking out the affected circuit boards in the Aux Box. This is delaying cooling down the instrument because we desire working temperature metrology before adding LN2.
• 2017-06-19: Instrument is on the vacuum pump on the observing floor. There were issues recabling it that required ordering parts. Hopefully cooldown later this week and on sky testing on the 24th.
• 2017-06-12: Brought instrument up to the observing floor for recabling and have on the vacuum pump for a few days.
• 2017-05-30: On the vacuum pump in the lab.
• 2017-05-26: Instrument is on the vacuum pump in the lab.
• 2017-05-19: UVa team left for what they feel is a successful trip. The found an issue with one of the clock lines leaving the leach board. Corrected. Found and corrected the slit wheel rotation issue with the stepper motor control. Found and corrected several cracked and damaged o-rings. Upgraded the temperature metrology boards and power supplies. They did leave with a few parts on order and uncompleted items for the APO staff to finish. Waiting on those items before we can start pumping on the dewar.
• 2017-04-17: UVa team is scheduled to service the instrument May 15, 2017.
• 2017-30-20: The tcamera has reached ambient temperature (290K), the rest of the dewar and the spectragram camera (speccam) is still warming and are a few degrees about freezing (274.3K). Power was removed from the Aux Tank resister this past Friday. The instrument will most likely be lowered off the floor the middle of May so the UVa team can investigate for the cause of the problem.
• 2017-03-13: Instrument is still warming.
• 2017-03-03: The thermal cycle was not successful in correcting the problem. We have started warming the instrument today. THe UVa team are currently planning a trip to work on the instrument. No dates have been decided but it will likely not be until mid May or mid June of 2017. Until then the instrument is warm and offline.
• 2017-02-24: Slitviewer and spectragraph cameras are both below 90K. Powered up. The slitviewer camera looks good after the first exposures. The spectragraph camera initial images still showed the same corrupted 'L' pattern. Tried a few resets to see if it would come back. On the 4th try the detector went blank and started producing zero's for all pixels. Tried one additional time, no change, all zeros. Informed UVa team. Will wait for their input before proceeding further.
• 2017-02-23: Cooled main tank. Will try powering up detectors when they cool to below 90K
• 2017-02-02: Started warming Triplespec for a thermal cycle. The hopes is this might be a possible correction for the failed state of the instrument.
• 2016-12-12: Candace reported that the instrument was taking good images for a brief period Saturday night but is now producing bad images again.
• 2016-11-14: Matt Nelson and Ed Leon are working on some experiments with controller voltages over the past week. The instrument has been powered down and unavailable while components on the control boards are changed.
• 2016-11-01: No change. The instrument has been producing curropted images whenever checked.
• 2016-10-24: Last week we thought we were able to correlate corrupted images with Instrument rotation or temperature. We were able to get the instruemnt to take proper images for a short few days. However over the majority of the past week and weekend images have been bad even though we were able to drop cryostat temps a few degrees and the instrument has been stored upside down. Please ensure you provide a backup instrument / program if Triplespec is your instrument choice. It may not be until next summer before the problem can be investigated by the UVa team.
• 2016-10-03: The instrument has been producing corrupted images for the past week. Resets have been not been effective. We are still trying multiple resets each day.
• 2016-09-26: The instrument still is intermittently having issues with corrupted images. Not all resets are successful in clearing the problem. We now strongly suspect the problem is inside the cryostat. If you plan to use Triplespec, please have a backup program ready with another instrument.
• 2016-09-08: A week ago the instrument produced a number of corrupted images on Thrusday evening during checkout. The Obs-spec was unsuccessful in resetting the controller and changing the state. A reset of the controller the next day brought the instrument back to normal mode of operation. Last night the controller again showed an unusual behavior during checkout (two bright glows in the image) but was producing otherwise good images. When checked this afternoon the instrument was again in a bad state. Multiple resets did not correct the problem. Later in the afternoon without further intervention the instrument had 'self corrected' and was back in a working state giving good darks. At this point we are investigating some possible avenues but consider the instrument intermittent and unreliable. Have a backup instrument availa/ble for scheduled science.
• 2016-03-14: Instrument continues to operate in an uncorrupted state.
• 2016-01-25: Instrument has been in an uncorrupted state for many weeks.
• 2015-11-30: Instrument is back in its bad state.
• 2015-11-02: Report that the instruemnt behaved well last night. 1500 test darks were taken with no corrupted images. >
• 2015-10-12: The instrument is still rather intermittent. It had troubles over the weekend but middle of the night self corrected after ~100 images after a reset. It then behaved for the rest of the night.
• 2015-10-05: Triplespec is back to misbehaving. Last night the observing specilist too several thoughsand exposures and there were perhas a hundred sequential images that were good, all the rest were bad. Attempting to do a reset this morning.
• 2015-09-18: The instrument still appears to be behaving. No failures seen in afternoon checkout images since mid August.
• 2015-08-24: The instrument has worked well the past few weeks. Nothing has changed with the instrument configuration that would have corrected the problem of corrupted images. We fully expect them to return at any time. We are planning a thermal cycle the beginning of September in hopes that it helps.
• 2015-06-15: Still in shared risk mode. Except for a few dozen bad images that were monitored over the weekend it appeared to perform well contrary to the 2 weeks where no good images could be obtained.
• 2015-06-08: The science camera is still producing corrupted images after repeated resets and hopes it will self heal. Check with the observing specialist before expecting to use it. Also plan your time with a back up instrument or program.
• 2015-06-01: Shared Risk Usage, currently we can not get it to take good images. Will continue to monitor.
• 2015-05-11: At this point the instrument appears to be more stable than unstable. Consider its use as shared risk. Plan a back-up instrument should you choose to use it. The observing specialist can try to recover from the corrupted image problem by power cycling and resetting the controller however this is not a sure fix. Sometimes it may take the instrument sitting idle for minutes, hours or days before it produces good images again. The problem is not well correlated with mounting / dismounting from the telescope, time of day, instrument rotation, etc. Nearly every piece of electronics outside the dewar has been tested or swapped with spares.
• 2015-05-07: No progress on determining the cause of the failures. It does not correlate with time of day, instrument rotation, rolling across the floor or jostling. We are running out of ideas.
• 2015-05-01: The controller is back from Leach. The controller and cards have been blessed by Bob Leach as being fine. The instrument is still exhibiting intermittent corrupted images. Continuing to test.
• 2015-04-13: The detector controller is still off at Leach for evaluation
• 2015-03-23: Another series of failures over this past weekend did not result in self correction by the controller or instrument. Taking the controller off the instrument and bringing it down to the bench to get to the individual boards to clean contacts, reseat and clean jumpers and their pins as well did not correct the problem. It was decided to ship the entire controller back to Astronomical Research Cameras (Bob Leach) for evaluation. The instrument will be unavailable and down until its return.
• 2015-03-18: Further troubleshooting of corrupted images problem. Got the instrument to work with the spare PCI card. We discovered however the reset switch on this card is bad. To reset the card, a full power-cycle of the computer is needed. After getting the instrument to take images again we saw only corrupted images. Tried a number of additional things with Matt's oversight. Will continue tomorrow. Unless the instrument self corrects itself again during checkout and starts taking good images again, the instrument is not usablefor science and is unavailable.
• 2015-03-16: The instrument has not corrected itself. Images taken during the weekend and this morning still show bad images.
• 2015-03-13: The obs-spec reported an occurange of the image corruption issue again during afternoon checkout.
• 2015-02-20: Another occurance of the image corruption error. The intermittant problem corrected itself again (within 24 hours) before further action of troubleshooting could take place.
• 2015-02-02: Power restored to the instrument. Images look fine.
• 2014-11-15: This problem is intermittant. It corrected itself by this morning. We will continue to watch each day if the problem returns but intermittant problems such as these are hard to troubleshoot.
• 2014-11-14: Image correction issue returned. Will investigate on Monday.
• 2014-11-05: Problem seems to have corrected itself. Will continue to monitor.
• 2014-11-04: Strange image corruption problem. Investigating. Instrument can not be used until corrected. An example can be found here.
• TRIPLESPEC Users Guide Instrument information
• 2014-07-24: Instrument is cold. Ready for science.
• 2014-07-16: Instrument is now warm. Started pumping.
• 2014-07-06: started warming for vacuum servicing.
• 2012-11-08: A reminder to users. Since the slitwheel is locked in place at the 1.1 arcsec slit, true instrument darks are not possible at this time.
• 2013-03-15: The ICC was found powered down yesterday. Recovery was a major undertaking requiring power cycles of both tspec and tcam leach controllers and reloading the dsp code. Most likely a gap in the documentation which will get updated.
• 2012-11-01: New instrument cart is now in full use. It was causing interference with the lip on the floor below the NA2 rotator so we cut a section of that kickplate lip away.
• 2012-10-20: Temperature logging fixed but not reported to APO however in the meantime critical alerts were off. They are back on now.
• 2012-09-21: Found the instrument was not properly logging temperatures. Power cycled the aux box for Matt Nelson. It is back up
• 2012-08-10: A reminder to users. Since the slitwheel is locked in place, True instrument darks are not possible at this time.
• 2012-02-19: The slitwheel mechanism has been powered off and disabled at this time. We are working with the UVa Instrument team to develop a repair plan. It currently is locked in the 1.1 arcsecond slit position.
• 2012-02-10: Exposure times for flat fielding using the truss lamps have not changed with the new lamps for Triplespec.
  
• 2012-02-10: Triplespec slit wheel has been experiencing problems where it freezes position or is sluggish to respond to commands. We are afraid that it will move to a non slit position and render the instrument unusable. As a result the engineers at UVa and APO staff are recommending the slit not be moved from the default 1.1 arcsecond slit, at least until we have a better handle on what is going on.
• 2012-02-07: Tripespec was exhibiting a low level intermittent pattern noise that can be seen in FS=8. It is on the order of +/- 20 DN noise. It has also been seen at lower levels (factor of 4) in FS=1 going back to April, 2010. This noise has been identified as getting worse than a few DN peak to peak as recently as this past December. For more information look here. APO is actively working on resolving this problem. We believe we may have made improvements today but it may take afew days to a week of monitoring to tell. Contact us if you see this in data after this date.