Feature: Telesat Integration of Telstar Fleet Operations
by Tom Protzman
System Test Engineer
Integral Systems Incorporated
Doug Mathias, M. Math Manager
Satellite Operations Systems
Space Systems, Telesat
In December 2006, the purchase of Telesat Canada by Loral Space and Communications and PSP Investments was announced. At this time, Loral Skynet, the satellite services subsidiary of Loral Space and Communications, was operating ten satellites. Four satellites were Loral-owned (Telstar), and six others were being maintained for two different customers. The ten satellites incorporated three different bus types. All of these satellites were operated by Skynet with the Epoch command and control software provided by Integral Systems, Inc (ISI).
The original business deal estimated a closing of early summer 2007. In the interim, only minimal high-level planning could be done concerning future spacecraft operations. Nine months were allocated to transfer operations for all ten satellites from Loral’s Hawley, Pennsylvania, facility to the Telesat control center in Ottawa, Canada.
As the actual purchase did not occur until the fall of 2007, the project team only had six months to accomplish this transfer.
Upon closing the deal, Telesat appointed Tony Grise as the project lead. Under his guidance, many people within the operations and network teams of both companies worked to get the entire task scoped, scheduled, and completed.
Successfully transferring control of a satellite requires many areas to be correctly addressed. These include training, procedures, network, software, hardware, and facilities. In this case, it was necessary to integrate six locations running the Integral Systems’(ISI) EPOCH software for satellite telemetry, tracking, and control with a total of eleven antenna sites. Toward the end of the project, it was necessary to remove the former primary site, Hawley, gracefully from operations. As EPOCH would continue as the real-time software in use, an additional requirement was the interfacing of EPOCH-to-legacy Telesat applications used for orbital analysis (Flight Dynamics System [FDS]) and data archival (Scalable Data Management [SDM]). These Telesat-developed applications support operations for their entire fleet.
The Balancing Act
As the careful operation of both the Skynet and Telesat fleets continued, great effort was required to not disrupt these activities at any time during the transition. The network team worked to ensure that core-required connectivity was not disrupted as the new WAN was implemented and tested. The software team used care when allowing configurations to migrate to their final, required form in small steps. Initially, Ottawa would serve as a monitor-only operation for the Skynet fleet. As the project moved forward, more Ottawa-centered control of hardware needed to be implemented and tested. Finally, full control of ranging and commanding was released to Ottawa.
One of the immediate milestones was to have ISI’s EPOCH software running at Telesat in Ottawa. This allowed for controller training to begin on both the system itself and legacy Skynet operational procedures. The decision was made that the primary Telstar console installed in the Ottawa control center be configured to passively monitor the Telstar fleet. This allowed personnel to begin monitoring real spacecraft data.
Capitalizing on the flexibility of the Epoch software, the system was configured to be essentially appended to the existing EPOCH system already used at five different locations. This provided telemetry reception and the ability to test commanding. To minimize impacts to existing Skynet fleet operations, the Ottawa system was configured to “see” all of the legacy Skynet system without the legacy system seeing Ottawa. Within two weeks of the closing, this system was up and running with initial command checks performed on each spacecraft.
During the training period, the backup EPOCH console in the Ottawa control center was configured to interface with Dynamic Satellite Simulators (DSS) located in the Hawley facility. This allowed Telesat personnel to fully exercise the system and operational procedures. Due to concerns regarding “flying” simulators from within the control center, a practice normally avoided, the command lockout features of the EPOCH software were used on the primary console to reduce the possibility of an errant command.
For the next four months, Telesat controllers and engineering staff exercised all legacy Skynet operational procedures against Hawley’s DSS computers. This was an around-the-clock effort and required significant participation by staff both in Ottawa and Hawley to ensure success. As the DSS’s were hosted by EPOCH servers located in Hawley, this process also provided for the discovery and reparation of network-related issues along some of the new WAN segments.
The transition’s final six weeks included a process of passing individual and then groups of spacecraft to Ottawa operations to maintain for single shifts, then a day, and so on. During these periods, Hawley shadowed operations to ensure nothing was missed. This effort began during eclipse season in order to help Ottawa gain experience with each spacecraft for this critical operation.
Telesat uses SDM and FDS software it developed with its entire fleet. Bridge software is developed as needed with each of their different Real-Time Systems (RTS) to provide processed telemetry, raw telemetry, range data, and events to SDM, which in turn feeds the FDS. EPOCH provides a number of very powerful API that customers can use to allow custom extensions to the core system. In this case, the CORBA-based API was used to develop the bridge software to SDM. Doug Mathias of Telesat, with guidance from ISI’s Brian Gray, developed this interface. Beginning with provided examples, decommutated data was flowing from EPOCH to SDM within three days. Raw telemetry frames and events followed within two more days.
Having the “live” system installed in Ottawa allowed for a thorough test of this new software. This also allowed Telesat engineers to begin to integrate the new fleet into their familiar trending tools that use SDM.
For satellite ranging, Telesat FDS/SDM software required a different output format than Skynet needed with the ISI Orbital Analysis System (OASYS) software. Telesat also used a different range tone set on the Cortex units, common baseband units used by both companies. Fortunately, the EPOCH ranging software can be configured to provide several different output formats, including one that is compatible with the Telesat FDS/SDM system. The integration team took these required changes as an opportunity to write new EPOCH Satellite Test and Operations Language (STOL) procedures to simplify maintenance and operation for the ranging process. These new procedures are scheduled using ISI’s Task Initiator (TI), resulting in automated range data collection. This new process was also designed to retain the production of OASYS-formatted output for use by Telesat operations customers. Clearly, the capabilities of the Epoch system were key to the success of the integration of the Epoch system with the legacy Telesat system components.
Many other significant activities were completed during the six-month transition. These activities included releasing and configuring antenna resources to accommodate the Hawley facility’s closure, as well as integrating scheduling and staff for the new fleet operations into Telesat’s overall fleet operations.
Control of the Skynet fleet was transferred from Hawley to Ottawa at the end of the first quarter of 2008. The original timeline was met. In the end, only nine spacecraft were transferred, as one was de-orbited before handover. Hawley personnel continued to monitor operations for another month, both on site in Ottawa, and remotely from Hawley. This was done in order to act both as a backup to Ottawa and as a continued training exercise for Ottawa staff.
Transferring control of nine spacecraft, in a time-span of six months, was quite aggressive from a scheduling perspective, requiring a tremendous effort by those at Skynet and Telesat, and involved considerable support by Integral Systems (training, setup and systems support). With the successful completion, all nine spacecraft are monitored from a single console running Epoch. A second console acts as a backup and high activity area.
About the authors
Tom Protzman is a System Test Engineer at Integral Systems Incorporated and has been in the satellite industry for more than 18 years. He has worked in satellite manufacturing, operations, and ground segment software. He can be reached at firstname.lastname@example.org.
Doug Mathias, M. Math Manager, Satellite Operations Systems, Space Systems, Telesat, has been an employee of Telesat for 23 years. During his career at Telesat he was worked in Flight Dynamics, Data Processing Systems, and Real-Time Systems.