We Are Borg: Crowdsourced Engineering and the Collective Mind of the Internet

We are Now Living in a Science Fiction World

In the science fiction universe of Star Trek, set several hundred years in the future, when we are a spacefaring civilization, humanity encounters a species called the Borg.  The Borg are a conglomeration of species who are assimilated into a collective mind numbering in the hundreds of billions.  All of the borg are connected to each other through a communications link that allows each of them to share each others thoughts, though in a manner that erases individuality.

This week, with the call that our ISEE-3 reboot team put out to the internet for help in debugging our propulsion system problem, I have come to realize that a significant portion of humanity has reached a Borg like state, one where the internet has become a collective mind for communications and knowledge sharing.  We still have our individuality, we can still decouple at will from the collective mind, but in a way that few philosophers or technologists have envisioned, we are connected in a way never before thought possible.  The implications are staggering, and here is how our little ISEE-3 project is an example of the operation of the collective mind.

ISEE-3 Trajectory Correction Maneuver Attempt

On July 8th and 9th the ISEE-3 reboot team attempted to fire the radial thrusters on the spacecraft.  Figure 1 shows the configuration of the thrusters:

Figure 2: The ISEE-3 Spacecraft Details

Figure 1: The ISEE-3 Spacecraft Details, including propulsion [1]

 The thruster firing on the 8th was a follow up to the successful spin up maneuver that was performed on July 2nd, 2014.  The engine firing on both the 8th and the 9th was unsuccessful in producing the thruster needed to change the course of the spacecraft to what is necessary for the lunar flyby on August 10th.  After the largely unsuccessful Trajectory Correction Maneuver (TCM), on the 8th, our team decided on a strategy to both use alternate thrusters and to troubleshoot the propulsion system.  On the 9th the TCM was also largely unsuccessful (a few pulses out but no substantial change in course).  After a bit of team depression that the mission was over, a failure investigation was started.  Following are the results of that failure investigation, and our plan for recovery.

First TCM Attempt July 8th

After the successful July 2nd spin up maneuver we had high hopes for the TCM.  We had learned quite a bit about the operation of the spacecraft and the spin up maneuver made us and our NASA partners confident in the next steps.  Indeed it was the successful spin up maneuver that allowed NASA to give us permission for the TCM.  However, as events would unfold, this confidence was to be misplaced.

As those who have read of our exploits know, the ISEE-3 is a spinning spacecraft.  It spins quite rapidly, at 19.75 (19.787 right now) RPM or about one rotation every 3.04 seconds.  The thrusters for the propulsion system that allows for an in plane change in velocity are located on panels 1 and 9 (redundant) of the spacecraft.  This is shown in figure 2:

Figure 2: ISEE-3 Thruster Locations and Designations

Figure 2: ISEE-3 Thruster Locations and Designations [2]

 Figure 3 shows the schematic of the propulsion system from the same AIAA paper from 1979:

Figure 3: ISEE-3 Propulsion System Schematic

Figure 3: ISEE-3 Propulsion System Schematic

The spacecraft has two completely redundant fuel, latch valve, and thruster systems.  The thrusters for dV maneuvers are located on the exterior of the spacecraft around the circumference as shown in figure 2.  These are the “radial dV” thrusters.  For the maneuver on the 8th we used thruster’s F and N, which are connected to the HPS 1 tank system.  Everything proceeded normally as we followed the worksheets provided by our documentation.  Things went much better from a commanding perspective as we were able to leave the transmitter and power amplifier in transmit mode at Arecibo while we received telemetry pipelined to us over the internet from the radio telescope at Bochum Germany, which is operated by the AMSAT DL team there.

When we fired the thrusters for the TCM maneuver, initially things looked great, but soon we saw a fall off in thruster performance as monitored by an accelerometer designed for this purpose.  This is shown in figure 4:

Figure 4: Telemetry Profile for July 8th TCM

Figure 4: Telemetry Profile for July 8th TCM

The upper graph is of the accelerometer output during the three firings.  The middle graph is the plot of the Fine Sun Sensor (FSS), which gives orientation.  The bottom graph is the telemetry value indicating whether or not the Latch Valve opened or not.   At this scale it is hard to see the actual acceleration values and they are in counts, not m/sec squared.  It is quite clear the fall off in thrust in the first maneuver.  We now know that the latch valve was not opened (we did not then).  Later, and before we did the second attempt, we see the sun angle decrease.  This so far has not been explained adequately.

The third attempt is the most interesting.  We started wondering about the latch valve and then sent the command for it to open again.  It opened.  The difference is that we did not have the +28 volts on the first time as none of this is in the procedures that we currently have available.  After we commanded the latch valve to open with + 28 volts on we saw telemetry confirmation opening.  Figure five shows what we saw in detail regarding the FSS sensors:

Figure 5: Fine Sun Sensor Angle Variations

Figure 5: Fine Sun Sensor Angle Variations

While this is probably hard to read on a screen but figure six is toward the end here when we first got confirmation that the latch valve between the tanks and the propellant lines was opened:

Figure 6: Latch Valve Opening Vibrations for FSS and Accelerometers

Figure 6: Latch Valve Opening Vibrations for FSS and Accelerometers

The latch valve opens at 9200 seconds and there is an immediate effect on the accelerometer and the FSS sensor.  The data rate is 8 hz for the accelerometer and 1/8th hz for the FSS sensor so there is a lot of aliasing but the influence is clear to see.  This is normally in the form of vibrations on the spacecraft that gives the appearance of a change in FSS pointing, though not in the longer term.  The end result is that we never got the thrust that we expected on July 8th, whether or not we had indications of latch valve open.

Evaluation of July 8th Attempt and Plan for the 9th

Our evaluation of what happened that day is different than what we know now but it informed how we operated on the ninth and thus is instructive to deciphering the nature of the overall problem.  Our evaluation that day was that we thought that the hydrazine tanks in fuel system 1 (on the left in figure 3) was depleted of fuel and pressure.  Our pressure transducers indicated that there was no pressure in fuel system 1 and only minimal (about 4 psi) in fuel system 2.  We also were very unsure of the telemetry indicator for the operation of the latch valves.  Thus the plan was to first attempt to do the maneuver with the fuel system 1 (but use latch valve C rather than A) and thrusters E and M on the other side of the spacecraft.  This would have the effect of using a different set of fuel lines but with the same fuel system tanks.  Then we would then repeat the test using fuel system 2 and the same thrusters (E,M).  If this did not work then we would open latch valve B on fuel system two and repeat with thrusters (E,M).  If that did not work then the final test would be to use latch valve D with thrusters (F,N).

The Attempt on the 9th

The results on the 9th were pretty much the same as on the 8th.  We initially made the same mistake by commanding latch valve C when the 28 volts was not on.  The results were basically as they were the day before.  Figure 7 shows the result of the first firing:

Figure 7: Accelerometer and Other Telemetry Indicators July 9th

Figure 7: Accelerometer and Other Telemetry Indicators July 9th

As you can see, the uppermost blue line is the accelerometer output.  It looks pretty much like the trace in figure 4.  It is opposite in sign because we used thrusters E,M, and sector 556 (the other side of the spacecraft) to start pulsing.  The pale flesh covered lines are where we pulsed (21-30) the latch valve.  There were other thruster firings done, but none of them with any effect.  This ended the pass for July 9th.

Failure Investigation

After a short bout of team depression, we got started on our failure investigation.  In learning how to be an engineer, it is just as instructive to study failure, and many times more so, than to study success.  I have read virtually every failure investigation in the last 50 years for in space failures, from the early Ranger days to GEO comsats, and Shuttle failures.  There is a common thread that not too many people write about, which is that these failure investigations almost always include people from outside of whatever organization was building or flying the system that failed, and that these guys were normally the best in the business.  Another common denominator of such failure investigations it that they take time and lots of money.  Our team has neither.

We seriously thought that there were no options going forward, but in order to understand what happened, and to see what we could learn, we decided to dig into the telemetry and see what happened and to see if we could determine the failure mechanism.  No one on our team is an experienced hydrazine expert.  My own expertise is more in communications, avionics, and power systems, with a good bit of experience in ion propulsion.  Marco Collelouri, our controls and AOCS engineer is very smart, but without a lot of experience.  Thus I felt, after receiving a few emails from people who offered suggestions on what might have happened, decided to throw the problem out to the world.  I was astonished at the response.

The Collective…..

On the 10th we threw out a few questions related to some suggestions that had been sent to us by some of our global distributed network of supporters unsolicited.  Keith put this on NASA Watch and http://www.spacecollege.org.  We immediately started getting responses.  While, as one might expect, many of them were uninformed, though enthusiastic, some were from the most qualified professionals in the world.  I am not going to name names at this time and without their permission but literally, the very top tier of experts started weighing in.

We sent them telemetry, and information on the spacecraft.  Fortunately there is a lot of information out there on ISEE-3, especially from the AIAA (see our references at the end of the article).  With this, and with our telemetry that is also public, we started to build a picture of a set of plausible failure modes and the state of the system.  I must stress that I have had multiple interactions with different groups of experts, some from industry, some from government, and some international.  I have not shared a lot of the information from one group to the other in order to get their unbiased responses, until a consensus started to emerge.  We also were able to be put in touch with (again, through a senior industry engineer), with some a retired person from TRW, the company that built the propulsion system in the first place.  Even though this person did not personally build the one from ISEE-3 he did know the general design and had extensive experience with hydrazine propulsion systems.

The Conclusions of the Ad Hoc ISEE-3 Hydrazine Propulsion System Failure Study

After just a few days of consultations with these groups, and a consensus started to form some common elements stand out.  Here is what we know about the state of the HPS system.  Please refer back to all previous figures as this explanation unfolds.

Integrity of the Propulsion System Finding 1

It is quite clear from the telemetry that both fuel system 1 and 2 downstream of the latch valves was still fully pressurized.  If this had not been the case, we would have not had a successful spin up maneuver, nor would we have had the initial thrust from both fuel systems on the 8th and the 9th.  This means that the following hardware is working:

1. All thrusters have their seals intact and the thrusters provided impulse, showing that the catalyst beds are also intact (or at least mostly so).

2. The propellant lines downstream of the latch valves are also intact and were fully pressurized.

3. Temperatures in the system are high, in some cases, in excess of what is desirable, though not dangerously so.  Temperatures in the propellant tanks are well within bounds.

Here is what we don’t know.

1. How much pressure is in the tanks (the options are failed sensors or depleted tanks).

2. The true status of the latch valves.

Procedures, Finding 2

We made a procedural mistake.  That mistake was turning the latch valves on without + 28 volts applied.  In our defense this was not in the AOCS procedure worksheet, though looking back into the old Mission Operations Plan, checking their status was part of the procedure.  We thought that when we did not have an indication of operation, that the telemetry was dead, which is also the case in the communications system.  Sometimes we imply the state of the spacecraft when telemetry is dead, by looking at whether the command that we sent was executed.  So the latch valve status we thought was dead.  It turns out this was incorrect.  We corrected this in the middle of firing on the 9th, but it did not effect the propulsion system operation as can be seen in figure 7 and right at the right edge where we initiated propulsion but got no response.  After we turned on the +28 volts and commanded the latch valves, they did successfully response (commands 21-30 as shown in figure 7).  You can see that the latch valves vibrated the spacecraft in the accelerometer data.  However, we did not have time before the end of that pass to fully investigate the issue.

Postulations

So the above is what we know.  Now we had to use our own knowledge and that of our outside experts to eliminate, one by one, possible failures.

Propellant Tanks and Upstream Propellant Lines

In examining the telemetry we know that the HPS-1 and 2 fuel systems downstream of the latch valve were fully pressurized.  This eliminates loss of fuel and nitrogen this way.  We also know, and it was our rookie mistake, that nitrogen does not dissolve in Hydrazine, more than just by fractional amounts.  This is why it is used as a pressurizing gas.  We also know that this is a blow down system (see reference 2), which means that the nitrogen gas is mixed with the hydrazine and not in a separate bladder or pressure tanks.  This is a simpler system and it eliminates failure modes.  Thus there is only three ways that we could have lost fuel and pressurizant from the system.

The first way is through the latch valves.  We already know that there was pressure downstream of the latch valves and thus that is eliminated.

The second way is that the fuel and pressurizant could have been lost through either the fill and drain valves or the fill and vent valves.  There are two reasons this is not plausible.  The first is that as far as our experts know, neither of these types of valves have ever failed in flight as they are physically capped before launch.  Also, if they had failed, it is even less likely that they would fail over ten years after launch.  Even less likely than that would those valves failed in both systems.

The third way would be a failure in the propellant lines upstream of the latch valves.  While this has happened in the past.  It is unlikely to have happened in both fuel systems, and if it had, there would be serious consequences for the spacecraft as Hydrazine eats spacecraft wiring and other hardware.  If that had happened it is likely that the spacecraft would have been lost.  Also, this would have had an effect on the attitude of the spacecraft, which we did not see when we first tabulated the telemetry after recovery.

Thus the conclusion of virtually all of our experts is that it is highly unlikely that fuel and propellant has been lost in the system.  This brings us to the next postulation.

Latch Valves

The latch valves on this spacecraft were built by Hydraulic Research (see reference 2).  These valves were popular and used on several spacecraft.  In researching the company, we found documents related to pressure testing and leak testing of the valves.  The valves do preferentially allow diffusion of nitrogen through the seals (we found this data on the NASA technical reports server).  However, since the lines downstream were pressurized, it is unlikely that this happened.  For this mechanism to operate would take further diffusion of the nitrogen through the thruster valves which is also unlikely with full pressure there.

What we did find, and I can’t be too specific here as this information is not in the public domain, is that there are different seal materials and that the type that most probably flew on this spacecraft is subject to temperature based swelling.  Since we also see very high temperatures, in excess of 62 degrees C on the upper propellant lines, which are in contact with the valve, we now have a plausible culprit for our problem.

This possibility is enhanced by our own mistake in how we operated the latch valves prior to figuring out that we had to have the +28 volts on to actuate the valve. This is somewhat mitigated by the fact that later we did actuate the valves and did see physical vibration of the spacecraft from our opening and shutting the valve (commands 21-30 in figure 7 indicate that something happened).  There is a case to be made that the valves did not actually open and we did not fully investigate this during our short Arecibo pass last week.

Temperature

The upper propellant lines near the latch valves is above their specified operating temperatures.  We found that the line heaters have been on since the last propulsion maneuver in 1987, or over 27 years.  While the temperatures are not dangerously high, the long term storage of hydrazine at elevated temperatures can cause the slow decomposition of hydrazine into ammonia and nitrogen and then eventually into nitrogen and hydrogen.  Did this happen?  We have no idea as there is no pressure transducers in the lines.  However, one of our outside experts worked on the Magellan to Venus mission and gas evolution in the propellant lines was seen there.

Since this was a mission to Venus, a planet that gets twice the radiative heat that the Earth gets, and since ISEE-3 came considerably closer to the sun every 354 days for 27 years, it is very plausible that this happened.  If it did, we would still get thrust out as we saw, but there would be gas in the propellant lines.  We did see large drops in temperature in the hot upper propellant lines and large increases in the lower propellant lines.  Rapid swings in temperature could be from gas and or hot ammonia (NH3 a decomposition product of Hydrazine) in the system.

Conclusion

Thus we have a plausible mechanism for our propulsion failures on the 8th and the 9th with the latch valves.  High temperatures expanding the seal material could have either impeded the flow, or have precluded the latch valve from opening even with the microswitch indicated to telemetry that the valve was open.  We also have high temperatures possibly evolving gas, causing a large gas bubble in the propellant lines Is this the case?  There is a way to test both cases.

Corrective Action

There is a pretty good possibility now that we have pressure and or fuel in the tanks but that it is not getting to the propellant lines and out the thrusters.  We are going to of course turn the +28 volts on this time!  We will also open both valves on one of the fuel systems, the primary and redundant.  We will also heat the tanks to see if we can see a rapid increase in temperature.  If we see a rapid rise, that would indicate no fuel in the tanks (testing for all eventualities).  There are several things we will do to test out and try propulsion to bleed all the gas out of the lines.

What we could see would be not much activity and then toward the end of the pulses from the thrusters we could see propellant flow, temperature increase, and thrust!

Cross your fingers.  We will have a pass on July 16th at Arecibo, so we will soon find out what the outcome is.

The Collective Consciousness of the InterNet

There was a great article on space.io9.com related to distributed engineering and our project, and how the people from the net came together to help us.  I first saw the term distributed engineering in the late 1980’s from the amateur radio community.  It began through using ham radio to do this, then it migrated to email before the advent of web browsers, and then to the web.  What happened with our call for help goes far beyond that as the distributed engineering meme begins with a pre organized group of people that collaborate in geographically disparate locations toward a common engineering goal.  Before our call for help last week, I knew maybe one or two of the experts that came in and helped us.  This goes well beyond distributed engineering to a collective consciousness.  I often characterize the internet as the global extension of my brain, with vast stores of knowledge that the brain organizes through the interface of the browser.

In the beginning of the net we used this to research information.  With the rise of the ubiquitous internet among the professional class and beyond in the world, we now have something never before seen on this scale in the history of mankind, a near instantaneous way to not only research information, but to rapidly organize humans to do “things”.  We now have crowd funded efforts that bring people together of like interests to fund interesting projects like ours.  We have crowdsourced collaboration in the arts, sciences, and engineering.  There is a lot of talk about singularities in the technology world, and for the most part they are marketing myths from my experience.  However, and this is what I leave the reader to ponder, we have reached a threshold where vast numbers of people can work together in a near real time manner to solve problems and do good and interesting (or evil) things.  One wonders where this will go….

For us it was great!

[1] Farquhar, R, Muhonen, D, Church, L; Trajectories and Orbital Maneuvers for the ISEE-3/ICE Comet Mission.  AIAA-84-1976, AIAA/AAS Astrodynamics Conference, Seattle, WA, August 20-22, 1984

[2] Curtis, M.S., Description and Performance of the International Sun Earth Explorer-3 Hydrazine Propulsion Subsystem, AIAA/SAE/ASME 15th Joint Propulsion Conference, Las Vegas, NV, June 18-20, 1979

Posted in Space | Tagged , , , , , , , , , | 4 Comments

ISEE-3 Reboot Update, Ranging The Spacecraft

Folks

It has been a bit since my last update.  Tomorrow I hope to have a more thorough one for you all but this one here should be of interest, especially for the more technically inclined.  We had a pass from the NASA Deep Space Network (DSN) that was to be used to improve our knowledge of the position of ISEE-3 in space.  This is not an easy task, though NASA’s DSN team makes it look easy.   Every measurement, in on way or another, depends on accurately measuring the time it takes for a signal to either come from the spacecraft (passive mode), or a two way round trip signal, (ranging mode) or the most accurate means, called coherent ranging where the spacecraft locks to a signal from the ground, and matches its phase coherently.  The ISEE-3 spacecraft, even thought it is of a 1975 design vintage has this ability.  However, things did not go as planned last week.  Here is what I posted on our http://www.spacecollege.org site and this was also sent to our NASA sponsors.

_________________________________________

The DSN pass on June 18th that went from 1:45 to 2:45 Pacific Daylight time was not a success. Here is a recap of the pass activity.  The DSN pass started at 1:45 pm PDT.  Here is a graphic of the pass through a very nice DSN Now web app:

Figure 1: NASA DSN Now Web Interface Showing ISEE-3 (ICE) Pass Via DSS 24 Goldstone

Figure 1: NASA DSN Now Web Interface Showing ISEE-3 (ICE) Pass Via DSS 24 Goldstone

The pass began with a +/- 3 KHz sweep across frequencies representing the input frequency of transponder A (2090.66) MHz + the Doppler offset + an additional 11.25 KHz that came from our most recent command session. The additional offset is due to thermal and or aging issues with the spacecraft transponder. The sweep is done with a carrier only, no modulation, to get the receiver on the spacecraft to lock to the DSN transmitted signal. The output of transponder A will start to vary in a 240/221 relationship when the carrier is locked. Then ranging can occur. The sweep was unsuccessful in establishing a coherent lock. The sweep rate was 60 Hz/sec. This conforms to the procedure used in 1985 by the DSN for the spacecraft during the ICE comet encounter.[1]

A second sweep with a bandwidth of +/-6 KHz was initiated at the same sweep rate. This was also unsuccessful. It should have been successful as a total sweep of 12 KHz encompasses more than the offsets that we have successfully used to command the spacecraft. After this did not work, the DSN ops team then did a sweep at +/- 20 KHz with the center frequency set at our Doppler + 11.25 KHz offset. This surely should have worked in obtaining a lock, but it did not. Due to the time involved to do the sweep this exhausted our available time on the Goldstone dish, and thus we completed the pass without any indication of lock from the spacecraft.

Troubleshooting the Pass

Link Margin

The configuration of the Goldstone system was using the DSS-24 34 meter dish, running 10 kilowatts of power. Table 1 gives the estimated link margin:

Table 1: Estimated Link Margin for Ranging with Carrier at Goldstone for 6-19-14:

Table 1: Estimated Link Margin for Ranging with Carrier at Goldstone for 6-19-14:

Looking at the link margin, it is evident that the link was not the problem.

Failed Subsystem or Procedural Issue?

There are two possibilities for why the ranging failed, when we know for certain that transponder A is functional for commanding the spacecraft. The first is that the ranging function on the transponder has failed. The second is that a procedural error regarding commanding the communications system into coherent mode is at fault. Thus by eliminating the DSN link as an issue and looking at what could be the problem otherwise we have the beginning of a fault tree established for an investigation.

Failed Subsystem

The possibility that the ranging function has failed, or was not functional was taken into account by our team. The spacecraft has two transponders, A, and B, and both are capable of ranging. There are some issues with lower gain on transponder B’s antenna that make it less desirable but with the robust margin from Goldstone seen in figure 2 this is not a problem. On our 6-15-14 Arecibo pass we placed transponder B into coherent ranging mode as well as transponder A. However, due to time constraints the DSN did not have time to attempt ranging to the B transponder. Also, due to lack of licensing to transmit to the B transponder from Arecibo, we have been unable to verify the functionality of that receiver. Thus one possibility would be to range to transponder B for our next DSN pass on 06-25-14. However, this does not address the issue of what the problem might be for transponder A.

Our success with transponder A has not been 100% in sending commands to it. The receiver input frequency seems to be drifting toward a higher frequency over time. We do not know at this time whether that is a random drift or one that is predictable. We overcome that when we do our commanding by sweeping the expected frequency plus Doppler plus an offset when we send commands. Since our first commanding session the offset frequency has appeared to drift to higher frequencies each time. This is not completely unexpected as we found test data that indicated a positive curve for transponder A. If this were the issue the +/- 20 KHz sweep would have locked the receiver. While this does not verify that the ranging mode is not functional, it does narrow the possibilities.

Operational Procedure

With the incomplete documentation at our disposal it is not unlikely that we have issues in operational procedure that preclude the coherent function of transponder A from working. On top of this problem is the legal issue that we can’t just use transponder B because we have been unable to obtain a license to transmit on the frequencies for that system from Arecibo. We have no schematics or vendor documentation on the transponder. We do have test data and we know the specifications and requirements that it had to meet. Worst of all is that as far as we can determine from the digital subcom that provides verification of communications operation, it is not functional, through either the Data Handling Unit (DHU) A or the redundant DHU B. This is not that surprising in that the DHU’s have absorbed more than five times the radiation that they were designed to handle. The only way that we have to verify a function for the communications system is to command that function and then see if it works in the intended way.

We actually were able to verify that the coherent mode is working and to validate the operational issue. Figure 2 shows our Eureka moment for coherent ranging:

Figure 2: Post Doppler Correction Spectrum Trace During Commanding for Coherency

Figure 2: Post Doppler Correction Spectrum Trace During Commanding for Coherency

The above graphic, produced by Phil Perillat from Arecibo, shows that our command bits for coherency did go up and did push the transponder into coherent mode, shown by the 18 kHz frequency jump by the downlink.  There was a 1.6 second lag afterwards and we send the ranging command again, which stabilized the carrier at the offset frequency.  After the end of that command, which at 256 bits/sec took 2.4 seconds, and after an additional 1.6 seconds, the downlink shifts back to its base frequency.  This was our first solid indication, on June 15th that the ranging transponder coherent mode was working.

We had tried ranging through the transponder on the previous pass we had on June 9th.  We did not think that this had been successful but after looking at some of the analyses by Phil Perillat we noticed that the ranging mode had indeed worked on that day as well.  Figure 3 shows this:

Figure 3: Ranging Tone Spectrum With Frequency Offset, Transponder A

Figure 3: Ranging Tone Spectrum With Frequency Offset, Transponder A

In looking at this spectrum we were able to completely verify that the coherent ranging function with tone had worked.  The spectrum above shows this.  We verified this by looking at a document with test data on it from the spacecraft acceptance test.  This is shown in figure 4:

[2]

Figure 4: 20 KHz Ranging Tone In the Time Domain on Oscilloscope from 1978

Figure 4: 20 KHz Ranging Tone In the Time Domain on Oscilloscope from 1978

The difference between the signal in figure 3 and figure 4 is that the first is shown in the frequency domain, and the second in the time domain.  With the correlation between the acceptance test data in figure 4and the waterfall plot in figure 3 we pondered why the DSN pass did not work.

Even though we have incomplete information, we still have a lot.  In our discussion related to the issue we recalled that somewhere in a document it was stated that the spacecraft had to be commanded into coherent mode and that if the carrier dropped for more than three seconds it would automatically drop out of that mode.  After searching this document was found and here is what it said:

Figure 5: Documentary Evidence of Procedure for Coherent Mode Operation of Transponder A

Figure 5: Documentary Evidence of Procedure for Coherent Mode Operation of Transponder A

This provides verification of why we were able to get into coherent mode and the DSN was not able to also do so.

Final Verification of Procedural Issue

Friday June 20th we were going to do the propulsion system test and spin up maneuver.  However, one of our pass/fail criterion was real time telemetry and reliable commanding.  Neither of these criterion were met and thus we cancelled that activity early in the pass.  This gave the team time to focus on operationally testing transponder A’s receive system and to retest the coherent mode to determine whether or not we could command that action and record the results.  This we were able to do and figure 6 is our final evidence of Coherent mode operation:

Figure 6: Coherent Mode Operation Confirmed 6-22-14

Figure 6: Coherent Mode Operation Confirmed 6-22-14

What we were doing, as shown in the waterfall plot above, is that we were sending dummy commands to the spacecraft in order to get the command counter to increment.  This was a test of the command counter but we also send a coherent mode command.  The difference between figure 2 and figure 6 is that we changed our procedure slightly and sent commands one after another without allowing the carrier to drop, which maintained receiver lock on the spacecraft.  This can be seen in the shifting of the waterfall plot and at the end the three seconds of drift before the carrier snaps back to the baseline frequency.

Findings

  1. Coherent Mode Operation

Coherent mode is operating in a manner consistent with the original acceptance test report.

  1. DSN Failure to Go into Coherent Mode

The failure of the DSN frequency sweep to lock the spacecraft into coherent mode is due to the lack of a command to go into coherent mode before the sweep.  The sweep method must be coupled with a command for coherency for coherent mode to be entered.

Solution

The ISEE-3 team can provide to the DSN a packaged coherency command that can be broadcast to the spacecraft as the frequency sweep happens.  At this time our best estimate for frequency offset is Doppler +10-15 KHz.  This is due to aging of the transponder on the spacecraft.  After lock is achieved ranging tones should be sent without dropping the carrier.  Discussion with the DSN should be centered around how our command can be integrated with DSN operations.

[1] DSN_Operations_Plan_1985.pdf (page 5-8)

[2] 1978-02_compatibility_test_report_isee-c_flight_model (page 162)

Posted in Space | 1 Comment

ISEE-3 Reboot Project Update: BULLSEYE! and More

Project Update June 1, 2014

We have returned from our foray (probably not the last) at Arecibo.  It was a wonderful trip, the people down there are marvelous, and Arecibo is the crown jewel of American radio astronomy.  Did you know that the Chinese are building a bigger one, modeled after ours?  The NSF keeps trying to cut their budget but this is truly an amazing asset to the American people and the world and it should be supported!!  I want to thank Mike Nolan, Phil Perillat, Dana Whitlow, Victor Negron, and all the great crew down there.

Technical Progress, Contact Made!

Commanding The Spacecraft

As we put out in our project very brief note on Friday, we have successfully contacted the bird!  At the time of the contact we had Morehead State University Space Science Center’s 21 meter dish, the 20 meter dish at Bochum Radio Observatory in Germany, and the SETI  Allen Array were all listening.  This was not without problems.  The spacecraft has two transponders, which are oddly enough called transponder A and Transponder B. Transponder B is normally the engineering telemetry transponder and transponder A is the ranging transponder.  The final state of the spacecraft at last contact was to have both of the transponders transmitters active and that is what people around the world have been tracking.  However, the spacecraft is set up with a lot of redundancy so you can use either transponder A or B to send telemetry or to range.

We tried several times to command the spacecraft’s B transponder at 2041.9479 MHz into the mode where it normally sends engineering telemetry, which is our first task.  It did not work.  We tried several variations of the proper commands and we tried several operational approaches such as scanning across the receive transponder band to make sure that there was not some offset that we did not know about or that the receive frequency had drifted over the years.  Nothing worked.   Then we tried the same process on transponder A and BINGO, telemetry! Well not really telemetry itself, but modulation from the output of the telemetry system.  The initial command was just to turn engineering telemetry on at 512 bits/second.  This was successful.  Figure one, generated by Achim Volhardt from AMSAT DL and Bochum, shows the modeled spectrum for a 512 bits/sec telemetry rate overlaid on the received signal, which is what we commanded:

Figure 1: Simulated (Solid Line) vs Received Spectrum from ISEE-3/ICE A Transponder

Figure 1: Simulated (Solid Line) vs Received Spectrum from ISEE-3/ICE A Transponder

The second image, figure 2, shows a waterfall plot that clearly shows the sidebands of the expected signal, as recorded from our Ettus Research USRP N 210 receiver as processed through GNU radio by Balint Seeber:

Figure 2: Waterfall Plot of Signal Showing Sidebands

Figure 2: Waterfall Plot of Signal Showing Sidebands

This provided our initial verification that we indeed had successfully commanded the spacecraft into engineering telemetry mode.  We later also, working through the A transponder’s receiver, we commanded through the B transponder’s command decoder to output engineering telemetry through transponder B’s transmitter.  Thus we have verified so far the following systems on the spacecraft.

1. Transponder A receiver

2. Transponder A’s Command Decoder and Data Handling Unit

3. Transponder B’s Command Decoder and Data Handling Unit

We reviewed some of our documents and found that neither of the ISEE-3/ICE receivers had met their specification in testing.  The specification was for -120 dbm sensitivity.  However, we found that receiver A was tested at about -114 dbm, and Receiver B at -111 dbm.  A difference of 6 and 9 db gain respectively.  Working the numbers backward with our 400 watt power amplifier and the gain of the Arecibo dish, we found that we were marginally ok with the A receiver and probably slightly short with the A receiver, calculating in radiated power vs db Eb/No.  Normally NASA and the Deep Space Network (DSN) uses transmitters in the tens of kilowatts range.  Since we could neither acquire one that big in the time that we had, or could afford to buy one, this had driven our decision to use Arecibo rather than a larger power amplifier at a smaller dish.

Our biggest disappointment was that we then tried to command the spacecraft into 64 bits/second mode, which was a mode that is much more complicated to set up, we did not get working successfully during the limited time that the spacecraft is visible from Arecibo.  We need to do this so that the smaller dishes at Morehead State and Bochum will have a positive signal margin so that we can record several hours of data.

THEN WE HAD AN EARTHQUAKE.  As many know who follow our social media, on Thursday after our end to end systems test we had an earthquake.  I was on the central part of the Arecibo dish, 450 feet in the air with Dana and Anthony, another engineer at the site when this happened.  We had just chatted about how observations could be affected by vibrations in the dome structure as it translates during an observation and then that happened!  The azimuth tracking system, which is the curved structure on the underside of the top part of the dish, was slewing while we were there as well as the dome.  We were sitting in a safe area when everything started shaking.  I was doing a video at the time but stopped it to hold on during the shaking!

Demodulating and Decoding the Received Signal

The first miracle was to command the spacecraft.  The second is to understand what it says.  Figure 3 shows a scope plot of the resulting data and clock plotted in time:

Figure 3: Clock and Data Recovery from Demodulator

Figure 3: Clock and Data Recovery from Demodulator

If you look at the bottom of this figure you can see 1’s an zero’s, the bits that come out of this process.  Now our guys are super exited about this and yesterday morning (Saturday) Austin Epps sent out an email based on the first set of bits that Balint got out of the demodulator:

I searched for the synch bits ‘11111010111100110011010000000000’ per the SIRD document.  That string was found in two locations…starting at bit 575 and again at bit 2623.  Note that the two locations are 2048 bits apart, exactly as expected.

We got our synchronization bits, which provides the framing indicator for a frame of data, out of the demodulator!  Not to be outdone, (actually everyone is collaborating and working beautifully together), our new volunteer, a very old hand at demodulating satellite data, Phil Karn, jumped on the data that Balint provided and we have the following fully processed first frame of data!

Gentlemen, feast your eyes on this:

7c 02 02 02 02 02 02 02 7c 02 02 02 02 02 02 02
7c 00 02 00 00 f2 00 00 7c 02 02 79 a0 00 00 00
7c 00 02 33 c8 02 4d 02 7c 4b 02 76 00 00 00 00
7c 02 02 53 01 02 39 02 7c 44 02 00 b1 49 00 00
7c 00 02 5a 00 19 5c 64 7c 4b 02 0e a0 00 00 00
7c 0e 02 4b 47 63 91 1d 7c 42 02 4d 36 00 00 00
7c 45 02 44 4e 8a 89 02 7c ce 02 50 a4 00 00 00
7c 48 02 32 4b b5 d2 ad 7c 33 02 12 fc 81 9f be

This is my very first Viterbi-decoded frame of ISSE-3 telemetry,
extracted from the first frame of the recording I received this morning.

Note that it ends with the 12 fc 81 9f be sequence, the 3-byte encoder
dump sequence 12fc81 followed by the 2-byte sync sequence 9fbe.

I forced the Viterbi decoder to end in the state 819fbe so those last
three bytes could not have been anything else, regardless of what was
received. HOWEVER, the 12 fc decoded just before that actually came from
the received symbol stream, and since that matches the values given in
the documentation this is a strong indication of correct decoding.

As another indication of correct decoding, I re-encoded the
Viterbi-decoded data and compared the encoded symbols to the raw symbols
received from the spacecraft. There were no errors. None.

I think we have it. Now I just need to polish this off so it’s useful.

Fantastic work!  Phil later processed our first day’s data dump from the spacecraft and we received 49 full frames of data at a bit rate of 512 bits/second.  Until the very end there were no errors on the downlink, and only then when the spacecraft was going beyond the horizon for Arecibo.  These are the milestones related to commanding and receiving data from the spacecraft that have been achieved:

1. Successful commanding multiple times of ISEE-3/ICE

2. Received engineering telemetry from both data multiplexing units on the spacecraft.

3. Successful demodulation on the ground of the received data, through the output of bits.

4. Verification of good data at 512 bits/sec, including frame synchronization, correct number of bits/frame, and with no errors, showing a very strong 30+ db link margin through Arecibo.

These milestones alone would be praiseworthy but there is more!

UPDATE: Sunday night.  Phil Karn sent me an email to say that he has now processed over a 1000 frames of telemetry from the ISEE-3 spacecraft.  We are going to have a LOT of fun decoding and displaying the data this week!

BULLSEYE

The Trajectory Problem

One of the major problems that we have, that has to be solved, is to update the range to the spacecraft so that its position, velocity, and trajectory into the Earth Moon system can be properly plotted so that we can then plot a course, and fire the engines for a  maneuver to target a lunar flyby at the proper altitude (around 50 km) on August 10, 2014.  The last trajectory solution that we have from the DSN is from 2001 and it is this one that is provided by NASA JPL in its Horizons prediction program that everyone has been using.

The problem is that this solution has been shown to be inaccurate when we are using the extremely narrow beam width (~2 arc minutes), at Arecibo.  When plotted out to the approximate distance of the spacecraft, 2 arc minutes is only about 16,800 km wide, meaning that if the spacecraft is more than about half this distance (assuming that we point exactly at the right location), then the spacecraft falls outside of this beam and the signal vanishes.  This is except for the fact that the Arecibo telescope is so powerful that the minor lobes of the main beam are still more powerful than most smaller telescopes and that was how we were receiving the spacecraft in the first few days at Arecibo.

Then, Phil Perillat who handles the hard problems in operating the telescope at Arecibo, performed a search to lock on the main beam of the spacecraft.  He was able to do this, and our signal level was over 50 db above the noise, a very strong signal for transponder A, and a bit less, around 45 db for transponder B.  Phil continued to do this almost every day when we could get time on the extremely busy telescope.

There is a huge side benefit to this technique.  In the scientific community for asteroid research radar and optical sighting of Near Earth Objects is used as the only means to determine orbits.  This community has gotten extremely good at this method.  The spacecraft engineering community uses coherent transponders (which is what transponder A is on ISEE-3/ICE) to lock on and do a two way ranging to allow the engineers to calculate a good orbit ephemeris for a spacecraft.

Amazing Accuracy of the 1986 Trajectory

Using the data from Phil’s daily targeting of the spacecraft, Mike Loucks at Space Exploration Engineering, along with the folks at Applied Defense, and then further verified by volunteers from APL working on their own time, we have narrowed down the location of the spacecraft.  It turns out that it is far closer to the Moon than the JPL Horizons propagated trajectory, and very near being on the course intended for it by the ICE trajectory team in 1986!  This is shown in figure 4:

Figure 4: Newly Plotted ISEE-3/ICE Course Compared to Horizons and JPL 2001 Trajectory

Figure 4: Newly Plotted ISEE-3/ICE Course Compared to Horizons and JPL 2001 Trajectory

The Blue Circle is the orbit of the Moon with the Moon’s location show at the right side of the circle (August 10, 2014 location).  The Yellow Horizons trajectory is shown intersecting the Moon’s orbit but no where near the Moon on that date.  The white line was a re-propigation of the JPL Horizons orbit in Systems Tool Kit (Satellite Tool Kit).  The dark blue trajectory is the intended trajectory of the ICE navigation team in 1986.  The red/green trajectory is the plotted trajectory based on Phil Perillat’s pointing data from the Arecibo telescope!

Consider this, the spacecraft has completed almost 27 orbits of the sun since the last trajectory maneuver. That is 24.87 billion kilometers.  They are off course by less than 30,000 km.  I can’t even come up with an analogy to how darn good that is!!  That is almost 1 part in ten million accuracy!  We need to confirm this with a DSN ranging, but if this holds, the fuel needed to accomplish the trajectory change is only about 5.8 meters/sec, or less than 10% of what we thought last week!

We truly stand on the shoulders of steely eyed missile men giants…

What is Next

If we can maneuver the spacecraft by June 17th we get the very small delta V number for the maneuver above.  However, this starts to climb rapidly as the spacecraft gets closer  to the moon.  Also we cannot at this time rule out a lunar impact.  It is imperative that we get a ranging pass as soon as possible.  We also need time to not only evaluate the health of the spacecraft, but to test the systems, the catalyst bed heaters for the propulsion system, the valve heaters, analyze the rest of the propulsion, power, and attitude control system as rapidly as possible.  This will be a lot of commanding so we have to move into high gear next week.

This is a very fluid situation and we have made amazing progress, thanks to the support of those who believed in us in our crowd funding and the support of our NASA sponsors at NASA Ames and NASA headquarters.  We also want to thank the members of the original ISEE-3/ICE engineering and science team.  Without their marvelous efforts, and without the documents that they saved and stored lovingly for decades in their homes this would have been a far more difficult task!  We also appreciate their looks over our shoulders and we will be relying on them to do this more as we work to successfully command the spacecraft to fire its engines.  More to come soon!!

 

 

Posted in Space | 5 Comments

ISEE-3 Reboot: Updates from the Front at Arecibo….

Sunday May 25 2014

We have had a very active week since we arrived and got started work.  We arrived on Saturday the 17th and checked into the visiting scientists quarters near the big dish.  And then we saw it…..

The Arecibo dish is too big to be taken in in one photo so here are a couple to give some sense of scale.

Figure 1: Central Focus of the Arecibo Dish

Figure 1: Central Focus of the Arecibo Dish

See the ring in the middle?  This picture below is what it looks like with people on it looking down to the center of the ring.

Figure 2: Arecibo Dish Rotating Section

Figure 2: Arecibo Dish Rotating Section

Those are people down in the center there.  That gives you a slight bit of sense of the size of the place.

Mike Nolan and the folks at the dish are just the best people imaginable.  There are several scientists in residence at this time running experiments and there are several who do things remotely.  One scientist showed us some amazing pictures of a celestial body that are amazing (keeping it quiet until the findings are published) that were obtained with the radar at the site.  The power amplifier that we had shipped to the site puts out about 400 watts on S-Band.  The one on site here, running at half power, is a megawatt.  We can’t use that one as it is very precisely tuned to its frequency and can’t be changed. The days kinda run together down here so the days are estimates for what we did starting but here are our day to day accomplishments while down here.

Saturday-Monday

Since the power amplifier from Germany was a bit late, we started out by conferring with the staff here on the best approach to be able to use the dish.  Mike Nolan, Phil Perillat, and Dana Whitlow, Victor Negron, and the group are all outstanding technical and scientific people and helped us figure out the ins and outs of using the dish down here.  The dish requires near constant maintenance and so it is near the end of these times, after the work is finished, that we have opportunities to receive signals from the spacecraft.  We brought our own transceivers down here, the Ettus Research USRP 210’s which use the GNU radio Software Defined Radio (SDR) open standards.  Operated by the right people, such as Ettus engineer Balint Seeber (Ettus graciously allowed Balint to come down here with us).

Tuesday

We began by replicating the basic reception of the spacecraft using the dish and using their equipment in parallel with ours.  The first day, which I think was Tuesday, Phil saw that the signal from the spacecraft was weaker than before.  We had some time to work and Balint did some configuring of the USRP receiver.  However, not much more than that happened and we were somewhat concerned about the signal.  Was the spacecraft failing as it approached Earth?  So we asked the folks at Morehead State to monitor the spacecraft on Wednesday in parallel with Arecibo in order to see what they could see.

Wednesday

On Wednesday we got the great news that the power amplifier had been shipped from Germany, next day Fedex!  The team worked during the day writing software that would be used for receive testing at the dish.  When the pass started, we were very surprised to see that the signal from the spacecraft was basically missing.  We consulted with Morehead and found that they were receiving the signal stronger than ever.  We also were receiving signal from the 6 meter dish at SETI in California.  Phil Perillat then did some investigating and found the signal, but at a very weak signal strength on a side lobe of the dish.  (Lobes are part of the spread of received energy from the dish).  Some calculations indicated that if Morehead could hear the signal in its main lobe but we could not, then there was a divergence between the position of the spacecraft and the ephemeris file provided by the NASA JPL Horizons server.  We did not have enough time on Wednesday to find the spacecraft on the main lobe.

Thursday

Huzzah!  We received the tracking number for the power amplifier and instead of waiting for it to be delivered around 5:00 pm, we and the film crew with us took off to the west end of the island to pick up the amplifier at Fedex.  We were greeted by Wilson at the office (if you don’t know who or what Wilson is, watch the Tom Hanks movie about the crash of a Fedex plane):

Figure 3: Wilson Greets us at Fedex in Puerto Rico

Figure 3: Wilson Greets us at Fedex in Puerto Rico

We picked it up early and got back to Arecibo before 1:30 in the afternoon, when it started raining which halts all activities in the center focus of the dish for safety reasons.  However, we did remove it and its power supply from their crates and set up the system to begin testing at the control center on Thursday evening.

We also were able to get some quality time in on the system for a receive test.  The ever diligent Phil Perillat initiated a search for the spacecraft and was able to lock onto it, but with an offset of -0.47 degrees in Right Ascension (RA) and 0.08 degrees in Declination (Dec).  After some calculations this indicated that the spacecraft was approximately 250,000 km away from its expected location.  With this information the signal was solidly locked to the spacecraft with approximately 55 db of signal to noise on one frequency and approximately 45 db on the other.  The difference is due to polarization differences between the two signals principally.  We were also able to lock onto the signal using the Ettus USRP 210 radios of ours.  This is shown in figure 4:

Figure 4: FFT and Waterfall Plot of ISEE-3/ICE Signal.

Figure 4: FFT and Waterfall Plot of ISEE-3/ICE Signal.

We had some very happy campers in the control room!:

Figure 4: The ISEE-3 Reboot Crew After Signal Confirmation on USRP Radio

Figure 5: The ISEE-3 Reboot Crew After Signal Confirmation on USRP Radio

With the successful reception of the signal on the USRP radio we confirmed at least the entire receive chain and processing through our laptops the signals in GNU radio.  Balint did a marvelous job there and we had help from John Malsbury who had been with us the previous two days.

Friday

On Friday the team from Arecibo lifted the Dirk Fischer Electronics power amplifier from the bottom of the dish up to the central focus and installed in the transmitter room. Figure 6 shows the power amplifier and its power supplies installed and ready to go:

Figure 6: Dirk Fischer Electronics Power Amplifier Installed in the Arecibo Transmitter Room

Figure 6: Dirk Fischer Electronics Power Amplifier Installed in the Arecibo Transmitter Room (in the Dome)

During the previous few days discussions had been had with Dana Whitlow and Victor Negron about how to get our transmitter signal from the USRP up to the power amp.  There are several fiber optic cables (as stray RF is a big no no there) that could carry the signal from the control room to the dome.  Then there was a coax to go from there to the power amp.  This created a lot of signal loss so preamplifiers were installed on top of the power amp that would allow us to boost the signal to the proper level to drive the transmitter to full power.  All of the transmit path from the control room to the preamps were tested and shown to be operational.

Final Testing Late Friday

With everything installed we then decided to do a final end to end test of the system.  We sent signals up to the preamps that then allowed Dana Whitlow adjust them and to monitor the output of the power amp.  After some trial and error this was accomplished and all four of the amplifiers that made up the combined power amp were tested and it was confirmed that they operated at full power while remaining relatively cool in the transmitter room.  This is the same room where the big megawatt klystrons are located.  In this test we found a couple of configuration issues to deal with which were successfully fixed.  Thus by the end of the day, we had a completely successful end to end test of our transmission and reception system.

Discussion and Agenda for Next Week

Originally we were going to head home this past Friday.  However, with the inevitable delays that are part of doing real work, we reached our first major success milestone here at Arecibo on Friday.  Thus we have extended our stay for another week in order to actually transmit to and attempt to command the spacecraft.  The end to end test revealed a couple of operational issues that we have to deal with such as that with a dish with this fine of a main transmit lobe, that we may have to offset the dish to “lead” the spacecraft so that it will be where it should be when it receives the signal.  This is doubly true as we now know that the spacecraft is off course compared to the existing ephemeris.

The error in position has just elevated the concern level greatly.  We know approximately what the offset error is from the existing ephemeris but we don’t have enough information yet to plot a new course and generate a new ephemeris file.  This has become extremely important as there is a solid statistical chance that the spacecraft could impact the moon or even by off course enough to threaten other spacecraft in Earth orbit.  We are working with Mike Loucks of Space Exploration Engineering (SEE) our trajectory guy on this issue.  An east coast company, Applied Defense (ADS) has also offered their help and engineering support to derive a new ephemeris from our new position reports.  ADS and SEE did the trajectories for NASA’s just completed LADEE mission.  It won’t be perfect but it will be an improvement over what we have now.

This is a necessary improvement as we are far enough off course that if we fired the engines now for a course correction, it could easily make matters worse.  So we have just had the level of concern increase dramatically!  We will know more when we are able to get time on the dish again, maybe Monday at best but definitely Tuesday so that we can have multiple plot points along an arc to give to Applied Defense for them to work with creating a new orbit plot.

We hope that if all of the stars align (some of the stars are paperwork), that we can attempt transmission to the spacecraft to command it into engineering mode on Tuesday.  It is imperative for us to do this and to attempt to range to the spacecraft to further refine the orbit.  Reading the engineering telemetry, debugging the demodulator and the telemetry system software are our biggest tasks this week.  We will have Morehead State and the SETI Allen Array (or some subset), listening to see if the telemetry is being transmitted.  This will allow several hours of reception instead of just the 2.5 hours here at Arecibo.

This is all for now.  Watch our @ISEE3Reboot twitter feed and http://www.spacecollege.org website for the latest!!

Oh, and thanks to all of those who donated to our project.  As you can see, your money is being well spent!

 

 

Posted in Space | Tagged , , , , , , | Leave a comment

ISEE-3 Reboot Project: Aiming for First Contact

Date 5/13/14

Today’s update regards the progress of the ISEE-3 Reboot Project team in our preparations to contact the spacecraft.  We started this effort 32 days ago on on April 12 2014.  Below is what we have accomplished in that time.

Technical Progress

The Learning Curve

Perhaps the toughest part of doing something like this in a very limited timespan is to climb the learning curve – and to do so with a spacecraft you knew very little about.  Early on we did a preliminary evaluation of the spacecraft and its systems so as to better understand it.  This was a long jump into deep water.   As we did with our Lunar Orbiter Image Recovery Project which concerns the 1960s era Lunar Orbiter spacecraft, the search for ISEE-3 documents has been intense and not without failure.

Most of the best information that we have been able to find has been from the people who worked on the project in the 1980’s when the spacecraft was fully operational.  This image shows the trajectory of ISEE-3/ICE during that period:

Figure 1: ISEE-3/ICE Trajectory through 1986

Figure 1: ISEE-3/ICE Trajectory through 1986

We have also obtained several documents from NASA as part of the development of our Space Act Agreement. Yet many holes still remain in our knowledge of the spacecraft that we have to deal with.  For example, we don’t really know the last state of the spacecraft.  In digging through documents and consulting with those who should know we have yet to find a record of the last commands sent to ISEE-3.  We know that both transmitters (2270.4 Mhz and 2217.6 Mhz) are fully operational but they are not sending telemetry.  Clarity is finally coming to us as our team (Austin, Cameron, Marco, and Tim) have diligently gone through the documents we have.  They are coming up with a series of commands that we will need to send to the spacecraft to set up its engineering telemetry mode.

Since there is no computer on board the ISEE-3 spacecraft our task is actually much easier since we are going to be directly commanding various subsystems. Our team has made a lot of progress in this area.  We are going to be ready to head off to Arecibo this weekend to attempt the first commanding of the spacecraft using their 305 meter dish, the largest in the world.

Hardware Status

Modulator/Demodulator

We now have our Ettus Research Software Radio’s inhouse at ISEE-3 Mission Control.  Figure 2 shows the spectrum of our signal at a ISEE- 3/ICE receiver frequency of 2090.66 Mhz:

Figure 2: Ettus Research USRP N210 Output at ISEE-3/ICE Receiver B Frequency

Figure 2: Ettus Research USRP N210 Output at ISEE-3/ICE Receiver B Frequency

We have a baseline configuration completed by the engineering team at Ettus to modulate (using GNU Radio) the waveforms required by the ISEE-3/ICE spacecraft.  We expect to be ready for receiving and debugging the demodulated stream if we are able to command the engineering telemetry mode to “on” aboard ISEE-3.  We have a couple of questions related to parity generation and convolutional coding that we hope that we have right. However, since things are all in software, we can play around with parameters while we are in a window waiting to see the spacecraft.

Command Generation Console and Telemetry Screens

Tim Reyes in Mountain View, California and Matt Sachs from Huntsville, Alabama have been working together to develop the consoles for commanding the spacecraft.  Figure 3 shows a mockup of the screens without the commands present:

Figure 3:

Figure 3: ISEE-3/ICE Command Console Screen (Tim Reyes and Matt Sachs)

We are ready to go with the commanding structure but we are still working on exactly what commands to send.  Since we don’t completely know the current state of the ISEE-3 spacecraft we are going to make some assumptions and then work from there with a contingency plan of multiple commands, being extremely careful!

The telemetry screens are coming along as well.  Our team has been focusing on climbing the steep learning curve for understanding the telemetry system.  In this effort details are everything.  We have to understand the calibrations required for things such as temperature and pressure sensors, solar array voltages and currents, attitude determination sensors, etc..  This is an ongoing process and we have, as usual, dug some of the pertinent information out of 35 year old IEEE or AIAA papers that are publicly available.  Figure 4 is from the old 1978 Mission Operation Plan that we obtained from Bob Farquhar:

Figure 4: Original Telemetry Screen from the ISEE-3 Mission

Figure 4: Original Telemetry Screen from the ISEE-3 Mission

Depending on how much time we have, we will do a very quick and dirty system that Austin Epps will build. Ideally  we will use a Labview version, provided to us by Eddie Rodriquez from National Instruments.  The people that make Labview have been incredibly supportive of our efforts and long term we will use Labview for our operational support of the spacecraft for engineering and science.

Another issue we have is that we have no idea what has failed on the spacecraft since telemetry was last obtained.  There are no surviving records that we can find of what was working the last time the spacecraft was operated.  This makes things more difficult in that we have to debug not only our telemetry system, our modulator and demodulator, but we also have to determine whether or not to believe what is coming back from ISEE-3’s telemetry indicators.  This makes things more complicated, but not impossible.

We are not going to use exactly the conventions that you see above for the telemetry system.  We may start out that way but we want something that is much more in keeping with modern graphical systems.  Keep in mind that all of this is being done on a rapid basis, and much of the work is being done by volunteers.  All of the folks working on this, whether they are volunteers, NASA folks helping us with the Space Act Agreement, or our own internal team, have done a marvelous job in getting rapidly up to speed on what is necessary to pull this off.  The next miracles that need to occur are related to transmitting to- and then commanding the spacecraft.

Ground Station Transmitters

We have two ground station power amplifiers that we are currently working with. These amplifiers are being sent to our ground station partners.  One amplifier is from AR RF/Microwave Instrumentation of Souderton, Pennsylvania.  This unit is a 700 watt transmitter (Model 700S1G4) that AR is loaning the project to be used on the big dish at the Morehead State University Space Science Center (MSU-SSC).  The Model 700S1G4 is a portable, self-contained, air-cooled, broadband, completely solid-state amplifier designed for applications where instantaneous bandwidth, high gain and linearity are required.  The model 700S1G4 is a 700 watt minimum output power amplifier at S Band frequencies (2.0.2.2 GHz for our application).  The folks at AR RF/Microwave have graciously loaned us the transmitter for when the spacecraft is close enough to the Earth (around mid July) for a link to close for commanding ISEE-3/ICE.  The power amplifier will be driven by a Ettus Research USRP N210 software radio transceiver.

The second power amplifier is coming from Dirk Fischer Electronics in Senifurt, Germany.  This power amplifier is being built specifically for us and will be shipped to Arecibo to be installed next week.  Figure 5 shows parts of this transmitter under construction:

Figure 5: DK2FD Power Amplifiers Ready for Testing

Figure 5: DK2FD Power Amplifiers Ready for Testing

The power amplifier as well as the Ettus Research Radio’s and other equipment has been purchased with the Rockethub-provided funding (well we are borrowing against it right now).

 Other Equipment

We have purchased a laptop onto which we are loading GNU Radio software and Linux operating system. As soon as we get our Labview developer software we will start working with it as well and integrating the work that Eddie and Mike have done for us in that area. We may not have everything working by next week but we will have the critical parameters loaded. There will probably be more hardware to buy but for now and for next week this will work.

Operational/Program Management

 Space Act Agreement

All the T’s are crossed and the I’s have been dotted for the Space Act Agreement (SAA) with NASA. The document has been put into the SAA formatter and (in theory) it will be forwarded to the lawyers for a final review before signing sometime in the next day or two. NASA has already provided us with the documentation from Goddard Space Flight Center and did it early so as to help foster our success in the effort.

The spacecraft is traveling at a quarter million miles per day, and with this in mind everyone is working together to fulfill the terms of the agreement as if it was already signed. Since what we are doing is setting a precedent for future activities of this type we are sure that they are just making sure that they have assured themselves that everything is done just right.

Arecibo

Our first official attempt to contact the spacecraft will be at the Arecibo antenna in Puerto Rico the week of the May 19.. The folks down there are working to fit us into their busy schedule of radar observations of Near Earth Objects. That kind of stuff is interesting to me as well so it will be a lot of fun to see a place that I’ve seen on TV and in movies for most of my life. We are working out an arrangement whereby to where the ISEE-3 Reboot Project will donate the power amplifier that we are getting from Germany to Arecibo in consideration for their support of our project.

Morehead State University

The Morehead State University Space Science Center 21 meter dish is going to be our primary ground station for the activities leading up to and including everything required to put the spacecraft into its final science operating state. However, until the spacecraft is within about 2-3 million kilometers of Earth, which will be in mid to late July, it does not have the wherewithal to close the link with the spacecraft so as to allow two way communications. During our time at Arecibo we will be commanding the spacecraft from there. If we are successful in putting the spacecraft into engineering telemetry mode, one of the USRP N210 radios and our software will be there to process and store the telemetry received. We will be shipping a radio there on Thursday to be ready for next week.

Bochum Observatory and AMSAT-DL

The Bochum Observatory, located in Bochum, Nordrhein-Westfalen, Germany is a private institution set up in 1946 by professor Heinz Kaminski. The observatory has a 20 meter dish that is used for radio science, amateur radio operations, and as a science receiver to receive data from many different satellites. The Amateur Radio Satellite Organization in Germany (AMSAT-DL) works closely with the Bochum observatory. It was this group that in early March of this year detected the signals from ISEE-3/ICE which first generated our interest in communicating with the spacecraft.

The group there with Achim Vollhardt and Mario Lorenz have been working with us and providing signal strength readings from the spacecraft. Independently of our effort they have developed a demodulator for the signals from ISEE-3/ICE. Due to ITAR limitations we have been careful in developing their site for commanding the spacecraft but we are working out a way to do so without compromising our requirements in the ITAR regime. What we hope to do is to have them receive the signals from the spacecraft, demodulate, store, and forward them to us after we command the spacecraft into engineering telemetry mode. With sites in Germany, Kentucky, and Arecibo we will have  good coverage, though the Arecibo dish only sees the satellite until the Earth turns away.

 Other Ground Stations

We are working to get on other ground stations. The SETI Institute in California has been tracking the signal as a means to calibrate and troubleshoot the Allen Array. We are working with them and hope to be able to provide them with a radio and or software so that they can demodulate the signal.

 Mystery Station

We have a another station here in California that we are talking to in order to get their support. Can’t report anything until we close that effort but things are in negotiation now.

Surprise! Google Creative Labs and the ISEE-3 Reboot Project

 Some folks had mentioned and recommended to us to get a film crew to go with us to Arecibo as this will be a very noteworthy event. We were recently contacted by the folks at Google Creative Labs about our project and we have worked out a collaboration for them to film us in operation at Arecibo! We have created a Google + page as well and are working to turn this into a longer term means of better communicating and disseminating our educational and scientific product for the ISEE-3 Reboot project.

We hope to be able to do a live stream from Arecibo with the Google Creative Lab folks for our attempt to command the spacecraft. It looks like we are also going to be there when the transmitter arrives from Germany so we will work with the Arecibo folks to get it installed as soon as possible.

Funding Status

On the evening of May 14, 2014 we reached our fundraising goal for the project! First of all a hearty thanks to everyone for your support and as soon as the funding gets to us we will start putting together the goodies to everyone. When we started this we only had a very vague idea of how much we would need to do this. We have spent about $35,000 so far with more spending coming as we go to Puerto Rico to Arecibo to attempt the first contact. We had to borrow and get terms to pay for some items that we will have to pay for when the funds arrive.

In all probability we will need more funds so we would like to ask for folks to continue to give as you can until the funding window ends on Saturday morning 17t May, at just about the time we get on an airplane to Puerto Rico. We will take a hiatus for fund raising but there is a strong possibility that we may have to pay for the Deep Space Network to do a ranging to the spacecraft for us. If so we will definitely have to do more fund raising. I still marvel at the technical ability of the DSN and how it does interplanetary spacecraft ranging. It is absolutely a non-trivial task!

Next Week’s Activities

Next week is crucial to the success of our project. Using the dish at Arecibo gives us the best chance of being able to command the spacecraft in the very near term. Every day is exceedingly important to us right now. The spacecraft gets about the distance from the Earth to the Moon closer each day and now every day the amount of propulsion burn it takes to make the trajectory correction grows.

The ISEE-3/ICE spacecraft was never really designed to be an interplanetary cruiser and thus the thrusters on board are very small. We estimate that if we wait until mid-June to do the course correction that it will take 17 hours of thrusting to get the course change of about 40 meters/second that we will need at that time. As such, everyday we wait, the risk increases. You see, as we have to continually command the spacecraft to fire for every 512 pulses, thus increasing the chance that something will go wrong.

Figure 6 shows the track of the spacecraft against the sky right now and the distance that it travels per day shown as the tick marks. It is pretty much impossible to see it now, but at least you can get an idea of its track in the sky as it approaches the Earth:

  Figure 6: Track of the ISEE-3/ICE Spacecraft The Sky Mid May 2014


Figure 6: Track of the ISEE-3/ICE Spacecraft The Sky Mid May 2014

Our flight operations engineer Tim Reyes put this graphic together.

Our team has done a marvelous job getting everything together and climbing an incredibly steep learning curve.  We have another new person Cameron Woodman, who has also done just a stellar job in helping everyone else get up to speed and to help with the back and forth engineering thought process related to getting the telemetry system going.  Marco Colleluori is putting in an incredible amount of effort to get Matlab to do what the old ICEMAN program did 35 years ago.

We have a great group of other people as well from our artist Mark Maxwell, who put together our logo, the folks at the ground stations, the team of ISEE-3/ICE alumni, without whom this would have been impossible to do!

And of couse without my project co-lead Keith Cowing, we would have never generated the immense public interest that the project has developed or collected donations from so many interested supporters that is going to translate our plans into reality.

This is a very interesting spacecraft and I think that this process has been a great learning experience for our team and we look forward to the possibility of being able to return this truly historic spacecraft to science operations.

 

 

Posted in Space | 1 Comment

ISEE-3 Reboot Project Near Term Objectives

Introduction

The ISEE-3 Reboot project is an effort to contact, evaluate, command, and place back into  operation in an Earth orbit the International Sun-Earth Explorer #3 (ISEE-3) spacecraft. In 1978 the ISEE-3 spacecraft was launched as part of a trio of spacecraft to monitor and understand the properties of the Earth’s magnetosphere (the Earth’s magnetic field) as it relates to how it is influenced by the various forms of radiation emitted by the Sun.  ISEE-3 basically wrote the book and invented the term heliophysics.  Later the spacecraft was renamed the International Cometary Explorer (ICE) and was the first to visit a comet (Gaicobini-Zinner on Sept 11, 1985), and Halley’s comet in March of 1986.

In 2014 this venerable spacecraft returns to Earth’s orbit and our primary objective is to regain control of the spacecraft and command its engines to fire on a trajectory that will result in a capture into a permanent Earth orbit.  following this, we hope to return the spacecraft to science operations, using its instruments as they were originally designed.  The data from the spacecraft will be open to the public and will be used by the heliophysics community and will be a tool for teaching operations and science data gathering from a spacecraft by students and the public.  In the following sections we will detail the engineering objectives of the project until it is in its final Earth orbit.

Engineering Objectives for the Recovery of ISEE-3

Problem Statement

Figure 1 shows the trajectory of the ICE/ISEE-3 spacecraft from its last propulsive maneuvers in 1986 until its return to the Earth in 2014:

Figure 1: ICE/IESS-3 Trajectory Through August 10, 2014

Figure 1: ICE/IESS-3 Trajectory Through August 10, 2014 (Tick Marks are 14 Day Intervals)

NASA has gotten rid of all of the hardware that understood how to talk to ISEE-3 around 1999.  The spacecraft was last commanded in 1986 or 1987.  The last time it was listened to, with a carrier only, was in 1999 and 2008.  Also, NASA has determined that it does not have the funding to recover the spacecraft and thus on April 12, 2014 the ISEE-3 Reboot Project was born.

Engineering Questions to Be Answered

The principal questions that have to be answered in order to successfully place the spacecraft into a stable Earth orbit and return to science operations are:

  1. Is the spacecraft still operational in the same state as for the previous reception by the   NASA Deep Space Network (DSN) in 2008 and can we receive signals without the support of the DSN?
  2. If the spacecraft is still operational can the equipment and software required to command the spacecraft and read its telemetry be reconstituted on a rapid basis?
  3. If the answer to (2) is positive, are there non DSN assets that can be used to command the spacecraft?
  4. if the answer to (2) and (3) are positive, can the spacecraft be commanded back into telemetry mode in order to debug the telemetry software system, and determine the health of the spacecraft for a thruster firing?
  5. If the answer to (2,3, and 4) are positive, can we obtain an updated ranging to the spacecraft in order to improve on the existing trajectory information.
  6. If all of the above are positive, can we reconstitute the programming necessary to fire the thrusters to modify the spacecraft’s trajectory as desired?
  7. After the spacecraft is placed into a stable Earth orbit, can we return it to science operations.

In the questions above, there are a plethora of subquestions within each one.   This is a summary document explaining where we are currently at in the process and to provide an abbreviated path to answering the questions posed above.

Spacecraft Operational Status (Question 1)

We have multiple reception reports from various telescopes around the world.  These include Arecibo, the Bochum radio observatory in Germany (AMSAT-DL), the big dish at Morehead State University Space Science Center (MSU-SSC) in Kentucky, and the SETI Allen Array telescope in California.  Both transponders have been received and a cursory analysis of the carrier signals indicate that  the spacecraft is still in a stable spinning mode with a rotation rate of 18.6 RPM, close to its last known values.  Thus question 1 has been answered in the affirmative.  The ISEE-3 reboot team now includes the above dishes, with the exception of the Allen array.  We expect to add more dishes in the following week.  Thus we can consider this question answered in the affirmative

Equipment and Software to Talk to the Spacecraft and Do We Have the Assets to Do So? (Question’s 2 and 3)

Modulator/Demodulator

Since we only started this on April 12th of this year, designing a hardware modulator/demodulator was out of the question.  In an extremely fortunate happenstance, Ettus Research has its office less than three miles from ours, and they are experts in the design of software radio systems.  Beyond this, the engineers at Ettus think that this is a cool project and they have developed, based upon information that we have obtained from public sources, the modulator that will allow us to talk to the spacecraft.  Additionally, the demodulator is under development now and will be available very soon.  We are reconstructing this from the documents that we have available.  However, soon we hope to have other documents from NASA that will possibly help to clarify things further so that we don’t make any mistakes.  We have ordered the hardware for four of these Software Defined Radios (SDR’s) and should have them in our possession before the 15th of this month from National Instruments, of Austin Texas, the company that owns Ettus Research.  Thus the answer to this question is that it is in an advanced state of progress.

Command/Telemetry System

We have been able to obtain the most current list of command codes and most of the information that we need for reconstructing the telemetry system for ISEE-3.  We obtained these from sources other than NASA.  Our internal team has converted the command lists and we have an internal and an external team developing the program to process the command codes.  This is obviously the most important part of the system and it will interface with the modulator/demodulator combination.

We are also diligently reading all of the telemetry documentation that we have been able to obtain from public sources.  We are going to develop telemetry screens for mission operations for the propulsion, the attitude determination and control system, and the power system.   Right now we have a team of experts in Labview, the graphical instrumentation software from National Instruments, developing these screens.  We will integrate our data regarding the calibrations, constants, variables, and other outputs from the spacecraft with the Labview telemetry display system.  The Ettus SDR system interfaces directly with Labview so we are able to develop this entire system in a fraction of the time otherwise needed. So the answer is that we are moving forward toward integrating the work of several people, spread around the country, in the next week.

Ground Stations

As of May 4th, we have three ground stations in our network.  The first and most important in the near term is the 305 meter in diameter big dish at Arecibo.  This is the largest radio dish in the world and the team down there, led by Mike Nolan, has graciously agreed to provide support to the project on a non interference basis.

The second ground station, and that we will be working with over the next few months, is the dish at the Morehead State University Science center.  This 21 meter dish is not large enough to contact the spacecraft now, but will be able to after the spacecraft gets within about 2-3 million kilometers distance.

Our third core ground station is at the Bochum Radio Observatory, and is operated in concert with the Amateur Radio Satellite Organization Germany (AMSAT-DL).   We are still working the numbers required to be able to do ranging as well as receiving telemetry at this station.

We are working now with another organization that will be named should our negotiations be successful that is an educational institution that has control over some very impressive dishes.  If this works, this may be our prime ground station in California. So this is an ongoing process.

Transmitters

Key to being able to command the spacecraft is the ability to transmit to it on the appropriate frequency.  We have two efforts ongoing at this time in this area.  The first is that we have a transmitter under construction now, from Dirk Fischer Elektronik in Germany.   Dirk is an expert in the design, development, and construction of microwave systems.  Microwave is still a black art and Dirk’s products are used around the world in the amateur radio community.  This transmitter is being provided to the Arecibo telescope facility expressly for the purpose.  We expect it to be delivered to the telescope sometime around May 18th.  This transmitter will be used, in concert with the Ettus SDR and Labview to attempt to command and receive telemetry from the spacecraft.  If time permits we will have Dirk built at least one more transmitter for the folks at AMSAT-DL and for California, depending on several factors still in the air…

We have another transmitter that is under construction that would be sent to Morehead State University Space Science Center.  When I get permission I will release that company’s name.  We will get a shipping date for this transmitter this week.  This system will be a loaner from the company and will be used to command the spacecraft after it gets closer to the Earth.  More on this as it evolves.

So the answer to question 2 and 3 above is that everything is in progress!  We have had a lot of success, driven by a lot of volunteers that have shown up that we will properly acknowledge in our next post about our team.  It is amazing to behold and if things work as they seem to be, we will fly to Arecibo around the 17th-18th of this month to attempt to command ISEE-3.

Commanding the Spacecraft (Question 4)

There is one prequel to commanding ISEE-3/ICE that we want to do.  We are going to send a tone to the ranging transponder.  We know the transmitter is operational.  We want to see if the receiver for the ranging transponder is turned on, and if we can get a return of that audio tone.  If that happens, this means that we can range to the spacecraft before commanding it.  This may change our sequence of events dramatically as we really would like to do ranging to update the spacecraft’s trajectory. (meaning the question 5 may be answered first, before 4)

The next thing we will do is to attempt to command the spacecraft to turn on its engineering telemetry mode.  We can determine that this happens by watching the high resolution spectrum from the engineering telemetry transmitter.  We then feed that signal into the SDR demodulator to see if we can get bits out.  If we get bits out, we feed them to the Labview/Matlab application that will take the bits produced and route them to the appropriate screens.  This is the big test and we have two things that we have to debug at once.

The first thing to debut is the demodulator itself.  This is done by reading out the bits and see if they make sense in terms of packet size, format, and content.  If that works, then we have to debug the telemetry screens to makes sure that for each sensor in each subsystem  (power, propulsion, and attitude system) is in the range that it is supposed to be.  The difficult part will be debugging the Labview code, our calibrations, and the determining that the spacecraft is functioning properly.  This may take a bit of time!!  Which is something we don’t have by the way.  We have experts in Labview working with us and our internal team is handling the telemetry and command formatting so they are giving it their undivided attention.

When all of the above turns out ok we go to the next step.

Ranging (Question 5)

Ranging, without the Deep Space Network is an incredibly difficult affair.  We have some ideas on this but they are still under development.  Thus, it is pre mature to discuss.  At the end of the day, this is our most challenging question to answer.  However, we have a few ideas that we are mulling around and should have at least a direction toward an answer by the end of this week.

The ICEMAN Cometh (Question 6)

There is a program that was originally used by the command team for ISEE-3/ICE called ICEMAN.  This program was used to implement firing sequences for the spacecraft’s thrusters.  ISEE-3/ICE has three sets of thrusters, two sets of which we will use to modify the trajectory of the spacecraft to put it on a more optimal path into Earth orbit.  Figure 2 shows the spacecraft with its thrusters:

Figure 2: The ISEE-3 Spacecraft Details

Figure 2: The ISEE-3 Spacecraft Details

There are two sets of radial and axial thrusters in addition to the spin/despin thrusters.  The spacecraft is spinning about the centerline that points toward the north ecliptic pole. The spacecraft is spinning at 18.6 RPM which is a little slower than once every three seconds.  So what has to happen is that if you are going to thrust in, or against the direction of travel,  you have to fire the radial thrusters in pulses.  These pulses are timed by a sun sensor that is also mounted on the spacecraft.  It is actually a very clever design.  Since the thruster is not firing exactly in the direction of travel (since it is spinning), that has to be taken into account as well.

ICEMAN took care of all of these variables.  However, it is in fortran code that we, as of this date, don’t have.  So, what we are doing is going back to first principles and rebuilding the functionality of ICEMAN in Matlab (well Marco Colleluori, our grad student from San Jose State is doing this).  We do have some support from some of the former flight team and we may get a copy of the original code.  The reason that this is important is that we have found that the thrusters can only be fired a certain number of times before overheating and thus we have to take that into account.  We are working to get the code from some of the retirees or from NASA.  We will be able to take the output of ICEMAN and feed it into Systems Tool Kit (formerly Satellite Tool Kit) astrogator, where Mike Loucks will simulate the thruster firings from the ICEMAN output to verify.

After the telemetry system is up and running and we know the state of the propulsion system we may try a couple of test firings before we go for the long sequence.  At the time when we think we can actually do this, sometime in June, it will take thousands of firing pulses to get the orbit trajectory that we want accomplished so a couple of test firings first is prudent.  If that works then we load the entire sequence, and pray.  Our objective of course is to modify the trajectory such as to capture into Earth orbit, not the final orbit, but a transition orbit whereby we can then see what we have to do for the final maneuvers.  Figure 3 is our graph of panic, which tells us by what date this has to be done before the spacecraft no longer has sufficient fuel for the maneuver:

Figure 3: dV required for Earth Capture Course Correction

Figure 3: dV required for Earth Capture Course Correction

When the number on the left goes higher than about 150, it is game over.  Thus if at all possible we are going to try and get this done by early June.

Earth Orbit and Science Operations (Question 7)

Due to the press of time and lack of resources, we have not given much thought as of yet to the science instruments.  If we get the burn done in June we will have time to work the science instrument issue before the August 10 flyby maneuver.  We have a critical issue for the flyby which is to turn on all the spacecraft’s heaters for its propulsion system so that it does not freeze during a 25 minute eclipse period during the flyby.  The battery on the spacecraft has been dead since 1981 so there is no help there.  Fortunately, this spacecraft does not have a computer!  The memory on board is not really memory as we understand it today so it will retain and regain its pre eclipse state after it comes back out of eclipse and power is generated by its solar cells.

We are still discussing the final orbit of the spacecraft in Earth orbit and this will be the subject of a future post.  By the time we get back into Earth orbit we will have put together the commissioning plan for the experiments.  Several of the original principle investigators are either already working with us or have indicated an interest in doing so.  It is our strong desire for a mentor relationship here with students to teach them about solar physics, the solar cycle, and the Earth’s magnetic field.  This is an important field of science that impacts our lives every day.  We take a lot of the science for granted that NASA, ESA, and other space agencies do in this area, but without them and the missions that have come after ISEE-3, we would have more difficulty in executing a modern civilization.

Soon we will do a short write up on the scientific objectives of the project.

Wrap Up

As I think you the reader can see, there is a heck of a lot going on, and not much time to get it done.  Thus, we are going to focus on getting the job done with a minimum of distractions and all but the most essential paperwork.  In a very short time, with the volunteers that have come forth, the corporate support we have gained, and the amazing outpouring of financial support in the crowd funding (yes we need every penny of the $125k to do this!), it is starting to look like we have a good shot of pulling this off.  The critical dates are in the third week of this month the transmitter is delivered and when we travel to Arecibo where we will attempt the first command session.

Please continue to help us reach our financial goal for the project.  We now have a really cool patch, designed by our incredible artist, Mark Maxwell.  The same for the images that are in our mission prints goodies now.  From being a completely insanely impossible thing to do on this short of a schedule, it is now merely insane.  Onward Sancho!

 

 

 

Posted in Space | 1 Comment

ISEE-3 Reboot Project Technical Update and Discussion

Today is April 30th, the 16th day after we started our Rockethub (http://rkthb.co/42228) project to raise $125k to allow us to attempt to contact, evaluate, and command the International Sun-Earth Explorer-3 (ISEE-3) spacecraft to fire its engines in such a way as to return it to Earth orbit after a swingby of the Moon on August 10th, 2014.  If you want to know all of the details, please read my previous posting on this subject here.  Today I want to give a discussion related to some of the technical issues and hurdles that we face in bring this spacecraft back into a stable Earth orbit.  I am leaving out the experiments for the time being as we have to focus on the engineering required before we get to that part.

First the Good News

The good news is that the team that did the ISEE-3/ICE mission really planned ahead.  In 1986 a series of maneuvers were done in three stages that targeted a flyby of the Moon at an altitude of 63 kilometers over the lunar surface on August 10, 2014 at around 8:30 PM UTC.  The first series of firings, with a total delta v of 1.51 meters/sec were done on February 27, 1986 when the spacecraft was near perihelion (closest point to the sun), to target the general vicinity of the Earth.  The second series of firings were done on April 7th 1986 to do a plane change to place the spacecraft on an intersect path to the Moon of ~38 meters/second.  The third   firing sequence was used to target the desired perilune target plane aim point (to the nearest kilometer of course!) with a 0.4 meter/sec burn that gave the lunar flyby distance of 63 kilometers.  Figures 1,2 and 3 show the resulting trajectory for these maneuvers:

Figure 7: ICE Trajectory from April 1986 Through August 2014

Figure 1: ICE Trajectory from April 1986 Through August 2014

Figure 1 shows the ICE/ISEE-3 trajectory relative to a fixed inertial frame from April of 1986 through the August 10 2014 lunar flyby.

Figure2: ICE/ISEE3 Lunar Flyby, August 10, 2014

Figure2: ICE/ISEE3 Lunar Flyby, August 10, 2014

This graphic, from Mike Loucks at Space Exploration Engineering, shows the flyby at ~15 km.  This was from just plugging the 1986 ephemeris into the modern Satellite Tool Kit application and running it.  It is shown here just as a general representation of the flyby.

Figure 3: ICE/ISEE3R Trajectory Post Lunar Flyby (1986 estimate)

Figure 3: ICE/ISEE3R Trajectory Post Lunar Flyby (1986 estimate)

This graphic, from the 1987 ICE Handbook shows the trajectory if everything went perfect, with no further maneuvering required.  For those who look at this with an engineering eye, the dark lines in the middle is the orbit of the moon around the earth.  There are a lot of out of plane orbits here that require course corrections to fix.  The lines to the side of the orbits are tick marks for days.  This is where the bad starts in that no matter how brilliant these guys were, they could not completely foresee the influences on the spacecraft from 1986 through now.  Also, the orbit chosen does not really, after the last ranging contact in 1999 and 2008, match and the orbit that the spacecraft is actually in, will result in the spacecraft not being permanently captured in Earth orbit.  This is shown by a new plot by David Dunham, who did a lot of the previous work and is now on our team:

The Bad

The above was the good part.  The targeting of the ISEE-3/ICE for its various comet rendezvous and return trajectory to the Earth is a tour deforce in understanding orbital dynamics and Dr. Farquhar and his team did a magnificent job.  However, there was only so much that computers of that era could do, and even if they had been perfect, in 28 years one has to expect that some course changes have to happen to put the vehicle on target to be captured into Earth orbit.  To recap from my previous post, figure 4 and 5 show what will happen if we do nothing vs what we want to do:

Figure 11: The Fate of ICE/ISEE-3 If We Fail

Figure 4: The Fate of ICE/ISEE-3R Without Course Adjustment

This is obviously not desirable.  The desired (baseline for our project) orbit is shown in Figure 5:

Figure 12: ISEE-3R Flight Path Baseline If We are Successful

Figure 5: ISEE-3R Flight Path Baseline If We are Successful

So, the goal of our project is to achieve the orbit that is in figure 5.  However, the trajectory of the spacecraft must be adjusted and it must be adjusted soon.  The current orbit is a nearly circular (eccentricity ~0.054) with an inclination relative to the ecliptic plane (the plane of the Earth and Sun projected on a two dimensional plane) of 0.06 degrees.  The perihelion (closest point to the sun) is 0.926 AU (astronomical unit or the average distance between the Earth and Sun of 146.6 million km) or 135.75 million kilometers [81.75 million miles]).  The aphelion (farthest point from the sun) is 1.033AU or 151.44 million km [91.196 million miles].  The orbital period is about 354 days.  So now, after 30 years the alignment of the orbit of ICE/ISEE-3 at aphelion lines up with the Earth’s orbit.  Figure six shows the decreasing range of the spacecraft at this time relative to the Earth:

Figure 10: Range from the Earth of the ICE/ISEE-3 Spacecraft

Figure 6: Range from the Earth of the ICE/ISEE-3 Spacecraft

As of May 1, 2014 ICE/ISEE-3 will be at .16 AU or 23.46 million km [14.1 million miles).  This is important as the closer the spacecraft gets to the Earth, the more energy it takes to nudge the course in the right direction for the right altitude lunar flyby that will result in capture into Earth orbit.  This can be seen in the figure 7  plot of delta v versus time done by our team member David Dunham:

Figure 7: dV required for Earth Capture Course Correction

Figure 7: dV required for Earth Capture Course Correction

There is approximately 150 meters/sec of dV left on the spacecraft in its hydrazine fuel system.  As can readily been seen the amount of energy needed as the range gets closer increases rapidly.  Table 1, also by David and a researcher at the University of Arizona, shows the problem in tabular form:

Table 1: ICE/ISEE Course Correction dV needed by 2014 Dates

Table 1: ICE/ISEE Course Correction dV needed by 2014 Dates

As you can readily see, as the spacecraft gets closer, the more energy needed to turn the spacecraft into the right trajectory.  Since there is only about 150 m/s of dV left (uncertain to at least 5%), then sometime very soon after July 1st it is game over.  However, it is worse than that in that we would like to have fuel left over in case of further maneuvers.  This includes more lunar flybys to trim the orbit or to have enough fuel to operate in Earth orbit for a while at the L1 point.  Thus time is of the essence!!!

The Ugly

So, we have a very hard deadline looming.  The problem has been that the people at NASA and the retirees that wanted to do this were unrealistic about the chances of obtaining funding.  With NASA getting ready to shut down operational missions there was no way that a 36 year old spacecraft coming back to the Earth would have any priority.  On April 10th of this year NASA headquarters in a teleconference with senior leadership in the Science Mission Directorate confirmed this to us and the people that wanted to do this mission with NASA funding.  Thus, four days later our effort was born.

The problem is that all of this should have started months ago!  However, just as we learned in our project to recover the Lunar Orbiter and Nimbus I, II, and III tapes from the 1960’s, modern technology can do things now that would have cost millions for NASA to do even ten years ago.  There are several technical problems that have to be solved and we are going to focus on them today and where we are at in the process.

Talking to and Commanding the Spacecraft

First we have to know what language the ICE/ISEE-3 spacecraft speaks in order to understand the formats of the data for commanding and telemetry.  Fortunately, the spacecraft does not have a computer!!  It has a sequencer that looks pretty much like a printer looks like to a computer.  In the days before NASA put computers on spacecraft the overhead for the operations team was huge as each individual command had to be developed on the ground, sequenced, and then sent to the spacecraft.  An acknowledgement, in the form of telemetry indicated that the command had been accepted and executed. What we have to do is to recreate that entire system.

Since this particular problem has a very strong ITAR flavor (International Traffic in Arms Regulation), we are doing this part in house with some help of some volunteers that have come on board (we will be paying them through the critical part of this activities).  We have to first figure out what the command structure is like for the engineering systems on the spacecraft (we are not bothering with the experiments yet) and figure out how to do that using modern computers.  We are making a lot of progress there but we at this time have insufficient documentation.  We are working with NASA to develop a Space Act Agreement that will give us access to this documentation.  We have enough to develop the command screens and we have an approximation of the data that we need to send.  However, there are (as of today) some gaps that we have to fill.  We continue to get documents from former engineers that worked the ISEE-3 mission so we are confident that we will have what we need.

Our initial tasks for commanding the spacecraft are as follows:

1. Turn Engineering Telemetry Mode on.

This mode commands the spacecraft to transmit telemetry regarding the functions of the onboard systems, including the attitude determination and control system, the power system, and the propulsion system.  This will help us evaluate the health of the spacecraft.

2. Two Way Ranging

This command will place the ranging transponder in a mode that allows ground stations to measure the distance (range) to the spacecraft and back.  We have to use multiple stations and use triangulation to get a better fix on the position.  We will have a team that takes this range data and then turns that into an updated position in space of the spacecraft.  Then the flight dynamics guys (Dr. Farquhar and David Dunham), will update the required energy to put the spacecraft into a proper Earth orbit.  Then….

3. Engine Firing

Back in the day, NASA had a program called ICEMAN that worked with a couple of programs called GMAT and GMAS.  These were all old mainframe programs that took the data and the developed a firing solution for the thrusters on the spacecraft.  This is an area where we don’t have the programming information for the old way so we going back to first principles of the spacecraft and re-deriving what we need to fire the thrusters.  We are working with Space Exploration Engineers (that did the NASA LADEE mission trajectories) to help us verify and validate what we are doing so that we can close in on a firing solution that looks good.

So this is what we are going to do.  To get there we have to solve the problem of how to read the telemetry, properly format commands, send the commands, and verify that the commands were executed.  We are developing the screens and the process to do this now.  More as this develops.

Modulator/Demodulator Development

Another big problem to solve is to redevelop the demodulator and modulator for talking to the spacecraft.  If this was even ten years ago it would be very difficult and expensive to do.  However, with the continuing advance in software technology for embedded computer systems we can now develop a demodulator in software and run it on some hardware that interfaces to the transmitters and receivers on the spacecraft.

We are working diligently with Ettus Research  on the development of the modulator/demodulator.  This basically formats the digital data that comes from the command generation software (also under development) and turns it into a waveform that the S Band transmitter can transmit to the spacecraft and that can then be demodulated on the spacecraft and sent to the command sequencer.  Note that there is no computer on the spacecraft and thus the commands are directly executed.  There are a lot of details related to this that we are glossing over for now in the interest of brevity.

When we command the spacecraft into engineering telemetry mode, the spacecraft transmits a signal back.  The signal is received, also on S-Band and then down converted to a signal that can turn the analog waveforms into bits that can then be received by our telemetry system software (also under development).  We will report back more on this later but suffice to say that the folks at Ettus are the best in the business and we will have this part of the system functioning soon.

Telemetry System

Another huge issue is redeveloping a telemetry display system.  When ISEE-3/ICE was operational this was comprised of several computers with old style displays for command line or numerical telemetry.  Today we have off the shelf software that can take the bits output from the demodulator and turn them into graphical displays and switch positions. This is a gross simplification in that we have to be able to take the incoming data, verify that the parity is right, then slice and dice all of the bits at their right addresses to push them to the right displays.  We are working with some people at National Instruments as well as working to develop our own displays for certain functions.  Following are the three spacecraft subsystems that we are focused on for the critical thruster firing.

  • Propulsion System
  • Attitude Determination and Control System
  • Power System

These are the screens that will be redeveloped and then when we command the spacecraft into engineering telemetry mode we will be able to verify its last known state, evaluate the condition of the spacecraft, and test a few of the subsystems without hitting the propulsion system and turning it on.  We are obviously going to be very careful about that one!

Ranging

Another and possibly our biggest issue is ranging.  What ranging is, is to send a signal to the spacecraft, have it returned, and measure the flight time to a very high order of accuracy (a few to tens of nanoseconds over a two and a half minute period).  We also really need to do this at multiple stations so that we can get differential flight times that can then be used to create a triangular baseline so that we can do a much better job at determining the spacecraft’s current position.  We know it to then tens to hundreds of kilometers level.  However, we need to know it much better than that in order to know exactly how much to fire the spacecraft thrusters for the lunar flyby at the right altitude.  NASA makes this look easy.  One time I read that the Voyager spacecraft’s position was accurate to a few kilometers out to Neptune!  This is an incredible feat of navigation that no one else in the world can do.  We are trying to replicate this at a much shorter range (about 23 million kilometers).  We know what to do, but we don’t have this one fully solved yet.

Management

The above are the biggies right now but another aspect is to keep all the parties in the loop, get our team of mostly volunteers working together, and to coordinate all the things that have to be done.  Part of the problem is that to read and absorb the documents takes time, and then to write formal documentation in a manner that should be done also takes time, and time is the one thing that we don’t have!  This is probably the biggest challenge overall and we have been bringing a couple of new team members that are local on board to help and start to get a handle on everything.  We also have to scare up resources, in people, in hardware, in time.  All of this takes time, time that is a very precious resource right now.

There is also the paperwork that we have to do with NASA.  We are currently working out a Space Act Agreement with NASA that will allow us to collaborate with them on a no exchange of funds basis.  This will give us access to the documents that we need and other things that as soon as the Space Act Agreement is signed, we will tell you about.  We will be sharing data about the spacecraft, its performance, and the condition and operation of the experiments with them in return for some things from them.

Current Status

We have a lot of balls in the air right now, from ordering transmitters, to coordinating ground stations and engineers, to getting documents from NASA, getting the Space Act Agreement signed, to getting the team organized and working as a team.  All of this while the crowd funding effort is still going forward and without being able to access the money from it!!

More next Monday, but for all of the Crowd Funding supporters, we are working very diligently to meet our deadline for the firing of the thrusters and so far we don’t see anything that makes this impossible!  Just darn difficult!

I would like to thank the following companies people at this time for their support.

Companies

Arecibo Observatory.

Ettus Research

National Instruments

Ball Aerospace

KinetiX

Persons

Keith Cowing, Media, and all around gadfly of making things happen and my Co-lead on this project.

Tim Reyes, Flight Operations

Cameron Woodman, Flight Operations

Karl-Marx Wagner, Transmitters and communications

Patrick Barthelow, Transmitter and antenna support.

My Wife Nikki!

Mike Loucks, Space Exploration Engineering

Mark Maxwell, Space artist extraordinary!

Leonard Garcia, overall great guy doing this on his own time at NASA

Dr. Robert Farquhar, former ISEE-3 Mission Design lead

David Dunham, Flight Dynamics

I also want to thank my internal team of Austin Epps and Marco Colleluori who are working some of the very difficult problems with telemetry.

and many others!!

 

 

 

 

Posted in Space | 1 Comment