Space Book Review: Peter Thiel’s “From 0 to 1

Peter Thiel and Ben Master's From 0 to 1

Peter Thiel and Blake Master’s From 0 to 1

From 0 to 1

Its not often that a book about the general development of companies has such direct applicability to space, but Peter Thiel’s book nails it in one.  It is a remarkable insight into the mind of a forward thinking venture capitalist and should be used as a general guide for those of us seeking to build a space business.

Since space is intrinsically a technology business, Thiel’s thesis on the subject (the book has far more of a feel of a thesis than a simple exposition) is that our general societal progress has stymied because of the lack of broad technological progress in the last few decades.  His statement defining from 0 to 1 is:

The single word for vertical, 0 to 1 progress is technology. The rapid progress of information technology in recent decades has made Silicon Valley the capital of “technology” in general. But there is no reason why technology should be limited to computers. Properly understood, any new and better way of doing things is technology.

Thiel’s observation is that the last few decades have been marked by broad horizontal progress (globalization) from 1 to n, where the advances of China, India, and other rapidly industrializing and emerging economies are built around basically replicating the American model. This is what he calls 1 to n progress.  While building skyscrapers, superhighways, and mega-airports may be progress, it is not innovation.  While it is obvious that information technology has advanced in the last 43 years (his horizon of interest is 1971 to the present), we have not had the broad advance in technology necessary to assure the continued prosperity and advance of a global civilization.

Why 0 to 1 is Important

Thiel states:

In a world of scarce resources, globalization without new technology is unsustainable.

This is absolutely correct and something that many of us, beginning with Gerard K. O’Neil in the 1970′s have discussed as the counter, to the Limits to Growth mindset that really defines the beginning of Thiel’s modern horizon.  Indeed the very premise of the Limits to Growth mindset is that technology cannot save us.  As former Vice President Gore said in his book, “Earth in the Balance”:

The environmental crisis is a case in point: many refuse to take it seriously simply because they have supreme confidence in our ability to cope with any challenge by defining it, gatherings reams of information about it, breaking it down into manageable parts, and finally solving it. But how can we possibly hope to accomplish such a task? The amount of information and exformation—about the crisis is now so overwhelming that conventional approaches to problem-solving simply won’t work.

So in a nutshell Thiel and those of us who are technological optimists are the Yin to the technological pessimists Yang of the political class.  The technological pessimists cannot fathom that technology can solve the problems that we have today but of course they (the political class) can solve them using their favorite tool, politics and the coercive power of government.  This is the defining tension of our age in the west, and thus the question is how and what do we change things in order to show that technological progress is indeed possible, desirable, is our solution to today’s problems, and to foster it?

 Mining Cordiner for Context

What Thiel writes about in postscript is exactly what the CEO and Chairman of the Board of General Electric, Ralph Cordiner spoke about (later written in a book with the chapter title Competitive Free Enterprise in Space) in prescript in 1960 and gets to the core of both the problem and the opportunity that we have today:

….But we must recognize that there are growth tendencies in these government agencies that could overexpand under the pressures of the space program, unless proper safeguards are established. As we step up our activities on the space frontier, many companies, universities, and individual citizens will become increasingly dependent on the political whims and necessities of the Federal government. And if that drift continues without check, the United States may find itself becoming the very kind of society that it is· struggling against-a regimented society whose people and institutions are dominated by a central government……

Ironically it was the state directed technology development of the space program that sewed the seeds of our lack of progress since the end of the Apollo era! There is no question that what Cordiner presented as what he called “an alarmist position” in 1960 has come to pass.  Not only has Cordiner’s alarmist position come to pass, it is now preached by some as the ultimate solution to our problems today.  Again from Albert Gore in “Earth in the Balance” (page 305) for his third strategic goal:

The third strategic goal should be a comprehensive and ubiquitous change in the economic “rules of the road” by which we measure the impact of our decisions on the environment. We must establish—by global agreement—a system of economic accounting that assigns appropriate values to the ecological consequences of both routine choices in the marketplace by individuals and companies and larger, macroeconomic choices by nations.

All guided by the government, the same people that have decided that technology cannot provide our solutions!

So, we have come to the point where regimentation is extolled as the only solution to securing our future.  You need only chart when the U.S. stopped fostering technology development and traded it for our current path, and it coincides almost exactly with the timing of Thiel’s modern age of reduced technological development.  Cordiner had an answer to this, and it brings us back to Thiel:

Therefore it is my view that national economic and military progress will be faster and more solid, and the freedoms we cherish will be preserved, if competitive private enterprise does just as much of the nation’s scientific and technical work as possible-and government provides the legal and policy framework to stimulate outstanding technical performance.

Thiel’s Thesis

Peter Thiel of course is a venture capitalist, and a leader on some of the most successful entities in Silicon Valley.  In some ways Silicon Valley is the ultimate enclave of technological optimism in the world today.  How else would you have a university (unaccredited) called Singularity University, where they preach the absolute opposite meme of the pessimists; that we are on the cusp of a convergence of technologies that will usher in a new golden age of plenty and prosperity for all mankind.  Unfortunately this is treated almost like the invisible hand of inevitability.  Thiel addresses this:

New technology has never been an automatic feature of history.!!!

This is the essence of the second part of his thesis in that we have traded the definite future of technological progress that was deeply embedded in American culture of the post war world to what he calls the indefinite future.  Basically that we have substituted definite progress toward the future with some indefinite hope in the future.  We now think that things are going to get better, we just believe it, think of it as automatic, but as Thiel writes, automatic progress has never been part of history.  As a general trend this is how he states it:

Everyone learned to treat the future as fundamentally indefinite, and to dismiss as extremist anyone with plans big enough to be measured in years instead of quarters. Globalization replaced technology as the hope for the future.

In the space business we see this every day with the infinite number of NASA trade studies leading to indefinite plans for space exploration, that somehow never quite make it to reality.

It is an embedded assumption in parallel with Cordiner by Thiel that the way forward is via private enterprise, startups, and technological development.  His favorite ones are ones with a secret, a new way of looking at things, and with plans to do more than write some code, build a bit of market an then flip it to the big boys for a cash out.  He introduces an interesting term that goes to back to the opening of his book where he talks about the his contrarian question.  He states that he asks this of people that come to him wanting investment.

His contrarian question is this:

What is it that you believe, that few others believe?

His answer is:

My own answer to the contrarian question is that most people think the future of the world will be defined by globalization, but the truth is that technology matters more.

In this he is absolutely correct as no amount of globalization or indefinite trust in the future can solve the fundamental problems of energy, natural resources, pollution, and the desires of even the most humble in the world to have a standard of living that Americans have enjoyed.

Thankfully Thiel’s solution is not to ask for more government but to take the Cordiner approach and put his faith and financial resources in the ingenuity of entrepreneurs, and their answers to the contrarian question.  His seven Socratic questions that he asks in order to judge which companies he thinks have the greatest potential to bring the technological breakthroughs that are necessary for our future:

The seven questions.

  1. The Engineering Question

Can you create breakthrough technology instead of incremental improvements?

  1. The Timing Question

Is now the right time to start your particular business?

  1. The Monopoly Question

Are you starting with a big share of a small market?

  1. The People Question

Do you have the right team?

  1. The Distribution Question

Do you have a way to not just create but deliver your product?

  1. The Durability Question

Will your market position be defensible 10 and 20 years into the future?

  1. The Secret Question

Have you identified a unique opportunity that others don’t see?

These are great questions that are geared toward the breakthrough companies.  To me it is hopeful as it is all geared toward developing enterprises dedicated to definite progress in technology.

Read this Book!

To me his book is much more of a reconfirmation of the research, readings, and writings that many of us have done over the past several years related to space development, with the added bonus of being able to get into the head of a leading venture capital visionary.  Thiel is part of the Paypal Mafia that also has produced Elon Musk, who all of us in the space business know is a stellar embodiment of what Thiel writes about.  What I hope is that this book will become a necessary book on the shelves of all of our bay area VC’s and that it helps to create a mindset of investing in technology.

Thiel postulates something that has already been cast as controversial in the media, which is his belief in the power of monopoly.  Early in the book he goes into the fact that only a very few companies in any VC portfolio are stellar performers, mostly companies that are defacto monopolies, something the companies themselves try to hide lest they attract the attention and ire of the government.

The monopoly companies, like Apple, Google, Amazon, and others have sufficient profits to be able to experiment and innovate on a large scale, and that companies that are hamstrung in their profits by excessive competition do not have the resources to continue innovating and thus their growth halts or they become focused on other (fake) metrics of success like increasing the stock price by large share buybacks.

I would posit that the VC business follows a similar trajectory as the companies that they invest in.   A Kauffman foundation report from 2012 goes into this in detail in that there are only a very few Venture Capital organizations that themselves have the outstanding performance that gives them the resources to innovate in their investments and take risks in 0 to 1 companies.  As one of my investors once said, it is always easier to write the second check, and it is always easier to follow what has already been done.  Thus it is the super successful ones that have the resources to experiment, if they have the mindset to do so.

Thiel’s Vision

There is no doubt that Thiel, and others like Marc Andresson of Andresson Horowitz and Steve Jurvetson of Draper Fisher Jurvetson have followed this path.  They are among the most successful in the business here in Silicon Valley.  Most of them have invested in SpaceX, Tesla, and other technology ventures.  Some of them have long investment horizons, like SpaceX and it is only those VC organizations with sufficient stature and track record that can make these investments.

As someone who has pushed radical technology development in spacecraft development for a long time, it is a very hopeful thing to see someone of Thiel’s stature write something that will be read by his peers here in Silicon Valley.  There are many 0 to 1 space companies out here that have been starved for resources during the era of 1 to n investing.  Elon Musk and SpaceX has helped lead the way for space, its time to take the advice proffered by Thiel and work to make more of them!

Posted in Space | Tagged , , , , , , , , , | 6 Comments

ISEE-3 in Safe Mode

Since the lunar flyby on August 10th the ISEE-3 Reboot team has continued to work with Google Creative labs to bring to full fruition the spacecraftforall.com website to provide real time data from ISEE-3. We have been working with the various dishes that have supported us until now, including Arecibo, Bochum, the SETI Institute, Morehead State and others. We now have a problem.

The ground stations listening to ISEE-3 have not been able to obtain a signal since Tuesday the 16th. Arecibo, Morehead, Bochum, SETI, as well as the Usuda 64 meter dish in Japan and the Algonquin 45 meter dish in Canada have all pointed at the spacecraft with no positive results. So, at this time we are assuming that the spacecraft has gone into safe mode.

What This Means

Safe mode on ISEE three can basically only occur from one problem, loss of power. Before the lunar flyby ISEE-3 orbited closer to the sun than the Earth. This resulted in a very good power profile for the spacecraft. However, as seen in the figure 1 here, since the flyby the spacecraft is traveling much farther away from the sun than it has been before:

Figure 1: ISEE-3 Post Flyby Trajectory (Courtesy of Mike Loucks www.see.com)

Figure 1: ISEE-3 Post Flyby Trajectory (Courtesy of Mike Loucks http://www.see.com)

We have not had many opportunities to get data from the spacecraft since the flyby as the antenna configuration has also been much worse from an attitude perspective. Also, we no longer have propellant to change the attitude of the spacecraft to improve this configuration. We can change the antenna pointing a bit but the first time we tried it, it did not work.

Next Steps

When ISEE-3 goes into safe mode it turns off all of the experiments and it turns off both transmitters and waits for help. Due to some uncertainty in the trajectory this may end up being a bit more of a problem than otherwise. We are working now to put together the commands to turn the transponders back on and obtain engineering telemetry. The last telemetry we have looked ok, but the spacecraft is still traveling farther away from the sun, and thus it is probable that last week the voltage on the power bus dropped enough to trigger the safe mode event. There is no functioning battery on the spacecraft now as it failed in 1981.

So, stay tuned for more information.

Dennis Wingo

ISEE-3 Project Co-Lead

Posted in Space | Tagged , , , | Leave a comment

ISEE-3 Post Lunar Flyby Status and Modification of Mission Goals

Mission Status

Communication with the ISEE-3 satellite was successfully re-established with the goal of commanding the satellite to change its trajectory with the goal of putting it into a libration point orbit that would allow it to resume its original mission goals of collecting data for solar physics research. The trajectory change goal unfortunately could not be completed due to the failure of the onboard thrusters. This failure was apparently the result of the loss of nitrogen pressurant in the Hydrazine fuel system.

This inability to change the spacecraft’s orbit rules out the original reboot mission goals which would have provided long-term data collection from the satellite instrumentation package using modest antennas. After the orbit change attempt, the ISEE-3 Reboot Team powered on the instrumentation package and began data collection from the instruments to assess their current physical status and usefulness for any ongoing scientific mission.

We are now redefining our mission goals to obtain the maximum scientific usefulness of ISEE-3 in its new interplanetary orbit. Figure 1 shows the flyby orbit and the long-term sun centered (heliocentric) orbit.

Figure 1: ISEE-3 Trajectory Through Aug 2016 (image courtesy www.see.com)

Figure 1: ISEE-3 Trajectory Through Aug 2016 (image courtesy http://www.see.com)

This rendering shows the projected position of ISEE-3 through August of 2016. The flyby of the Moon increased the orbital energy of the spacecraft. The original orbit, from 1983 through August 10th 2014 was interior to the Earth’s orbit around the sun. The orbit of the Earth is the light blue ellipse. The red line is the new orbit, but with respect to the Earth. This is why there is the small red ellipse in the first year near the Earth/Sun L5 location. That the large and small ellipse represents is the eccentricity of the orbit. The top of the large ellipse represents the maximum distance from the sun, which is about 1.2 astronomical units or about 180,000 kilometers.

The ISEE-3 spacecraft is traveling slower than the Earth during this time. The top of the small ellipse is the closet point to the sun, which touches the orbit of the Earth, which is about 146 million kilometers from the sun. Figure 2 shows the orbit for the next 15 years, until the return of the spacecraft for another encounter in seven years at about 30 million km and then back to Earth orbit in 15 years in August of 2029.

Figure 2: ISEE-3 Trajectory through August of 2029

Figure 2: ISEE-3 Trajectory through August of 2029

Besides being a beautiful example of how orbits look different (depending on how the data is displayed), figure 2 provides a lot of information to our team. Due to the malfunction of the propulsion system the flyby of the Moon on August 10th 2014 at an altitude of between 14,000-17,000 km resulted in ISEE-3 gaining enough energy to shift its orbit around the sun by a total of about 50,000 km. The flyby also changed the plane of the spacecraft’s orbit, from nearly co-planer with the ecliptic (the zero plane of the Earth’s orbit with respect to the sun) to a maximum of about 3.5 million kilometers above the plane of the ecliptic. Figure 3 shows the near term magnitude of this change:

Figure 3: Near Term Change in ISEE-3 Distance Above Ecliptic Plane

Figure 3: Near Term Change in ISEE-3 Distance Above Ecliptic Plane

This change in Z altitude creates a near term problem for our operations. Figure 4 shows the nature of the problem:

Figure 4: ISEE-3 Medium Gain Antenna Beam Angle

Figure 4: ISEE-3 Medium Gain Antenna Beam Angle

There are two main antennas on the ISEE-3 spacecraft. The Medium Gain Antenna (MGA) has a gain of 7.5 dbi maximum and an intermediate gain antenna with a maximum gain of -0.5 dbi. The spin axis of the spacecraft is not exactly normal to the ecliptic. It is currently about 1.8 degrees in the wrong direction, but this is accounted for in the graph in figure 4. Figure 5 shows the relevant part of the pattern of the medium gain antenna:

Figure 5: MGA Beam Pattern vs Angle to the Earth (late August 2014)

Figure 5: MGA Beam Pattern vs Angle to the Earth (late August 2014)

It turns out that while at the present time, the MGA primary beam is not pointed at the Earth, one of the secondary beams (first minor lobe), is pointed close enough. What this means to us is that the MGA is still the best bet for our continued reception of the signals for ISEE-3 for the duration of its new interplanetary sojourn. However, in recent days this has caused us problems in receiving data from our ground stations, a problem that will solve itself soon, that is until the spacecraft gets too far away from us, which will happen by the end of the year as shown in figure 6:

Figure 6: ISEE-3 Distance from the Earth Through The End of 2014

Figure 6: ISEE-3 Distance from the Earth Through The End of 2014

It is not that easy to tell from the graph in figure 6, but the distance from the Earth of ISEE-3 is increasing faster as it moves away from the Earth, than it did on the way in. This is because the spacecraft is farther from the Earth now in heliocentric distance, than on the way in, about twice as far. This is why that we will not be getting much data after early 2015 until it gets close to the Earth again in 2021.

Spacecraft Health

As far as the health of the spacecraft goes, it is pretty good. Our biggest concern for the long-term health of ISEE-3 is related to temperature. The spacecraft will be an average of about 50 million km farther from the sun than the previous orbit, and this will have an effect on temperatures internal to ISEE-3. Even though there is no pressure in the hydrazine propulsion system, we still have to be concerned about the hydrazine freezing. If the lines or the hydrazine in the tanks freeze, then thaw in several months when the spacecraft gets closer to the sun again, then this will present a danger. Hydrazine eats spacecraft, and leaking hydrazine is not a good thing to have. We can leave the propulsion system heaters on but that has an impact on our next largest concern, which is power

As the spacecraft moves farther away from the Earth, sunlight gets weaker and thus the amount of power generated by the solar arrays is less. We were astonished when we found out prior to the flyby that the solar arrays were putting out considerably more than current computer modeling of array degradation suggested.   This is a major mission enabler now that the spacecraft is outside of the Earth’s orbit.

Marco Collelouri (our controls and AOCS engineer) ran a calculation that indicates that our minimum power will still be about 86 watts at 1.2 astronomical units, the aphelion of the ISEE-3 orbit. This does mean that we will have to turn off one of the two transponders, but if we do that we can then power the most critical experiments, which begins to lead us to the science part of this status report.

One last note about spacecraft health is that though we do see some degradation in the electronics on the spacecraft, including the communications system, we feel that there is every chance that in seven years, and even in fifteen years, ISEE-3 will still be a functional spacecraft, a remarkable testament to the people that built it!

The ISEE-3 Interplanetary Science Mission

Though the planned mission goals cannot be met the reboot team feels ISEE-3 still has a useful scientific role to play. In its current orbital configuration the spacecraft will be able to provide data from a number of instruments as outlined in the previous section. Ground station tracking and data collection is the most critical of the factors. With increasing distance of the satellite from Earth we face the challenge of receiving a weakening signal. The ability to command the satellite to turn on/off or configure instruments must also be considered and the window for determining which experiments to turn on and support is very short if we are to realize the maximum scientific benefit.

The ISEE-3 Reboot team has reviewed data from the spacecraft obtained since repowering the instrumentation package to determine the validity of the data and health of the instruments onboard. In the case of a number of the instruments the original Principal Investigators have been contacted and sent samples of the data for review and we are currently waiting for their analysis. A list of the onboard experimental packages and the current instrument status can be found in table 1:

Table 1: Experiments and Their Proposed Status (Conditional)

Table 1: Experiments and Their Proposed Status (Conditional)

The ISEE-3 Reboot team feels that at this point time is of the essence in redefining mission goals and configuring the satellite and propose the following plan to be implemented immediately.

  1. Redefine the mission as a solar physics data acquisition of indefinite length. The new mission length will be measured in weeks and months and will be ultimately defined by loss of telemetry. The limited power budget of the satellite and the slower telemetry data rates will require us to select only a subset of the onboard instruments for the ongoing mission with more instruments being shut down as lower data rates render them less useful. An initial list of instruments and their proposed status is contained in Table 1. Initially, the cosmic ray instruments will be powered off and further instruments will be powered off as the data rate slows. We are soliciting the input of the ISEE-3 science community for their recommendations.

2) Solidify current tracking station commitments and solicit new resources (dishes) to extend data collection time. The amount of useful data will be determined by the signal strength which determines the data transfer rate we can use. The signal strength will be a function of the gain of the ground stations we use.  Currently the ground stations we have do not provide 24 hour coverage and the satellite cannot buffer data so there will be data lost regardless of satellite distance. Since ISEE-3 has passed the closest point to earth and is now on an outward bound trajectory the soliciting of new ground station tracking with higher gain antennas and longer coverage is imperative to the success of the new mission goals.

Public Outreach

The spacecraftforall.com website was set up in collaboration with Google Creative Labs, a kind of think tank for the company. They have poured considerable resources into this, partially as a way to tout the graphics and capabilities of the Chrome browser, but they have been incredibly supportive of our science goals. This commitment continues. We are working toward getting the full real time data display system working within the next several weeks. The purpose of this is to promote STEM education and the dissemination of science data to the public as well. Not only is the real time data going to be available for use by citizen scientists, the data will also be archived and preserved. The data will also be provided to the science community for conversion to their preserved science data formats for their own purposes.

It should be clear that one of the core ideas behind the ISEE-3 Reboot Project has been citizen science. What does that term mean? For us it is the ability for people to see our data, and for us to provide explanations and context for the data. You might ask, don’t scientists do this already? Well, yes but often it gets lost in translation. Solar physics and the study of it is exceptionally important to our modern civilization. In the recent solar cycle (cycle 24), the majority of the community was considerably mistaken in their predictions related to solar activity. It is of major importance to our civilization to understand why we don’t understand the sun and to build support for investments to increase our level of understanding.

Recent articles have reached the public consciousness about how a solar flare could basically fry all of our technology and how one missed the Earth in 2012. It is a known fact that one such flare happened in 1859 and caused havoc on even our primitive telegraph system at the time. Anyone that says that we fully understand the Sun, does not fully understand the sun or the state of the science today. In an era of competing financial priorities in government, communicating this to the public is important for all of us as without modern electrical systems, we would all be in a very difficult position, extremely quickly.

What’s Next

So expect to see more from us in the very near future. After the rush of the flyby we took a bit of a break but we are moving forward with the new “ISEE-3 Interplanetary Citizen Science Mission”. As soon as we have the real time display systems and reliable ground station communications support we will start putting more data out to be seen. We cannot sufficiently express our appreciation to the ground stations that have helped us make this a reality. Arecibo, our national treasure, in Puerto Rico. The Bochum Radio Observatory in Germany, our reliable friends and the ground station that first got us interested in ISEE-3. Morehead state in Kentucky has provided crucial support for transmitting to the bird of late. And the SETI Institute has been listening in as well. We are working to increase this list, but without these folks and their dishes, this would not have happened…

 

Posted in Space | Tagged , , , , , , , | 2 Comments

We Are Borg: Crowdsourced Engineering and the Collective Mind of the Internet

We are Now Living in a Science Fiction World

In the science fiction universe of Star Trek, set several hundred years in the future, when we are a spacefaring civilization, humanity encounters a species called the Borg.  The Borg are a conglomeration of species who are assimilated into a collective mind numbering in the hundreds of billions.  All of the borg are connected to each other through a communications link that allows each of them to share each others thoughts, though in a manner that erases individuality.

This week, with the call that our ISEE-3 reboot team put out to the internet for help in debugging our propulsion system problem, I have come to realize that a significant portion of humanity has reached a Borg like state, one where the internet has become a collective mind for communications and knowledge sharing.  We still have our individuality, we can still decouple at will from the collective mind, but in a way that few philosophers or technologists have envisioned, we are connected in a way never before thought possible.  The implications are staggering, and here is how our little ISEE-3 project is an example of the operation of the collective mind.

ISEE-3 Trajectory Correction Maneuver Attempt

On July 8th and 9th the ISEE-3 reboot team attempted to fire the radial thrusters on the spacecraft.  Figure 1 shows the configuration of the thrusters:

Figure 2: The ISEE-3 Spacecraft Details

Figure 1: The ISEE-3 Spacecraft Details, including propulsion [1]

 The thruster firing on the 8th was a follow up to the successful spin up maneuver that was performed on July 2nd, 2014.  The engine firing on both the 8th and the 9th was unsuccessful in producing the thruster needed to change the course of the spacecraft to what is necessary for the lunar flyby on August 10th.  After the largely unsuccessful Trajectory Correction Maneuver (TCM), on the 8th, our team decided on a strategy to both use alternate thrusters and to troubleshoot the propulsion system.  On the 9th the TCM was also largely unsuccessful (a few pulses out but no substantial change in course).  After a bit of team depression that the mission was over, a failure investigation was started.  Following are the results of that failure investigation, and our plan for recovery.

First TCM Attempt July 8th

After the successful July 2nd spin up maneuver we had high hopes for the TCM.  We had learned quite a bit about the operation of the spacecraft and the spin up maneuver made us and our NASA partners confident in the next steps.  Indeed it was the successful spin up maneuver that allowed NASA to give us permission for the TCM.  However, as events would unfold, this confidence was to be misplaced.

As those who have read of our exploits know, the ISEE-3 is a spinning spacecraft.  It spins quite rapidly, at 19.75 (19.787 right now) RPM or about one rotation every 3.04 seconds.  The thrusters for the propulsion system that allows for an in plane change in velocity are located on panels 1 and 9 (redundant) of the spacecraft.  This is shown in figure 2:

Figure 2: ISEE-3 Thruster Locations and Designations

Figure 2: ISEE-3 Thruster Locations and Designations [2]

 Figure 3 shows the schematic of the propulsion system from the same AIAA paper from 1979:

Figure 3: ISEE-3 Propulsion System Schematic

Figure 3: ISEE-3 Propulsion System Schematic

The spacecraft has two completely redundant fuel, latch valve, and thruster systems.  The thrusters for dV maneuvers are located on the exterior of the spacecraft around the circumference as shown in figure 2.  These are the “radial dV” thrusters.  For the maneuver on the 8th we used thruster’s F and N, which are connected to the HPS 1 tank system.  Everything proceeded normally as we followed the worksheets provided by our documentation.  Things went much better from a commanding perspective as we were able to leave the transmitter and power amplifier in transmit mode at Arecibo while we received telemetry pipelined to us over the internet from the radio telescope at Bochum Germany, which is operated by the AMSAT DL team there.

When we fired the thrusters for the TCM maneuver, initially things looked great, but soon we saw a fall off in thruster performance as monitored by an accelerometer designed for this purpose.  This is shown in figure 4:

Figure 4: Telemetry Profile for July 8th TCM

Figure 4: Telemetry Profile for July 8th TCM

The upper graph is of the accelerometer output during the three firings.  The middle graph is the plot of the Fine Sun Sensor (FSS), which gives orientation.  The bottom graph is the telemetry value indicating whether or not the Latch Valve opened or not.   At this scale it is hard to see the actual acceleration values and they are in counts, not m/sec squared.  It is quite clear the fall off in thrust in the first maneuver.  We now know that the latch valve was not opened (we did not then).  Later, and before we did the second attempt, we see the sun angle decrease.  This so far has not been explained adequately.

The third attempt is the most interesting.  We started wondering about the latch valve and then sent the command for it to open again.  It opened.  The difference is that we did not have the +28 volts on the first time as none of this is in the procedures that we currently have available.  After we commanded the latch valve to open with + 28 volts on we saw telemetry confirmation opening.  Figure five shows what we saw in detail regarding the FSS sensors:

Figure 5: Fine Sun Sensor Angle Variations

Figure 5: Fine Sun Sensor Angle Variations

While this is probably hard to read on a screen but figure six is toward the end here when we first got confirmation that the latch valve between the tanks and the propellant lines was opened:

Figure 6: Latch Valve Opening Vibrations for FSS and Accelerometers

Figure 6: Latch Valve Opening Vibrations for FSS and Accelerometers

The latch valve opens at 9200 seconds and there is an immediate effect on the accelerometer and the FSS sensor.  The data rate is 8 hz for the accelerometer and 1/8th hz for the FSS sensor so there is a lot of aliasing but the influence is clear to see.  This is normally in the form of vibrations on the spacecraft that gives the appearance of a change in FSS pointing, though not in the longer term.  The end result is that we never got the thrust that we expected on July 8th, whether or not we had indications of latch valve open.

Evaluation of July 8th Attempt and Plan for the 9th

Our evaluation of what happened that day is different than what we know now but it informed how we operated on the ninth and thus is instructive to deciphering the nature of the overall problem.  Our evaluation that day was that we thought that the hydrazine tanks in fuel system 1 (on the left in figure 3) was depleted of fuel and pressure.  Our pressure transducers indicated that there was no pressure in fuel system 1 and only minimal (about 4 psi) in fuel system 2.  We also were very unsure of the telemetry indicator for the operation of the latch valves.  Thus the plan was to first attempt to do the maneuver with the fuel system 1 (but use latch valve C rather than A) and thrusters E and M on the other side of the spacecraft.  This would have the effect of using a different set of fuel lines but with the same fuel system tanks.  Then we would then repeat the test using fuel system 2 and the same thrusters (E,M).  If this did not work then we would open latch valve B on fuel system two and repeat with thrusters (E,M).  If that did not work then the final test would be to use latch valve D with thrusters (F,N).

The Attempt on the 9th

The results on the 9th were pretty much the same as on the 8th.  We initially made the same mistake by commanding latch valve C when the 28 volts was not on.  The results were basically as they were the day before.  Figure 7 shows the result of the first firing:

Figure 7: Accelerometer and Other Telemetry Indicators July 9th

Figure 7: Accelerometer and Other Telemetry Indicators July 9th

As you can see, the uppermost blue line is the accelerometer output.  It looks pretty much like the trace in figure 4.  It is opposite in sign because we used thrusters E,M, and sector 556 (the other side of the spacecraft) to start pulsing.  The pale flesh covered lines are where we pulsed (21-30) the latch valve.  There were other thruster firings done, but none of them with any effect.  This ended the pass for July 9th.

Failure Investigation

After a short bout of team depression, we got started on our failure investigation.  In learning how to be an engineer, it is just as instructive to study failure, and many times more so, than to study success.  I have read virtually every failure investigation in the last 50 years for in space failures, from the early Ranger days to GEO comsats, and Shuttle failures.  There is a common thread that not too many people write about, which is that these failure investigations almost always include people from outside of whatever organization was building or flying the system that failed, and that these guys were normally the best in the business.  Another common denominator of such failure investigations it that they take time and lots of money.  Our team has neither.

We seriously thought that there were no options going forward, but in order to understand what happened, and to see what we could learn, we decided to dig into the telemetry and see what happened and to see if we could determine the failure mechanism.  No one on our team is an experienced hydrazine expert.  My own expertise is more in communications, avionics, and power systems, with a good bit of experience in ion propulsion.  Marco Collelouri, our controls and AOCS engineer is very smart, but without a lot of experience.  Thus I felt, after receiving a few emails from people who offered suggestions on what might have happened, decided to throw the problem out to the world.  I was astonished at the response.

The Collective…..

On the 10th we threw out a few questions related to some suggestions that had been sent to us by some of our global distributed network of supporters unsolicited.  Keith put this on NASA Watch and http://www.spacecollege.org.  We immediately started getting responses.  While, as one might expect, many of them were uninformed, though enthusiastic, some were from the most qualified professionals in the world.  I am not going to name names at this time and without their permission but literally, the very top tier of experts started weighing in.

We sent them telemetry, and information on the spacecraft.  Fortunately there is a lot of information out there on ISEE-3, especially from the AIAA (see our references at the end of the article).  With this, and with our telemetry that is also public, we started to build a picture of a set of plausible failure modes and the state of the system.  I must stress that I have had multiple interactions with different groups of experts, some from industry, some from government, and some international.  I have not shared a lot of the information from one group to the other in order to get their unbiased responses, until a consensus started to emerge.  We also were able to be put in touch with (again, through a senior industry engineer), with some a retired person from TRW, the company that built the propulsion system in the first place.  Even though this person did not personally build the one from ISEE-3 he did know the general design and had extensive experience with hydrazine propulsion systems.

The Conclusions of the Ad Hoc ISEE-3 Hydrazine Propulsion System Failure Study

After just a few days of consultations with these groups, and a consensus started to form some common elements stand out.  Here is what we know about the state of the HPS system.  Please refer back to all previous figures as this explanation unfolds.

Integrity of the Propulsion System Finding 1

It is quite clear from the telemetry that both fuel system 1 and 2 downstream of the latch valves was still fully pressurized.  If this had not been the case, we would have not had a successful spin up maneuver, nor would we have had the initial thrust from both fuel systems on the 8th and the 9th.  This means that the following hardware is working:

1. All thrusters have their seals intact and the thrusters provided impulse, showing that the catalyst beds are also intact (or at least mostly so).

2. The propellant lines downstream of the latch valves are also intact and were fully pressurized.

3. Temperatures in the system are high, in some cases, in excess of what is desirable, though not dangerously so.  Temperatures in the propellant tanks are well within bounds.

Here is what we don’t know.

1. How much pressure is in the tanks (the options are failed sensors or depleted tanks).

2. The true status of the latch valves.

Procedures, Finding 2

We made a procedural mistake.  That mistake was turning the latch valves on without + 28 volts applied.  In our defense this was not in the AOCS procedure worksheet, though looking back into the old Mission Operations Plan, checking their status was part of the procedure.  We thought that when we did not have an indication of operation, that the telemetry was dead, which is also the case in the communications system.  Sometimes we imply the state of the spacecraft when telemetry is dead, by looking at whether the command that we sent was executed.  So the latch valve status we thought was dead.  It turns out this was incorrect.  We corrected this in the middle of firing on the 9th, but it did not effect the propulsion system operation as can be seen in figure 7 and right at the right edge where we initiated propulsion but got no response.  After we turned on the +28 volts and commanded the latch valves, they did successfully response (commands 21-30 as shown in figure 7).  You can see that the latch valves vibrated the spacecraft in the accelerometer data.  However, we did not have time before the end of that pass to fully investigate the issue.

Postulations

So the above is what we know.  Now we had to use our own knowledge and that of our outside experts to eliminate, one by one, possible failures.

Propellant Tanks and Upstream Propellant Lines

In examining the telemetry we know that the HPS-1 and 2 fuel systems downstream of the latch valve were fully pressurized.  This eliminates loss of fuel and nitrogen this way.  We also know, and it was our rookie mistake, that nitrogen does not dissolve in Hydrazine, more than just by fractional amounts.  This is why it is used as a pressurizing gas.  We also know that this is a blow down system (see reference 2), which means that the nitrogen gas is mixed with the hydrazine and not in a separate bladder or pressure tanks.  This is a simpler system and it eliminates failure modes.  Thus there is only three ways that we could have lost fuel and pressurizant from the system.

The first way is through the latch valves.  We already know that there was pressure downstream of the latch valves and thus that is eliminated.

The second way is that the fuel and pressurizant could have been lost through either the fill and drain valves or the fill and vent valves.  There are two reasons this is not plausible.  The first is that as far as our experts know, neither of these types of valves have ever failed in flight as they are physically capped before launch.  Also, if they had failed, it is even less likely that they would fail over ten years after launch.  Even less likely than that would those valves failed in both systems.

The third way would be a failure in the propellant lines upstream of the latch valves.  While this has happened in the past.  It is unlikely to have happened in both fuel systems, and if it had, there would be serious consequences for the spacecraft as Hydrazine eats spacecraft wiring and other hardware.  If that had happened it is likely that the spacecraft would have been lost.  Also, this would have had an effect on the attitude of the spacecraft, which we did not see when we first tabulated the telemetry after recovery.

Thus the conclusion of virtually all of our experts is that it is highly unlikely that fuel and propellant has been lost in the system.  This brings us to the next postulation.

Latch Valves

The latch valves on this spacecraft were built by Hydraulic Research (see reference 2).  These valves were popular and used on several spacecraft.  In researching the company, we found documents related to pressure testing and leak testing of the valves.  The valves do preferentially allow diffusion of nitrogen through the seals (we found this data on the NASA technical reports server).  However, since the lines downstream were pressurized, it is unlikely that this happened.  For this mechanism to operate would take further diffusion of the nitrogen through the thruster valves which is also unlikely with full pressure there.

What we did find, and I can’t be too specific here as this information is not in the public domain, is that there are different seal materials and that the type that most probably flew on this spacecraft is subject to temperature based swelling.  Since we also see very high temperatures, in excess of 62 degrees C on the upper propellant lines, which are in contact with the valve, we now have a plausible culprit for our problem.

This possibility is enhanced by our own mistake in how we operated the latch valves prior to figuring out that we had to have the +28 volts on to actuate the valve. This is somewhat mitigated by the fact that later we did actuate the valves and did see physical vibration of the spacecraft from our opening and shutting the valve (commands 21-30 in figure 7 indicate that something happened).  There is a case to be made that the valves did not actually open and we did not fully investigate this during our short Arecibo pass last week.

Temperature

The upper propellant lines near the latch valves is above their specified operating temperatures.  We found that the line heaters have been on since the last propulsion maneuver in 1987, or over 27 years.  While the temperatures are not dangerously high, the long term storage of hydrazine at elevated temperatures can cause the slow decomposition of hydrazine into ammonia and nitrogen and then eventually into nitrogen and hydrogen.  Did this happen?  We have no idea as there is no pressure transducers in the lines.  However, one of our outside experts worked on the Magellan to Venus mission and gas evolution in the propellant lines was seen there.

Since this was a mission to Venus, a planet that gets twice the radiative heat that the Earth gets, and since ISEE-3 came considerably closer to the sun every 354 days for 27 years, it is very plausible that this happened.  If it did, we would still get thrust out as we saw, but there would be gas in the propellant lines.  We did see large drops in temperature in the hot upper propellant lines and large increases in the lower propellant lines.  Rapid swings in temperature could be from gas and or hot ammonia (NH3 a decomposition product of Hydrazine) in the system.

Conclusion

Thus we have a plausible mechanism for our propulsion failures on the 8th and the 9th with the latch valves.  High temperatures expanding the seal material could have either impeded the flow, or have precluded the latch valve from opening even with the microswitch indicated to telemetry that the valve was open.  We also have high temperatures possibly evolving gas, causing a large gas bubble in the propellant lines Is this the case?  There is a way to test both cases.

Corrective Action

There is a pretty good possibility now that we have pressure and or fuel in the tanks but that it is not getting to the propellant lines and out the thrusters.  We are going to of course turn the +28 volts on this time!  We will also open both valves on one of the fuel systems, the primary and redundant.  We will also heat the tanks to see if we can see a rapid increase in temperature.  If we see a rapid rise, that would indicate no fuel in the tanks (testing for all eventualities).  There are several things we will do to test out and try propulsion to bleed all the gas out of the lines.

What we could see would be not much activity and then toward the end of the pulses from the thrusters we could see propellant flow, temperature increase, and thrust!

Cross your fingers.  We will have a pass on July 16th at Arecibo, so we will soon find out what the outcome is.

The Collective Consciousness of the InterNet

There was a great article on space.io9.com related to distributed engineering and our project, and how the people from the net came together to help us.  I first saw the term distributed engineering in the late 1980′s from the amateur radio community.  It began through using ham radio to do this, then it migrated to email before the advent of web browsers, and then to the web.  What happened with our call for help goes far beyond that as the distributed engineering meme begins with a pre organized group of people that collaborate in geographically disparate locations toward a common engineering goal.  Before our call for help last week, I knew maybe one or two of the experts that came in and helped us.  This goes well beyond distributed engineering to a collective consciousness.  I often characterize the internet as the global extension of my brain, with vast stores of knowledge that the brain organizes through the interface of the browser.

In the beginning of the net we used this to research information.  With the rise of the ubiquitous internet among the professional class and beyond in the world, we now have something never before seen on this scale in the history of mankind, a near instantaneous way to not only research information, but to rapidly organize humans to do “things”.  We now have crowd funded efforts that bring people together of like interests to fund interesting projects like ours.  We have crowdsourced collaboration in the arts, sciences, and engineering.  There is a lot of talk about singularities in the technology world, and for the most part they are marketing myths from my experience.  However, and this is what I leave the reader to ponder, we have reached a threshold where vast numbers of people can work together in a near real time manner to solve problems and do good and interesting (or evil) things.  One wonders where this will go….

For us it was great!

[1] Farquhar, R, Muhonen, D, Church, L; Trajectories and Orbital Maneuvers for the ISEE-3/ICE Comet Mission.  AIAA-84-1976, AIAA/AAS Astrodynamics Conference, Seattle, WA, August 20-22, 1984

[2] Curtis, M.S., Description and Performance of the International Sun Earth Explorer-3 Hydrazine Propulsion Subsystem, AIAA/SAE/ASME 15th Joint Propulsion Conference, Las Vegas, NV, June 18-20, 1979

Posted in Space | Tagged , , , , , , , , , | 8 Comments

ISEE-3 Reboot Update, Ranging The Spacecraft

Folks

It has been a bit since my last update.  Tomorrow I hope to have a more thorough one for you all but this one here should be of interest, especially for the more technically inclined.  We had a pass from the NASA Deep Space Network (DSN) that was to be used to improve our knowledge of the position of ISEE-3 in space.  This is not an easy task, though NASA’s DSN team makes it look easy.   Every measurement, in on way or another, depends on accurately measuring the time it takes for a signal to either come from the spacecraft (passive mode), or a two way round trip signal, (ranging mode) or the most accurate means, called coherent ranging where the spacecraft locks to a signal from the ground, and matches its phase coherently.  The ISEE-3 spacecraft, even thought it is of a 1975 design vintage has this ability.  However, things did not go as planned last week.  Here is what I posted on our http://www.spacecollege.org site and this was also sent to our NASA sponsors.

_________________________________________

The DSN pass on June 18th that went from 1:45 to 2:45 Pacific Daylight time was not a success. Here is a recap of the pass activity.  The DSN pass started at 1:45 pm PDT.  Here is a graphic of the pass through a very nice DSN Now web app:

Figure 1: NASA DSN Now Web Interface Showing ISEE-3 (ICE) Pass Via DSS 24 Goldstone

Figure 1: NASA DSN Now Web Interface Showing ISEE-3 (ICE) Pass Via DSS 24 Goldstone

The pass began with a +/- 3 KHz sweep across frequencies representing the input frequency of transponder A (2090.66) MHz + the Doppler offset + an additional 11.25 KHz that came from our most recent command session. The additional offset is due to thermal and or aging issues with the spacecraft transponder. The sweep is done with a carrier only, no modulation, to get the receiver on the spacecraft to lock to the DSN transmitted signal. The output of transponder A will start to vary in a 240/221 relationship when the carrier is locked. Then ranging can occur. The sweep was unsuccessful in establishing a coherent lock. The sweep rate was 60 Hz/sec. This conforms to the procedure used in 1985 by the DSN for the spacecraft during the ICE comet encounter.[1]

A second sweep with a bandwidth of +/-6 KHz was initiated at the same sweep rate. This was also unsuccessful. It should have been successful as a total sweep of 12 KHz encompasses more than the offsets that we have successfully used to command the spacecraft. After this did not work, the DSN ops team then did a sweep at +/- 20 KHz with the center frequency set at our Doppler + 11.25 KHz offset. This surely should have worked in obtaining a lock, but it did not. Due to the time involved to do the sweep this exhausted our available time on the Goldstone dish, and thus we completed the pass without any indication of lock from the spacecraft.

Troubleshooting the Pass

Link Margin

The configuration of the Goldstone system was using the DSS-24 34 meter dish, running 10 kilowatts of power. Table 1 gives the estimated link margin:

Table 1: Estimated Link Margin for Ranging with Carrier at Goldstone for 6-19-14:

Table 1: Estimated Link Margin for Ranging with Carrier at Goldstone for 6-19-14:

Looking at the link margin, it is evident that the link was not the problem.

Failed Subsystem or Procedural Issue?

There are two possibilities for why the ranging failed, when we know for certain that transponder A is functional for commanding the spacecraft. The first is that the ranging function on the transponder has failed. The second is that a procedural error regarding commanding the communications system into coherent mode is at fault. Thus by eliminating the DSN link as an issue and looking at what could be the problem otherwise we have the beginning of a fault tree established for an investigation.

Failed Subsystem

The possibility that the ranging function has failed, or was not functional was taken into account by our team. The spacecraft has two transponders, A, and B, and both are capable of ranging. There are some issues with lower gain on transponder B’s antenna that make it less desirable but with the robust margin from Goldstone seen in figure 2 this is not a problem. On our 6-15-14 Arecibo pass we placed transponder B into coherent ranging mode as well as transponder A. However, due to time constraints the DSN did not have time to attempt ranging to the B transponder. Also, due to lack of licensing to transmit to the B transponder from Arecibo, we have been unable to verify the functionality of that receiver. Thus one possibility would be to range to transponder B for our next DSN pass on 06-25-14. However, this does not address the issue of what the problem might be for transponder A.

Our success with transponder A has not been 100% in sending commands to it. The receiver input frequency seems to be drifting toward a higher frequency over time. We do not know at this time whether that is a random drift or one that is predictable. We overcome that when we do our commanding by sweeping the expected frequency plus Doppler plus an offset when we send commands. Since our first commanding session the offset frequency has appeared to drift to higher frequencies each time. This is not completely unexpected as we found test data that indicated a positive curve for transponder A. If this were the issue the +/- 20 KHz sweep would have locked the receiver. While this does not verify that the ranging mode is not functional, it does narrow the possibilities.

Operational Procedure

With the incomplete documentation at our disposal it is not unlikely that we have issues in operational procedure that preclude the coherent function of transponder A from working. On top of this problem is the legal issue that we can’t just use transponder B because we have been unable to obtain a license to transmit on the frequencies for that system from Arecibo. We have no schematics or vendor documentation on the transponder. We do have test data and we know the specifications and requirements that it had to meet. Worst of all is that as far as we can determine from the digital subcom that provides verification of communications operation, it is not functional, through either the Data Handling Unit (DHU) A or the redundant DHU B. This is not that surprising in that the DHU’s have absorbed more than five times the radiation that they were designed to handle. The only way that we have to verify a function for the communications system is to command that function and then see if it works in the intended way.

We actually were able to verify that the coherent mode is working and to validate the operational issue. Figure 2 shows our Eureka moment for coherent ranging:

Figure 2: Post Doppler Correction Spectrum Trace During Commanding for Coherency

Figure 2: Post Doppler Correction Spectrum Trace During Commanding for Coherency

The above graphic, produced by Phil Perillat from Arecibo, shows that our command bits for coherency did go up and did push the transponder into coherent mode, shown by the 18 kHz frequency jump by the downlink.  There was a 1.6 second lag afterwards and we send the ranging command again, which stabilized the carrier at the offset frequency.  After the end of that command, which at 256 bits/sec took 2.4 seconds, and after an additional 1.6 seconds, the downlink shifts back to its base frequency.  This was our first solid indication, on June 15th that the ranging transponder coherent mode was working.

We had tried ranging through the transponder on the previous pass we had on June 9th.  We did not think that this had been successful but after looking at some of the analyses by Phil Perillat we noticed that the ranging mode had indeed worked on that day as well.  Figure 3 shows this:

Figure 3: Ranging Tone Spectrum With Frequency Offset, Transponder A

Figure 3: Ranging Tone Spectrum With Frequency Offset, Transponder A

In looking at this spectrum we were able to completely verify that the coherent ranging function with tone had worked.  The spectrum above shows this.  We verified this by looking at a document with test data on it from the spacecraft acceptance test.  This is shown in figure 4:

[2]

Figure 4: 20 KHz Ranging Tone In the Time Domain on Oscilloscope from 1978

Figure 4: 20 KHz Ranging Tone In the Time Domain on Oscilloscope from 1978

The difference between the signal in figure 3 and figure 4 is that the first is shown in the frequency domain, and the second in the time domain.  With the correlation between the acceptance test data in figure 4and the waterfall plot in figure 3 we pondered why the DSN pass did not work.

Even though we have incomplete information, we still have a lot.  In our discussion related to the issue we recalled that somewhere in a document it was stated that the spacecraft had to be commanded into coherent mode and that if the carrier dropped for more than three seconds it would automatically drop out of that mode.  After searching this document was found and here is what it said:

Figure 5: Documentary Evidence of Procedure for Coherent Mode Operation of Transponder A

Figure 5: Documentary Evidence of Procedure for Coherent Mode Operation of Transponder A

This provides verification of why we were able to get into coherent mode and the DSN was not able to also do so.

Final Verification of Procedural Issue

Friday June 20th we were going to do the propulsion system test and spin up maneuver.  However, one of our pass/fail criterion was real time telemetry and reliable commanding.  Neither of these criterion were met and thus we cancelled that activity early in the pass.  This gave the team time to focus on operationally testing transponder A’s receive system and to retest the coherent mode to determine whether or not we could command that action and record the results.  This we were able to do and figure 6 is our final evidence of Coherent mode operation:

Figure 6: Coherent Mode Operation Confirmed 6-22-14

Figure 6: Coherent Mode Operation Confirmed 6-22-14

What we were doing, as shown in the waterfall plot above, is that we were sending dummy commands to the spacecraft in order to get the command counter to increment.  This was a test of the command counter but we also send a coherent mode command.  The difference between figure 2 and figure 6 is that we changed our procedure slightly and sent commands one after another without allowing the carrier to drop, which maintained receiver lock on the spacecraft.  This can be seen in the shifting of the waterfall plot and at the end the three seconds of drift before the carrier snaps back to the baseline frequency.

Findings

  1. Coherent Mode Operation

Coherent mode is operating in a manner consistent with the original acceptance test report.

  1. DSN Failure to Go into Coherent Mode

The failure of the DSN frequency sweep to lock the spacecraft into coherent mode is due to the lack of a command to go into coherent mode before the sweep.  The sweep method must be coupled with a command for coherency for coherent mode to be entered.

Solution

The ISEE-3 team can provide to the DSN a packaged coherency command that can be broadcast to the spacecraft as the frequency sweep happens.  At this time our best estimate for frequency offset is Doppler +10-15 KHz.  This is due to aging of the transponder on the spacecraft.  After lock is achieved ranging tones should be sent without dropping the carrier.  Discussion with the DSN should be centered around how our command can be integrated with DSN operations.

[1] DSN_Operations_Plan_1985.pdf (page 5-8)

[2] 1978-02_compatibility_test_report_isee-c_flight_model (page 162)

Posted in Space | 1 Comment

ISEE-3 Reboot Project Update: BULLSEYE! and More

Project Update June 1, 2014

We have returned from our foray (probably not the last) at Arecibo.  It was a wonderful trip, the people down there are marvelous, and Arecibo is the crown jewel of American radio astronomy.  Did you know that the Chinese are building a bigger one, modeled after ours?  The NSF keeps trying to cut their budget but this is truly an amazing asset to the American people and the world and it should be supported!!  I want to thank Mike Nolan, Phil Perillat, Dana Whitlow, Victor Negron, and all the great crew down there.

Technical Progress, Contact Made!

Commanding The Spacecraft

As we put out in our project very brief note on Friday, we have successfully contacted the bird!  At the time of the contact we had Morehead State University Space Science Center’s 21 meter dish, the 20 meter dish at Bochum Radio Observatory in Germany, and the SETI  Allen Array were all listening.  This was not without problems.  The spacecraft has two transponders, which are oddly enough called transponder A and Transponder B. Transponder B is normally the engineering telemetry transponder and transponder A is the ranging transponder.  The final state of the spacecraft at last contact was to have both of the transponders transmitters active and that is what people around the world have been tracking.  However, the spacecraft is set up with a lot of redundancy so you can use either transponder A or B to send telemetry or to range.

We tried several times to command the spacecraft’s B transponder at 2041.9479 MHz into the mode where it normally sends engineering telemetry, which is our first task.  It did not work.  We tried several variations of the proper commands and we tried several operational approaches such as scanning across the receive transponder band to make sure that there was not some offset that we did not know about or that the receive frequency had drifted over the years.  Nothing worked.   Then we tried the same process on transponder A and BINGO, telemetry! Well not really telemetry itself, but modulation from the output of the telemetry system.  The initial command was just to turn engineering telemetry on at 512 bits/second.  This was successful.  Figure one, generated by Achim Volhardt from AMSAT DL and Bochum, shows the modeled spectrum for a 512 bits/sec telemetry rate overlaid on the received signal, which is what we commanded:

Figure 1: Simulated (Solid Line) vs Received Spectrum from ISEE-3/ICE A Transponder

Figure 1: Simulated (Solid Line) vs Received Spectrum from ISEE-3/ICE A Transponder

The second image, figure 2, shows a waterfall plot that clearly shows the sidebands of the expected signal, as recorded from our Ettus Research USRP N 210 receiver as processed through GNU radio by Balint Seeber:

Figure 2: Waterfall Plot of Signal Showing Sidebands

Figure 2: Waterfall Plot of Signal Showing Sidebands

This provided our initial verification that we indeed had successfully commanded the spacecraft into engineering telemetry mode.  We later also, working through the A transponder’s receiver, we commanded through the B transponder’s command decoder to output engineering telemetry through transponder B’s transmitter.  Thus we have verified so far the following systems on the spacecraft.

1. Transponder A receiver

2. Transponder A’s Command Decoder and Data Handling Unit

3. Transponder B’s Command Decoder and Data Handling Unit

We reviewed some of our documents and found that neither of the ISEE-3/ICE receivers had met their specification in testing.  The specification was for -120 dbm sensitivity.  However, we found that receiver A was tested at about -114 dbm, and Receiver B at -111 dbm.  A difference of 6 and 9 db gain respectively.  Working the numbers backward with our 400 watt power amplifier and the gain of the Arecibo dish, we found that we were marginally ok with the A receiver and probably slightly short with the A receiver, calculating in radiated power vs db Eb/No.  Normally NASA and the Deep Space Network (DSN) uses transmitters in the tens of kilowatts range.  Since we could neither acquire one that big in the time that we had, or could afford to buy one, this had driven our decision to use Arecibo rather than a larger power amplifier at a smaller dish.

Our biggest disappointment was that we then tried to command the spacecraft into 64 bits/second mode, which was a mode that is much more complicated to set up, we did not get working successfully during the limited time that the spacecraft is visible from Arecibo.  We need to do this so that the smaller dishes at Morehead State and Bochum will have a positive signal margin so that we can record several hours of data.

THEN WE HAD AN EARTHQUAKE.  As many know who follow our social media, on Thursday after our end to end systems test we had an earthquake.  I was on the central part of the Arecibo dish, 450 feet in the air with Dana and Anthony, another engineer at the site when this happened.  We had just chatted about how observations could be affected by vibrations in the dome structure as it translates during an observation and then that happened!  The azimuth tracking system, which is the curved structure on the underside of the top part of the dish, was slewing while we were there as well as the dome.  We were sitting in a safe area when everything started shaking.  I was doing a video at the time but stopped it to hold on during the shaking!

Demodulating and Decoding the Received Signal

The first miracle was to command the spacecraft.  The second is to understand what it says.  Figure 3 shows a scope plot of the resulting data and clock plotted in time:

Figure 3: Clock and Data Recovery from Demodulator

Figure 3: Clock and Data Recovery from Demodulator

If you look at the bottom of this figure you can see 1′s an zero’s, the bits that come out of this process.  Now our guys are super exited about this and yesterday morning (Saturday) Austin Epps sent out an email based on the first set of bits that Balint got out of the demodulator:

I searched for the synch bits ’11111010111100110011010000000000′ per the SIRD document.  That string was found in two locations…starting at bit 575 and again at bit 2623.  Note that the two locations are 2048 bits apart, exactly as expected.

We got our synchronization bits, which provides the framing indicator for a frame of data, out of the demodulator!  Not to be outdone, (actually everyone is collaborating and working beautifully together), our new volunteer, a very old hand at demodulating satellite data, Phil Karn, jumped on the data that Balint provided and we have the following fully processed first frame of data!

Gentlemen, feast your eyes on this:

7c 02 02 02 02 02 02 02 7c 02 02 02 02 02 02 02
7c 00 02 00 00 f2 00 00 7c 02 02 79 a0 00 00 00
7c 00 02 33 c8 02 4d 02 7c 4b 02 76 00 00 00 00
7c 02 02 53 01 02 39 02 7c 44 02 00 b1 49 00 00
7c 00 02 5a 00 19 5c 64 7c 4b 02 0e a0 00 00 00
7c 0e 02 4b 47 63 91 1d 7c 42 02 4d 36 00 00 00
7c 45 02 44 4e 8a 89 02 7c ce 02 50 a4 00 00 00
7c 48 02 32 4b b5 d2 ad 7c 33 02 12 fc 81 9f be

This is my very first Viterbi-decoded frame of ISSE-3 telemetry,
extracted from the first frame of the recording I received this morning.

Note that it ends with the 12 fc 81 9f be sequence, the 3-byte encoder
dump sequence 12fc81 followed by the 2-byte sync sequence 9fbe.

I forced the Viterbi decoder to end in the state 819fbe so those last
three bytes could not have been anything else, regardless of what was
received. HOWEVER, the 12 fc decoded just before that actually came from
the received symbol stream, and since that matches the values given in
the documentation this is a strong indication of correct decoding.

As another indication of correct decoding, I re-encoded the
Viterbi-decoded data and compared the encoded symbols to the raw symbols
received from the spacecraft. There were no errors. None.

I think we have it. Now I just need to polish this off so it’s useful.

Fantastic work!  Phil later processed our first day’s data dump from the spacecraft and we received 49 full frames of data at a bit rate of 512 bits/second.  Until the very end there were no errors on the downlink, and only then when the spacecraft was going beyond the horizon for Arecibo.  These are the milestones related to commanding and receiving data from the spacecraft that have been achieved:

1. Successful commanding multiple times of ISEE-3/ICE

2. Received engineering telemetry from both data multiplexing units on the spacecraft.

3. Successful demodulation on the ground of the received data, through the output of bits.

4. Verification of good data at 512 bits/sec, including frame synchronization, correct number of bits/frame, and with no errors, showing a very strong 30+ db link margin through Arecibo.

These milestones alone would be praiseworthy but there is more!

UPDATE: Sunday night.  Phil Karn sent me an email to say that he has now processed over a 1000 frames of telemetry from the ISEE-3 spacecraft.  We are going to have a LOT of fun decoding and displaying the data this week!

BULLSEYE

The Trajectory Problem

One of the major problems that we have, that has to be solved, is to update the range to the spacecraft so that its position, velocity, and trajectory into the Earth Moon system can be properly plotted so that we can then plot a course, and fire the engines for a  maneuver to target a lunar flyby at the proper altitude (around 50 km) on August 10, 2014.  The last trajectory solution that we have from the DSN is from 2001 and it is this one that is provided by NASA JPL in its Horizons prediction program that everyone has been using.

The problem is that this solution has been shown to be inaccurate when we are using the extremely narrow beam width (~2 arc minutes), at Arecibo.  When plotted out to the approximate distance of the spacecraft, 2 arc minutes is only about 16,800 km wide, meaning that if the spacecraft is more than about half this distance (assuming that we point exactly at the right location), then the spacecraft falls outside of this beam and the signal vanishes.  This is except for the fact that the Arecibo telescope is so powerful that the minor lobes of the main beam are still more powerful than most smaller telescopes and that was how we were receiving the spacecraft in the first few days at Arecibo.

Then, Phil Perillat who handles the hard problems in operating the telescope at Arecibo, performed a search to lock on the main beam of the spacecraft.  He was able to do this, and our signal level was over 50 db above the noise, a very strong signal for transponder A, and a bit less, around 45 db for transponder B.  Phil continued to do this almost every day when we could get time on the extremely busy telescope.

There is a huge side benefit to this technique.  In the scientific community for asteroid research radar and optical sighting of Near Earth Objects is used as the only means to determine orbits.  This community has gotten extremely good at this method.  The spacecraft engineering community uses coherent transponders (which is what transponder A is on ISEE-3/ICE) to lock on and do a two way ranging to allow the engineers to calculate a good orbit ephemeris for a spacecraft.

Amazing Accuracy of the 1986 Trajectory

Using the data from Phil’s daily targeting of the spacecraft, Mike Loucks at Space Exploration Engineering, along with the folks at Applied Defense, and then further verified by volunteers from APL working on their own time, we have narrowed down the location of the spacecraft.  It turns out that it is far closer to the Moon than the JPL Horizons propagated trajectory, and very near being on the course intended for it by the ICE trajectory team in 1986!  This is shown in figure 4:

Figure 4: Newly Plotted ISEE-3/ICE Course Compared to Horizons and JPL 2001 Trajectory

Figure 4: Newly Plotted ISEE-3/ICE Course Compared to Horizons and JPL 2001 Trajectory

The Blue Circle is the orbit of the Moon with the Moon’s location show at the right side of the circle (August 10, 2014 location).  The Yellow Horizons trajectory is shown intersecting the Moon’s orbit but no where near the Moon on that date.  The white line was a re-propigation of the JPL Horizons orbit in Systems Tool Kit (Satellite Tool Kit).  The dark blue trajectory is the intended trajectory of the ICE navigation team in 1986.  The red/green trajectory is the plotted trajectory based on Phil Perillat’s pointing data from the Arecibo telescope!

Consider this, the spacecraft has completed almost 27 orbits of the sun since the last trajectory maneuver. That is 24.87 billion kilometers.  They are off course by less than 30,000 km.  I can’t even come up with an analogy to how darn good that is!!  That is almost 1 part in ten million accuracy!  We need to confirm this with a DSN ranging, but if this holds, the fuel needed to accomplish the trajectory change is only about 5.8 meters/sec, or less than 10% of what we thought last week!

We truly stand on the shoulders of steely eyed missile men giants…

What is Next

If we can maneuver the spacecraft by June 17th we get the very small delta V number for the maneuver above.  However, this starts to climb rapidly as the spacecraft gets closer  to the moon.  Also we cannot at this time rule out a lunar impact.  It is imperative that we get a ranging pass as soon as possible.  We also need time to not only evaluate the health of the spacecraft, but to test the systems, the catalyst bed heaters for the propulsion system, the valve heaters, analyze the rest of the propulsion, power, and attitude control system as rapidly as possible.  This will be a lot of commanding so we have to move into high gear next week.

This is a very fluid situation and we have made amazing progress, thanks to the support of those who believed in us in our crowd funding and the support of our NASA sponsors at NASA Ames and NASA headquarters.  We also want to thank the members of the original ISEE-3/ICE engineering and science team.  Without their marvelous efforts, and without the documents that they saved and stored lovingly for decades in their homes this would have been a far more difficult task!  We also appreciate their looks over our shoulders and we will be relying on them to do this more as we work to successfully command the spacecraft to fire its engines.  More to come soon!!

 

 

Posted in Space | 5 Comments

ISEE-3 Reboot: Updates from the Front at Arecibo….

Sunday May 25 2014

We have had a very active week since we arrived and got started work.  We arrived on Saturday the 17th and checked into the visiting scientists quarters near the big dish.  And then we saw it…..

The Arecibo dish is too big to be taken in in one photo so here are a couple to give some sense of scale.

Figure 1: Central Focus of the Arecibo Dish

Figure 1: Central Focus of the Arecibo Dish

See the ring in the middle?  This picture below is what it looks like with people on it looking down to the center of the ring.

Figure 2: Arecibo Dish Rotating Section

Figure 2: Arecibo Dish Rotating Section

Those are people down in the center there.  That gives you a slight bit of sense of the size of the place.

Mike Nolan and the folks at the dish are just the best people imaginable.  There are several scientists in residence at this time running experiments and there are several who do things remotely.  One scientist showed us some amazing pictures of a celestial body that are amazing (keeping it quiet until the findings are published) that were obtained with the radar at the site.  The power amplifier that we had shipped to the site puts out about 400 watts on S-Band.  The one on site here, running at half power, is a megawatt.  We can’t use that one as it is very precisely tuned to its frequency and can’t be changed. The days kinda run together down here so the days are estimates for what we did starting but here are our day to day accomplishments while down here.

Saturday-Monday

Since the power amplifier from Germany was a bit late, we started out by conferring with the staff here on the best approach to be able to use the dish.  Mike Nolan, Phil Perillat, and Dana Whitlow, Victor Negron, and the group are all outstanding technical and scientific people and helped us figure out the ins and outs of using the dish down here.  The dish requires near constant maintenance and so it is near the end of these times, after the work is finished, that we have opportunities to receive signals from the spacecraft.  We brought our own transceivers down here, the Ettus Research USRP 210′s which use the GNU radio Software Defined Radio (SDR) open standards.  Operated by the right people, such as Ettus engineer Balint Seeber (Ettus graciously allowed Balint to come down here with us).

Tuesday

We began by replicating the basic reception of the spacecraft using the dish and using their equipment in parallel with ours.  The first day, which I think was Tuesday, Phil saw that the signal from the spacecraft was weaker than before.  We had some time to work and Balint did some configuring of the USRP receiver.  However, not much more than that happened and we were somewhat concerned about the signal.  Was the spacecraft failing as it approached Earth?  So we asked the folks at Morehead State to monitor the spacecraft on Wednesday in parallel with Arecibo in order to see what they could see.

Wednesday

On Wednesday we got the great news that the power amplifier had been shipped from Germany, next day Fedex!  The team worked during the day writing software that would be used for receive testing at the dish.  When the pass started, we were very surprised to see that the signal from the spacecraft was basically missing.  We consulted with Morehead and found that they were receiving the signal stronger than ever.  We also were receiving signal from the 6 meter dish at SETI in California.  Phil Perillat then did some investigating and found the signal, but at a very weak signal strength on a side lobe of the dish.  (Lobes are part of the spread of received energy from the dish).  Some calculations indicated that if Morehead could hear the signal in its main lobe but we could not, then there was a divergence between the position of the spacecraft and the ephemeris file provided by the NASA JPL Horizons server.  We did not have enough time on Wednesday to find the spacecraft on the main lobe.

Thursday

Huzzah!  We received the tracking number for the power amplifier and instead of waiting for it to be delivered around 5:00 pm, we and the film crew with us took off to the west end of the island to pick up the amplifier at Fedex.  We were greeted by Wilson at the office (if you don’t know who or what Wilson is, watch the Tom Hanks movie about the crash of a Fedex plane):

Figure 3: Wilson Greets us at Fedex in Puerto Rico

Figure 3: Wilson Greets us at Fedex in Puerto Rico

We picked it up early and got back to Arecibo before 1:30 in the afternoon, when it started raining which halts all activities in the center focus of the dish for safety reasons.  However, we did remove it and its power supply from their crates and set up the system to begin testing at the control center on Thursday evening.

We also were able to get some quality time in on the system for a receive test.  The ever diligent Phil Perillat initiated a search for the spacecraft and was able to lock onto it, but with an offset of -0.47 degrees in Right Ascension (RA) and 0.08 degrees in Declination (Dec).  After some calculations this indicated that the spacecraft was approximately 250,000 km away from its expected location.  With this information the signal was solidly locked to the spacecraft with approximately 55 db of signal to noise on one frequency and approximately 45 db on the other.  The difference is due to polarization differences between the two signals principally.  We were also able to lock onto the signal using the Ettus USRP 210 radios of ours.  This is shown in figure 4:

Figure 4: FFT and Waterfall Plot of ISEE-3/ICE Signal.

Figure 4: FFT and Waterfall Plot of ISEE-3/ICE Signal.

We had some very happy campers in the control room!:

Figure 4: The ISEE-3 Reboot Crew After Signal Confirmation on USRP Radio

Figure 5: The ISEE-3 Reboot Crew After Signal Confirmation on USRP Radio

With the successful reception of the signal on the USRP radio we confirmed at least the entire receive chain and processing through our laptops the signals in GNU radio.  Balint did a marvelous job there and we had help from John Malsbury who had been with us the previous two days.

Friday

On Friday the team from Arecibo lifted the Dirk Fischer Electronics power amplifier from the bottom of the dish up to the central focus and installed in the transmitter room. Figure 6 shows the power amplifier and its power supplies installed and ready to go:

Figure 6: Dirk Fischer Electronics Power Amplifier Installed in the Arecibo Transmitter Room

Figure 6: Dirk Fischer Electronics Power Amplifier Installed in the Arecibo Transmitter Room (in the Dome)

During the previous few days discussions had been had with Dana Whitlow and Victor Negron about how to get our transmitter signal from the USRP up to the power amp.  There are several fiber optic cables (as stray RF is a big no no there) that could carry the signal from the control room to the dome.  Then there was a coax to go from there to the power amp.  This created a lot of signal loss so preamplifiers were installed on top of the power amp that would allow us to boost the signal to the proper level to drive the transmitter to full power.  All of the transmit path from the control room to the preamps were tested and shown to be operational.

Final Testing Late Friday

With everything installed we then decided to do a final end to end test of the system.  We sent signals up to the preamps that then allowed Dana Whitlow adjust them and to monitor the output of the power amp.  After some trial and error this was accomplished and all four of the amplifiers that made up the combined power amp were tested and it was confirmed that they operated at full power while remaining relatively cool in the transmitter room.  This is the same room where the big megawatt klystrons are located.  In this test we found a couple of configuration issues to deal with which were successfully fixed.  Thus by the end of the day, we had a completely successful end to end test of our transmission and reception system.

Discussion and Agenda for Next Week

Originally we were going to head home this past Friday.  However, with the inevitable delays that are part of doing real work, we reached our first major success milestone here at Arecibo on Friday.  Thus we have extended our stay for another week in order to actually transmit to and attempt to command the spacecraft.  The end to end test revealed a couple of operational issues that we have to deal with such as that with a dish with this fine of a main transmit lobe, that we may have to offset the dish to “lead” the spacecraft so that it will be where it should be when it receives the signal.  This is doubly true as we now know that the spacecraft is off course compared to the existing ephemeris.

The error in position has just elevated the concern level greatly.  We know approximately what the offset error is from the existing ephemeris but we don’t have enough information yet to plot a new course and generate a new ephemeris file.  This has become extremely important as there is a solid statistical chance that the spacecraft could impact the moon or even by off course enough to threaten other spacecraft in Earth orbit.  We are working with Mike Loucks of Space Exploration Engineering (SEE) our trajectory guy on this issue.  An east coast company, Applied Defense (ADS) has also offered their help and engineering support to derive a new ephemeris from our new position reports.  ADS and SEE did the trajectories for NASA’s just completed LADEE mission.  It won’t be perfect but it will be an improvement over what we have now.

This is a necessary improvement as we are far enough off course that if we fired the engines now for a course correction, it could easily make matters worse.  So we have just had the level of concern increase dramatically!  We will know more when we are able to get time on the dish again, maybe Monday at best but definitely Tuesday so that we can have multiple plot points along an arc to give to Applied Defense for them to work with creating a new orbit plot.

We hope that if all of the stars align (some of the stars are paperwork), that we can attempt transmission to the spacecraft to command it into engineering mode on Tuesday.  It is imperative for us to do this and to attempt to range to the spacecraft to further refine the orbit.  Reading the engineering telemetry, debugging the demodulator and the telemetry system software are our biggest tasks this week.  We will have Morehead State and the SETI Allen Array (or some subset), listening to see if the telemetry is being transmitted.  This will allow several hours of reception instead of just the 2.5 hours here at Arecibo.

This is all for now.  Watch our @ISEE3Reboot twitter feed and http://www.spacecollege.org website for the latest!!

Oh, and thanks to all of those who donated to our project.  As you can see, your money is being well spent!

 

 

Posted in Space | Tagged , , , , , , | Leave a comment