The MMAOC meeting of March 2-3, 2000 included presentations from the AUI and NRAO management of the US MMA project. For the first time, the committee also heard presentations from the management of the European project that is working with the US team to define a combined project, ALMA. The new members of the European Oversight Committee for that project also joined us.

The MMAOC feels that dramatic progress has been made toward defining a combined international project for the next step in millimeter wavelength astronomy. The emerging coordination and integration of the current parallel design and development activities demonstrates rapid and enthusiastic progress toward a single efficient and focused program. Bilateral discussions towards resolving the remaining technical decisions, construction responsibilities and management arrangements are proceeding in an open and success-oriented manner. The management teams on both sides of the ocean are expending a great deal of effort and diligence on these difficult agreements. There is every reason to believe that this diligence is headed to a successful outcome.

The technical work of the Design and Development phase (now dubbed Phase 1 in the joint planning documentation) is proceeding well. This is commendable given the obvious distractions faced by the project’s management team who are clearly diverted to the work of forming an international partnership.

Indeed, the recent arrival of a US Project Manager, and the developing shared management and consultation by the two teams leads us to believe that a strengthening management team is developing. Nevertheless, the details to be worked out still lie ahead. The progress to this point leads to optimism.

Based upon this meeting, the MMAOC feels that the NSF is in a good position to recommend that this project proceed to construction (Phase 2) in 2002. We learned that the NSF has requested that the US MMA project carry out a fourth year of the D&D phase (Phase 1) during 2001 and the President’s budget requests $6 million for this additional year of development. The MMAOC is concerned that this funding level is not adequate to maintain the progress of building the team and design definition. The NRAO projects that this funding level is short of needs by $6.5 million. We urge that the project proceed to early commitment of the funds in 2001 under the planning assumption that construction funding will be awarded in the federal fiscal year 2002. We urge the NSF to work with the project team to minimize the impacts of the possible funding shortfall in 2001.

We arrange more detailed comments within the structure of the charge to the MMAOC.

MMAOC Charge, Meeting of March 2-3, 2000

At its sixth meeting the MMAOC is requested to consider the following issues:

This meeting was focused on a review of the process leading to merger of the US and European projects. We spent less time than normally scheduled for review of technical accomplishments. Nevertheless, we did review the January, 2000 Monthly Report for the project, and we heard presentations on the antenna, computing, correlator and mixer systems and received summaries of other technical efforts. We appreciate the open and candid discussion of the development efforts provided by the NRAO staff.

These discussions indicate that progress is being made in the technical efforts and that the schedule is being held. It was worthy of note that the development work is being integrated with the Europeans and we learned of several collaborative and supportive efforts. Technical choices between some European and US alternates have been made and others are proceeding toward resolution. The project has carried out 11 joint USA/European design reviews to date! This is an especially promising achievement.

The most noteworthy technical development is the coordinated procurement of the two prototype antennas, one by each partner. This is making good progress and should lead to definitive tests of a two-antenna interferometer; a major Phase 1 goal.

At future MMAOC meetings we should return to more thorough presentations on the technical developments, using the format that we have requested in the past, including technical, cost and schedule status information on each area.

A concern of the MMAOC has been the software area. The NRAO presentation allayed many of our fears primarily because it defined a working relationship between the USA and the Europeans with development of a joint software program. This is a key first step. A lot of effort is required to implement a working program and NRAO indicated they understand the task at hand.

The project schedule has reached a very mature stage of merger with the European plan. NRAO has made appropriate adjustments in the project plan to accommodate the merger. The MMAOC was presented with a detailed schedule that clearly separated Phase 1 and Phase 2 activities, and activities for which the US or European team was responsible, or both partners were jointly responsible.

The planning, however, is not yet mature enough to support a presentation of progress or detailed earned value. This planning should continue to full definition of the joint construction project.

Based on data provided by NRAO, there is a $6.5M shortfall in funding to successfully carry out the program in CY01.  NSF has submitted a budget request of $6M for FY01.  The $6.5M shortfall is based on approval of the $6M request. 

The budget request for FY02 will not be known for roughly another year.

This lack of funding for CY01 places the US ALMA Project Office in a very difficult position to properly plan the program relative to schedule, production cost, staff and program risk.

The oversight committee is impressed with the cooperation, communication and progress being accomplished by the USA/European ALMA Project Offices.  Important and critical agreements are being made by them.  We realize that these agreements still require the approval of NSF.  It is anticipated that most, if not all, of the scientific and technical requirements will be approved by NSF/ESO. This will place on NSF and NRAO a stronger commitment to meet their obligations to agreements reached.

The oversight committee, after careful consideration of all of the factors described, recommends that the NRAO ALMA Project Office plan the program anticipating that the NSF will request construction phase funds in FY02, and it will be approved by Congress.  Hence early upfront spending of the $6M that has been requested for FY01 should be considered in their planning.

The oversight committee fully realizes that this recommendation has high potential negative impact to the program. Completing the last 6 months of FY01 with very little funds would have considerable impact on maintaining a management and technical staff if the $6.5M is not approved.  However, the combined experience of the oversight committee familiar with programs of this size and complexity puts us in the awkward position of making the recommendation of early FY01 spending.

The NSF should also strongly consider increasing the budget request for FY01 to a number more consistent with the needs of project/NRAO.

The MMAOC is highly impressed by the rapid progress made by NRAO in achieving many technical and managerial agreements with Europe on ALMA in the last few months. A strong sign of European interest was the presence of potential European members of a future combined oversight committee for ALMA at this MMAOC meeting. The suggestion that the MMAOC and this future ALMA OC be rapidly merged with the MMAOC was one topic of the European presentation, discussed below.

The process (joint meetings of agencies, science, management, and technical aspects of the project) appears to be thorough and complete with dedicated efforts on both sides of the ocean. Agreed upon technical specifications of the two project offices were presented. Some of these requirements (e.g. frequency coverage) may increase cost/schedule. The MMAOC recommends that each agreed-upon requirement be evaluated against the current NRAO design-to-cost requirement. The MMAOC is concerned by possible "requirements creep" driven by the merger.

The probability of success is extremely high. The completion of eleven joint design reviews is a significant accomplishment. Communication is very important to completing a program successfully and this is an excellent indication that it is taking place.

The OC recommends that the merger to a single project, with a single project office, be carried out as expeditiously as possible. Events and actions are occurring rapidly and they should be given that the project must continuously build definition and strength. The merger should proceed as fast as the project teams are able to reach agreement and a working arrangement. It should not be delayed for agency or government agreement. Rather, the project teams are responsible to develop the best project scope and management and to present this for agency review and agreement.

The MMAOC commends the progress made to date with regards to the prototype antenna procurement. The procurement allows two competitive technologies to be assessed and retains an important element of competition between two vendors. The procurement is properly structured to test the technical specifications of a 12-m submillimeter antenna, has adequate plans to assess the antenna metrology, and to prepare for interferometric testing. The proper execution of this procurement activity has the potential to control the cost of the ALMA project and the elements are in place to successfully execute this activity. The NRAO should be commended for structuring a potentially successful prototype procurement that may lead to a cost efficient production procurement for the rest of the program.

It will be advantageous to clarify the criteria to be used in making the final selection of the antenna contractor. The choice could be one of the contractors, or both. However, the contractors are not given clear guidance on the criteria to be used. Rather than allow them to guess what will be important and less important, the project should complete a disciplined exercise to define written guidance to the contractors. Contractor comment on this criteria document should be considered in making it definitive. The criteria document should not unduly limit the nature of the final selection.

NRAO has selected an experienced contractor with a proven track record. The contractor is making extensive use of carbon composites, which have many advantages over materials used in the past. The first mode of the antenna was stated to be about 6.5 Hertz. This was obtained from analysis. Experience from the past has shown that analysis overpredicts the mode by about 10%, resulting in what some may consider a very low mode. It is recommended that a thorough dynamic response analysis be carried out considering all forcing functions that the antennas will be subjected to. It is important to demonstrate that the fundamental mode is not being excited during science operations, and if it is, it will be damped out quickly.

Management of the contract should include considerable effort to carry out value engineering to reduce costs/schedule in production. Management of the merged project should be aware of what is proceeding in both the US and European contracts, with due sensitivity to later competition. The project should learn from both contracts.

The MMAOC recommends prompt merger of the US and the emerging European oversight committees. This will support oversight of the merger. However, it is important for the NSF to be able to seek advice specifically from the US members of the committee. This should not be taken by either party as a hint of dissension or disunion, but as evidence that the NSF has its own management and decision process and that these may benefit by the best possible advice from the US members acting separately.

During the meeting, the European members numbered 3. The US MMAOC currently numbers 5 members. We recommend that a joint committee be formed by the NSF and the European Coordinating Committee with a membership approximately the size of the two currently appointed groups, but that the NSF and ECC determine the membership and how the committee will be organized as they are the recipients of the advice.

Given the intense focus on merger, the NRAO has made notable technical progress.

In future meetings of the OC, we should return, as stated earlier, to presentation of technical, cost and schedule progress relative to the project baseline plan.

The appointment of a US Project Manager is an important step. Mark Rafal will assume the important responsibilities of this position at a time when the project management has many competing and critical demands for its attention. It is very important to complete the development of the full merged management team at the earliest date.

NSF needs to work very closely with the NRAO in developing the budget and technical justification to be used to seek approval through the NSF, OMB, and Congress. The communications between the NSF and the NRAO is infrequent and is not sufficiently focused on this process. NSF needs to give clear guidance to NRAO about the inputs that they need, and they should do it well in advance of the need dates. NRAO should keep NSF fully briefed about potential changes of scope of the project (e.g. the addition of a 350 micron channel, or the entry of Japan to the partnership). NSF should declare the major reviews that are to be held before putting the project into the budget submission, before giving the final go-ahead to start construction, and the nature and schedule for periodic progress reviews.

For a project of this size, the greatest danger is a "loss of control incident," in which the good work being done is overshadowed by apparent runaway budgets and technical requirements. None of these are necessary, but they have happened to other projects. It is critically important that NSF and NRAO prevent any such event. This requires good management to a fixed cost and fixed technical requirements, it requires excellent communications among the participants, and it requires the ability to show progress against resources in a candid and convincing way.

NSF and NRAO should be clear about whether this is a cost-constrained project, in which performance will be sacrificed to meet the budget. If so, then the project should produce a descope plan showing how and when decisions must be made to control costs. Considering the international nature of the project, this is not so simple and needs much attention.

Similarly, it is important in the international agreement to specify the consequences for non-delivery or late delivery of agreed contributions. With 17 European institutions and NRAO and NSF and Congress all involved, there will certainly be such events. It must not be left to chance how this is done. Similarly, it is essential that a method of valuing contributions must be established, and that it not be just a count of effort expended (or wasted or spent on institutional overheads). It must be a count of the actual accomplishments (earned value) that are required by the project plan.

The joint project efficiency of 17 European groups and 3 NRAO sites and several Chilean sites must certainly be considered. Interface control across so many time zones is an organizational challenge. It is essential that the organizations pledge to work closely with each other and that they develop a cooperative approach, in which all participants help to work out problems as they arise. Some scheme for asserting central management control of the systems engineering is the most important part of this. Configuration control requires much thought.