Since our last review, the MMA Project has made very substantial progress in technical developments within the D&D program and in developing the planning for the reference MMA construction project. Moreover, the progress in defining an international partnership matches the most optimistic of the Millimeter Array Oversight Committee (MMAOC) scenarios. The MMA team, and the NSF, are to be commended for working vigorously to bring the partnership to the stage of a fully workable MOU for the D&D phase, together with the good will expressed in mutual accommodation and in alignment of goals.

A pressing task is the fixing of the scope and cost of the US reference project. The MMAOC feels that it is essential that the NRAO and NSF establish a recognized baseline agreement that will clearly define the level of support that the NSF can contemplate and set the level of participation in a combined international enhancement of the project. We recommend a comprehensive cost/schedule/technical scope review as early as possible in order verify and establish this reference level for the project.

The MMAOC recommended, in its earliest advice, that the MMA D&D program be pursued vigorously and without any delay associated with the development of a partnership. We further recommended that the pursuit of partnership be vigorous and intended to merge the programs with minimal impact on the D&D schedule. These ambitious goals, somewhat paradoxical and fraught with contradictory tensions, have, nevertheless, been accomplished with remarkably little delay, through the concerted efforts of NRAO and NSF. Both the NRAO and the NSF have demonstrated great vision and diligence in playing their respective roles in advancing the partnership. This should be sustained through the remainder of the D&D phase. We provided early advice that a second antenna to be used in an array should accompany the test of a single prototype antenna. This appears to now be feasible. The single antenna test was dictated by budget limitations in the D&D phase. However, the procurement of two antennas with the same requirements, in parallel procurements by the US and European partners, now offers the opportunity for a two antenna test at an NRAO site without US budget impact. This is an early indication that an improved system may well be the product of a combined, and cost-optimized, US-European partnership.

As part of the developing partnership, the MMAOC recommended in our last report that the US designs accommodate the European concept by increasing the diameter of the procured prototype antenna to 12 meters. This step would support parallel procurements, set a tone for the negotiations that we felt was positive, was consistent with the MAC statement on the physics sensitivity of the antenna diameter for the US physics goals, and would position the development for the much more powerful array that could emerge from the partnership. The goal of matched technical specifications and parallel procurements has now been carried out with both potential partners having released coordinated Requests for Proposals to industry.

If these solicitations are successful in leading to parallel design/prototype contracts with competing vendors, another early concern of the MMAOC could be addressed. We were hopeful that competition between vendors could be maintained until the final production selection was made. The single prototype/single vendor plan dictated by the US D&D budget permitted only an early commitment to a single source. The development of the parallel efforts by the US and European teams opens the possibility that a technical and cost selection can be made at the later stage of initiation of production.

The consolidation of both partners on a 12-meter design is a very advantageous step. However, it stresses the MMA D&D plan. We reaffirm our earlier advice that the NSF work with NRAO to support any impacts from this positive step.

We organize our remaining comments to parallel the questions in the charge to the MMAOC for this meeting.

At its fourth meeting the MMAOC is requested to consider the following issues:

• Based on your preliminary review of NRAO's cost estimate for constructing the U.S. reference design of the MMA, as well as on NRAO' presentation at this meeting,

• the MMAOC is asked to recommend a process for its own intensive and timely review of this document for NSF; and
• the MMAOC is asked to recommend a process for subsequent detailed external review of this document for NSF.

MMA AOC response: We believe that a prompt review of the NRAO cost estimate for the US reference design be carried out as soon as possible for the reasons outlined earlier. We recommend that the two possibilities outlined in the charge be combined into a single review consisting of MMAOC members and others selected for appropriate expertise. We have separately provided an outline of the structure and type of report from such a review and we understand that the NSF is currently planning a review of the type that we have recommended.

This review is prerequisite for any fixing of the MMA project scope, cost or schedule, whether as a single US project, or a combined partnership. Fixing the baseline must be founded on a credible and carefully audited plan. If the reviewed and audited plan demonstrates unresolved issues or areas requiring further development, these should be encountered now in order to facilitate early attention. If the review leads to fixing a baseline cost acceptable to the NSF, the NRAO must resolve and commit to develop, design and construct to this cost. The MMA is a remarkable scientific opportunity worthy of timely and vigorous development.

• Based on the initialed Memorandum of Understanding establishing a U.S-European effort to carry out a joint Design and Development, as well as on other information presented at this meeting,

• Does the Committee have specific comments or recommendations on the steps taken and proposed to be taken by NRAO in order to accommodate the antenna procurement process to the interests of its prospective European partners?

MMAOC response: The steps taken and planned to date represent impressive progress, demonstrating early openness and aligned interests. We comment above on how the coordinated solicitations revive the prospects for dual-antenna tests and for improved competition in the procurement. The matching of technical specifications also promises early firming of the technical baseline for the antenna. For this strategy to succeed, it is imperative that both partners provide a suitable and coordinated way to manage the two selected vendors. Many decisions will have to be made during the execution of the contracts. These will require coordinated decisions during a time beyond the initial partnership "honeymoon". This coordination will provide a real opportunity to develop a robust partnership.

• Does the Committee have comments regarding NRAO's plans for managing the impact of a potential European partnership?

MMAOC response: The early steps to fashion a partnership are appropriate for the D&D phase. The MOU promises a follow-on agreement to establish the partnership for the full construction phase. In order to support definition of the combined technical, cost and schedule baseline, and to make optimum use of the combined partnership resources (material and intellectual), we recommend that attention be given to early definition of the individual institutional and subsystem responsibilities. Where technically feasible, both partners should consider less parallelism and more integration of their respective D&D activities in order to conserve resources, to focus on needed work, and to establish the matrix of responsibilities for the construction. This will require negotiation, some timely technical decisions, and greater integration of tasks and interfaces. However, these integrating steps may be more difficult later on when readiness for construction dictates that prompt commitments be made in the face of competing developments and that institutions alter their roles when the press of construction commitments is urgent. In our discussions with MMA management, we came to believe that these early choices are feasible. Thus, we recommend that the D&D phase be used to advance definition of the combined construction project into a seamless project team.

• Please provide a critique of MMA project planning and organization, as described by NRAO in the Millimeter Array Management Plan (Version 3.0), the Millimeter Array Project Book (Version 2.5), and as described at this meeting. As in the past, please use the following criteria to frame the Committee's assessment: probability of success, completeness, quality of project management and planning.

• Has NRAO responded satisfactorily to the MMAOC's previous recommendations?

MMAOC response: NRAO has, indeed, responded well to our previous recommendations. The cost estimate and schedule definition has advanced. The MMAOC urges that progress toward a performance measurement system be pursued vigorously.

• Based on the information presented at this meeting,

• Does the MMAOC have comments on NRAO's technical progress for MMA design and development efforts?

MMAOC response: The technical progress is consistent with the D&D plans.

• Are there any other issues that the MMAOC deems relevant?

MMAOC response: We have commended the MMA team for impressive progress in technical, planning and partnership areas. We have commented that these have placed great demands on the MMA management team. We believe that the demands will increase through the cycle of the project and that the needs of an enlarged and international project will be especially demanding. We believe that the key areas of project direction, project management, antenna procurement and development, system integration, and project science coordination be strengthened by reexamining and clarifying the defined roles of each position, and by augmenting the leadership team, where advantageous, with additional experienced leading staff to fill these roles. The success of MMA will be promoted if attention at the top of the organization is robust in all of these areas. The D&D phase must produce a design, a cost and a schedule. Most important, however, it must produce a team capable of executing the MMA construction effectively. This is an equally important deliverable of this phase.