Notes from Reviewer Recommendations ----------------------------------- Overall the project has done an excellent job to date and the reviewers are very confident that the project will deliver on the technical aspects of the project. However, the project must also serve real users and real user communities and this is where the reviewers see the project as falling behind. User Interaction ---------------- For the second aspect, the reviewers see a lack of focus on exactly what user community we're trying to serve and on what use cases we want to support. They see this lack of focus as diluting the effort necessary to make a strong demonstration of the project's cloud technology. They recommend specifically: 1) To choose 1 or 2 concrete use cases, 2) To bring users directly on board with these use cases, 3) To work directly with those users to ensure success. This will help to avoid the current situation in which the project is somewhat decoupled from the users and ensure that there are points for strong dissemination, creating for example, a video demonstration with the user's application. Keep in mind that a demonstration is NOT a tutorial. Overall, there needs to be a better strategy for working with users. This strategy should be developed between the various activities within the project and documented. Related to this is the dissemination strategy. Overall, the dissemination effort and strategy needs to be much more ambitious. Technical Roadmap ----------------- For the technical roadmap, there are no problems. The roadmap is well-adapted to the needs and goals of the project. The reviewers have confidence (based on the first year results) that the project can deliver the functionality defined in the roadmap. From the outside however, it is very difficult to get an overview of what has been developed within the project, what external tools have been modified, and what tools have simply been integrated. The project needs to make this information available in a concise and unified format. One technical point was to remain neutral on the low-level data management layer (i.e. shared file system). Different sites will choose different technologies and this can't be imposed. We should also investigate scalable shared file systems that work, like GPFS. Consortium & Management ----------------------- Overall, the management of the project is good. However, more effort needs to be made to produce the periodic report on time and permit the reviewers to adequately review it. The analysis of the current underspend was insufficient, including the explainations for the underspend. The reviewers recommend to develop a spending forecast for Year 2 to understand better the use of resources and to investigate scenarios for "correcting" the underspend. It was evident from the participants body language and interactions that there is a good chemistry and good communication between the team members. This is evidently one reason for the good technical success of the project. The reviewers were impressed with the technical progress made by the project and specifically suggested that the current agile processes are continued in the project. Deliverables ------------ Specifically on the deliverables, all of them have been accepted by the reviewers, except the Periodic Report for Year 1, which hasn't been submitted because of the missing financial information. They did comment that the executive summaries of all of the documents are "crap". These summaries for the second year need to be improved; they should contain the concise "take home message" of the deliverable, what are the benefits, and how it can be used; it should not be a summary of the project or the report. The project should absolutely ensure that the Periodic Report for the second year is available to the reviewers on time. In general, the deliverables should be written with much less "fat". There are often 2-3 pages of really good material within the documents hidden by lots of fat. Remove the fat and provide concise, relevant documents. Demo ---- The first part of the demo with the manual installation and tutorial went smoothly. The second part encountered several problem, although most of the individual parts worked a problem with the APEL node prevented the scalability from working. Irregardless of these problems, the reviewers didn't find the demo interesting at all. The use of the command line and inclusion of details rendered the demo "magic" for the reviewers. In addition, they wanted much more "marketing" videos for our target communities that demonstrate the benefits of the technology and an overview of how it works (without the low-level technical details). Sustainability and Strategy --------------------------- Sustainability is a critical point for the project right now and must be resolved to give people confidence for adopting and using the StratusLab distribution. The project's confused focus (see below) makes developing a convincing sustainability plan more difficult. The project should consider adding an outside expert to the PMB and/or creating a "Strategy Director" to help develop a general sustainability and exploitation strategy for the project. The reviewers recommend working with a business or economic school. Students could do a project around the sustainability and exploitation of StratusLab. This would provide valuable feedback to the project. The students would ask "dumb questions", but these are exactly the points that would need to be tackled for such a plan. Grid vs. Cloud -------------- The reviewers see the "confusion" between grid and cloud technologies as one reason for the lack of a sharp focus for the project. They quite clearly recommended that we concentrate primarily on the cloud aspect (which are very strong in the project) and "ignore" the grid aspect. It was asked whether we could actually change the focus in this way and still be consistent with our stated program of work within the Technical Annex (TA). The response from the reviewers was quite clear: 1) we've already essentially fulfilled the stated interactions with the grid by the work already accomplished and 2) both aspects are described in the TA and this could be accommodated within the scope of the current document without needing contractual changes.