From EGI Knowledge Base

Jump to: navigation, search


    Download this content here


[edit] Operations and security

The Operations and Security Function includes those EGI services needed to ensure optimal functionality of the pan-European infrastructure and the overall seamless effective interoperation of national and regional Grids. At the same time, a common authentication trust domain is required to persistently identify all Grid participants, common security policies need to be defined and enforced. In a European e-Infrastructure, coordination will be required on security policies and operational security to support and coordinate the work of teams drawn from the NGIs. The various tasks of this Function need to be structured according to a common operational model that meets various requirements: scalability and interoperability, availability and reliability, sustainability, and autonomy of NGIs. Many of the EGI operations and security tasks are jointly delivered by EGI.org and the NGIs, i.e. the EGI.org tasks complement those carried out by NGIs themselves in the regions. NGI International Tasks are those activities aimed at allowing the sharing of the national IT resources at pan-European and international level in a uniform, robust, and seamless way. Depending on the needs of the individual NGIs, the international tasks are integrated by the NGI national tasks, which are carried out to satisfy the NGI local requirements. In this scenario, common standards and/or specifications for interoperation between NGIs play a critical role in order to ensure interworking within EGI. To this end, collaboration from the NGIs is important to jointly define specifications, policies, best practices, and in general, to share operational responsibilities. It is important to note that at the time of writing, the devolution of operational and security activities and responsibilities is already a common approach adopted by the main European Grid infrastructure projects.

In what follows the Operations and Security tasks of EGI.org on one side, and of the NGIs on the other, together with the corresponding effort, are presented.

[edit] EGI.org Tasks

EGI Operations and Security activities belong to five broad categories:

  1. operation of tools and services;
  2. support;
  3. other tasks;
  4. security;
  5. development.

Notation: EGI.org and NGI tasks are numbered according to the following scheme. Prefix O-E identifies operations services provided by EGI.org, whereas O-N identifies those provided by NGIs. Explicit indication is given for those tasks that are deemed necessary (as opposed to optional services) and for those EGI.org activities to be run under the EGI.org responsibility which may be technically distributed to NGIs, as they do not need to be located in the EGI.org site.

[edit] Operation of tools and services

         O-E-1 and O-N-1: Operation of the Grid topology and configuration repositories (EGI.org and NGIs) –– necessary, can be distributed

Many aspects of operations rely on the availability of information (as applicable) from NGIs about service nodes, contact details, security contacts, certification status, sites in scheduled downtime, etc. The Grid repository provides all such information. Information input is devolved to regions and sites. The current central repository (known as GOCDB in EGEE) may need to be adapted to support a two-tier distributed model. This requires the definition and implementation of common interfaces and transport mechanisms to ensure the exchange of information between different Grid domains.

         O-E-2 and O-N-2: Operation of accounting repositories for international VOs (EGI.org and NGIs) –– necessary, can be distributed

The accounting repository is responsible of keeping records about usage of compute, storage, networking and other types of resources as required by the users, resource providers, NGIs, etc . It is the responsibility of a NGI to collect accounting data, and to keep a permanent master copy of usage records. Accounting information is needed by international VOs in order to allow VO managers to know about the amount of IT resources “consumed” by the respective users across different domains of the e-Infrastructure. For this reason, the deployment of standard interfaces between accounting systems in different NGIs, is important to ensure the interoperable exchange of records between different domains. For each NGI EGI.org is responsible of gathering and making publicly available accounting information (as applicable and in agreement with local laws and the privacy requirements of the EGI actors). The availability of a pan-European accounting infrastructure is a key enabling component of the EGI business model.

         O-E-3 and O-N-3: Operation of Grid repositories storing monitoring and performance data and other related information (EGI.org and NGIs) – necessary, can be distributed Availability, status and performance information about Grid services and sites are needed to check the health of the infrastructure and to verify the Quality of Service delivered to VOs and other NGIs. Gathering and publication of monitoring information – regarding Grid functionality, Grid service status, assessment of quality of the services delivered by various EGI actors (resource providers, the NGIs, etc.) – is consequently important to help the infrastructure to assess its level of service and compare it to the VO requirements. This also requires the operation of repositories and supervision of the processes to populate them, the maintenance of schema for publishing of site and service status information, the ownership of the information schema used, the preparation of reports, etc.

This task includes network performance for network quality assurance/reporting and metrics follow up, to ensure the underlying network infrastructure is working properly and efficiently, and that network providers are respecting their contractual obligations. To this extent, EGI.org tasks are the publication of statistics, the maintenance of schema for central publication of site and service status information, the deployment of monitoring-related tools such as the dashboard and the alarm system, and the preparation of performance reports.

         O-E-4 and O-N-4: Operation of the Grid Operations Portals (EGI.org and NGIs) –– necessary, can be distributed

The Grid operations portals provide an entry point for various actors to support their operational needs. Different "views" are necessary according to the role of the customer (Grid operators, VOs, Grid site managers, Region Operations Managers, etc.). The information displayed is retrieved from several distributed sources (databases, Grid information systems, etc). It provides static information about sites/VOs, and dynamic information about resources/services status and allocation. The central Operations portal is the aggregation point of regional information, which is also accessible via regional operations portals.

         O-E-5 and O-N-5: Grid operation and oversight of the e-Infrastructure (EGI.org and NGIs) –– necessary, can be distributed

EGI.org operation and oversight activities over the detection and coordination of the diagnosis of problems affecting the entire EGI e-Infrastructure during the entire lifecycle until resolution, the reporting of middleware issues to the developers, the execution of quality checks of the services provided by NGIs, and the handling of operational problems that can not be solved at the NGI level. This task coordinates the oversight of the NGI e-Infrastructures (run under the responsibility of the NGIs), which– at the NGI level – includes the monitoring the services operated by sites, the management of tickets and their follow up for problem resolution, 1st and 2nd line support to operations problems, the suspension of a site when deemed necessary, etc. This EGI.org task is currently done in EGEE in cooperation with the relevant Regional Operations Centres (via rotating shifts) according to a two-level hierarchical model [COD]. We foresee the possibility to evolve this model, in such a way that NGIs can autonomously run oversight activities in the region, or to federate in order to share efforts.

[edit] Support

         O-E-6, O-E-7 and O-N-6, O-N-7: central and regional Grid user support and ticketing system (EGI.org and NGIs) –– necessary, can be distributed

User support relies on a central helpdesk, which is a regional support system with central coordination [GGUS]. It gives access to user documentation and support, and to a ticketing system. The central system is interfaced to a variety of other ticketing systems at the NGI level in order to allow a bi-directional exchange of tickets (for example, those opened locally can be passed to the central instance or other areas, while user and operational problem tickets can be open centrally and subsequently routed to the NGI local support infrastructures). Support to network end-to-end problems in the Grid is also important – especially for applications requiring high-availability – as connectivity is provided by the pan-European network research backbone and by a large number of National Research and Education Networks, each providing links to sites within countries. A Network Operation Centre provides the operational interface between the Grid and the relevant network players to check the end to end connectivity of Grid sites [ENOC]. The NGIs provide 1st line local/regional support to users and centres, while EGI.org takes care of the Maintenance and Operation of the central ticketing system (GGUS like) and of the Triage of incoming problems. a.Maintenance and Operation (can be distributed): run a central ticket handling system for Grid and network end-to-end problems. User support relies on a central helpdesk, which is a regional support system with central coordination [GGUS]. It gives access to user documentation and support, and to a problem ticketing system. b.Triage of tickets entering the central user support system (also known as ticket processing management in EGEE) – can be distributed, consists of the monitoring and routing of all active tickets in the Grid user support system by Grid and VO experts, who are responsible of addressing the problems to the appropriate second-line specialized support units. This process combines manual as well as automated procedures.

         O-E-8 Gathering of requirements for user support tools and process –– necessary, can be distributed 

Tools and the process for user support are designed to meet the requirements of customers taking input from NGIs, VOs and resource centres. Additional requirements may arise with the evolution of the middleware stacks in use, and with the support of new user communities. EGI.org is responsible of the coordination of this process.

[edit] Other tasks

         O-E-9 Coordination of middleware roll-out and deployment (EGI.org, necessary, centralized), middleware pilot and certification testbeds (EGI.org and NGIs, necessary, can be distributed) 

It is important to ensure that middleware updates move from certification and into production as quickly as possible, while also assuring that the updates are suitable for deployment in the production Grid. EGI.org coordination will be needed for strategy decision, for example to decide significant changes to processes, and to ensure that resource sites are encouraged to upgrade whenever new critical updates of supported middleware stacks are released. Being still in a phase where middleware is subject to frequent bug fixing cycles, prompt alignment of the Grid services and components to the latest releases, contributes to better functionality and availability of the overall infrastructure. In addition to this, the operation by NGIs of facilities for testing and certification of middleware are important for the deployment of high-quality middleware by allowing VOs and site managers to test Grid components during the early development and release phase.

         O-E-10 Coordination of resource allocation and brokering support for VOs from NGIs (EGI.org) –– optional, centralized

VOs can specify requirements in terms of resources to be guaranteed by the overall pan-European Grid infrastructure used. In this case, coordination – as required by VOs – contributes to ensure that a suitable production infrastructure (Grid core services and resources offered) is in place, to meet such requirements. Development is still needed to provide tools for the automation of the management and the negotiation of SLAs. EGI.org is responsible of support and coordination of this process.

         O-E-11 Coordination of interoperations between NGIs and with other Grids (EGI.org) –– necessary, centralized

Coordination is needed to foster the creation of a seamless operations model across administrative boundaries, in order to pursue pervasiveness and sustainability of the infrastructure. This is of great importance as users who want to cross Grid boundaries need to know that the environments will be similar, and applications must function properly without major changes. Interoperation covers a number of aspects, such as the availability of common tests for monitoring of site status, the interconnection between helpdesks/ticketing systems, etc. “Other Grids” includes Asia-Pacific regional Grids, OSG, Naregi, and related infrastructure projects. This role owns the definition of the operational tools interfaces, the procedures and the operational activities allowing the NGIs to interoperate. EGI aims at continuing the collaboration established with operations centres outside Europe in order to preserve the current integration of non-European sites into the production infrastructure. EGI.org is responsible of support and coordination.

         O-E-12 Coordination of network support (EGI.org) – necessary, centralized. Network operation design, handling of troubles affecting international VOs, and network assessment allow EGI to keep the state of the network under control, and to establish link between Grid operations and network operations. A centralized approach is proposed here in order to keep this task is close relationship with the other External Liaison tasks run by EGI.org.
         O-E-13 Definition of best practices, operations procedures, operations requirements (EGI.org and NGIs) –– necessary, can be distributed 

Interoperation relies on the definition of best practices and of general operational procedures for daily monitoring activity for sites and federations. EGI.org is responsible of the coordination of these activities.

         O-E-14 and O-N-8: Operation of the production Grid Core Software Services, catch-all services for international VOs, catch-all VO (EGI.org) – necessary and distributed

Grid core services are components of the EGI e-Infrastructure. They are software components that typically run on server machines. With Grid service we refer to a software instance (a Web service in many cases) "that is designed to operate in a Grid environment, and meets the requirements of the Grid(s) in which it participates." [GLO] In particular, Core Software Services are those necessary components provided by the Middleware Consortia on which the overall Grid functionality relies in order to operate. Catch-all instances can be required to support small user communities. It is a responsibility of EGI.org to ensure that user communities are properly supported by the NGIs of reference. Examples of gLite Core Software Services are: the VO management service (e.g. VOMS), the File catalogue and transfer services (e.g. LFC and FTS), Job management services (e.g. WMS), Information services (e.g. BDII), Security services, etc. Authentication is also fundamental to get access to resources in the Grid. This is why a catch-all Certification Authority needs to be available to any user community in EGI.

[edit] Security

The character of the security vulnerabilities and risks presented by Grid infrastructures provides a rationale for coordination among the Grid participants at various levels: at the NGI level between site managers, the NGI itself and CERTs from the National Research and Education Networks, between NGIs and EGI.org to adopt and enforce common policies, and between EGI and international bodies such as EUGridPMA (the European Policy Management Authority for Grid Authentication) and IGTF (the International Grid Trust Federation). In a European e-infrastructure some central coordination will be required on security policies and operational security. Support and coordination of the work of teams drawn from the NGIs, will be the task of EGI.org.

         O-E-15 Coordination of security policy development and enforcement –– necessary, centralized Security policy development and enforcement are needed to define an agreement on matters such as best practices, security policies, CA policies, etc. A team of security people in NGI’s will take care of ensuring the definition and application of standard security policies. EGI.org is responsible of support and coordination.
         O-E-16 Coordination of security and incident response - necessary, centralized

It is needed to ensure that common policies are followed for coordinated incident response by EGI members from NGIs. EGI.org is responsible of coordination and support.

[edit] Development

         O-E-17 Coordination of development and maintenance of operational tools – necessary and centralized

While the tools for accounting are included in the Middleware, other tools are needed to support operations. Examples are: tools for monitoring, dashboards and alarm systems, ticketing systems, portals, etc., and new tools to improve automation. EGI.org is responsible of coordination of the maintenance of the set of the tools presently used in Europe production Grids and of the upgrades that will be necessary for keeping them in step with the quantitative and qualitative evolution of the Grid. This includes monitoring tools to measure and report on the quality of networks used by Grid project to ensure the underlying network infrastructure is working properly and is efficiently used, and that SLA constraints with network providers are met. It is foreseen that EGI.org will only take coordination responsibility (necessary task) while a set of willing NGI’s will take care of the development work, to be co-funded by the EC.

[edit] EGI.ORG effort and timing

Given the detailed description of activities provided above, the following paragraphs summarize the list of activities carried out by EGI.org and NGIs, and indicate the effort needed to support those tasks in the first three years of EGI. For the sake of simplicity, estimations are expressed in Full Time Equivalents.

[edit] Operation of tools and services

         O-E-1.Operation of the Grid topology and configuration repositories. EGI.org FTE: 1	
         O-E-2.Operation of accounting repositories for international VOs. EGI.org FTE: 1
         O-E-3.Operation of the grid repositories storing monitoring and performance data, and other related information. EGI.org FTE: 2.5
         O-E-4.Operation of the Grid Operations Portals, EGI.org FTE: 0.5
         O-E-5.Grid operation and oversight of the e-Infrastructure. EGI.org FTE: 1

[edit] Support

         O-E-6.Maintenance and operation of central ticketing system: EGI.org FTE: 2.
         O-E-7.Triage of incoming problems: assignment of tickets to the 2nd line support units, ticket escalation end ticket follow-up to ensure they get closed, EGI.org FTE: 2
         O-E-8.Gathering of requirements for user support tools and process: EGI.org FTE: 0.5

[edit] Other tasks

         O-E-9.Coordination of middleware roll-out and deployment, middleware pilot and certification testbeds. EGI.org FTE: 1
         O-E-10.Coordination of resource allocation and of brokering support for VOs from NGIs, EGI.org FTE: 0.5
         O-E-11.Coordination of interoperations between NGIs and with other Grids. EGI.org FTE: 0.5
         O-E-12.Coordination of network support, EGI.org FTE: 0.5
         O-E-13.Coordination of definition of best practices, operations procedures, operations requirements, FTE: 0.5 
         O-E-14.Operation of production Grid Core Software Services, catch-all services for international VOs, catch-all CA: EGI.org  FTE: 1

[edit] Security

         O-E-15.Coordination of security policy development and maintenance; EGI.org FTE: 0.5

Coordination of security and incident response: EGI.org FTE: 1

[edit] Development

         O-E-16.Coordination of development and maintenance of operational tools. EGI.org FTE: 1
Table 1: overall effort for EGI.org operations and security critical services
Operation of tools and services 6
Support 4.5
Other tasks 4
Security 1.5
Development 1

[edit] NGI TASKS

The list of tasks in this paragraph is not intentionally comprehensive, as it is meant to only include the necessary international tasks of an NGI. Many of the tasks in this section are performed by the NGIs and coordinated by EGI.org. The necessary property of such tasks does not prevent an NGI from devolving the operation of the task itself to a third party, or from choosing the option of purchasing it from EGI.org. Tasks not relevant to the overall EGI operation model, or specific to national VOs, are omitted. NGIs are free to choose the most suitable supply model. For instance, it can federate with other NGIs to share their joint effort, it can buy a set of services from other NGIs or other partners, or request them to EGI.org. In order to facilitate NGIs, especially during the transition phase, we foresee the possibility for EGI.org to supply catch-all operational services – in addition to the central ones – according to the demand. We believe the number of FTE needed by EGI.org to run catch-all services remains fairly constant with the number of NGIs requesting it.

         O-N-1.Operation of the NGI Grid topology and configuration repository - necessary
         O-N-2.Operation of the NGI accounting repository - necessary
         O-N-3.Operation of repositories storing monitoring and performance data, and other related information – necessary
         O-N-4.Operation of the NGI Operations Portal – necessary 
         O-N-5.NGI e-Infrastructure oversight (monitoring of status of services operated by sites, opening of tickets and their follow up for problem resolution), 1st and 2nd line support in case of operational problems, site suspension, reporting to EGI.org in case of middleware problems and general operational issues, etc. – . – necessary
         O-N-6.Operation of the NGI ticketing system, gathering of new requirements for user support tools in the region – necessary
         O-N-7.Regional helpdesk: support to users and site managers via a local/regional helpdesk - necessary
         O-N-8.Operation of production Grid Core Software Services, catch-all services for international VOs, catch-all CA: running the required Grid services provided by the NGI, and services required by international VOs – optional; availability of Certification Authority: to distribute X.509 certificates to users and servers in the region - necessary	
         O-N-9.Operations Coordination at the NGI level - necessary
               a)Security and incident response coordination in the region
               b)Roll out of middleware updates in the NGI
               c)Resource allocation in the NGI
               d)Interoperation with national and regional Grids 

[edit] NGI Effort and timing

The estimation for the NGI total manpower depends of course from the size of the NGI, the service level requirements to be met in the respective region, the level of participation to EGI activities, and from its organizational structure (e.g. an NGI can decide to outsource tasks or take care of extra tasks on behalf of other NGIs or EGI, etc.). Here tentative estimations are provided for the initial three years of EGI. NGIs are divided among three categories: “small”, “medium” and “large”. Estimations are based on the present EGEE experience, assuming that increasing automation and expertise will at least partly make up for the increase of Application variety and Middleware complexity. Small NGI: 2-4 FTE Medium NGI: 5-10 FTE Large NGI: 14-22 FTE

Note that for countries presently involved in EGEE, we estimate that in the first years of EGI an amount of people similar to the one currently available, will be funded to work on NGI international activities. The amount of FTE currently involved in operational activities for the EGEE III project alone is 189.9, of which 85.9 are funded by the EC. We envisage that when all the NGIs that have expressed interest in EGI will have been properly constituted and will have joined EGI, in EGI will comprise 6-7 large NGIs, 12-16 medium NGIs and 16-20 small NGIs. Nevertheless, for the very first year the number of NGIs could be somewhat smaller. More details on the NGI effort estimation are given in Appendix B. The operations and security function is supported by manpower effort and additional hardware resources, that are needed (mainly at the NGI level) to host Grid Core Software Services, operational tools, testbeds and auxiliary IT services (wiki pages, agenda pages, databases, etc.). Based on the current status, it is estimated that for some large EGEE ROCs about 150 servers are needed for these functions. Hardware resources needed for the realization of the NGI e-Infrastructure are funded via national funding sources (i.e. no EC co-funding is requested in this case).

[edit] Evolution

FTE estimates refer specifically to the overall amount of effort needed during the EGI transition phase (about three years). Efficiency after a few years might reduce the staff requirement for the initial operational model but we expect this to be partially matched by the requirement for new activities to meet the evolving requirements of new communities. As to development, we foresee a reduction in cost in about three or five years when operational tools will likely reach maturity. At this point, still a small fraction of funding will be needed for maintenance of existing tools. In three years we foresee the possibility to evolve some Operational and security tasks of EGI.org into services, of which some will be necessary and sold as a bundle, while others will be optionally subscribed by the NGIs. Depending on the type of service, these will be charged through a per-use or flat rate.

[edit] Middleware development and support

This chapter provides the technical details that complement and complete the Middleware Section (3.2) of the Blueprint. The concepts expressed there will not be repeated here, in general, and the reader is assumed to have preliminary knowledge of the Blueprint.

[edit] Middleware tasks and services

The general goal of EGI is to realize a large-scale, production Grid infrastructure for the sharing of IT resources and data, built on National Grids that interoperate seamlessly at many levels. EGI will offer reliable services to a wide range of applications, from “mission critical” to prototyping and research. The goal will be reached by distributing very different responsibilities between the various players. The EGI specific technical objective, to be achieved by a coordinated effort of the EGI.org central organization and the National Grid Initiatives, is to take care, on behalf of all the stakeholders, of the procurement, certification, deployment, and operation of the software services (i.e. of the software infrastructure) and to set up the organizational rules, policies and procedures which will provide the required standard access and sharing mechanisms for all sort of IT resources and data which currently are and will continue to be made available to the researchers by national resource providers. These today are mostly constituted by public or semi-public Resource Centers of very different sizes and dimensions, which will continue to be completely funded at national level. The Blueprint justifies the establishment of a Middleware function in EGI and why it needs to remain in full control of the software infrastructure, which constitutes one of its key services offered to all the stakeholders. The Blueprint proposes to implement such function, at least in the first EGI years to avoid the risk of disruption of the current services, used daily by thousands of researchers, with a limited manpower in EGI.org and without requiring any further immediate mandatory contribution from the NGIs. The maintenance and support of the Middleware components used in the current e-Infrastructures should continue to be co-funded at a level of 50% by the EC, without excluding contributions from other interested partners, and sustained for the remaining part by the Middleware Consortia and other development teams, that have agreed to move toward a Unified Middleware Distribution (UMD), under the steering of EGI. In the future, once the EGI e-Infrastructure will be well established, the maintenance and support of the legacy widely adopted services will be progressively covered by the NGIs through the payment of service charges, while the necessary innovation and the new developments, as defined by the need of the user communities and operations, will continue to be co-funded by the EC through competitive calls. This Chapter provides more details about:

  1. Middleware Components and Middleware Consortia.
  2. Guidelines for UMD.
  3. Role of the EGI.org Middleware Unit.
  4. Components and Services proposed for inclusion in UMD in the first stage of EGI.
  5. Cost estimates for Middleware maintenance

[edit] Middleware Components and Middleware Consortia

Currently a variety of Middleware components are deployed in the EU e-Infrastructures. They are the result of several years of European and international competitive efforts aimed at satisfying the needs of a large number of user communities with complementary requirements and dimensions, ranging from teams of few individuals to very large international collaborations with thousands of researchers. They all adhere to a general service oriented approach, aimed at compliance with the evolving Web Services and the Open Grid Forum standards. A large part of components come from three middleware distributions: ARC, gLite and UNICORE. All the three of them are mostly developed in Europe and are used in production in the three main EU e-Infrastructures: EGEE, DEISA and NDGF. Each of the distributions provides a middleware stack with a good coverage of the most fundamental needs, however none of them represents a fully generic satisfactory solution for all the needs. Other middleware projects also exists in Europe (such as GridWay, pGrade, AssesGrid, GRIA, ...), funded by the European Commission and by National funds, and provide higher-level services, which complement the basic services provided by those three EU stacks. Recent efforts, in particular by the OMII Europe project, have already improved their interoperability. Pragmatically (after eight years of successful developments), the ARC, gLite and UNICORE stacks, with the mentioned enrichments coming mainly from other EU initiatives and together with a few components taken from the US-based Globus and VDT provide a very large fraction of the services in use in the largest general-purpose EU e-Infrastructures (EGEE, DEISA and NDGF), serving every day thousands of researchers. The ARC, gLite and UNICORE stacks thus constitute the basis for the creation of the open-source Unified Middleware Distribution (UMD), that the future European Grid Initiative (EGI) will make available to the national Resource Providers as a key integral part of its offer and business model. The availability of certified grid services that can be easily downloaded from a common UMD repository, together with the set of common procedures, policies and rules that EGI will establish will make it easy for the research teams to access and share all type of IT resource and data made available by their national resource centers and funded at national level. The EGI Middleware function needs to continue to guarantee the current level of quality of the deployed services to the European scientific user communities during the transition and in the initial years of consolidation of the new EGI organization. During the transition to the new sustainable European organization embodied by EGI, the middleware currently represented by these stacks and other identified services needs to continue to be supported, maintained and further developed, in particular in view of emerging standards, and, in some parts, completed and hardened from their current stage to fully satisfy the operational quality requirements. The requirements will be established by the Middleware Coordination Board, including representatives of VOs, Operations and Middleware development teams, which represent a straightforward evolution of current best practices of EGEE, DEISA and other national experiences. The maintenance, development and evolution towards standards for these three EU stacks and related middleware projects is currently co-funded by national Institutions or Consortia and by the CEC via competitive bids.

[edit] ARC

The Advanced Resource Connector (ARC) is developed by the NorduGrid collaboration (http://www.nordugrid.org/) and associated projects since 2001. It features a decentralized architecture, leading to high efficiency, low maintenance costs and robust performance. It is highly portable and is available for all major Linux flavors. This in turn allows a decentralized deployment of ARC in over 60 sites, with over 30.000 cores. In particular, ARC is adopted by the NDGF (Nordic DataGrid Facility) to support the world’s only distributed heterogeneous Tier1 center. Currently, the next generation of ARC is under development, which minimizes dependencies on third-party components, improves extensibility, interoperability and allows portability to non-Linux platforms. The NorduGrid consortium was established in 2001, and currently includes 5 academic institutes from Scandinavia and Finland. It is based on a general Memorandum of Understanding (MoU), which is not legally binding. The MoU establishes the Steering Committee and the Chairperson and defines their duties. The consortium has no termination date and has no resources owned collectively. NorduGrid currently carries out consultations towards establishing an international “ARC consortium” as a legal entity in very near future, before the EGI start. NorduGrid thus guarantees middleware support, maintenance and further development of ARC beyond the scope of the current projects.

[edit] gLite

The gLite middleware is the result of a truly pan-European development effort made by the EDG-EGEE project series started in 2001 and co-financed by EC via competitive bids. The services offered by gLite provide the backbone of the EGEE infrastructure, is the largest multi-disciplinary grid infrastructure in the world, which brings together more than 140 institutions to produce a reliable and scalable computing resource available to the European and global research community. At present, it consists of approximately 300 sites in 50 countries and gives its 10,000 users access to 80,000 CPU cores (largely commodity clusters with some HPC systems) and to very large (>15 PB) distributed storage systems around-the-clock, processing up to 300.,000 jobs per day from scientific domains ranging from biomedicine to fusion science. The gLite middleware consists of an integrated set of components compliant with open standards and covering all the aspects of the Grid infrastructure. It is developed for the Scientific Linux environment, but extensive effort is currently dedicated to make it more widely available on other platforms. The gLite software is managed in all its phases, from development to testing to release, with the tools provided by ETICS (eInfrastructure for Testing, Integration and Configuration of Software, an EC-funded project). The gLite community is actively pursuing the constitution of the gLite Open Consortium with the goal to maintain and evolve the gLite middleware beyond the EGEE series of projects, to provide a long-term sustainable roadmap for the gLite software to meet the needs of its diverse user community. The Consortium will be established as a not-for-profit organization and will be open not only to the Institutions currently providing components but also to any other partner willing to contribute to the Consortium goals. The organizational model for the software development and related activities will be based on teams fully responsible for the individual middleware components; the coordination among the teams will happen within a Technical Coordination Board, lead by a Technical Coordinator.

[edit] UNICORE

The UNICORE middleware (http://www.unicore.eu/) has a traditional HPC background (since 10 years). It is used in HPC-related infrastructures like DEISA (serving a similar amount of CPUs as in EGEE but concentrated on a few powerful supercomputers) and in the future PRACE (European PetaFlop/s Supercomputers), but also in non-HPC-focused NGIs like D-Grid and some Swiss SwiNG projects. UNICORE is characterized by its open, extensible, lean, and interoperable Web services architecture which supports many open standards, providing a seamless, secure and intuitive access to Grid resources. UNICORE comes with a strong focus on workflow capabilities, security, application support and ease of installation and configuration. Since 2004 the UNICORE middleware is open source under a BSD license and publicly available at SourceForge (http://sourceforge.net/projects/unicore/). It is developed by the open source developer community of UNICORE with a set of core partners who contribute major elements of the software and are responsible for the development of the core components as well as for the release management. Institutions that have a long-term interest in the UNICORE Grid technology have joined the “UNICORE Forum e.V.” (http://www.unicore.eu/forum/) established in 1999; its legal status is a registered, open, non-profit association according to German law. Its objective is to promote the development and distribution of UNICORE without being bound to limited durations of funded EU or national projects. Currently the UNICORE Forum e.V. has 32 members comprising research institutions as well as commercial organizations. The Technical Advisory Board of the UNICORE Forum e.V. develops the future roadmap and strategy of the UNICORE development. It evaluates technical proposals, discusses technical solutions to be implemented in UNICORE and thus drives and monitors the open source development process of UNICORE.

[edit] Guidelines for the Unified Middleware Distribution (UMD)

The Consortia mentioned in the previous section have agreed that the middleware components, tools and services they presently support have to evolve into a Unified Middleware Distribution: UMD. UMD will contain components satisfying the needs of the user communities and of the resource providers and conforming to quality criteria defined by the EGI.org Middleware Unit and endorsed by the Middleware Coordination Board. The quality criteria that will select which components will be included in UMD include:

  • Interoperability: services included in a UMD release should be fully interoperable with all other UMD implementations adopted in the EGI Grid infrastructure.
  • Completeness: the set of available components and tools included in UMD once adopted by any NGI should allow the national infrastructure to be operated in a fully self-functional and autonomous way and at the same time completely integrated with the rest of the pan-European EGI infrastructure. The Grid services included in UMD should address the needs of all current VOs and a process should be in place to allow them to evolve according to the requirements of new scientific communities.
  • Scalability: available services should allow the management of resources and services in an e-Infrastructure which must satisfy scientific user communities ranging in size from a few to thousands of researchers. Different service implementations should be included to take into account both the need of simplicity for small user communities and scalability for the largest ones. In addition the services should be able to face the expected growth in scale (in terms of users, services and sites operated) over a short time period.
  • Simplicity: UMD should contain tools to download the appropriate services, to provide assistance during their configuration, and to perform as much automatic set up as possible.
  • Extensibility: UMD must provide interfaces (and “hooks”) that allow independent development (by any interested party) of higher level and additional services that will create a software pool from which further UMD innovation will be drawn. Gateways to the other (EU and non-EU like Globus) grid systems and components will be one example of services built on the extensibility interfaces.

Given the wide variety of needs, it is acceptable that different implementations of the same service or of the same interface are available at the same time in UMD, provided they are actually requested and conforming to the UMD quality criteria. However, wherever possible, a progressive specialization of the different services will be pursued, so to avoid unnecessary duplication of effort.

[edit] Role of the EGI.org Middleware Unit

The current environment of EU middleware development consists of distributed multiple teams of experts specialized in one or more services in general organized around the three Middleware Consortia with additional teams with complementary expertise belonging to other EU and international initiatives. In order to leverage the existing clusters of competence it is then advisable to maintain this decentralized model based on autonomous teams while introducing with EGI an effective pan-European technical and financial coordination. The decentralization will also leave open the road for the introduction of other development teams, including eventuallyteams who develop components on a commercial basis. The central technical coordination of the development teams will be supported by an EGI.org unit, called Middleware Unit (MU), lead by the Chief Technical Officer (CTO). The main objective of e the MU is to guarantee the availability of the required middleware services at the pan-European level with the assistance of additional technical bodies including the relevant experts appointed by the Consortia. The coordination requires the establishment of bodies like a Middleware Architect Group and a Middleware Technical Management Group to follow, respectively, the high-level long-term architectural issues and the daily execution of the plans. The exact scope and function of these bodies, which complement the MCB, where representatives from the middleware community meet with representatives of users and operations, will be more precisely defined when this general proposal will be ready for implementation to guarantee the availability and evolution of the UMD distribution and repository. The EGI.org and its technical bodies should be the unique place in Europe where the needs concerning the middleware for EGI will be planned and coordinated, in particular with respect to:

  • Common baseline architecture.
  • Full interoperability of existing services through standardization.
  • Validation and testing of the released services included in UMD.
  • Increasing complementarities and specializations of the included services.
  • Adoption of application and operation requirements.
  • Convergence and interoperability through the implementation of standard interfaces with Globus and other non-EU stacks.
  • Definition of additional interfaces to allow independent development of higher level services.

Special care needs to be taken to assure that the UMD software components are easily installed and configured. The goal of UMD is to make it as easy as possible for the NGIs national resource providers to deploy, maintain and use the grid services that need to guarantee to the VOs teams a uniform access to their resources. Another important objective for the EGI.org MU is to provide the necessary testing and certification of the services included in UMD to guarantee a seamless operation and interoperation of all the components included in UMD. This will also include provision of test suites for Quality Assurance and to validate standards compliance of new or modified existing services. To guarantee these functions in an efficient and effective way it is necessary that the MU could rely on appropriate tools for software configuration, build and test that the MU has to make available also to the Middleware providers. In addition, the MU will establish effective collaborations on an equal basis to promote the inclusion, with their related support, of services coming from outside Europe (like for instance Condor and Globus) compliant to the same set of EGI rules. In summary the tasks under the responsibility of the EGI.org Middleware Unit are reported in the following table, taken from the Blueprint.

MW Tasks in EGI.org FTE
Maintain and document processes and quality criteria common to all the middleware providers. 1
Provide and support tools to enable and monitor the processes (such as configuration management system, bug and task tracker, wiki). 1
Define quality and conformance criteria that UMD components need to satisfy in areas such as security, performance, scalability, functionality, usability, interoperability, adherence to standards. Verify that accepted components are certified according to the agreed process and satisfy the quality and conformance criteria, specifically aimed also against security vulnerabilities. 3
Maintain a repository of certified middleware components or references thereto. 2
Follow the daily execution of the strategic plan endorsed by the MCB. Promote the EGI participation to standardization bodies. 1
Sum of Resources in EGI.org Middleware Unit 8

[edit] Components and Services proposed for inclusion in UMD in the first stage of EGI

The main services developed by ARC, gLite and UNICORE that are extensively used on the current production infrastructures by thousands of researchers from many different scientific disciplines are summarized in the following table. The information has been originally provided by the OMII Europe project (see http://omii-europe.org/OMII-Europe/docs/DJRA20.pdf)), but it has been updated since deliverable D3.1. It is advisable that the maintenance and further development of these services continue to be supported in the EGI scenario.

OGSA capability ARC gLite UNICORE
Security.Accounting SGAS, APEL DGAS, APEL OGSA-RUS + UR
Data.Management.Storage Smart-SE, dCache ARC Gridftp StoRM, DPM SMS
Data.Management.Transfer FTS, GridFTP2 FTS, GridFTP JMS, GridFTP
Data.Access.Relational OGSA-DAI (future)
Data.Access.FlatFiles ARC Caching GFAL TSI
Information.Model GLUE, arcschema GLUE GLUE
Information.Discovery OpenLDAP, WSRF OpenLDAP CIP
Information.Monitoring NG-Monitor GridICE,R-GMA LLview, CIS, RSS
ExecMan.ExecService Grid-Manager with A-REX (BES) or GridFTP interface GT2 Gram, CREAM + BES TSS,OGSA-BES
ExecMan.JobManager ARC Client WMS XNJS

Deliverable D3.1, “First EGI functions definition”, includes a more detailed description of the mentioned components.

[edit] Cost stimates for Middleware maintenance

An approximate evaluation of the effort needed by the three Consortia to maintain and support the existing middleware components and to adopt standards aimed at interoperability is about 70 FTE. The estimation includes all the phases of software preparation, from development to integration, full testing and packaging. Only the final conformance tests are under the responsibility of the EGI.org Middleware Unit. The table below classifies the estimate of 70 FTE

Security 1 8 1.5 10
Data Management 5 9 1.5 15
Job Management 8 10 3 24
Information System 3 6 1 10
Other 4 5 4 11
Total 21 38 11 70


The description of the Middleware function in EGI provided in this Chapter and in the Blueprint refers to the first few years after the establishment of EGI. In the longer run the middleware components should evolve into services that may be charged to customers and for which the maintenance and support may be more easily outsourced also to commercial partners.

[edit] User Community Services

This section discusses the proposed organisation of the User Community Services. The first draft of these functions was referred to in D3.1 as “Application Support and Training” or “Extended Support Services”. Following several rounds of feedback by the NGIs and by representatives of the user communities, and considering the intuitive purpose of these functions, the descriptions have been modified in parts and the entire section has been renamed User Community Services (UCS).

These services include activities such as:

  1. Gathering requirements from the user communities and providing efficient channels for their representation vis à vis the middleware – and other software – providers.
  2. Carrying out a review process to integrate useful “external” software – i.e. software packages that can help application developers use the Grid infrastructure, but are not part of the core Middleware distribution(s)
  3. Establishing Science Gateways that expose common tools and services (e.g. workflow engines, web services, semantic annotation) in a transparent and user-friendly manner to user communities in the various disciplines – see also http://knowledge.eu-egi.eu/index.php/Science_Gateways.
  4. Establishing technical collaborations with the large European Reasearch Infrastructure projects (e.g. ESFRI) in support of customers of the European Organisations.
  5. Providing “umbrella” services for collaborating projects, to streamline information management tasks and ensure some continuity of service between project cycles (e.g. maintenance of repositories, FAQs, wikis, etc.)
  6. Maintaining a European Grid Application Database that allows applications to be “registered”, permitting people to search for similar applications and contact the authors for guidance.
  7. Organising European events such as the User Forum meetings and topical meetings for specific user communities.
  8. Providing services for new communities – e.g. “Front desk” services, VO creation counselling, etc.

See also http://knowledge.eu-egi.eu/index.php/UCS_Front_Desk.

  1. Ensuring that the user communities and Grid administrators are provided with high quality documentation and training services.

See also http://knowledge.eu-egi.eu/knowledge/index.php/Documentation_and_Training

This is carried out mainly by the NGIs in the context of a structured network of User Community Services, with coordination by a small team in EGI.org. Activities such as providing support to porting activities and training of the users and administrators is generally delivered through NGIs, either on a national level or via specific agreements with other NGIs in the context of the creation of a Specialised Support Centre (see below).. To avoid excessive length in this document, most of the content from D3.1 is not repeated here; where relevant (e.g. on topics such as Science Gateways or a possible Front Desk Acquisition process), the reader is referred to parts of the EGI Knowledge Base at http://knowledge.eu-egi.eu.

[edit] The EGI User Forum and Specialised Support Centres

The two main organisational entities in EGI which provide representation and services to the user communities are the EGI User Forum (UF) and the EGI Specialised Support Centres (SSCs). The EGI User Forum was introduced in the final EGI Blueprint in response to requests by several NGIs and representatives from the user comunities. It is a body specifically designed to provide representation to the various communities (e.g. Bioinformatics, Earth Sciences, new small communities, etc.). The EGI User ForumUF is established by the communities themselves and is headed by a Steering Committee (UFSC), which interacts directly with EGI.org management, and with the EGI Council via a Chairperson, who is a non-voting representative in the Council.

The User Coordination Officer (UCO) in EGI.org and the related UCS team are of course expected to interact with the UF at a central level. As the main managerial body that represents users in EGI, the UF will include representatives from any number of user groups – national, international, thematic, and “functional” (e.g. new and small user communities). In EGI.org, the User Coordination Officer (UCO) and the related UCS team are of course expected to interact with the UF at a central level. The EGI SSCs are also established by the user communities, as is any support centre in Europe. However, in the context of EGI, an SSC is defined as a centre (or cluster of activities) that has a formal relationship with EGI. The characteristics of this relationship are to be determined in the proposal phase of EGI; some initial suggestions are given here to aid in the preparation of relevant proposals.

It is expected that each SSC will have a User Forum Representative, but the UF Representatives (UFSC members) may not all be associated with an SSC. The overall organisation of the EGI UF and the SSCs with their main interfaces in EGI is shown in Figure 2:

Image:UCS_EGI.jpg Figure 2 User Community Services in EGI

The SSCs will contribute to collecting and transferring requirements and feedback from the user communities to EGI via a User Technical Support team covering the day to day technical needs in cooperation with the Operations help desk team, and a Grid Planning team which participates in the EGI Middleware Coordination Board, which provides more long term technical planning and may establish other advisory committees to work with the EGI.org director. An SSC could also have Front Desk services, as described more in detail elsewhere. This option would be particularly advisable for an SSC dedicated to new communities. SSCs, as well as NGIs in fulfilling their international tasks, will in addition collaborate in several other interdisciplinary UC Services, including contributing to the EGI Application Database, the External and UMD Candidate Software Review (similar to EGEE RESPECT), the building and maintenance of wikis, repositories, gateways, etc. These tasks are considered proper EGI UCS tasks, and the SSCs will be supported by EGI in fulfilling them. On the other hand, it is assumed that SSCs will be able to leverage support from their communities and collaborations from other relevant projects. A large SSC could then be structured as in Figure 3:

Image:bigSSC.jpg Figure 3 Example of a large SSC

[edit] Today’s

Image:currentSSC.jpg There is no obligation for an NGI to support or endorse any particular SSC. SSCs will be encouraged to have a clear procedure in place to allow new members to join at a later date, or – if appropriate – to allow a community within an NGI to make partial use of its services, which would be properly acknowledged by the relevant NGI. An SSC is expected to have European scope and visibility. In some cases, the SSCs will take over some functions of the existing EGEE strategic discipline clusters connected to specific international communities, and could be highly structured themselves. While these centers are not all expected to structure themselves according to a given template, it is assumed that some elements should be in place to facilitate communications related to EGI policy issues, such as participation in the User Forum Steering Committee, and the usage of the general technical services outlined elsewhere.

[edit] EGI SSC Guidelines

The set of guidelines described in this subsection is an initial proposal aimed to provide some structure with respect to the UCS layer in an SSC. Again, it is assumed that as SSC is governed by its user community. The guidelines below only refer to the aspects of an SSC which are relevant to its relationship with EGI.

         Establishment of an SSC

In the proposal phase, the EGI partners (NGIs and experts) will make specific proposals for initial SSCs. and establish the User Forum and its Steering Committee. Once EGI is established, the EGI Council will be responsible for evaluating proposals for new SSCs in consultation with the UFSC. Details and formalities on this process are not addressed here

         General Rights and Responsibilities

The SSCs are assumed to have a European-level existence. An SSC must have a cohesive community behind it that is able to take ownership of the SSC and to drive its evolution. Most SSCs will be created around scientific domains. However, an SSC may also be created to fulfil certain “functional” requirements (e.g. a Training SSC, or one for new and small communities). The SSC will have representation in the EGI Council, Middleware, middleware Coordination Boardgroup, and other appropriate bodies. The SSCs must have the ability to feed their technical and non-technical requirements into EGI. The SSCs will also be able to interact with each other via the UFSC and other means. The SSCs are expected to be "good citizens" of EGI and to follow the defined EGI policies (both security and operational policies). The SSCs are expected to be (relatively) stable entities. The UCS layer in an SSC may be “readjusted” to meet the requirements of the user communities being served, and where an adjustment in personnel is requested, this will be negotiated with the UFSC and EGI Council. However, the SSC itself will evolve independently from this layer. EGI may provide resources to the SSCs,. in particular to support the central UCS layer of services (mainly manpower). EGI will also have a mechanism to provide seed resources for new communities, as discussed elsewhere. SSCs will have access to resources. This includes mechanisms for making their own resources available and potentially (priority) access to centralized services, for example, help desks, central grid services, etc. This includes access to "community services" that are made available to the full EGI user community such as operations help, middleware help, etc. SSCs will also have access to training and documentation. The SSCs will report facts and figures about their use of EGI in order to help EGI (and its funding agencies) understand the scope of the work accomplished with EGI. The SSCs are expected to operate transparently allowing a clear view of the SSC's activities and making the list of provided services available to the full EGI community.

         Typical UCS Personnel in an SSC

An SSC will have a high level User Forum Representative, who can nominate a deputy. For large SSCs, this individual is automatically a member of the EGI UFSC. An SSC will have a Grid Planning Officer who participates in the EGI MCB meetings. An SSC will have personnel for User Technical Support and other tasks. An SSC will have personnel for assisting with dissemination efforts and (web) content management. An SSC might have a Gateway Officer to coordinate the development and maintenance of its Science Gateway.

Several user communities have already been contacted, and many are producing hypotheses on specific SSCs. An articulated example is provided for the Life Sciences in Appendix C. This case is also under review by people involved with the ESFRI projects related to Life Sciences (in particular ELIXIR), who have graciously shared their materials.

[edit] NGI International Tasks including SSC Tasks

There is currently a projected budget of 9.9 M€ per year – equivalent to 110 FTE – for all the NGI international tasks, including the SSC UCS layer. It was initially assumed that roughly 50 FTE would be allotted to the SSCs which provide continuity to the current scientific clusters in EGEE and other projects, but depending on the requests by the communities there may be more NGI international tasks that could be clustered as SSC-related services. At this stage, it is hypothesisedhypothesise that the international (“central”) services will require the kind of effort described below. It should be kept in mind that the ranges of manpower indicated are deliberately broad, to accommodate the heterogeneity of the communities’ needs for UCS services. For instance, one may assume that in the short term there will two very large SSCs (for Life Sciences and Earth Sciences) which may require 9 to 15 FTE (the latter also including activities with several collaborating projects) – hence the upper bound of the estimated effort for SSCs. Thus the global estimated ranges for manpower presuppose a rough division into “small”, “medium” and “large” SSCs, as was done with the other NGI International tasks. Also please note that the UCS tasks for the NGIs include only the NGI international tasks that are not otherwise allocated to SSCs. NGI national tasks are not included here; this is a departure from the global estimates at the end of chapter 8 of D3.1, where an estimate was attempted for some common national tasks.

[edit] User Forum

  • Representation in EGI

The UF Representatives (UFSC members) represent their user communities at the EGI.org management level, collectively as an advisory body, and in the EGI Council via the participation (as a non-voting rep) of the UFSC chair.

  • Coordination

Coordination of all of the activities that concern a given user community. This includes common problems with the middleware, development of high-level services, political issues concerning SSC, funding, etc. The coordinators, led by their UF representative, will interact “externally” with the EGI council, EGI.org administration, middleware coordinators, and (other) SSC coordinators. They must also provide information about the activities within their community or SSC to external parties and disseminate information from the external parties within their user community. The main coordinator or UF representative is likely to be a researcher for scientific SSCs, as technical competence in the field is usually necessary to understand how grid technologies can advance the scientific research. The UF Reps are members of the UFSC, and can count on the collaboration of the entire EGI.org UCS team.

  • Feedback

The UFSC should include members from all the EGI communities, and the process of collecting, evaluating and representing user needs is one of the primary purposes of this body.

  • User Conference – User Forum Events

Organization of a large conference for users of the grid infrastructure. This involves several parties, including the UFSC, the dissemination officers from SSCs and NGIs and other members of the UF, and is coordinated in EGI.org by the Event Organisation and User Forum Support team and the dissemination officer. We assume that each SSC will have a UF representative, plus often one deputy. These can be part time assignments but there need to be named personnel for this task.

Range of effort
SSC Other NGI International effortInternatl Total
8 ~ 12 4 ~ 7 12 ~ 19

[edit] Dissemination and Events

  • Public Relations

Dissemination of grid activities/technologies within a particular (scientific) community. Make dissemination efforts and their results available to EGI. This typically happens through direct interactions between scientists and through the domain’s conferences. Additionally, EGI should sponsor/organize meetings for specific scientific disciplines. Direct interaction between SSC and its user community. Interaction with EGI to obtain funding for grid-focused meetings. Funds for organizing meetings can be drawn from various sources; an SSC can also choose to “convert” its budget, currently expressed in FTE, into monetary funds. Logistical support for those meetings. Most efficient dissemination comes through others in the same field. Probably also need general dissemination for general public or for new communities.

  • Events

The dissemination teams are also involved in the planning and organising of events such as the User Conference. The UF is expected to coordinate dissemination efforts and event organisation. In the large SSCs there may be some extra assistance by 0.5 to 1 FTE (e.g. a Dissemination officer). It is also auspicable that the central EGI.org dissemination team has some “central” assistance – perhaps on a rotating basis – by dissemination experts in the NGIs, for a total of 1 or 2 FTE In addition, it is expected that some collaboration on dissemination activities is provided by each NGI as an international task.

Range of effort
SSC Other NGI International effort Total
3 ~ 6 4 ~ 6 7 ~ 12

[edit] Front Desk / Acquisition and support to New Communities

The process of bringing new communities onto the infrastructure can be relatively simple or very challenging. In D3.1 we proposed a schema for a relatively large community, which is now available at http://knowledge.eu-egi.eu/index.php/UCS_Front_Desk, and is expected to evolve at that site. The following tasks are classified as Front Desk activities. Note the interaction of these activities with others within UCS and in Ops and MW, as well as the new communities themselves. This task is currently very labour intensive, and may require a “functional” SSC in the initial phase of EGI. It is expected, however, that the acquisition of new communities should evolve into a simpler process as the infrastructure itself evolves.

  • Consulting

This service requires direct interaction with application developers to get their applications running on the grid infrastructure, and with middleware (and RESPECT software) developers to understand the best ways to use their services/APIs. It also requires interaction with other support activity personnel and with SSC leaders to identify “clients”. In EGI.org, the UCS Consultant for new communities is expected to coordinate these activities. The bulk of this activity is expected to be taken up by the SSCs and collaborating projects, and could require specialised effort from the hypothesised SSC for new communities. Consulting teams in general interact with the personnel that manages informational resources, here referred to as Technical Coordination B.

  • Integration of Domain’s Resources

The integration of a user community’s computing resources with the grid infrastructure. This includes community data sources, standard computing services, and/or instruments. This team interacts with middleware developers or with the application porting group if it exists. Direct interaction with their user community to understand what resources need to be interfaced to/made accessible from the grid. Interaction with NGIs regarding integration of hardware resources; expect EGI.org to provide contacts for NGIs. The team must also have access to reasonable documentation and support to interface resources to the grid infrastructure. In turn, it will provide information to the community on how to access its resources via the grid. If these are generic resources, then this information should be shared with other communities

  • Assistance for Application Porting

This service provides information concerning the porting of applications to the EGI infrastructure and on the integration of grid services with the application. It interfaces with application developers, either via GGUS or other forums (mailing lists, chat rooms, etc.); and with core middleware developers and “integration software” developers (e.g. the RESPECT software) to understand how the software can be used effectively. At the EGI.org level this activity is seen as a shared responsibility of the SSC coordination team and the Front Desk personnel, with assistance where needed by the technical coordinators. Specialised help desk personnel are involved in this effort. It is expected that these activities will be strongly sustained by the SSCs, plus perhaps 2 or 3 general consultants sought from the NGIs – i.e. an NGI can opt to devote some of its international UCS effort to this task. In alternative to these consultants, there may be specific dedicated effort for this task in a possible Training SSC or an SSC for new communities. The effort table below assumes no such SSC, but this may be subject to change.

Range of effort
SSC Other NGI International effort Total
8 ~ 16 2 ~4 12 ~ 20

[edit] Documentation and Training Coordination

As mentioned in the previous task, it is possible that a dedicated Documentation and Training SSC might be proposed; thus the global UCS effort expected for this task may vary from current assumptions, which for the moment are that this entity is not in place. Here we give a brief overview of the task. For a more detailed discussion please see http://knowledge.eu-egi.eu/knowledge/index.php/Documentation_and_Training.

  • Documentation

Systematic review of documentation produced by entities within the project. Organization (indexing) of the available documents along with some information about their quality and whether they are current. Additionally, high level documentation that treats the grid infrastructure as a coherent system must be written and maintained. As this type of documentation is above any particular service, middleware developers cannot really be asked to provide it. Instead, EGI – either as part of the NGI international tasks or by means of a dedicated SSC, must employ technical writers to create this documentation and to keep it up to date. Support for multiple middleware stacks will complicate this task and will probably require dedicated manpower for each of the supported stacks. In EGI.org, the Documentation Review Coordinator is responsible for all these activities, with the assistance of the UCS: Technical Coordinator B and the middleware coordinator.

  • Training Coordination

Related to the high level documentation is the creation of training courses geared to 1) new users, 2) application developers, and 3) system administrators. Training is required by operations centres for system operators, by application developers who are developing programs to use the system and by users to allow them to access the services. Training is also required for trainers and educators regionally to support them in disseminating experience of changes in the system which they must then pass on to their communities (local and in different user communities / VOs). In EGI.org, the Training Coordinator is responsible for all these activities, with the assistance of the UCS: Technical Coordinator B and the middleware coordinator. Since the EGI.org effort for this task is minimal, it is expected that the central team will require some assistance in the amount of 1 or 2 FTE. In addition, the larger SSCs may have effort dedicated for creating documentation specific for their user communities – perhaps 0.5 or 1 FTE. This does not include documentation for Applications, which is not in the domain of the EGI project. National efforts in this area are not considered here, but are assumed to be present.

Range of effort
SSC Other NGI International effort Total
3.5 ~ 7 1 ~ 2 4.5 ~ 9

[edit] Technical Coordination A – Operations related

  • VO Registration and VO Database

EGI will have a central VO database with an interface for VO registration. These tools are part of the Operations portals described under Operations tasks O-E-4 / O-N-4, and under the responsibility of EGI.org Operations and UCS: Technical Coordinator A. This service includes running the VO registration process, including providing support to VO managers and validating provided information; interface with VO managers for registration, with developers of other services regarding extent, format, and access to registration information, and with operations to ensure comprehensive configuration information is provided. VO registration is performed in collaboration with Front Desk personnel.

  • Site Validation Tests

EGI will also ensure maintenance of a battery of reusable site validation tests in support of VO managers and VO members, and interfacing with Operations and Middleware personnel to provide help to the VOs in using these tests. This service presupposes that the SAM infrastructure (or equivalent) will be available in EGI. We also assume that there will be some mechanism (CVS, SVN, etc.) for versioning and maintaining code for the tests. The creation, maintenance and availability of these tests will be ensured by a small central team consisting of operations and UCS personnel. The actual running of tests is the responsibility of the VOs, with some assistance in the UCS layer of the SSCs Many actors do some level of testing, but it is the SSCs that will do detailed testing in close collaboration with middleware providers. We expect that an SSC may have some dedicated effort for this task. In EGI.org the central service is the responsibility of the operations team and the UCS: Technical Coordinator A.

  • Core VO Service Provision

These services (VOMS, LFC, etc.) are already described in Operations O-E-14 and O-N-8. Their provision for all VOs running on the grid infrastructure should be guaranteed by EGI.org, but probably actually run by various NGIs. The UCS: Technical Coordinator A will be responsible for working with his or her operations counterpart within EGI.org to handle requests for the deployment of core services. The NGIs must provide part of their international effort to sustain this task, which must be guaranteed by the operations teams.

  • Help Desk and User Technical Support

The description of the Grid User Support and ticketing system is given in Operations tasks O-E-6, O-E-7, and O-N-6, O-N-7. These activities require a UCS element to interface directly with users of the grid infrastructure on issues of documentation and utilization of the grid. It is expected that the NGIs will provide this kind of service, in collaboration with their operations counterparts. It is also expected that the large SSCs will have a User Technical Support unit which should include some (part time) effort on the UCS side, providing help desk support focused on using community-specific software, services, data sources, etc. These activities would in any case use the common ticketing system to interact with users and other supporters. This task is in part the responsibility of Technical Coordinator A and the personnel involved in SSC coordination.

Range of effort
SSC Other NGI International effort Total
4 ~ 8 3 ~ 6 7 ~ 14

[edit] Technical Coordination B – Information Tools

This activity is also involved with front desk activities, VO registration, and making sure that informational tools are integrated with the Science Gateways. In addition, the task may be responsible for designing and maintaining these gateways, in which case the relevant SSC must seek personnel with specific competences for the scientific field in question.

  • Case Studies

Providing written case studies of applications that have been successfully ported to the grid infrastructure. These serve as guidelines for future (similar) applications. Works with application developers to port their applications to the grid. From this case studies are written. Expect these studies to be made available through the EGI. The written case studies are a mechanism to document application porting techniques and to provide a guide for future applications. Expect this to be done for each application receiving “consulting”; therefore this task interacts closely with the Front Desk teams. This is the responsibility of the SSCs and the NGIs. At a central level a depository or wiki should be provided, under the responsibility of Technical Coordinator B – see for instance [ reference / url ]

  • Application Database

A central database containing information about the applications running on the grid infrastructure. Serves as dissemination tool and a support resource. This should be the responsibility of EGI.org, centrally coordinated by UCS: Technical Coordinator B, possibly with some assistance by a dedicated person sought among the NGIs international tasks. Expect end users and funding agencies to access the database, and contributions from the user community to provide information. The central team should interact with various user communities to understand what information is relevant and how to make database easily accessible and easy to use. Any effort to populate the database is considered NGI international effort. In addition, it is hoped that there will be “external” effort from collaborating projects – in particular projects aimed at new communities.

Range of effort
SSC Other NGI International effort Total
4 ~ 8 3 ~ 7 7 ~ 15

[edit] Grid Planning

  • Development of Services

The EGI ecosystem expects the continuation of development efforts for high-level grid services and APIs (both generic and highly customised) in service of its user communities. In the EGI project there is a structured set of services for collecting and evaluating user requirements and transmitting them to development teams, for instance through the MCB and the RESPECT process, as well as by means of the teams which assist with application porting. However, there is no budgeted effort for actual development in the EGI project. The SSCs are then expected to take on this responsibility, which should be properly acknowledged at the level of the EGI Council and the funding agencies. The SSCs will have access to appropriate documentation and support for interfacing new services to the core grid services, and in turn they should provide the relevant software to their user communities. If the services are generic, dissemination (and support) of services should also be offered to other communities. At the EGI.org level, the SSC coordination team will oversee these activities to prevent duplicated development and to encourage sharing, in collaboration with the Grid Planning Coordinator. Global effort for actual development is then 0; however, each large SSC should have a dedicated Grid Planning Officer to oversee the processes mentioned above, participate in MCB meetings etc.

  • Coordination of Grid Planning

Much of the technical coordination between difference disciplines now happens within the NA4 Steering Committee or through TMB working groups with strong NA4 participation. To avoid duplication and ensure a coherent evolution, this technical coordination must continue in the EGI era. Expect to be able to raise issues with the infrastructure and to be able to influence the priority for resolving them. Expect that EGI.org will provide the coordination/tracking of raised issues. SSC coordinators should develop a consensus within their communities regarding issues and their priority. Provide funds for sponsoring these meetings and for inviting strategic people to attend.

  • Feedback

Providing feedback to EGI with respect to the middleware requirements, the utility of the services, operational problems, and administrative processes. The SSCs will interact with their user communities and then provide collected information / experiences with the relevant coordinator in EGI.org. This is part of the Grid Planning activities in collaboration with members of the User Forum. Each large SSC will have a UF Representative who is a member of the User Forum Steering Committee. We assume 8 FTE for these.

Range of effort
SSC Other NGI International effort Total
4 ~ 8 2 ~ 4 6 ~ 12

[edit] Science Gateways / Portals

For any structured scientific community, the grid is useful insofar as it provides added value to the work of that community – i.e. if the work is carried out in a manner that is easier, faster, cheaper, etc. For many communities, this entails the presence of specific tools that are easy to find and use for that particular community. The one-size-fits-all model is not appropriate for end users who need to use their particular applications, have their particular language, and are accustomed to particular kinds of interfaces. Hence the idea of Science Gateways, which expose these specialised services to specific communities and are built by (or in consultation with) people who know the specific user community. In D3.1 we gave an initial description of the purpose and organisation of science gateways, and more information can be found at http://knowledge.eu-egi.eu/index.php/Science_Gateways. It is strongly recommended that the SSCs design, build, and then maintain these tools for the benefit of their communities. These gateways can be initially simple, and evolve from currently existing portals, or they can be very sophisticated. In all cases, it is recommended that there be named specialised personnel (e.g. people with strong experience in content management and the ability to make decisions) in charge of the content and structure of these sites. The creation of a gateway may initially take more effort than its subsequent maintenance; this effort, however, should never be 0. The table below gives estimates for the initial effort on the gateways.

Range of effort
SSC Other NGI International effort Total
6 ~ 12 0 6 ~ 12

[edit] Other NGI International Tasks

The estimated ranges of effort for the above tasks should not be added up without considering the discussion at the beginning of this section. However, we expect that compatibly with the guidelines for the tasks given above there is still a fair budget (perhaps in the order of 15 ~ 25 FTE) for cooperative NGI international tasks. We will not give specific descriptions of these coordination activities, as each NGI needs some flexibility in assigning its international effort, and the basic recommendations by task have been given above. We recommend that some cooperative effort go toward the support for specific actions that are of interest to more than one large user community. In particular, both the Life Sciences and Earth Sciences communities have a strong need to work in a focused manner on fostering the integration between grid and cluster computing / supercomputing. Currently in projects such as EGEE the regional coordinators are responsible for interfacing between regional support and corresponding centralized support teams. They also provide overviews of user activities within their region and act as first-line support. Expect interfacing between users within a region and centralized support structures to continue. The regional coordinators will also continue to report about use of the grid within the region. Tools and information to effectively make use of centralized services. To report on activities within the region and to provide an efficient liaison between EGI.org and the regional user community.

Figure 4

[edit] The Role of EGI.org

EGI.org will provide overall coordination for the services described above, structured as shown in Table 2. Aside from the activities that are carried out by senior personnel and therefore directly associated with two full time employees, the estimated effort for the other activities are overall activity averages – e.g. event organisation requires more than 2 FTE in certain periods and less in others, documentation-related activities are often performed in conjunction with coordination of SSC activities.

Table 2: User Community Services in EGI.org
Coordination of SSC activities A small team of coordinators to assist the User Coordination Officer (UCO) in all collaborative activities, such as (1)-(6) above, attend meetings, and work with the Grid Planning team to organise the representation of user community needs, new software etc. in EGI management. 2
Services for new and small communities & Front Desk coordination This will include a Consultant for new communities and a Front Desk Coordinator. These personnel oversee the availability of seed resources for new communities, and works with Grid Planning in analysing new trends in typology of Grid users and new resources. 2
Event organisation & User Forum Support 2 people in EGI.org will provide liaison and support for the work of the UFSC, and coordinate the organisation of the main User Forum Events, plus others as needed in collaboration with their counterparts in the NGIs and SSCs. 2
Grid Planning & Technical Coordination A (help desk & other ops tools) One senior person to represent the UCS team in the Middleware Coordination Board and to liaise with any user committees that are established for technical representation and advisory activities with respect to the EGI Council and EGI.org management on behalf of their communities. One Technical Coordinator for all User Technical Support activities – e.g. Help Desk.

Representatives of the International user communities will be members of the MCB who will steer, define priorities and provide feedback to the technical work program of the EGI.org Director and the group of technical units in charge of the UMD component evolution and deployment. || 2

Technical Coordination B (information tools) and documentation One Documentation Review Coordinator. One Technical Coordinator to perform activities related to technical information gathering, content and material creation, and support of central services such as material repository and online resources. 2
Coordination of training efforts Covering the activities in (9) related to management and coordination of training efforts in the NGIs and management of Grid central services. 1
Sum of User Community Services in EGI.org 11

[edit] Function of egi: External LIAISON Functions

[edit] Task and services

[edit] Dissemination

A small team within this function will execute the dissemination activities of the EGI.org. The team will focus on content production and coordinating activities. Technical and specific services will preferably be bought from third party. The objectives of the dissemination activities of EGI.org are to ensure the visibility and inform about EGI among decision makers, funding bodies, research communities, industry partners and other grid initiatives in Europe and worldwide

  • to inform the user communities and NGIs
  • to arrange activities in collaboration with the NGIs
  • to create and maintain excellent PR/media relations
  • to coordinate publishing of activity and management reports
  • to organise events such as EGI conferences and user forums

The dissemination activities need to be effective and well targeted. For EGI the dissemination activities at large must be executed both by EGI.org and the NGIs with a clear division of responsibilities. The EGI.org will typically be in charge of tasks requiring coordination between NGIs. In general terms, the EGI.org will deal with the common actions of the EGI while the NGIs are responsible for the EGI dissemination in their local and regional areas. It is important to note that in order to achieve good results the dissemination team needs to act in close collaboration with the user-oriented, grid operational and technical activities of EGI. The dissemination team of the EGI.org will serve as a horizontal link between the stakeholders (NGIs) and existing user communities, and has therefore a central role in maintaining the information flow to these parties. A dynamic and up to date website is a key element in maximizing the visibility, providing support to users and stakeholders and informing about the EGI. It is therefore a clear need for a professional and dedicated web editor. The dissemination team of the EGI.org will support and coordinate the public relations publication work activities of the EGI. Press releases and Newsletters of the work and key achievements will be published and widely distributed in order to increase the visibility of the EGI. NGIs have to contribute by providing material to paper and electronic publications. The EGI.org will also be in charge of organisation of annual events and conferences, similar to e.g. EGEE User Forum and DEISA Symposium. These events not only increase the visibility and inform existing users but also aim to enlarge the user community. Also presence in other major events in Europe and outside will be coordinated and organised by the dissemination team of EGI.org, whereas NGIs are responsible of the EGI presence in local and regional events. The presence can be for example, a presentation, where the dissemination team would support in finding the right experts. NGIs who are active on the international scene represent themselves but are of course free to propose coordination of any international activities with EGI.org. Further in practical arrangements, such as drafting of abstracts, or planning for an exhibition booth. The core EGI.org dissemination team will be small at the beginning, but can be augmented by a rotation of 1-2 colleagues from the NGIs. The NGIs will further be requested to provide a contact person for the dissemination activities within their organisations.

[edit] Industry take up

It has been identified that sustainability of EGI would benefit from a persistent activity aimed at increasing participation of the private sector in the European Grid Infrastructure, bringing additional competences and financial resources to the initiative. As a publicly funded infrastructure aimed for research, the usage policies will be determined not only by EU policies but primarily by national law and policies. The usage policies can be expected to be comparable to those of other similar research infrastructures, such as the GÉANT network. and thus dedicated to research usage. BusinessComercial usage is then limited, and usage by the business and industry sectors has primarily to be in form of research collaborations with European and national research institutes, universities and other educational institutions. The EGI.org management must develop a business model for the grid infrastructure although it by essential qualities will be limited in its commercial potential. The general interest and potential use by industry can be very different. The most likelycome in many forms are; use of the EGI infrastructure in R&D (collaboration with the publicly funded research community). the EGI infrastructure as “state-of the-art”/”best practice” for industry. industry use of the EGI infrastructure for testing and learning. industrial projects with occasional exceptional requirements (critical computing on demand). Preferably EGI.org would initiate a work together with the stakeholders, to elaborate on policies for access for industrial research projects in the pre-competitive domain and industrial production projects accessing innovative technologies or deploying innovative strategies. The NGIs are expected to work along similar lines on a national level. Following the recommendations of the e-Infrastructure Reflection Group (e-IRG) Task Force on Sustainable e-Infrastructures, industry has to be seen as both a potential user and a service provider. Today it is possible to identify an emerging business based on the major European grid technologies. EGI.org should act positively towards such initiatives and establish policies allowing emerging companies and other initiatives a fair competition in providing services for the EGI.

[edit] Other External Relations

External relations are defined as relations with organisations and initiatives outside of the EGI and of direct relevance for the EGI in terms of collaboration or interoperation. Examples of such organisations and initiatives are:

  • Grids outside Europe
  • Commercial grids (e.g. cloud computing efforts)
  • Large-scale international research collaborations (e.g. the EIROForum organisations, ESFRI projects and WLCG)
  • Networking organisations (e.g. NRENs, DANTE, TERENA)
  • Policy and standard shaping bodies (e.g. e-IRG, ESFRI, OGF)

The EGI.org management and specifically the Director should be in charge of External Relations. This responsibility should primarily be focused on establishment of formal relations when necessary promotion of common understanding on policies in scope of grid interoperation influence on policy and Standards shaping activities networking and enlargement of the EGI “sphere of influence” The operational aspects in interoperation with other grids are handled by the EGI.org Grid Operations function. The activity is not actively pursuing standardisation work but handles the relations of EGI.org with organisations such as OGF, e-IRG and OASIS. EGI.org should consider membership in organisations like OGF and OASIS if it is found beneficial for EGI. The work could include coordination and reporting of participation in different Standards working groups and interfacing with the technical teams doing the actual standardisation. To maximise the outcome of the external relations activity the EGI.org management should encourage bridging to external organisations and initiatives through the NGIs.

[edit] outline of time evolution

The description provided in this chapter refers as usual to the first year of EGI. These kind of activities however are expected to be rather constant in time, with the very important exception of Industry Take-up, which will start very small, as a kind of feasibility study, but that hopefully will much grow, and may in future also require some change in the EGI structure, for better accommodating profit comercial partners.

[edit] Resources effort

Dissemination: FTE estimation: 3 2 (or 2) FTE for EGI.org and 0,5 for each NGI According to the above analysis the following expertise is proposed: A dissemination manager – 1 FTE for EGI.org A dissemination support person – 1 FTE for EGI.org (this can be combined with the one of the admin person at the EGI.org management function). A web editor – 1 FTE for EGI.org NGI dissemination interface for EGI – 0,5 FTE for each participating NGI. As said above 1-2 of the NGI interfaces can also further staff the EGI dissemination team.

Industry Take-up: FTE estimation : no additional manpower It is proposed that the EGI.org director and the management team cover these activities, in the first years of EGI. The effort could increase substantially in the subsequent years when effective way of collaborating with the business world.

Other: FTE estimation: 2 FTE for EGI.org. According to the above analysis the following expertise is proposed:

  1. A policy and external liaison manager – 1 FTE for EGI.org
  2. A standardisation liaison manager – 1 FTE for EGI.org

[edit] Functions of egi: management

[edit] EGI Council and its members

The main actors of EGI are the National Grid Initiatives (NGIs), which ensure the operation of the grid infrastructures in each country, as well as a transparent representation of the requirements of all their scientific communities together with resource providers and all e-Infrastructure-related institutions. The top level management layer in EGI is the EGI Council, which is constituted by the NGIs accepting the statutes. The NGIs govern EGI.org and voice their views on all EGI matters as voting members in the EGI Council. Other members of this body are the Associate Members, i.e. European institutions represented in the EIROFORUM or ESFRI, and non-voting representatives of extra-European partner grid infrastructures. It is expected that this representation could be reciprocated and that the EGI Council will be represented in the governing bodies of those partner grids. The EGI Council may designate committees that will work on topics specified by the Council. It may furthermore elect an Executive; details will be defined when the EGI.org statutes are worked out and when the future EGI Council will decide on them. The director and heads of units of EGI.org as well as the Chair of the User Forum Steering Committee will be ex officio Council members.

[edit] EGI.org and its Management

The EGI.org full time Director provides the organizational interface to the EGI Council, to funding and policy bodies (EC etc.) and to several EGI committees on the one hand, and to the heads of the EGI.org units on the other hand. For all internal and external activities the EGI.org Director has an assistant who will help with handling the work. The EGI.org Director will be supported by a secretariat and by dedicated staff to prepare policy developments, representation on European level, and to support the EGI Council. In EGI.org four permanent units are identified: the Administration Unit headed by the Central Administration Officer (CAO), the Operations Unit headed by the Central Operational Officer (COO), the Middleware Unit headed by the Central Technical Officer (CTO) and the User Community Services headed by the User Coordination Officer (UCO). The administration also includes staff to cover public relations, human resources, administrative and legal services. Projects may, based on EGI.org’s findings, be embedded in these units or they may be organized as a separate project oriented unit within EGI.org, but always embedded in the organization’s structure. The following graph summarizes the features of the EGI.org management structure: Image:EGI_mgmt.jpg Figure 5 EGI Management Structure The following table quantifies the management-related positions mentioned above:

Position FTE
Director 1
Assistant to the Director 1
Secretaries 2
Admin. Staff 2
Legal expert 1
Total (positions paid by membership fees) 11

[edit] EGI User Forum

The user communities will have representation and support mechanisms in the EGI User Forum (UF). The UF will organize an annual general meeting of all user communities favoring information exchanges at all levels.... More details on User Communities Support and its structures are given in Chapter.8. User communities are represented in the User Forum Steering Committee (UFSC) through the respective SSC or – if there is no SSC - through their bigger international grid based projects. The chairperson of the UFSC is an ex officio member of the EGI Council. The UFSC advises both the Council and the EGI.org Director on all matters regarding the involvement of the users of the EGI e-Infrastructure. At the management level, the SSCs will contribute to collecting and transferring the requirements and feedback from the user communities to EGI through the Grid Planning team (see Chapter 8); the team participates in the EGI Middleware Coordination Board providing a more long term technical planning; it may also establish other advisory committees to work with the EGI.org Director.

[edit] EGI Middleware Coordination Board (MCB)

The Middleware Coordination Board (MCB) is the EGI body that sets technical priorities and makes all decisions concerning the maintenance, support and evolution of the middleware deployed on the EGI e-Infrastructure; more details about the Middleware Support are provided in Chapter 7. The MCB is composed of representatives of the following areas, appointed in agreement with the EGI.org management:

the main middleware developers of the components in use in the EGI e-Infrastructure as the three European Middleware Consortia; the operation function representing globally the operational requirements of EGI.org, NGIs and Resource providers; the User Community Services (UCS) teams on behalf of the Specialized Support Centers, representing the various user communities organized in thematic disciplines.

[edit] Bibliography

    [1] EGI BluePrint, http://www.eu-egi.eu/vision.pdf.
    [2] Grid Operations Cookbook; EGEE-II Project Deliverable DSA1-5, Oct 2007.
    [3] Draft Definition of the EGI Organization, EGI_DS Deliverable, Mar 2008
    [4] The EGEE Network Operation Centre (http://technical.eu-egee.org/index.php?id=144).
    [5] The EUgridPMA: coordinating Grid authentication in e-Science (https://www.eugridpma.org/)
    [6] GGUS SUpport Model, EGEE and LCG Document N. 9100, Jul 2006.
    [7] Treadwell, J.; Open Grid Services Architecture Glossary of Terms, GFD-I.120.
    [8] The International Grid Trust Federation (http://www.gridpma.org/http://www.gridpma.org/ )
    [9] Grid Security Incident Handling and Response Guide, Open Science Grid, Nov 2004
    [10] OMII Europe, http://omii-europe.org/OMII-Europe/docs/DJRA20.pdf
    [11] D2.1. EGI consolidated requirements and use cases.
    [12] The SA1 Operational Procedures Manual: https://twiki.cern.ch/twiki/bin/view/EGEE/EGEEROperationalProcedures#1_SA1_Operational_Procedures_Man
    [13] GGUS monthly reports (https://gus.fzk.de/pages/metrics/download_metrics_reports.php).

[edit] Appendix A: Description of European Projects

[edit] A.1 Infrastructure Projects

[edit] A.1.1 EGEE

As results of investments from member states into national resources as well as by the European Commission with project such as EGEE (Enabling Grids for E-sciencE), Europe has developed a scientific grid infrastructure in and across many member states, which is being used by many research communities. More then 250 sites in 48 countries contribute to the EGEE infrastructure which can at the present deliver, 24 hours-a-day, seven days per week, more then 45000 CPU’s to communities like Archeology, Astronomy, Astrophysics, Civil Protection, Computational Chemistry, Earth Sciences, Finance, Fusion, Geophysics, High Energy Physics, Life Sciences, Multimedia, Material Sciences, across all Europe; the infrastructure is serving more then 8000 registered users spread across about 90 Virtual Organizations. In the year 2007 was stored about 25PB of data in disk and tape/MSS storage, Peaks of 3.5 Mjob per month have recently been observed on the EGEE infrastructure, which corresponds to 115 kjobs per day. During last year, 20578 kSI2kyears of CPU’s have been used. One third of such computing resources is today used by the “other research” communities, the rest by the high energy physics community. In the near future, it is expected that the HEP community alone will contribute to a factor of 5 increase in the computing resources usage during the next year. Massive data transfers rates up to 1.5 GB/s have been reached.

EGEE III Effort Table
Project Activities Effort in FTE
Middleware JRA1: Middleware Engineering SA3: Integration, Testing and Certification TNA5.3: Monitor EGEE contributions to standardisation activities Total 52.8
Funded 26.4
Operations SA1: Grid operations SA2: Networking Support Total 189.9
Funded 94.9
User oriented activities NA3: User Training and Induction NA4: User community support and expansion NA2: Dissemination, Communication and Outreach Total 121.7
Funded 60.9
Global Effort Total 364.4
Funded 182.2

Project duration: 24 months Project’s home page: http://www.eu-egee.org/

[edit] A.1.2 BalticGrid-II

The BalticGrid Second Phase (BalticGrid-II) project is designed to increase the impact, adoption and reach, and to further improve the support of services and users of the recently created e-Infrastructure in the Baltic States. This will be achieved by an extension of the BalticGrid infrastructure to Belarus; interoperation of the gLite-based infrastructure with UNICORE and ARC based Grid resources in the region; identifying and addressing the specific needs of new scientific communities such as nano-science and engineering sciences; and by establishing new Grid services for linguistic research, Baltic Sea environmental research, data mining tools for communication modelling and bioinformatics. The e-Infrastructure, based on the successful BalticGrid project, will be fully interoperable with the pan-European e-Infrastructures established by EGEE, EGEE associated projects, and the planned EGI, with the goal of a sustained e-Infrastructure in the Baltic Region. The e-Infrastructure of 26 clusters built in five countries during the first phase of the BalticGrid is envisaged to grow, both in capacity and capability of its computing resources. The BG-II consortium is composed of 13 leading institutions in seven countries, with 7 institutions in Estonia, Latvia and Lithuania, 2 in Belarus, 2 in Poland, and one each in Sweden and Switzerland.

The overall vision is to support and stimulate scientists and services used in the Baltic region to conveniently access critical networked resources both within Europe and beyond, and thereby enable the formation of effective research collaborations.

BalticGrid-II Effort Table
Project Activities Effort in FTE
Middleware JRA1: Enhanced Application Services on Sustainable e-Infrastructure NA4: Policy and Standards Developmen Total 4.25
Funded 4.25
Operations SA1: Grid Operation SA2: Network Resource Provisioning Total 16.25
Funded 16.25
User oriented activities NA2: Education, Training, Dissemination and Outreach NA3: Application Identification and Collaboration SA3: Application Integration and Support Total 16.87
Funded 16.87

Project duration: 24 months Yearly effort: PM 448; Annual budget: €1,499,000 Project’s home page: http://www.balticgrid.org/

[edit] A.1.3 SEE-GRID-SCI

The South-East European e-Infrastructure initiatives are committed to ensuring equal participation of the less-resourced countries of the region in European trends. SEEREN initiative has established a regional network and its GÉANT connection and the SEE-GRID initiative the regional Grid. SEE-GRID-SCI leverages the SEE e-Infrastructure to enable new scientific collaborations among user communities. SEE-GRID-SCI stimulates widespread e-Infrastructure uptake by new user groups extending over the region, fostering collaboration and providing advanced capabilities to more researchers, with an emphasis on strategic groups in seismology, meteorology and environmental protection. The initiative thus aims to have a catalytic and structuring effect on target user communities that currently do not directly benefit from the available infrastructures. In parallel, it aims to enlarge the regional e-Infrastructure to cater for demands of the communities by increasing the computing and storage resources and involving new partner countries in the region. Finally, SEE-GRID-SCI targets to help mature and stabilise the National Grid Initiatives in the region, allowing them to join the new era of longer-term sustainable Grid infrastructure in Europe.

SEE-GRID-SCI Effort Table
Project Activities Effort in FTE
Middleware JRA1 Development of application-level services Total 3.6
Funded 2.7
Operations SA1 Infrastructure Operations Total 9.6
Funded 9.6
User oriented activities NA4: User communities support NA3: Dissemination and Training Total 12
Funded 12

Project duration: 24 months Yearly effort: PM 302; Annual budget: €1,014,443 Project’s home page: http://www.see-grid.eu/

The following table summarizes the effort for the infrastructure projects (EGEE-III, BalticGrid, SEE-GRID-SCI).

All Infrastructure Projects Effort in FTE
Middleware Total 60.65
Funded 33.35
Operations Total 215.75
Funded 120.75
User oriented activities (includes Application support, Training, and Dissemination) Total 150.57
Funded 89.77
Global Effort Total 426.97
Funded 243.87

[edit] A.2 Development Projects

[edit] A.2.1 ETICS

The ETICS 2 project aims at establishing a software build, test and quality assurance validation service across different infrastructures and promoting the widespread adoption of grid-based software engineering technologies by existing and new infrastructures. This is done by consolidating and expanding the availability, flexibility and efficiency of the existing ETICS services across those infrastructures, capturing commonalities and promoting open standards on software build, testing and quality assurance. During the first phase of the ETICS project (2006-2007), a number of major challenges in the adoption of common build and test services across several projects have been identified through the close collaborations with many projects and active dissemination events. The challenges can be summarized as follows:

  • The lack of skilled personnel able to design and implement efficient validation test suites for complex grid scenarios associated to the complexity of the deployment and management of grid software
  • The lack of widely adopted validation procedures and metrics as well as a relative lack of trust at technological level between users and providers
  • The diversity of utilisation scenarios, the need for specialized validation methods and tools and the necessity of supporting emerging technologies and standards (e.g. IPv6)
  • The lack of grid-aware, implementation-independent test design and workflow management tools
  • The complexity of setting up and managing secure, multi-platform validation testbeds
  • The dispersion of resources across multiple software repositories, third-party testbeds and private resources, which are not accessible through common mechanisms, making it difficult to share information and protect existing investments in legacy systems

ETICS 2 addresses the identified challenges by providing a common software configuration model, multi-platform and language-independent build and test tools, an open repository of packages and reports produced during builds and tests runs, extensible tools to collect software metrics, generate reports and monitor the overall quality and standard compliance of distributed software and a standard-based certification model (Grid-QCM). The focus is on automating as much as possible the software development process from build to release minimizing the time and effort required to perform complex tests in realistic grid and distributed environments. In addition, ETICS extends the availability of distributed build and test services to multiple infrastructures allowing the automatic deployment of complex multi-node tests using the major European and international middleware implementations, like gLite, UNICORE and Condor.

ETICS 2 Effort Table
Project Activities Effort in FTE
Middleware JRA1 - Testbed Management Technologies JRA2 - Test Management Tools Total 4.6
Funded 2.8
Operations SA1 - Service Management SA2 - Infrastructures Support Total 10.3
Funded 7.7
User oriented activities NA2 - Dissemination, training and certification Total 2.0
Funded 2.0

Project duration: 24 months Yearly effort: PM 218.25; Annual budget: €1,336,000.00 Project’s home page: http://www.eticsproject.eu/etics

[edit] A.2.2 OMII-Europe

OMII-Europe is an EU-funded project which has been established to source key software components from major Grid middleware platforms and re-engineer them to be interoperable. Components are selected for their potential in the field of interoperability: similar functionalities, availability and maturity of standards, open nature of the standard, etc. The focus is on individual components and not on full middleware distributions in the spirit of a service-oriented approach and to prove that interoperability can be achieved even among completely different Grid middleware architectures. The final objective is to make available the quality-assured re-engineered components in a common repository with the expectation of their re-introduction in their original middleware releases. The work involves a set of 16 established partners from Europe, the USA and China. The selected middleware platforms for the initial work are gLite, UNICORE and Globus and the protocols or services to implement include job execution (BES/JSDL), data integration (OGSA-DAI), VO management (VOMS), accounting (RUS) and portal capability (GridSphere). The first year was dedicated to building the connections among all internal and external partners, organize participation in OGF and other working groups, design and prototyping of the components with the aim of delivering alpha versions by the end of the project year. The second year saw the beginning of QA tests, ramp-up of training events, continued cooperation with partner projects and participation in standardization events and the bulk of development leading to the delivery of final versions of all components at the end of the project. OMII-Europe's work is intended to be the beginning of an effort to spread the definition and the implementation of open standards in all fields of Grid computing. The project established the concept that standards are fundamental for the future of Grid middlewares and proved that interoperability can be achieved even between very different architectures.

OMII-Europe Effort Table
Project Activities Effort in FTE
Middleware JRA1: Re-engineering of services JRA2: Identification of new services JRA3: Infrastructure integration JRA4: Benchmarking Total 38.2
Funded 17.3
Operations SA1: Repository SA2: Quality Assurance SA3: Support Total 14.9
Funded 6.2
User oriented activities NA2: Outreach and inreach NA3: Training Total 3.4
Funded 2.2

Project duration: 24 months Yearly effort: PM 678; Annual budget: €3,174,191 Project’s home page: http://www.omii-europe.org

[edit] A.2.3 GridCC / DORII

While remote control and data collection was part of the initial Grid concept, most recent Grid developments have been concentrated on the sharing of distributed computational and storage resources. In this scenario applications that need computational power only have to use these Grid elements in order to access an unlimited amount of computational power and disk storage. However scientific and technical facilities both provide concrete use cases where a strong interaction between the instrumentation and the computational Grid is required. The GRIDCC project, launched in September 2004 by the European Union, provides a well proven technology that can be deployed on top of existing grid middleware, extending the grid e-infrastructure to the control and monitoring of remote instrumentation. EGEE gLite is the natural reference grid middleware for GRIDCC and the EGEE e-infrastructure is the natural framework where to deploy and integrate the instrument grid technology. The goal of GRIDCC was to build a geographically distributed system that is able to remotely control and monitor complex instrumentation, ranging over a large number of diverse environments, from a set of sensors used by geophysical stations monitoring the state of the earth to a network of small power generators supplying the European power grid. These applications need real-time and highly interactive operation of GRID computing resources. To achieve this goal the project has pursued three main objectives:

To develop generic Grid middleware, based on existing building blocks (Grid Services), which will enable the remote control and monitoring of distributed instrumentation. To incorporate this new middleware into a few significant applications to validate the software both in terms of functionality and quality of service aspects. These applications include, among others, European Power Grid, Meteorology, Remote Operation of an Accelerator Facility, High Energy Physics Experiment. To widely disseminate the new software technology and the results of the application evaluations on the test beds, and to encourage a wide range of stakeholders to evaluate and adopt our Grid-oriented approach to real-time control and monitoring of remote instrumentation.

GridCC Effort Table
Project Activities Effort in FTE
Middleware WP1: System Architecture WP2: Real-time and Interactive web services WP3: Grid-Enabled Instrumentation WP4: Brokering access to existing Grid resources WP5: Cooperative Environment (user-oriented?) Total 21.7
Funded 10.8
Operations N / A Total 0
Funded 0
User oriented activities WP6: Integration and Pilot Applications WP7: Information dissemination and exploitation Total 15.7
Funded 7.8

Project duration: 36 months Yearly effort: PM 449; Annual budget: €1,763,000 Project’s home page: http://www.gridcc.org/cms/

[edit] A.2.4 Interactive European Grid

The objective of the Interactive European Grid project is the deployment of an advanced Grid empowered infrastructure in the European Research Area specifically oriented to support the execution of interactive demanding applications. The Interactive European Grid, whilst interoperable with EGEE, will focus on interactive use for medicine, environment, physics and other research areas (from robotics to archaeology) that have demanding interactive applications that can benefit from being Grid-enhanced. The initiative exploits the expertise generated by the EU CrossGrid project to provide researchers with an interactive and simultaneous access to large distributed facilities through a friendly interface with powerful visualization.

Project duration: 24 months Annual budget: €1,318,500 Project’s home page: http://www.interactive-grid.eu/

[edit] A.3 Field-Specific Projects

[edit] A.3.1 BIOINFOGRID

Since the completion of the Genome Project, due to the vast number of sequences available for consultation, the problems associated with the calculation resources needed to process biological data have increased dramatically. Moreover, the amount of data continues to increase at a high speed because new technology of high throughput expression analysis gives to researches a continuous flow of information to be appropriately elaborated and interpreted. In the meantime the study of comparative genomics and genetic variation using modern analysis methods, to identify in details the different set of genes involved in diseases, amplify the computational load problem. The major result of the BioinfoGRID project is the demonstration that Grid computing can be a reliable solution to face with the appropriate tools the most recent computational problems in Bioinformatics. More specifically the BioinfoGRID project has evaluated applications in the fields of Genomics, Proteomics, Transcriptomics and Molecular Dynamics, showing that a reduction in the time required to reach the final result can be obtained by distributing the calculation on thousands of computers using the Grid infrastructure network created by the EGEE Project (6th Framework Program). By exploiting these resources, challenges that were not possible in a recent past, can now be deployed walking through the sequencing of the Human Genome and working with new perspectives at the study of complex multigenic diseases analysing in parallel thousands of molecular components. However, the BioinfoGRID project has also shown that the situation is far from being ideal, for example for what concerns the friendliness, completeness, robustness and adherence to standards of the existing tools for the biological data access and management as well for the grid jobs submission, monitoring and bookkeeping. On the other end the BioinfoGRID project has also pointed out that a continue dissemination activity is necessary in order to bring the Grid vision to pervade campuses, departments and laboratories and set up the basis for the sharing of information and of a collaborative work.

Project Activities Effort in FTE
Middleware N / A Total 0
Funded 0
Operations N / A Total 0
Funded 0
User oriented activities WP1: Genomics applications in GRID WP2: Proteomics Applications in GRID WP3: Transcriptomics Applications in GRID WP4: Database and Functional Genomics Applications WP5: Molecular Dynamics Applications WP6: Coordination of technical aspects and relation with Grid infrastructure Projects, user training, application support and resources integration WP7: Dissemination and Outreach Total 12.2
Funded 9.3

Project duration: 24 months Yearly effort: PM 146; Annual budget: €527,104 Project’s home page: http://www.bioinfogrid.eu/

[edit] A.3.2 CYCLOPS

CYCLOPS brings together two important Communities: GMES (Global Monitoring for Environment and Security) and GRID, focusing on the operative sector and needs of European Civil Protection. The main objectives of CYCLOPS are:

  1. To disseminate EGEE results to the CP Community, assessing EGEE infrastructure for CP applications. A variety of activities will focus on dissemination and outreach, training, workshops, possibly in close relation with EGEE events and on promoting a close collaboration between the two communities.
  2. To provide the EGEE Community with knowledge and requirements that characterise the CP services. These requirements will also be used to assess the possibility for the development of an advanced grid platform enabling Real Time and near-Real Time services and implementing a security infrastructure very close to the defence systems standards.
  3. To evaluate the possibility to utilise the present EGEE services for CP applications, developing the research strategies to enhance EGEE platform.
  4. To develop the research strategies to enhance EGEE platform, especially for Earth sciences resources.

CYCLOPS will contribute to the EU policy developments establishing liaisons and synergies with other existing projects and initiatives dealing with GMES, GRID and complementary sectors, among them: PREVIEW, Risk EOS, RISK-AWARE, BOSS4GMES, EGEE Networking Activities and Application Support, e-IRG and INSPIRE. In fact, Consortium partners are involved in all these projects and initiatives. Furthermore, CYCLOPS aims to address the OGF standardization needs as far as the Earth and Space Science Communiy, GMES and gLite are concerned. In this context, it is contributing to OGC (Open Geospatial Consortium) OGF initiative."

CYCLOPS Effort Table
Project Activities Effort in FTE
Middleware N / A Total 0
Funded 0
Operations N / A Total 0
Funded 0
User oriented activities WP2: Coordination with EGEE activities WP3: Civil Protection System analysis WP4: research and Innovation Strategies definition WP5: Dissemination & Exploitation Total 5.1
Funded 5.1

Project duration: 24 months Yearly effort: PM 61; Annual budget: €412,500 Project’s home page: http://www.cyclops-project.eu/

[edit] A.3.3 e-NMR

e-NMR aims at deploying and unifying the NMR computational infrastructure in system biology, a project funded under the 7th framework programme of the European Union (Contract no. 213010 - e-NMR). NMR plays an important role in life sciences (biomolecular NMR), and structural biology in particular, at both European and international levels. Our main objective is to optimize and extend the use of the NMR Research Infrastructures of EU-NMR through the implementation of an e-Infrastructure in order to provide the biomolecular NMR user community with a platform integrating and streamlining the computational approaches necessary for NMR data analysis and structural modelling (e-NMR). Access to the e-NMR infrastructure will be provided through a portal integrating commonly NMR software and GRID technology.

e-NMR Effort Table
Project Activities Effort in FTE
Middleware WP3: Design and development of the e-NMR Grid platform Total 5.1
Funded 5.1
Operations WP2: e-NMR Grid deployment and operation Total 1.4
Funded 1.4
User oriented activities WP1: Monitoring, Standardization and Outreach Total 1.8
Funded 1.8

Project duration: 36 months Yearly effort: PM 100; Annual budget: €922,217 Project’s home page: http://www.e-nmr.eu/

[edit] A.3.4 Ithanet

Ithanet is a Euromediterranean network of research centres conducting molecular and clinical research of thalassaemia and related haemoglobinopathies. Participants of Ithanet include all major European research institutions active in haemoglobinopathy research and a number of collaborating partner institutions from non-EU Mediterranean and Black Sea countries. The main objective of Ithanet co-ordination action is to enhance the scientific potential of this research community using infrastructures and tools of European Research Networks. Ithanet aims to harmonize and develop these resources for the coordination of existing research activities as a base for future collaborative projects. Using eInfrastructure tools to consolidate and strengthen a research community with a specific geographic distribution and research topic, Ithanet strives to create new opportunities for high-impact collaborative research in the European Research Area.

Ithanet Effort Table
Project Activities Effort in FTE
Middleware N/A Total 0.0
Funded 0.0
Operations WP2: e-Infrastructure (collaboration tools) Total 1.05
Funded 1.05
User oriented activities WP3: Tools for clinical research WP4: Tools for molecular research WP5: Training and knowledge transfer WP6: Portal WP7: Dissemination Total 3.3
Funded 3.3

Project duration: 24 months Yearly effort: PM 52; Annual budget: €603,650 Project’s home page: http://www.ithanet.eu/

[edit] A.3.5 DEGREE

A major challenge for DEGREE is to build a bridge linking the ES and GRID communities throughout Europe, and focusing in particular on the EGEE-II Project. An ES applications panel with a range of candidate applications suitable for porting to GRID will make sure key ES requirements for porting and deployment on the GRID middleware are identified, communicated and discussed within the GRID community. At the same time the DEGREE SSA will ensure the ES community is informed and up to date on GRID developments and potential benefits. The results will provide feedback to the GRID community and dissemination in the ES community will increase awareness of and involvement with GRID developments. In order to ensure that ES requirements are taken into account in the next Grid generation, DEGREE will initiate different collaborations; at short, medium and long term via EU horizontal collaborations, specific collaboration with Grid projects and participation to the e Infrastructure Reflection Group (e-IRG). Objectives:

  • Disseminate, promote uptake of Grid in wider ES community
  • Reduce the gap between ES users and Grid Technology
  • Explain and convince ES users of Grid benefits and capability to tackle new and complex problems.

Project duration: 24 months Annual budget: €670,000 Project’s Home page: http://www.eu-degree.eu/

[edit] A.3.6 EuroVO-DCA

The concept of a Virtual Observatory is that all the worlds astronomical data should feel like it sits on the astronomers desktop, analyzable with a user selected workbench of tools and made available through a standard interface. Euro-VO is the European implementation of this idea that will produce a unified data and service resource (a data and service grid) with the ability to perform complex data discovery and manipulation tasks across the whole range of astronomical research topics. The Euro-VO Data Centre Alliance project will co-ordinate the national and European Agencies Virtual Observatory initiatives, supporting implementation of the Virtual Observatory framework by the European Data Centres to populate the Virtual Observatory with data produced by the European astronomy infrastructures.

Project duration: 28 months Yearly effort: PM 72.9; Annual budget: €702,857 Project’s home page: http://www.euro-vo.org/pub/index.html

[edit] A.4 International Cooperation Projects

[edit] A.4.1 EUChinaGrid

Co-Funded by the European Commission in the framework of FP6, EUChinaGRID Project officially started on 1st January 2006 with the aim to support the interconnection of the existing European and Chinese Grid Infrastructures and enable their interoperability, thus creating a network of collaboration between Europe and China. EUChinaGRID provided specific support actions to foster the integration and interoperability of the Grid infrastructures in Europe (EGEE) and China (CNGrid) for the benefit of e-Science applications and worldwide Grid initiatives, in line with the support of the intercontinental extension of the European Research Area (ERA). The project studied and supported the extension of a pilot intercontinental infrastructure using the EGEE-supported applications and promoted the migration of new applications on the Grid infrastructures in Europe and China; this was done by training new user communities and supporting the adoption of grid tools and services for scientific applications. A set of existing Euro-Chinese collaborations in research, marked by strong requirements in terms of analysis of large quantities of data and needs for wide amounts of computing power, were selected as pilot applications in order to validate the infrastructure. During the 27 months of duration, the Project achieved several goals. The pilot infrastructure includes 12 sites 5 of which are in China (4 in Beijing and one in Shandong). All the relevant Grid services were started and are maintained to facilitate the access of users and Virtual Organizations (VO) through the web portal (www.euchinagrid.eu). Some of those core Grid services are hosted in China. A special stress was posed on designing an e-Infrastructure allowing full interoperability, both horizontally (i.e. between European and Chinese middleware) and vertically (i.e. between Grid middleware and the different versions of the IP protocol). Works towards both objectives lead to interesting results and the EUChinaGRID findings in this field raised interest amongst middleware developers in EGEE and ETICS communities leading to common activities such as a code checker for IPv6 compliance implemented in the ETICS building system. A Gateway between gLite and GOS has been build and extensively tested and improved. The Gateway allows exchanging jobs between the two infrastructures taking care of the differences related to the Job Description Languages and the Security mechanisms. Application deployment has also achieved significant impact in several science fields:

  • High Energy experiments (ATLAS and CMS) at the CERN Large Hadron Collider (LHC) can run their applications on the pilot infrastructure.
  • Astroparticle experiment ARGO-YBJ, a joint collaboration between Chinese and Italian researchers, is currently collecting data on Cosmic Ray showers in the YangBaJing laboratory in Tibet; a complete system has been deployed to perform the data transfer from YangBaJing to IHEP (Beijing) and INFN-CNAF (Bologna) sites, using the EUChinaGRID Grid Infrastructure and the 2.5 Gbps link provided by the ORIENT project.
  • EUChinaGRID also supported Biological applications in the field of simulation and discovery of new proteins. The work in this field, carried out in the laboratories of Biology Department of University of Roma Tre (UROM3), Jagiellonian University – Medical College (JU-MC) and Peking University (PKU), led to first ab-initio protein structure prediction processes ever deployed in a Grid environment. The parallel approaches adopted by UROM3 and JU-MC have been compared on a large sample of candidates (2x104), while the predicted protein structures are being experimentally verified by the PKU group.

EUChinaGRID had an intense dissemination activity with two website versions in English and Chinese and more than 300 Chinese researchers, engineers and students took part to the advanced knowledge tutorials held in China. A specific dissemination action was carried out towards the community of middleware developers, to raise their awareness about IPv6 compliance and interoperability issues, and to suggest actions and best practices for overcoming these problems. This included the delivery of focused workshops and tutorials, that involved more than 150 developers, and the publication of a dedicated IPv6 website (http://www.euchinagrid.org/IPv6/index.html), as well as the collaboration with other projects such as 6DISS.

EUChinaGrid Effort Table
Project Activities Effort in FTE
Middleware N / A Total 0
Funded 0
Operations WP2: Network planning and interoperability study WP3: Pilot infrastructure operational support Total 20.08 (10 Non-EU)
Funded 14.01 (6 Non-EU)
User oriented activities WP5: Applications WP5: Dissemination Total 37.64 (13 Non-EU)
Funded 21.81 (5 Non-EU)

Project duration: 27 months Yearly effort: PM 693; Annual budget: €577,777 Project’s home page: http://www.euchinagrid.org/

[edit] A.4.2 EUMEDGRID

Funded by EC within the Sixth Framework Program for Research and Development and Coordinated by INFN, EUMEDGRID aimed to support the development of a Grid e-Infrastructure in the Mediterranean Area and promote the porting of new applications on the Grid platform, thus allowing Mediterranean scientist to collaborate more closely with their European colleagues. EUMEDGRID has disseminated Grid awareness and competences across the Mediterranean and, in the meanwhile, identifying new research groups to be involved in the project helped them to exploit Grids’ enormous potential to improve their own applications. The implementation and coordination of a grid infrastructure at a national (or larger) level can be regarded, especially in the beneficiary Countries, as an opportunity to optimize the usage of existing, limited storage and computing resources and to enhance their accessibility for all research groups. The EUMEDGRID project was conceived in this perspective and has set up a pilot grid infrastructure for Research in the Mediterranean Region, which is interoperable and compatible with EGEE and related initiatives. The EUMEDGRID’s vision focused on improving both the technological level and the know-how of networking and computing professionals across the Mediterranean, thus fostering the introduction of an effective Mediterranean Grid infrastructure for the benefits of eScience. Accordingly, the project objectives can be regarded as belonging to two main areas: the first focusing on softer actions, with the overall aim of creating a human network in e-Science across the Mediterranean, and the second one addressing technical issues and intended to support the implementation of a pilot Grid infrastructure and applications in the area. The Project lasted for 26 months and made a considerable step forward during the second year of the project and a number of achievements give evidence of the success of its activities. Cooperation among all the participants has been demonstrated by the enthusiastic participation to common workshops and meetings organized during the duration of the project and the great success obtained fostering the creation of National Certification Authorities and National Grid Initiatives. Impressive results were also obtained in the events of knowledge dissemination on Grid Technology and services. A large community including system administrators, researchers, and final users was involved with good results in terms of number of participants (more than 700 people) and feedback obtained through dedicated questionnaires. The promotion of National Grid Initiatives carried out in all non-EGEE Partner Countries registered a good level of success with programs already operational in Algeria, Egypt, Morocco, Tunisia and Turkey and well advanced plans, with clear commitments, in Cyprus, Jordan, Syria and Palestinian Territories. The project was very active in promoting the creation of national Certification Authorities which will issue digital certificates necessary for allowing secure Grid access to the users. The process is completed in Morocco, the first African Country to become member of EUGridPMA and well advanced in the other countries and, in the meanwhile, a temporary catch-all CA was created in order to fulfil the needs of EUMEDGRID users. A pilot grid infrastructure, composed to date of 25 sites in 13 countries, was set up during the project’s duration. Several applications have been proposed to run on the EUMEDGRID e-Infrastructure and many were selected to be supported, spanning several fields of interests: High Energy Physics, Biology and Biomedical, Hydrology, Archaeology, Seismology and Vulcanology. New communities and applications of Regional interest were also discovered by means of a survey based on web questionnaires2. The works to port the first applications on the EUMEDGRID eInfrastructure began in the 1st quarter of 2006 with CODESA and ArchaeoGrid, respectively a hydrological and an Archaeological application which are of interest for the Mediterranean Region. Another large bunch of applications was deployed during a dedicated event in Cairo: the first “EUMEDGRID School for Application Porting” (EGSAP-1 http://www.EUMEDGRID.org/egsap-1/) on 17-28 April 2007. Conceived as a full immersion experience for selected new communities of regional interest, the school was deemed of paramount importance for the uptake of new applications on the regional pilot infrastructure. EGSAP-1 was accordingly one of the largest dissemination efforts of the whole lifetime of the project, contributing in involving new communities in the project activities, while providing them the knowledge needed to build upon the eInfrastructure and deploying their own application. All selected applications were ported to the EUMEDGRID e-infrastructure. Moreover these applications have been also ported to the GENIUS web portal. The interest of the EUMEDGRID experience does not however restrict to scientific issues - although the opportunity to port applications of regional importance, such as the hydro-geological and medical ones, on the pilot infrastructure sounds really exciting. Fostering Grid awareness and the growth of new competences in EU Neighbours’ scientific communities is a concrete initiative towards bridging the digital gap and, moreover, to promote a peaceful and effective collaboration among all Partners. At Social level e-Infrastructures can contribute to mitigate phenomena such as Digital Divide and, possibly, revert Brain Drain to allow brilliant minds in the area to contribute significantly to cutting edge European Scientific activities concretely enlarging the European Research Area (ERA). Research and Education Networks and Grids are fundamental infrastructures that will allow non-EU researcher to make high quality work in their home laboratories without the need to migrate in most advanced countries. An extended Mediterranean Research Area could thus be seen as a first step towards the suggestion of more politically ambitious plans of open market, open transportation infrastructures, free circulation of citizens, etc.

EUMEDGRID Effort Table
Project Activities Effort in FTE
Middleware N / A Total 0
Funded 0
Operations WP3: Pilot infrastructure operational support Total 22.33 (15.09 Non-EU)
Funded 12.1 (4.76 Non-EU)
User oriented activities WP4: Application support WP2: Requirement capture and analysis WP5: Dissemination and Outreach Total 26.57 (10.56 Non-EU)
Funded 20.84 (8.36 Non-EU)

Project duration: 26 months Yearly effort: PM 587; Annual budget: €759,231 Project’s home page: http://www.eumedgrid.org/

[edit] A.4.3 EUAsiaGrid

The EUAsiaGrid proposal contributes to the aims of the EU Research Infrastructures FP7 Programme by "promoting international interoperation between similar infrastructures with the aim of reinforcing the global relevance and impact of European e-Infrastructures". The project's main goal is to pave the way towards an Asian e-Science Grid Infrastructure, in synergy with the other European Grid initiatives in Asia, namely EGEE-III via its Asia Federation, and both the EUChinaGRID and EU-IndiaGRID projects and their eventual follow on efforts. Taking advantage of the existing global Grid technologies, with the specific emphasis on the European experience with the gLite middleware and applications running on top of it, the project plans to encourage federating approaches across scientific disciplines and communities. EUAsiaGrid acts as a support action, aiming to define and implement a policy to promote the gLite middleware developed within the EU EGEE project across Asian countries. Its main actions will be to spread dissemination, provide training, support scientific applications and monitor the results. The use of the Grid e-Science infrastructure is not only promoted on a geographical base, but also to new communities which can profit of the resources made available, like Social Sciences, Disaster Mitigation, building on the knowledge of more experienced fields, like High Energy Physics and Bioinformatics. The project would envision any collaboration with standard bodies and other projects which helps in making the results sustainable over time.

EUAsiaGrid Effort Table
Project Activities Effort in FTE
Middleware N / A Total 0
Funded 0
Operations N / A Total 0
Funded 0
User oriented activities WP2: Requirement capture and coordination policy definition WP3: Support of scientific applications WP4: Dissemination WP5: Training Total 15.0
Funded 13.1

Project duration: 24 months Yearly effort: PM 180; Annual budget: €727,075 Project’s home page: http://www.euasiagrid.org/

[edit] A.4.4 EU-IndiaGrid

EU-IndiaGrid is the European project that has established and currently maintains e-Infrastructure ties with the Indian generalized Grid infrastructure. Among the partners of the project are the Indian NREN (ERNET) and the Indian NGI (GARUDA). EU-IndiaGrid is formally supported by the Indian Government, as witnessed by a letter sent to Ms Reding (EC) by the Indian Government Principal Scientific Advisor, Dr Chidambaram. In addition to extensive dissemination and training activities, EU-IndiaGrid has set up a testbed running applications from several scientific communities, and has reported on its interoperation efforts in the context of many collaborative and standardization / interoperability events. The work has produced some specific requirements which would benefit interoperation between the European gLite middleware and the Indian middleware. The project would envision extending the collaboration with standards bodies and projects such as (a possible continuation of) OMII-Europe by implementing these requirements (either directly or in an effort mediated by EGI), and in general to continue and ideally stabilize the elements which maintain the current EU-India Grid relationship.

EU-IndiaGrid Effort Table
Project Activities Effort in FTE
Middleware N / A Total 0
Funded 0
Operations WP3: Network Planning Support WP4: Pilot grid infrastructure operational support Total 4.7
Funded 3.1 User oriented activities WP5: Applications WP2: Building an eScience Network Community WP6: Dissemination & Networking Events Total 8.9
Funded 6.0

Project duration: 24 months Yearly effort: PM 163; Annual budget: €640,410 Project’s home page: http://www.euindiagrid.org/

[edit] A.4.5 EELA-2

EELA-2 aims at building a high capacity, production-quality, scalable Grid Facility, providing round-the-clock, worldwide access to distributed computing, storage and network resources needed by the wide spectrum of Applications from European - Latin American Scientific Collaborations, with special focus on: Offering a complete set of versatile services fulfilling Applications requirements; Ensuring the long-term sustainability of the e-Infrastructure beyond the term of the project. Such an ambitious project would not be possible without the prior existence of a consolidated e-Infrastructure, set up with the early intention to build a sustainable Grid platform. This was the objective of the EELA Project (www.eu-eela.org/first-phase.php) that provided its users with a stable, well supported Grid Infrastructure based on 16 Resource Centres (RCs) summing up to over 730 CPU cores and 60 Terabytes of storage space, thus proving that the deployment of an European-Latin American e-Infrastructure was not only viable but is also responding to a real need of a significant part of the Scientific Community. The EELA-2 vision is two-fold: Consolidate and expand the current EELA e-Infrastructure built on the GÉANT2/European and RedCLARA/LA National Research & Education Networks (NREN), to become an e-Infrastructure Facility, providing a full set of enhanced services to all types of Applications from multiple Scientific Areas of European and Latin American Scientific Communities; Ascertain the conditions of the durability of the e-Infrastructure, beyond the Project duration.

EELA-2 Effort Table
Project Activities Effort in FTE
Middleware JRA1: Development of Services for Applications and Infrastructure Total 7.5
Funded 5
Operations SA1: Grid Infrastructure Service Activity SA2: Network Resource Provision Total 31.5
Funded 18.3
User oriented activities NA3: Application Support NA2: Dissemination and Training Total 17.0
Funded 8.7

Project duration: 24 months Yearly effort: PM 672; Annual budget: €1,284,160 Project’s home page: http://www.eu-eela.eu/

[edit] A.5 Data Management Projects

[edit] A.5.1 D4Science

D4Science is one of the main European e-Infrastructure projects, involving 11 participants and co-funded by the European Commission's Seventh Framework Programme for Research and Technological Development. The project started in January 2008 and has duration of 2 years. D4Science aims to continue the path that the GÉANT, EGEE, and DILIGENT projects have initiated towards establishing networking, grid-based, and data-centric e-Infrastructures that accelerate multidisciplinary research by eventually overcoming several crucial barriers that stand in the way, primarily those related to heterogeneity, sustainability and scalability. The main objective of D4Science is laying the foundations for next generation of collaboration and knowledge management environments by realizing an infrastructure that allows members of dynamic Virtual Research Environments (VREs) to create on-demand transient digital libraries based on shared computing, storage, multi-type content and application resources. Knowledge sharing and support of collaboration in a secure, coordinated, dynamic and cost-effective manner are to be the two major facilities offered by the combination of hardware, network, software and content elements that constitute a D4science infrastructure. Whereas this infrastructure is designed to support many different research and industrial applications, two communities are used to demonstrate and validate the project: the Environmental Monitoring and Fisheries and Aquaculture Resources Management communities. The objectives of the project will be achieved through the synergetic operation of Networking, Service, and Joint Research Activities. The overall objective of the Networking Activities (NA) is to serve the needs of the communities. The experience done with these large communities will facilitate a future extension of the e-Infrastructure capabilities to other scientific communities. This will be done by: disseminating the project outcomes, training of the various players, and exploiting and collecting feedback to the D4Science e-Infrastructure through the implementation of the communities VREs. The Service Activities (SA) aims at making available and maintaining a stable, reliable and usable e-Infrastructure to these (and possible other) D4Science user communities. Finally, the Joint Research Activities (JRA) address the technical requirements raised by the Environmental Monitoring and Fisheries and Aquaculture Resources Management communities against the gCube framework.

D4ScienceEffort Table
Project Activities Effort in FTE
Middleware JRA4: gCube Development Total 6.1
Funded 4.6
Operations SA1: Infrastructure Operation SA2: Community Specific Operations SA3: Software Integration, Testing and Distribution Total 9.4
Funded 9.4
User oriented activities JRA1: Overall Planning and Development Coordination JRA2: Environmental Monitoring Community-specific Software Development JRA3: Fishery Resources Management Community-specific Software Development NA3: Communication and Dissemination NA4: Training NA5 Communities VREs Definition, Validation and Exploitation Total 15.25
Funded 10.75

Project duration: 24 months Yearly effort: PM 200; Annual budget: €1,575,000

[edit] A.5.2 DRIVER

DRIVER is building the testbed for a future knowledge infrastructure of the European Research Area. Aimed to be complementary to GN2, the successful infrastructure for computing resources, data storage and data transport, DRIVER will deliver the content resources, i.e. any form of scientific output, including scientific/technical reports, working papers, pre-prints, articles and original research data. The vision to be accomplished in a second phase is to establish the successful interoperation of both data network and knowledge repositories as integral parts of the e-infrastructure for research and education in Europe. The knowledge infrastructure testbed, delivered by DRIVER, will be based on nationally organized digital repository infrastructures, similar to GN2 and the NRENs. The successful DARE network in the Netherlands, recently presented to the public by the project partner SURF, will serve as a model to DRIVER. DRIVER with its testbed will not build a specific digital repository system with pre-defined services, based on a specific technology and serving dedicated communities. The testbed will in its inception focus on the infrastructure aspect, i.e., open, clearly defined interfaces to the content network, which allow any qualified service-provider to build services on top of it. Like the data network GÉANT, DRIVERs knowledge infrastructure offers mainly a well structured, reliable and trustworthy basis. DRIVER opens up knowledge to the communities; it does not prescribe how to use the knowledge.

Project duration: 18 months Yearly effort: PM 244.7

[edit] A.6 Policy and Public Relations Projects

[edit] A.6.1 Belief

BELIEFs aim is to create a platform where e-Infrastructure stakeholders can collaborate, reach out to new audiences and exchange knowledge, thus helping to ensure that e-Infrastructures are both developed and used effectively worldwide. It will be a one stop shop for information on e-Infrastructure documentation and activities for both research and industry and will thus aid the knowledge transfer between them.

Project duration: 24 months Annual budget: €604,226.5 Project’s home page: http://www.beliefproject.org/

[edit] A.6.2 e-IRGSP

The e-IRGSP project provides a number of services to support the work of the e-Infrastructure Reflection Group (e-IRG), such as a secretariat (in The Hague, The Netherlands), a knowledge base and policy and editorial support. e-IRG consists official government delegates from the 25 EU member states, as well as associated countries.

Project duration: 24 months Yearly effort: PM 22.5; Annual budget: €183,042 Project’s home page: http://e-irg.eu/

[edit] A.7 Other Projects

[edit] A.7.1 ICEAGE

At European level, e-Infrastructure has been identified as a key element of the construction of the European Research Area (ERA) so as to stimulate industry, improve the lives of citizens, accelerate research and gain international competitive advantage. For Europe to realise this expectation, there needs to be a diverse, knowledgeable and creative community to skilfully exploit e-Infrastructure. With the support of the European Union, the ICEAGE project aimed to encourage and support the incorporation of education in distributed computing in academic courses throughout the ERA. Built on EGEE, ICEAGE has enabled students and educators to obtain and develop Grid Education via sustained, large-scale, multi-purpose e-Infrastructures. ICEAGE differs from EGEE in that its primary goals are educational and therefore embraces a wide variety of approaches to e-Infrastructure. ICEAGE has catalysed the necessary infrastructure and skills by establishing a worldwide initiative to inspire innovative and effective Grid Education. Grid Education implies the use of education in the Grid, but also the use of the Grid in education. In the context of ICEAGE the term "Grid" is indeed used in a broad sense to include computing and communications technology, working practices, and policies that underpin e-Infrastructure.

ICEAGE Effort Table
Project Activities Effort in FTE
Middleware t-Infrastructure – development and provision (with several middleware co-existent) Total 3
Funded 2
Operations t-Infrastructure operation (during Grid Schools) Total 2
Funded 1
User oriented activities WP1 - Extend and Advance Grid Education – Grid Education Policy Development WP2 - Advanced Grid Education Support, Outreach, Induction & Training Services WP3 - Educational events and Summer Schools WP4 - t-Infrastructure – development and provision Total 13
Funded 9

Project duration: 24 months Yearly effort: PM 216; Annual budget: €600,000 Project’s home page: http://www.iceage-eu.org/

[edit] A.7.2 ISSeG

ISSeG aims to contribute to the consolidation of the European Grid infrastructure in the field of computer security, by creating and disseminating practical expertise on the deployment of Integrated Site Security (ISS), as a complementary action to Enabling Grids for E-sciencE (EGEE) projects Grid Security. ISS is a concept where all Site Security components (technical, administrative, educational) are developed in a coordinated fashion. The ISSeG vision is that Grid Security, which focuses on inter-site security, middleware, and authentication, needs to be complemented by a comprehensive ISS strategy at every centre. The ISSeG consortium comprises three large scientific centres, CERN, CCLRC and FZK, all involved in EGEE. The project objectives will be achieved by the creation and capture of raw expertise through full-scale ISS deployment at CERN and FZK, and by dissemination through the provision of applicable recommendations and methodologies for further ISS deployments.

Project duration: 24 months Yearly effort: PM 102.5; Annual budget: €655,000 Project’s home page: http://www.isseg.eu/

[edit] A.7.3 RINGrid

RINGrid provides an architecture which integrates scientific instruments in the e-Infrastructure and promotes a vision towards next-generation Remote Instrumentation Systems. It encompasses the current state-of-the-art and near future technology, delivers a conceptual design of missing architectural pieces to achieve such vision and assumes a Grid environment and high-speed network interconnections.

Project duration: 18 months Yearly effort: PM 123; Annual budget: €666,110 Project’s home page: http://www.ringrid.eu/

[edit] Appendix B

[edit] B.1 NGI effort estimation

Purpose of this Section is to further elaborate on NGI effort requirements for Operations and Security activities in EGI. In particular, this document elaborates on which metrics can be adopted to estimate the effort needed for international tasks by NGIs of different sizes and with different levels of complexity and involvement in EGI. Various metrics can be adopted to estimate the size and complexity of an NGI, as shown in:

Table 3: examples of metric for NGI size estimation
Category Metric Description
Size metrics Number of production sites Number of certified Grid sites that are part of the NGI e-Infrastructure, not including other sites just configured for testing purposes. The effort needed by an NGI to support site managers in the country and to monitor the infrastructure, is somehow related to this quantity. This information is typically provided by the Grid Information services.
Total computing power Amount of computing resources made available to international VOs. The power of a computing device typically depends on the type of application it runs and can not be quantified univocally. It can be approximated by the total number of CPUs or cores provided and the related computing power, which can be estimated via a given benchmark of choice (e.g. SpecInt2000). Information about this metric is typically provided by the Grid Information services.
Total disk and tape space Amount of storage space made available to international VOs. Information about this metric is typically provided by the Grid Information services.
Operations metrics Overall NGI availability/reliability Overall level of service offered to the supported VOs, intended to be a composed function of the availability/reliability on various services, such as: the Grid technical services operated by the sites, the Grid central services operated by the NGI, the Grid operational tools, etc.
Number of trouble tickets per NGI The amount of tickets reaching final status in the EGI central ticketing system can be computed for every NGI, to estimate the workload sustained to support users and site managers. This number should include both central and regional tickets.
Usage metrics Number of users per NGI The number of users served by a given NGI can be approximately estimated by gathering the number of VOs supported, and by calculating the distribution of users per country according to the accredited Grid CA that released the corresponding personal certificate.
Accounting records The aggregated amount of (normalized) wall clock time/cpu time consumed by applications running in the region.

Operation metrics such as availability and reliability are of great importance, as they can have a significant impact on the amount of effort needed to operate a given NGI e-Infrastructure. For example, according to the requirements of the supported VOs, some NGIs may be interested in operating a highly reliable Grid, while others may just provide a best-effort service. In the former case, the enforcement of high-level services requires manpower to run shifts, for Grid oversight activities, etc. To address this variety of needs, NGIs may be periodically requested to declare a target level for their services, according to which the effort requirements are estimated. During the transition period discrepancies between the target and the actual service level delivered may be accepted, in order to give the chance to all NGIs to develop the level of maturity needed.

[edit] B.2 metric Analysis

[edit] B.2.1 Operations metrics: Number of trouble tickets per NGI

Tickets for an NGI surely imply a management workload, but they can be an indication of a poor level of service, and are not necessarily proportional to the size of the infrastructure operated, as it is currently the case in EGEE. A study has been performed to estimate the current distribution of tickets per country. Information about the number of central tickets (i.e. those tickets managed via the central helpdesk system in EGEE) was gathered from the GGUS monthly reports [REP]3. Shows the number of GGUS tickets for each EGEE ROC from Jan to Sep 2008. As we can see, the number of ticket is not always related to the number of sites operated by a given ROC (e.g. this is the case of Italy and South East Europe). This can be related to a number of factors, such as the middleware used and the middleware update frequency in the region. Secondly, the reports used in this study, only count the tickets managed via the central ticketing system (i.e. tickets limited to the regional scope are not included), so the numbers considered in this study may be partly incomplete. Image:GGUS_tckts.jpg Figure 6: number of EGEE GGUS tickets per ROC (Jan-Sep 08), and comparison with the number of sites operated in each region

[edit] B.2.2 Usage metrics: Number of Grid users per NGI

The distribution of Grid users among countries involved in EGEE, SEE-Grid and Baltic-Grid infrastructures was estimated during Jan 2009. The purpose of this analysis was two fold: firstly, to get an idea of what type of procedure can be put in place to calculate the density of users per country, and secondly to verify the level of correlation between the numbers of users, the amount of tickets handled in a country and its size. 25 production VOMS servers have been queried to get a list of registered VOs and the related users with certificates released by an accredited Grid CA. The number of VOs registered at the time of writing is 139, and they are very different in size. For each valid VO in the list, a query was issued to extract the list of the member certificates registered. This list of users was subsequently reprocessed to:

  • remove those VOs enabled for testing, demonstration activities, or past projects, and as such not used for production activities;
  • remove certificates from non-accredited CAs (expired CAs, CAs for testing and demonstration activities, etc.);
  • remove double entries in case of multiple certificates hold by individual users;
  • avoid double counting in case of a user certificates that is part of two or more VOs;
  • remove the Kerberized CA of Fermilab (extensively used by the CDF VO), where a single user typically olds three or more different X.509 certificates;

As at the time of writing, most of the accredited Grid CAs can be easily associated to a given European country, the distribution of users among countries was determined from the country associated to the respective certificate. The three notable exceptions are the CERN CA (arbitrarily assigned to Switzerland even if it is a European International Research Organization, and many certified users hold both a certificate from the home institute and a certificate from CERN), and Baltic-Grid and SEE-Grid, which both include users from multiple countries.

Those CAs with less than 10 registered users were not counted, and only the top 30 VOs were included in the diagrams which follow.

Table 4: Grid users per country and per CA in descending order
CA ShortName Country Users %
CERN Trusted Certification Authority CH 902 14.42%
GridKa-CA DE 897 14.34%
DOEGrids CA 1 USA 789 12.62%
INFN CA IT 718 11.48%
GRID-FR FR 497 7.95%
Baltic Grid Certification Authority BalticGrid 398 6.36%
UK e-Science CA UK 351 5.61%
IRISGridCA ES 224 3.58%
NIKHEF medium-security certification auth NL 214 3.42%
HellasGrid CA 2006 GR 199 3.18%
Russian Data-Intensive Grid CA RU 144 2.30%
Academia Sinica Grid Comp. Certification Authority & Mercury TW 231 3.69%
Grid Canada Certificate Authority CA 116 1.85%
KEK GRID Certificate Authority JP 93 1.49%
Polish Grid CA PL 78 1.25%
gridca-cn CN 63 1.01%
CyGridCA CY 46 0.74%
IUCC IL 45 0.72%
AEGIS-CA RS 43 0.69%
SEE-GRID CA SeeGrid 37 0.59%
TR-Grid CA TR 36 0.58%
BEGrid CA BE 33 0.53%
LIP Certification Authority PT 31 0.50%
AustrianGrid CA AT 24 0.38%
CESNET CA CZ 23 0.37%
KISTI Grid Certificate Authority KR 22 0.35%

Image:user_distr.jpg Figure 7: Users distribution per country/project

[edit] B.2.3 Size metrics: Number of production sites and Total computing power

The current size of the 36 NGIs operated in the framework of the main infrastructural Grid projects in Europe (namely, BalticGrid, EGEE, and SEE-Grid) has been estimated to verify how well size metrics can be composed to estimate their size today. The current status of several NGIs shows that size cannot be just based on singe metrics such as the number of sites, or the amount of resources provided. For example, as shown in Figure 36, some countries provide a large fraction of CPU power, but it is concentrated in a relatively small number of sites (e.g. France, UK, etc.), while other countries feature a large number of sites and a small average amount of computing power per site (e.g. Italy, Greece, Russia, etc.). Consequently, we tried to estimate the NGI size according to a different formula. The size of an NGI has been estimated by considering the overall amount of CPU offered and of the number of sites operated7. The gstat tool, gathering information from the production Grid information services, was used in this exercise as a source of input. Data was collected at different times during the second half of October 2008. The reference parameters were the total number of CPUs (Ctot) and sites (Stot) and the respective number of CPUs CNGI(NGI) and sites SNGI(NGI) per NGI currently available in the EGEE, BalticGrid and SEE-Grid infrastructures. Only production sites have been considered. The Size of an NGI was calculated according to this formula:

Size(NGI) = ½ [CNGI(NGI) / Ctot * 100 + SNGI(NGI) / Stot * 100] Note that at the time of writing, the number of sites belonging to NDGF – a site distributed across DK, FI, NO and SE – is equal to seven. Figure 2 illustrates the size distribution in Europe, estimated as the percentage of the overall size of EGEE, plus SEE-Grid and BalticGrid.

Image:NGI_size.jpg Figure 8: NGI size distribution (size %)

[edit] B.3 NGI Effort distribution: Simulation

In this paragraph we considered the NGI size estimation formula illustrated above, we propose a linear function to assign effort according to the NGI size, and we compare – where possible – the result to the current level of funding of NGIs in EGEE.

[edit] B.3.1 Current funding in EGEE-III

The following table summarizes the level of Operations and Security funding per NGI provided to countries in EGEE III. For every NGI this amount has been computed from the overall amount of funding of the project, by calculating the proportion between FTE assigned to the activity SA1 and the number of FTE provided to the NGI for other project activities. Data were extracted from the EGEE Consortium Agreement v1.8. The funding level per NGI is compared to the number of CPUs offered and the number of sites operated (in this case not including resources from BalticGrid and SEE-Grid).

Image:opsec_egee.jpg Figure 9: Operations and Security funding in EGEE-III The following table provides detailed information on the current number of FTE funded by the EC for every NGI/EIRO in EGEE III.

Table 5: NGI/EIRO distribution of funded FTE in EGEE III for operation and security activities
AT 1.54
Belarus 0
Belgium 0
Bulgaria 2.5
CERN 17.5
CH 1
Cipro 1.96
Croatia 1.96
CZ 2.41
DE 16.33
ES 13.2
Est. 0
FI 1
FR 18.75
GR 5.46
HU 1.58
Ireland 1.5
Israel 2.16
IT 19.5
Latvia 0
Lith. 0.00
NL 8.5
NO 0
PL 6.33
PT 4.16
Romania 2.37
Russia 17.66
SE 5
Serbia 2.29
Slovakia 1.37
Slovenia 0.66
Turkey 2.75
Ucr. 0
UK 15.5

[edit] B.3.2 Effort distribution

Given the NGI size function illustrated previously (given by the combination of the number of sites and the amount of computing power provided), a linear function was defined to compute the NGI effort as a function of its size. In this function the minimum number of assigned FTE was defined to be 2 FTE (for small NGIs), while the maximum is 22 (for larger NGIs). In particular, the number was calculated by multiplying the size to a coefficient (2 in our case) and by rounding up the result to the minimum larger integer. The effort was capped to 22 FTE or fixed to 2 FTE in case of out-of-range results. The resulting FTE distribution is illustrated in Figure 4.

Image:NGI_FTE_distr.jpg Figure 10: simulated distribution of FTE among NGIs

[edit] B.3.3 Conclusions

The simulated FTE distribution provides results very similar to the current EGEE effort distribution, and to the distribution obtained from keys where manpower is assigned proportionally to the Gross Domestic Product of a country. EGI_DS plans to review the effort distribution function illustrated above to take into account additional metrics as discussed, and possibly new metrics according to the operational tools available in the future.

[edit] Appendix C

[edit] Life Sciences SSC

The Life Sciences community is very large and heterogeneous, and although it will probably be the case that in the beginning of EGI there might be just one SSC for this entire community, there is a high likelihood that, as grid usage in the field evolves, more specialised SSCs may spin off from the original one. There are several thematic clusters of LS grid users: Bioinformatics, Medical Imaging, Drug discovery and Health. The needs of these communities are fairly homogeneous with the exception of certain security requirements (anonymisation of data) for the Health group. All the LS communities have expressed a need to access cluster grids and supercomputers, and are thus very interested in the integration of these resources on the grid. At the same time, there are about 8 ESFRI design studies related to life sciences which are establishing their own infrastructures – in particular sustained by the ELIXIR infrastructure – and need to be properly interfaced with EGI:

Image:esfri.jpg Figure 11: General schema of the planned ESFRI infrastructures. Image courtesy of A. Lyall (Elixir).

In addition to the needs mentioned above, the LS community expects more friendly user interfaces and is prepared to dedicate specific effort to the design of a Life Sciences Gateway, which will serve not only to facilitate access and use of the grid by its growing communities, but also as a template for gateways in other disciplines. Furthermore, the importance of international standards (OGF, Web Services) cannot be overestimated, as the community requires integration of resources into european infrastructures and initiatives, such as the ESFRIs and the Virtual Physiological Human [ref]. A possible scenario for this SSC in relation to the ESFRIs could be the following:

Image:lifeSSC.jpg Figure 12: The LS SSC and two ESFRI SSCs

[edit] VO Structure

In terms of usage typology, the community can be divided into three categories:

  • Research Teams coming together for a limited time (at the National, European or International level) to constitute a project which pursues some research objectives. This use case is very well discussed in the documents. These users are keen to have their own VO for the duration of their project and therefore their own scientific gateway as well.
  • Research teams having a continuous scientific activity that requires regular access to limited resources (equivalent to 1-2 CPU years) and from time to time, access to important resources (equivalent to 10-20 CPU years). These people are the customers of the biomed VO that they join once for all. The biomed VO should have its own scientific gateway.
  • The ESFRI infrastructures will structure communities with well defined long tern scientific topics and therefore specific requirements. Each ESFRI should have its own international VO, its own scientific gateways and be managed from the ESFRI collaboration board.

[edit] UCS Layer

The Life Sciences SSC proposes to structure its UCS layer services as follows:

  1. One User Forum Representative and member of the UFSC; if possible, this individual should have assistance by at least a part time representative for each thematic subcluster, for a total of 1.5 or 2 additional FTE. These people also do coordination.
  2. One Grid Planning Officer to participate in the EGI MCB meetings. Possibly assisted by the VO managers (not counted here).
  3. 2 FTE for User Technical Support and other support tasks. The personnel above will also collaborate on collecting, evaluating and representing the requirements of the LS community to the relevant bodies – EGI Council, MCB, etc. The UF rep and GPO are also responsible for recommending specific actions (e.g. the creation of projects) for the development of new services.
  4. A team of 2 FTE to design and build the Life Sciences Gateway.
  5. One web content manager to work with the Gateway team and other information-related tasks.
  6. 1 FTE for coordination of Documentation and Training specific to the LS community.
  7. A team of 2 FTE for Consulting with new communities, Application Porting Support, and to maintain and monitor the Application Database for the Life Sciences.

The global UCS layer for the LS SSC is thus initially estimated at between 9 and 14 FTE.

Personal tools
hidden pages