From EGI Knowledge Base
This section defines the European Grid Initiative (EGI) Business Model (BM), i.e. the proposed EGI framework to create value. The goal of EGI is to provide services for a standard and easy access to potentially any type of resource from multiple distributed and heterogeneous domains, made available by the respective providers.
EGI is not a simple continuation of EGEE and other grid projects. Most existing grid infrastructure projects include in their Consortia specific national Resource Providers or Research Institutions and naturally satisfy mostly their specific requirements. In contrast, the EGI model for sustainability is built on each member state’s establishment of its own NGI which will be responsible for the provision and operation of a national grid infrastructure satisfying all the Resource Providers and Research Institutions of its country, and for representing these in the EGI Council and in the relations with EGI.org and the other NGIs. Some structuring of the national grid infrastructure efforts has begun with EGEE III with the constitution of Joint Research Units (JRUs) which need to be leveraged by EGI.
The EGI service offer is organised in a non-hierarchical NGI-based environment. It is governed by the subsidiarity principle, meaning that tasks which are efficiently fulfilled at the national or regional level should be performed at that level by the NGIs.
 Resource Sharing
The distributed resources to be accessed can be dedicated to users from a given community (the so-called Virtual Organization – VO), or shareable by users from different VOs. In the former case, EGI supports intra-VO sharing, while in the latter cross-VO sharing is performed. The primary mandatory purpose of EGI is to enable intra-VO sharing, i.e. the possibility to get access to the distributed pool of resources allocated to a given VO. In addition to this, EGI optionally supports cross-VO sharing (if requested by some of its stakeholders and if technically feasible). Cross-VO sharing is a resource supply model that addresses the needs of users that do not own (neither directly nor indirectly) resources, but that are willing to pay for the best-effort access of distributed resources.
Cross-VO sharing needs to be transparent to its primary users (the amount of resources negotiated with the providers must be available to them at any time when needed) and is technically implemented by the providers. This is currently technically enabled by various mature implementation techniques which allow fair sharing by giving cross-VO access to resources only when resources are left idle by their primary users.
The EGI BM optionally supports full cross-VO sharing (it is an optional service offered by the NGIs) to create added value to the respective Resource Providers and the related Funding Agencies, whose typical main interest is the optimal usage of available funds. EGI promotes and encourages the maximum of cross-VO sharing because this is a value that EGI can offer to Research Institutions, Funding Agencies and Resource providers (its important stakeholders).
 Main Actors in the European Grid Initiative (EGI)
|1.||The National Grid Initiatives (NGIs), which are the primary members of EGI|
|2.||EGI.org, the new common organization owned by the EGI Members|
The NGI is a new legal national organisation responsible for:
- Establishing, managing and operating the national Grid infrastructure to an agreed level of service and ensuring its integration with the European e-infrastructure.
- Maintaining relationships with its national stakeholders:
- Research Institutes (RIs) and Research Projects carried out by Virtual Organizations constituted by Research Teams (RTs)
- Resource Providers (or centres) which offer resources to support the computing needs of the RTs in the country. The NGI and its Resource Providers form a national "business alliance" to jointly develop and “sell” a specific national marketplace solution (the national grid infrastructure) to their national researchers, each with its specific responsibilities
- Mobilizing the national funding and resources to guarantee the sustainable availability and operation of the national grid infrastructure as required by national users and to contribute to EGI.org for the common tasks.
- Representing all its national stakeholders in the EGI Council and in the relations with EGI.org; have the capacity to sign the Statutes of EGI.org – either directly or through a legal entity representing it.
- Contributing to the decisions of the EGI Council and the EGI.org technical bodies regarding international standards and to the EGI policies and quality criteria, and ensuring adherence at the national level to the agreed criteria.
- supporting user communities (application independent, and open to new user communities and resource providers).
EGI.org is a new legal organisation owned by the NGIs (and interested European Organizations), that will provide those common general services and tasks that are most effectively done centrally rather than duplicated by each NGI. These include the overall coordination of the operation of the infrastructure, of the user and application support and of the middleware procurement, standardization and final certification together with few general managerial positions.
At this point in time, all NGIs in Europe are at different stages of their implementation, ranging from individuals claiming to represent an NGI to early implementations of NGIs collecting all national players within some form of association, with a preliminary legal status and offer of services. During the development of EGI, these early forms of NGIs are expected to evolve into legal entities, which are able to collaborate on formal grounds in the European landscape. The EGI effort intends to support this development until a mature level for all NGIs in Europe is achieved. The NGI’s are fully autonomous in their choices, within the boundary condition of the EGI model and the established EGI policies and rules.
 Use Case
Research Teams (RTs) in general come together for a limited time (at the National, European or International level) within a Virtual Organization (VO), to constitute a project which pursues some research objectives. Such project is normally approved by peer review committees (acting at national and/or European or International level), set up by the involved Research Institutions or funding Agencies, that allocate the necessary funds including those for the IT resources.
VOs have different requirements on the IT resources they need to share, their usage model and the set of baseline services which EGI has to offer to enable this, e.g.: authentication and authorization services, accounting services for NGIs and VOs, services for data sharing at different levels of abstraction, services for compute sharing for different types of resources, monitoring services, etc. The Research teams belong to different Research Institutions (Universities, Laboratories, Applied Research Institutions, etc).
Resource consumption may be organised in different ways through VOs. A VO may fulfil its resource requirements through resources from its constituent research institutions, or through resources provided by a resource provider (either by another research institution or by the market) or any other means which fits the VO’s needs.
An e-Research project needs to make available to its members a set of software tools which enable the secure sharing of all the partner Organizations’ “local” IT resources and data located in the different administrative domains (Requirement n.1).
Such sharing may concern the CPU cycles of the commodity clusters used for the analysis of HEP, BIO, Astro… data, the fast interconnect parallel systems for the MPI applications of Computational Chemists, Earth Observation, Biomed, Weather forecast etc., the files contained in distributed storage systems for image visualization of Astronomers, Medical doctors etc, the metadata located in distributed archives systems related with a large variety of applications, etc.
The sharing occurs through software services (grid middleware) which expose to the user a uniform interface hiding the local diversities and allowing a distinct level of authorization according to the member’s role and the agreed project policies. In this way a project common pool for all kind of resources is created, allowing the most efficient exploitation by the project partners of all the available distributed resources and data.
The reference Resource Centres/Providers are asked by the VOs to operate the set of services which enable them to reach the above goal. A distributed accounting at the level of Institutions is also required together with tools to monitor the activities and eventually a support to the VOs to enable their legacy user applications to execute in this multi-administrative domain pool.
It should be noticed that many VOs, especially the new ones, tend to consider their sharing requirements always a special case which need the development of integrated special vertical services dedicated to them (see LHC experiments, ESFRI, etc). However the use of a layer of services common to the other VO’s, when possible, has the advantage of manpower saving, for the development and even more important for the seamless operation and the maintenance of the more fundamental baseline services enabling the sharing, as well as the advantage of allowing partial sharing with different VO (thus increasing the pool of available resources), see below.
It is economically much more convenient and efficient for the funding bodies as the Research Institutions, or in general the national and European funding agencies, to promote, support and fund the procurement and the operation of a common, robust, secure, certified set of baseline grid services which the EGI.org, at European level, and the NGIs, at national level, can offer and operate as part of the general EGI/NGI e-Infrastructure to enable global sharing, rather than a chaotic set of tools that each VO may freely ask to adopt or develop (Economy argument).
In this way most of the past investments made at EU or at national level will be continuously reused for the benefit of the new VOs and the high level special services that these may still need to develop will be less expensive and founded on a mature layer with a longer lifetime and wider user spectrum (Reuse and long-term perspective argument).
The point of view of a better global return for the money invested by the funding bodies, pushing for the creation of global pools based on a well defined certified set of services, constitutes the important Requirement n.2 for EGI which could possibly not always be shared by all VOs.
Resource Centres/Providers have obtained so far from external providers (EU projects or Middleware Consortia) the general grid middleware services they need to operate. Obeying to different VO consolidated practices in some cases they are currently supporting more than one middleware solution. To avoid to charge the high costs of the support of chaotic and very expensive multiple special environments and the real operational challenge of being able to offer a well defined quality of service for multiple solutions, they will normally appreciate and support the coordinated action of EGI.org and of their national NGIs in moving towards a progressively unified solution for the services they will have to operate in line the funding Research Institutions and Funding Agencies but moved by the need of simplified and easy operations (Operational argument). This constitutes the Requirement n.3 for EGI.
Of course while the above 3 Requirements provide the foundation of the EGI.org and NGI’s Business Model they should not be perceived as limiting by any means the strategies that each NGI could adopt in its reference country. It may well happen that some countries in addition to the common set of supported services obeying the EGI rules and policies decide to continue to support also "islands" based on more private or not certified services.
 Template and Related Terminology
The template by Osterwalder (2004) was adopted to illustrate the EGI business logic. Nine building blocks are identified:
Offering, composed of:
- value proposition, the products and services a business offers;
Infrastructure, composed of:
- core capabilities, the capabilities and competencies necessary to execute a company's business model,
- partner network, the business alliances which complement other aspects of the business model,
- value configuration, the rationale which makes a business mutually beneficial for a business and its customers;
Customers, composed of:
- target customer, the links a company establishes between itself and its different customer segments,
- distribution channel, the means by which a company delivers products and services to customers,
- customer relationship, the target audience for a business' products and services;
Finances, composed of:
- cost structure, the monetary representation of the means employed in the business model,
- revenue streams, the way a company makes money through a variety of revenue flows.
|1.||Service "Secure sharing of distributed IT resources and data"
Among these are CPU cycles available from commodity clusters or fast-interconnect parallel systems supporting MPI applications, storage space, geographically distributed data as files for image visualization, metadata from distributed archives, etc. This constitutes the main joint offer made by EGI.org and the NGIs to all their customers: Research Teams, VOs and Resource Providers.
|2.||Service "Middleware provision"
Resource sharing occurs via the EGI e-Infrastructure which exposes to the user a uniform interface, thus hiding the local diversities and allowing a distinct level of authorization according to the member’s role and the agreed project policies. EGI collects requirements from the Research Teams and Resource Providers and ensures that the middleware offered (originally provided by the Middleware Consortia), meets those requirements, is certified, interoperates and enable the creation of common pan-European pools for all sort of IT resources and data. Note however that each NGI and Resource Provider is free to adopt any other service implementations to continue to support more private or regional islands. EGI has very little to say about this.
|3.||Services for User Communities
EGI supports RTs and user communities that need to use the e-Infrastructure and offers specific support in gathering requirements from the RTs and representing them vis à vis the middleware (and other software) providers. This is carried out in part by a small central team plus a set of User Community Services (optional services). Activities such as providing support to porting activities and training of the users and administrators is generally delivered through NGIs, either on a national level or via specific agreements with other NGIs in the context of the creation of a clustered set of services referred to as a Specialised Support Centre.
|4.||Service "Resource brokerage"
(optional service): gathering resources for European and international RTs can be a lengthy and difficult task, especially for smaller RIs who do not own resources. If requested (this is an optional service), EGI can facilitate the process by connecting customers with the resource providers via the relevant NGIs, and by ensuring that the RT has access to shared resources in the e-Infrastructure. The establishment of an SLA and the related contract only involves the RI and the resource provider.
|5.||Service "Help desk"
support is provided to resource providers and users in case of problems with the use and the operation of the local and global e-Infrastructure respectively.
 Partner Network
NGIs and related Resource Providers (RPs). A given NGI and the national/regional RPs join to form a national partner alliance to jointly offer the services needed to implement a global pan-European shared pool of resources as the most efficient way to satisfy the requirements of the national RTs. Resources from national/regional resource owners, are made available to customers via RPs. One owner is also a provider if it both owns and provides resources to the Grid (RIs which operate their own resource centres which share resources via the Grid, are typically both owners and providers).
The NGI represents the RIs and the Resource Centres/Owners of one country for all matters concerning the national e-Infrastructure towards the other NGIs, the EGI.org and the European Commission.
The existence of a National Grid Initiative and of EGI.org operating the general European e-Infrastructure streamlines the actions required to share resources by reducing the O(N2) problem of establishing bi-lateral contacts with other providers.
In addition, the NGI can provide help and recommendations to guarantee the “balance” between the national amount of IT funding (which determines the national resource global offer) and the national RI global demand for each individual type of resource. It should be stressed that in the EGI model at the substantive level each country is completely responsible for guaranteeing such balance.
Note that this article only describes activities and offerings by an NGI that are needed to support international RTs and to be seamlessly integrated in the European e-Infrastructure. Any other activity and offering that is specifically targeted to address local needs, are not part of the EGI BM. For these latter activities the NGI is completely free to adopt a BM of its choice. Similarly the role of RPs described in this document complements the traditional resource provider BM typically based on the offering of IT resources often according to annual flat service fees (needed to cover capital expenditures).
EGI.org offers those services that are of common interest to all NGIs, such as the overall coordination framework for the development, operations including security across national borders and user support activities, and
- the hosting of operation/maintenance of operational tools and critical services which are more conveniently performed centrally.
- the overall coordination framework for the development, operations including security across national borders and user support activities.
The main customers of EGI.org are the National Grid Initiatives (NGIs), which operate the Grid infrastructures in each country.
Middleware Consortia: the current environment of EU middleware development consists of multiple teams of experts specialized in the development of one or more of the middleware services. At this time, these teams are organized around Middleware Consortia with additional distributed teams. They normally work in close collaboration with the US teams which provide the solutions for TeraGrid or the Open Science Grid (OSG) like Globus and Condor. In order to leverage the existing clusters of competence it is then advisable to maintain this model based on decentralized teams while introducing with EGI an effective pan-European technical coordination.
Almost all the EU middleware developments have been supported so far by a significant EC co-funding which has allowed especially for gLite, but also for other solutions, the strong interaction between Operation, Application and Middleware activities (included in the same project) and the delivering of the services with the functionalities and quality that user VOs and site administrators needed for their daily work. The same has happened for Globus and Condor in US which have received substantial support by the National Science Foundation (NSF) or the Department of Atomic Energy (DOE).
ARC, gLite, and UNICORE represent the today leading edge European reference MWare solutions that have demonstrated the capability to support large and diverse pan-European communities with a great variety pf requirements and huge amounts of computation and data needs. There are some European user communities also relying on the US Globus services, but these Globus based grids are usually much smaller and independent, not creating a truly general shared grid infrastructure.
The Open Source software development and the offering of related maintenance and support services by the middleware Consortia and the additional European teams of experts has a less straightforward economic sustainability model compared to current service activities (e.g. network bandwidth offering by the Dante/NRENs organization). The Consortia cannot expect to immediately gain economic sustainability through standard fee-for-service contracts, as the Open Source software can be freely downloaded and used by skilled communities (e.g Linux is used by the HEP scientific communities without buying extra services).
A possible way to overcome these difficulties and to make profit of the significant progress made so far is the creation of an Open Source Universal Middleware Distribution (UMD) under the steering of EGI which could progressively include all certified interoperable services adopted in the pan European e-Infrastructure.
Specific actions to expand the usage of UMD by less skilled user communities should be undertaken, in order to progressively increase its economic sustainability through this additional income. Sustainability should rely on the generalization of the UMD services to adapt to requirements from a growing user community, including business and government (similarly to the case of Red Hat or the Apache consortium). This process will take time and will require continuity of explicit support of the development efforts leading to general and standard products (the case of pre-competitive services).
The target customers of middleware consortia are EGI.org, NGIs and, indirectly, Resource Centres/Owners and RTs.
 Core Capabilities (IT Resources)
- T resourceswhich can be brokered via different channels:
- RIs already have them internally available via consolidated centres which host their own IT resources and data infrastructures. These are made available to the RTs according to well defined amounts and usage policies.
- RIs acquire and operate their own resources only when needed (this is typically applicable when the demand is imited).
- RIs sign a long term contract/agreement with external RPs, which guarantee that the agreed resources are provided, and host the RI data (for example, at the cost of a negotiated flat rate for a given yearly share).
- Operation of Grid resources to make them shareable.
- Operation of Grid technical services (at the RP and NGI level): these operate the set of services needed by the customers for the sharing of resources. Additional tools need to be operated, for example for the accounting of the RI resource usage and for monitoring.
- Cooperation with partner NGIs and EGI.org to define common policies, specifications, standards, procedures, etc. for the usage of the resources included in the shareable common pools
- Operation of Grid core technical services at the NGI level (workload management services, data management services, catalogues, etc.) following EGI operational requirements and recommendations, and a number of other auxiliary services such as the issuing of certificates, authorization services, etc.
- Rollout and deployment of Grid middleware to the resource centres: the NGI ensures that the middleware services adopted by the partner resource centres comply with the international standards and with the set of EGI policies and quality criteria necessary to guarantee the full functionality and interoperability which are at the foundation of general pan-European pools.
- Operation of the central monitoring and accounting service of the national e-Infrastructure. Accounting is needed to provide information about the overall resource usage to national RIs or NGIs.
- Technical coordination (if and when needed) of the customization of middleware components released by Middleware Consortia.
- Ticketing system and help desk (NGI): support to the RTs is needed to facilitate the enable the running of existing user applications on the e-Infrastructure.
- Training of users and resource centre administrators.
- Resource Brokerage (optional service) towards other NGIs when some required services are not provided by the national Resource Centres/Owners ;
- The NGI ensures global monitoring and accounting at the national level for VOs and other NGIs, in order to account for the global resource usage balance between the NGIs.
- Offering and operation (possibly via outsourcing) of the high level services required by the national customers.
Note that some of the aforementioned activities can be considered services which in due time could be offered by the NGI under annual contract fees. There is also a large number of practical possibilities to be chosen by an NGIs to implement such services ranging from delivering them directly to completely outsourcing most of them to one or more Resource Centres/Owners.
- Coordination of middleware development, standardization, operation and application support activities carried out by the partner NGIs.
- Liaison with international bodies, standardization bodies and other e-Infrastructures.
- Infrastructure for middleware testing and certification: to check if middleware released by MC meets the RT requirements and to certify its interoperability.
- Provisioning of Grid services, support, dissemination and outreach activities at the European level, complementing and coordinating the NGI ones.
- Integration, testing, validation and packaging of software from Grid middleware providers, along with its distribution.
- Definition of common policies and procedures.
Middleware Consortia and other development teams
- Middleware development and maintenance to satisfy the EGI requirements
- Active involvement in international standardization efforts