Germany
From EGI Knowledge Base
|
| |
| |
| NGI Full Name: | D-Grid |
|---|---|
| Acronym: | D-Grid |
| URL: | http://www.d-grid.de/ |
In September 2005 six Community Grid projects and the D-Grid Integration Project (DGI) started to build a sustainable Grid infrastructure in Germany, to establish methods of e-science in the German scientific community. D-Grid doesn't form an institute nor a company. It's a 5-year project funded by the German government. It is lead by the D-grid Steering Committee which reports - through the Chairman - to the government.
D-Grid is officially recognized and funded by BMBF, the German Federal Ministry for Education and Research.
Contents |
[edit] Key Objectives
The aim of the D-Grid Initiative is to design, build, operate and deploy a national-scale digital infrastructure of distributed, integrated and virtualized high-performance computing resources and related services to improve global collaboration in research and industry and to enable the processing of large amounts of data and information; to collaborate on an international basis; exchange information, documents and publications in real time; and guarantee efficiency and stability even with huge amounts of data from measurements, laboratories and computational results.
In the past, organizations strived to successfully collaborate within their own communities. But in D-Grid, for the first time ever, all these different communities are working together on a single, inter-community grid middleware platform to share computing resources, middleware tools, and applications. This will result in an IT infrastructure which is interoperable with other international grids, scalable and extensible for more community grids in the future, and available for all of our scientists for national and international collaboration.
[edit] Project History
In 2003, German scientists and scientific organizations started the D-Grid initiative, publishing a strategic paper in July 2003. This paper examined the status and consequences of grid technology for scientific research and recommended a long-term strategic grid research and development initiative.
The e-Science Initiative and D-Grid started on September 1, 2005. The German Federal Ministry for Research and Education (BMBF) is funding over 100 German research and industry organizations with 100 Million Euro over the next 5 years. For the first 3-year phase of D-Grid, financial support is approximately 25 Million Euro. The goal is to design, build and operate a network of distributed, integrated and virtualized high-performance resources and related services to enable the processing of large amounts of scientific data and information. The Ministry for Research and Education is funding the assembling, set-up and operation in several overlapping stages:
D-Grid 1, 2005-2008: IT services for scientists, designed and developed by the ‘early adopters’ of the computer science community. This global services infrastructure will be tested and used by so-called Community Grids in the areas of high-energy physics, astrophysics, medicine and life sciences, earth sciences (e.g. climate), engineering sciences, and humanities.
D-Grid 2, 2007-2010: IT services for scientists, industry, and business, including new applications in chemistry, biology, drug design, economy, visualization of data, and so on. Grid service providers will offer basic services to these users.
[edit] Organisational Form
The D-Grid Steering Committee (StA) consists of the representatives of the community grid projects and of the four work package leaders of the D-Grid Infrastructure (DGI) project. The StA meets 12 times a year, two telephone conferences (one hour) and one face-to-face meeting (5 hours) per quarter. The Agenda includes mainly inter-project related issues such as co-operations within D-Grid and beyond (international), workshops and conferences, joint developments, tests, and work papers, inter-project dependencies, and so on. The StA meetings are chaired by the D-Grid coordinator on behalf of the German Government.
[edit] Grid Resources and Services
The current D-Grid resource infrastructure consists of resources offered by the participating partners. In addition, in 2006, the Government funded dedicated grid resources worth 5.5 MEuro, for specific grid-related development, test and production work. These computing and storage resources are distributed among 8 Backbone and 17 partner nodes. In 2007, The Government funded additional grid computing and storage resources worth more than 20 MEuro, to encourage grid production activities.
During the last two years, the D-Grid Infrastructure project (DGI) developed the following grid services which are currently available for the D-Grid community projects:
- The core D-Grid infrastructure offers central grid services. New resources can be easily integrated in the help-desk and monitoring system, allowing central control of resources to guarantee sustainable grid operation.
- DGI offers several grid middleware packages (gLite, Globus und Unicore) and data management systems (SRB, dCache und OGSA-DAI). A support infrastructure helps new communities and "Virtual Organizations" (VOs) with the installation and integration of new grid resources via a central Information Portal ("Point of Information"). In addition, software tools for managing VOs are offered, based on the VOMS and Shibboleth [36] systems.
- Monitoring und Accounting prototypes for distributed grid resources exist, as well as an early concept for billing in D-Grid.
- DGI offers consulting for new Grid Communities in all technical aspects of network and security, e.g. firewalls in grid environments, alternative network protocols, and CERT (Computer Emergency Response Team).
- DGI partners operate "Registration Authorities" to support simple application of internationally accepted Grid Certificates from DFN (German Research Network organization) and GridKA (Grid Project Karlsruhe). DGI partners support new members to build their own "Registration Authorities".
- Core D-Grid is offering resources for testing, via middleware systems (gLite, Globus and UNICORE). The Portal Framework Gridsphere serves as the graphical user interface. Within the D-Grid environment the dCache system takes care of the administration large amount of scientific data.
[edit] Grid Projects
[edit] D-Grid Infrastructure Project
Scientists in the D-Grid Infrastructure project DGI are developing and implementing a set of basic grid middleware services which will be offered to the other Community Grids. For example, services include access to large amounts of distributed data in the grid, the management of virtual organizations, monitoring and accounting. In addition, a core-grid infrastructure will be built for and provided to the community grids for testing and experimentation. High-level services will be developed which guarantee security, reliable data access and transfer, and fair-use policies for computing resources. This core-grid infrastructure will then be further developed into a reliable, generic, long-term production platform which can be enhanced in a scalable and seamless way, such as the addition of new resources and services, distributed applications and data, and automated “on demand” provisioning of a support infrastructure.
[edit] Community Grid Projects in D-Grid-1
HEP-Grid - The High-Energy Physics community is developing applications and components for evaluating terabytes of data from large high-energy physics experiments, including the Large Hadron Collider at CERN.
ASTRO-Grid – The Astrophysics community combines research institutions in astronomy and astrophysics into a single, nationwide virtual organization for distributed collaboration and integration of distributed astronomical data archives, instruments and experiments.
MEDI-Grid represents the medical and bio-informatics community in Germany. It focuses on application scenarios in medical image processing, bioinformatics, and clinical research, and their interaction.
C3-Grid – Collaborative Climate Community has the goal to develop a highly proficient grid-based research platform for the German earth-system research community to efficiently access and analyze distributed, high-volume scientific data from earth-system modeling and observation.
InGrid – Industry applications. A grid environment will be developed to enable modeling, optimization, and simulation of engineering applications from areas such as foundry technology, metal forming, groundwater flows, turbine simulation, and fluid-structure interaction.
TextGrid is developing tools and standard interfaces for publication software, modules for scientific text processing and editing, and administration and access to distributed data and tools on the grid.
WISENT is developing tools and methods in the area of energy meteorology to accurately forecast energy usage to be matched with energy provisioning on demand.
[edit] Grid Projects in D-Grid-2
In D-Grid-2 there is a stronger focus on grids in the industry, with more than 40 companies participating. The new grid projects will all be using the D-Grid core infrastructure developed during the first two years of D-Grid.
AeroGrid – Aerospace Engineering: The AeroGrid project aims at providing an efficient grid-based working environment for the national aerospace research community. The AeroGrid environment will be a permanent and effective grid infrastructure for the cooperation between industry, research centers, and universities in aerospace engineering and research.
BauVOGrid – Construction Industry: BauVOGrid develops an extensible Construction-Community-Solution as the basis for next generation VO grid services. The goal is to improve the structure, functioning and operability of virtual organizations in the construction sector via a flexible and reusable infrastructure achieved by a D-Grid-based approach.
BIS-Grid – Business Information Systems: The BIS-Grid project intends to realize a horizontal Service Grid for business information systems. The overall goal is to enable grid technologies to be used for the integration of decentralized business information systems. This will be achieved by developing and providing organizational and technical extensions based on the current state of the art in grid technologies, EAI, and SOA.
Biz2Grid – Grid Technology for Enterprises: The main objective of Biz2Grid is to provide foundations for an effective application of grid technologies in enterprises. Business and economically driven questions have to be answered and technical challenges have to be solved. Two case studies from commercial scenarios within the automotive industry gain specific attention as “best practice” for future business grid solutions.
D-MON – Resource and Service Monitoring: D-MON’s vision is to realize a D-Grid wide monitoring architecture across several underlying, heterogeneous systems taking in consideration multiple resource providers, virtual organizations and the integration of different middlewares with independent monitoring and information systems which differ in functionality, information models, implemented standards and architectures.
F&L-Grid – Service Grid for Research and Education: The F&L-Grid project has the goal to set up a service grid for IT services for research and education. The project is based on a public/private partnership and the services will be provisioned on the base of the German Research Network (DFN). The concept is open to integrate new providers and also for an extension towards other market segments, specifically for small and medium enterprises.
FinGrid – Financial Business: the Financial Business Grid project strives to identify suitable services and processes in the financial services sector and to develop grid-based systems that enable financial service providers to reorganize their processes efficiently and to realize applications that have been impossible so far in terms of computational requirements. A rigorous empirical analysis of the potentials of grid from an economic perspective will be performed and grid-based prototypes designed and developed.
GDI-Grid – Spatial Data Infrastructure: The GDI-Grid project is integrating geo-information technologies with grid technologies in order to establish a Spatial Data Infrastructure Grid. While the Geo-Information Systems (GIS) and Spatial Data Infrastructures (SDI) mainly provide access to spatial data resources, the GDI-Grid will perform the processing of enormous amounts of spatial data. Evaluation is with three scenarios: spatial simulation of flood disasters - flood hazard; noise dispersion simulation for noise pollution predictions; and real-time route optimization for disaster management.
IVOM – VO Management: IVOM aims at designing a D-Grid wide management infrastructure for Virtual Organizations. Currently deployed VO management technologies developed by international VO management projects have been evaluated and compared with requirements from the German Grid Communities, ranging from PKI-based VOMS to Shibboleth-based my-Vocs. IVOM will also identify remaining gaps on the way to an interoperable and integrated VO management in D-Grid.
PartnerGrid – eCollaboration for the Industry: PartnerGrid develops a platform for the co-operation of companies on the basis of D-Grid. Special focus is on grid technology for small and medium enterprises. The collaboration platform will be tested and demonstrated with two typical industry scenarios: the collaboration of foundries and their customers in designing and optimizing casting processes; and the collaboration in the metal processing industry for the planning of forming processes.
ProGRID – Collaborative Product Development: Main objectives of the project are to utilize grid technology for collaborative product development and to demonstrate benefits by means of use cases for virtual product development. Three scenarios will deal with virtual verification and structure optimization. Another application scenario will connect product data management (PDM) and CAE systems. The fifth will be about collaborative product development and collaboration environment.
SuGI – Sustainability Concepts for Academia and Industry: Major task is to disseminate the knowledge of grid technology and to spread its use. Therefore, SuGI addresses small and Medium academic computing centers as well as small and medium enterprises, which still have not adopted grid technology. SuGI will offer training courses to the D-Grid communities.
[edit] Next Steps
Potential next steps, within the D-Grid-3 phase, will extend the D-Grid infrastructure with a business layer (including work on Service Level Agreements), and a knowledge management layer (including maintenance, utilization, and preservation of digital data and objects), encouraging and connecting global service-oriented architectures in the industry, and using this grid infrastructure for the benefit of our whole society.
The Vision: In the near future, on such an enhanced Internet, all kinds of service providers will offer their services for computing, data, applications, and more. On an enhanced World Wide Web, via secure Web Portals, we will access grid components like Lego building blocks, which enable us to dynamically build and dismantle grids ‘on the fly’, according to our specific needs. We will rent or lease the resources required and just pay for what we use. From a bird’s eye view, the business model for grid services will be similar to those for electrical power, water, or telephony: our payments will be based on widely agreed billing units which include cost for computers, storage, software, applications, work, electrical power, square footage for the equipment, and personnel for maintenance.
| Number of users with valid Grid Certificate: | currently over 250 |
| Number of sites (Resource Centres): | 30 |
| Number of CPUs: | 3000 - 5000 |
| Total storage (in TB): | 2000 |
[edit] EGI Functions: Current Rating
[edit] Functions proposed in survey of late 2006
| Coordination of infrastructure operations | No Opinion |
| Testing, certification and validation service including middleware | Quite Important |
| Managed resource centers to provide initial resources for new user communities | Not Important |
| Coordination of user and application support | Not Important |
| Coordination of dissemination and training efforts | Quite Important |
| Representation of European Grid efforts on standards bodies | Not Important |
| Representation of European Grid efforts with similar bodies from other continents | Quite Important |
Comments...
[edit] Other EGI Functions
Provisioning of functions which at present are not available in most national or regional infrastructures. The subsidiary principle should apply. Main focus should be on coordination of NGIs. It would be difficult to understand from an economic and from a policy point of view that resource centers should appear on a European level and furthermore NGIs are implicitely asked to fund these resources and their operations. The European organizational layer should be defined bottom-up (and not top-down).
