Derived clustered information
From EGI Knowledge Base
Contents |
[edit] Middleware
[edit] Smooth integration of new components
EGI should be prepared for accepting new MW components, coming from external or internal development, which will co-exist or replace one another eventually.
- Interoperability is critical. Despite interface standardization is necessary UC-NGI-D-standardization, it does not solve the problem completely UC-NGI-SW-elmroth, UC-NGI-NT-II.
- Replacable small components with well-defined functionality are prefered to huge monoliths UC-NGI-SW-elmroth
- Procedures have to be defined to accept new components coming from external development, including roles of EGI and NGI's
- It is useful to share MW components (and even service instances) among different user groups/communities. How candidates for sharing are identified? UC-NGI-D-tools
- User communities may reqiure completely new or tailored components. How these are provided, who triggers, decides and is responsible for the development? UC-CERN-pm-IV,UC-NGI-FR
[edit] Component lifecycle
- contributed components may not arrive in a state ready for production EGI has to define procedures for their "hardening", "experimental services" deployment may be an efficient way UC-NGI-CZ-VII
- maintenance and production updates of all components is an issue in the large-scale deployment, EGI should define procedures for both fast propagation of security updates to avoid vulnerability risks, as well as gradual deployment of new versions which may introduce backward incompatibility UC-cern-middleware
[edit] One or more MW stacks
- Running multiple middeware stacks is perceived unavoidable, at least in mid-term UC-KnowARC, UC-NGI-SW-elmroth
- Applications tend to build "grid of grids" UC-KnowARC, even defining services at the meta-level UC-VO-ILDG-access; the approach is probably unavoidable when extending outside Europe
UC-NGI-UK-III, UC-NGI-UK-V, UC-NGI-UK-VI, UC-NGI-UK-VII, UC-NGI-PL-infrastructure-I, UC-NGI-UK-XVI
[edit] Support of interactive use
- Visualization and steering of simulations UC-IEG-I, UC-IEG-II, UC-IEG-III, UC-IEG-IV, UC-IEG-V, UC-IEG-VI, UC-IEG-VII, UC-IEG-VIII, UC-IEG-IX
[edit] Security issues
- Security based on X509 certificates has clear limitations. The concept should be reviewed and reconsidered eventually UC-cern-middleware
- Deployment in highly restricted environments (firewalls etc.) must be foreseen UC-UPV-epidemiology, UC-UPV-medical-images, UC-UPV-metagenomics
- Advanced security models are required for sensitive data, e.g. medical application UC-UPV-epidemiology, UC-UPV-medical-images, UC-UPV-metagenomics
- On the contrary, relaxed security approaches have to be sought where appropriate UC-cern-middleware
[edit] Large redesign is required
- "rethinking grids from scratch", taking account of experience, with more strict control UC-NGI-NT-III
- Introduction of novice users cannot be achieved with the current middeware due to its design, its reconsideration is necessary UC-cern-middleware
[edit] Other specific functionality requests
- Co-operation of Storage and Compute element to prevent waste of resources and stalled jobs. UC-cern-middleware, UC-cern-user
- MPI jobs, even across clusters, efficient access to very large number of small files UC-IEG-I, UC-IEG-II, UC-IEG-III, UC-IEG-IV, UC-IEG-V, UC-IEG-VI, UC-IEG-VII, UC-IEG-VIII, UC-IEG-IX
[edit] Infrastructure and Operation
[edit] Joining the Grid
Definition of basic elements/subjects for interaction within European grid environment.
- user/user communities UC-NGI-FR-II-VOs-man-communication, UC-NGI-FR-II-VOs-man-general-II, UC-NGI-FR-II-VOs-man-general-III, UC-NGI-FR-II-VOs-man-general-IV, UC-NGI-FR-II-VOs-man-general-V, UC-NGI-FR-II-VOs-man-general-VI, UC-NGI-FR-II-VOs-man-production-I, UC-NGI-FR-II-VOs-man-production-II, UC-NGI-FR-II-VOs-man-production-III, UC-NGI-FR-II-VOs-man-production-IV, UC-NGI-FR-II-VOs-man-production-V, UC-NGI-FR-II-VOs-man-production-VI, UC-NGI-FR-II-VOs-man-software-I, UC-NGI-FR-II-VOs-man-software-II,
UC-NGI-FR-II-VOs-man-software-III, UC-NGI-FR-II-VOs-users-I, UC-NGI-FR-II-VOs-users-II, UC-NGI-FR-II-VOs-users-III, UC-NGI-FR-II-VOs-users-IV, UC-NGI-FR-II-VOs-users-V
- resource providers UC-NGI-PL-infrastructure-II, UC-NGI-D-continuity, UC-NGI-D-data, UC-NGI-D-infrastructure, UC-NGI-D-international-dimension, UC-NGI-D-standardization, UC-NGI-D-sustainability, UC-NGI-D-tools, UC-NGI-PL-infrastructure-IV, UC-NGI-UK-II
- service providers/developers
- local versus global access UC-NorduGrid
UC-IPC-I, UC-NGI-NW-I, UC-NGI-UK-I, UC-VO-ILDG-certificate, UC-NGI-UK-X, UC-NGI-UK-XV
[edit] Working with the Grid
Set of elementar steps required for seamless interaction with grid environment.
- user view (registration, ...) UC-NGI-UA-user, UC-NGI-UA-user-II, UC-cern-user, UC-NGI-CZ-IV
- provider view (allocation, ...) UC-NGI-FR-II-VOs-man-general-II, UC-NGI-FR-II-VOs-man-general-III, UC-NGI-FR-II-VOs-man-general-IV, UC-NGI-FR-II-VOs-man-general-V, UC-NGI-FR-II-VOs-man-general-VI
- security incidents UC-NGI-FR-I-sites-man-interaction-I, UC-NGI-FR-I-sites-man-interaction-II, UC-NGI-FR-I-sites-man-interaction-III, UC-NGI-FR-I-sites-man-interaction-IV, UC-NGI-FR-I-sites-man-operation-I, UC-NGI-FR-I-sites-man-operation-II, UC-NGI-FR-I-sites-man-operation-III, UC-NGI-FR-I-sites-man-operation-IV, UC-NGI-FR-I-sites-man-security
- users registration, access to VOs UC-NGI-CZ-VI
- operational support for VO management (VOMS, ...) UC-NGI-FR-I-ROC-man-grid-oper-I, UC-NGI-FR-I-ROC-man-grid-oper-II, UC-NGI-FR-I-ROC-man-monitoring-I, UC-NGI-FR-I-ROC-man-monitoring-II, UC-NGI-FR-I-ROC-man-monitoring-III, UC-NGI-FR-I-ROC-man-monitoring-IV, UC-NGI-FR-I-sites-man-general-I, UC-NGI-FR-I-sites-man-interaction-I, UC-NGI-FR-I-sites-man-interaction-II, UC-NGI-FR-I-sites-man-interaction-III, UC-NGI-FR-I-sites-man-interaction-IV, UC-NGI-FR-I-sites-man-operation-I, UC-NGI-FR-I-sites-man-operation-II, UC-NGI-FR-I-sites-man-operation-III, UC-NGI-FR-I-sites-man-operation-IV, UC-NGI-FR-I-sites-man-security
[edit] Organization of operations
What approach is the environment built on (best effort? SLAs?). Possibility to require to label resources as "EGI proofed/confirmed" (based od selected requirements).
- Grid operations administration UC-CSC-applications, UC-CSC-broker, UC-CSC-ca, UC-CSC-resources, UC-CSC-settingup, UC-CSC-tailoring, UC-NGI-D-continuity, UC-NGI-D-data, UC-NGI-D-infrastructure, UC-NGI-D-international-dimension, UC-NGI-D-standardization, UC-NGI-D-sustainability, UC-NGI-D-tools, UC-NorduGrid
- Grid of Grids-integration UC-NGI-NT-I
UC-NGI-UA-centers, UC-NGI-UA-infrastructure, UC-NGI-UA-support, UC-NGI-SK-cell-II, UC-NGI-SK-cell-III, UC-NGI-SK-kwf-infrastructure, UC-VO-ILDG-storage, UC-VO-ILDG-data-management
[edit] Policies
Set of rules to cope with international relations, responsibility to take care of this issue (international partner itself or special EGI group?).
- relation to non-European contacts UC-EELA-infrastructure, UC-EELA-newcomers, UC-EELA-WISDOM, UC-EUChinaGrid-infrastructure, UC-EUChinaGrid-installation, UC-EUChinaGrid-interoperability, UC-EUChinaGrid-networking, UC-EUChinaGrid-new-community, UC-EUMEDGrid-access, UC-EUMEDGrid-infrastructure, UC-EUMEDGrid-installation, UC-EUMEDGrid-integration, UC-EUMEDGrid-interoperability, UC-EUMEDGrid-Windows, UC-SEEGRID-IV, UC-SEEGRID-V, UC-SEEGRID-VI, UC-SEEGRID-VII, UC-SEEGRID-VIII
[edit] Commercial related issues
Definition of procedures for handling private sector requirements.
- interaction with private sector (no-JRU subjects, ...) UC-CSC-broker, UC-NGI-PL-end-users-I, UC-NGI-PL-end-users-II, UC-IPC-II, UC-IPC-III, UC-IPC-V, UC-IPC-VI
[edit] Training & User Support
Responsibility and way of delivering training/user support for work within the Grid.
- how and who UC-cern-user, UC-NGI-CZ-VIII, UC-EELA-newcomers, UC-SEEGRID-IX, UC-SEEGRID-X
[edit] Monitoring and Accounting
Definition of procedures how to evaluate the resources.
- what will be monitored, using which tools, who will have access to monitoring results and of which elements, possibility to allow different views (ROCs, VOs) UC-NGI-FR-I-ROC-man-monitoring-I, UC-NGI-FR-I-ROC-man-monitoring-II, UC-NGI-FR-I-ROC-man-monitoring-III, UC-NGI-FR-I-ROC-man-monitoring-IV
- reliability measurements UC-assessgrid
- problem recognition and tracking UC-NGI-PL-infrastructure-III
[edit] Resource provisioning
UC-bazaar-resources-I, UC-bazaar-resources-II, UC-bazaar-resources-III, UC-NGI-GR-V, UC-NGI-UA-VOs, UC-degree-join, UC-NGI-PL-application-I, UC-NGI-UK-IV, UC-degree-create, UC-NGI-UK-XI, UC-NGI-UK-XII, UC-NGI-UK-XIII, UC-NGI-UK-XIV
[edit] Applications
Majority of application-related use cases have specific requirements. It seems better to match those directly to identified WP3 functionalities. However, several following issues had been identified as critical.
[edit] Commercial software licences
- dealing with commercial software packages UC-NGI-CZ-II, UC-NGI-PL-application-III, UC-CSC-licenses
[edit] Parallel job run
- support for parallel jobs UC-NGI-CZ-I
- business model UC-NGI-PL-application-IV
[edit] Interactive jobs
- running interactive jobs UC-IEG-I, UC-IEG-II, UC-IEG-III, UC-IEG-IV, UC-IEG-V, UC-IEG-VI, UC-IEG-VII, UC-IEG-VIII, UC-IEG-IX
[edit] User support for individual application group
- selection of relevant services for specific application UC-EUChinaGrid-new-community
- is my application already on the Grid UC-cern-user
UC-SEEGRID-I, UC-SEEGRID-II, UC-SEEGRID-III
[edit] Specific individual applications
UC-bioinfogrid, UC-IPC-IV, UC-NGI-GR-III, UC-NGI-GR-IV, UC-NGI-GR-VI, UC-NGI-GR-VII, UC-NGI-GR-VIII, UC-NGI-GR-IX, UC-NGI-GR-X, UC-NGI-NW-II , UC-NGI-NW-III, UC-NGI-PL-application-II, UC-NGI-SK-astro, UC-NGI-SK-cell-I, UC-NGI-SK-kwf-infrastructure-II, UC-NGI-SK-kwf-infrastructure-III, UC-NGI-SK-kwf-user, UC-NGI-SK-kwf-user-II, UC-NGI-SW-persson, UC-NGI-UA-application, UC-NGI-UK-disaster, UC-VO-ILDG-search, UC-degree-degree, UC-RINGRID
