State scientific-technical Program - UNG
Ukrainian National Grid is the targeted state scientific and technological project (program) on the development and implementation of grid-technologies for 2009-2013, adopted by the Cabinet of Ministers of Ukraine in 2009. “Goal of the program is creation of national Grid infrastructure and wide implementation of Grid technologies into all the spheres of social-economical life in Ukraine”. The project goal is to build a national grid-infrastructure and to introduce grid-technologies in all areas of scientific, social and economic activities in Ukraine, as well as to train specialists on grid-technologies. The objectives of the project are:
• Introduction and application of grid-technologies in scientific research
• Creation of conditions for implementation of grid-technologies in economy, industry, financial and social spheres.
• Creation of multilevel interdepartmental grid-infrastructure with elements of centralized control that takes into account the peculiarities of grid-technologies usage in various fields.
• Creation of specialist training system to work with grid-technologies.
The following management bodies are created for project control:
• Interdepartmental Coordinating Council, which defines the general principles of development, grid infrastructure program and operational plans.
• Project Coordination Committee, which is the executive body and has the authority to represent the national grid-infrastructure at the national and international levels.
• Basic Coordination Centre, which is responsible for operation of the national grid-infrastructure.
The main project activities are:
• Creation of new and update of existent clusters. Raising the bandwidth of internet channels.
• Middleware and technical support.
• Security of Grid Environment.
• Implementation of Grid technologies and Grid applications in scientific research, in economics, industry, financial activity.
• Implementation of Grid technologies and Grid applications in medicine.
• Development for purpose of storing, processing and open access to scientific and educational information resources (data bases, archives, electronic libraries) by using Grid applications.
• Organizational and methodical providing of specialists training for them to work with Grid technologies.
The additional information about state program implementation can be found on the site - http://grid.nas.gov.ua.
It is known that grid system is based on three principal elements: computer resources (clusters), high-speed and reliable access of resources to Internet and middleware which unifies these resources into one computer system. Though the middleware was available even at the beginning of the grid infrastructure creation, the clusters and fiber-optic network of NASU were simultaneously built together with the grid facilities system development. The well-known Beowulf idea (www.beowulf.org) as a conceptual model for the computer cluster construction was selected and adopted for realization in UNG. This conception is based on using servers with standard PC architecture, distributed main storage and Gigabit Ethernet technology which unifies computer system. So, all grid clusters in UNG are built with x86, x86_64 architecture, two- (four-) processor servers of 1-4 GB main storage and 36-500 GB HDD. 1GB/s switchers have been used to provide inter-server exchange and the InfiniBand is used only in some clusters. At present time the UNG shares resources of 30 institutes and universities (more than 2700 CPU and 200 TB of disk memory).
It should be emphasized that the footprint on hard disks of computer nodes is used for operating system (loading from local disks), program packages and temporary files, but it is inaccessible for user files storage. Each cluster has its disk array to store programs, user’s data and information of common use. A free distributed operational system Linux of various modifications (Scientific Linux 2.6.9, Fedora 2.6, CentOS-4.6) is installed on clusters and the task management system OpenPBS is used to start tasks and allocate the cluster utilization. To enlarge number of grid users the idea of so-called grid platforms has been fulfilled, its core idea is to install the grid clusters with “minimal configuration”. Such a cluster includes the control server with installed middleware, two working nodes and network equipment. Working permanently in the conditions of limited funding, such grid platforms have allowed access to the grid for the specialists of Institutes without operable cluster. With financing available any mini cluster can be easily extended to the full scale cluster. This strategy enables to train system administrators and users for the work with the grid. The high-speed and reliable access channel to Internet networking is one of the necessary conditions at the grid infrastructure building. Fiber optic channels in UNG are owned by two providers: UARNet (which works mainly with academic institutes) and URAN (which works mainly with educational structures). At present, UARNet is one of the biggest Internet providers in Ukraine with its own data transmission network and external channels to the global Internet. The total capacity of UARNet external channels amounts to over 100 Gb/s.
The access to non-Ukrainian Internet resources is provided via Tier1-providers “Level3”, “Cogent” and “Global Crossing”, Russian provider “ReTN.net”. UARNet is a member of European Internet Traffic Exchanges DE-CIX, AMS-IX and PL-IX. UARNet has its sites (POPs) in all regional centers of Ukraine as well as in Frankfurt (Germany) and Warsaw (Poland). In 2009 cable from Lvov to Poland border has been built. Network equipment has been installed for connection to Pioneer network. In 2009 Ukraine has been connected to a GEANT. All sites are interconnected by ring topological structured by multiple 10 Gb/s data channels. UARNet is also connected to the Ukrainian Internet Traffic Exchange Network UA-IX through four 10 Gb/s data channels. To KIPT cluster (Kharkov) that takes part in CMS experiment and has status of Tier-2 CERN with guaranteed capacity 300 MBit/s. To BITP cluster (Kiev), that takes part in ALICE experiment at CERN, with guaranteed capacity 1 GBit/s.
The main efforts this year were aimed at the harmonization of Ukraine’s grid infrastructure to the requirements of EGI. According to the WLCG scheme the infrastructure of UNG has been built as three-level system in respect to an organization and management:
• First level: Basic Coordinating Centre (BCC) which is responsible for core services and controls UNG.
• Second level: Regional Operating Centers which coordinates the activity of grid sites in regions.
• Third level: Grid sites (institutes) or minimal grid network access platforms which belong to any virtual organization (VO). VO temporarily joins institutes (not necessarily from the same region) of common scientific interests to solve a problem.
Basic Coordinating Centre is a non-structural subdivision of the Bogolyubov Institute for Theoretical Physics of the NAS of Ukraine, which is the basic organization for the implementation of the project. BCC provides management and coordination of works on supporting the operation of the national grid-infrastructure and resource centres (grid-sites) to provide grid-services (high-performance distributed computing, access to distributed database, access to software) for users. BCC on technological and operational level coordinates the work of grid-sites and entire UNG grid-infrastructure according to the requirements of European and international grid-infrastructure. BCC on technological and operational level serves as the National Operations Centre (Resource infrastructure Provider) of UNG grid-resources in relation with the International grid-communities. BCC has the right to conclude international agreements in the field of cooperation at the operational level, which shall become valid after ratification by the Program Coordinating Committee.
The Basic Coordination Centre includes:
• Centre for monitoring of grid-infrastructure and registration of grid-sites.
• Centre for registration of virtual organizations and members of virtual organizations.
• Certification authority.
• International relations group.
• Scientific and analysis group.
• Technical assistance and middleware maintenance group.
• User support group.
• Study and training centre.
The principal BCC teams and their functions are following:
International relations group. The key task of this group is creating the conditions for UNG participants for cooperation with the international organizations and the international grid collaboration.
Scientific and analysis group. The task of this group is to analyze the applicability of grid technologies in the different fields, analysis of perspective research trends of the grid technologies and their realization, expert evaluation of new propositions and projects for development and the grid technology applications.
Technical assistance and middleware maintenance group. The group assists administrators of computer clusters in installation of system-wide software and providing assistance in computer security topics. The team coordinator maintains the permanent contact with the system administrators and security administrators of each local grid site. Software support team assists administrators of grid site on installation of middleware and maintenance. Moreover, the team experts are responsible for task analysis of the whole grid system, new middleware installation, and compatibility of software with middleware.
User support group. The main task of group is supporting of the GGUS system - monitoring the questions of grid users. Usually this is a web-portal which should answer to the practical questions: how to involve in the grid activity, how to obtain the grid certificate, how to join to the virtual organization. Web-site contains the information about the structure of national grid, references to the grid activity documentations and reference to international grid projects and virtual organizations, announcements on the grid seminars and conferences.
Study and training centre is in charge of organization of the training process on theoretical and practical implementation of the grid technology. The individual education programs for system administrators and grid users are to be created.
The Coordinating Committee of the Ukrainian State Program has accepted and approved the basic documents defining the operation the grid – sites in UNG. They are:
- Ukrainian National Grid. Operation architecture.
- Agreement for the use of grid resources in the UNG;
- Procedure of registration of grid sites in the UNG;
- Procedure of registration of virtual organizations in the UNG;
- Basic Coordination Center. Operation structure.
All this documents were developed according to EGI requirements. Negotiation on «Memorandum of Understanding» between EGI.eu and BCC are on. «Resource Infrastructure Provider MoU» document is coming. The purpose of this Memorandum of Understanding is to define a framework of collaboration between EGI.eu and BCC for access of Ukraine to the operation level in EGI. Additional information about this activity can be found on the sites: