Tags

    Cluster Mangue - User Guide

    1. System Overview


    The Mangue system is AMD Linux Cluster based on 5 SuperMicro server nodes, each outfitted with 4-16 AMD Opteron(TM) 6276 (Interlagos) processors and two NVIDIA Tesla C2075 GPGPUs (Fermi Architecture) and 9 Intel Xeon server nodes outfitted with 2 Intel Xeon E5606 processor.

    The peak performance of each AMD node is 294 GP [source], while the NVIDIA accelerators deliver an additional peak performance of 1.03 TF (single precision) and 515 GF (double precision)[source]. The system also includes a login node that provides management and file system services.

    One of the important design considerations for Mangue was to create a accelerated cyberinfrastructure resource, offering large data transfer and GPU capabilities for data-intensive, accelerated computing and molecular dynamics. By augmenting the compute-intensive nodes within the system with GPUs there is no need to move data for data-intensive computing and GPGPU computing.

    Compute Nodes: All nodes are configured with 4-16 AMD processors and two NVIDIA Tesla C2075 GPGPUs (on a PCIe card). These compute nodes are configured with 12GB of "host" memory with an additional 5GB of memory on the NVIDIA cards. Each GPGPU has the Error-correcting Code (ECC) enabled.

    File Systems: The Mangue system supports a 8TB global file systems. Each node contains a local 2TB disk. Also, a 19TB archival system is accessible from the login node, but not from the execution nodes.

    Interconnect:Nodes are interconnected with QLogic FDR InfiniBand technology in direct connection topology.


    2. System Configuration


    All Mangue nodes run CentOS 6.4 and are managed with batch services through SGE 6u5. Global $HOME and $SCRATCH storage areas are supported by an single file system. Inter-node communication (MPI) is through an FDR QLogic InfiniBand network (The network configuration for the compute nodes is shown in Figure 1.4.).

    The five compute nodes are housed in one rack, along with one 8-port Mellanox leaf switch and a 3COM Gigabit Switch. Each node has 4-16 AMD processors and two NVIDIA C2075 GPGPU card, connected by two x16 PCIe bus. The host and accelerator are configured with DDR3 12GB and DDR5 5GB memory, respectively.

    The configuration and features for the compute nodes, interconnect and I/O systems are described below, and summarized in Tables 1.1 through 1.4.


    Intel(R) Xeon(R) CPU E5606 @ 2.13GHz

    cache size : 8192 KB

    mague 126.0G

    octa 47.1G


    Table 1.1 System Configuration and Performance

    Component Technology Performance/Size
    Node 4-16 core AMD 6276 Opteron(TM) Processors, two NVIDIA C2075 GPGPU cards 5 Nodes
    Memory Distributed, 12GB/node 60 GB (Aggregate)
    Shared Disk xfs filesystem 8 TB
    Local Disk ext4 filesystem 2 TB
    Interconnect InfiniBand Mellanox Switch/HCA 40 GB/s


    2.1 Compute node


    A Compute node consists of a single sled in a 2 rack-unit chassis with 4 other sleds. Each node runs CentOS 6.4 with the 2.6.32 x86_64 Linux kernel and contains 4 16-Core 64-bit AMD Opteron(TM) 6276 (64 cores in all) on a single board, as an SMP unit. The core frequency is 2.3 GHz and supports 8 floating-point operations per clock period with a peak performance of 73.5 GFLOPS/core or 294 GFLOPS/node. Each node contains 16GB of memory (2GB/core). The memory subsystem has 4 channels from each processor's memory controller to 4 DDR3 ECC DIMMS, each rated at 1600 MT/s (51.2GB/s for all four channels in a socket). The processor interconnect runs at 6.4 GT/s between sockets. The two Intel NVIDIA C2075 Graphics Card model have 448 cores with a peak performance of 515 GFLOPS/card in double precision or 1.3 TFLOPS/card in single precision. Each GPGPU contains 5GB of GDDR5 memory.

    Table 1.2 Compute Node Configuration and Performance

    Component Technology
    Sockets per node/Cores per socket
    NVIDIA GPGPU cards/node
    4/16 AMD Opteron 6276 (2.3 GHz
    2 Tesla C2075
    Motherboard
    Memory per host
    Memory per GPGPU
    12 GB 4 channel DDR3 1600 MHz
    5 GB
    Interconnect
    Processor - Processor
    Processor - GPGPU

    6.4 GT/s
    PCI-e 16x
    2 TB Disk 7.5K RPM SATA


    2.2 Login Node


    Component Technology
    1 login node
    Sockets per node/Cores per socket
    Motherboard
    Memory
    $HOME
    $ARCHIVE
    $SCRATCH

    Comments

    /groups/geekstuff/search/index.rss?tag=hotlist/groups/geekstuff/search/?tag=hotWhat’s HotHotListHot!?tag=hot6/groups/geekstuff/sidebar/HotListbiomatBiomat2016-11-01 16:19:20+00:002016-11-01 16:19:20updated22added the Terceira opção sectionbiomatBiomat2016-11-01 16:19:07+00:002016-11-01 16:19:07updated21Added tag - xorgbiomatBiomat2016-11-01 16:18:54+00:002016-11-01 16:18:54addTag20Removed tag - sibiomatBiomat2016-11-01 16:18:46+00:002016-11-01 16:18:46removeTag19Added tag - sierrabiomatBiomat2016-11-01 16:18:41+00:002016-11-01 16:18:41addTag18Added tag - portbiomatBiomat2016-11-01 16:18:34+00:002016-11-01 16:18:34addTag17Added tag - macportsbiomatBiomat2016-11-01 16:18:31+00:002016-11-01 16:18:31addTag16biomatBiomat2016-10-06 21:05:20+00:002016-10-06 21:05:20updated15biomatBiomat2016-10-06 20:44:07+00:002016-10-06 20:44:07updated14added sierra updatebiomatBiomat2016-10-06 20:43:39+00:002016-10-06 20:43:39updated13Added tag - sibiomatBiomat2016-10-06 20:30:38+00:002016-10-06 20:30:38addTag12added xcode partbiomatBiomat2015-10-27 17:12:34+00:002015-10-27 17:12:34updated11Added tag - x11biomatBiomat2015-10-27 17:11:03+00:002015-10-27 17:11:03addTag10Added tag - xquartzbiomatBiomat2015-10-27 17:11:01+00:002015-10-27 17:11:01addTag9added xquartz partbiomatBiomat2015-10-27 17:10:50+00:002015-10-27 17:10:50updated8biomatBiomat2015-10-24 17:48:37+00:002015-10-24 17:48:37updated7Added tag - securitybiomatBiomat2015-10-24 16:35:37+00:002015-10-24 16:35:37addTag6Added tag - openmotifbiomatBiomat2015-10-24 16:35:34+00:002015-10-24 16:35:34addTag5Added tag - el capitanbiomatBiomat2015-10-24 16:35:30+00:002015-10-24 16:35:30addTag4Added tag - xmgracebiomatBiomat2015-10-24 16:35:21+00:002015-10-24 16:35:21addTag3Added tag - hotbiomatBiomat2015-10-24 16:35:13+00:002015-10-24 16:35:13addTag2First createdbiomatBiomat2015-10-24 16:34:04+00:002015-10-24 16:34:04created1wiki2016-11-01T16:19:20+00:00groups/geekstuff/wiki/be4a5FalseXmgrace no Mac OS X El Capitan e Sierra/groups/geekstuff/wiki/be4a5/Xmgrace_no_Mac_OS_X_El_Capitan_e_Sierra.htmlBiomat22 updatesXmgrace no Mac OS X El Capitan e Sierra Prefácio Olá. Este tutorial tem o objetivo de auxiliar e guiar na instalação do software Xmgrace no sistema operacional Mac OS X El Capit...Falsebiomat2016-11-01T16:19:20+00:00biomatBiomat2016-08-24 18:35:48+00:002016-08-24 18:35:48updated10Added tag - printerbiomatBiomat2016-08-24 18:11:40+00:002016-08-24 18:11:40addTag9biomatBiomat2016-08-24 18:11:40+00:002016-08-24 18:11:40updated8Added tag - twain sanebiomatBiomat2016-08-24 18:11:34+00:002016-08-24 18:11:34addTag7Added tag - osx mavericksbiomatBiomat2016-08-24 18:11:28+00:002016-08-24 18:11:28addTag6Added tag - hp laserjetbiomatBiomat2016-08-24 18:11:20+00:002016-08-24 18:11:20addTag5Added tag - scannerbiomatBiomat2016-08-24 18:11:12+00:002016-08-24 18:11:12addTag4Added tag - hotbiomatBiomat2016-08-24 18:11:04+00:002016-08-24 18:11:04addTag3biomatBiomat2016-08-19 21:43:08+00:002016-08-19 21:43:08updated2First createdbiomatBiomat2016-08-19 21:13:54+00:002016-08-19 21:13:54created1wiki2016-08-24T18:35:48+00:00groups/geekstuff/wiki/08c03FalseTutorial: Scanner da LaserJet M1120 MFP no iOS/groups/geekstuff/wiki/08c03/Tutorial_Scanner_da_LaserJet_M1120_MFP_no_iOS.htmlBiomat10 updatesTutorial: Scanner da LaserJet M1120 MFP no iOS Scanner da LaserJet M1120 MFP no iOS Prefácio Olá. Este tutorial tem o objeti...Falsebiomat2016-08-24T18:35:48+00:00biomatBiomat2015-10-24 16:35:01+00:002015-10-24 16:35:01updated11biomatBiomat2015-10-23 22:29:08+00:002015-10-23 22:29:08updated10added Mac OS X partbiomatBiomat2015-10-23 22:28:31+00:002015-10-23 22:28:31updated9biomatBiomat2014-09-23 22:12:48+00:002014-09-23 22:12:48updated8Added tag - installbiomatBiomat2014-09-23 22:12:43+00:002014-09-23 22:12:43addTag7Added tag - csgbiomatBiomat2014-09-23 22:12:38+00:002014-09-23 22:12:38addTag6Added tag - cgbiomatBiomat2014-09-23 22:12:33+00:002014-09-23 22:12:33addTag5Added tag - coarse-grainbiomatBiomat2014-09-23 22:12:31+00:002014-09-23 22:12:31addTag4Added tag - votcabiomatBiomat2014-09-23 22:12:25+00:002014-09-23 22:12:25addTag3Added tag - hotbiomatBiomat2014-09-23 22:12:21+00:002014-09-23 22:12:21addTag2First createdbiomatBiomat2014-09-23 17:57:47+00:002014-09-23 17:57:47created1wiki2015-10-24T16:35:01+00:00groups/geekstuff/wiki/c0fa7FalseInstalação do VOTCA-CSG/groups/geekstuff/wiki/c0fa7/Instalaco_do_VOTCACSG.htmlBiomat11 updatesInstalação do VOTCA-CSG Prefácio Olá. Este tutorial tem o objetivo de auxiliar e guiar na instalação do software package VOTCA (http://www.votca.org/). O VO...Falsebiomat2015-10-24T16:35:01+00:00biomatBiomat2014-04-19 13:48:51+00:002014-04-19 13:48:51updated51biomatBiomat2014-04-19 13:40:24+00:002014-04-19 13:40:24updated50biomatBiomat2014-03-07 18:21:37+00:002014-03-07 18:21:37updated49biomatBiomat2014-02-27 12:30:24+00:002014-02-27 12:30:24updated48biomatBiomat2014-02-27 12:27:38+00:002014-02-27 12:27:38updated47biomatBiomat2014-02-26 22:08:24+00:002014-02-26 22:08:24updated46biomatBiomat2014-02-25 14:35:18+00:002014-02-25 14:35:18updated45biomatBiomat2014-02-25 14:33:33+00:002014-02-25 14:33:33updated44biomatBiomat2014-02-25 14:32:16+00:002014-02-25 14:32:16updated43biomatBiomat2014-02-25 14:30:06+00:002014-02-25 14:30:06updated42biomatBiomat2014-02-25 14:03:37+00:002014-02-25 14:03:37updated41biomatBiomat2014-02-25 13:59:32+00:002014-02-25 13:59:32updated40biomatBiomat2014-02-25 13:57:47+00:002014-02-25 13:57:47updated39biomatBiomat2014-02-25 13:56:52+00:002014-02-25 13:56:52updated38biomatBiomat2014-02-24 19:35:19+00:002014-02-24 19:35:19updated37biomatBiomat2014-02-03 15:15:39+00:002014-02-03 15:15:39updated36biomatBiomat2014-02-03 15:15:09+00:002014-02-03 15:15:09updated35biomatBiomat2013-11-06 23:36:18+00:002013-11-06 23:36:18updated34biomatBiomat2013-11-06 23:36:13+00:002013-11-06 23:36:13updated33biomatBiomat2011-09-03 09:42:19+00:002011-09-03 09:42:19updated32biomatBiomat2011-08-23 21:52:46+00:002011-08-23 21:52:46updated31biomatBiomat2011-07-26 22:37:00+00:002011-07-26 22:37:00updated30biomatBiomat2011-07-09 20:45:07+00:002011-07-09 20:45:07updated29tasoaresThereza Soares2011-05-21 01:52:32+00:002011-05-21 01:52:32updated28biomatBiomat2011-05-20 09:52:18+00:002011-05-20 09:52:18updated27biomatBiomat2011-05-20 09:03:48+00:002011-05-20 09:03:48updated26biomatBiomat2011-05-19 01:09:24+00:002011-05-19 01:09:24updated25biomatBiomat2011-05-19 01:09:09+00:002011-05-19 01:09:09updated24biomatBiomat2011-05-07 05:12:52+00:002011-05-07 05:12:52updated23biomatBiomat2011-05-06 18:42:54+00:002011-05-06 18:42:54updated22biomatBiomat2011-05-06 10:13:59+00:002011-05-06 10:13:59updated21biomatBiomat2011-05-06 10:12:46+00:002011-05-06 10:12:46updated20biomatBiomat2011-05-06 01:39:45+00:002011-05-06 01:39:45updated19biomatBiomat2011-03-09 23:58:20+00:002011-03-09 23:58:20updated18Added tag - hotbiomatBiomat2011-03-09 23:58:15+00:002011-03-09 23:58:15addTag17biomatBiomat2011-03-04 11:43:58+00:002011-03-04 11:43:58updated16biomatBiomat2011-03-04 11:33:45+00:002011-03-04 11:33:45updated15biomatBiomat2011-03-04 11:33:14+00:002011-03-04 11:33:14updated14biomatBiomat2011-03-04 11:23:04+00:002011-03-04 11:23:04updated13biomatBiomat2011-03-04 11:22:25+00:002011-03-04 11:22:25updated12biomatBiomat2011-03-04 11:21:56+00:002011-03-04 11:21:56updated11biomatBiomat2011-03-04 11:19:50+00:002011-03-04 11:19:50updated10biomatBiomat2011-03-04 11:19:04+00:002011-03-04 11:19:04updated9biomatBiomat2011-03-04 11:14:52+00:002011-03-04 11:14:52updated8biomatBiomat2011-03-04 10:43:23+00:002011-03-04 10:43:23updated7biomatBiomat2011-03-04 10:42:05+00:002011-03-04 10:42:05updated6biomatBiomat2011-03-04 10:40:44+00:002011-03-04 10:40:44updated5biomatBiomat2011-03-04 10:38:59+00:002011-03-04 10:38:59updated4biomatBiomat2011-03-04 10:38:06+00:002011-03-04 10:38:06updated3biomatBiomat2011-03-04 10:37:18+00:002011-03-04 10:37:18updated2First createdbiomatBiomat2011-03-04 10:35:29+00:002011-03-04 10:35:29created1wiki2014-04-19T13:48:51+00:00groups/geekstuff/wiki/c24e6FalseCluster barolo - Guia do usuário/groups/geekstuff/wiki/c24e6/Cluster_barolo__Guia_do_usuario.htmlBiomat51 updatesCluster barolo - Guia do usuário Introdução Antes de você começar a rodar cálculos na barolo, por favor, siga todos os passos abaixo. Gostaria de adicionar que a barolo está c...Falsebiomat2014-04-19T13:48:51+00:00biomatBiomat2013-03-14 22:07:29+00:002013-03-14 22:07:29updated29biomatBiomat2013-03-14 22:06:03+00:002013-03-14 22:06:03updated28biomatBiomat2013-03-14 20:51:48+00:002013-03-14 20:51:48updated27biomatBiomat2013-03-14 20:51:28+00:002013-03-14 20:51:28updated26biomatBiomat2013-03-14 20:47:35+00:002013-03-14 20:47:35updated25biomatBiomat2013-03-14 20:46:59+00:002013-03-14 20:46:59updated24biomatBiomat2013-02-11 01:01:04+00:002013-02-11 01:01:04updated23biomatBiomat2013-02-11 00:58:48+00:002013-02-11 00:58:48updated22biomatBiomat2013-02-11 00:56:47+00:002013-02-11 00:56:47updated21biomatBiomat2013-02-11 00:56:04+00:002013-02-11 00:56:04updated20biomatBiomat2013-02-08 04:57:24+00:002013-02-08 04:57:24updated19biomatBiomat2013-02-08 04:57:14+00:002013-02-08 04:57:14updated18biomatBiomat2013-02-08 04:54:53+00:002013-02-08 04:54:53updated17biomatBiomat2013-02-08 04:50:29+00:002013-02-08 04:50:29updated16biomatBiomat2013-02-08 04:39:37+00:002013-02-08 04:39:37updated15biomatBiomat2013-02-08 03:13:49+00:002013-02-08 03:13:49updated14biomatBiomat2013-02-08 03:11:55+00:002013-02-08 03:11:55updated13biomatBiomat2013-02-08 02:56:49+00:002013-02-08 02:56:49updated12biomatBiomat2013-02-08 02:29:20+00:002013-02-08 02:29:20updated11biomatBiomat2013-02-08 02:26:06+00:002013-02-08 02:26:06updated10Added tag - hotbiomatBiomat2013-02-08 02:19:24+00:002013-02-08 02:19:24addTag9biomatBiomat2013-02-08 00:00:46+00:002013-02-08 00:00:46updated8biomatBiomat2013-02-07 22:17:35+00:002013-02-07 22:17:35updated7biomatBiomat2013-02-07 22:12:20+00:002013-02-07 22:12:20updated6biomatBiomat2013-02-07 22:09:38+00:002013-02-07 22:09:38updated5biomatBiomat2013-01-31 22:07:20+00:002013-01-31 22:07:20updated4biomatBiomat2013-01-31 22:06:53+00:002013-01-31 22:06:53updated3biomatBiomat2013-01-31 22:06:06+00:002013-01-31 22:06:06updated2First createdbiomatBiomat2013-01-31 22:01:42+00:002013-01-31 22:01:42created1wiki2013-03-14T22:07:29+00:00groups/geekstuff/wiki/2ebb2FalseGROMACS 4.6 Installation Guide/groups/geekstuff/wiki/2ebb2/GROMACS_46_Installation_Guide.htmlBiomat29 updatesGROMACS 4.6 Installation Guide 1. Introduction GROMACS 4.6 has another procedure to be installed compared to the previous 4.5.X versions. Now it does not rely on the ...Falsebiomat2013-03-14T22:07:29+00:00hot/groups/geekstuff/search/index.rss?sort=modifiedDate&kind=all&sortDirection=reverse&excludePages=wiki/welcomelist/groups/geekstuff/search/?sort=modifiedDate&kind=all&sortDirection=reverse&excludePages=wiki/welcomeRecent ChangesRecentChangesListUpdates?sort=modifiedDate&kind=all&sortDirection=reverse&excludePages=wiki/welcome0/groups/geekstuff/sidebar/RecentChangesListmodifiedDateallRecent ChangesRecentChangesListUpdateswiki/welcomeNo recent changes.reverse5search