Active Nodes

• 192.168.11.1 - ipepdvcompute1.ka.fzk.de (master, storage)
• 192.168.11.2 - ipecamera.ka.fzk.de
• 192.168.11.3 - ipekatrinadei.ka.fzk.de (storage)
• 192.168.11.4 - ipeusctcompute1.ka.fzk.de
• 192.168.11.5 - ipepdvcompute2.ka.fzk.de
• 192.168.11.6 - ipepdvkepler.ka.fzk.de

Installation

• OpenSuSE 12.x
• OpenSuSE 11.4

Diagnostic

• Port information: ibstat
• Hardware graph: iblinkinfo
• Network Diagnostic: ibdiagnet -ls 10 -lw 4x
• Ping: ibping -S ; ibping <Baselid_of_server_reported_by_ibstat>

Storage

• Fast storage for camera streaming
  • First 8 TB on storage boxes attached to ipekatrinadei and ipepdvcompute1
  • Exposed over iSER protocol using tgt server
    • No access sharing, single user only
  • Both devices allocated to the camera computer using openiscsi
  • Software Raid 0 is assembled from the devices
  • Formatted using XFS and mounted under /mnt/server
• Big and slow storage
  • glusterfs is used currently, but fhgfs is faster and may be better option if they go open source (there are some plans)
    • OpenSuSE packages does not include RDMA support, the source RPM should be recompiled on the system with installed Infiniband stack
  • Storage mounted under /pdv
    • /pdv/home/ clustered home folders
    • /pdv/data/ sample data sets
  • External computers may mount the storage over NFS

    ipepdvcompute1:/storage> /pdv nfs defaults,_netdev,mountproto=tcp 0 0
    

• Configuration