Reducing Software Overheads in Parallel Linear Algebra Libraries
P.E. Strazdins,
Reducing Software Overheads in Parallel Linear Algebra Libraries ,
Proceedings of The 4th Annual Australasian Conference on
Parallel And Real-Time Systems (PART'97), Springer,
Newcastle Australia, September 1997, pp 73--84.
Contents
Abstract
Software overheads can be a significant cause of performance degradation
in parallel numerical libraries. This paper examines the nature and
extent of software overheads in an implementation of parallel LAPACK on
distributed memory multiprocessors, where block-partitioned algorithms
with a general block-cyclic matrix distribution scheme present special
challenges. It then describes various techniques that have been used to
reduce these overheads, and evaluates their effectiveness. While there
is a tradeoff between the software engineering properties of high data
and procedural abstraction, modularity and portability (which are
particularly important in parallel programming) and achieving low
software overheads, it is shown that a good balance can be achieved in
the case of parallel LAPACK, at least for important classes of
computations.
Keywords
dense linear algebra, block cyclic decomposition,
sowtware overheads, procedural abstraction, data abstraction.