Paper _ Models of Parallel Computation_ A survey and Synthesis

Document Sample
Paper _ Models of Parallel Computation_ A survey and Synthesis Powered By Docstoc
					A critique on the research paper: LogP: Towards a Realistic Model of Parallel Computation
[Authors : David Culler, Richard Karp y , David Patterson, Abhijit Sahay, Klaus Erik Schauser, Eunice Santos, Ramesh Subramonian, and Thorsten von Eicken]

by Rajvikram Singh


Paper : LogP: Towards a Realistic Model of Parallel Computation
[Authors : David Culler, Richard Karp y , David Patterson, Abhijit Sahay, Klaus Erik Schauser, Eunice Santos, Ramesh Subramonian, and Thorsten von Eicken] Main Contribution(s): The authors of the paper have stressed more than once that the present popular models used for depicting parallel architectures do not accurately describe the machines. The present models, which are derived from the ideal, PRAM models are described by the authors as too “simple” One of the main drives for presenting a new model was given as the trend that the computing industry has been taking since the past few years. Statistical and empirical data has been given in the paper to show that the latest trend has been towards more “multi computer” type of machines. These are generally massively parallel machines constituted of many nodes, each containing a powerful processor and substantial memory and cache, so as to be regarded as independent computer in itself. In such „distributed architecture‟ machines, the following features can be considered as the salient ones:  Performance of the interconnect network – The bandwidth, latency, jitter, congestion aspects of the interconnect networks typically become important in deciding the overall performance of the machine.  Unsynchronized nodes – Since each of the nodes is more or less an independent computer in itself, the various nodes in a MPP machine can no longer considered to be synchronous.  Shared memory absent – Since each node now carries its own memory modules, the architecture is more suited for a message-passing system than a shared memory one. All the three points mentioned above are starkly opposite to the features of the popular PRAM model, which describes a synchronized, unbounded shared memory, MIMD model. Thus acknowledging the latest trend in the HPC industry, the authors have suggested the LogP model, which takes into consideration, the parameters, namely: latency, communication overheads, communication bandwidth and the number of processors to describe a more accurate model. The model also takes care of some other important aspects such as affect of load on network performance and affect of routing for describing a more pragmatic model.

Evaluation of the contributions to advancing science and the state-of-the-art of the topic: The LogP model suggested by the authors does seem to alleviate quite a few of loopholes of the overly simplified PRAM model and in the present scenario where indeed focus on MPP machines has shifted towards a more distributed architecture, the LogP model definitely provides a much more accurate depiction of practical MPP machines. Thus though the model was suggested in‟93, it can still be considered as state-of-the-art


and of relevance even now and with HPC systems like clusters becoming popular day by day, the model might well be of use even in the future. Deficiencies: Though the authors have presented a model, which they feel is accurate for depicting present day MPP architectures, they have used parameters, which concentrate primarily on the performance of the interconnection network. They seem to neglect the other factors such as network topology and message-passing v/s shared memory topology, which also affect performance. The paper also seems to sort of re-instate the obvious concerns for designing a parallel computing machine. And though the authors have published results of their experiments showing the predicted performance and the actual performance for some algorithms but they have not compared the results to those predicted by the existing models.

Directions for future improvements: The model seems to give too much importance to the underlying interconnect network. Maybe some of the parameters in the model should be dropped and replaced with others pertaining to the nodes. Especially with the recent development of inexpensive and high speed optical interconnects, which provide excellent latency, near zero contention environments (WDM) and bandwidths higher than the processors, it might be a good idea to drop some of the parameters like L and o or g in lieu of future trends. Also the authors agree that in most of the cases all the parameters are not required. Questions –Regarding the paper:  Can one of the parameters out of „o‟ or „g‟ be dropped? Or can one be expressed as a function of the other? They seem to depict the same thing.  The authors claim that the model takes care of network saturation due to loading of the network. How do L, o , g or P reflect the loading of the network, when there is no parameter to define the network topology ?  Why haven‟t the results of performance predicted by other known models compared to the ones predicted by the LogP model? (eg. For the FFT algorithm)



Paper : Algorithms in CGM, BSP and BSP Model – A Survey
[ Author: Silvia Gotz]

Summary: The paper has been written with the intention of evaluating the pluses and minuses of some of the most popular parallel models that have received considerable attention in the past few years of its publication. The paper talks about the following models : The BSP (Bulk synchronous Parallel Model)  The BSP* - Which is an extension of the basic BSP model but takes into account an additional parameter B (minimum size of transmitted message that fully utilizes n/w bandwidth) - which defines the interconnect more accurately and leads to better bandwidth utilization.  The CGM (Coarse Grained Multicomputer ) model After surveying these popular models and evaluating them for different algorithms, the author concludes that the BSP model is the one that comes close to being the one model that can be standardized. But at the same time there exist whole classes of problems which can not use the BSP model.

Note : Since the paper was just a survey of existing models, it by itself does not contribute anything, nor does it contain any deficiencies which can be attributed to the survey.


Paper: BSP - The Bulk Synchronous Parallel model
[University of Oxford Parallel Applications Center] Main contribution(s): This paper is not a research paper and provides a brief but insightful view into the BSP model developed primarily at the University of Oxford. The paper defines the characteristics of a BSP computer, the parameters that define a BSP model and the computation and programming model for a typical BSP computer.


Paper : Models of Parallel Computation: A survey and Synthesis
[ Authors: B.M Maggs, L.R Matheson and R.E Tarjan ]

Main Contribution(s): The paper tries to form present the past trends in the field of parallel computing and tries to explain why there has been no standard model which can define all the existing and possible future parallel machines. The paper evaluates the research undertaken by the computer scientists in coming up with a unified model and the levels at which models have been introduced (eg. Memory, network etc) The authors stress on the point that the models should not only be „descriptive‟ i.e they should not only reflect the existing parallel architectures, but they should also be prescriptive – i.e they should be able to provide guidelines for the machines of the future. And while the authors agree that the models should not contain a lot of parameters, they have described a few parameters, which they feel symbolize a parallel architecture. They recommend that a subset of these parameters be used for modeling.


Paper : Models for Parallel Computation
[ Author: Susanne E. Hambrusch]

Summary: The paper gives a brief discussion of the most popular models, which have received the most attention over the past years and describes the major advantages / disadvantages of one over the other. The paper acknowledges that in order for a successful standard model to evolve, the model should:  Accurately reflect the constraints of the existing classes of machines  Should also broadly apply to existing as well as future machines  Allow accurate measure and prediction of performance. The paper was not intended as a research paper and thus does not make any major contributions or has any deficiencies.


Shared By: