Poor Mans Cluster (PMC) Johny Balian Outline What is PMC How it works Concept Positive aspects Negative aspects Good and Bad Application ideas Monte Carlo example Q&A What is it? PMC is a library for PHP Provides parallelization methods like MPI Written in PHP Open source, still in beta stage, you can participate on: http://sourceforge.net/projects/poormanscluster/ How it Works 1) Master sends jobs to Slaves Slave 1 Slave 2 Master Slave 3 Slave 4 How it Works 2) Master waits, slaves execute job Slave 1 Slave 2 Master Execute job Slave 3 Slave 4 While(#results != #slaves) sleep(1); How it Works 3) Slaves are done, they can either send result to Master or to another place (eg: database) The code of the slaves and the results are stored in files (we’ll see later how communication is done) Slave 1 Slave 2 Master Slave 3 Slave 4 Concept The concept is the same as MPI But MPI is limited, its not practical to use on the web, you can’t install it on 4$ servers! That’s where PMC comes handy, its main idea is to make it run on almost any web server. Hence the name, Poor Mans Cluster Positive Aspects (to name a few) Very cheap to maintain, servers cost 4$/month, a cluster of 100 slaves will run at 400$/month (relatively cheap) Easy to install, upload the files on the slaves and add the slaves to the masters config file, the master takes care of everything after that. Open source all the way (Linux,Apache,PHP,PMC) Very easy to use for web applications since PHP is a web language. … Negative Aspects Big communication overload (a msg takes from 1 - 3 seconds) Good Application Ideas… PMC is mainly used for high bandwidth web applications, such as crawling the internet, because PMC is installed on web servers which have very high speed internet connections. 100 slaves with 1mbps each = 100 mbps for the application! The applications can range from security to anything you can think of, example of security: you can tell the slaves to ping a website (DoS) with large packets, this way you can test if the website is vulnerable to DoS attacks… (not recommended but doable!) You can make a powerful search engine by crawling the web using the slaves… Bad Application Ideas Applications that heavily rely on the processor are not recommended unless you have dedicated servers (over 100$/month) Examples of bad applications for PMC are: Monte Carlo Genetic Algorithms… Monte Carlo example Master sends the jobs to the slaves Slave 1 Slave 2 Master Slave 3 Slave 4 Monte Carlo example The slaves will fetch random numbers from random.org Slave 1 Slave 2 Master Slave 3 Random.org Slave 4 While(#results != #slaves) sleep(1); Monte Carlo example Slave 1 Slave 2 Master Execute job Slave 3 Slave 4 While(#results != #slaves) true sleep(1); Monte Carlo example Slave 1 Slave 2 Master Slave 3 Slave 4 While(#results != #slaves) false sleep(1); Calculate average and output Source Code Screenshot Questions?