Poor Mans Cluster by liuqingyan

VIEWS: 7 PAGES: 18

									Poor Mans Cluster (PMC)

  Johny Balian
Outline

  What is PMC
  How it works
  Concept
  Positive aspects
  Negative aspects
  Good and Bad Application ideas
  Monte Carlo example
  Q&A
What is it?

    PMC is a library for PHP

    Provides parallelization methods like MPI

    Written in PHP

    Open source, still in beta stage, you can participate on:

 http://sourceforge.net/projects/poormanscluster/
How it Works
 1) Master sends jobs to Slaves




                                  Slave 1

                                  Slave 2
                Master
                                  Slave 3

                                  Slave 4
How it Works
 2) Master waits, slaves execute job



                                       Slave 1

                                       Slave 2
                Master                           Execute job
                                       Slave 3

                                       Slave 4
     While(#results != #slaves)
      sleep(1);
How it Works
 3) Slaves are done, they can either send result to Master
    or to another place (eg: database)

 The code of the slaves and the results are stored in files
 (we’ll see later how communication is done)

                                                 Slave 1

                                                 Slave 2
                 Master
                                                 Slave 3

                                                 Slave 4
Concept

  The concept is the same as MPI
  But MPI is limited, its not practical to use
   on the web, you can’t install it on 4$
   servers! That’s where PMC comes
   handy, its main idea is to make it run on
   almost any web server.
  Hence the name, Poor Mans Cluster
Positive Aspects (to name a few)
   Very cheap to maintain, servers cost 4$/month, a
    cluster of 100 slaves will run at 400$/month (relatively
    cheap)
   Easy to install, upload the files on the slaves and add
    the slaves to the masters config file, the master takes
    care of everything after that.
   Open source all the way (Linux,Apache,PHP,PMC)
   Very easy to use for web applications since PHP is a
    web language.

  …
Negative Aspects

    Big communication overload (a msg
     takes from 1 - 3 seconds)
Good Application Ideas…
  PMC is mainly used for high bandwidth web
   applications, such as crawling the internet, because
   PMC is installed on web servers which have very high
   speed internet connections.
  100 slaves with 1mbps each = 100 mbps for the
   application!
  The applications can range from security to anything
   you can think of, example of security: you can tell the
   slaves to ping a website (DoS) with large packets, this
   way you can test if the website is vulnerable to DoS
   attacks… (not recommended but doable!)
  You can make a powerful search engine by crawling
   the web using the slaves…
Bad Application Ideas

  Applications that heavily rely on the
   processor are not recommended unless
   you have dedicated servers (over
   100$/month)
  Examples of bad applications for PMC
   are: Monte Carlo
        Genetic Algorithms…
Monte Carlo example

     Master sends the jobs to the slaves




                                           Slave 1

                                           Slave 2
              Master
                                           Slave 3

                                           Slave 4
Monte Carlo example

      The slaves will fetch random numbers from random.org



                                    Slave 1

                                    Slave 2
        Master
                                    Slave 3             Random.org

                                    Slave 4

 While(#results != #slaves)
  sleep(1);
Monte Carlo example


                                      Slave 1

                                      Slave 2
            Master                              Execute job
                                      Slave 3

                                      Slave 4
  While(#results != #slaves)
                               true
   sleep(1);
Monte Carlo example


                                         Slave 1

                                         Slave 2
                 Master
                                         Slave 3

                                         Slave 4

  While(#results != #slaves)
                                 false
   sleep(1);

  Calculate average and output
Source Code
Screenshot
Questions?

								
To top