FOSDEM 2005 Free and Open source Software Developers' European

							FOSDEM 2005:
Free and Open source Software Developers' European Meeting




                       Matthew Grove

                       DSG Seminar
                       1st March 2005
Outline
• Presenting some topics which were
  presented at FOSDEM that relate to DSG’s
  research.
• Four short talks:
  –Wikipedia,
  –Debian Release Cycle,
  –The Linux Kernel,
  –Linux Kernel Device Mapper.



                                             1
Wikipedia
• Wikipedia is a Web-based free content
  encyclopedia that is openly edited and freely
  readable.
• It is interesting to us because of our work writing
  portals.




                                                        2
Some Facts About Wikipedia
• 187 independent language editions sponsored
  by the non-profit Wikimedia Foundation,
• 80 million hits per day,
• 1.3 million articles,
• Wikipedia is run by MediaWiki open source
  software,
• 52 servers in several data centers,
• 1 part time member of staff :)



                                                3
Wikipedia Growth




                   4
Wikipedia Sub Projects
• Wikipedia - free-content encyclopedia,
• Wikiquote - free online compendium of
  quotations,
• Wikibooks - collection of open-content textbooks,
• Wikisource - repository of source texts,
• Wikimedia Commons - provides a central
  repository for free images, music, sound, video,
  used in any Wikimedia project,
• Wikispecies - free directory of species,
• Wikinews - free-content news source,
• Wiktionary - free multilingual dictionary.
                                                      5
Wikipedia Problems
• Wikipedia is generally perceived not not be
  entirely credible,
• Regardless of whether it is credible or not, the
  issue has become the acceptance that people
  generally perceive it not to be credible,
• The popularity means they need to add hardware
  and write software, both of which are hard to do,
• Spam - defacing a site by wrecking it (obscene
  pictures or deleting content) or adding
  advertising,
• “Revert wars”.
                                                      6
The Debian Release Cycle
• An explanation of how Debian produce a new
  stable release of their distribution from the testing
  branch,
• We run Debian Sarge (Testing) on all DSG
  servers.




                                                          7
Debian Releases
• ?? -- Etch, release date unknown,
• 3.1 -- Sarge, partially "frozen" July 2004,
  expected release in 2005,
• 3.0 -- Woody, July 2002,
• 2.2 -- Potato, August 2000,
• 2.1 -- Slink, March 1999,
• 2.0 -- Hamm, July 1998,
• 1.3 -- Bo, June 1997,
• 1.2 -- Rex, December 1996,
• 1.1 -- Buzz, June 1996.
                                                8
Debian Main Branches
• Unstable
• Testing
• Stable
• Experimental




                       9
Simplified Package Lifecycle
       Developer


        Incoming   Experimental


        Unstable
                    Developer
        Testing

                                Automated
        Frozen
                                Manual

         Stable
                                            10
The Path To Testing
• Must have been in unstable for the appropriate
  length of time,
• Must not have a greater number of "release-
  critical" bugs filed against it than the current
  version in testing,
• Must be compiled for all architectures slated to
  release,
• Must be a package for an architecture that is
  slated to release,
• Must not depend on versions of any packages
  which do not meet the above conditions.
                                                     11
From Testing To Stable
1. Start freezing the Testing branch - increase the
   propagation delay to reduce risk of adding
   bugs,
2. Proper freeze - only bug fixes make it through,
3. When all release critical bugs are fixed the new
   stable is released.




                                                      12
Where We Are Now




  Number of bugs concerning the next release
                                                13
  (excluding ignored and not-in-testing): 117
The Linux Kernel
• How the current Linux Kernel (2.6) is being
  developed,
• The DSG runs 2.6 on servers, which use SATA
  and all of the desktops,
• We still use 2.4 on SCSI and PATA servers.




                                                14
Some 2.6 Trees
• Stable (also known as vanilla or official), maintained
  by Linus,
• -mm is Andrew Morton’s patch tree,
  – He has been working on Linux since 2.4 - did some ext3 stuff
    and the low latency patch,
  – Frequently more experimental in nature than the official series,
  – Likely to end up maintainer of 2.6.
• -ac is Alan Cox’s patch tree,
  – He has worked on the kernel since 2.0, seen by many as the
    number two to Linus,
  – -ac is more conservative than –mm,
  – Alan is currently the maintainer of 2.2.

                                                                       15
The Old Development Model
• There was a stable kernel (2.2) which just had
  bug fixes,
• And a development kernel (like 2.3) which all the
  new features were added to.




                                                      16
The New Development Model
• No unstable tree (yet),
• New code gets put into –mm by Andrew,
• After testing Linus puts some of it into 2.6.x,
• Alan uses 2.6.x and picks patches to make it
  more stable and fix bugs.




                                                    17
Effect Of The New Model
• There is a longer time between stable tree
  releases than there used to be (so security and
  bug fixes take longer to appear),
• -ac is probably more stable than the vanilla
  kernel,
• -mm has more bleeding edge features.




                                                    18
What Alan Cox Thinks
• 2.6.x.y - point releases to do the security updates
  and bug fixes while we wait for a new stable
  kernel release,
• The counter argument is that the Linux
  distributions (vendors) will do the patching for you
  and you should use their kernel.
  –However people like using the stable tree as we trust Linus
   more than our vendors and picking and mixing patches
   ourselves can be hit and miss.
• Perhaps we should be using –ac for now?


                                                                 19
Device Mapper
• A brief introduction to the Linux Kernel Device
  Mapper and LVM2 with some examples.




                                                    20
An Introduction To LVM
• The Device Mapper is a the kernel driver for
  virtualising storage.
• It allows you to use LVM2:
  –LVM2 is the second implementation of Logical Volume
   Management on Linux.
  –LVM allows you to create virtual volumes (and volume
   groups) which can be resized and moved around
   dynamically.




                                                          21
Simple Example (Part 1)
• I have a small hard drive which I install Linux on.
• I make an educated guess as to what size
  partitions I need.
 /boot    /dev/hda1     10 Megabytes
 swap      /dev/hda2    256 Megabytes
 /       /dev/hda3     2 Gigabytes
 /home     /dev/hda4     6 Gigabytes




                                                        22
Simple Example (Part 2)
•    My root partition becomes full and I want to
     install some software, but there is still space on
     other partitions, my options are:
    1. Reformat the disk, change the partitioning scheme and
       reinstall.
    2. Buy a new disk and figure out some new partitioning
       scheme that will require the minimum of data movement.
    3. Set up a symlink farm on / pointing to /home and install
       the new software on /home.




                                                                  23
Simple Example (Part 3)
•       If I had used LVM my drive would have looked like
        this:
    /boot    /dev/hda1       10 Megabytes
    swap      /dev/vg00/swap 256 Megabytes
    /       /dev/vg00/root   2 Gigabytes
    /home      /dev/vg00/home    6 Gigabytes
•       I could reduce /home by 1 GB and add it to the
        root partition.
•       In the future if I bought a new disk I could add it to
        the existing volume group and extend the /home
        logical volume to include the new disk.
                                                                 24
Device Mapper Targets
•   Linear,
•   Striped,
•   Crypt,
•   Mirrored,
•   Snapshot,
•   Multipath.




                        25
Summary
•   It will be be interesting to see if Wikipedia is a
    success or whether it turns into the new Usenet,
•   Debian Sarge will be released when it’s ready,
•   We should think about using the –ac patch to the
    stable kernel branch unless they adopt 2.6.x.y,
•   LVM2 can help you whether you have 800MB or
    800GB of storage.




                                                         26
27
Links
• Wikipedia,
 http://en.wikipedia.org/
• The Debian Release Cycle,
 http://debian.org/releases/
• The Linux Kernel,
 http://kernel.org/
• Linux Kernel Device Mapper.
 http://sources.redhat.com/dm/



                                 28