2.2. Firefox Extension - Technion

Document Sample
2.2. Firefox Extension - Technion Powered By Docstoc
					            Technion – Israel Institute of Technology
             Department of Electrical Engineering

                     Mr. Ittay Eyal

               Oren Kalinsky 300731247
               Amir Tepper 037832052
                               Table of Contents
ABSTRACT .................................................................................... 3
1. INTRODUCTION ................................................................... 4
  1.1 OBJECTIVES OF THIS DOCUMENT ............................................ 4
  1.2 PROJECT OVERVIEW ................................................................ 4
  1.3 PROJECT DESCRIPTION ............................................................ 5
  1.4 INTRODUCTION TO PROJECT CONCEPTS .................................. 6
2. HIGH-LEVEL DESIGN ......................................................... 11
  2.1 SYSTEM STRUCTURE ............................................................. 11
  2.2. FIREFOX EXTENSION ............................................................ 14
  2.3 DISTRIBUTED HASH TABLE(DHT)........................................ 17
3. LOW-LEVEL DESIGN .......................................................... 20
  3.1 PROBLEMS & SOLUTIONS ...................................................... 20
  3.2 MODULES APIS ..................................................................... 22
  3.3 TK EXTENSION MAIN LOGIC ................................................. 23
  3.4 ASSUMPTIONS ABOUT USAGE ................................................ 27
  3.5 PERFORMANCE OPTIMIZATIONS ............................................ 27
  3.6 DEBUG SUPPORT .................................................................... 28
4. INSTALLATION ..................................................................... 30
5. DEMO ....................................................................................... 35
6. FUTURE EXPANSIONS ........................................................ 39
7. DEVELOPERS PERSONAL NOTES................................... 41
8. BIBLIOGRAPHY & REFERENCES ................................... 42
9. APPENDIX ............................................................................... 45
    Commands From TK to DHT .................................................. 46
  PROBLEMS & SOLUTIONS APPENDIX .......................................... 48
  FIREFOX EXTENSION STRUCTURE APPENDIX ............................. 54

Many of us use the internet from different locations – home, office or a friend’s house.
When we move from one location to the other we cannot access the results of our
previous surfing like history and bookmarks.

Until now it was only possible to save the surfing data on a server, and load it later. In
this project we developed a software which uses a distributed database and imbued into
the Firefox web browser.

The distributed database is fast and dependable which makes it more appropriate to our
software than a single server that might crash and less efficient with handling numerous
requests at the same time.

Imbuing the software into the Firefox web browser supplies the user with a user friendly
and transparent way to access his surfing data.
                               1. Introduction

1.1 Objectives of this document
The project main goal is to develop an extension for Firefox, which allows the users to
backup their profiles and load them from a different location.
This document best describes our different steps in accomplishing this goal.

Generally this document consists of two major parts;
The first part describes our work and different studies made about the Firefox and DHT
(Distributed Hash Table). It elaborates on the high and low levels of the design.
The second part contains the information on how to use the extension, which consists of
an installation, user manual and a demo.

This document main purpose is to help the user understand the extension’s functionality
and structure.

1.2 Project Overview
The main goal of Firefox roaming profile is to develop an extension for the Firefox web
browser which uses a distributed databaseto allow the clients to backup any part of their
profile and easily load it from any computer connected to the internet. Firefox profile is
the data each user configures or adds to his Firefox such as bookmarks, history,
passwords and extensions. Allowing the user to easily load it in any location helps
provides a better and friendlier working environment.
1.3 Project Description
The Firefox roaming profile project consists of two main modules, the Firefox extension
and the distributed database. One of the main design guidelines of this project was
implementing each module as an individual, which allows us to switch, change or add
another module with minimum impact on the other modules.

The extension serves as a user friendly GUI that provides the user an easy way to
configure which parts of the profile he wants backup or load. The backup option stores
the data in a distributed database called a DHT which is constructed from all the
connected users, and the load option retrieves the data from the DHT to the user’s
computer and loads it to the Firefox.

The DHT is activated when Firefox is opened in order to allow the DHT to work in the
background as long as the web browser is open. This option contributes to the number of
users that are connected to the DHT, which improves the efficiency of the DHT due to a
better distribution of the work load for every computer in the system.

Another key design guideline is using the DHT as a database. The DHT is designed in
order to efficiently retrieve data and handle continual user number changes; therefore it
fits best to demands of this project’s database. For more information about the DHT
please refer to chapter 2.3 Distributed Hash Table (DHT)
1.4 Introduction to project concepts

1.4.1 DHT and Chord: What is a Distributed Hash Table(DHT)?

DHT provides an interface very similar to a normal hash table, by including the
commands put and get with a given key. On the other hand, the DHT is built over a
distributed network and designed to contain a potentially extreme large number of nodes
and to handle continual node arrivals, departures, and failures. The Distribution provides
the DHT its reliability using redundancy, and also the ability to store an enourmous
quantity of data by dividing the load by the different nodes. The most common use of the
DHT is peer-to-peer with implementations used by torrent trackers such as Kad (in
eDonkey) or Azureus. DHTs are designed to efficiently combine distribution, resilience
(mostly to continual change of the nodes set, called churn), reliability and distribution.

The common ground between all DHTs is that they all depend on consistent hashing. For
example there is a naive DHT implementation which is based on modulo. At the moment,
there are 8 nodes and 100 keys. In order to assign a key to a node the DHT calculates
modulo of the key number by the node count. For instance, key 16 will end up on node 2
and key 58 on node 7. The problem occurs when a new node arrives to the DHT or a
node leaves. The trivial solution is modifying the node count and changing all the
assignments of the keys, therefore all the stored key/value pairs have to be moved around,
which leads to a mess in small systems and to chaos in real ones which usually have an
enormous number of nodes and churn. Consistent hashing solves that problem.
In Order to explain consistent hashing we use the next diagram:

Initially there is a hashing space which usually corresponds to the target space of the
SHA1 hashing function, although there are some implementations with other hashing
functions. The key of each value is calculated by its SHA1 target. Usually the space of
the DHT will be represented as a circle and will be followed clockwise in a cyclic method
(when you get past the maximum, you get back to zero). Each key\value pair will be
attributed to the closest node in the hashing space, going clockwise.
In case a node leaves the network, its key/value pairs need to be re-attributed and in case
a node joins the network, only its follower’s node key/value pairs which are closer to the
new node in the hashing space, need to be re-attributed to the new node. The rest of the
hash space stays untouched.

For example, there are two nodes in the DHT at the moment, one is positioned 50 in the
hashing space and the other positioned at 200. If we want to store a key / value pair in the
DHT and the key hash is 100, it will be attributed to the node positioned at 200. Another
key hash of 30 would go to the node at 50. In case the first node leaves, the pair
positioned in 30 will be re-attributed to the second node. On the other hand, in case a new
node arrives and positioned at 40, the pair positioned in 30 will be re-attributed from the
node at 50 to the node at 40. The chord implementation

The basic idea of the chord DHT implementation is once there are nodes on the hash ring;
each node only needs to know its successor. For example, in the diagram above, a key
lookup starting at node (a), in case it doesn’t have the key, node (a) would just forward
the lookup to (b). In case (b) doesn’t know about the key either, it will forward to the
lookup to (c) and so on. Eventually, the lookup will find a node which is familiar with the
key that was being searched. The advantage of this method is that it only requires a node
to know about a single other node, its successor. On the other hand, the lookup time with
this method is linear to the number n of nodes (O(n)) and also not very resilient against
node disappearance.

To improve the lookup time, Chord extends the basic idea by introducing a routing table
containing O(log n) elements. The first node in this table is the closest successor from our
node’s position (say p) plus 2, the second element is the closest successor of key (p + 4),
the third of key (p + 8) and so on. We can see that some nodes will be duplicated in this
table, for instance it’s fairly unlikely you’ll have a node between p + 2 and p + 4, so the
routing information to maintain is decently low. Low routing information is important
because maintaining valid routing information requires constant traffic to make sure
known nodes are still alive. With this improvement, the area “close” to a node in the ring
is very well known by that node and it’s proven that a lookup will take, with high
probability O(log n) hops. This lookup time is very good in classic P2P systems: in a
network of one million nodes, you shouldn’t get much more than 20 hops to find a value.

When using the new improvement in chord it’s causing the routing table to jump forward
very quickly, and when a close node disappears it might not have enough backups. To
improve reliability and allow replication for backups, Chord introduces a successor table
of size O(log n) which simply contains the successors of a given node and acts as a
backup. OpenChord as an open code DHT
OpenChord is an open source Java-based implementation of the Chord DHT described by
Stoica et al. It provides an interface for Java applications to take part as a peer within a
DHT and to store and retrieve arbitrary data from this DHT. So Java-based P2P
applications can benefit from properties of DHTs. Open Chord is called open, as it is
distributed under GNU General Public License (GPL), so that it can be used and
extended for own purposes for free and as desired.

After searching and trying different DHT implementation, such as Bamboo and Project
Voldemort, we have chosen to use OpenChord as a basic DHT to implement our
distributed DB on for the next reasons:
   1) Open code – the usage and manipulation of the code is free and easy.
   2) Java based implementation – java applications use a JVM and therefore usually
       the same program operates on all of the popular operating systems. Also, java is
       one of the most popular, rich and easy OOP languages known today.
   3) Functions on all of the popular operating systems – in the contrary of some DHT
       implementations of DHTs in Java we’ve seen, OpenChord is designed to support
       all the popular operating systems (Windows, Linux, MacOs).
   4) Great documentation – OpenChord has a good and user-friendly documentation
       either in the manual or the code.

Good support – the developers of OpenChord still support the software

1.4.2. FireFox- Firefox is an open source web browser developed by the Mozilla
       Foundation. The web browser was picked as the platform for the project due to it's
       flexibility and strong developers community (for a more general explanation
       please refer to
1.4.3. Firefox Extension - Firefox extensions are small add-ons that enhance the
Firefox browser with additional functionality. A Firefox extension may add anything
from a toolbar button to a completely new feature.
The extension allowed us to install the new feature of saving the user information on a
DHT almost completely transparent to the user, while allowing us having the power to
change the functionality and look and feel of the Firefox browser to fit our requirements.

1.4.4. Xul: Xul,XML user interface markup language, is the language that extensions
are written in, it allows creating a rich user interface and it integrate JavaScript scripts
into it's code. TK extension user interface was written with Xul while the logic was
integrated using JavaScript. (for a more general explanation please refer to,

1.4.5. Javascript: JavaScript is an object oriented scripting language. Javascript was
used in TK in order to imbue the logic and separate the code into different modules.
( for a more general explanation please refer to

1.4.6. Firefox profile: a Firefox profile holds all the user information: extensions,
bookmarks, history, passwords etc…, the project main goal was to enable a user to save
his profile over the network so he could retrieve it no matter where he uses his Firefox
browser (for a more general explanation please refer to

1.4.7. Febe: Febe (Firefox Environment Backup Extension) allows you to quickly and
easily backup your Firefox data and profile locally.
TK extension lays heavily on FEBE code, taking mainly the logic of save and setting the
user profile data (for more information please refer to
                        2. High-Level Design

2.1 System structure
One of the main design guidelines of this project was to assemble the software from
individual modules. The purpose of this guideline is to allow easy changes in one of the
modules (such as changing a web browser or a database) or a change in the software
design (such as adding another module). Therefore the software is divided into two main
modules: web browser extension & database.

In our case the web browser is Firefox and the database is a DHT implementation called
Chord. We chose Firefox mainly for its rich support resources over the internet compared
to other web browsers known today. As for the DHT, its ability to handle node arrivals,
departures, and failures and still efficiently retrieve the wanted data is what pushed us
into choosing it as our database.

The Firefox extensions main goals are:
   1) Provide a user-friendly GUI that gives the user the possibility to choose which
       part of his profile he wants to save or load.
   2) Retrieve or load the wanted parts of the profile from Firefox and save them on a
   3) Notify the DHT that it needs to send or retrieve a specified file to or from a
       specified location on the hard drive.

The DHT main goals are:
   1) Implement a distributed database (in the form of a distributed hash table).
   2) Transparently Handle frequent users’ arrivals and departures.
   3) Efficiently retrieve or send the wanted data.
   4) Function properly on as many operating systems as possible.
   5) Supply a transparent functionality to the Extension.
We chose to build our database over an implementation of a DHT in Java (for further
information please read the chapter (2.3)DHT module).

In addition to its advantages, the individual modules design method has its shortcomings,
and the most essential one is the communication between the different modules. After
trying many different processes communication methods such as loading java from
Javascript or opening a socket (q.v. – (3.1)problems and solutions), we decided to use
files as the input\output streams of the modules.

The communication algorithm is as follows:
   1) The extension runs the DHT as an independent process.
   2) The DHT starts listening to his input stream, the “js” file (frequently reading the
   3) The extension sending a command with the wanted parameters to the DHT input
   4) The DHT executes the command and sends the result (“true”\”false”) to the
       extension input stream, the “java” file.
   5) In case the command is “leaveDHT” the DHT disconnects from the Chord
       network and terminates itself. Otherwise, go back to step 2.
A diagram of the System structure:

                                 (JavaScript +


                          RUN .exe
                                                         in pu

                 Open Chord             DHT output
                    DHT                  stream

                                             DHT INPUT

                                        Diagram 1

Diagram explanation:
As explained in this chapter, the modules are divided and individual. Some of the
modules are imbued into the Firefox process and some are running as an individual
As we can see in “Diagram 1” the modules that are inside the Firefox square are part of
the Firefox process and imbued into it. The other processes are individual processes that
are running in the background.
2.2. Firefox Extension

2.2.1 What is a FireFox extension:
A Firefox extension is an add-on to the Firefox browser application and
allows adding different customized behaviors.
The Extension is implemented using XUL (an XML User Interface markup language
created by Mozilla) that operates under the Gecko Engine on Firefox.
On startup the Gecko engine load the default XUL files of Firefox and on top of them
inject all the extensions XUL files.
XUL allow a feature rich GUI environment and uses JavaScript and c++ to imbue logic.

2.2.2 Extension Purpose implications:
The extension purpose was to allow users to use their Firefox profile data freely on
numerous computers.
This required incorporating 3 main modules in the TK extension:
      Main extension module: Connection between the modules and a GUI to configure
       and run the extension.
      Profile module: Connection to the Firefox database.
      DHT connection module: Connection to a DHT that will save the data over the

2.2.3 Main extension module:
The Main extension module binds all the modules in the extension and invokes all the
commands in the system.
The commands implemented are:
      Save Profile command:
       description: save a pre configured part of the profile to the DHT.
          Load Profile command:
           description: load a profile part from the DHT and save it on the current profile.
               o   Instruct DHT connection module to receive a part of a profile from the
                   DHT and save it to the temp directory.
               o   Instruct profile manager to load the part received into the current user
               o   Display the results to the user.

          Startup command:
           description: load the entire extension configuration and open a connection to the
               o   Instruct DHT connection module to start the DHT.
               o   Start Timers for timed Saved if configured.
          Shutdown command:
           description: close the connection to the DHT.
               o   Instruct DHT connection module to close the connection to the DHT.

2.2.4 Profile module:
Profile module proposes are to access the data of FireFox,get it for backups and set it for
Firefox profile holds the following data:
              Extensions: holds all the extension of the profile.
              Themes: Style the browser GUI.
              Bookmarks: holds bookmarks to different site.
              Preferences: holds user defined preferences.
              Cookies: holds cookies to different sites.
              Usernames and passwords: holds the user.
          Browser history: holds the user site history.
The profile module uses different LiveConnect classes to access the data (for example the
PlacesUtils class for access Bookmarks), these classes allow serialization and de
serialization of the data to and from Json files.

2.2.5 DHT connection module:
Original Design: in the original design of the connection module, the module connected
to a java class that implemented the DHT interface (as described in chapter 3.2 Modules
APIs) to allow flexibility and future change of a DHT.
During the project development the connection module design was changed due to the
problem of load a java class (as described in chapter 3.1 Problems & solutions), thus
module was changed to support File I/O communication with an external application that
runs a DHT and support an the DHT API.
2.3 Distributed Hash Table(DHT)

2.3.1 Adjustments for the Firefox Roaming Profile project
OpenChord is a good base for a distributed DHT database, but there were a few demands
which forced us to adjust, add or modify the OpenChord:
   1) Demand – the data that is transferred on the DHT is files that contains the firefox
       backup data.
       Problem – the OpenChord only supports serialized classes (classes that implement
       the serialization interface).
       Adjustment – in case of a backup, we loaded the files to a array of bytes and sent
       it through the web. In case of a restore, the data from the web is transferred into a
       array of bytes and then being written to a file.

   2) Demand – the DHT need to communicate with the Firefox extension through file
       Problem – running the OpenChord is only possible through a console or using its
       Adjustment - an overlaying class has been built over the DHT to handle the
       communication to the Firefox extension through the files I/O streams.

   3) Demand – the DHT API needs to be absolutely transparent to the outside.
       Problem – in the chord there is no support for a failing the initializing of the chord
       instance. Furthermore, the API uses classes that are not known to the outside.
       Solution – we wrapped the Chord with an interface called DHT that gives the
       outside user a transparent usage of the DHT through handling errors and using
       strings instead of classes that just wrap strings (such as Key & Bootstrap).
2.4. Modules Communication
The Connection between the modules is based on 2 files, each functioning as an input and
output file for TK and to the Open Chord DHT.
The file names are:
"js" – TK output file\ Open Chord DHT input file.
"java" – Open Chord DHT output file\ TK input file.
"log" – log Open Chord DHT commands.

Command Format:
The format is the same for sending and receiving.
The command is a string, that start with the command name string, followed by the
command arguments with a " " (white space) delimiter, ended by the command delimiter
"!", an example of a command:
sendProfile c:\backup\sami.txt !
example command explanation:
"sendProfile" – the command name.
"c:\backup\sami.txt" – command argument number 1.
"!" – End command delimiter.

For more information on the different commands between the modules please read the
Appendix Commands From TK to DHT.
Communication Example:

      TK              Init atep djk !
                                                                                         JS                     DHT_run

                                                                                                                to DHT

                                                                                               Tr u e


     file            dPr
                           o fi l e
                                                                ma r

                                                                                                                 Send file
                                                                                                                  to DHT

                                                                                                Tr u e

       TK                                                                               JAVA
                         3. Low-Level Design
3.1 Problems & solutions

1. Load DHT java class to the Firefox:
  The Problem: running the DHT class from the Firefox environment (and not as an
  individual process).

  Note: this was the most critical problem we had and a lot of effort was put into
  solving it. More information can be found in the Appendix.

  Solution: load the DHT as an individual process and support the communication
  between the two processes using files.

2. Load a jar file using the extension:
  The problem: Starting a java jar Application through FireFox extension.

  Solution: converted the jar file into an executable file and run the using the run

3. Closing External Application (DHT):
  The problem: The DHT should be closed whenever FireFox is closed.

  Solution: We found a FireFox event that indicates whether the browser is shut down
  and wait until we safely send a termination command to the DHT through the file IO.

4. JavaScript and XUL Developing Environment:
  The problem: FireFox doesn't have a developing environment for extensions.
  We searched the Internet for solutions because a decent developing environment
  meant the difference between a slow code reviewing (with no debugging) to a full
  functioning environment. The main problem was that due many developing
  environments support JavaScript and even Xul, none could do it using the full
  FireFox and Gecko functionality, thus they were unusable to debug and develop our

  Solution: The selected solution was a mix of all the failed solutions:
     o JavaScript Development –visual studio that enabled IntelliSense and minimal
         Class explorer.
     o Xul Development – XulEclipse – enabled a simple designer and helpful code
     o Debugging – using FireFox with a small help of the JavaScript debugger
         extension and FireFox message console.

5. Modifying FEBE Code to TK, GUID:
  The Problem: Since every extension need a unique GUID (the identifier of the
  extension), we needed a new GUID to replace the FEBE GUID.
  Normally The GUID is defined in the install.rdf file, but because FEBE support
  automatic updates and is registered in the FireFox add-on library it holds many more
  hard coded GUID references.

  Solution: The solution required a thorough code review of all GUID related methods
         (but no documentation meant that all the code went a code review).
3.2 Modules APIs

 * initializes the connection to the DHT
 * @param bootStrap - the URL of the bootstrap computer
 * @param userName - the user name, used for key
 * @param password - the password given by the user, used for key.
 * @return true if succeeded false otherwise
boolean init(String bootStrap, String userName, String password);

 * send a profile to the DHT according to the key of the user
 * @param filePath - contain path to the Firefox profile to send
 * @return true if succeeded false otherwise
boolean sendProfile(String filePath);

 * receive a profile from the DHT
 * @param filePath - contain path to the firefox profile to receive
 * @return true if succeeded false otherwise
boolean receiveProfile(String filePath);

 * close the connection to the DHT
 * @return true if succeeded false otherwise
boolean leaveDHT();

Firefox extension (aka TK) module API:
The TK module API to the DHT, is under tkDHT.js, which uses a DHTinterface (defined
under 2.3 DHT) it controls all the connection between the DHT and the extension, thus if
a new connection paradigm is implemented (such as implementing the DHT under the
extension) tkDHT.js should be changed.

API Methods:
       o DHTenabled() – return true if the DHT connection enable preference is true.
      o DHTstart() – if the DHT is enabled, the method get all the DHT connection
         parameters (user name,password,bootstarp) and connects to the DHT.
      o DHTinit(bootstrap,user,pass) – tries to connect to the DHT, using the DHT
         connection parameters.
      o DHTleave() – sends a "DHTleave" command to the DHT, that closes the
      o DHTsend(filepath) – sends the file (from filepath) to the DHT, return true for
         success false otherwise.
      o DHTrecive (filename,filepath) – tries to retrive a file with the key "filename"
         from the DHT, to the directory "filepath", return true for success false
      o testDHT () – sends a message to the DHT, the content of the message is taken
         from the debug window in the TK options menu.

3.3 TK extension Main Logic
Tk.js Main methods:
                tkLoad(): tries to connect to the DHT (or open one if the bootstrap is
                 "NONE"), load all the default values of the extension, checks for
                 missed scheduled backups and starts the backup reminder tool.
                tkRemind(): starts a listener that starts backups according to the
                tkInit(): enables the different privileges that are needed (such as
                 XPConnect, browser write, browser read) and checks if the backup
                 directory is valid.
                tkGetBDD(): checks that the directory backup defined by the user is
                tkAbortBackup(): Aborts scheduled backup before it starts (only if
                 statusbar icon is in "warning" state)
                tkSanityCheck(): Checks to see if there is anything to backup, for
                 selective profile backup.
   dotkBackup(): start the backup, calls all the backup methods, shows
    the progress bar, creates and open the results page when backup ends.
   tkStoreBUdate(): Writes last backup info to preferences
   tkWriteResults(): Creates results page
   tkStartBackup(): checks that all the backup paramertes are ready for
    backup and if so starts "dotkBackup" method
   tkPickFile(filter): tries to retrive the file from the DHT, and if
    succeeded returns true, if the file was not found in the DHT or the
    DHT is not connected it return fails (uses tkDHT.js).
   tkConfirmRestore(msgNum,bDate): checks if the file that should be
    restored is not outdated.
   Backup related methods:
        o tkBackupBookmarks() – backup the user's bookmarks
        o tkBackupPreferences() – backup the user's preferences
        o tkBackupCookies() – backup the user's cookies
        o tkBackupUserChrome() – backup the user's chrome
        o tkBackupPasswords() – backup the user's passwords
        o tkBackupSearchPlugins() – backup the user's search plug-ins
        o tkBackupBrowserHistory() – backup the user's browser history.
        o tkBackupFormFillHistory() – backup the user's form fill
        o tkBackupPermissions() – backup the user's premissions.
        o tkBackupUDBu() – backup the user's UD data.
        o tkBackupProfile() – backup user's profile.
   Restore related methods:
        o tkRestoreBookmarks() – restore the user's bookmarks
        o tkRestorePreferences() – restore the user's preferences
        o tkRestoreCookies() – restore the user's cookies
        o tkRestoreUserChrome() - restore the user's chrome
        o tkRestorePasswords() – restore the user's passwords
        o tkRestoreSearchPlugins() – restore the user's search plug-ins
                       o tkRestoreBrowserHistory() – restore the user's browser history.
                       o tkRestoreFormFillHistory() – restore the user's form fill
                       o tkRestorePermissions() – restore the user's premissions.
                       o tkRestoreProfile() – restore user's profile.

tkfileIO.js Main Methods:
      o FileIOInit(): connects to the files "js" and "java", and start the "dht_run.exe"
          (the DHT).
      o DHTstartJar(): starts the DHT with "dht_run.exe"
      o tkGetExtDir(): returns the extension's directory as a string value.
      o tkCreateFile(filename,clear): creates a file named "filename" in the
          extension's directory, if clear is true, the file will be cleared if he is already
          exist. A pointer to the file is returned from the method.
      o tkGetNextCommand(): reads the next command from the read stream of
                   Method variables:
                          Count : the number of time the method started in the current
                           operation (operation is from tkWaitForcommand start, until
                           end), at the start of operation count=0.
                          DataRead : the data that was read from the stream in the
                           current operation, at start of operation DataRead = null.
                          tmpStr : the buffer to which the next char is read each read
                   Algorithm:
                          read the next char in the buffer to tmpStr
                          if read nothing assume end of file
                                 o increase count by one
                                 o if count is bigger then 20 return "timeout"
                                 o else start tkGetNextCOmmand() asynchronously after
                                    100 ms and return.
             If read=="!" (end of command) trim last char in DataRead and
             Else
                 o Add tmpStr to the end of DataRead
                 o start tkGetNextCOmmand() asynchronously after 10
                        ms and return.
   tkWaitForcommand(): initializes all the "tkGetNextCommand"
    recurring method variables and then starts the method and returns its
    result which is the last command that was in the read file "java"
   tkInitFileForReadFromStream(file): initializes the file "file" for read
    and return a read stream of the file.
   tkPrepareFileForRead(filename): creates a new file named "filename"
    and returns a read stream for the file.
   tkInitRead(): checks if the current "readstream" exists and if it doesn’t
    builds a new file for readstream.
   tkCheckTrue(): checks if the next command that came in the read
    buffer is "true !" or "false !" and return the result.
   tkPrepareFileForWrite(filename): creates a new file named "filename"
    and returns a write stream for the file.
   tkInitFileForWriteFromStream(file): initializes the file "file" for write
    and returns a write stream of the file.
   tkInitWrite(): checks if the current "writestream" exist and if it doesn’t
    builds a new file for writestream.
   tkWriteMsg(msg): writes the message "msg" on the write stream.
   tkInitAllIO(): opens the write file and stream and the read file and
   tkShutfileIO(): closes the read stream and write stream if they exist.
   myDump(aMessage): sends a log message "aMessage" about the file
    IO operations to the FireFox message log.
3.4 Assumptions about usage
The user must declare the following definitions for the Extension to function properly:
   1. FireFox version – The user's FireFox version must be between 3.0 – 3.5
   2. Java Version – The java versions must be over 6.12.
   3. Directory – A local Backup directory must be declared (Under "Directory" Tab on
   4. Enable DHT connection - enables the connection to the DHT (Under "TK" Tab on
   5. Bootstrap – the bootstrap of the Chord DHT (Under "TK" Tab on options).
   6. User Name – the user name in the Chord DHT (Under "TK" Tab on options).
   7. Password – the password in the Chord DHT (Under "TK" Tab on options).
   8. Port – the port 8080 needs to be available for the DHT to connect.
   9. Backup files – the paths that are sent to the DHT in send\retrieve must exist and
       available in order that the DHT could use them.

3.5 Performance optimizations

   1. DHT optimizations:
          Sleep: part of the DHT communication algorithm is a continual read from the
           “js” file for the next command. After a implementing a busy wait listener to
           the “js” file, the DHT_run process has ~98% CPU usage and 50kb memory
           usage due to the frequent IO requests. In order to reduce CPU usage we
           decided to force the process to sleep between each file read for one second in
           case there was no new command and for 10 milliseconds in case a new
           command is being written.
           This optimization has reduced the DHT_run process to 0% CPU usage and
           13kb memory usage.
   2. TK Communication Optimizations:
          Sleep: in order to prevent the use of a busy wait loop while reading data
           written from the DHT on the communication files, a sleep method was created
           using the JavaScript method "settimeout" (start a method after a certain time

           given as an argument), without this optimization The browser got stuck every

           time a command was sent to the DHT.

          Timeout: the communication to the DHT is based on files which is not a

           stable way of communication thus a Timeout feature was essential to make
           TK a relatively low performance extension on FireFox.

3.6 Debug support

Debug messages:
The extension sends numerous messages to the FireFox Error console (can be found
under Tools in the FireFox toolbar or accessed by the Ctrl+Shift+J shortcut).

TK debug menu:
The TK debug menu includes the following functionality:
   1. Clear TK preferences: Clear all the TK preferences and return them to their
       defaults in the FireFox Chrome.
   2. Verify extension directory: Verify that the local backup directory is valid.
   3. Init Files: try to connect and empty the IO files "java" and "js".
   4. Message to send: when the button is pressed and the DHT is opened on the
       computer, the content of the Textbox would be sent to the DHT.
   5. Close Files: close the streams and handles to the "js" and "java" files.
   6. Start Jar: tries to start the dht_run.exe file.
In order to automate the extension packaging process, a small c# "script" was written that
Zip the extension content, Turn it to an xpi file and enables a faster installation to a new

DHT log file
Originally the DHT works as a console application and many debug messages are sent to
the console. When loading the DHT application from the Firefox the console doesn’t load
and therefore there isn’t any information about the DHT run status. For that reason, we
added a log file called “log” where some debug notes and run status comments are
written at, which can help understand the status of the DHT in case of a bug.

DHT Debug function
An extra function was added to the DHT interface called “testDHT “ which its sole
purpose is to allow the developer to easily test the connectivity and functionality of the
DHT class. This function was frequently used while trying to load the DHT class from
the Firefox (q.v. problems & solution, problem 1).
                               4. Installation
The Installation will demonstrate how to install TK and define a
basic configuration.

   1. Download TK xpi file (xpi file is an install file for a FireFox extension).
   2. Drag xpi file into FireFox window, or open it with FireFox.

   3. Software installation window: Software installation window would be open, click
         the "Install Now" option (as shown in the picture) the button will become valid
         only after 4 seconds so have patience!
Initial Setup:
1. Tk Options: open the TK options menu, under Tools (in the FireFox main
   toolbar), TK tab, and then choose the options tab.
2. Options definition window:
          1) Press the Options tab, the following window would be open.
          2) Backup type: choose your backup type, selective, full profile or both.
          3) Backup items: choose the items you want to be backup and retrieved
              (this option will work only if you selected a selective backup type)
3. Directory definition window:
          1) Press the Directory tab, the following window would be open.
          2) Press the Browse Button: a dialog window named "Browse For
              Folder" (labeled "3" in the picture) where you can choose the
          3) Choose the directory for backups.
4. TK definition window:
          1) Press the TK tab, the following window would be open.
          2) Press the "Enable DHT connection" to enable DHT.

5. TK definition window:
          1) Define a bootstrap, in the format: url:port or NONE for a new DHT.
          2) Define the user name (no spaces are allowed).
          3) Define the password (no spaces are allowed).
          4) Press the save login button to save the login into FireFox.
6. Enjoy!
                                   5. Demo
In this demo we’re going to backup and reload the Firefox bookmarks using the TK

Initial state:
User’s home computer

User’s work computer

User’s home computer steps:
  i. Define on the TK options which profile parts is wanted for the backup(see 4.
 ii. Press ToolsTKPerform Backup.
iii.   Information tab on the backup will open.
User’s work computer steps:
   1) Press ToolsTKRestoreRestore Bookmarks.

   2) A window will popup on the bottom right side of the screen, press OK.
3) Done!
                        6. Future Expansions

                     All the future expansions
Support all profile Backup Features:
TK should have support for all the parts of the FireFox profile (extensions, themes etc…)

Enable FireFox to share files on the DHT:
The DHT can enable file sharing through all the DHT to all users, thus the extension can
be upgraded to a backup service to numerous files and just profile backup file of each
user or even to use it as a file sharing system with other users.

Restructure TK extension:
TK was built on top of the FEBE FireFox extension, which doesn't utilize JavaScript
Structural capabilities and uses a rather "c like structure" (global methods, global
variables), a restructure to a more "c++ like structure" (classes, inheritance) will make the
extension easier to maintain and debug.

Load the DHT class from FireFox:
As discussed in the previous topics, we have tried to integrate the DHT in the Firefox
extension and encountered many problems which are related to the immature
environment of the Java-Javascript communication. Hopefully, when the integration will
be supported by Mozilla and Sun, we could successfully use the DHT class from the
Firefox extension and not as an individual process.

Security expansion:
There are many security breaches in the software which makes the extension not safe for
usage at the moment. The critical security bugs that need to be fixed are:
           a. The password of the user is passed openly through the communication
              files of the modules.
The profiles are sent through the DHT unencrypted, which are then saved on other users’
computers and they can access private information of the user.
              7. Developers personal notes
Working on this project, we got introduced to a new kind of challenge, which is working
on a pioneer system.
We’ve developed the project from stage one, from determining the project requirements
to high-level design all the way to the small details and manipulating and modify open
code to meet our requirements. Motivated by the huge potential, we believe to have
delivered a successful system that may be used even in its current version.
We are thankful for this opportunity given to us by the Networked Software Systems
Laboratory lab in the EE-department of the Technion. And hope to see this system
growing into a product that serves every day users of Firefox all over the world.
              8. Bibliography & References
       FireFox related:
      Mozilla Developer Center:
      Mozilla Support Center:
      Article about Firefox extension development:
      A forum for FireFox extension development:
      A forum for FireFox extension development:
      Firefox extension development guide:
       Offline SQLiteSVG database applications with Firefox
      FireFox add-ons Library:
      References to other Mozilla projects:

JavaScript related:
      JavaScript tutorials and references:
      JavaScript tutorial:
      JavaScript tutorial:

DHT related:
     Open Chord Information:
     Open Chord implementation explanation:
     Bamboo Reference:
     Why not use Bamboo:
     Project Voldemort:

Java & JavaScript communication:
     A Discussion about connecting to Java
     Core JavaScript Guide 1.5: 9 LiveConnect Overview
     LiveConnect Data Types (JavaScript: The Definitive Guide, 4th Edition)
     JavaScript: The Definitive Guide, 4th Edition
     Java Firefox Extension - SIMILE
     Java in Firefox Extensions - MDC
     Can a Java application communicate with a firefox extension? • mozillaZine
     Extend a Java class in javascript - Daniel -
      rhino - MarkMail
     aberrare humanum est!: Java Firefox Extension
     New To Java - How to run Java from Firefox Extension using liveconnect?!

     Rapid Application Development with Mozilla – can be found online on:
9. Appendix
                     Commands From TK to DHT

Init "bootstarp" "user name" "password":
Connect the computer to the DHT, if "NONE" was sent in the bootstrap argument a new
Open Chord DHT will be opened on local machine.
"bootstarp" – the bootstrap to the DHT, if "NONE" argument is sent a new Open Chord
DHT will be opened on the local machine.
"user name" – Identify the user on the DHT network.
"password" – allow secure connection to the DHT.

True ! – Connection succeed.
False ! – Connection failed.


Disconnect the computer from the DHT and closes the DHT execulabte application

True ! – disconnected and close DHT execulabte application succeessfully.
False ! – disconnection failed.
sendProfile "filepath":

Send the File at "filepath" to the DHT using the filename as key.
"filepath" – the file path.

True ! – Sent succeefully.
False ! – Sent unsucceefully.

receiveProfile "filename" "filepath":
Get the file with the name "filename" into the directory "filepath".
"filename" – the file to get from the DHT.
"filepath" – the directory the file should be copied to.

True ! – recivied succeefully.
False ! – recivied unsucceefully.

Commands From DHT to TK:
                    Problems & solutions Appendix

1. Load DHT java class to the Firefox:
   The Problem: running the DHT class from the Firefox environment (and not as an
   individual process).

   Note: this was the most critical problem we had and a lot of effort was put into
   solving it. More information can be found in the Appendix.

   Failed solution:
   in order to understand the solution, there are two tools that the reader needs to be
   familiar with:
          LiveConnect – a feature of Web browsers that allows Java and JavaScript
           software to intercommunicate within a Web page. From the Java side it allows
           an applet to invoke the embedded scripts of a page or to access the built-in
           JavaScript environment, much as scripts can. Conversely, from the JavaScript
           side, it allows a script to invoke applet methods, or to access the Java runtime
           libraries, much as applets can
          XPCOM - a cross platform component object model, similar to Microsoft
           COM. It has multiple language bindings, letting the XPCOM components be
           used and implemented in JavaScript, Java, and Python in addition to C++.
           Interfaces in XPCOM are defined in a dialect of IDL called XPIDL. The
           XPCOM allows us to build a COM-like component for Firefox, platform
           independent (because it is written in javascript).

We tried to create an XPCOM component that will use LiveConnect to get to the Java
code. Why didn't we do it only with LiveConnect? Because XPCOM allows a single
instantiation of the component through all the browser instances (shared).
A diagram of the original system is:

                 chrome                                           JVM



     LiveConnect                  XPCOM

       Java class

After overcoming some difficulties with using liveconnect and loading a simple java
class into Firefox, we successfully loaded our DHT class. What little that we know that
all the Firefox extensions are running in a sandbox, a closed security environment that
checks the extensions before letting it run on the operating system. While running in the
sandbox, our extensions didn’t get enough permission to open a socket, which caused the
DHT connection to fail.
The exception stack contains the next information:
Caused by: Could not set up IO channel to host

at de.uniba.wiai.lspi.chord.service.impl.ChordImpl.joinHelp( As   we can see, the joinHelp
function in the DHT that suppose to connect to the DHT network, has failed to open a

A wide internet search for a solution has made us realize that the problem was connected
to security privileges. After authorizing all the security options for an extension in the
Firefox, the extension still failed to open a socket using the liveconnect. Eventually, after
a month work on the issue, we decided to abandon the design pattern of loading the DHT
class from the Firefox.

Solution: load the DHT as an individual process and support the communication between
the two processes using files.

2. Load a jar file using the extension:
     The problem: Starting a java jar Application through FireFox extension.

     Failed solutions:
               Use nsILocalFile Component run method – assuming the system know how to
                run a jar file, let the Opearting system use it's defualt running application for
                Jar file ( if JRE is installed on the computer, the OS knows how to run a jar
                file by defualt) – the run method could not use the Operating system defualt
                run method.
               Use Javaw directly – using the run method we tried to run the javaw with the
                DHT application jar – we had privleges problem.
  Solution: converted the jar file into an executable file and run the using the run

3. Closing External Application (DHT):
  The problem: The DHT should be closed whenever FireFox is closed.

  Solution: We found a FireFox event that indicates whether the browser is shut down
  and wait until we safely send a termination command to the DHT through the file IO.

4. JavaScript and XUL Developing Environment:
  The problem: FireFox doesn't have a developing environment for extensions.
  We searched the Internet for solutions because a decent developing environment
  meant the difference between a slow code reviewing (with no debugging) to a full
  functioning environment. The main problem was that due many developing
  environments support JavaScript and even Xul, none could do it using the full
  FireFox and Gecko functionality, thus they were unusable to debug and develop our

  Failed solutions:
        Fully working developing environment for FireFox extension – there is none!
        Extensions to existing developing environment –
                      1. Eclipse plug in – OrangeVult XulEclipse – the plug in could
                         not open an extension developing project and debug feature
                         were enabled.
                      2. Eclipse plug in –Xul– Though an extension project is available
                         on the plug in, and even a simple Xul "Designer" the plug
                         didn't supply any other functionality and didn't work well with
                         JavaScript files.
                      3. JavaScript Add-on for Visual Studio – could not run or debug
                         JavaScript but have a helpful IntelliSense feature.
        FireFox Message console - the console allowed us to implement a simple TK
         logger that helped us to debug the application more rapidly.
        FireFox extensions –
                     1. Extension Developer – by most the most effective tool for
                           Developing extension, holds a JavaScript minimal debugger, a
                           JavaScript Injector to inject JavaScript on a running FireFox
                           and a Chrome reload button.
                     2. MozRepl – Though had a very promising description, the
                           extension didn't work properly.
                     3. SQLite Manager – helped connect to the FireFox database.
                     4. Firebug – Helps check Xul logic and Pages preview.
                     5. Venkman JavaScript Debugger – a debugger with breakpoints
                           capability and a good file explorer, the problem was that
                           JavaScript could not be edited in it.

  Solution: The selected solution was a mix of all the failed solutions:
     o JavaScript Development –visual studio that enabled IntelliSense and minimal
         Class explorer.
     o Xul Development – XulEclipse – enabled a simple designer and helpful code
     o Debugging – using FireFox with a small help of the JavaScript debugger
         extension and FireFox message console.

5. Modifying FEBE Code to TK, GUID:
  The Problem: Since every extension need a unique GUID (the identifier of the
  extension), we needed a new GUID to replace the FEBE GUID.
  Normally The GUID is defined in the install.rdf file, but because FEBE support
  automatic updates and is registered in the FireFox add-on library it holds many more
  hard coded GUID references.
Solution: The solution required a thorough code review of all GUID related methods (but
no documentation meant that all the code went a code review).
            FireFox extension Structure Appendix
Short description: An Install Manifest is the file that is being used by Add-on Manager-
enabled XUL application to determine the information needed to install the extension. It
contains metadata identifying the extension, providing information about who created it,
where more information can be found about it, the versions of the related applications it
is compatible with, how it should be updated, and so on. The format of the Install
Manifest is RDF/XML.

      Extension ID - "{AB130631-9CB3-422e-BA8F-C82F0202D8BB}".
      Extension name: “tk”
      Extension description: “ Backup your Firefox data”.
      Creators: “Amir Tepper & Oren Kalinsky”
      Options URL: chrome://tk/content/settings/tkOptions.xul
      Target application: the extesnion is compatible with firefox versions: 3.0 -> 3.5

Extension Chrome Package:
Short description: chrome package is the way Firefox extensions are able to add
features to the browser. The extensions are small packages of XUL files, JavaScript, style
sheets and images packed together into a single jar file. This file can be created by using
a ZIP utility. When the user downloads it, it will be installed onto the user's machine. It
will hook into the browser using a XUL specific feature called an overlay which allows the
XUL from the extension and the XUL in the browser to combine together.

Short description: "The Gecko runtime maintains a service known as the chrome
registry that provides mappings from chrome package names to the physical location of
chrome packages on disk.
This chrome registry is configurable and persistent, and thus a user can install different
chrome providers, and select a preferred skin and locale. This is accomplished through
xpinstall and the extension manager.

In order to inform the chrome registry of the available chrome, a text manifest is used:
this manifest is "chrome.manifest" in the root of an extension."( Mozilla developer center,

Contains: the manifest hold the following mapping:
   Browser overlay:
       overlay chrome://browser/content/browser.xul chrome://tk/content/tkOverlay.xul
      Navigator overlay:
       overlay chrome://navigator/content/navigator.xul
      Chrome content:
       content tk chrome/content/ contentaccessiable=yes
      Skin path:
       skin tk classic/1.0 chrome/skin
      Style path:
       style   chrome://browser/content/browser.xul          chrome://tk/skin/tk.css
      Locale, en-US (english US locale package):
        locale tk en-US chrome/locale/en-US/

Main XUL overlays:

       o tkOverlay: tkOverlay overlays the browser and navigator and add the main
           GUI to FireFox. The overlay contains the definition for all the toolbars, menus
           and all visible GUI in the main browser window. tkOverlay acts as the
           extension starting point and connects all the JavaScript logic to the different
           menus and controls.
       o tkOptions: tkOptions contains the main options in the TK window control,
          which allows all the different TK definitions for each user. The control
          include the following sub menus:
                 TK : contains all the DHT connection module definitions:
                        Bootstrap: the bootstrap of the DHT (for further information
                         please read the (2.3)DHT module summary), NONE represent
                         that a new DHT will be created at the user computer.
                        Username: the user's ID as registered in the DHT.
                        Password: the user's password to the DHT (ensures security
                         and profile theft), the password is saved in the Firefox
                         password manager.
                 Directory: holds the directory to which the extension will save all
                  profile backup locally, must be determined prior to backups.
                 Schedule: allows different time related backup such as Daily &
                  Weekly, and backup reminders alert.
                 Debug: various debug functions (for further information please read
                  the (2.3) 3.5 Debug support summary).
                 Options: different backup options, such as saving full profiles and
                  checking which part of the profile the user wish to save.
                 Info: general info about the extension.

   Main Logic JavaScript:

       o Tk.js: holds the core backup functions, init, load and close methods of the
          extension and connects between all the different layers of the extension.
                 Backup methods: each method work with the same basic algorithm but
                  deals with different FireFox services to get the item (bookmarks,
                  extensions, History).
                     o Algorithm:
                                Check if the item should be backed up, if not return
                                 Retrieve the relevant item's data from the FireFox
                                 Store the data as a file in the backup directory.
                                 Send the file to the DHT.
                  Restore methods: each method works with the same basic algorithm
                   but deals with different FireFox services to restore the item
                   (bookmarks, extensions, History) back to FireFox.
                       o Algorithm:
                                 Check if the item should be restored up, if not return
                                 Get the file from the DHT into the backup Directory.
                                 Get the data from the file.
                                 Retrieve the relevant item's data from the file into
                                  FireFox database.

       o tkCommon.js: contains all of TK common methods, such as Error Message,
           Help Message, TK version, TK sound handling, all the global variables that
           are used by all JavaScript files, initialize preferences .It Also contains simple
           generic method to deal with preferences.

       o tkDHT.js: controls all the connection between the DHT and the extension,
           implement all the methods that were determined at the DHT connection
           interface class.

           tkfileIO.js: handle all file communication with DHT

Skin and style:
Short description: " A skin is a set of style sheets, images and behaviors that are applied
to a XUL file. By applying a different skin, you can change the look of a window without
changing its functionality. Firefox provides a skin by default, and you may download
others. The XUL for any skins is the same, however the style sheets and images used are

For a simple personalized look to a Firefox window, you can easily change the style
sheets associated with it. Larger changes can be done by creating an entirely new skin.
Firefox has a Theme Manager for changing the default skin. (Although the underlying
code for Mozilla calls them skins and the user interface calls them themes, they're both
referring to the same thing).

A skin is described using CSS, allowing you to define the colors, borders and images
used to draw elements. The file classic.jar contains the skin definitions. The global
directory within this archive contains the main style definitions for how to display the
various XUL elements. By changing these files, you can change the look of the XUL

"( Mozilla developer center,
content: the skin contains the TK css file (that define the different controls styling and
modes) and all the images resources of the extension, a few examples:

       help_icons.png : TK help icons.

       tk_pref.PNG : holds the TK menu icons.

       tk6-100x100.png : the TK extension icon.

 Localization: DTD – multi language support:
Short description: " Many applications are built such that translating the interface into
a different language is as simple as possible. Usually, a table of strings is created for
each language. Instead of hard-coding text directly into an application, each piece of text
is only a reference into the string table. XML provides entities which can be used for a
similar purpose.
Entities are declared in Document Type Definition (DTD) files. Files of this type are
normally used to declare the syntax and semantics of a particular XML file, but they also
let you declare entities. In the Mozilla chrome system.

If you look in the chrome directory, you should see an archive for your language. (en-
US.jar is the default for English.) You might have locale files in multiple languages, for
example, US English (en-US) and French (fr). Inside these archives, you will find the
files that hold the localized text for each window. The structure of the archives is very
similar to the directory structure used for skins."( based on Mozilla developer center,

Content: TK at its early stage only support en-US translation, due the entire infrastructure
is built in order to support different languages and dialects.
Each language holds 4 different localization files:
       o Tk.dtd : contains all the Xul related entities.
       o contains all the TK user messages.
       o contains all the TK help messages.
       o contains the welcome page localization strings.

Shared By: