Get documents about "pace
Document Sample
PACE Yourself: Power Aware Collaborative Execution on
Mobile Phones
Eric Jung Yichuan Wang Iuri Prilepov
Department of Computer Department of Computer Department of Computer
Science Science Science
University of California, Davis University of California, Davis University of California, Davis
eajung@ucdavis.edu yicwang@ucdavis.edu iprilepov@ucdavis.edu
Frank Maker Xin Liu Venkatesh Akella
Department of Electrical and Department of Computer Department of Electrical and
Computer Engineering Science Computer Engineering
University of California, Davis University of California, Davis University of California, Davis
flmaker@ucdavis.edu liu@cs.ucdavis.edu akella@ucdavis.edu
ABSTRACT be garnered from other phones or from the cloud (or a
In this paper, we propose a framework for power aware collabo-
local server).
rative execution (PACE) for mobile phones. Mobile phones have
Though, over the past decade, collaborative execu-
scarce resources, such as battery power and bandwidth. In the
tion has been studied in research literature, there was
context of mobile phones, collaborative execution is a way to pool
no compelling reason to put it to practice. The main
resources together to give the abstraction to the end user as if the
reason being, the incentives to the various stakeholders
application is running on the local phone. The pool of resources such as the users and network operators were not com-
could be garnered from other phones or from the cloud (or a
pelling. However, two significant developments in the
local server). For collaborative execution to be practical it is im-
past two years, warrants a reexamination of the poten-
portant that the helper phone is protected from jeopardizing its
tial of collaborative execution and consider the imple-
battery life for its own usage until its recharging time. To ad-
mentation issues.
dress this issue, we develop a Markov Decision Process (MDP)
First, with the advent of smart phones with GPS
framework that enables collaborative execution while protecting
and cameras (such as the iPhone and Android-based
the interests of the helper phone and taking into account the user phones), there has been an explosion in location-aware
call profiles and preferences such as when they expect to recharge
or context-aware applications. This is placing an enor-
their phones. Both centralized and decentralized policies are de-
mous burden on the network infrastructure of carriers.
veloped and validated through numerical results using real mobile
The recent woes of ATT’s 3G network is a good testi-
usage traces. We implement the proposed policies on a network mony to this problem 1 . So, from a network operator’s
of HTC G1 phones running the Android 1.5 operating system.
perspective there is a clear incentive to support col-
laborative execution to off-load the cellular data traffic
to another network such as WiFi or WiMax - basically
1. INTRODUCTION using other mobile users as mobile relay to improve net-
Mobile phones are resource constrained embedded de- work coverage or bandwidth. This is a much faster and
vices - with the scare resources being compute capabil- less expensive mechanism to improve service quality,
ity, energy (battery), memory and network bandwidth. which complements infrastructure improvement. The
Over the past decade several researchers [22, 16, 3, 17, network operator can provide free minutes/data for the
25] have proposed techniques to overcome these limi- helper phones.
tations via techniques called remote execution or more The second development is the explosive growth in
generally collaborative execution. For example, in [22, social networking. User groups that are based on trust
18] remote execution is proposed as a way to off-load and friendship can be leveraged to provide the incentive
computation from a mobile device to a server and in [3, to the users to help each other [10]. For example, one
23, 15] devices collaboratively share each others network could imagine a Facebook group (just like a carpooling
bandwidth. In the context of mobile phones, collobora- group) where users sign up to help each other. Incen-
tive execution is a way to pool resources together to give tives for sharing could take many forms including the
the abstraction to the end user as if the application is 1
http://blogs.wsj.com/digits/2009/12/09/
running on the local phone. The pool of resources could att-to-new-york-and-san-francisco-were-working-on-it/
1
following: being part of a collaborative effort where the 4. We have a working system (demo) with multiple
outcome is beneficial to all the group members, such as HTC G1 cellphones running Android 1.5 that im-
searching for the location of a restaurant or playing a plements the MDP framework
multiplayer video game. As a social activity where you The rest of the paper is organized as follows. We start
help your friends to maintain good will. Social stand- with a detailed problem definition and the formulation
ing could be increased by tracking how much a user of the solution using Markov decision processes. We
shares and recognizing their contributions. More con- provide a simplified framework in the paper and leave
crete incentives could also be explored such as a credit the details to the Appendix. Next we describe how
based model [3] which ensures no user takes advantage we obtained the different parameters used in the model
of other users or even monetary incentives which would followed by a detailed presentation of the results and
pay individuals or companies for sharing their resources. their discussion. The last section describes the imple-
So, in this paper we reconsider collaborative execu- mentation issues. We conclude with related work and
tion with the practical implementation issues in mind. directions for future work.
Specifically, we propose a systematic framework called
PACE which allows a set of users collaboratively share 2. DESIGN CHALLENGES
resources while balancing the costs and benefits. We
To implement collaborative execution, in addition to
provide a mathematical optimization framework based
the incentive issue addressed in Section 1, there are
on Markov decision processes (MDP) to study the dif-
other technical challenges: 1) means to locate poten-
ferent trade-offs and the costs and rewards to the users,
tial collaborators; 2) protection for helper phones; 3)
the helpers and the network operators. We address the
decision of whether to request help and which phone to
technical issues such as centralized versus decentralized
request help (if multiple choices are available).
policies, peer discovery and the actual deployment on
Next, we use the collaboration downloading as the
the Android platform based mobile devices.
major example to illustrate the concept of PACE. As
In the process we address two new issues that have
illustrated in Figure 1, phone 1 (named the requester
not been considered by previous researchers in this area.
phone) has a download request from its user. It has
First deals with the protection of the helper phone and
two potential “helper” phones (2 and 3) in the vicinity.
the second deals with the taking the user profile into
As their names imply, the “requester” may choose to
the optimization framework. Protection of helper phone
serve its own user or request to offload its download
implies that the policy should not seriously deplete the
to neighboring helper phones, while the helper phones
resources of the helper such that it jeopardizes the abil-
may decide to accept or reject these requests. A phone
ity of the helper to make phone calls. So, the trade-offs
does not always request for help because such a request
between helping another user and its impact on its own
may be rejected and thus incur waste of energy and
requirements have to be explicitly modeled and consid-
extra delay. In addition, we assume that the incentive
ered in the decision process. User profiles are important
is setup such that a phone does not always request for
because mobile phones are personal, so their usage pat-
help. We assume that the helper phones are willing to
tern varies significantly from one user to another - so
aide the requester under the right incentive. A helper
a one-size-fits-all approach to decision making will not
phone may be in a better location, have better network
work. We consider the incorporation of real call profiles
coverage, higher bandwidth, or more abundant battery
and data download profiles of the users and their bat-
compared to the requester phone.
tery charging preferences into the model. This makes
the proposed approach more practical and useful.
In summary, this paper makes four main contribu-
tions.
1. We formulate collaborative execution as an opti-
mization problem using Markov decision processes
and develop heuristics to solve the problem effi-
ciently.
2. We explicitly model the costs and benefits of the
helper devices in the decision framework. Figure 1: Illustration Example of Collaborative
Downloading.
3. We model user preferences and the context of the
user (such as signal strength) in the framework For users to collaborate they must have a means of
to make the decision making more practical and finding each other, the enabling technology can be ei-
realistic. ther network-assisted and network-oblivious. When a
2
service provider is involved, it can easily provide a list 3. PACE POLICIES
of neighboring users, notify users, and facilitate con- We present how PACE works using the collabora-
nection establishment (e.g., when to turn on WiFi in- tion downloading example as discussed in Section 2.
terface, what is the IP address, etc). In a network- A PACE policy decides when and how a requester
oblivious case, collaboration could be achieved using phone initiates requests and how a helper phone decides
auto service discovery protocols such as Bonjour and whether to assist or not.
UPnP [4, 1]. Both decentralized and centralized meth- In this section, we present three PACE policies:
ods can be created as frameworks for application use Policy I: a fully centralized optimal policy with al-
and easily integrated during the software development truistic helpers; Policy II: a centralized policy with
process. Furthermore, these frameworks can mine the autonomous helpers; and Policy III: a decentralized
user profile to make smart decisions about tradeoffs be- energy-threshold policy. The objective of Policy I is
tween sharing resources and the penalty for allowing to maximize the collective utility of all phones. In this
them to be shared. In our demo, we allow distributed case, the helper phones may sacrifice their own perfor-
neighbor discovery, and use a simple server to collect in- mance, thus named altruistic. The objective of Policies
formation and to make decisions, as discussed in detail II and III is to improve requester phone performance
in Section 6. with built-in protection for the helper phones. The
For collaborative execution to be successful, in ad- centralized policies correspond to a network-aware col-
dition to incentive, a critical component is to protect laboration scheme where service providers administrate
helper’s own performance. For example, when helper collaboration and provide incentives to collaborate to
phone 2 assists phone 1 to download a file, it consumes the end-users, while the decentralized heuristic policy
its own energy, and thus later in the day, it might not relates to the network-oblivious case where users decide
have enough battery power for its own phone usage. If to collaborate on their own.
this is severe, it will significantly limit the potential of For the ease of discussion, we make the following as-
collaborative execution because users will be much less sumptions all of which can be easily generalized. We as-
willing to help at their own performance detriment, even sume that if a phone call arrives, it preempts download
with incentives. The challenge in addressing this issue is actions (i.e. a helper phone cannot serve, a requester
that a user’s future usage is random and dynamic, which phone cannot ask for help downloading). In addition,
cannot be predicted precisely. In addition, users have a call arrival at phone 1 clears any pending download.
very different usage profile, and thus one size does not We assume Phone 1 will always serve the download if
fit all. Therefore, decision has to be adaptive, adjusting it cannot find a helper. We assume all downloads can
to usage pattern, time, energy, channel condition, etc. be handled within one unit of time.
Therefore, we propose PACE (Power-Aware Collab-
orative Execution) to address the protection and selec- 3.1 Markov Decision Process
tion issues. The essence of PACE is to pace the usage
of resource (in this example energy) according to user The two centralized policies are based on Markov
profile so that battery does not run out prematurely, i.e., Decision Process (MDP), a well-known mathematical
before its charging time. framework which models decision-making in a system
We make the following assumptions. First, each where the outcomes are partly random and partially
phone has a regular recharging time. Therefore, if the dependent on user decision. In our context, future us-
battery lasts until the charging time, we consider the age of phones and the corresponding energy need are
user well protected. For presentation simplicity, we as- random; i.e., a phone may not precisely decide its bat-
sume the phone charging time is 9PM for all users. A tery need later in the day. The key of the MDP frame-
more dynamic and realistic model, such as proposed work is to balance between obtaining immediate reward
in [21], can be adopted here. We assume that phone 1 and maximizing possible future reward. For example,
is the potential requester and phones 2 and 3 are helpers a helper phone may obtain reward (e.g., free minutes)
as the example to present our PACE policies. In prac- for helping a requester download a file. However, this
tice, all phones can be requesters as well as helpers, consumes its energy and may result in battery depletion
depending on their instantaneous conditions. For the and thus loss of future phone calls.
helper phones, we assume that the main priority is to Intuitively, at a given time, a phone with a relatively
preserve their phone calls, and use voice profile to count large amount of remaining battery may choose to serve
for their future usage. This can be generalized to other a request even at high energy cost because it poses little
usage. For example, for heavy data users, the energy risk to affecting its future reward, while the same phone
threshold can be easily adjusted to account for energy with low battery may choose to request for help. The
usage due to data service as well as future phone call MDP framework quantifies these competing interests
times. and provides optimal decisions.
The general idea of MDP is that for any system state
3
s, an optimal action a can be found which probabilis- general, is the extreme case of collaborative execution,
tically maximizes total reward over the lifetime of the and thus serves as a benchmark.
system. In this paper, we consider discrete MDP, where Recall that phone 1 is the requester phone and phones
the system can be defined by a finite number of states. 2 and 3 are the helper phones. First, the following states
An MDP can be defined by four objects: are the input to the MDP formulation.
S - State Space: The state space S contains all possible • t: time since last recharge period
states of the system, which is assumed to be known • T : time when next recharge period will occur
to the decision-maker. Each state s ∈ S can further
be defined by a tuple of state variables which specify • Ei : remaining energy of phone i
different aspects of the system. For example, a phone
• Si : signal strength of phone i
state s could specify values for the phone’s remaining
battery energy level, network connectivity, time since • Q: download request at phone 1 (0: no request, 1:
last recharge, and service request state, and S would be request)
the set of every possible combination of these variables.
• Ci : call status of phone i (0: no call, 1: call)
A - Action Space: This is the set of all possible actions First, all state variables are discretized into a finite
that can be taken by the decision-maker. An optimal number of levels. We assume that the system is “slot-
action a ∈ A will be associated with each state s ∈ ted“, i.e. that the time t since last recharge is repre-
S, that maximizes the reward over the duration of the sented by a finite set of intervals (each time slot is 1
system’s operation. The action space for the requester minute in our numerical results).
may be which user to request or to serve itself, while The action space is A = {0, 1, 2, 3}, which specifies
the helper action may be to reject, or accept and serve the phone to download the file. We note that a = 0
the request. indicates an idle action when no download is pending,
1 indicates that the requester phone has chosen to self-
Ra - Reward: This is the immediate reward for taking serve, and 2 (3) means that helper phone 2 (3) will help
action a in a state s. For example, successfully down- download.
loading a file gives us an immediate reward, and so does Depending on the state and action, the following re-
making a phone call. wards and (energy) costs may occur:
Pa - Transition Probability: The transition probability • Rc : Reward for one unit of call time
is the probability that an action a taken in state s at • Rd : Reward for successful download
time t will result in a system state s′ in time t + 1.
Therefore, the evolution of the system state from one • ec : Energy cost of one unit of call time
time to the next is random but also depends on the
• eo : Energy overhead for establishing collaboration
action taken. We also note that events that are outside
the user’s control, such as call arrival, are also transition • ed (S): Download energy cost with signal strength
probabilities. S
In our system, the objective is to maximize user ex-
perience collectively until charging time, which means • ef : Energy for helper to forward download
that the design of rewards must reflect user experience • eg : Energy for requester to receive download
and usage priorities. In this discussion, we assume that
the user’s main priority is phone calls, but priority can Let
be adjusted easily by adjusting reward functions. E1 S1 C1
E= E2 ,S = S2 ,C = C2 .
E3 S3 C3
3.2 Policy I: The Centralized Altruistic The goal of the MDP formulation is to find an opti-
Scheme mal action a for all possible combinations of these vari-
Policy 1 is a fully centralized optimal policy with al- ables, including both immediate reward and future re-
truistic helpers. In other words, when a download re- ward. In particular, V (t, E, S, C, Q) is the maximum
quest is generated by the user, an “oracle” with perfect expected reward that can be obtained including both
state information of all phones makes decisions on which immediate reward and future reward. An optimal pol-
phone should perform this action in order to maximize icy can be calculated numerically through backward in-
the aggregate reward for the group. In this case, the duction. More details on the analysis can be found in
helper phones may sacrifice their own performance, thus Appendix 10.1. A major challenge of using MDP is the
named altruistic. This policy, although not practice in curse of dimensionality; i.e., the number of states and
4
computational complexity increase exponentially with later time, or incentive from the service provider for im-
the number of system parameters. We address this is- proving performance for the requester, and/or offload-
sue in Appendix 10.2. ing data traffic onto a different network, as discussed
previously.
3.3 Policy II: PACE with Helper Protection We can now define the helper decision policy. For
In the previous scheme, we assumed a centralized or- helper i, if it is not on a call, it will help if and only if
acle attempted to maximize the aggregate utility of all Rh +E[Yi (t+1, Ei −(eo +ed +ef ))]Rc ≥ E[Yi (t+1, Ei )]Rc
i
phones, which is the extreme case of collaborative exe- (3)
cution. In this case, it is possible that a helper phone Note the left and right hand sides represent the helper
may significantly jeopardize its own usage due to bat- phone’s reward for collaborating and rejecting, respec-
tery depletion. tively. Clearly, the more abundant the energy of the
In Policy 2, we explicitly address the concern that helper with respect to its future usage, the more likely
helper phones want to protect their own communication the helper phone will help. In addition, the larger the
need. In particular, we allow a helper phone not to help value of Rh , the more the incentive to help, and the less
even if it is in the best position to download to protect the protection of the helper itself. Thus, the value of
its own future performance. Rh can be adjusted to achieve the desired balance.
We note that we do not take into account possible
3.3.1 Helper Decision Policy
future request arrivals. This is a more realistic assump-
To determine an autonomous helper decision policy, tion, since a fully modeled data usage profile is gener-
we must first assess the helper’s potential to serve ex- ally not be available. We also note that this scheme is of
ternal requests. As stated before, we assume a user’s practical computational complexity, such that it could
top priority is to prevent missed call or call disruptions be calculated locally on the phone.
due to battery depletion. With its call profile, a helper
phone can obtain an estimate of its future needs. Based 3.3.2 Requester Phone Decision Policy
on this estimate, we can produce a decision policy that The policy of the requester phone can be calculated
reserves energy for its own future need. similar to that derived in section 3.2. The only dif-
First, we define Ci,t ∈ {0, 1} as a random variable ference is that the new policy here essentially removes
indicating the call status at time t. Therefore: phones that refuse to serve due to their own energy
need. The rest is the same.
P r[Ci,t = 1] = P r[Phone i on a call at time t] = pc (t).
i
(1) 3.4 Policy III: PACE with Energy-Threshold
We have
We note that the helper decision in Policy II (Eq. 3) is
T −1
a threshold policy, depending on time and user profile.
Xi,t = Ci,j , (2) Motivated by this observation, we present the following
j=t
simple heuristic policy for both requester and helper
with Xi,t ≤ T − t, which is a r.v. representing the total phones.
phone call minutes remaining. The probability mass Recall that in Section 3.3.1, we have determined the
function for Xi,t can be easily calculated based on user pmf (probability mass function) for Xi,t , a random vari-
call profile. We define able denoting the number of minutes that arrive after a
time t. The idea of an energy-threshold is that a phone
Ei
Yi (t, Ei ) = min ⌊ ⌋, Xi,t , can specify how conservative it wants to be in protecting
ec its future communications.
where ⌊ Eci ⌋ is the maximum number of minutes that can We define the energy-threshold as follows. First we
e
be served because of the remaining energy constraint. denote a call threshold probability function for phone i
In other words, Yi (t, Ei ) denotes the number of call min- as pi (t) ∈ [0, 1], e.g, pi (t) = 0.95. This value specifies
th th
utes that can be served on phone i. the desired probability that the phone can serve all calls
We must also consider a helper’s incentive to collab- that arrive after time t. For each time t, we wish to find
orate. Although a phone is primarily interested in pro- a service threshold Γi (t) which is the smallest value that
tecting its call time, if provided with enough incentive satisfies
Γi (t)
by the service provider, it may be willing to give up
some performance. To this end, we define a new pa- pXi,t (j) ≥ pi (t),
th (4)
rameter, the help reward Rh , as the reward that the j=1
helper phone receives for serving a download request. where pi (t) is a user’s protection level, so that the
th
This reward could represent local incentive such as an probability that the total remaining phone call is be-
agreement for service from the requester phone at a low Γi (t) minutes is larger than pi (t).
th
5
From this value of Γi (t), we can obtain a power bud- in a sixteen hour time period, the number of times a
i
get function Eth (t) call occurs within that minute is counted to form a his-
i togram, as shown in Figure 2. These histograms are
Eth (t) = Γi (t) · ec . (5)
normalized over the number of days sampled for each
Then we define the energy cushion of user i as user (61 days), and these numbers are used to estimate
i pi (t), the call arrival rate for user i at time t. While
c
Ei − Eth (t),
users 1 and 2 have somewhat heavy call profiles, user
which is intuitively the “extra” energy available at 3’s is significantly lighter. In all simulations, we use
phone i. Phone i will help if the user 1 call profile for the requester phone, and user
i 2 and 3 as the helper phones. Intuitively, we expect
Ei − Eth (t) ≥ eo + ed (Si ) + ef .
that user 3 will be more willing to collaborate since it
In other words, phone i is willing to help if it has enough expects fewer calls than the other helper phone.
energy for its own use after serving the download. In the case of data usage characteristics, the diffi-
The decision policy for phone 1 at phone 1 has the culty lies in the low number of samples of data usage
three steps. First, phone 1 self-serve if even for heavy data users. For example, the maximum
1 number of 3G accesses in a single day from our heavi-
E1 − ed (S1 ) ≥ Eth (t) or eo + eg > ed (S1 ).
est data user (roughly 700MB/month) was only 5, and
In other words, it has enough energy or if it is more ranged from <200KB to >150MB. This reflects the wide
energy efficient to download itself. If not, step 2 is to range of uses of smartphones: people perform low-traffic
find a helper phone with the largest energy cushion. If functions like web search, and high-traffic functions like
not, then phone 1 has to self-serve. streaming video. In this paper, we impose a simple but
high-traffic model to evaluate performance. We leave
3.5 Complexity and Scalability the characterization of data usage patterns and their
The three proposed policies vary significantly in effects on PACE to future work.
terms of complexity. Policy I is a full-size MDP formu-
lation, which easily involves a large number of states. 4.2 Signal Strength Profile
It is complexity is high. With a desktop PC, we can For signal strength, the modeling difficulty lies in
barely manage to find optimal policies for three users. infrastructure and sufficient measurement time. Long
We have presented mechanisms in Appendix 10.2 to re- measurement durations (on the order of weeks) must
duce complexity. Policy I is meant for benchmark, not be obtained to study mobility patterns, which in turn
real implementation. dictate the average signal strength and service avail-
Policy II has much lower complexity. The decision al- abilities to a particular user at different times of day.
gorithm on the helper side can be easily calculated nu- Here, we use average signal strength numbers obtained
merically, even on a phone. The decision on the helper from a study done in Houston, Texas [20]. In the work,
side is more complex, but still doable. measurements of signal strength parameters, such as
Policy III is a heuristic policy with very low complex- the number of cell towers and Wi-Fi access points ob-
ity, which can be easily computed on the phone for both served, were taken every five minutes over the course of
helper and requester. four weeks using proprietary software.
We note that none of these policies change frequently
because user profiles do not rapidly change. Therefore, 4.3 Power Profiling
it is possible to precompute in a server and just down- We used the Android Developer Phone (HTC G1
load the decision table on the phone. [14]) to obtain all energy costs used in the previous sec-
Last, locality is a very important factor here. The tion. The Android platform [12] is open source and uses
number of devices in the vicinity of a mobile phone is a developer-friendly Java environment over a modified
usually limited. Therefore, we do not need to consider Linux kernel. This allows us to modify the operation
the scalability issue as the number of participating users of the phone as we see fit, and to turn off unnecessary
increases. processes during testing to remove their influence on the
power measurements taken. This is essential to obtain
4. SYSTEM MODELS AND MEASURE- accurate energy constants to test our framework.
MENTS
4.3.1 Device Setup
4.1 Call Profiling We measured the power consumption of certain
We obtained the call records of four users for a two events specific to the Android G1 mobile phone by using
month period. Three of them were selected for the rel- a DC power supply (Agilent E3644A [2]) to power the
ative difference in their call profiles. For each minute phone instead of the phone’s battery. We connected
6
User 1 Call Profile User 2 Call Profile User 3 Call Profile
0.4 0.4 0.4
0.35 0.35 0.35
0.3 0.3 0.3
Normalized Call Count
Normalized Call Count
Normalized Call Count
0.25 0.25 0.25
0.2 0.2 0.2
0.15 0.15 0.15
0.1 0.1 0.1
0.05 0.05 0.05
0 0 0
7:30 9:30 11:30 13:30 15:30 17:30 19:30 21:30 23:30 7:30 9:30 11:30 13:30 15:30 17:30 19:30 21:30 23:30 7:30 9:30 11:30 13:30 15:30 17:30 19:30 21:30 23:30
Time Time Time
(a) User 1 (b) User 2 (c) User 3
Figure 2: Call Profiles for 3 Users
the power supply to the phone and a computer using Constant Energy (Joules)
the IEEE-488A General Purpose Interface Bus (GPIB). ec 34.92
eoi 6.47
The power measurements were then sampled using a ef (800KB) 5.87
Python script using the PyVISA package[5]. The script eg (800KB) 5.33
also sets maximum voltage and current levels to avoid edl (Low) 107.6779
edl (Med) (estimated) 81.9
damage to the phone during experimentation. Each edl (High) 56.1278
measurement was performed for a user-specified dura- edl (WiFi) 5.33
tion of time, with a frequency of approximately 12 mea-
surements per second.
Table 1: Energy costs for defined constants
4.3.2 Downloading over EDGE and Wi-Fi
We obtain the following energy measurements: the Figures 3 (a) and (b) display power consumption
download energies over the EDGE and Wi-Fi net- graphs for some sample downloads over the different sig-
works; i.e., the energy to download ed , and the energy nal strengths. To obtain a starting and stopping point
for a helper/requester to forward/receive a file ef , eg for the download, a smoothing filter was run over the
(through WiFi interface). All other energy-related pa- numbers, and a threshold power of 450 mW was then
rameters were obtained from [7] using the same setup, applied to determine the download durations. From the
discussed earlier. figures, we can see that the mean power for the down-
To test download energies over EDGE, we obtained load at either strength is roughly 1.5 W. However, we
a prepaid SIM card and wrote a simple script that see that the download speeds are quite different, with
maintained the minimal phone functionality required average download times of 40.42s and 70.44s. Several
to download the same file several times. The file was downloads were taken and their energies averaged to
800KB, which is a sufficient size to obtain good average obtain the energy numbers. We then performed the
power numbers over the course of the download. We same type of measurement over Wi-Fi, this time also
then found locations for different signal strengths, and forwarding the file to another phone. In Figure 3 (c),
obtained the power readings from the download events. we see four cycles of download and file forwarding over
As described in [20], average coverage percentage of Wi-Fi. We see that the both the power and time are
EDGE was taken over three signal strength divisions, - significantly smaller in this case than in EDGE, at 1.3
112 to -94 dBM (one or two bars), -94 to -81 dBM (three W and 5s per download, which is to be expected.
or four bars), and > -81 (4 bars). A fourth division, <- For the overhead cost of collaborative download, we
112 dBM, occurred less than 1% of the time in that pa- use the Wi-Fi startup cost obtained in [7]. Because the
per, so we do not attempt to measure it here. We mimic actual control messages are sent over SMS, our mea-
this measurement scheme by obtaining power measure- surement apparatus is unable to obtain power readings
ments at locations where one to two bars, and strictly because of the small size of the transmission. Therefore,
four bars occur. However, due to the size constraints we assume that their energy is negligible, and use the
of the measurement apparatus, we were unable to find startup energy as the overhead energy eo . The energy
a suitable location for the second signal strength divi- costs are listed in Table 1.
sion (three or four bars), so we infer this power number
linearly from the other two points, which should have 5. PERFORMANCE EVALUATION
lower and higher download energies than the missing In this section, we evaluate the performance of the
point. proposed PACE policies. The performance metrics can
7
Figure 3: Power measurements for different signal strengths
Power for 800KB Download at Low Signal Strength (1−2 Bars) Power for 800KB Downloads at High Signal Strength (4 Bars) Power for 800KB Download and Forward (WiFi)
3500 3500 2000
1800
3000 3000
1600
2500 2500 1400
1200
Power (mW)
Power (mW)
Power (mW)
2000 2000
1000
1500 1500
800
1000 1000 600
400
500 500
200
0 0 0
0 10 20 30 40 50 60 70 20 40 60 80 100 120 140 160 180 200 40 50 60 70 80 90 100
time (s) time (s) time (s)
(a) EDGE Low Signal (1-2 Bars) (b) EDGE High Signal (4 Bars) (c) Wifi
Ave Data (MB) vs pd
be broadly separated into two categories: benefits to
350
Policy 1
the requester and the service provider, and costs to the Policy 2
300
requesters. Real voice usage traces and measured power 90% Threshold
95% Threshold
consumptions are used in the numerical study.
Data Received (MB)
250
99% Threshold
No PACE
200
5.1 Simulation Setup 150
The requester phone (phone 1) follows user 1’s voice 100
profile shown in Figure 2, and helper phones 2 and 3 50
follows that of users 2 and 3. We artificially generate
0
download request for phone 1; in particular, at each 0 0.1 0.2 0.3 0.4 0.5 0.6
p
time unit, with probability pd , user 1 has a download d
request from its user. Phone 1 may decide to serve itself
or request help following the policies, and helper phones Figure 4: Throughput vs. pd
decide to assist or not following their corresponding
policies.
to the requester, the amount of traffic potentially of-
All energy costs used are the measurement data in
floaded onto other networks, call times, and battery life
Table 1. The download energy is scaled up because we
times of all phones involved. Each data point represents
assume 1MB download size in the simulation. We as-
the average performance of the 30-day simulations. In
sume requester and helper communicate through WiFi.
all figures, the x-axis is the phone 1 download rate (pd ),
The overhead cost eo is the startup energy for Wi-Fi.
which varies from 0.05 (light) to 0.6 (heavy).
The time horizon is T = 960 with each slot represent-
ing a minute. The maximum energy for all phones is 5.2 Requester Benefits
400 units, where each unit represents 11J. This corre-
sponds to roughly 133 minutes of talk time. In the sim-
ulations, we assume that the requester phone only has 5.2.1 Throughput
EDGE network connectivity as modeled in [20], while In Figure 4, we present the throughput (total data
the helper phones both have Wi-Fi connections. We downloaded) under the different policies as a function
note that phone 1 may not have access to the WiFi of download rates. First, we note that all PACE poli-
network because of authentication issues. In addition, cies perform better than acting alone (SOLO), where
phone 1 may be outside the range of the WiFi network. the difference is significant when the download rate is
We compare the following five cases: SOLO case high. This is a result of the low data rate of the EDGE
where no collaborative execution is considered, Pol- connection. We also see that Policy I results in the
icy I (centralized policy with altruistic helpers) with highest throughput for the user. This is to be expected
Rd /Rc = 1, Policy II (centralized policy with helper since all phones work towards aggregate performance,
protection) with Rh /Rc = .25, and Policy III with and as specified, serving a download obtains the same
pth = 0.9, 0.95 for all users, respectively We simulate reward as voice.
the performance of these policies. For fair compari- At lower download arrival rates, Policy I and II have
son, all results are generated from the same voice traf- matching throughput. In this case, the potential risk
fic, download arrival and signal strength evolution pro- for a helper phone to miss future phone calls is small,
cesses. so the helpers are more likely to accept requests. As
The metrics used for performance are the throughput the download rates increase, more requests are received,
8
and phones begin to reject requests to pace their energy
use for their expected local needs, leading to a diver- Figure 6: Average downloads served vs. pd
Ave Data Served vs. pd
gence with Policy I.
Policy III can be understood the same way. In the 200 Phone 3
policies simulated, the requester phone sets its threshold
Data Served (MB)
level at 90% while the helper phones set their levels 150
at 90, 95, and 99%, respectively. In addition, user 3
also adds a cushion of 60 energy units to its threshold 100
(corresponding to twenty minutes talk time) due to its Phone 2
sparse profile. This allows the phone to budget for its 50
bursty voice profile. As expected, the higher energy
thresholds result in less throughput for the the requester 0
0 0.1 0.2 0.3 0.4 0.5 0.6
due to helper protection. p
d
5.2.2 Requester Call Time and Battery Life
Figure 5 shows the corresponding average phone call 5.3 Helper Performance and Costs
times and battery life times for the same simulations. The most important metric for a helper is its own
The call time is the average number of minutes that performance while collaborating. In particular, we con-
phone 1 could serve with and without PACE, given the sider both the battery depletion time and the talk time.
download arrival model imposed upon it.
The trends in relative performance exhibited in the 5.3.1 Who Serves More?
throughput are also present in both of these metrics.
Figure 6 shows the average number of downloads
We note that both drop gradually over higher download
served by each user. Clearly, phone 3 dominates down-
probabilities, which is natural because more downloads
load service for all cases. This is due to its much lighter
are initiated earlier, leading to earlier battery depletion.
voice traffic profile from Figure 2, which leads to much
lower thresholds for Policy 2 and the explicit threshold
Table 2: Total Data Served vs. Data Served policy, even given the added energy cushion. For Policy
through EDGE with pd = .2 1, its weaker profile also indicates that it is unlikely to
obtain any call reward for its energy, and thus allocates
its resources to serve phone 1 requests.
Policy EDGE Traffic (MB) Total Data (MB)
Policy 1 4.9070 170.8307 5.3.2 Helper Call Time and Battery Life
Policy 2 4.6523 170.3703
Threshold 90% 19.6203 150.1857 We now look at call time and battery life time for
Threshold 95% 19.6630 149.268 the helper phones, shown in Figures 7 and 8. However,
Threshold 99% 19.7653 148.2193 since we are now evaluating the cost to the helpers, we
No PACE 41.5740 41.5740 would actually like to see how well each policy preserves
call time and battery life for the helper phones. We
note that the battery life in the absence of PACE is the
entire time horizon T , since none of the call samples
5.2.3 Network Benefit used have more call time than battery life according to
One benefit of collaborative execution is to exploit our models.
better network connectivity. Depending on the network We first notice that the threshold policies are the
condition of the helpers, traffic from overly congested strongest in terms of maintaining both call times and
(slower) networks can be offloaded onto other networks. battery life times that are close to the original case.
In our simulation, cellular traffic is offloaded to WiFi This is due to their explicit purpose of reserving energy
networks. In other scenarios, there is also the poten- for expected future needs. We note that phone 2’s bat-
tial of offloading traffic onto other cellular networks un- tery life and call time drop gradually for increasing pd ,
der some type of service agreement. In Table 2, we while phone 3’s call time and battery time begin to sta-
compare the average number of downloads served on bilize. This is due to the extra energy cushion added
the requester’s data network when PACE is being used to phone 3’s threshold level. We note that our energy
to collaborate and when the phone acts alone. Aside cushion budgets for twenty minutes, while its average
from the throughput improvements exhibited in Figure call time is roughly fifteen minutes. This means on an
4, traffic on the requester’s network is also significantly average day, this cushion is enough to serve its entire
reduced in all cases. This provides incentive for cellular voice requirement. Policy 1 loses the most minutes,
service providers to support collaborative execution. which is due to its consideration of global reward, while
9
Figure 5: Call Time and Battery Life of Requester
Ave Calltime of Requester vs. pd Ave Lifetime of Requester vs. pd
60
Policy 1 Policy 1
1200
Policy 2 Policy 2
50 90% Threshold
90% Threshold
1000 95% Threshold
Calltime (minutes) 95% Threshold
Lifetime (minutes)
40 99% Threshold 99% Threshold
Solo 800 No PACE
30
600
20 400
10 200
0 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0 0.1 0.2 0.3 0.4 0.5 0.6
p P
d d
(a) Call time (b) Battery life
Figure 7: Average call times for helper phones
Phone 2 Ave Calltime vs. pd Ave Calltime of Phone 3 vs. pd
85 16
15
14
Calltime (minutes)
Calltime (minutes)
13
12 Policy 1
80 Policy 1
11 Policy 2
Policy 2
90% Threshold
90% Threshold 10 95% Threshold
95% Threshold
9 99% Threshold
99% Threshold
No PACE
No PACE
8
75 7
0 0.1 0.2 0.3 0.4 0.5 0.6 0 0.1 0.2 0.3 0.4 0.5 0.6
p p
d d
(a) Call time for phone 2 (b) Call time for phone 3
Figure 8: Average battery life for helper phones
Ave Lifetime Phone 2 vs. pd Ave Lifetime Phone 3 vs. pd
960 950
900
940
Lifetime (minutes)
Lifetime (minutes)
850
920
Policy 1 800
900 Policy 1
Policy 2 Policy 2
90% Threshold 750
90% Threshold
880 95% Threshold 95% Threshold
99% Threshold 700
99% Threshold
860
650
0 0.1 0.2 0.3 0.4 0.5 0.6 0 0.1 0.2 0.3 0.4 0.5 0.6
p p
d d
(a) Battery life for phone 2 (b) Battery life for phone 3
10
Policy 2 again exhibits the pacing performance, where connection is established the file is transmitted to the
the download probability begins to test the threshold of requester and stored on the external storage card.
the helpers. To make sure that a requester’s choice of the helper
phone is optimal, every peer periodically uploads its
6. SYSTEM IMPLEMENTATION battery level to the server.
We have implemented a collaborative download ap- Phone-to-Phone Request/Serve Protocol:.
plication using three Android Developer Phones under Every time the user wants to download a file, the
the Android 1.6 Mobile Operating System. The appli- phone can either perform the download itself or request
cation is a basic form of the collaborative downloading its assigned helper phone to do it, where the decision
application, which allows phone to request other phones is made based on the power-threshold decision, which
to download a file for it. The implementation handles can easily generated on the local phone during recharge
initial peer discovery to find helper phones, the updat- time. SMS is used perform the overhead communica-
ing of a database that stores phone status information, tion for setting up a collaborative download. Theoret-
and the request/serve protocol between phones, includ- ically, this can be locally using Bluetooth, but because
ing the use of a decision table of the decentralized form of bugs in the Android implementation, SMS is used as
described in the formulation. Since the implementa- a placeholder.
tion’s main purpose is as proof-of-concept, we make two To make a request, the phone sends an SMS message
assumptions: the phones will remain in the same loca- to its assigned helper, which, in turn, either accepts
tion indefinitely, and only one collaborative download the request or rejects it. Since the helper phone is as-
occurs at a given time. The first assumption allows sumed to have the highest chance of accepting such a
us to run the peer discovery process only once, while request, in the case that the request is rejected, the re-
the second allows us to avoid some synchronization is- quester phone has to perform the download by itself.
sues. The three main implementation modules are peer When the helper phone accepts the request, it noti-
discovery, server status reporting, and phone-to-phone fies the requester with an SMS message and downloads
serve/request protocol based on decision tables. the file. After download completes, another SMS mes-
sage is sent to signal that the peer-to-peer file transmis-
Peer Discovery. sion should begin. When this SMS message is received,
The list of the phones that are in close proximity with the requester starts a Wi-Fi ad-hoc network which the
each other is obtained using the Bluetooth discovery helper phone then associates with, and the forwarding
process. As mentioned above, the bluetooth API is not of the downloaded file is completed.
supported by the majority of the Android OS versions, Android 1.6 generally does not allow access to the
therefore, we resorted to the open source experimental functionality of Wi-Fi. Therefore, the P2P connec-
bluetooth API [11]. In order to discover each other, tion between the phones is implemented by modify-
both peers alternate between acting as a server or as a ing a Wi-Fi tethering application from another open
client, while broadcasting short messages, called heart- source project [13], whose authors have compiled bina-
beats. The list is then uploaded to the server along ries that allow access to the ad-hoc capabilities of the
with the phone number of each peer. Each peer is then Wi-Fi adapter. For our purposes, we have modified the
assigned a helper phone which is guaranteed to have ad-hoc network discovery protocol implementation and
the highest probability of accepting a help request. Be- added the ability to create TCP/IP connections for re-
cause the selection process with several phones may be liable file transmission over ad-hoc Wi-Fi.
computationally intensive, this selection occurs in the
server.
The assumption that only one request can be made 7. RELATED WORK
during a full file transmission cycle allows us to main- The roots of the proposed work can be traced to re-
tain one ad-hoc P2P connection at a time. This makes mote execution [22] which advocates executing a task
the discovery process simpler by statically assigning the on a remote server and shipping back the result to the
helper to act as a server and the requester to act as a client to save energy. More recent incarnations of re-
client. When one of the phones receives a heartbeat, mote execution are clone cloud [8] that advocates creat-
it stops broadcasting, saves the IP address specified by ing a clone of the smartphone in cloud so that resource-
the heartbeat and sends an acknowledge message us- intensive tasks can be run in the cloud, and [18], where a
ing this address as the destination. When the other proxy-based approach is used for transparently support-
party receives the acknowledgement, it stops broad- ing active web content (i.e., flash) on mobile devices.
casting as well and either starts listening for incoming Bandwidth aggregation, originally proposed by [15]
TCP/IP connections or tries to establish one depend- and extended by [3] can also be viewed as a form of col-
ing on whether it is the server or the client. When the laborative execution. The latter uses HTTP-level strip-
11
ing to get rid of proxies proposed in [15] and unifies the the Android operating system.
available bandwidth and energy using a monetary cost.
It uses credit and accounting systems to encourage col- 8. CONCLUSIONS AND FUTURE WORK
laboration. Similarly, [24] proposes an architecture to
According to the October 2009 Morgan Stanley re-
share internet among residential users (e.g., DSL). A
port2 ,, ATT’s mobile data traffic increased by 50X in
credit system to manage incentive and the activity on
the past three years and there are over 35 million WiFi
each link is monitored and predicted and the informa-
hotspots. From a network operators perspective the
tion is used for scheduling. Another form of bandwidth
cost per bit is 70% cheaper to send data over WiFi
aggregation is in reported in [16] where the server par-
compared to 3G monthly plans. This, coupled with the
titions the web page and send them to multiple collab-
explosive growth in social networking, which could be
orative mobile users.
leveraged to overcome the trust and incentive issues,
Collaboration between multiple radios is discussed
paves the way for collaborative execution to be consid-
in [19] where WiFi and bluetooth radios are switched
ered seriously. We believe for this to happen, we need
dynamically to save battery power. Context-aware bat-
a systematic design methodology which is based on a
tery optimization [21] advocates giving priority to voice
mathematical framework that takes various costs and
application over others (similar to what we do in this
benefits into account and can be implemented on off-
paper) and they try to predict the next charging time
the-shelf smart phones. The proposed work is a first
based on call arrival and call duration and battery
step in this direction. In future we propose to extend
draining function. In [7] a technique is proposed to for-
the models to include network conditions (e.g., network
malize this using Markov decision processes, however,
congestion) and mobility patterns of the users and de-
it is limited to applications running on a single mobile
velop better models for users profiles and contexts. We
phone. It does not deal with collaborative execution.
also plan to deploy the current system in the field to col-
Applications of collaborative execution that leverage
lect real usage statistics, which could be used to refine
social networks can be found in [10] and [9]. In the
the policies.
former mobile users update their location, and pub-
lish context-aware multimedia blogs to a central server
which hosts a website. Internet users browser the web- 9. REFERENCES
site, issue location-sensitive queries which could be an- [1] UPnP forum. http://www.upnp.org/.
swered by the blogs. If no blogs match the query, the [2] Agilent Technologies. Agilent Technologies
server can send the query to the mobile users in or near E364xASingle Output DC Power Supplies,
the location. The mobile users may answer the query 1999-2008.
using blogs. The latter ([9]) proposes a model to col- [3] G. Ananthanarayanan, V. N. Padmanabhan,
laboratively produce content. It includes how to route L. Ravindranath, and C. A. Thekkath.
message among users which may have different roles COMBINE: leveraging the power of wireless peers
to interact with the message (e.g., ask question, answer through collaborative downloading. In ACM
it, provide suggestions, provide examination and valida- MobiSys, pages 286–298, New York, NY, USA,
tion, etc). FAQ generation system to illustrate the idea. 2007. ACM.
Finally, [17] proposes a tool to enable multiple users to [4] Apple. Networking – Bonjour.
search together, synchronously or asynchronously. The http://developer.apple.com/networking/
key is to provide awareness (know what other members bonjour/index.html.
are doing), division of labor, and persistence (up-to- [5] T. Bronger. Python gpib etc. support with pyvisa,
date information about this search session). Collabora- controlling gpib, rs232, and usb instruments.
tive location is another application that is related to this http://pyvisa.sourceforge.net/.
work. In [25], location is estimated for devices without [6] L.-W. Chan, J.-R. Chiang, Y.-C. Chen, C. nan
GPS using motion-tracking sensors based on the rough Ke, J. jen Hsu, and H.-H. Chu. Collaborative
ideas of where the neighbors are located and in [6] nodes localization: Enhancing wifi-based position
have their own estimate, then collaboratively compute estimation with neighborhood links in clusters. In
a more accurate location. IEEE Pervasive, 2006.
The main contribution of the proposed paper is to [7] T. L. Cheung, K. Okamoto, F. Maker, X. Liu,
formalize the underlying decision process in collabora- and V. Akella. Markov decision process (MDP)
tive/remote execution and cast it as an optimization framework for optimizing software on mobile
problem to systematically maximize certain reward for phones. In International Conference on Embedded
the different players such as the users, network opera- Software, Grenoble, France, Oct. 2009.
tors etc. We also develop a methodology to implement
2
this effectively on a network of mobile phones running http://www.morganstanley.com/institutional/
techresearch/internet ad trends102009.html
12
[8] B.-G. Chun and P. Maniatis. Augmented smart SIGMOBILE Mob. Comput. Commun. Rev.,
phone applications through clone cloud execution. 2(1):19–26, 1998.
In HotOS XII, 2009. [23] P. Sharma, S.-J. Lee, J. Brassil, and K. G. Shin.
[9] J. Davitz, J. Yu, S. Basu, D. Gutelius, and Aggregating bandwidth for multihomed mobile
A. Harris. iLink: search and routing in social collaborative communities. In IEEE Transactions
networks. In ACM KDD, pages 931–940, New on Mobile Computing, 2007.
York, NY, USA, 2007. ACM. [24] N. Thompson and H. Luo. PERM: A
[10] S. Gaonkar, J. Li, R. R. Choudhury, L. Cox, and collaborative system for residential internet
A. Schmidt. Micro-blog: sharing and querying access. In UIUC Technical Report, 2009.
content through mobile phones and social [25] P. Zhang and M. Martonosi. LOCALE:
participation. In ACM MobiSys, pages 174–186, Collaborative localization estimation for sparse
New York, NY, USA, 2008. ACM. mobile sensor networks. In IEEE IPSN, pages
[11] gerdavax. Experimental unofficial bluetooth API 195–206, Washington, DC, USA, 2008. IEEE
for Android. Computer Society.
[12] Google. Android, official website, April 2009.
http://www.android.com/. 10. APPENDIX
[13] harald.mue ulfada bbuxton. Wireless tether for
root users. 10.1 Policy I: MDP formulation
[14] HTC. Htc g1 overview.
http://www.htc.com/www/product/g1/overview.html. To derive V (t, E, S, Q), we first define the value func-
[15] K.-H. Kim and K. G. Shin. Improving TCP tions corresponding to all pertinent events in the system
performance over wireless networks with (e.g. call arrivals and help request actions). In all of
collaborative multi-homed mobile hosts. In ACM these equations, it is assumed that the remaining ener-
MobiSys, pages 107–120, New York, NY, USA, gies of the phones are sufficient to commit the chosen
2005. ACM. action; if an action is chosen for which the phones have
[16] T. Maekawa, T. Hara, and S. Nishio. A insufficient energy, the immediate reward obtained and
collaborative web browsing system for multiple the remaining energy after that action is zero. We de-
mobile users. In IEEE PerCom, pages 22–35, note the states for signal and queue in the next time
Washington, DC, USA, 2006. IEEE Computer slot as S ′ and Q′ . We consider the following actions
Society. and events.
[17] M. R. Morris and E. Horvitz. Searchtogether: an 1) If phone 1 has no call arrival and no download
interface for collaborative web search. In ACM request pending, its value function vi (·) satisfies
UIST, pages 3–12, New York, NY, USA, 2007. vi (t, E, S, 0) = EC2 ,C3 ,S ′ ,Q′ [(C2 + C3 )Rc
ACM. (6)
[18] A. Moshchuk, S. D. Gribble, and H. M. Levy. +V t + 1, E − e(t), S ′ , Q′ ,
Flashproxy: transparently enabling rich web where e(t) is the power consumption vector of all phones
content via remote execution. In ACM MobiSys, at time t. We have e(t) = [0, C2 · ec , C3 · ec ]′ , because
pages 81–93, New York, NY, USA, 2008. ACM. phone 1 has no call and phones 2 and 3 both can po-
[19] T. Pering, Y. Agarwal, R. Gupta, and R. Want. tentially receive phone calls. Note that (C2 + C3 )Rc is
Coolspots: reducing the power consumption of the reward for the phone calls of phones 2 and 3, and
wireless mobile devices with multiple radio V t + 1, E − e(t), S ′ , 0 is the future reward.
interfaces. In ACM MobiSys, pages 220–232, New 2) When a call arrival occurs at phone 1, there is no
York, NY, USA, 2006. ACM. action but to make the phone call. The system evolves
[20] A. Rahmati and L. Zhong. Context-for-wireless: as:
context-sensitive energy-efficient wireless data
transfer. In ACM MobiSys, pages 165–178, New vc (t, E, S, Q) = EC2 ,C3 ,S ′ [(C2 + C3 + 1)Rc
(7)
York, NY, USA, 2007. ACM. +V t + 1, E − e(t), S ′ , 0 ,
[21] N. Ravi, J. Scott, L. Han, and L. Iftode.
Context-aware battery management for mobile where e(t) = [ec , C2 · ec , C3 · ec ]′ . We note that, be-
phones. In IEEE PERCOM, pages 224–233, cause of the call preemption assumption, the download
Washington, DC, USA, 2008. IEEE Computer request is dropped.
Society. 3) If phone 1 has a download request and chooses to
[22] A. Rudenko, P. Reiher, G. J. Popek, and G. H. serve it itself, we have
Kuenning. Saving portable computer battery d
v1 (t, E, S, Q)
power through remote process execution. (8)
= ES ′ ,Q′ V t + 1, E − e(t), S ′ , Q′ + Rd
13
where e(t) = [ed (S1 ), C2 · ec , C3 · ec ]′ . > 2 · 1011 entries, which means storing the value ta-
4) If phone 2 helps phone 1 download, we have ble size is greater than 800GB; clearly this is too large.
d While this MDP table is not meant to be actually im-
v2 (t, E, S, Q) plemented on the phones, even calculating it on a high-
(9)
= ES ′ ,Q′ V t + 1, E − e(t), S ′ , Q′ + Rd (Q) resource server may be too expensive.
In this case, we perform table reductions by reducing
where e(t) = [eo + eg , eo + ed (S2 ) + ef , C3 · ec ]. Similarly,
d the granularity of the Ei variables to a reduced energy
we can obtain v3 . ′
state Ei , where the factor of reduction is an integral
In the altruistic scheme, our objective is to maximize
value γ. Therefore, any value Ei in the system evolution
the aggregate reward of all users. Therefore, we can now ′
maps to the action for Ei , where
define the value function V (·) in two parts for Q = 0
and Q > 0. When Q = 0, there is no decision to be ′ Ei
Ei = ⌈ ⌉.
made, and the system evolves as γ
V (t, E, S, 0) = pc1 (t)vc (t, E, S, 0)+ This requires some modification of the MDP value func-
(10) tion itself. If γ is larger than any of the energy costs,
(1 − pc1 (t))vi (t, E, S, 0),
any action performed may or may not lead in a drop in
When Q > 0, the decision is to select a phone to down- Ei if ⌈ Ei ⌉ = ⌈ Ei −ecost ⌉.
′
γ γ
load that maximizes the overall reward: To work around this, we calculate the probability
V (t, E, S, Q) = pc1 (t)vc (t, E, S, 0) that a certain energy cost will lead to a drop in en-
ergy. We assume that whenever we are in a reduced
+ (1 − pc (t)) max (vi (t, E, S, Q))
i
d
′
energy level Ei , we have uniform probability that the
i∈{1,2,3}
i∈{1,2,3} actual energy state Ei is an integral value between the
′ ′
+ pc
3 (1 − pc (t)) max (vi (t, E, S, Q) + Rc )
i
d interval [γ(Ei − 1), γ(Ei )]. For an energy cost e, we
i∈{1,2} derive the probability pe that the corresponding action
i∈{1,2}
leads to a drop in the reduced energy level,
+ pc
2
d
(1 − pc (t)) max (vi (t, E, S, Q) + Rc )
i
i∈{1,3} e
i∈{1,3} pe = .
γ
d
+ pc · pc (1 − pc (t))(v1 (t, E, S, Q) + 2Rc )
2 3 1
Using this probability, we can now modify the table
(11)
calculations (7)-(9) with an extra probability parame-
To determine the optimal policy from the above equa- ter, and calculate the optimal actions corresponding to
′
tion, we also state boundary conditions for the value these reduced Ei states.
function. The boundary conditions specify values for We note that, due to these approximations, the MDP
when no action can be performed, either because the table is no longer optimal for the original system. How-
charging time is reached or all phones are dead. ever, simulations with two user cases (not presented in
this paper) show negligible differences between the orig-
V (T, E, S, Q) = fr (E); (12) inal policy and state-reduced policy. We also note that
other state variables (such as t) can also be reduced in
V (t, 0, S, Q) = 0 (13) a similar manner.
where fr (E) defines the reward for remaining energy at
charge time. From equations (10)-(13), backward in-
duction can be used to obtain the optimal policy. For
all numerical results presented, fr (E) = 0, but this can
be easily generalized. As stated previously, the policy
obtained here is optimal in terms of the aggregate re-
ward obtained by all phones.
10.2 Reducing the State Space
The main shortcoming of MDP is the so-called ”state
explosion” problem, where introducing new system pa-
rameters leads to exponentially increasing table sizes
and computation times. This is particularly true for the
centralized “altruistic” scheme presented above. With
three users (Phones 1-3), the MDP table can become
prohibitively expensive to calculate and store. For ex-
ample, in our simulation section, the table size contains
14
