Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out
Get this document free

Virtualized Screen: A Third Element for Cloud - Mobile Convergence

VIEWS: 16 PAGES: 9

In recent years, we have witnessed the strong growth of cloud computing, which refers to both applications delivered as services over the Internet and the infrastructure (including both hardware and software systems) in the data centers that supports those services.

More Info
									                                                                                                            Wenjun Zeng
Multimedia at Work                                                                       University of Missouri-Columbia




                        Virtualized Screen: A Third Element
                        for Cloud—Mobile Convergence
       Yan Lu,
Shipeng Li, and         I n recent years, we have witnessed the strong
                          growth of cloud computing, which refers to
                                                                            screen to refer to the display images. There are
                                                                            a number of advantages to putting screen ren-
  Huifeng Shen          both applications delivered as services over        dering in the cloud.
      Microsoft         the Internet and the infrastructure (including          First, because screen rendering is closely
  Research Asia         both hardware and software systems) in the          coupled with program execution and data stor-
                        data centers that supports those services.1 In      age, putting screen rendering in the cloud
                        general, cloud computing provides a central-        where data storage and program execution are
                        ized platform where programs can be executed        centralized actually simplifies the cloud com-
                        and data can be stored. Cloud services built on     puting architecture. Second, screen rendering,
                        the platform can offload many computing tasks       especially graphics-rich screen rendering, is
                        normally solely done on the client devices and      not a trivial task. It often requires a powerful
                        can be accessed instantly from anywhere at any      CPU and GPU on the client devices. Moving
                        time. However, until recently, the place where      screen rendering to the cloud will greatly re-
                        screen images that interface with users are ren-    duce the hardware requirement for client de-
                        dered has been mostly done locally on client        vices and thus make low-cost devices possible.
                        devices.                                            Third, even if the client devices are equipped
                           Our argument is that the screen rendering        with powerful CPU and GPU processors, off-
                        can also be moved to the cloud and the ren-         loading all or part of screen-rending task to
                        dered screen can be delivered as part of the        the cloud will spare the client processing
                        cloud services. In general, the screen represents   power to handle more effectively higher prior-
                        the whole or part of the display images. In a       ity tasks (such as the local UI) and rich interac-
                        broad sense, it also represents a collection of     tions (such as touch and gesture recognition)
                        data involved in user interfaces such as display    that require fast responses locally. Last,
                        images, audio data, mouse, keyboard, pen and        screen rendering in the cloud also brings a
                        touch inputs, and other multimodality inputs        new avenue for optimizing the overall comput-
                        and outputs. In this article, we use the term       ing experience.
                                                                                Screen virtualization or screen rendering in
                                                                            the cloud doesn’t always mean putting the en-
                          Editor’s Note                                     tire screen-rendering task in the cloud. Depend-
   Mobile and cloud computing have emerged as the new computing             ing on the actual situations—such as local
platforms and are converging into a powerful cloud—mobile comput-           processing power, bandwidth and delay of the
ing platform. This article envisions a virtualized screen as a new dimen-   network, data dependency and data traffic,
sion in such a platform to further optimize the overall computing           and display resolution—screen rendering can
experience for users. In a virtualized screen, screen rendering is done     be partially done in the cloud and partially
in the cloud, and delivered as images to the client for interactive dis-    done at the clients (that is, scalable screen virtu-
play. This enables thin-client mobile devices to enjoy many computa-        alization); collaboratively, a complete rendered
tionally intensive and graphically rich services. Technical challenges      screen is presented to the user. This is very sim-
are discussed and addressed. Two novel cloud-mobile applications,           ilar to traditional cloud computing in which
Cloud Browser and Cloud Phone, are presented to demonstrate the             we have to decide where we should put the
advantages of such a virtualized screen.                                    program execution and data storage, either
                                                           —Wenjun Zeng     remotely in the cloud or locally on the device,
                                                                            to obtain an optimized computing experience.




 4                                         
                       1070-986X/11/$26.00 c 2011 IEEE                      Published by the IEEE Computer Society
       Screen                                           Cloud      Client                                  Figure 1. The
   virtualization                  Virtual
                                 display/UI                                          Display/UI            conceptual diagram
                                                                                                           of the cloud—client
                                                                                                           computing
                                   Screen                                             Screen               architecture.




                                   Cloud                                          Local computing
                                 computing           Program         Program
                                                                                    (CPU/GPU)



                                    Data                                               Data




                                   Data                                                 Local
                                   farm                                                storage


This flexibility added by screen virtualization       computing architecture, the data and the pro-
can enable us to provide an even further opti-        gram can be stored, loaded, and run remotely
mized computing experience by balancing be-           and/or locally. To take advantage of the
tween local client devices and remote clouds          cloud, computationally intensive tasks are usu-
in terms of data storage, program execution,          ally undertaken in the cloud to generate some
and now, screen rendering.                            intermediate results (for example, HTML
   However, rendering a screen in the cloud           data), which are then delivered to the clients
also introduces obstacles for the client devices      for the creation of a locally processed display
to access the virtual screen, if it needs to main-    screen. In other words, local screen rendering
tain high-fidelity display images and responsive      is separated from the data storage and program
user interactions. Fortunately, we have already       execution, and is connected to them through
developed a number of advanced multimedia             the Internet.
and networking technologies to address these              Up until now, the virtualization of the
issues. Ultimately, we would like to define a         screen in the cloud has been an underdevel-
common cloud API for cloud computing with             oped area in cloud computing. An even less
scalable screen virtualization, with which the        addressed area is how to leverage the virtual
developers never have to care where the data          screen in the cloud and combine it with
storage, program execution, and screen render-        local rendering capabilities to give the same
ing actually occur because the cloud services for     or even better user experiences across differ-
the API will adaptively and optimally distribute      ent devices, regardless of their computing ca-
the storage, execution, and rending among the         pability, rendering capability, bandwidth,
cloud and the clients. Similar to the GUI, which      and screen resolution. Figure 1 depicts the
drove the wide grassroots adoption of personal        proposed conceptual architecture for cloud—
computers, the cloud API, which bridges the           client computing, in which the virtual screen
cloud and a wide variety of clients, will intro-      is rendered in the cloud. Behaving like the
duce new computing experiences both locally           data and the program, the screen can be pro-
and remotely. It will drive the evolution of          cessed adaptively and collaboratively between
cloud—mobile computing into a revolution.             the cloud and the clients, determined by the
                                                      client and cloud capabilities. As we men-
                                                      tioned earlier, the cloud—client architecture
                                                                                                                                 April—June 2011




Screen in the cloud                                   with screen adaptation could provide the fol-
   Rapid development of the Internet has pro-         lowing benefits:
vided opportunities for using remote comput-
ing and storage resources hosted by powerful,         Š an optimized interactive experience to end
parallel, distributed machines in a public or            users across different screens, such as laptop,
private data center. In a typical cloud-client           television, and phone;




                                                                                                                             5
Multimedia at Work


                                                                                             System architecture
                               User-input
                                                    Decrypt        Receive                       Thin-client, remote-computing systems are
                               transform
                                                                                             expected to provide high-fidelity displays and
                                Screen              Screen          Screen                   responsive interactions to end users as if they
                                capture            encoder          sender
                                                                                             were using local machines. However, the com-
                                                                                             plicated graphical interfaces and multimedia
                                                                                             applications usually present technical chal-
                      Previous screen frame
                                                                              Network        lenges to thin-client developers for achieving
                         in codec buffer                                                     efficient transmissions with relatively low-
                                                                                             bandwidth links. Figure 2 depicts the proposed
                                                                                             thin-client, remote-computing system, which
                                Screen              Screen          Screen                   decouples the application logic (remote) and
                               presenter           decoder         receiver                  the user interface (local) for clients to use re-
                                                                                             mote servers deployed as virtual machines in
                               User-input
                                                    Encrypt         Send                     the cloud. The servers and the clients commu-
                               transform
                                                                                             nicate with each other over a network through
                                                                                             an interactive screen-remoting mechanism.
                                                                                             The clients send user inputs to the remote serv-
Figure 2. The                        Š a simplified programing model for develop-            ers, and the servers return screen updates to the
interactive screen-                         ers, as if they were building local applica-     clients as a response.
remoting system.                            tions without struggling on various types            The screen-update model determines whether
                                            of real-time data communications; and            the screen can be efficiently compressed and
                                                                                             transmitted to the client. Most of the existing
                                     Š a quick model for software providers to de-           thin-client systems, such as Virtual Network
                                            ploy software services in the cloud, through     Computing (VNC)2 and Remote Desktop Pro-
                                            a mechanism of virtual machines plus screen      tocol (RDP),3 represent screen updates with
                                            remoting—for example, this model could           graphics primitives of arbitrarily-sized regions.
                                            help to quickly deploy trial software without    Such a mechanism allows the server to simply
                                            piracy concerns.                                 forward the graphics primitives to be updated
                                                                                             into the compressors and discard other stable
                                        Is it feasible to deliver real-time screen as ser-   regions directly. At the client side, the screen
                                     vices through the Internet to achieve near-local        presenter renders the received graphics primi-
                                     interactive experiences? The screen as display          tives and overlays rectangular areas of pixels
                                     images indeed contains a large volume of                in destined regions. However, the regions to
                                     data. Fortunately, its representation as image          be updated are usually small and at arbitrary
                                     sequences has a lot of redundancy that can be           positions, such as menus or edit boxes. Encod-
                                     efficiently removed. Moreover, screen refresh-          ing these small regions with arbitrary locations
                                     ment with a full-frame unit can make the la-            will result in the system suffering from com-
                                     tency of screen updating at the client side             pression efficiency degradation.
                                     relatively constant. While breakthroughs of                 In contrast to the architecture based on
                                     multimedia, hardware, and networking tech-              arbitrarily-sized regions, our thin-client system
                                     nologies are around the corner, our recent              employs a frame-based screen-representation
                                     progress on a remote-computing system with              model. This model reads all the pixels on the
                                     virtualized screen implies that screen compres-         screen from the frame buffer at once and
                                     sion and transmission can be very efficient             feeds the whole screen image into the compres-
                                     even with technologies available today.                 sors and transmitters. At the client side, the
                                                                                             screen presenter replaces the whole screen
                                     Remote computing with                                   with the newly decoded one. As shown in
IEEE MultiMedia




                                     virtualized screen                                      Figure 2, both the server and the client store
                                        Following the cloud-computing conceptual             the same reference frame, which is used to re-
                                     architecture depicted in Figure 1, we have              move the redundancies between consecutive
                                     developed a thin-client, remote-computing sys-          frames. Besides the advantage in compression,
                                     tem that leverages interactive screen-remoting          the frame-based representation model also sim-
                                     technologies.                                           plifies the system architecture without the




                  6
                                                              Previous
overhead in scheduling the updates of screen                   frame
regions. Furthermore, this frame-based repre-
sentation model could recover from errors due                                                      Skip-block
                                                                              Skip                  coding
to packet loss very quickly without retransmis-
sions through keyframe refreshing.
                                                                                                  Picture-block       Entropy
   Compared with existing remote-computing                       Difference
                                                                 Diffe
                                                                                                     coding           coding
systems, such as VNC and RDP, the proposed                       dete
                                                                 detection           Block
solution could significantly improve user expe-     Current                      classification
                                                     frame                                         Text-block
rience in terms of both the smoothness of the                                 Nonskip                coding
screen update and the response in interaction,
which are enabled through the advanced
screen-compression and transmission technol-        which is specifically designed to handle pixels        Figure 3. Block diagram
ogies introduced next.                              belonging to a background picture in a text-           of the proposed screen
                                                    on-picture region.                                     compression algorithm.

Screen compression                                      Figure 3 depicts the proposed screen-
   Screen images might include, but are not         compression algorithm. The captured screen
limited to, webpages, slides, posters, images,      image is segmented into nonoverlapping
videos, and anything showing up on your com-        16 Â 16 blocks. After comparison with the pre-
puter screen. For natural pictures, many exist-     vious frame, each block is identified as either a
ing image- and video-coding standards (such         skip or a nonskip block. Further, a classification
as, JPEG2000 and H.264/Advanced Video Cod-          is performed to separate the nonskipped blocks
ing) have demonstrated excellent coding per-        into picture blocks and text blocks, followed by
formances. However, they are inefficient for        the adaptive-coding schemes associated with
compressing screen images that in most cases        the block types. The statistical gradient histo-
contain rich text. The challenges for compress-     gram is employed for the purpose of block
ing screen images are summarized as follows:        classification. Obviously, 100 percent accurate
                                                    block classification is almost impossible be-
Š Computational complexity. The screen codec        cause of the diversity of text-on-picture regions.
  has to handle a large amount of data to           Fortunately, the proposed escape pixel-coding
  maintain high resolution and high-frame-          scheme can avoid quality loss with a marginal
  rate screen updates. Meanwhile, the codec         rate increase in case of incorrect classification.
  also has to spare most of the processing              This screen codec outperforms traditional
  resources for other regular applications.         natural image and video codecs for coding
                                                    screen images in terms of rate, quality, and
Š Compression performance. The text, graphics,      computational complexity. The encoding and
  and pictures composing a screen image             decoding processes are also GPU-friendly in im-
  each possess a unique set of characteristics      plementation. GPU acceleration with a main-
  and susceptibilities to encoding artifacts.       stream video card could easily double the
  The screen codec has to handle them effi-         frame rates of screen capturing and coding
  ciently within a single framework, and yet        over systems without GPU acceleration.
  maintain low computational cost.

   In general, a typical screen image can be seg-   Screen transmission
mented into four categories of regions: smooth,        The latency of screen transmission is proba-
text, picture, and text-on-picture. Based on our    bly the most important factor to user experi-
statistical analysis, the smooth and picture        ence in a remote-computing system. Because
regions are more suitable for coding in the         the screen images are organized as a time-series-
transform domain, while the text and text-on-       like a video, some existing video-transmission
                                                                                                                                April—June 2011




picture regions can be efficiently compressed       technologies could be leveraged here. The
in the pixel domain. To simplify the codec ar-      strong dependency between predictively
chitecture, we propose a block-based coding al-     coded frames makes the video stream sensitive
gorithm with only two coding types: picture         to transmission errors. Therefore, some buffer-
and text. The text-block coding scheme also         ing and error-control mechanisms have to be
includes an escape pixel-coding technique,          employed, which causes additional delay on




                                                                                                                            7
Multimedia at Work


Figure 4. Block diagram     Codec layer                                        Error-concealed screen reconstruction
of the proposed screen                          Screen-encoding pipeline
transmission                                                                                                     Error-concealed
                                      Screen            Flexible              Hybrid
algorithm.                                                                                                       screen decoder
                                      capture            slicing             encoder


                            Control layer                                         Congestion-distortion optimization

                                    In-pipeline congestion             Encoded-screen-                           Received-screen-
                                          controller                    buffer control                           buffer controller


                            Transport layer
                                                                                            Coded screen
                                                                   User Datagram Protocol                    User Datagram Protocol
                                   TCP-friendly rate-control
                                                                         w/automatic                               w/automatic
                                  based congestion controller
                                                                        repeat request                            repeat request
                                                                                                  ACK
                                                             Server                                                    Client


                          top of the inherent network-transmission                     help error-concealed screen reconstruction
                          delay.                                                       at the client. Moreover, the screen decoder
                             Furthermore, interactive screen remoting                  can handle out-of-order screen packets with-
                          has much more stringent requirements on la-                  out producing drifting errors.
                          tency than real-time video communications.
                          For example, the user usually expects an imme-          Š At the control layer, the congestion-
                          diate response on the local display after clicking           distortion optimization algorithm sched-
                          a button, similar to what would be expected                  ules the packets in the sending buffer.
                          when using a local machine. Achieving this per-              These packets are scheduled on the basis
                          formance requires quick round-trip message                   of their impact on network congestion
                          processing and almost instant updates between                and display distortion. Some packets may
                          the virtual screen in the cloud and the local dis-           be dropped without reencoding the frames
                          play on the device. Therefore, existing video-               followed.
                          transmission technologies may not be sufficient
                          to the interactive screen-remoting scenario.            Š At the transport layer, the User Datagram
                             Fortunately, the proposed screen codec                    Protocol transport with automatic repeat
                          doesn’t introduce strong dependency between                  request is employed to balance the trans-
                          consecutive frames, which can mitigate to a cer-             mission rate and error control. To avoid
                          tain extent the drifting errors in decoded screen            congestion in the network, TCP-friendly
                          images caused by transmission errors. More-                  rate control is used to estimate available
                          over, screen remoting presents some unique                   bandwidth.
                          features different from video-transmission sce-
                          narios. For example, it’s not always necessary
                          to update the whole screen frame all at once.           Cloud-mobility convergence
                          Rather, some out-of-order screen regions can               The rapid evolution of mobile computing
                          be presented early as they become available.            offers a wide variety of freedom to mobile
                          Therefore, it’s possible to achieve extremely-          users. Besides communication functions, a
                          low-latency transmission with the help of               mobile device could be a computing, sensor,
                          some content-aware scheduling schemes.                  control, gaming, and natural-interaction plat-
                             Figure 4 illustrates the proposed screen-            form. However, it’s difficult for a mobile device
                          transmission algorithm, which is based on the           to serve as a dominant device for all the user’s
IEEE MultiMedia




                          joint optimization of screen compression and            computing needs, mainly because of its limited
                          transmission. The screen images are processed           capabilities in terms of computing, storage,
                          in three layers:                                        display, and interaction. On the other hand,
                                                                                  cloud computing offers unlimited computing
                          Š At the codec layer, the screen images are             and storage capabilities through centralized
                             encoded with a flexible slicing structure to         data centers. The capacity of mobile devices




                  8
                         Virtual browser                                                       Mobile browser
              Ctrl Msg                                                                                            Ctrl Msg

HTML            Sample*              Sample           Ctrl Msg                              Sample             Sample*             Rendered
  files                                                                                                                            webpage
                                                             Real-Time
          Browser          Sample              Sample                               RTP              Sample            Sample
                                                              Transfer
          engine           encoder            scheduler                           network            decoder         composition
                                                              Protocol



                                                                                                                         Cache
                                                Encoded sample buffer


     Rendered screen                                                                                                     Phone screen
   image and extracted                                                                                                     location
     webpage objects




                                                                                                                           Nonupdate
                                                                                                                         screen regions




could be significantly augmented by cloud                   exemplar mobile applications: Cloud Browser             Figure 5. The system
computing:                                                  and Cloud Phone.                                        architecture of the
                                                                                                                    Cloud Browser.
Š Remote-computing terminal. A mobile device                Cloud Browser
   with remote-desktop solutions (for example,                 As most mobile phones come with prein-
   RDP and VNC) could use remote computer                   stalled Web browsers, mobile browsing is be-
   programs in the cloud, though existing com-              coming increasingly popular. However, many
   puter programs are usually designed for a rel-           mobile browsers have limited support for com-
   atively large display and difficult to adapt to          plicated HTML objects, such as full-feature
   mobile screens.                                          Flash, JavaScript, and Silverlight. Installation
                                                            of add-ons from some websites in many cases
Š Cloud-services harnessing platform. A mobile              isn’t possible. Even worse, the website owners
   device could access unlimited services in                might have to develop and maintain two ver-
   the cloud through the Internet. In an ex-                sions of the same content, one full version for
   treme case, except for the user interfaces               desktop browsers, and the other simplified
   and sensors, all program execution, data                 version for mobile browsers. In most cases,
   storage, and screen rendering could be                   the mobile version loses rich multimedia infor-
   done in the cloud. In general, by balancing              mation and results in a compromised and
   program execution, data storage, and screen              degraded browsing experience. It’s desirable to
   rendering between the cloud and the mobile               bring the desktop-like browsing experiences to
   device, we could achieve an optimized com-               mobile phones.
   puting experience adaptive to the capacities                Recently, we have built a cloud-assisted
   of the cloud and the mobile device.                      mobile browser, the Cloud Browser, which is
                                                            an extension to the proxy-based Web browser
                                                                                                                                              April—June 2011




   The virtualization of the screen in the cloud            we developed.4 Figure 5 illustrates the architec-
introduces a new avenue to further optimize                 ture of the Cloud Browser, which consists of a
the cloud—mobile computing experience and                   virtual browser deployed in the cloud and a mo-
could potentially accelerate cloud—mobile con-              bile browser installed on the device. The virtual
vergence. On the basis of the proposed interac-             browser extracts and renders webpage objects
tive screen-remoting system, we showcase two                into samples, encodes and multiplexes them




                                                                                                                                          9
Multimedia at Work


                                                                                        assemble these samples into a webpage for
                                       Cloud      Program   Phone                       the users to view to further shorten the web-
                        Virtual                                            Cellphone
                        phones                        +                                 page rendering time. While the HTML webpage
                                                    data
                                                                                        could be converted to a screen image, the
                                                      +
                                                   screen                   Browser     screen image could also be embedded into the
                                   Cloud Phone               Cloud Phone
                                                                                        HTML file as a webpage element to help real-
                       Unlimited      service                   client
                       programs                                                         time cloud-client interactive communications.
                                                                             Laptop
                                                                                        Cloud Phone
                                                                                           The smartphones as a personal computing
                                                                                        device are broadly available to mobile users.
                                                                                        In addition to a wide variety of phone brands
                                                                                        and a number of different operating systems
Figure 6. Conceptual                into a composite screen format, and then sends      in the market, the rapid development of hard-
diagram of the Cloud                the data to the client with optimized schedul-      ware and application software accelerates the
Phone.                              ing. The mobile browser receives and decodes        pace at which phone hardware is upgraded. Is
                                    these samples, and assembles them for presenta-     it possible to replace the frequent hardware
                                    tion to mobile users. The mobile browser also       upgrades with only software and services
                                    sends back control messages, such as visibility     upgrades? Recently, we have built a prototype
                                    information about the client display and user-      of a complete cloud-service-based phone solu-
                                    interaction events, to the virtual browser.         tion, the Cloud Phone, with which the phone
                                        As described previously, the composite          user can access the ever increasing programs
                                    screen format is defined as a sequence of sam-      and services available in the cloud, even on a
                                    ples, where the sample can be either a screen       low-cost device without the need to update
                                    image or a webpage object. The webpage object       phone hardware anymore.
                                    consists of object ID, type, and properties such       Figure 6 illustrates the conceptual diagram
                                    as the position in a screen. We define four types   of the Cloud Phone. On the one side, with
                                    of webpage objects: hyperlink, text input, pass-    the virtual phones deployed in the cloud with
                                    word input, and dynamic. The object samples         user-preferred mobile operating systems (for ex-
                                    are not intended to be used for direct display,     ample, Windows Phone, iPhone, and Android)
                                    but for improving user interaction experiences      and the unlimited apps or services, the Cloud
                                    and avoiding unnecessary bandwidth con-             Phone service could switch between these
                                    sumption. For example, the client can use the       phone user interfaces and programs at any
                                    object information to execute some simple in-       time and instantly. On the other side, the
                                    teraction logic to give quick responses to the      Cloud Phone client could be implemented in
                                    user. Moreover, all or part of the screen could     any form, such as a cellphone, browser, or lap-
                                    be selectively updated according to the object      top. As a complete cloud-service-based phone
                                    types and the user’s options.                       solution, the Cloud Phone could provide the
                                        The Cloud Browser is a complete end-to-end      following benefits:
                                    solution that offloads the rendering of web-
                                    pages from the client to the cloud or some
                                                                                        Š It could make a low-cost smartphone possi-
                                                                                          ble, where the phone is just a terminal
                                    proxy servers, reformats the webpages to elimi-
                                                                                          with basic communication functions and a
                                    nate incompatibility with mobile devices, and
                                                                                          screen decoder. Complicated applications
                                    maximizes the object-level interactivity while
                                                                                          become much simpler by shifting comput-
                                    keeping the full desktop browsing experience.
                                                                                          ing tasks to the cloud.
                                    In many cases, the webpage’s bits after com-
                                    pression could be even less than that of the        Š It could emulate any existing devices
                                    original HTML files. Even better, the proposed        (phones, game consoles, PDAs). It also could
IEEE MultiMedia




                                    mobile browser could significantly reduce             allow the user to enjoy the same personal
                                    page loading time by pulling only a visible por-      experience on any phone.
                                    tion of the screen image rendered in the cloud
                                    for that webpage. Moreover, because most            Š It could have complete knowledge of users’
                                    screen samples have already been rendered             interactions in the cloud for potentially pro-
                                    in the cloud, the local client could quickly          viding better services.




                  10
   There still remains some future work            evolution of cloud computing become a
required to deploy the services of Cloud           revolution.                       MM
Phone. For example, the offline experience
should be taken into account, because the net-
work might not always be available. Neverthe-      References
less, the concept of Cloud Phone inspires us       1. M. Armbrust et al., ‘‘A View of Cloud Comput-
to think about the next-generation mobile plat-       ing,’’ Comm. ACM, vol. 53, no. 4, 2010,
forms with the consideration of cloud—mobile          pp. 50-58.
convergence. A middleware layer might exist        2. T. Richardson et al., ‘‘Virtual Network Comput-
on top of the phone OS, which could be more           ing,’’ IEEE Internet Computing, vol. 2, no. 1, 1998.
suitable than the general Web user interface       3. Microsoft, ‘‘Remote Desktop Protocol,’’ 2009;
for running rich cloud applications.                  http://msdn.microsoft.com/en-us/library/
                                                      aa383015.aspx.
Conclusions                                        4. H. Shen et al., ‘‘A Proxy-Based Mobile Web
    Cloud computing is not a simple extension         Browser,’’ Proc. ACM Multimedia, ACM Press,
of Web services. Breakthroughs may come               2010, pp. 763-766.
with the introduction of a new application in-
terface model between the cloud and the cli-         Contact author Yan Lu at yanlu@microsoft.com.
ents. User interaction with the cloud through
client—cloud communications bounds the user          Contact editor Wenjun Zeng at zengw@
experience and presents many technical chal-       missouri.edu.
lenges. We have shown that advanced multime-
dia compression and networking technologies
can bridge the devices and the cloud effectively
and efficiently, and potentially could help the




                                                                                                                  April—June 2011




                                                                                                             11
                   This article was featured in




    For access to more content from the IEEE Computer Society,
                 see computingnow.computer.org.




Top articles, podcasts, and more.




                               computingnow.computer.org

								
To top