Mashups Software Composition for the Web
Dr. Cesare Pautasso
IBM Zurich Research Lab
c.pautasso@ieee.org http://www.pautasso.info/
17.4.7
Wikimapia.org
Today’s Lecture
• • • • • Why Study Computer Science? Mashups in Context What is a Mashup? Building Mashups Discussion and Limitations
Why Study Computer Science?
Why Computer Science?
Swiss Job Market Statistics (1997-2005) Every year: 7% more IT Jobs
Computer Science Exciting?
10,000,000,000 1,000,000,000 100,000,000 10,000,000 1,000,000 100,000 Number 10,000 of 1,000 Transistors 100 10 1 1960
R = 0.9733
2
1970
1980
1990
2000
2010
• Exponential growth in raw hardware performance (Your computer is obsolete)
Computer Science Exciting?
1,000,000,000 100,000,000 10,000,000 1,000,000 100,000 10,000 Number 1,000 of Hosts 100 10 1 1975 1980 1985 1990 1995 2000 2005 2010
• Exponential growth in Internet network size (16.9% of the world population is online – March 2007)
Why Study Informatics Now?
1000000 100000 10000 1000 100 10 1 1000000 800000 600000 400000 200000 0
• With exponential growth, the most interesting things happen at the end…
What about the Software?
Intel giveth, Microsoft taketh away Microsoft MS-DOS (1985) “Nobody will ever need more than 640KB of RAM” Microsoft Windows Vista (2007) Minimum System Requirement: 1’048’576KB of RAM
Software Most Exciting
• Without software, nothing works today!
The Beauty of Software
• Software-intensive systems are perhaps the most intellectually complex artifacts created by humans • Software is invisible, but:
– “The best software is simple, elegant, and full of drama as manifest in the cunning patterns that form its structure and command its behavior” (*)
(*) Grady Booch, The Promise, The Limits, and The Beauty of Software, Turing Lecture 2007.
Why is Software interesting?
• Software is a special kind of artifact
– Doubling the size of your program does not make it twice as strong
• A software product can be shipped at 0 cost
– Why deploying upgrades is so expensive then?
• Most people will only buy used software
– It is difficult to sell brand new software
• Software does not wear out when using it
– Instead, it changes
Software Evolution
• Software used to come together with hardware
– People didn’t know how to sell it independently
• Nowadays Software is usually licensed as a reusable component
– People should pay for every new installation
• In the future Software will be more and more delivered as a service
– People will be charged as they use the software without having to install it
Mashups are made out of Web services
Software Composition Mashups Software Architecture Web Services
Software Composition Mashups Software Architecture Web Services
Layered Architectures
Client Presentation Layer Server Application Layer Database Resource Manager Layer
Defining the Layers
User Interface Presentation
•
Application Logic
Business Rules
•
Resource Manager Business Objects
Client
Client
•
Server
Business Processes Persistent storage
Database
Client is any user or program that wants to perform an operation over the system. Clients interact with the system through a presentation layer The application logic determines what the system actually does. It takes care of enforcing the business rules and establish the business processes. The application logic can take many forms: programs, constraints, business processes, etc. The resource manager deals with the organization (storage, indexing, and retrieval) of the data necessary to support the application logic. This is typically a database but it can also be a text retrieval system or any other data management system providing querying capabilities and persistence.
Why we need Layers?
1-2 years Clients and external interfaces
Presentation Layer
2-5 years Business logic at the Server
Application Layer
Resource Manager
10 years Layer Database Management
Layers for Composition
Integration at the presentation layer (screen scraping) is always possible, but the least recommended. Mashups typically work at the UI and API level It is also possible to bypass the application and directly access its database (Data Integration)
UI Application DB
Well designed systems have an API that we can call directly (EAI)
Software Composition Mashups Software Architecture Web Services
Is this a Web Service?
Web Sites (1992)
Web Browser HTML HTTP Web Server
WS-* Web Services (2000)
SOAP Client XML (HTTP) WSDL Server
RESTful Web Services (2007)
PO-XML JSON RSS WADL Web Server
Web Browser
HTTP
Mashups use these lightweight Web protocols to access and compose together remote software components and data sources
Web Services APIs
Software Composition Mashups Software Architecture Web Services
Software Composition in Context
• • • • Databases Networking Software Engineering Programming Languages How to build applications from scratch
• Middleware • Software Composition
How to integrate two or more existing applications
Aspects of Composition
1. Component Model 2. Composition Technique 3. Composition Language
Uwe Assman, Invasive Software Composition, Springer 2003
1970s Composition (UNIX)
1. Component Model
– UNIX Commands (Filters)
2. Composition Technique
– Pipes (Stdin, Stdout, Stderr)
3. Composition Language
• Shell Scripts • Makefiles
cat $1 | tr -cs A-Za-z ' | \n' tr A-Z a-z | sort | uniq -c | sort -rn | sed 10q
Solution: return the 10 most frequently used words in a file
2000s Composition (Web Services)
1. Component Model
– Web Services (WSDL Interface)
2. Composition Technique
– SOAP/HTTP Messages
3. Composition Language
– BPMN, BPEL, UML, … JOpera, WebML, XL, … Java, C#, …
BPMN = Business Process Modeling Notation BPEL = Business Process Execution Language XL = XML Language
Mashups and Composition
1. Component Model
– Web Services
2. Composition Technique
– RPC over HTTP (with SOAP, PO-XML, JSON, RSS/ATOM)
3. Composition Language
– Javascript, PHP, Ruby, JOpera…
RPC = Remote Procedure Call PO-XML = Plain Old XML JSON = JavaScript Object Notation
Software Composition Mashups Software Architecture Web Services
What is a Mashup?
• Web Application composing existing Web services and Web data sources Ease of software reuse inspires innovation through original mix and assembly Revolution in Web development – anyone can do it!
•
•
•
“Mashup” originates from musical domain
WheaterBonk.com
Mashup Properties
• Aggregate content from more than one source
– – – – Public Web Service APIs Screen Scraping (scrAPIs) of existing Web sites Additional local data owned by Mashup provider User-provided information (shared wiki or “private”)
• • •
Lightweight programming effort
– HTML with some JavaScript is enough to get started
Interactive Web application
– Can also provide its own API
Ad-hoc composition
– Not concerned with long term stability and robustness
Long-tail of Mashup Development
Commercial Off The Shelf Software • Millions of Generic Users • Long-term, Expensive Development • Good Quality (Reliability, Scalability) • Professional Software Engineering Mashups • Few Specialized Users • Short Lifespan, Cheap Development • Good Enough Quality • End-User Development (à la Excel)
One-size-fits all
Number of Users
Personalized (Ad-Hoc)
Application Requirements
Mashup Logical Architecture
Web Browser Web Web Server Service APIs
AJAX
L XM / TP ON T H /JS P TT H
SOAP HTTP RSS
Mashup Local Database
HTTP/RSS/ATOM
Client
Databases Web Data Feeds
Client-Based Mashup
Web Browser Mashup
(HTML JavaScript)
HT
T HT
G JPE P/
Web Service APIs Web Service APIs Web Server
HTTP/JPEG
TP /XM
L/J S
ON
Problem: JavaScript Same-Origin Security Policy
Server-Based Mashup
Web Browser Mashup
(HTML JavaScript Flash)
HTTP XML/ JSON
Web Server Mashup
(PHP, JSP, ASP.NET Ruby, JOpera…)
P OA S
Web Service API Web Service API Web Data Feeds
HTTP/XML
RS S/A T
OM
Data Formats for Mashups
• XML
– PO-XML – SOAP (WS-*) – RSS, ATOM
• Meta-language (Standard textual syntax) • Many tools available: XML Schema, DOM, SAX, XPath, XSLT, XQuery • Everyone can parse it (not necessarily understand it) • Slow and Verbose
• JavaScript Object Notation (JSON) • Serialization of JavaScript data structures (also textual syntax) • Wire format introduced for AJAX Web applications • Supported in most languages • Lightweight alternative to XML for structured data
API Building Mashups User Methaphor The Glue
Mashup – Do it yourself
Pick 2 or more Web services APIs Study their functionality + data formats Test API access (registration?) Select a Mashup Metaphor for the User Design the Mashup (Client- vs. Server-Based) 6. Build it using your favourite languages
– Client/Browser: HTML/JavaScript/Flash – Server: PHP/JSP/Ruby/JOpera…
1. 2. 3. 4. 5.
Mashup Toolkit
• Mashup Tools
– Yahoo Pipes – MapBuilder (already obsolete!) – GVisit
• Widget Libraries
– Google Maps/Yahoo Maps… – dojo – Google Web Toolkit
• Firefox Extensions
– Web Developer/Java Script Debugger
Mashup User Methaphors
• Map
– Not only Google Maps
Example: immo.search.ch
• Calendar
– Collection of Events
• Data Stream • Wiki
– Shared Document Space
Example: diggdot.us
– Aggregated over multiple source streams
• Portal
– Juxtaposed Web Widgets
Example: netvibes.com
Doggdot.us
FlashEarth.com
NetVibes.com
Immo.Search.ch
Legal Issues Mashups Discussion Technical Limitations Business Model
Technical Limitations
• JavaScript: Same-Origin Sandbox
– Cannot mix everything in the browser/client
• Brittleness
– Due to Screen-Scraping – In the long term Web APIs change format (without telling the Mashup)
• Loosely Coupled?
– The mashup will not run if any of the backend API providers are down
Legal Issues
• Even if reusing software has become much easier technically, lawyers still play an important role!
– Always check the terms of use of an API
• Fair-Use
– Do we have the “Right to Remix”?
• Privacy
– Would you entrust a mashup with the password to your email?
Business Model
• Mashup by service provider
– What is the added value of a mashup? – How to share revenue with the original service/data provider? – Is it sustainable to support the mashup with Ads?
• Mashup by end-user
– Tooling is missing to make this possible – Opportunity for tool-makers?
Summary and Conclusion
Summary
• Mashups are a new kind of Web Application composing existing software services (APIs) and data sources found on the Web • Two views over Mashups:
– Revolutionary way of building lightweight composite Web applications on the fly – Free R&D for the Web services APIs providers (which will end up inglobating the mashup functionality eventually)
The Physics of Mashups
• Mashups have only now become possible because the physics is right • Client CPUs are now powerful enough to run a significant amount of JavaScript inside browsers • The networks have now become fast enough to deliver both the software and large amounts of data to the client
Adam Bosworth, Physics, Speed and Psychology: What Works and What Doesn'in Software, and Why, January 2007 t
Conclusion
• Software is rarely built from scratch • Software Systems do not live in isolation • Learn how to reuse software by composing, assembling, or mashing it up together in different ways
Mashups Software Composition for the Web
Dr. Cesare Pautasso
IBM Zurich Research Lab
c.pautasso@ieee.org http://www.pautasso.info/
17.4.7