MATLAB Server Pages

MATLAB Server Pages Ali KIZIL Istanbul, Turkey alikizil@su.sabanciuniv.edu Abstract MATLAB Server Pages (MSP) is an open source technical web programming language using MATLAB in background, which supports 3-Tier Architecture (Web Tier – Business Tier and Database Tier) including Distributed Computing and Parallel Processing via remote procedure calls and web services. MSP uses JavaServer Pages technology to define custom tag libraries, called MATLAB & MATLAB RMI TLDs, to handle a rapid development environment to web developers. Underlying Java RMI support acts as a gateway for distributed computing and parallel processing and enables MSP to use a remote MATLAB just like on local computer. Expression Language Mechanism of JSP Standard Tag Library and MSP Custom Tag Libraries work together to realize database connectivity. MSP implements JavaServer Faces for defining business logics and business rules including advanced UI components. MSP contains web service description language files to create a possibility of MATLAB as a Web Service. MSP is BPEL ready and WS-I Compatible. Keywords MATLAB, Toolbox, Simulink, JavaServer Pages, JSP, JavaServer Faces, JSF, JSP Standard Tag Library, JSTL, Remote Method Invocation, RMI, Java DataBase Connectivity, JDBC, Java Web Services, MATLAB Tag Library, MATLAB RMI Tag Library, MATLAB Web Service, MATLAB Server Pages, MSP, MSP Architecture, MATLAB Architecture. Introduction MATLAB Server Pages is technical web programming language that uses MATLAB in background. MATLAB is a high-level technical computing language and interactive environment for algorithm development, data visualization, data analysis, and numerical computation. Using MATLAB, it is possible solve technical computing problems faster than with traditional programming languages, such as C, C++, and FORTRAN. MATLAB has a wide range of applications, including signal and image processing, communications, control design, test and measurement, financial modeling and analysis, and computational biology. Add-on toolboxes (collections of special-purpose MATLAB functions, available separately – Figure 1) extend the MATLAB environment to solve particular classes of problems in these application areas. Figure 1: MATLAB Architecture extends with toolboxes and blocksets. MATLAB provides a number of features for documenting and sharing work. MATLAB can interact in code level with other languages and applications, and distribute MATLAB algorithms and applications. Key Features • High-level language for technical computing • Development environment for managing code, files, and data • Interactive tools for iterative exploration, design, and problem solving • Mathematical functions for linear algebra, statistics, Fourier analysis, filtering, optimization, and numerical integration • 2-D and 3-D graphics functions for visualizing data • Tools for building custom graphical user interfaces • Functions for integrating MATLAB based algorithms with external applications and languages, such as C, C++, Fortran, Java, COM, and Microsoft Excel. [1], Figure 2. Figure 2: Overview of MATLAB Product Family. Computer Aided Electrics & Electronics Engineering (For ex: ASELSAN) and Advanced Financial Computing areas (For ex: Central Bank of Turkish Republic) have the largest user groups of MATLAB in Turkey. The field of Electrics & Electronics Engineering use MATLAB for Rapid Prototyping, Code Generation, Real Time Embedded Control System Development, Data Acquisition, Data Visualizing, and Advanced Engineering Topics (Neural Networks, Fuzzy Logic, Genetic Algorithms etc). Financial division use MATLAB for Financial Modeling, Data Mining, Data Feed, Curve Fitting, Optimization, Database Connectivity, Excel (Microsoft) Connectivity, MATLAB Compiler, Data Visualizing, and Advanced Engineering Topics in only one platform. At the beginnings, users in Turkey selected MATLAB as a tool. They used the already coded graphical user interfaces (GUI) to compute results. Just like using functions in Excel or Scientific Calculators. Last two years; arranged conferences, user meetings, education programs and F GES A.Ş. put forward the use of MATLAB as a Programming Language. As mentioned before, MATLAB’s easy syntax, rapid prototyping ability and one platform for all problems made users to code applications on MATLAB. Moreover, users use included compilers to generate standalone applications. This is a common usage of MATLAB in worldwide by low level programmers. Nowadays, companies demand on N-Tier applications, which mean that users want to drive MATLAB as a computation engine from other popular programming languages (For ex: Java, C#, C++). As a scenario, users have User Interfaces (via a web browser or a stand alone file) to run process’. These interfaces send commands to application server, which run MATLAB and if there is Database or other routines, they should call procedures on other server like Database Server. MATLAB provides COM, DDE and C Routines to call MATLAB from other languages. MATLAB has documentation for calling MATLAB from C and C++, but does not have any for Java and C#. In a detailed look, COM and DDE Support is just for Microsoft Windows Operation Systems. Using C Routines is a supported methodology for C/C++ and C# languages. If you want to use C Routines in Java, Java Native Interface (JNI) has to be used. C and C++ languages are not well formed and rapid choice for developing N-Tier applications (For ex: Remote Method Invocations (RMI) and Web Services). C# is a suitable for the solution but not platform independent in practice (Some projects like Go-Mono still underdevelopment and is not supported by Microsoft). In the other hand, MATLAB has a product called “MATLAB COM Builder” to generate DLLs from MATLAB programs to be used in C/C++/C#. Main purpose is to reduce time in development and easy code management. The MathWorks Inc. uses external interfaces as mentioned above. There are other products in market offers parallel products such as Mathematica, one of the most popular. Mathematica has full documented C/C++/C# and Java support. Also Mathematica can run as web service server, beside MATLAB. Mathematica offers Java packages and C# namespaces to talk with Mathematica Engine directly. These features bring Mathematica in a situation one step forward than MATLAB at subject of external interfaces. In conclusion, MATLAB users want to have a platform to develop N-Tier application rapidly. MATLAB Server Pages (MSP) is a Java based open source platform independent application to fill this gap. MSP offers web based application development platform including database connectivity, Remote Method Invocation, Distributed Computing, and Web Services. Not only MATLAB users can use these products, web developers and application engineers can now prefer MATLAB for N-Tier architectures. Proposed Solution MATLAB Server Pages (MSP) is an open source technical web programming language using MATLAB in background, which supports 3-Tier Architecture (Web Tier – Business Tier and Database Tier) including Distributed Computing and Parallel Processing via remote procedure calls and web services. Some Features of MSP: • MATLAB Tag Libraries: Develope technical web pages as simple as using HTML Tags. • MATLAB RMI Tag Libraries: Use a remote MATLAB just like it is running on local. • MSP Expression Language Support: For DB Connectivity and Easy Computing. • MATLAB Server Faces: Advanced Web Components such as Uploads & Validators. • MATLAB Beans: Handle Business Rules and Logics. • MATLAB Web Service: Use MATLAB as a Web Service. MSP Architecture: Figure 3: Overview of MSP Architecture. MATLAB Server Pages connects to MATLAB with Stefan Mueller’s Java Wrapper Class called JMatLink [2]. MSP classes drive this class for JavaServer Pages including new methods for MATLAB Features, JSP Page Output, Imaging On Browser, Remoting and Web Services. MSP implements JavaServer Pages and JSP Standard Tag Library to define MATLAB Custom Tag Library, with support for Expression Language and Scriptlets that can evaluate methods of MSP classes. This lets developers to call MATLAB with in JSP page by using tags. MSP main classes are declared as Remoting classes. It is possible to register MSP classes and bind them from remote classes. Marshalling serializes MSP classes’ methods and properties to remote site and unmarshalling deserializes them at remote site. MSP includes ftp client classes to pass and retrieve files to/from remote computer. This is designed for not transferring images, MATLAB files and outputs by serialization. Both sides run a web server. If needed, MSP FTP client puts necessary MATLAB files to server and evaluates command on remote server. Generated output, beside images and HTML pages, are transferred back to local computer. Images and HTML Pages are displayed on local computer using HTTP protocol. This provides escapeness for serialization problems. End users, developers, do not need to know these background processes. There are ready custom tags that handle these operations. This custom tag library is called MATLAB RMI Tag Library. Apache Software Foundation’s Axis project has tools for Java 2 WSDL generations. MSP contains Axis libraries to generate Web Services for its MSP classes. There are two types of WSDLs in MSP. Nowadays, WS-I Group tries to minimize web services interoperability problems. MSP has WSDLs for both Java and C# proxy class generators. All of these implementations create Web Tier. MSP Web Tier can be standalone from application server using MSP RMI features. By default, MSP comes with MySQL JDBC driver. JSP Standard Tag Library has SQL tag library, which can run SQL expressions on database server. Expression Language Mechanism handles MSP MATLAB Tag Library and JSTL SQL Library interactions. From MSP, developers can connect to a database server. They can retrieve data from MATLAB and store to database and vise versa. This is the Database Tier – EIS Tier of MSP. JavaServer Faces is a technology that has advanced user interface components for JSP and can define navigation rules for business processes by running beans in background. MSP uses Apache Software Foundation MyFaces implementation. The pages that use JSF have MLSPX extension in MSP. Default extension is MLSP for MSP. MLSPXs can call beans and do operations defined in them. Developers can add their beans to MSP. MATLAB Beans works like a Plug-in Logic. If developers use remoting in MATLAB Beans, they can separate business logic and business rules to an other server. This puts “Business Tier” to forward. MSP has several demos and examples on how to use MSP Engine. Client systems can be dynamic web pages, a part of distributed computing and web service operations. These usage examples can be called as a “Client Tier”. Contribution of all tiers creates MSP Architecture. Figure 3. MSP Technologies: Table 1 below, gives implementation of technologies to MSP. Table 1. Technology JavaServer Pages Description JavaServer Pages (JSP) technology lets you rapidly develop and easily maintain, information-rich, dynamic Web pages. JSP pages contain display markup and tags that encapsulate the logic for generating dynamic content. JSP technology includes some standard tags and support for creating custom tags. [3] Implementation Custom Tag Library: MATLAB Tag Library and MATLAB RMI Tag Library JSP Standard Tag Library JavaServer Faces The JavaServer Pages Standard Tag Library (JSTL) goes beyond the JSP standard tags, providing custom tags that encapsulate core functionality common to many Web applications. JSTL has support for structural tasks such as iteration and conditionals, tags for manipulating XML documents, internationalization tags, SQL tags, and commonly used functions. [4] JavaServer Faces or JSF simplify the development of user interfaces for J2EE applications using JavaServer Pages. It is a framework for building user interfaces for web applications. It includes: A set of APIs for representing UI components and managing their state, handling events and input validation, defining page navigation, and supporting internationalization and accessibility • A JavaServer Pages (JSP) custom tag library for expressing a JavaServer Faces interface within a JSP page. [5] Java DataBase Connectivity. JDBC is a Java API that enables Java programs to execute SQL statements. This allows Java programs to interact with any SQLcompliant database. Since nearly all relational database management systems (DBMSs) support SQL, and because Java itself runs on most platforms, JDBC makes it possible to write a single database application that can run on different platforms and interact with different DBMSs. [6] Java Remote Method Invocation (Java RMI) enables the programmer to create distributed Java technology-based to Java technology-based applications, in which the methods of remote Java objects can be invoked from other Java virtual machines, possibly on different hosts. RMI uses object serialization to marshal and unmarshal parameters and does not truncate types, supporting true objectoriented polymorphism. [7] • Support for; • Expression language, • Iterations, • Conditional, • SQL, • XML, • Formatting, in MSP. (MSP uses MyFaces from Apache Software Foundation Group.) Support for; • MATLAB Beans: To define Business Logic and Business Rules. Advanced UI Components like Upload Form, Validation etc. • in MSP. Java DataBase Connectivity Work with JSTL and JDBC Drivers to store data from MATLAB to DB and retrieve data from DB to MATLAB. Uses JSTL SQL Tag Library and MSP Core Package “line” to process requests. Remote Method Invocation Custom Tag Library: MATLAB RMI Tag Lib. Enables web developers to develop web pages using a remote MATLAB just like it is on local. Java 4 Services Web Web services are Web-based enterprise applications that use open, XML-based standards and transport protocols to exchange data with calling clients. Java 2 Platform, Enterprise Edition (J2EE) provides the APIs and tools you need to create and deploy interoperable Web services and clients. The Java 2 Platform, Enterprise Edition platform provides the APIs and tools that are needed to quickly design, develop, test, and deploy Web services and clients that fully interoperate with other Web services and clients running on any platform. This full interoperability is possible because application data is translated "behind-the-scenes" to a standardized XML-based data stream. [8] (MSP uses Axis from Apache Software Foundation Group.) MSP uses generating Services. Java2WSDL for MATLAB Web MSP supports two WSDLs and one of them is WS-I Compatible. MSP supports BPEL. 3-Tier System Description: Figure 4: Overview of MSP 3-Tier Architecture Figure 4 demonstrates the connection of layers. First Layer: Application Server MATLAB is installed on this server. There are MSP Main Classes on this PC to data exchange over RMI with Web Server. Main Classes have several methods to handle custom JSP Tags at Web Server. For example:  tag on Web Server runs; public void engineOpen() { // Reads MSP settings from Bundle. ResourceBundle rb = ResourceBundle.getBundle("MSP"); // Code Dir is the base for MATLAB M-Files, Simulink Models and // Matlab Server Files. All runnable codes and models must be // there. Code_Dir = rb.getString("Code_directory"); // Image Dir is the base for MATLAB created images. Image_Dir = rb.getString("Image_directory"); // System out out for Java debugger. System.out.println("MATLAB Server Pages Engine"); javaLink = new JMatLink(); javaLink.engOpenSingleUse(); // Running MSP at Code Dir. javaLink.engEvalString("cd '" + Code_Dir + "'"); } This class handles all RMI operations, database connectivity, distributed computing, and web services. Some MATLAB Tags access to these classes: • • • • • • • • • • • • • • • • • • • • • • • Engine: Starts MATLAB engine for computing (Single Use). SessionStart: Starts MATLAB engine for computing (Multi Use, Session based). SessionGet: Connects to specified MATLAB session. SesisonEnd: Closes MATLAB session. Command: Executes MATLAB commands. Clean: Deletes all JPG and BMP images in Image directory, removes MAT files in Code dir. Debug: See where the problem is in MATLAB syntax. EvalMDL: Executes Simulink Models. EvalMFile: Executes MATLAB M-Files. EvalMLSF: Executes MSP Files, which can be a bunch of MATLAB commands. GetArray: Get MATLAB array (2D) in to Java. Ex: magic(10) GetVector: Get MATLAB vector (1D) in to Java. Ex: rand(1,10) or rand(10,1) GetScalar: Get MATLAB scalar in to Java. Ex: sqrt(9), exp(8) etc. GetCharArray: Get MATLAB string (char) array in to Java. Ex: 'Hello World' GetParam: Gets HTTP input form. PlotData: Plots MATLAB figure. PlotModel: Draws Simulink model. PlotSim: Plot Simulink result. Thumbnail: Creates thumbnail for MATLAB figure. PutArray: Put Java double[][] to MATLAB. PutVector: Put Java double[] to MATLAB. PutScalar: Put Java double to MATLAB. WriteData: Writes MATLAB variable to web page; including array, vector, scalar. Second Layer: Web Server Web Server acts as a RMI Client to connect to Application Server. This Web Server has a Remote Method Invocation Interface to MSP Main Classes located at Application Server. This interface includes all methods in these classes. So application in this server can use all functionality of MSP as running on local. Programmers can deploy applications on this server with an easy syntax: Test.mlsp ---------- <%@ include file="/Scripts/header.inc" %> Third Layer: Database Server Web Server connects to this layer to exchange data using JSTL SQL Library and MSP Tag Libraries. Fourth Layer: User Interface End Users can connect to Web Server via a Web Browser or using Web Service. Research Methods Methodology: MATLAB Server Pages is a green field project. This project targeted to convert MATLAB abilities to a “Technical Web Programming Language”. As a methodology “Rapid Development” technique is chosen to handle the project, because there is no other work to inspire from and it is a must to measure user demands needs in these types of projects. Although this is a green field project, MSP takes into consideration of some customer compliments about MATLAB Web Server product to not to make same mistakes. Before release of MSP, prototypes of MSP are created and is distributed over open source community spaces. User opinions are added to project. This development methodology handles to see bugs and application problems in development state. So it is much easer to adobe changes. Development Tools: • • • • • • Eclipse 3.1 (Free to use): IDE for Java Development Visual Studio .NET 2003: IDE for C# Development Eclipse WTP 0.7R (Free to use): IDE for Web Developments in Eclipse EclipseME (Free to use): IDE for MIDlet Development in Eclipse Genady RMI (Free to use): Plug-in for RMI Stub Generation Sun Wireless Toolkit Platform (Free to use): Tool for Mobile Web Service Stub Generation Challenges: • JMatLink is only usable, when it is not in a package format for JSP. The reason for this is its C Code among the Java Native Interface. This situation causes Java Server Pages ability to be unusable in current development kits, because it only supports “package” hierarchies. To avoid this paradox, I used Java Compiler 1.3 version. Javac 1.3 supports “packageless” forms in JSPs. • Axis Tomcat add-in needs Java Run Time 1.4.x to run. Latest versions of Tomcat are coming JRE 1.5 compatible. To run Axis web services, Tomcat has to be made runnable with JRE 1.4.x. Tomcat has another extension called “compat” that handles Axis and Tomcat functionalities run smoothly together. • To prove platform independency of MATLAB Server Pages, this project involves makefiles of shared library of JMatLink for Solaris and Linux Oss from open source community users. • J2ME only supports HTTP Connections in threats to avoid deadlocks. • Pocket PC, ASP.NET and Windows Clients for MSP Web Services have a common WSDL proxy generation bug. C# does not understand array of double arrays. It needs manually editing from reference files. • J2EE Web Services using keywords of C# have interoperability problem for C# in proxy class generation. (For ex: value) • ASF MyFaces does not run on Tomcat, if Sun’s JSP compiler class library is installed. • Smart Application runs only with IP address when using MSP Web Service. • C# Wrapper Class for MATLAB C Engine Library does not support Web Service developments. This is a reason to prefer Java in development stage of MSP. • Reading a 200x200 dimensional array from MATLAB takes 10 ms for C Engine Library calls, 30 ms over COM and 902 ms for Runtime Invocation of MATLAB. [9] MSP uses C Engine Library. Related Work MATLAB Web Server: • Commercial Product • Based on CGI Executable • Creating of a simple web application needs 3 code pages at least • Has no debugging features • Can not interact with a web programming language • Does not has DataBase support • Does not has Web Services • Does not have tag libraries for rapid development • Restrictions to some toolboxes The Ohio State University – “A Java Based Web Interface to MATLAB”: • Unavailable for Public Use and Not Open Source • Works with HTML Embedded Servlets • No automated installation • Harder syntax to code • Only supports Linux/Unix Operating Systems • Does not support JavaScript interactivity • Does not has DataBase support • Does not has Web Services • Does not have tag libraries for rapid development • Advantages; o Supports MATLAB generated audio/video playback on HTML o Supports Kerberos Authentification o Supports dynamic zooms to graphics on pages Conclusion MATLAB Server Pages adds several new features to MATLAB for rapid development. It is also an open source product, so developers can add their enhancements. Moreover, MSP has some similar tools to MATLAB toolboxes. Table 2 describes the similarities of products. Table 2: MSP & MATLAB Equivalence Table The MathWorks Inc Description MATLAB Toolbox MATLAB Web The MATLAB® Web Server Server enables you to create MATLAB applications that use the capabilities of the World Wide Web to send data to MATLAB for computation and to display the results in a Web browser. MATLAB Database The Database Toolbox is one of Toolbox an extensive collection of toolboxes for use with MATLAB®. The Database Toolbox enables you to move data (both importing and exporting) between MATLAB and popular relational databases. MATLAB Distributed The Distributed Computing Computing Toolbox and the MATLAB Products Distributed Computing Engine enable you to coordinate and execute independent MATLAB operations simultaneously on a cluster of computers, speeding up execution of large MATLAB jobs. MATLAB Compiler Compiles MATLAB M-Files to system libraries or executable files. Price[10] 2000$ MSP Equivalent MATLAB Server Pages Engine 1000$ MSP and JSTL Connectivity 1900$ MSP Remote Procedure Calls and Web Services 5000$ Commonly used for System Library (For ex: DLL) generation, which will be used in other applications. MSP invokes MATLAB, no need for compiling process. MSP may decrease the total ownership cost of MATLAB. Future Works: • Providing Report Generation Mechanism: PDF, Excel, HTML, Doc and PowerPoint Slides. • Providing a web based user interface, which seems like Excel, to make a more user friendly environment. • Providing Add-Ons for BPEL based applications. • Arranging more secure RMI and Web Service features to MSP. • Adding audio and video multimedia support to MATLAB Tags. • Adding servlets for some figure operations like zooming an area in figure. • Adding an online MLSP editor. • Adding an Administrator console to set properties easily. • Adding Load Balance to MSP Systems. • Adding connection limits to MSP. • Fixing code in tag libs for user demands. • Fixing WS-I Compability issues for goOnlineDotNet.wsdl. • Fixing bugs found by users. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] http://www.mathworks.com/products/matlab/description1.html Stefan Mueller’s JMatLink, http://jmatlink.sourceforge.net/ Sun Java Products – JavaServer Pages, http://java.sun.com/products/jsp/ Sun Java Developers Forum – JSP Standard Tag Library, http://forum.java.sun.com Wikipedia – Definition of JSF, http://en.wikipedia.org/wiki/JavaServer_Faces GreenSuite.com – Definition of JDBC, www.greensuite.com/whitepapers/whitepapers_03.html Sun Java Products – Java Remote Method Invocation, http://java.sun.com/products/jdk/rmi/ Sun Java Technology and Web Services, http://java.sun.com/webservices/index.jsp CodeProject, “1..2..3 ways of integrating MATLAB with the .NET”, Ruffaldi E., November 2003, http://www.codeproject.com/dotnet/matlabeng.asp The MathWorks Inc. United States Individual Price List, September 2005. Links used for researches are: • • • • • • • • • • • • • Apache.org JGuru.com TheServerSide.com Sourceforge.net Codecomments.com Experts-exchange.com Techworld.com Codezoo.com Javaalmanac.com C-sharpcorner.com Dotnetforums.com Onjava.com Javaranch.com