Learning Center
Plans & pricing Sign in
Sign Out

Business Model Process Omnigraffle

VIEWS: 315 PAGES: 21

Business Model Process Omnigraffle document sample

More Info
									Software Architectures and Tools for
Model Sharing
             Christopher Mueller
      Open Systems Lab/Indiana University

  IMAG Model Sharing Strategies Mini-Workshop
                April 10, 2007
    Research Software Projects
    Software is an essential research tool
    Many projects use custom software
        Data gathering and processing
        Simulation
        Analysis and visualization
        Algorithm/protocol development
        Glue for 3rd party applications
    Many research areas are developing common application
    Software is developed by a combination of grad students,
     undergrads, Pis, collaborators, and consultants
    Results are often published with no verification and
     accepted into the computational canon
Evolution of an Application
                                              • First application written in Fortran
                                              • “A Model for Baconian Dynamics” *
                                              • Tom ports to C
       F77                                    • Adds command line parameters, makefile
                                              • “An Application Framework for Baconian Dynamics”
                                              • Jenny ports to F90
                                              • Extends model
                                              • “An Extended Model of Baconian Dynamics”
   C         F90
                                              • Brad ports to C++
                                              • Models system using objects
                                              • “An Object Oriented System for Dynamical Baconian Systems”
                                              • Jeremy (consultant) consolidates existing versions in C++
  C++        C++          Java
                                              • Advanced template and object patterns lead to fast and
                                                extensible code that is indecipherable by scientists
                                              • Maria implements model in Java for the Grid
                                              • Implements original model
                                              • “A Scalable, Grid Enabled Toolkit for Baconian Systems”
                                              • Baconian Dynamics predicts summer blockbusters
                                              • Everyone wants a copy of the software

       * Baconian Dynamics predicts the success of a movie using models based on the cast’s Bacon Numbers, i.e. how many
       degrees of separation are between the actors and a movie staring Kevin Bacon
A Closer Look

                                   There were 6 major versions
                                       13 actual implementations
                                       5 Languages
 C1 C3                             2 major versions advanced the science
             F903 F902
      C2                           4 major versions were simply software
                   F904             projects
 C++1                              All versions re-implemented basic
              C++         Java      features
                                   The implementations used for the
                                    papers were not always used for the
 Version used for paper
                                    next major version
 Versions that advanced science
Research Software Crisis!

        Reproducibility of computational
     results is difficult and the lack of good
       engineering practices is negatively
        impacting scientific productivity.
    Rethinking Research Software

    Architect software for reproducibility and productivity

    Use existing technologies for application infrastructure

    Use Agile development processes

    Develop with a team of dedicated engineers and

    Know your community’s limitations
Designing for Productivity
Goal: Design software that enables all users to optimize their productivity.

              User                            Tasks

                                        Run simulations
  Scientist                             Manage workflow
                                        Extend models

                                        Maintain code base
  Developer                             Develop applications and libraries
                                        Deploy applications

                                        Evaluate models (journal review!)
  Community                             Use applications
                                        Share models
Designing for Reproducibility
  Goal: Design software that enables all users reproduce results.

             User                         Tasks

                                    Run existing simulations
 Scientist                          Integrate new models into simulations
                                    Share model extensions

                                    Reproduce bugs
 Developer                          Ensure compatibility across versions
                                    Validate alternate (optimized) components

                                    Evaluate models (journal review!)
 Community                          Use applications
                                    Share models
Designing for Scientists
Provide a integrated user interface for
common use cases

Use domain-specific user interfaces and                         QuickTime™ and a
                                                            TIFF (LZW) decompressor
                                                         are needed to see this picture.


Provide a scripting interface for
workflow management and model

Use “projects” to save workflows and runs

Optimize when software impacts scientific productivity
  Designing for Developers
Place all resources under
version control

Encourage basic coding standards

Use unit tests

Use multiple languages
e.g., Python for infrastructure, C for kernels, SQL for data

Use a component architecture

“Buy”, then build

Use separate systems for development,                          CompuCell3D
testing, and deployment
  Designing for the Community
Know the community and respect
its standard practices

Share as a Web application, if
possible (and it’s usually possible)

Provide binary installers for
common platforms

Include a validation suite              

Provide collaboration services for discussion and support
 Designing for Performance
Address performance as it
impacts productivity

Understand performance
challenges as early as possible
Is it data size, communication, and/or computation?

Test performance assumptions
before committing code to address

Have a clear strategy to move
from prototype to production code

Design the component framework to scale with the data and computation
 Agile Software Development
         Software development is an iterative process that consists of
                           three main phases:

                             Business Modeling
 Business Modeling
                              Understand the main roles , resources, and procedures
                              used in the application domain

                             Development Iterations
Deploy              Design    Identify features and refinements, design components,
                              implement in software, and deploy to the users using
                              short, manageable iterations (days to weeks)

                              Deliver the application to the users and provide ongoing
                              support (60% of application lifecycle!)
Software Development Tools
        Diagram Software                                      Word Processors
                                                                    Rapid Prototyping Tools
                                  Business Modeling
                                                                  VB, Python, PowerPoint, HTML

     Packaging Systems                                                   Interpreted Languages
       InstallShield, RPM           Requirements                             Java, VB, Python

 Bug and Feature
 Tracking Systems                Deploy              Design                 Compiled languages
     Bugzilla                                                                    C/C++/Fortran

                                    Implement/Test                   Integrated Development
         Test Suites                                                   Environments (IDEs)
                                                                        Visual Studio, Eclipse

      Automated Nightly Builds                                     Libraries/Components
                                                                       numerical, plotting,
                                                                   instrument communication

         Version Control         Debuggers and Profilers
       subversion, Perforce
   Architecting for Model Sharing
Use a component based           Compile time: language level components (#include)
architecture                    Link time: binary objects (external languages)
                                Dynamic run-time: arbitrary objects (COM, native
                                 interfaces, OSGI)
                                Local processes (maximum flexibility)
                                Remote processes (Web-based applications, network services)

Use reference data formats      Conversion to/from reference format
                                Support industry standards first, invent new ones only when
                                 absolutely necessary

Have minimum documentation      Allow HTML
requirements                    Embed documentation in domain MLs

Have minimum validation         Unit tests for each shared component

Have a community Web site       Central location for model sharing resources
Software Development Roles
End User                                        Anyone who uses the software

                                                Coordinates development efforts, resolves conflicts, ensures project is
Project Manager                                  moving along. This is the hardest job to fill.

Lead Developer                                  Experienced member of the team, understands technologies and is able
    (aka Architect, Sr. Software Engineer)       to advise other developers. Same responsibilities as developer.

                                                Responsible for all aspects of a portion of the application (requirements,
Developer                                        design, implementation, testing)

Web developer                                   Similar to a developer, but with a skill set targeted at designing and
                                                 implementing Web sites and applications

                                                Maintains and optimizes the database and helps developers design
Database Administrator (DBA)                     database applications

Technical Writer                                Develops tutorials and user manuals

                                                On projects released to a wide audience, a separate QA team is
Quality Assurance                                responsible for testing

System administrator                            Responsible for maintaining the computers, software licenses, file
                                                 systems, and security policies
Keys to Success                                     …or, Process and People are Necessary but not Sufficient

Developer-User interaction                         The more levels of communication required, the higher the chance
                                                    that requirements will be mis-communicated

Small, incremental deliverables                    This ensures the application evolves based on user’s
                                                    needs and that requirements can be adjusted

Implement what’s needed…                           This keeps developers and users focused on the
 …not what might be needed                          current problems

Experienced developers in lead roles               Would you make an undergraduate a lead scientist?

Keep management neutral                            The project manager’s role is to keep things moving
                                                    smoothly without getting in the way

…Except for the PI                                 He who has the gold makes the rules (and the tough
                                                    decisions no one else wants to make)
Mutual Respect
            The hierarchy and reward systems for software and science are different.
            Scientists should treat developers as colleagues, not as servants
            Developers should respect the ideals and institutions of science
            Developers should be willing to understand the scientific field they are supporting
Final Thoughts
Developing software is a complex process
     Proper training and tools are important for success

Separating research and development improves
  the quality of research software
     Existing staff can do this at first, but outside help is
      needed as projects expand*

Using appropriate languages and tools increases
  productivity, performance, and reproducibility
     Many problems have adequate solutions. Take
      advantage of them!
        *Plea to funding agencies: Fund software development as if it’s essential infrastructure!
   “The Mythical Man-Month”, Frederick P. Brooks, Jr.
   “Peopleware: Productive Projects and Teams”, Tom
    DeMarco & Timothy Lister
   “Software Project Survival Guide”, Steve McConnell
   “Facts and Fallacies of Software Engineering”, Robert
    L. Glass
   “Software Carpentry”, Greg Wilson, et al.,
Example: WebMail Component Diagram

To top