Docstoc

PowerPoint-presentation - Viktoria Institute

Document Sample
PowerPoint-presentation - Viktoria Institute Powered By Docstoc
					Interaction with sound & music (pt. I)
Lalya Gaye, Viktoria Institute




                    Information Art & Design Course
                   Tama Art University, Tokyo, Japan
                                 26 september 2005
Introduction
Interaction with sound & music

Aims and scope
• Introduction to interaction design and prototyping
  methods for the design of interfaces for:
  – music performances / installations
  – use of sound in everyday life
• Project examples
• Simple prototyping activities
Introduction
Schedule
* Monday 13.00-16.00
• Brief overview of my work
• Design space: music controller
• Design space: everyday interfacing with sound
  (e.g. mobile music technology)

* Tuesday 13.00-16.00
• Prototyping methods
• Case study
• Prototyping activities
Lalya Gaye
Future Applications Lab, Viktoria Institute
Göteborg, Sweden
Future Applications Lab

One group at the research institute in applied
information technology Viktoria Institute, in
Göteborg, Sweden.
6 PhD students + one senior adviser (Lars Erik
Holmquist)
Research: Ubiquitous computing, mobile and locative
media, information visualisation, tangible computing,
human-computer interaction, interaction design, etc

www.viktoria.se/~lalya
Education

BSc Physics – University of Geneva, Switzerland
MScEng Engineering Physics – Royal Institute of
Technology (KTH), Stockholm, Sweden
PhD student in Informatics / Applied Information
Technology – University of Göteborg / IT-University of
Göteborg, Sweden




www.viktoria.se/~lalya
Research
Focus
Ubiquitous Computing for Everyday Aesthetic Practices
Keywords: Ubicomp, sound, photography, physical
interaction, everyday life

Method
Experimental projects:
- Concepts
- Prototypes
- User studies
Workshops: Mobile music technology
Design sketches: Aesthetics of physical interaction

www.viktoria.se/~lalya
Sonic City                                     www.viktoria.se/fal/projects/soniccity

Lalya Gaye (FAL, Viktoria Institute), Ramia Mazé (Re:Form, Interactive Institute),
Daniel Skoglund (8Tunnel2), Margot Jacobs (Re:Form, Interactive Institute)


Creating a real-time personal soundscape of electronic
music by walking through and interacting with urban
environments

Technology: a wearable using sensor-based information
to control real-time audio processing of urban sounds and
turn them into music


www.viktoria.se/~lalya
Tejp                                         www.tii.se/reform/projects/pps/tejp.html
                                           www.tii.se/reform/projects/pps/tejp.html

Margot Jacobs (Re:Form, Interactive Institute) & Lalya Gaye (FAL, Viktoria Institute)



Embodied interaction with locative media (e.g. space
annotations)
Example: Audio tags on walls whisper to people leaning
towards them; space of intimacy in public space




www.viktoria.se/~lalya
Context Photography                            www.viktoria.se/fal/projects/photo

Maria Håkansson, Sara Ljungblad, Lalya Gaye, Mattias Rost (FAL, Viktoria)
Panajotis Mihalatos (Art & Technology, IT-University in Göteborg)



A context camera captures the invisible context of a scene
with sensors (e.g. level) and translates it visually into the
resulting still picture noise, as you are taking it




www.viktoria.se/~lalya
Tap-n-Bass                        http://www.timebend.net/projects/?tap_and_bass
Lalya Gaye, Alexander Berman, Valerie Bugmann (Goutte d’Or)




Live drum-n-bass with re-mixed tap-dance shoes
Sound of wired-up tap dance shoes is re-mixed in real time
into live drum&bass




www.viktoria.se/~lalya
Attack of the PANTS
(Portable Art Noise Things)
David McCallum, Lalya Gaye, Tobias Good (Fringe)



DIY workshop for amateurs with simple electronics:
Building portable audio units with contact microphones
and speakers made out of junk boxes




www.viktoria.se/~lalya
Mobile Music workshops                         www.viktoria.se/fal/events/mobilemusic
Lars Erik Holmquist (FAL, Viktoria), Atau Tanaka (Sony CSL Paris), Lalya Gaye (FAL,
Viktoria) , Frauke Behrendt (Univ. of Sussex), Drew Hemment (Univ. of Salford,
Futuresonic + PLAN)



Mobile Music: Music technology meets mobile computing.
Devices used anywhere, with awareness of place, in
distributed / ad hoc networks…
Workshop sessions with presentations, in-depth
discussions, group brainstormings, hands-on activities…



 www.viktoria.se/~lalya
Interfacing with sound
Interfacing with sound
Types of interaction with sound

* Performance/installations vs everyday use
•   Music making: performance / composition
•   Sound-art installations
•   Sound as design material
•   Mobile audio
•   Ambient displays
•   etc
  Interfacing with sound
Design of music controllers
Design: music controllers
Acoustic vs. electronic instruments

* Acoustic instruments
• Sound source and interface are the same artefact
• Nature of expected sound & laws of physics dictate the
instrument’s form and how it is designed
• Fixed and relatively transparent mapping
• Acoustic + haptic feedback
Design: music controllers
Acoustic vs. electronic instruments

* Electronic instruments
• Sound soure and interface separated  mapping,
interaction and physical attributes are free
• Need for methods and constraints in order to fulfill criteria of
expressiveness, transparancy, audio quality
• Need for feedback
Design: music controllers
Issues

* Purpose of design
 Users
Experts vs. amateurs?
• Uses
Composition? Performance? Education? Cognitive stimulation?
• Designing an…
Interface? Instrument? Composition tool?
Design: music controllers
Issues

* Criterias
• Relationship between performer and audience
• Physical effort
• Complexity / transparency
• Ergonomics
• Cultural context

It is not just about producing sound, it is about the whole
experience of producing sound
Design: music controllers
Interaction loop

* Simplified model



input                 mapping               output
                                sound
          interface
                                generator
1rst fb

2nd fb
Design: music controllers
Interaction loop

* Simplified model



input                 mapping               output
                                sound
          interface
                                generator
1rst fb

2nd fb
Design: music controllers
Interfaces
* Object-based
Starting with existing instruments
- augmented (hyperinstruments…)
- digitalised (ex: piano synth)
- interface used as controller (ex: MIDI keyboard)
Use metaphor of object




   Taku Lippit, ITP/NYU, 2002-03     Machover & Ma, Hypercello, MIT, 1991
Design: music controllers
Interfaces
* Object-based
Repurposed everyday objects
and materials: water, fabric,
chemicals, vegetables …




 Daniel Skoglund, 8Tunnel2   Particles, Horio Kanta, 2003   MIDI Scrapyard Challenge,
                                                            Brucker-Cohen & Moriwaki, 03-04
Design: music controllers
Interfaces
* Object-based


Take advantage of the material properties of objects f.e.x
bendable, conducts electricity, etc
Take into consideration human activities surrounding the
objects: build upon it and / or break from it
Design: music controllers
Interfaces
* Body-based
Human body as start for design




                 The Hands, Waisvicz, STEIM, 1984
Design: music controllers
Interfaces
* Body-based

Human body as start for design:
- Ergonomics
- Existing gestures
- Expressive qualities of human movements
- Scale and continuity of movements
Design: music controllers
Interfaces
* Environment-based
Interactive environments
- Reactive floors
- Digital realm: networked audio
Everyday environments, etc




 Magic Carpet, MIT Medialab, 1996   Global String, Tanaka   Sonic City, Gaye et al., 02-04
                                    & Toeplitz, 1998
Design: music controllers
Interfaces
* Environment-based

Take advantage of the features of space

- Interactive environments:
many people together, control of interaction parameters…

- Everyday environments:
rich environment, unpredictable, dynamic, heterogeneous
Design: music controllers
Interfaces
* Wearables
Musical jeans jacket
(MIT Medialab, 1992)

Tgarden
(FoAM & sponge, ~2001)

Expressive Footwear
(MIT, 1997-2000)

ensemble (Kristina Andersen, ~2003)
Design: music controllers
Interfaces
* Wearables


Intimate interfaces
Body movement and posture
Theatrical vs. daily life dimensions
Design: music controllers
Interfaces
* Representations
- Tangible algorithms



                        Audiopad, Patten, Medialab, 2001   Block Jam, Newton-Dunn et al.,
                                                           Sony CSL, 2002


- Virtual instruments



                                           Mulder, Simon Fraser Univ., 199?

- Screen-based (laptop musicians using MAX/MSP, Pd, etc)
Design: music controllers
Interfaces
* Representations


Taking familiar sound manipulation metaphor and making it
tangible, into space.
Design: music controllers
Interfaces
* Circuit bending
Hacking is fun!




                    Modified Toy Orchestra
Design: music controllers
Interfaces
* Hybrids
Controlling e.g. audio and visuals together
Balancing and adapting interaction so that both dimensions
are satisfactory




                  Manual Input Sessions, Golan Levin, 2005
Design: music controllers
Interaction loop

* Simplified model

input                 mapping               output
                                sound
          interface
                                generator
1rst fb

2nd fb



                      control
Design: music controllers
Control

Levels of indeterminancy
• Control vs. randomness ( interactive improvisation)
• Total predeterminancy: push a button  deterministic output
• Total undeterminancy: random machines
• Unexpected vs. expected input / output


Control characteristics
• Continuous vs. discrete control
• Implicit vs. explicit
• Micro- to macro-level control: sound spectrum to details of
articulation to overall structure
Design: music controllers
Interaction loop

* Simplified model



input                 mapping               output
                                sound
          interface
                                generator
1rst fb

2nd fb
Design: music controllers
Mapping
* Issues
• Complexity to stimulate creativity
• Transparency to keep link between input and resulting
sound (otherwise, danger of loosing the audience)
Design: music controllers
Mapping
* One-to-one
   input              mapping               output

    X                                         X
    X                                         X
    X                                         X

 • Each independent input assigned to one musical parameter
 • Simplest mapping scheme, but usually the least expressive
Design: music controllers
Mapping
* One-to-many
   input               mapping               output

     X                                         X
     X                                         X
     X                                         X

 • One input controls more than one simultaneous musical
 parameter
 • Conductor model: provides a macro-level expressivity control,
 but does not allow access to internal (micro) features
Design: music controllers
Mapping
* Many-to-one
   input               mapping               output

     X                                         X
     X                                         X
     X                                         X

 • Many inputs coupled to produce one musical parameter
 • Requires previous experience with the system in order to
 achieve effective control
 • But far more expressive than the simpler unity mapping
Design: music controllers
Mapping
* Many-to-many
    input               mapping           output

     X                                      X
     X                                      X
     X                                      X

 • Many inputs coupled to many musical parameters
 • Control on different levels
Design: music controllers
Interaction loop

* Simplified model



input                 mapping                output
                                 sound
          interface
                                 generator
1rst fb

2nd fb
                      feedback
Design: music controllers
Feedback

Feedback
• Helps articulating control
• Passive vs. active
• From mono- to multi-modal (modalities: audio, haptic, visual)
• 1rst FB: from interface
• 2nd FB: audio


Feedforward
• Guides user by providing information about the internal state of
the system (as opposed to information from output)
Design: music controllers
Interaction loop

* Simplified model

input                 mapping               output
                                sound
          interface
                                generator
1rst fb

2nd fb
Design: music controllers
Output

* Mechanical
Guitarbot
(Eric Singer et al., LEMUR,
2003-)


* Tactile output (haptics)
Cutaneous Grooves
(E. Gunther, MIT Medialab, 2001)
Design: music controllers
Output
* Alternative speakers
SoundbugTM speakers & piezos
Spherical speakers (Curtis Bahn)
Flower Speakers (LET’S corporation, Japan, 2004)
Design: music controllers
Interaction loop

* Simplified model

input                 mapping                  output
                                   sound
          interface
                                   generator
1rst fb

2nd fb


              General forms of interaction
Design: music controllers
Interaction
* User movement
- Choreographed body movement
- Traditional instrumental gesture
- Novel gestures




Dark around the Edges,   Machover & Yoyo Ma,   The Hands, Waisvicz, STEIM, 1984
Winkler, 1997            Hypercello, 1991
Design: music controllers
Interaction
* User movement
- Full-handed gesture

                              Unfoldings, Interactive         Stranglophone,
                              Inst., 2003                     Sharon, ITP/NYU, 03

- Empty-handed gesture




                         Lady glove, Bongers & Sonami, 1991



+ Unvoluntary movements, embodied actions...
Design: music controllers
Interaction
* Real-time music
Improvising new music



vs. interpreting existing one
(conductor model)


                                        Radio Baton, Max Mathews, 1987


vs. navigating through non-linear musical narratives
Design: music controllers
Interaction
* During performance
                                Tooka, Fels et. Al, UBC, 01-03
Interaction with environment,
audience, etc

- Performer-performer

- System-audience
                                  Crackle-family, STEIM, 1976

- Performer-system-audience



                                  Dialtones, Golan Levin, 2001
Design: music controllers
Interaction
* During performance

- Audience as collaborative performer




                                          Sine Wave Orchestra, Tokyo, 2003-04
- Private performances in public spaces
…or over the internet



                                               Le Placard headphones concerts
 Interfacing with sound:
Performance/installations
     vs everyday use
Properties of sound in everyday life
• Ubiquitous (sometimes obtrusive)
• Dynamic and transient
• Broad yet subtle information carrier (emotions,
  data)
• Socio-cultural meaning
• Strong link to space and time
• Physicality (body and space)
• Additive: layers
• Foreground vs. background awareness -> implicit
  vs. explicit interaction
Sound in everyday interactions
Audio as input
Examples from art & research
Blendie
(Kelly Dobson, MIT Medialab,
2003-04)


Context Photography
(FAL, Viktoria Institute,
2003-04)

-> physicality, cultural meaning…
Sound in everyday interactions
Outputing sound

* Ambient audio displays
- Street crossing auditory displays
etc
- Sonification of network activity:
AmbientROOM
(Hiroshi Ishii et al., MIT, 1996-97)




-> Peripheral awareness
Mobile music and locative audio
Mobile music and locative audio
Locative audio in public space
* Motivations
Sound as public display
Peripheral awareness
Community re-appropriation of public space
Mobile music and locative audio
Locative audio in public space
* Space annotation
Hear&There
(Rozier, MIT Medialab, 1999)


Tacticle Sound Garden
(Mark Shepard, 2004)


Tejp / Audio tags
(PLAY & FAL, 2003-04)
Mobile music and locative audio
Locative audio in public space

* Radio pirates

Bit Radio
(Bureau of Inverse Technology)




Key Chain Radio Station
(Rikako Sakai, Ivrea, 2004)
Mobile music and locative audio
Mobile Music Technology

Music technology meets mobile computing. Devices used
anywhere, with awareness of place, in distributed / ad hoc
networks…
Device follows user’s displacement and connects to the world
(physical, social, located virtual)

- Mobile music making, listening, sharing
- Wearable audio
- Sound walks, etc
Mobile music and locative audio
Mobile music
* Mobile music sharing

Social aspect of mobile computing: ad hoc networks,
distributed social networks, etc
-> spontaneous and situated music sharing with people in
public space
Mobile music and locative audio
Mobile music
* Mobile music sharing
SoundPryer (Mattias Östergren, Interactive Institute, 2001)




TunA
(Arianna Bassoli et al.,
Medialab Europe, 2002)


Push!Music (Håkansson et al., Viktoria Institute, 2005)
Mobile music and locative audio
Mobile music

* Mobile music sharing
Bass Station
(Mark Argo & Ahmi Wolf,
ITP/NYU, 2003)
Mobile music and locative audio
Mobile music

* Mobile music making

Music making away from computer screen or performance
setting: in the everyday
Sensor technology + GPS -> situated music making
Ad hoc & distributed networks throughout the city ->
collaborative music making
etc
Mobile music and locative audio
Mobile music

* Mobile music making

Sonic City
(Gaye et al., FAL & PLAY, 2002-04)




Malleable Mobile Music
(Atau Tanaka, Sony CSL, 2004)
Mobile music and locative audio
Mobile music

* Mobile music making

Sound Mapping
(Mott et al., Reverberant, 1998)




Sonic Interface
(Akitsugu Maebayashi, 1999)
Mobile music and locative audio
Mobile music

* Mobile music making
CosTune
(Nishimoto, ATR, 2001)

Sound Lens
(Toshio Iwai, 200?)
Mobile and locative sound
”Walking through sound” (D. Toop)

* Sound-art installations
Electric walks
(Christina Kubisch)
Drift
(Rueb)
* Walking through digital space
Seven Mile Boots
(Beloff et al., 2003-04)
* Non-linear audio narratives
The Case at Kulturhuset (Knifeandfork, 2004)
Mobile and locative sound
Wearable audio

”Personal instruments”
(Krzysztof Wodiczko, 1969)




(Chelle Hugues, RCA/CRD, 2000)
Mobile and locative sound
Wearable audio

Nomadic Radio
(Nitin Shawney, MIT Medialab, 1998)




Sonic Fabric
(Alice Santaro, 2002)
Design: everyday interactions
Considerations

• Is the interaction implicit or explicit? Is the awareness
  foreground or background?
• Mapping issues too
• How does the sound fit in the environment?
• Cultural assumptions about how things sound (familiar
  movie effects vs. unusual real life sounds)
• Emotional content and conotations
• Role in everyday life (balancing everyday activities with
  interaction with sound, etc)
Etc.
Design: everyday interactions
Considerations


How to find out: interaction design methods &
 prototyping!
For tomorrow
Design exercices

• Get into groups of 4 people
• Discuss potential sound application ideas (very
  basic and simple ones!)
• Choose interesting everyday artefacts to bring
  tomorrow
• Tomorrow: design exercises!
ありがとうございました!
   またあした !

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:14
posted:7/17/2012
language:English
pages:77