Musical Keyframe. Final experiance.

Final experiance. Musical keyframes on Vimeo

Final experiance

Interface developement

Principles of movement
multi-sensory modal cube chooses between various modes. Axis movment for accessing various submodes (origo-access) Circlular and rotational movement for accessing individual parameters(Circle-lines)

Concept III, Image Schemata

Image Schemata is a term from cognitive psychology

we organize our impressions accourding patterns
which is founded in the experience of our own
bodies in interaction with the world.
”…..embodied patterns of meaningfully
organized experience, such as structures of
bodily movement, and perceptual interactions..”
”…structures which our experiances must fit if
it is to be coherent and comprehensable..”
(Mark Johnson)

Container
blockage
Path
Cycle
Part/whole
Full/empty
Surface
ballance
Attraction
Link
Near-far
object
Center-periphery
Inside/ Outside


Early ideas for hollow central object which could detect RFID-tagged objects inserted in a central object. This could involve marbles representing eather soundfile change or effect-adding. The system would involve a wireless RFID-reader.

Consept II, Keyframe- interpolation

Keyframe- interpolation extention of animation terminology
The system has a double interpolation.
Interpolation between contrasting settings within one effect, and interpolation in time
Taking a few contrasting musical situations, represented by
a graphical object, and interpolating between the situations in time.
The system offers both graphical and auditive feedback and antisipation in the playback moment, and offers the user a creative feedback for further idea-generation
/ Interpolation as grenerative principle in time-based art.
This simple, but powerfool technique creates variation and consistency.
An attractive principle in time-based art, like music

Musical objects implementation

My overall aim has been to create a graphical envirement which gives a more intuitive impression of the various properties of each sound and the relation to other sounds.


The graphics should not depict the sound itself (which is an abstrack unspeakable entity), but the controll parameters assigned to the sound
The objects represents a sound with indication of strength, location in space,
location in time and various effectsettings (tentacles)

Musical objects

My approach to graphical representation is also inspired by P. Schaeffers ideas of the musical object. A well established metaphor in music-research

Multidimentional anaalysis

To solve this problem I have drawn upon experiences from multidimentional
representation in music and design research,

Plotting qualities of sound in a three-axis system. Each axis being a dicotonymic set of sound metaphores.
This multidimentional representation-techniques give a good overview of the subject and an easy access to comparitative analysis.

Concept I , Holistic representation

Concept I
Holistic representation
Traditional music software is relying on a vast amount of modes and submode to perform various tasks. This creates problems of multi-point attention.

Consepts foundation for project

Future directions

Technical spesifications

Tracking a mag-light

Since the modal-issues was channeled out to a secondary object I decided to consentrate the tracking on position and rotation

By placing a massive photo-filter over a webcamera and using a strong mag-light I can easily get the tracking-info without disturbance from the graphics

Multi-sensory modal Cube

Due to weak IR-light results and focus problems of hacked camera, I decided to move my attention to a secondary multi-senory object.

Meassuring various degrees of resistance the software triggers various modal interfaces. The resistor-meassuring is done with an arduino-chip, which sends ist results to Max/MSP through the serial object.
I also experimented with tactility to solve the problem of multi-point attention
By adding various surface qualities(spunge, sandpaper etc) the user can keep his focus on the screen an recognize the various modes through the tactile quality (The metal points for the 3 resistors is on the opposite sides of the three collours)

Example of numeric diodal tracking.

Example of numeric diodal tracking.

Technical developement of tracking and modal shifting.

I planned to do the tracking with jitter/cv-jit
I was going to send out infra-red light from the physical objects to seperate the graphics of the screen from the information from the physucal object.
I did some experiments with removing the IR filter from ordinary web-cams.
I wanted to have all functions could being handled by one central physical object
I imagined a geometrical object which could be tipped from side to side, triggering the 3 modes and handeling both position and rotation.
By combining triangular shapes and various nr of light diodes I would both take care of rotation, position and modal issues.

Example of drilling triangular shapes into accrylic surfaces. By sending horisontal light through the plate thedrilled triangle is illuminated. (lightning sollution Stig Skjellvik)

Navigation

Each parameter of each effetc-tentacle consist
of two contrasting settings (red/green mode) which may be edited seperatly.
The two settings can be interpolated(orange mode) to create new
consistently variated musical situations in the orange mode

2 Modes

Playing intuitively with the soundqualities in an editing mode, using a physical object on an illuminated surface.
Composing in a playback-mode by lending the keyframe consept from animation arts. Creating variation and consistencyby interpolating between a few

User-group

Medium-advanced creative user
Probably not for Sound engeneers. - Controll freaks
Probably not for children. - Abstract consepts

Purpose

Structuring and simplification of the hetrogenic, modal style of traditional music-softeware.
Offering creative feedback through the system-design to battle the overwhelming technical possibilities of modern music-software.
Multimodal antisipation and experience of the sound and its controll-parameters. Individually and in relative to oother soundfiles

Project goal

Table projected music-software with physical interaction.
Timbral tools. (No midi grids)
Representing several aspects of a sound-file in a holistic graphical object.