(Universtiy of Sydney teaching until November 2008)
VISUAL PERCEPTION + DIGITAL IMAGING (DESC9155)
Sessions
1. Saturday - Sunday October 18 - 19, 2008 - 10am - 6pm General Access Lab
2. Saturday - Sunday October 25 - 26, 2008 - 10am - 6pm General Access Lab
+ assessment tasks
Learning modules
- Colour (theory and treatment in PhotoShop CS)
- Photographic Digital Image Capture (SLR camera technology, lenses, and photographic techniques)
- Digital Image Composition & Correction (framing, creative colour, cropping, effects, collage, pre-press restoration/correction/optimisation)
- Print Preparation
Practical Tasks
1. Colour Quiz (10%) Saturday Oct. 18 in class
2. Colour adjustment (10%) Saturday Oct. 18 in class: image 01 | image 02 | image 03
3. Digital Imaging Quiz (photographic technical knowledge and digital imaging) (10%) Sunday Oct. 19 in class
4. Texture and colour capture field work (5%) Sunday Oct. 19 in class
5. Image tag team process (10%) creative collaboration Saturday Oct. 25 6pm by email
6. Collage (15%) saturation, filtering, blending modes, masking, integration, October 29 on CD-ROM 300dpi, A3 edge-to-edge (do not print)
7. Composition Folio (30%) 6 images 4x6" + 1 A3 300dpi (your choice best image) - colour, texture, light, composition, depth of field, portrait/wildlife, landscape/architcture; and demonstrate understanding of print process and resolution October 29 on CD-ROM (do not print)
8. Attendance (10%)
Outcomes
Exhibition-quality prints of best images, web posting of best images from the course, animated sequence of image tag, individual folios
Summaries
Printable Assessment Descriptions
Interactive Sound Design Studio (DESC9187)
BLOG: http://interactive-sound.blogspot.com/
Intensive Schedule
*Information session 1 August 2-6pm Sentient Lab
Part 1: What is the assessment, form groups for major work
Part 2: Technical introduction to software, approach, technical information sources and how to prepare BEFORE first studio session (assessment)
- where tutorials are for acquiring technical knowledge
- review technologies out there for capturing data (info or about interaction such as gesture/motion)
- techniques for creating original sounds
- techniques for propagating/generating sounds over time
Assignments/Tasks
1. Tech submission due: 25 September - presentation of tech submission + project proposal 15%
2. Review design + code + approach for 2 sample projects from this list ... [in the form of a researched report] due: 25 September submission by blog (individual assessment) 10%
3. Complete workshop ATSC safety course, wear suitable shoes. Linda Fienberg (formative not summative)
4. Final presentation 10%
5. Final project and documentation (online submission to blog) 50%
6. Participation - this mark involves attendance, studio discussion participation, and exhibition setup (layout, power, computer requirements, equipment responsibility, publicity) 15%
Assessment 1 description / assessment criteria - 1,000 words blog post, (diagrams and images helpful) + in-class presentation (5 powerpoint/keynote slides per individual)
Describe major project prototype design
1. Describe design basis and discuss influential approaches
2. Describe intended user experience
3. Outline interactive elements, data types used, mapping methods + (integration of technologies) + audio output elements, audio production tasks (provide structure chart)
4. Discuss major technologies required (in detail), e.g. camera tracking
Assessment 2 description / assessment criteria (1,000 words blog post, diagrams and images helpful)
Review design + code + approach for 2 sample projects from this list ... [in the form of a researched report] due: 25 September submission by blog (individual assessment). Assessment is based on understanding and correctly attributing knowledge from the literature and critical evaluation. Use the headings 'design review', 'code review' and 'approach' to structure your response. All material sourced from the Internet or books should be correctly referenced.10%
Here are some interesting examples:
boiingg
http://docs.monome.org/doku.php?id=app:boiingg
Algorithmic music generator with monome interface for interaction.
Forester 1.4
http://leafcutterjohn.com/Software.html
Interface for controlling various digital audio samples. Leafcutter John makes some software
Many more available on:
http://cycling74.com/twiki/bin/view/ResourceGuide/InterestingWork
But remember they should be something that involves both interaction and sound.
Assessment 3 description
ATSC Workshop safety training course. You should complete this in order to make physical objects for interaction and interface design. This is a formative task (i.e. must be completed but will not be evaluated as part of the final mark). Students who do not complete the 3-hour training (scheduled during the studio hours) will be unable to use tools, equipment and resources of the ATSC.
Assessment 4 description / assessment criteria - Present interactive project design
In a team of between 1-3 people, present your interactive major project design. 10%
The purpose of presenting your work in progress is to hear feedback from the lecturer, tutor and peers on conceptual and technical improvements, suggestions, queries and to demonstrate an understanding of the requisite technologies, methods needed to develop the work and to explain the conceptual paradigm underlying your work. You should discuss the interface design and technical method of execution, motivation for the design, consideration of the user experience, discuss the hypothetical question - how might you evaluate the efficacy of your design from a user perspective? Refer to influential and inspiring designs that compare or contrast with your approach. Prepare no more than 8 PPT/PDF/Keynote slides to present to the class. Each individual will be marked and each person in the group should make part of the group presentation (usually addressing the part you are responsible for in the design).
Assessment 5 + 6 description / assessment criteria
About the project. The culminating presentation will be in the hearth/atrium.
The culminating presentation will be in the hearth/atrium. 50%
The final project should entail an interactive experience chosen from the following possibilities:
1. augmented, hyper-, new or physically interactive "instrument" music making interface for performance; typically performative single-user driven
2. interactive installation; typically public/audience-driven and may be multi-user or single-user. Gestural, or object triggered/manipulated interactive installation - create a work for exhibition or public performance that uses gesture controllers or computer vision, fiducial marks or motion controllers to produce a dynamic audio-visual experience. Other modes of physical input are eligible (please discuss with us).
3. information sonification, ambient display, data-driven audio, could be an interactive sonification for manipulating and examining information in new ways; typically single-user, though an ambient display could be for public or domestic spaces. An audio representation of data input (live) or data file. The interaction/customization should enhance information access or understanding (e.g. playback controls, manipulate presentation, expose interesting attributes of the data).
Available equipment includes: webcams for computer vision, Wii controllers, motion sensors. Also available to download are data-streams, fiducial markers and software for reactivision tracking.
Exhibition. This is the presentation of your completed functioning design. You should demonstrate fully how it works and discuss your rationale and considerations on reflection of the final outcome. Documentation 1000-words addressing each area (brief overview description, related and influential works, design motivation, technical method, user experience / usability considerations, resources/references and a few examples of media documentation published on the blog (upload web-sized photos, audio recording, QTmovie) should be included of an appropriate nature to record/archive the project after the installation/exhibition is finished. Retain your hi-res images on a CD or DVD-ROM for your archives and future reference. Your blog submission will be the documentation assessed.
Participation. Your participation mark is based on studio attendance and participation and your preparation and involvement in the final exhibition event. Students are responsible for exhibition setup (layout, power, computer requirements, equipment responsibility, publicity). Different responsibilities for ensuring the technical, interactive, presentation and social smooth-running of the final event. You may invite parents and friends to experience your work.15%
Studio Schedule
... coming soon ...
Guest lecture: Densil spatial hearing and auditory perception.
Other interesting info
(from New Music Now network news) Masterclass with Alex Davies. Saturday 2 August 10am-4pm FREE @ Campbelltown Arts Centre. The masterclass offers artists working with video, sound and interactivity an insight into Davies' techniques for creating illusion and a sense of presence in his installation works. More information: Phone (02) 4645 4100.
Advanced Interactive Multimedia Design (DECO3005)
Unit of Study blog |
Results + Feedback
Here are anonymous overall results for the course. These have been submitted to the SAC and will be officially available shortly. Congratulations! Everybody did really well and everybody participated well on the blog. Here are some feedback remarks for the competency tasks (anonymous) and presentations (well, you know who you are already?).
Aim + Description
The aim of this Unit of Study is to look at motivations for designing interactive environments, installations and art works, such as social activism, curiosity + playfulness, and consequences of human-centered computational interaction, e.g. effect and affect, engagement and usability then to develop the technical skills to design an interactive project enabled by physical-, sensing- or data-driven-computing.
This Unit of Study reviews site-specific, contextual installation and other interactive art and design works. We investigate technologies and modes of interaction such as physical computing, spatial and gestural interaction, interfaces for tactile, tracking and wearable interaction. We look at methods for real time contextual content generation, e.g. data-driven art, informative interaction, sonification/visualization and generative/creative computing models. Cycling 74's Max/MSP is the main software platform, in conjunction with camera vision, wireless sensors and motion controller hardware or purpose-specific physical devices designed by students. The 'design-interact' blog is a forum to share the dialogue of investigation. Certain tasks involve reviewing events, literature and projects of existing interactive works and peer review and feedback of each other's projects. Sharing links and technical information promotes an open discussion supporting the creation of interactive experiences.
Schedule
Date |
Lecture |
Crit |
Tutorial |
Assessment |
| 7 March | Introduction (10-11) |
|||
| 14 March | Interactive art (10-12) |
(12-1) | Blog post links | |
| 4 April | (10-1) Intensive workshop | Blog lit review | ||
| 11 April | Site-specific (10-11) |
(11-1) Max/MSP intro | Short research report (blog post) | |
| 18 April | Physical + spatial (10-11) |
(11-1) Interfacing hardware (cv, Kroonde, reactivision, Wii) | ||
| 2 May | Wearable computing (10-12) |
(12-1) Workshop project plans + tech requirements | ||
| 9 May | Motivation + purpose (10-11) |
(11-12) Tech method for project | (12-1) Present project design | |
| 16 May | Data-driven art (10-12) |
(12-1) Jitter - computer vision, calibrating sensor data | Technical competency demo | |
| 23 May | Generative/creative computing (10-11) |
(11-1) | Blog review event | |
| 30 May | (10-1) | |||
| 6 June | (12-1) Post review of peer project | (10-12) Explain/present design demo + major project evaluation Blog project idea development (continuous) |
Lecture Topics
1. Introduction (overview, Unit of Study outline, assessment, blog, introductory examples)
2. Interactive art + social design (not site-specific)
3. Site-specific, locative + contextual interactive art
4. Physical + spatial interfaces (tactile + tangible, tracking, gestural, sensing - embedded + moveable)
5. Wearable computing (e-fashion + worn devices)
6. Motivation + purpose (social activism, curiosity, playfulness, effect + affect, engagement, usability)
7. Data-driven art (interactive [live] data + display)
8. Generative / creative computing in interaction design (esp. in relation to A.I. + A-Life modeling in Max/MSP)
Discussion / Criticism / Blog
1. Review literature (interactive works of others and literature on technologies and conceptual thinking)
2. Review and event/experience - critical appraisal of an interactive multimedia design user experience
3. Feedback on peers' work
4. Project development - conceptual and technical method
Technical knowledge (tutorials)
See topic listed under Assessment task 3. Tutor: Sam Ferguson in Mac Lab
Assessment Tasks
1. Participation and contribution to blog 25%
(paper-based references will be added to the virtual library listing of related books/texts and links to web resources or electronically published writing will be added to the related links section to build up a repository of knowledge over the semester)
Method: http://design-interact.blogspot.com/ (each class member has been sent an invitation [ to your uni email account] to join this blog as an author)
Minimum, complete a task from each of these categories:
- electronic resources/references/links (5%) 14 March
provide 5 hyperlinks to related topics or sites with a 1 paragraph summary of each. Try to find a unique link not already listed on the blog. - critical literature review (5%) 4 April
1000-word summary of a key themes in interactive multimedia design, identifying references/sources, including up to 5 images and hyperlinks to sources or related sites. Choose from: social activism explored through interaction design; human-centered interaction design/usability; tele-immersive or distributed network interactions; methods of evaluating user experience; tangible computing; haptic interfaces; interaction in virtual reality; robotic interaction + cyborgism - review event interactive experience (10%) 23 May
500-word review your user-experience of an interactive multimedia design (event attended, installation in a gallery, commercially environment - e.g. a game and entertainment station, an interactive performance - e.g. a performed work, observed by you - such real time music/dance/AV. Include a relevant image. investigate your reactions, the usability, responsiveness, technical infrastructure, artist/designer motivations, contextual and cultural themes. (Some places where you might find suitable work include the Museum of Contemporary Art, Music/Art/New Media festivals, BetaSpace at the Sydney Powerhouse, The Performance Space ...) - project idea formation (5%) 6 June (accumulative continuous throughout semester)
This can be posting plans and ideas behind the project for peer comment or dialogue with your project team, approaches to technical, social and design issues. References to related work, inspirations, motivations. Consider method of development and user experience in your idea formation.
2. Short research report (electronic publication) 15%
11 April. Method: upload as a post to http://design-interact.blogspot.com/
Investigate i) an interactive art, installation, design work or ii) hyper-instrument design or iii) visualization/sonification work by an influential designer and write a critical review of the work in the context of its genre and the artist's/designer's works.
Your submission of a written report of the format and quality appropriate for conference submission. As the idea is to broaden your experience and develop new collaborations, it is a good idea to explore outside your area of current expertise to gain new knowledge.
assessment criteria and structure:
2,000 words + images, scholarly referencing
Your own interpretation, analysis, original point of view or argument utilizing existing research to substantiate ideas are the criteria for successful assessment.
Improperly acknowledged and reference material - test, images, charts, media, code - or any unoriginal, unattributed ideas from others will be considered as plagiarism.
3. Technical competency tasks 20%
16 May
Submission of small milestone summative tasks demonstrating competence with Max/MSP + Jitter. Zip/archive your files for submission by email to kirsty@arch.usyd.edu.au
The intention of this assessment task is for you to develop and demonstrate facility with the tools you will need to use in your major project. Obviously, the tools you will need will be determined by the type of project you wish to accomplish. Below is a list of categorized Topics to Investigate, and it is suggested that you investigate at least two (if not a lot more) from each category. At the same time you should be investigating artworks and environments that implement interactive multimedia, and should be developing an idea of your own so that you know which of these topics you will need to use for your project. We will review your submissions and deliberations throughout the semester and will provide hints and directions. Your assessment is based on the demonstration of technical mastery.
Core Software Environments
Max/MSP/Jitter – Visual programming language with extensive facilities for midi, synthesis and visualization. (Commercial)
Pure Data - Open source version of the above, lacking in documentation and somewhat less user-friendly but adequate for many tasks. puredata.info (Free)
SuperCollider - Small-talk-like text language with extensive audio abilities. Least user-friendly but very terse/elegant. supercollider.sourceforge.net (Free)
Helpful Software Tools
Processing - simplified java environment - extensive but simple programming environment with somewhat restrictive sound support. processing.org (Free)
AudioSculpt - Sound manipulation and cross synthesis tool (Commercial IRCAM License)
Protools - mixing and sound manipulation tool (Commercial)
CSound - language with inheritance directly from the pioneers of computer music. (Free)
TOPICS TO INVESTIGATE
general programming
* data structures, functions and program flow - Learn how to write simple programs by following the tutorials and reading the documentation.
* connecting programs/tools with OSC (Open Sound Control) - OpenSoundControl is a flexible network protocol that most of the software you will use speaks. You can use it to connect Max/MSP to supercollider within the same computer, or you could use it to connect two computers remotely. See the cnmat site for general details.
* synthesis, filters and envelopes - Tutorials 1 and 2 deal with synthesis using simple oscillators. Investigate using different types of waveform: sawtooth (saw~), triangle (tri~) and rectangular (rect~) oscillators will give different types of sound. For envelopes you can look at the help for the adsr~ object, for the most useful example. For filters the filtergraph~ and biquad~ partnership is pretty much all you need. Look at their help.
* frequency modulation synthesis - Basic FM synthesis is dealt with in tutorial 10. FM synthesis is mainly interesting for the role it has played historically, rather than necessarily being a good choice these days. Tutorial 11 has a bit more information if this topic interests you.
* MIDI - stands for Musical Instrument Digital Interface. It is a very complicated topic as it has been around for 25 years. It essentially boils down to being a way to number all the notes on a keyboard and send those numbers between note sequencers (old electronics or modern computers) and instruments that play those sequences of notes (keyboards, synthesisers and computers as well). I avoid using it, as there are now better ways for communicating between sequencers and synthesisers, but it is still very popular and quite handy.
* physical modelling - Physical modelling is a method of producing sounds based on modelling the physics of a musical instrument. The resulting sounds can be extremely realistic, but the computational cost is high.
data input
* text data file reading - look at the 'text' object's help.
* http data sources - There are an excellent set of externals for talking to google (which appear not to work anymore) or to a mysql database at Bill Orcutt's site, although unfortunately he has not updated them to intel universal binaries. The windows versions may still work. Alternatively we can attempt to recompile the source if there is something you really want. For an alternative mysql object there is a java version at http://www.loadbang.net/space/Software/net.loadbang-SQL/MySQL Configuration for these is probably relatively difficult. Alternatively something like processing can be used, and then the results can be sent to max via OSC.
* camera input (cv.jit) - tracking location, mass, colour, point - cv.jit is a library that allows relatively simple extraction of information from webcam input. There are also a number of examples in the Jitter Tutorials.
* sensor/controller input - Wii, Kroonde or aka.listen, apple remote, tilt macbk - i.e.
- The Wii can be used with the 'aka.wiiremote' object. Various other sensors on apple computers can also be used with some of the 'aka' objects from http://www.iamas.ac.jp/~aka/max/. For instance aka.listen allows speech recognition, aka.appleremote gets apple remote data, and aka.booklight gets light data from a macbook's keyboard backlighting sensor. The 'Kroonde Gamma' is a system for using sensors wirelessly. It is a rack-mounted unit that attaches to a computer through ethernet cable, and receives information from a small pack that is attached to a set of sensors. This means the sensors and the pack can be attached to a person for instance.
* computer listening - beat tracking and feature detection (pitch, intensity, spectrum)
- The Tristan Jehan Max/MSP Objects are very useful for this. They are at his website at www.media.mit.edu/~tristan. There are a number of different features, including pitch, loudness, brightness and beat tracking that can be used for various purposes, and in various contexts. The IRCAM objects also include yin~, which is a very good pitch detection algorithm.
* basic scaling (e.g. Max/MSP scale object) - The Max/MSP 'scale' object scales an input range (say between 0-127) to an output range (say between 5-15). This is very important when mapping data sources to outputs of some kind. This can also be performed with some simple maths.
* Lindenmeyer systems - jit.linden is a Jitter object that implements the core of a lindenmayer system. The most popular use of lindenmayer systems is to replicate a 'tree growth' structure, with a trunk, branches and leaves.
* evolutionary algorithms - Probably not an easy thing to implement in Max, but worth thinking about. Evolutionary algorithms are a field of study that cover a wide variety of biologically inspired computational paradigms. See http://en.wikipedia.org/wiki/Evolutionary_algorithm for some more details.
* neural oscillators - have a look at the pictures in this paper - http://www.springerlink.com/content/n402g24m81gt3884/ , If you like the idea of an oscillator network that produces these types of interwoven waveforms you can see me for some code that will allow you to produce this.
* recursive oscillatory neural networks - Check Ollie Bown's site for a Java Continuous TIme Recursive Neural Network that you can use in MaxMSP, at http://igor.gold.ac.uk/~map01ob/ Whether it can be used for musical purposes is up to you.
miscellaneous
* visual controllers and displays - Jitter can be used to make a visual display or interface. Processing can also be used, and then can be hooked up with OpenSoundControl.
* live coding - Live Coding is the idea of improvising a musical performance on stage. Often it is performed in a `battle' format with two programmers facing off to perform to their best ability. Programs are often written in languages such as superCollider or ChucK (note that Max/MSP is not necessarily suited, as it is far too easy to crash it while programming). There are a set of good links at the wiki page. http://en.wikipedia.org/wiki/Live_coding
* embedded architectures - Whilst it is not necessarily logistically possible for this class, it is possible to send messages between arduino boards (http://www.arduino.cc) and Max/MSP. What you use the arduino board for is up to your electronic skills.
* ambient sonification - Ambient sonification is sonification (the representation of information through sound), that is used in an ambient context. By ambient we mean that it does not need to be attended to in the environment it is used in, but will still impart its information in an unobtrusive, aesthetically pleasing manner.
4. Present interactive project design 10%
6 June. In a team of between 1-3 people (individual work is fine), present your interactive multimedia major project design. The purpose of presenting your work in progress is to hear feedback from lecturer, tutor and peers on conceptual and technical improvements, suggestions, queries and to demonstrate an understanding of the requisite technologies, methods needed to develop the work and to explain the conceptual paradigm underlying your work. You should discuss the intended interface design and technical mode of execution, motivation for the design, consideration of the user experience, discuss the hypothetical question - how might you evaluate the efficacy of your design from a user perspective? Refer to influential and inspiring designs that compare or contrast with your approach. Prepare no more than 8 PPT/PDF/Keynote slides to present to the class.
About the project. The culminating presentation will be in the hearth/atrium.
Create one of:
a) Digital instrument - an interactive interface for producing music/sound with the dual purpose of recording music or live performance. The controls of your 'instrument' can be an on-screen interface or gesture controllers for affecting the sound.
b) Interactive data-driven work (visualization/sonification) - an audio visual representation of data input (live) or data file. The interaction/customization should enhance information access or understanding (e.g. playback controls, manipulate presentation)
c) Gestural interaction installation - create a work for exhibition or public performance that uses gesture controllers or computer vision, fiducial marks or motion controllers to produce a dynamic audio-visual experience.
5. Explain/present design demo + major project evaluation + documentation 30%
6 June. This is the presentation of your completed functioning design. You should demonstrate fully how it works and discuss your rationale and considerations on reflection of the final outcome. Documentation 1000-words addressing each area (brief overview description, related and influential works, design motivation, technical method, user experience / usability considerations, resources/references and a few examples of media documentation published on the blog (upload web-sized photos, audio recording, QTmovie) should be included of an appropriate nature to record/archive the project after the installation/exhibition is finished. Retain your hi-res images on a CD or DVD-ROM.
Curriculum
Selected subjects taught between 2001-2008
Undergraduate (Bachelor of Design Computing)
Sound Design and Sonification
(Acoustic and audio theory, methods, sound design, auditory cues, auditory display, sonification)
Advanced Interactive Multimedia Design
(responsive + interactive multi-modal art installation, hyper-instrument or data-driven display (visualization/sonification); Max/MSP + Jitter, computer vision, Wiring + Processing, Kroonde/WiSeBox wireless sensors)
Graduate Coursework (Master of Design Science - Digital Media)
Interactive Sound Design Studio
(responsive + interactive sound and multi-modal art installation; Max/MSP + Jitter, ProTools/Logic, Wii controllers, cv.jit colour- and motion-tracking computer vision, reactivision fiducial marker-tracking with computer vision and reactable, boidal flocking, Wiring + Processing, Kroonde/WiSeBox wireless sensors, physical computing, sensate environments, ambient display, data-driven design, spatial + gestural interaction)
Visual Perception and Digital Imaging
(physiology of vision, visual cognition, image composition, aesthetics, criticism, montage, digital imaging, PhotoShop)
Digital Media Design Studio
(Project-based applications of interactive and time-based media design, physical computing, artificial intelligence, A-Life and other generative computing methods, sensor and tracking data, research papers)
Web Design and Programming
(scripting and programming, and media element design)
Interactive Multimedia Design
(usability, HCI, interaction design, Flash + ActionScript, Director)
Advanced Interaction Design
(responsive + interactive multi-modal art installation, hyper-instrument or data-driven display [visualization/sonification]; Max/MSP + Jitter, computer vision, Wiring + Processing, Kroonde/WiSeBox wireless sensors)
Digital Media Production
(image editing - PhotoShop, Illustrator/InDesign, audio editing, video editing)
Postgraduate Research
Ph.D supervisions (Principal Supervisor):
Joanne Jakovich, Hong Jun Song, Michael Bates, Nathan Wilson
Ph.D (Associate Supervisor):
Ian Stevenson, Sam Ferguson, Baki Kocaballi
Honours supervisions (3)
Independent Study supervisor
Units of Study Coordinated
Unit administration and supervision (not teaching)
3D Computer Graphics Concepts (formerly 3D Modeling)
3D Animation 1
3D Animation 2
Digital Compositing and Visual Effects
Modeling and Animation for Games Studio
Digital Video Design and Production
Digital Culture