Situation
Working with computers opens up a whole realm of possibilities for creating interactive content that is not available to us when we work with conventional media. Books, music, movies, these things don't tend to offer much interaction with the users. They are static. They provide content for the user to absorb. Computers however, allow us to create content that is dynamic. What users see and hear can change based on their interactions. Working with a programming language such as Processing, we can create such interactive content with ease. And the objects that we produce can be shared online, on a desktop computer or even on a handheld device.
Situation Specific
For this project, I will design and create a Digital Gift for my step-father for his birthday. I will use the Design Cycle to guide my efforts through the various phases of the project's development, collect feedback along the way to ensure that my efforts result in a suitable, high quality product and document my work here on this page.
Research and Analysis
Brainstorm
Here are a few ideas that I considered for my project. Your list may be a lot longer. The important thing is that, at this point in the process, you are still open to exploring new ideas. Try to come up with at least 3 viable ideas that you can present to a group of your peers. In the next stage, you will use their feedback to help you choose the best one.
Possibilities
Here are 3 ideas that I developed from my brainstorm. Each one was discussed with a group of my peers to help me decide on the most appropriate choice. We used Edward DeBono's 6 Thinking Hats as a tool to help analyse each idea from different points of view.
Idea #1: Interactive e-card
While greeting cards in general can seem impersonal compared to something you made yourself, electronic cards may appear even more-so. Taking the time to create one yourself makes it a truly personal gift. I will design and create an interactive e-card where he can click on some floating balloons to pop them and spell out a birthday message with letters that fall out.
The feedback I got on this idea was generally positive. The idea is appropriate for the occasion and would be well-received. The programming is likely to be difficult however and given the time that I have available to complete the project, it may not be the best choice. Perhaps next year, when I am a more experienced programmer!
Idea #2: Digital photo album
When members of a family live far apart, it is important to reinforce shared memories to help maintain a sense of closeness. A simple collection of photos of time together can help keep those special memories alive. I will design and create a digital photo album of family pictures where the pages will appear to 'flip' like a real book.
I share photos with my family all the time and we already have a number of established avenues for exchanging pictures. While I'm sure that a special birthday album would be appreciated, there is nothing particularly new about this. The page flip idea might make it cool, and I already know how to do it using a tool called Megazine3 and some simple XML programming, the focus for this project is supposed to be interactivity, so I think it wold be better to select a project that does a bit more on that front.
Idea #3: Generative art project
Using mathematical algorithms, it is possible to create interesting and beautiful artworks that morph and change and are never exactly the same twice. I will design and create an original, interactive artwork with images and sounds and embed it into a website for easy sharing.
I have recently been talking to my step-dad about generative art and he found it very interesting so it would make for a nice surprise to create something like this for him. Since retiring, computers have become a hobby for him so, again, it would be a relevant gift to suit his interests. Since it is a piece of artwork, it is something that could possibly be submitted to an art class for additional credit. There is a lot of mathematics involved in creating the algorithms to produce the images and sounds which might be of some interest to a math teacher or even a music teacher. I know how to do the programming to create the random images already, but will have to learn about using the MIDI libraries to generate sound.
Final choice: Generative art project
This project provides a good challenge to expand my knowledge of programming and getting me to work with MIDI libraries in Processing, but, given the timeframe, is still realistic. It is an appropriate gift for the intended recipient and it allows me to make connections with what I am learning in other classes.
Design Brief
I will design and create an interactive, generative artwork as a 'digital gift' for my stepfather for his birthday. He will be able to view it in a webpage and by clicking on the mouse or keyboard, he will be able to change what he sees and hears. By creating an original piece of artwork, I am giving a gift that is unique, personal and memorable.
Guiding Questions
- How will it be viewed?
- When does it need to be finished? Sent?
- What skills must be demonstrated to fulfill the requirements of the assignment?
Designing
Design Specifications
- The project will be embedded into a publicly viewable webpage.
- The project will be completed by July 23rd.
- The project will be written using a computer programming language.
- The project wil incorporate images and sounds.
- All images and sounds must be free from copyright restrictions
Appearance
Design idea #1:
As the user clicks and drags the mouse across the screen, randomly coloured circles and squares are spawned at the cursor's location and begin to move outwards at different, random rates. Each is accompanied by a musical note.
Design idea #2:
Lines of random lengths and thicknesses appear as each, random, musical note plays. Lines come in from the top, bottom and sides.
Randomly coloured circles of various sizes spawn from the location of the mouse cursor, move outwards and bounce off of the sides of the screen. Musical notes are played as they bounce off of different 'keys' around the screen. After a few tests, this offers some technical problems, where the notes interfere with one another. It sounds more like noise than music.
Sounds
If the notes are completely random, and they are based on a chromatic scale, then it will sound too chaotic. I looked at using pentatonic scales and they work pretty well. Below is a major pentatonic and an Am blues pentatonic but they didn't quite give the feeling that I was looking for.
A Chinese pentatonic scale is nice since it doesn't really have a tonal center at all. The feeling is a sort of meandering that is sort of what I was looking for where any note can be a temporary root.
Then I thought of a Chinese bamboo wind chime which offers the same feeling but the percussive nature might make it a better match to the lines and squares that I am planning to use for the visuals.
Interactivity
Idea #1: Click and drag
The user clicks and drags the mouse and shapes spawn from the location of the mouse (see design idea #1.)
-After some tests this is actually quite tricky to program. With endless shapes 'flowing' from the mouse, they need to be a part of an ever-expanding array and spawn at timed intervals. This is tricky on its own. To then associate each shape with a sound is tough. I would have to learn some new tricks to pull this off.
Idea #2: Occasional click
In this case, the program mostly runs itself. Shapes appear randomly around the screen and notes play each time they do. But with a click, the screen is cleared (like a reset) and, to make things a bit more interesting, the key could change--probably going up or down by a 5th so it isn't too jarring.
Idea #3: Simple click
Each time the user clicks, a shape spawns at the location of the mouse and begins to 'float' outwards, randomly (see design idea #3.) When it makes contact with the edge of the screen, a note plays based on where it hit.
-This also makes use of arrays, but with a simple test, I have been able to produce the effect. When I added sound though, it was noisy and the notes interfered with one another so you wouldn't hear each note play fully before the next one.
Final Design
I have decided to incorporate elements from a number of the ideas above. While I think that just one shape will be a bit boring, after a couple of experiments, I didn't like the look of having several different shapes on the screen at the same time either. It looked a bit too random and didn't appear very artistic. So instead, the user will be able to click on the screen to change the patterns from rectangles, to ellipses, to lines. This works nicely with the idea that with each click, the key will change and the screen will clear briefly.
So, the program will randomly generate patterns, drawing rectangles around the screen in different colours and dimensions. Each time a new rectangle spawns, it will play a note--a MIDI-generated percussion instrument of some sort. When the user clicks on the screen, the screen will clear and the pattern will resume, this time with ellipses. The musical key will go up or down (randomly selected) by a musical 5th. With the next click, the pattern will change to lines. Another click will take us back to the rectangles.
Realizing the Solution
In this section, you list everything that you did, step-by-step, describe any difficulties that you encountered and how you overcame them and you explain and justify any changes from your original plan. Depending on your project, your may be quite a bit longer than this.
Step by Step
I have included most of the steps as comments in the code below. The first box is just the sound. the second box is all of the code. The completed project is below that.
Start with the random sounds//special thanks to Andrew R. Brown, who created the SoundCipher MIDI library and accompanying //tutorials. This script borrows from tutorial #4: 'unlimited art'
// import the soundCipher MIDI library import arb.soundcipher.*;
//create an instance called 'sound1' SoundCipher sound1 = new SoundCipher(this);
//create an array of the notes to play, 60=C, these notes are a C major pentatonic scale float[] notes = { 57, 60, 60, 62, 64, 67, 69, 72, 72, 74, 76 };
//create an array of musical instruments, represented by numbers in MIDI, to select randomly from // wikipedia has a list of the MIDI instruments here: http://en.wikipedia.org/wiki/General_MIDI float[] inst = { 112, 113, 114, 115, 116, 117, 118, 119 };
// by adding the same number to every note in the array, you can change the key float root = 0;
void setup() { size(400, 500); frameRate(8); colorMode(HSB, 100); //randomly selects the first instrument to play from the array sound1.instrument(inst[(int)random(inst.length)]); }
void draw() { //changes to a new instrument every time the program loops (8 times per second--see framerate above sound1.instrument(inst[(int)random(inst.length)]); if (random(2) < 1) { //plays a random note from the array, the duration and loudness are also random sound1.playNote(notes[(int)random(notes.length)/4] + root, random(50, 100), random(2, 8)); } } The first step was to get the sounds to play. Everything went pretty much according to plan.
You can see a list of MIDI sounds and their program numbers on Wikipedia here:
Note that they are numbered starting from 1 but in Processing, you start from 0, so they all shift down by 1. So, percussion was from 112-119.
Adding the Shapesimport arb.soundcipher.*;
SoundCipher sound1 = new SoundCipher(this); SoundCipher sound2 = new SoundCipher(this);
float[] notes = { 57, 60, 60, 62, 64, 67, 69, 72, 72, 74, 76 };
float[] inst = { 112, 113, 114, 115, 116, 117, 118, 119 };
float root = 0;
//this variable (1,2 or 3) controls whether the program //draws rectangles, ellipses or lines int shapes = 1;
//rather than being totally random, the colur is determined using a Perlin //noise function float col = 0.0;
void setup() { size(400, 500); frameRate(8); //the colours came out more consistently when I used Hue, Sat, Brightness, //rather than RGB colorMode colorMode(HSB, 100); sound1.instrument(inst[(int)random(inst.length)]); //sound 2 is a gunshot sound that you hear when the screen is cleared //when the user clicks sound2.instrument(127); }
void draw() { col = col + 0.01; float n = noise(col) * random(100); //it looked better with a line around the shapes stroke(color(n, random (70, 100), random(70, 100))); strokeWeight(random(10)); sound1.instrument(inst[(int)random(inst.length)]); //if the notes play on ever frame, it is noisy and boring, adding a little //randomness makes it a bit more interesting if (random(2) < 1) { sound1.playNote(notes[(int)random(notes.length)/4] + root, random(50, 100), random(2, 8));
if (shapes < 2) { //when you hear the big japanese taiko drum, it the background appears to change colour //but it is actually an overlay that is not completely opaque if (sound1.instrument == 116) { fill(color(n, random (70, 100), random(70, 100), random(50, 80)));
// fill(color(random(100), random (70, 100), random(70, 100), random(50, 80))); rect(0, 0, width, height); } else { fill(color(n, random (70, 100), random(70, 100))); // fill(color(random(100), random (70, 100), random(70, 100))); rect(random(width), random(height), random(80), random(80)); } }
if (shapes > 1 && shapes < 3) {
if (sound1.instrument == 116) { fill(color(n, random (70, 100), random(70, 100), random(50, 80))); rect(0, 0, width, height); } else { //change to ellipses with a mouse click or keypress fill(color(n, random (70, 100), random(70, 100))); ellipse(random(width), random(height), random(80), random(80)); } }
if (shapes > 2 && shapes < 4) {
if (sound1.instrument == 116) { fill(color(n, random (70, 100), random(70, 100), random(50, 80))); rect(0, 0, width, height); } else { //change to lines with a mouse click or keypress float lineX = random(width); float lineY = random(height); float q = random(2); if (q < 1) { line(lineX, 0, lineX, height); } else { line(0, lineY, height, lineY); } } } //if the mouse is clicked or a key is pressed, you hear a gunshot sound, if ((mousePressed == true) || (keyPressed == true)) { sound2.playNote(100, 100, 100); //the screen is cleared to a white background fill(100); rect(0, 0, width, height); //this changes the variable so you get rectangles, ellipses or lines if (shapes == 3) { shapes = 0; } shapes++; float r = random(2); if (r < 1) { root = root + 7; } else { root = root - 7; } } } }
I quite liked the 'taiko drum'. It is a big, Japanese drum with a big, bass sound. I added a little spice with that one, where, every time it plays, it appears that the background changes color. In fact, what I did was draw a rectangle the size of the screen overtop of everything that is there, but made it a bit transparent (in Processing, if you add a 4th parameter to the fill colour, you can control the alpha) so that it looks like everything fades out a bit.
Initially, I was using a 'random()' function to get the colours to change but I found that it was a bit jarring sometimes when it would make a huge jump across the spectrum. Instead, I used a Perlin noise function where things are random, but the changes are always incremental so it looks a bit more organic. The Perlin noise function can be handy if you want to generate something like the surface of moving water or a rock face or fire--pretty much anything from nature. In my case, it made for much smoother transitions from one colour to the next.
Here is the final project.
Evaluating
Feedback
The feedback that I got was very positive. I used a number of people's suggestions along the way to tweak things, such as the maximum and minimum sizes of the shapes and thicknesses of the lines. I also used people's opinions to help get the frequency of the instruments right (if you look at the array where the pentatonic scale notes is located, you will see that the root appears twice in two different octaves, for example.) While it appears here as an embedded object in a web page, I was also able to email it as an application. That way, he can keep it on his desktop and just double click on it to run and it can the run full-screen. He liked that since he can show it off better than if it just on a webpage. He said it seemed more personal that way. Online, it seems like it is for everybody. I wish I had more time to make it look a bit more like a card in this regard. My mom thought a splash screen at the start that says, "Happy Birthday!--Click here to begin..." or something like that would have improved it and made it more appropriate. But considering that 6 months ago, I had never really written a computer program, he was quite impressed and appreciated the effort that went into it. That was sort of the point, really.
Ideas for Improvements
I would like to do a number of things to improve it. I could add some other instrument sounds, perhaps as chords, that come in once in a while. I could add more 'modes' such as a triangle mode, or make more complex shapes, such as snowflakes. This will take some more study in how to draw the shapes, but shouldn't be too hard. (The snowflakes might be a nice Christmas gift to someone else, perhaps with a short, repeating animation of a person walking or something like that...) In the next version, I would like to have a different set of instruments or tonal range for each mode. Triangles should 'sound' different from circles and ellipses I would think. A mouse click/key-press would not only change the key and clear the screen, but have much more impact, changing the shapes, intensity of the music, colour range, instruments, etc... Finally, I would like to get it to run on a handheld device. This would make the project seem much more professional. I would include a start screen where the user would have some choices, or the very least, as noted above, a splash screen with a title would be an improvement. Processing sketches can be exported to Android devices and this is something I have done before, but I am not so sure about the MIDI library. This is something I will have to explore. A handheld offers some interesting possibilities since it has an accelerometer. The scene could change depending on how one holds and rotates the device. This could be a lot of fun!
Personal Reflection I feel good about how I created a plan and then executed it fairly systematically. I was able to explore a wide range of possibilities, then create a project that fit the requirements well and i was able to do it in time. In retrospect, I should have saved my work under different names along the way (step 1, step 2) which would have made it much easier to document my work and share what I did at each part. Then, I could have included various versions of the project that actually run for each stage and the viewer would have a really good sense of how I built it up one part at a time. My process journal could use some more detail regarding how each function works so that it could be of more educational value to the beginning programmer, but I think that the comments that I did put in the code should be relatively helpful.
I have been expecting my students to document their work in this format for a long time, but this is the first time that I have gone through the whole process, top to bottom, myself. It is a lot to expect and I am starting to understand why my students find this class to be one of the toughest that they take. I will try to be a bit more understanding in the future and give them more time and guidance as they do their written work.
|
