Rainbow Notes Help               Advanced features                    How to use Saved files

Why a Melody class      Functions of a Melody Class          Technical Stuff

The principle behind it is simple: the applet functions as an electronic sheet of music and can, when requested, convert this symbolic information into an array of byte values that will sound like the corresponding music when sent to the loudspeaker of your machine through your WWW browser.  Performing the actual byte-by-byte conversion is the tricky part, but that's nothing the end-level users will need to concern themselves with. Those who are interested can take a look at the rough description of it further down on this page.

For each melody, only the symbolic note data needs to be loaded from the server. That's a very compact way of representing sound. So compact, in fact, that I decided to write a Java class, named "Melody", for making use of these data files in general applets.

---

Rainbow Notes: Help

Click on any of the small buttons marked R, O, Y, G, B, I, V (for red, orange, yellow, ... Duh!) or press the corresponding key to select a sound type. Put notes on the sheet by placing the cursor where you want the main "dot" to be and press 1, 2, 3, 4, 6 or 8 for the appropriate length of the note in 1/8 units. Hold down the CTRL key if you want the note to be raised half a step.

All notes can be dragged up or down (keep this in mind -- it makes things a whole lot easier). They will vanish if you drag them off the sheet, or if you place the cursor over them and press delete. A whole bar can be cleared at once with the c key. The applet inserts pause marks at appropriate places and makes sure you don't do anything illegal. You are not allowed to play more than one note of the same color at once.

When you want to hear the result, click on the Make button. The applet will then compile the melody (which takes a few seconds -- you should be able to see the progress in the status bar at the bottom of your browser window) and play it from beginning to end. Once it's been compiled, you can play it without having to go through the computational process again if you just click on Play, but whenever you have made some changes (or if you just want to play it at a different speed) you need to click on Make again in order to hear the new version.

If you just want to hear how a single note at the current cursor position would sound, press p (and hold down SHIFT to get the note note raised half a step).

I assume I don't need to explain the use of the Load and Clear buttons. To switch to 3/4 or 4/4, you need to clear the sheet first.

---

Advanced features

You will notice that when you flip between voices, the envelope or waveform changes you make usually won't look exactly the same when you get back. That's because different browsers will assign differently sized drawing areas to these graphs, so it wasn't possible to find a curve data representation with a one-to-one correspondence to what you see on your screen. The applet consequently stores an approximation of what you've drawn, which will turn out slightly different when it shows up again.

When you "save" the finished melody, you don't get the result as a file (since that would violate the security restrictions of Java). You get a bunch of odd characters in the "Java Console" window. (Look under "Options" in Netscape.) Copy all of it into whatever text editor you've got and save it from there.

---

How to use the saved files

The resulting text file can be read by a particular melody object I've created, which can be included in applets and used in place of AudioClip objects.

For the source code of a very simple test applet using the Melody class, go here.

Why a Melody Class

The main advantage is storage space (consequently download time) and to some extent sound quality. A half-minute tune, taking up more than 230 kilobytes as a normal audio file (or 115 K at 2:1 compression) normally doesn't take more than about 2 K as note data. That's a significant gain if you've got a slow connection or a crowded network. However, to process the notes into audible data once they're in your computer takes a bit of time too (though not as much time as the "making" in Ranibow Notes, where I haven't optimized the algorithm), so the gain won't be quite that dramatic, but even for a run-of-the-mill machine it should beat downloading a normal audio file through a 28.8 modem hands down.

What about the sound quality? Well, since this audio information hasn't been recorded through a physical microphone, it doesn't have any static or any other real-world distortion. It sounds clearer, but then again the things you can create with the melody editor are rather limited and primitive. They might not live up to the coolness you'd want for your own applet sounds. So go write your own melody editor! It's not that difficult.

Functions of a Melody Class

Melody just loads the note data and converts it into an audio byte array. You can then play that by sending it as an AudioDataStream to the AudioPlayer. Each melody object has an InputStream variable called soundStream, which is preferably used for that purpose.

mel.soundStream=new AudioDataStream(new AudioData(mel.rawAudio));
AudioPlayer.player.start(mel.soundStream);

If you'd rather "loop" the melody, create a ContinuousAudioDataStream instead:

mel.soundStream=new ContinuousAudioDataStream(new AudioData(mel.rawAudio));
AudioPlayer.player.start(mel.soundStream);

You can stop it by calling the AudioPlayer.player.stop() method:

AudioPlayer.player.stop(mel.soundStream);

Technical Stuff: Building Sound from Scratch

An AudioDataStream (or ContinuousAudioDataStream) object is based on an array of byte values (0-255) representing sound amplitudes in the range -8192 to 8191. The applet first computes the latter value by adding together sound contributions from all voices (which in turn are products of waveform and envelope values), and then picks the final byte value from a lookup table prepared during initialization. This has to be done 8000 times for each second of resulting music, which is why it takes a while before you get to hear anything.

If we have two voices, red and blue, defined by the waveform and envelope curves

and

and a bar with these notes

the note data is converted into an amplitude curve like this:

(In reality the curve oscillates much more quickly, but this is just a schematic illustration.)

Now, this data in turn needs to be converted to a form the AudioPlayer Java class can understand. Assuming I've got the rules behind the byte value conversion right, this is how it's intended to work:

The maximum volume I've given any voice is 4032. This creates a problem: what happens if three or more voice sound data contributions add up to more than 8191 (or less than -8192)? I avoid this by first searching through the note data for such occurrences, find the theoretical maximum volume and use that to compute a scale factor I later use throughout the making of the melody. The inevitable result is that if you use many voices at the same time, they will have to "share" the max volume and each one will sound slightly weaker than if they had been played individually.

Rainbow Notes Help               Advanced features                    How to use Saved files

Why a Melody class      Functions of a Melody class         

free2explore.com