Interactive chord progression estimator project
Implemented a self-modified Hidden Markov Model in java in the way that given a piece of melody, it’s able to firstly give a possible chord progression based on its own data base(manually typed txt files of existing music), then the user would be asked whether he/she is satisfied with each chord. Each time the user selects an unsatisfied chord, the program would take this into account and modify HMM model by strengthening the user’s choice chain in HMM network, re-evaluate the whole network and give a new chord progression, until the user is satisfied. In this way, the user could get reasonable chord progression usually within 3 times unsatisfactory selections.
This is a self-designed project for my 2-week long workshop of algorithmic computer music in University of California at Santa Cruz, supervised by Prof David Cope.
I’ve always been interested in how technology could be applied to help non-musicians to compose and perform music. Since chord progression is always considered the backbone of tonal music, playing an important role, as well as an important barrier for non-musicians, to compose music, I started to investigate in how to develop a program to help non-musicians for this process.
Hidden markov model
For each style of music, the possible chord progressions are decided by two factors:
1. 1. The given melody
2. 2. The succession of chord
This inter-dependency structure of melody-chord happens to be similar to the hidden markov model, with melody as the state, chord as the “hidden state”. In addition, For a certain type of music, certain rules can be statistically encoded.
Previous work done by Microsoft: “Songsmith”
In 2008, Microsoft released a software called “Songsmith”, which applied Hidden markov model in music composition for non-musicians. It enables a non-musician to sing to the computer along with a pre-fixed rhythm, and does pitch approximation according to FFT algorithm, turning amplitude wave form into frequency form, and finally to pitch. With the given melody, a hidden markov model, which is built with statistical analysis of over 100 classical existing songs, is applied to the melody. Also, a “happy” factor is added as a parameter to control the subjective feeling of chord progression.
It’s an interesting application that enables people to write music without having to spend years to for ear training and music theory study. And the “happy” factor gives the user some control of the flavor of music composition.
However, I think non-musicians have more potentials of music creation and deserve more detailed control and representations of music. Here are some flaws of “Songsmith” in my opinion:
1. 1. The pre-fixed rhythm. When non-musicians hum some new melody from mind casually, they tend to sing not in fixed rhythm. The pre-fixed rhythm is having a negative influence to music creation. Even when people try to sing along with the pre-fixed rhythm, they often sing out of the rhythm, thus reducing the accuracy of pitch segmentation algorithm.
2. 2. The “Songsmith” interface does not give the user an interface to correct wrong pitch segmentation generated by computer. When the user find obvious errors, they still can not correct it.
3. 3. Due to the above two errors, hidden markov model will not give an accurate estimation of chord progression
4. 4. Even assuming the above three situation does not happen, the hidden markov model, which is based on existing songs, can never be tailored to the user’s personal flavor of music.
Improvement I did:
For the first two flaws, my project pitch detection from human voice gave a new solution. In this documentation, a personalized and controllable Hidden markov model algorithm is presented.
Description of the algorithm:
1. Analyze existing songs chosen by the user:
Take existing songs that the user personally chose for statistical analysis. Analyze the relationship between melody notes and chord of each song, as well as the probability of one chord jumping to next chord and one chord jumping from the last chord.
2. Train the hidden markov network:
Accumulate the analysis into one hidden markov model
3. Set certain restrictions of the model. Set a starting chord probability.In this case, in order to simplify the process, I set it to C chord as the start of any song after switching the melody the key of C.
3. Ask the user to input a new melody line, and decides where to cut the melody and add a chord, in order to apply hidden markov model.
4. After first evaluation of the hidden markov model. Present the user with output chord progression, and “ask” the user: which chord does not sound satisfying.
5. After the user selected the unsatisfying chord, present the user other possible chord selection here according to probability. Ask the user to choose one. And then “Strength” connection between the last chord and this new selected chord in the hidden markov network by amplifying the parameter by 100 times.
6. With the new HMM model, re-evaluate the input melody.
7. Repeat step 5, 6 until the user is satisfied.
Three objects were implemented:
1. harmonizer: It's an object of hidden markov model network matrix
2. chord: It represents a chord and stores the melody line it corresponds to
3. song: It represents a song consisting of chord objects
And here's the main framework of how the whole system works: