Tools: Java abc4j notation package, Jfugue, Jsound library, Java GUI, multi-thread programming

Based real time frequency analysis and amplitude tracking,I designed pitch approximation, rhythm approximation algorithm to export melody notation of singing voice. Non-musicians could use this program to write music by singing to the computer.       Full documentation

Tools: java, jfugue library

Implemented a self-modified Hidden Markov Model in java in the way that given a piece of melody, it’s able to firstly give a possible chord progression based on its own data base(manually typed txt files of existing music), then the user would be asked whether he/she is satisfied with each chord. Each time the user selects an unsatisfied chord, the program would take this into account and modify HMM model by strengthening the user’s choice chain in HMM network, re-evaluate the whole network and give a new chord progression, until the user is satisfied. In this way, the user could get reasonable chord progression usually within 3 times unsatisfactory selections. Full documentation

Tools: Pure Data, java, Arduino board, beagle board and 4 FSR sensors

Using 4 FSR sensors as both buttons and handles with tempo segmentation algorithm and prefixed MIDI melody notes text files generated by a java program written by myself, which can read any midi file and output the MIDI melody notes in specified format to be read by Pure Data. The user can press the FSR sensors in his/her own preferred strength(controlling volume and rhythm) to play the melody in text files. 

Human Singing proficiency research for new music interface design

Under the supervision of Prof Takako Fujioka, I conducted music perception research focused on singing proficiency among non-musicians. The purpose is to understand how human music perception research could provide insight on designing intuitive music interface for non-musicians. A scholar research paper was written for this research. Below is the PDF of the paper I wrote.

Voice-controlled music controller

Tools: Supercollider, a standard midi keyboard controller

Using pitch follower and amplitude follower to catch the two parameters of human voice input in real time and then using the two parameters to synthesize sound, enabling people to express their music ideas from voice instead of writing down notations, which requires ear training. And use a keyboard to trigger different timbre of synthesized sound and control volume.

Guitar chord controller for non-musicians

Tools: Pure Data, java, Arduino board, beagle board, 5 FSR sensors and 1 position sensor.

With 4 fixed-pitch virtual guitar sound signal input and real time pitch shifting object, built a virtual guitar that enables the user to control the volume, rhythm and pattern of prefixed chord progression. By pressing 4 FSR sensors(controlling 4 notes of a chord) with right hand fingers on user’s preferred position of position sensor(position controlling volume of each note of this chord), and by pressing 1 FSR sensor with left hand finger, the user can control when to go to the next chord to pitch shift the 4 right hand FSR sensors to control next chord’s 4 notes, thus controlling the rhythm. 

iPhone app projects

Tools: Xcode C-objective, C, C++ audio programming

Use audio programming to create an iPhone recorder, player, basic FM synthesis, interactive sound-image app.

An audio DSP tutorial: Understanding convolution for digital filters 

A 13-page tutorial was written trying to explain the fundamental and application of convolution assuming the reader has no prior experience of advanced level mathematics. Further writing on reverberation is in progress. Below is the tutorial I wrote.

Automatic guitar body production machine

This is a whole semester's group work for mechanical engineering design class. We designed, calculated and drew detailed machine to cut 20 blank board into 20 guitar body totally in automatic procedure, and financial justification, materials, presentation was done at the end of semester. Below is the PPT presentation file for this project.

Intelligent game player-“Connect 4” gamer and “Color ball” gamer 

Tools: java python

Designed two evaluation functions for two games, implemented alpha-beta algorithm to play with human and play with another computer.Full documentation

gesture detection using webcam

tools: C++ OpenCV

designed and implemented algorithms that delineate gestures in video images and analyze their movements. The vision-based techniques largely rely on color of the target objects and background. Full documentation

Threebody in universe simulation

Tools: Processing

Inspired by the famous Chinese science fiction "Threebody", I created 3 planet objects in the java based graphic programming language-Processing by implementing the gravity in universe with Newton's second law, mass, diameter, initial speed, positions of planets, etc. 

Smart Guitar patent-A guitar design for NON-Musicians

A traditional guitar (both acoustic and electronic) requires very complex left hand technique for countless hand positions for chords and strength to press strings firmly and stretch fingers. Many people give up guitar learning very soon because it hurts their fingers, especially for girls when it comes to F chord, which requires one finger to press all six strings together, and repeated practice tend to be boring. While music is fun, to play music becomes much harder and not accessible for non-professionals. There might be interesting music idea inside non-guitarist's mind, but it just can not be expressed, because he or she doesn't have strong muscle or good memory of hand position. This is not fair.   Full documentation

Water oscillation pipe organ project

Inspired by my Air Conditioning Design Engineering background, I measured and cut 8 plastic pipes in certain length and attach them right on the side surface of a water bucket, with one end of pipe closed by water surface. I made a hole on the back a coke cola bottle, fill it with water and put it right on top of the other end of pipes, With water crushing the water surface in the other end of each pipe, different length pipes would generate different pitch. In this case, it generates an octave scale with water flowing sound. And with slightly vibrating the depth of pipes into water, vibrato effect could be generated. 

Extracting water from air

Developed a new method of synthesizing complex water absorption material based on Carbon and calcium chloride to extract water from air for people to drink, with a paper and a patent published.