...Click left to dowload non-audible screen capture of the implementation software...
In this thesis, 3D animation of human facial expressions and lip motion and their synchronization with a Turkish Speech engine using JAVA programming language, JAVA3D API and Java Speech API, is analyzed. A three-dimensional animation model for simulating Turkish lip motion and facial expressions is developed.
In addition to lip motion, synchronization with a Turkish speech engine is achieved. The output of the study is facial expressions and Turkish lip motion synchronized with Turkish speech, where the input is Turkish text in Java Speech Markup Language (JSML) format, also indicating expressions.
Unlike many other languages, in Turkish, words are easily broken up into syllables. This property of Turkish Language lets us use a simple method to map letters to Turkish visual phonemes. In this method, totally 37 face models are used to represent the Turkish visual phonemes and these letters are mapped to 3D facial models considering the syllable structures.
The animation is created using JAVA3D API. 3D facial models corresponding to different lip positions of the same person are morphed to each other to construct the animation.
Moreover, simulations of human facial expressions of emotions are created within the animation. Expression weight parameter, which states the weight of the given expression, is introduced.
The synchronization of lip motion with Turkish speech is achieved via CloudGarden®’s Java Speech API interface.
As a final point a virtual Turkish speaker with facial expression of emotions is created for JAVA3D animation.
Keywords: 3D facial modeling, facial animation, lip motion, lip/speech synchronization, facial expression simulation.
INPUT
Turkish text and Facial Expression information in JSML. The Java™
Speech Markup Language (JSML) is used by applications to annotate text
input to Java Speech API speech synthesizers.
MORPHING
Animation of 3D shapes involves the change of vertex attributes overtime.
Morphing represents such changes over time as the interpolation of two
given shapes. We can extend this concept to more than two base shape and
use morphing to produce blend of several objects. In this study, a Turkish-speaking
agent is constructed using the simple idea of morphing. The animation
is built using JAVA3D machine.
FACIAL EXPRESSIONS
During animation six basic emotions (namely; sadness, happiness, surprise,
anger, fear and disgust) are added to the speaking agent using a weighted
morphing method.
ALIVENESS EFFECTS
Aliveness effects such as eye blinking, head movement and eye movement
are added to the animation structure providing a more realistic facial
animation. Also these effects can be conttrolled within the animation
console, such as setting the gaze direction to 2 degrees left and 1 degree
up.
SPEECH SYNTHESIS
A speech synthesizer is a speech engine that converts text to speech.
In this study we have used JSAPI (Java Speech API) to synthesize Turkish
phonemes synchronized with 3D animation.
OUPUT
Turkish-speaking 3D agent constructed in JAVA3D and JSAPI, compatible
to any virtual environment.
Ugur HALICI {halici@metu.edu.tr}
Erdem Akagündüz {erdema@metu.edu.tr}
Middle East Technical University
Electrical & Electronics Engineering Dep.
http://vision1.eee.metu.edu.tr/~vision