FORM1 Kinematic Gesture was produced by Linguistic Data Consortium (LDC) catalog number LDC2004V01 and ISBN number 1-58563-299-6.
FORM is a gesture annotation scheme designed to capture the kinematic information in gesture from videos of speakers. This publication is a detailed database of gesture-annotated videos stored in the Anvil and FORM file formats. FORM encodes the "phonetics" of gesture by giving geometric descriptions of location and movement of the right and left arms. Other kinematic information such as effort and shape are also recorded.
FORM gesture data has applications in statistical natural language processsing, gesture recognition and generation, information extraction from video, and human-computer interaction.
Please go to the FORM website for more information. The FORM2 publication was released in 2003 by the LDC, and encoded much of the same data provided here using a more recent tag set.
This publication contains gesture annotations created using the FORM 1.0 tag set. The Anvil annotation files used in their creation are also included, as are 29.5 minutes of the original audio and video recordings excerpted from a lecture given by Brian MacWhinney on January 24, 2000 at Carnegie Mellon University. A second data set, with 5.5 minutes of Paul Howard telling a story in conversation while being motion captured, is also supplied. These video recordings were chosen because they are part of the NSF-funded TalkBank project.
There are a total of 69 data files: 21 movie (.mov) files, 24 Anvil (.anvil) files, and 24 FORM (.form1) files. One video, jan24bm02f1.mov, was annotated 4 times, each by a diferent annotator. The annotations are labeled jan24bm02af1, jan24bm02bf1, jan24bm02cf1, and jan24bm02df1. These files are included for Inter-Annotator Agreement studies.
The movie files are in Quicktime format with the following specs:
| Size | 360 x 240 pixels |
| Compression | H.261 |
| Video rate | 29.97 fps |
| Audio rate | 48 kHz |
| Audio format | 8-bit/16-bit stereo |
Anvil files can be opened using the Anvil video annotation tool, which is freely available from Michael Kipp. A specification file that describes the FORM 1.0 tag set is also needed to open anvil files: form1.xml. All .anvil files have been validated by the DTD schema anvil.dtd. The specification file has been validated by spec.dtd.
The .form file format is an intermediate data format that contains only the FORM2 values from each .anvil in a comma-delimited, frame-by-frame listing of the following form:
frame,upper_arm_lift,forearm_orientation,handshape,wrist_up_down,wrist_side_side,effort,tension
A full description of the FORM tag set with explanations of each value can be found in annotation_codebook.pdf.
A chart displaying the various handshapes in FORM is provided in handshapes.pdf.
The handshape values are mapped to numbers in .form files in sequence. Thus, 0A=1, 0B=2, and so on, ending with 6I=51.
Please see the docs directory to consult the documentation.
Please see file.tbl for the directory structure of this publication, as well as a complete list of files.
This research was conducted using funding from the following grant sources:
ISLE - 9910603
NSF: Talkbank (via subcontract from Carnegie Mellon University) - BCS-9980009 and BCS-9978056
NSF: Discourse and Gesture w/ Joshi, Liberman, and Martell - EIA98-09209
Additional information, updates, bug fixes may be available in the LDC catalog entry for this corpus: LDC2004V01.
Please contact Craig Martell at cmartell@ldc.upenn.edu for information regarding this publication.
Due to restrictions imposed by copyright issues and third-party sources, all LDC corpora are governed by some kind of user license agreement. If you are a member of the LDC, your use of LDC corpora (barring any specialized corpus license) is governed by a Membership License Agreement. Please see your company or department administrator for a copy of this license or find the appropriate agreement on the LDC website.
If you or your organization are not a member of the LDC, then this corpus is governed by the following LDC End User License Agreement for Non-Members.
Certain LDC corpora are governed by a specialized End User License Agreement which supercedes any other member or non-member license agreements. This corpus does not require any specialized license.
Portions © 2004 Trustees of the University of Pennsylvania, © 2000 Brian MacWhinney