Automated Objective Metrics and Instruments
for Motor Assessment, Therapy, and Rehabilitation
MARS – Motor Analysis Realtime System
VRAMS – Virtual Reality Analysis of Motor Skills
MMP – Multidimensional Motor Profile
A. Specific Aims
The availability and effectiveness of practical, affordable, automated objective techniques for measuring motor performance of
Parkinson’s Disease patients may offer the opportunity to maximally manage incremental changes in patient performance and
to offer an accurate, quantitative method for measuring independent symptom severity for improved quality of life through
appropriate clinical interventions in the form of drugs and or surgery. Existing measurements are expensive, offer gross scales,
are flawed, suffer from inter-tester variability, and are highly subjective. Patients’ clinical signs are multidimensional, vary
over relatively short time frames, and over the course of the disease and as a result of treatment. It is not possible to speak of
incremental, individual symptoms, or minimal clinically relevant changes with a Unified Parkinson’s Disease Rating Scale
(UPDRS) which has a five-point scale and lumps multiple symptoms into single categories. A new method is needed that
offers hope for assessing motor skills more precisely. We propose a new system, Motor Analysis Realtime System (MARS)
and method including the use of a video camera architecture to capture motion samples. With automated image processing
techniques1, we will measure and characterize arm, hand, and finger movement including various characteristics of tremor,
fatigue, bradykinesia, festination, freezing, re-emergent tremor, and rigidity. We will analyze discrete measurements,
correlation coefficients, variability, etc. We will create a scale for the measurements, the Multidimensional Motor Profile
(MMP) for use in the clinical environment, and we will calibrate the measurements with the existing test2, the UPDRS. These
measurements will be available to additional applications including but not limited to the Virtual Reality Analysis of Motor
With the new MMP testing instrument, clinical intervention may be expanded to include increased frequency and modification
of drug dose and administration for improved “on time” statistics, better determinations and target identification for surgery,
and balancing of drug therapy and stimulation levels for deep brain stimulation patients, and ultimately, better overall patient
motor management. The hypothesis has been developed to support clinical and health policy choices and to test questions
faced by clinicians in their practice with Parkinson’s Disease patients. Of course, MARS may benefit other populations
including those who have had stroke or essential tremor, where motor skills are an issue. This tool is meant to provide
capabilities to objectively measure incremental and progressive changes in motor conditions thereby characterizing motions,
indicating refractory, progressive, or other clinically significant symptoms perhaps even before they may be apparent in clinical
assessment. The MARS tool and its architecture are cost effective, easy to use, and are designed to improve access for patients
and clinicians. MARS may be integrated into other systems to provide measurement feedback into applications including but
not limited to diagnostic, therapeudic, and rehabilitative, as in VRAMS.
Objectives of the study include: 1) to determine the resolution of measurements of amplitude, frequency, correspondence,
phase, and distance of movement using video input and image analysis (MARS, VRAMS); 2) to create a scoring scale for
motions measured, tremor analysis (dimensions, latencies, re-emergence), festinations, freezing, fatigue, bradykinesia,
variability analysis, and correlation coefficients using the new video technique (MMP); 3) to calibrate measurements from
video techniques with the traditional Unified Parkinson’s Disease Rating Scale (UPDRS) testing instrument; 4) to determine
measurements of motion using video during Chopstick Task.
Goals of the study are to capture video of Parkinson’s Disease patients and control subjects performing the tasks of the Unified
Parkinson’s Disease Rating Scale and the Chopstick Test and to analyze these video images using image processing techniques
to determine measurements of arm, hand, and finger movement. This exercise will determine that the technical aspects of the
architecture function as expected. The processed information will be used to create a scoring scale including heterogeneous
dimensions associated with tremor. The MMP will be created. The scoring scale will then be correlated to the UPDRS testing
instrument. The Chopstick Task is designed to focus on an optimal method of capturing important tremor associated
characteristics for motion processing. This study will determine whether or not this instrument provides improved signal for
Long term, the MARS system is expected to provide scoring of tremor characteristics for clinical applications: diagnosing
essential tremor or Parkinson’s Disease, indications for adjusting drug intervals, identifying target locations for electrode
placement and treatment with deep brain stimulation, and providing feedback including realtime patient performance statistics
Segen, J., Controlling Computers with Gloveless Gestures, Gesture VR: Vision Based 3D Hand Interface for Spatial
Interaction, International Conference on Automatic Face and Gesture Recognition, June 1995, pages 166-171.
Louis, E.D., Levy, G., Cote, L.J., Mejia, H., Fahn, S., Marder, D., Diagnosing Parkinson’s Disease Using Videotaped
Neurological Examinations: Validity and Factors that Contribute to Incorrect Diagnoses, Movement Disorders, 2002 May;
for Virtual Reality physical therapy applications, and accessibility. Additional applications will be investigated, and the system
will be extended to include additional motor skills such as the lower limb.
In addition, analysis of the patients and their tremor characteristics is expected to lead to greater understanding of the
progressive nature3 of the diseases by investigating independent tremor characteristics, transitions, the nature of re-emergent
tremor (which flies under the radar screen of the UPDRS), time delays 4, growth or decays of oscillations amplitude and
frequencies, duration, correlations, bilaterally or unilaterally5, etc. festination, fatigue, and freezing, without, under adequate
treatment or insufficient treatment conditions6, in the short term and over the course of the diseases.
B. Background and Purpose of the Study
Parkinson’s Disease (PD) is a serious, chronic and progressive neurodegenerative condition effecting the quality of life of
approximately one million persons in the US accounting for 0.3% of the total population, 3% of those over 65, and 10% of
those over the age of 80. As the third most common neurological diagnosis in outpatient neurology clinics behind headaches
and seizures, 50,000 to 60,000 new patients are diagnosed annually. 7 PD patients have bradykinesias and either tremor at rest
or rigidity, with postural instability developing in later stages and autonomic nervous system symptoms that interfere with
activities of daily living. Postural tremor is a more debilitating form seen in PD patients where re-emergent characteristics
differentiate this tremor from essential tremor 8. Deep brain stimulation (DBS) involves the use of a stimulator implanted in a
pocket in the chest and electrodes inserted into the brain. DBS delivers treatment by means of high frequency, pulsatile,
electrical stimulation to a selected target, either the VIM thalamus, subthalamic nucleus (STN) or globus pallidus (Gpi), for the
adjunctive treatment of advanced stages of Parkinson’s Disease used in combination with drug therapies to reduce some of the
symptoms of advanced levodopa-responsive PD that are not adequately controlled with medication.9
Diagnosis, staging, and adequate monitoring of motor disorders is a prerequisite for successful clinical intervention including
surgical decisions. Instruments used to monitor, measure extent, and detect changes in motor disorders include: Unified
Parkinson’s Disease Rating Scale (UPDRS), Modified Hoehn and Yahr Staging, Schwab and England Activities of Daily
Living Scale, and others. Differences reported in the success rates of treatment including drugs or surgery might be influenced
by inter-rater differences, subjectivity of the testing instruments, day-to-day differences in patient symptoms, unmeasured or
reported heterogeneity of symptoms, or patient self-reporting.
The Movement Disorder Society Task Force for Rating Scales for Parkinson's Disease critiqued the UPDRS reporting
weaknesses: several ambiguities in the written text, inadequate instructions for raters, some metric flaws, and the absence of
screening questions on several important non-motor aspects of PD. The Task Force recommends the development of a new
version of the UPDRS and encourages efforts to establish its clinimetric properties, especially addressing the need to define a
Minimal Clinically Relevant Difference and a Minimal Clinically Relevant Incremental Difference. Of course the new metrics
should demonstrate correlation with the current UPDRS.10
Glass, L., Synchronization and Rhythmic Processes in Physiology, Nature, 410 (2001), 277-284.
Jankovic, J., Schwartz, K.S., Ondo, W., Re-emergent Tremor of Parkinson’s Disease, Journal of Neurosurgical Psychiatry,
(1999): 67: 646-650.
Calzetti, S., Baratti, M., Gresty, M., Findley, L., Frequency/amplitude Characteristics of Postural Tremor of the Hands in a
Population of Patients with Bilateral Essential Tremor: Implications for the Classification and Mechanism of Essential Tremor,
Journal of Neurological Psychiatry. 1987 May; 50(5):561-7.
Beuter, A., Titcombe, M.S., Modulation of Tremor Amplitude During Deep Brain Stimulation at Different Frequencies, Brain
and Cognition 53 (2003), 190-192.
Deep brain stimulation for Parkinson’s disease: an update, Medscape, 2003.
Jankovic, J., Schwartz, K.S., Ondo, W., Re-emergent Tremor of Parkinson’s Disease, Journal of Neurological Psychiatry,
Limousin, P, Krack, P, Pollack, P, Benazzouz, A, Ardouin, C, Hoffmann, D, Benabid, A, Electrical Stimulation of the
Subthalamic Nucleus in Advanced Parkinson’s Disease, N Engl J Med, Volume 339(16) October 15, 1998.
Collective Name: Movement Disorder Society Task Force on Rating Scales for Parkinson's Disease, The Unified
Parkinson's Disease Rating Scale (UPDRS): status and recommendations, Mov Disord 2003 Jul;18(7):738-50.
Current tests provide gross scoring categories. For example, the UPDRS scores range from zero to four. It is not possible to
speak of small or incremental changes in patient performance using gross metrics. It is also not possible to account for the
heterogeneous components of complex motions such as tremor, bradykinesia, festination, fatigue, and freezing. Video offers
us vast opportunity to precisely measure motion and create refined clinimetric instruments that can objectively quantify small
differences in amplitude or frequency, performance over time, left-side, right-side differences, correlation coefficients, re-
emergence, delays, and other properties of the subject’s movement. 11
Patients with Parkinson’s Disease experience dynamic changes in their motor control – they wait for “on” conditions so they
can enjoy more control of their motor skills. With better drug and DBS management, patient “on” time can be improved from
38% to 72% while ADL scores improve by 27%.12 Testing techniques that are administered infrequently or which don’t
represent the total experience of the patient may not offer the best representative picture of the person’s motor condition. With
more refined and incremental objective metrics, it may be possible to create a heterogeneous motor profile and precisely
determine objective performance metrics, to take samples more often, to detect refractory symptoms more quickly, to
determine disease progression, or to isolate specific disease indicators and as a result, to fine tune treatment including adjusting
drug dosage and administration periods and frequency, and or to target electrode placement, or adjust electrode stimulation
patterns and periods for DBS, thus improving quality of life for Parkinson’s Disease patients. Affordable automated objective
techniques for measuring motor performance calibrated with existing test methods may offer the opportunity to maximally
manage incremental changes in patient performance and to offer quick intervention for improved quality of life. Clinical
intervention may be expanded to include increased frequency of drug administration, better targeting and resolution of
administration of deep brain stimulation, etc. and ultimately, better overall patient motor management. Future work may
extend MARS to include facial recognition image processing, gait analysis, and speech analysis.
The same metric techniques applied to Parkinson’s Disease patients can be applied to patients with stroke and other motor
disorders. With an improved method of measuring motion by using video cameras and image processing and automated
objective evaluative techniques, virtual reality therapy and rehabilitation applications may be implemented, providing a wide
range of treatment alternatives to people with motor disorders at a lower cost and with greater access. Reach of clinicians into
community locations, assisted living, and patient homes (tele-rehabilitation)13 can be improved through the use of video testing
C. Preliminary Studies
Five Parkinson’s Disease patients and five age-matched control subjects will be recruited from the community, from
Parkinson’s Clinical Programs, and from the Neurology Clinic of UMDNJ to participate in the preliminary study to calibrate
the MARS and VRAMS software. The participants must give informed consent, and the local ethics committee must approve
the study. The participants will be asked to complete the UPDRS, the Chopstick Test, and the Survey. MARS will record the
participants and the VRAMS software will analyze the motions.
PD patients will be tested before medication in the morning, then one hour after medication. Deep brain stimulation subjects
will be tested off medication in the morning, then one hour after medication with effective stimulation level, off stimulation,
then effective stimulation, then ineffective stimulation, then off, then ineffective (E, O, E, I, O, I). The entire test is expected to
take no more than four hours for participants who must wait for one hour between tests. For control subjects, the study should
take less than two hours. More precise timing for test duration will be determined during this preliminary study.
Segen, J, Controlling Computers with Gloveless Gestures, Gesture VR: vision-based 3D hand interface for spatial
interaction, International Conference on Automatic Face and Gesture Recognition, June 1995, pages 166-171.
Pahwa, R, Wilkinson, SB, Overman, J, Lyons, KE, Bilateral subthalamic stimulation in patients with Parkinson’s disease:
long-term follow up. J Neurosurg. 2003 Jul;99(1): 71-7.
Merians, A.S., Jack, D., Boian, R., Tremaine, M., Burdea, G.C., Adamovich, S.V., Recce, M., Poinzner, H., Virtual Reality-
Augmented Rehabilitation for Patients Following Stroke, Physical Therapy, September 2002 Volume 82(9)898-915.
For objective one, video will be examined using image processing. Measurements will be made of amplitude,
frequency, correspondence, phase, correlation of coefficients, re-emergent and delay characteristics, and distance of
movement using video input and image analysis. Variance of amplitude modulation (vam) time series for the left and
the right side will be calculated. The two will be compared. Mean relative phase (mrp) and the variance of relative
phase (vrp), the mean and variance of the relative time series indices will be calculated. Mrp and vrp will be defined
by the mean and variance of the absolute value of difference between and 180(i.e. - 180), respectively.
Mean slope of regression lines (mrl) and variance of slope of regression lines (vrl) will be calculated to extract
information reflecting cross-correlation between the left and right velocities. LR-plots will be developed.14 Motion
delays will be recorded and analyzed. The above described motion analysis will be carried through all portions of the
For objective two, results from the image processing evaluation will be organized into a scoring instrument. The goal
is to be able to reliably measure the five possible scores for the UPDRS test. The researchers would like to increase
the possible scores reliably measured to more than four for each movement category. Score clusters will be evaluated.
Intervals will be determined. Characteristics of the intervals will be enumerated. Tremor characteristics will be
isolated and scored. Bradykinesia and fatigue characteristics will be isolated and scored. Heterogeneous
Multidimensional Motor Profiles will be created.
For objective three, scores from the image processing instrument will be compared with the Parkinson’s Patient
records as a means to calibrate the tests to the existing gold standard.
For objective four, the Chopstick Test results will be analyzed to determine if more motion data captured by MARS
was processed more easily using the chopstick to track tremor. The chopstick task does not require skill, expense, or
In a previous study by Bronte-Stewart, et. al., a digital keyboard was used to quantify digit movement in Idiopathic Parkinson’s
Disease patients. The patients were instructed to perform 60 seconds of alternating finger tapping. Key, time of the strike, and
velocity of the strike were recorded. Only activation and not displacement of the key was recorded. Bronte-Stewart, et. al.,
were successful in testing and scoring participants, measuring differences before and after medication or other treatment, and
measuring the differences between PD patients and control groups, even when patients were effectively treated. 15 The test
scored festination, freezes, and fatigue separately. In existing tests such as the UPDRS, for repetitive finger tapping, three
qualitative areas of motor dyscontrol are noted by a single score, “2” which is defined as: “Moderately impaired. Definite and
early fatiguing. May have occasional arrests in movement.” Bronte-Stewart points out that for repetitive movement tasks,
patients who are moderately bradykinetic but who maintain the rhythm may be given the same score as patients who can
perform the repetitions more rapidly but with several pauses or freezes in the ongoing movement and the differences in
frequency and amplitude cannot be quantified. We believe that is a characteristic of the UPDRS that makes the test ineffective
in a clinical environment, for determining drug therapy doses and intervals, determining changes or progress of disease state,
for planning surgery, targeting locations for deep brain stimulation, and for effectively treating the patient. These are the very
symptoms clinicians are attempting to treat and yet the testing instrument fails to isolate or objectively measure them.
However, Bronte-Stewart, et. al., have had success in QDG measurements and correlation of these measurements with the
UPDRS. Tests traditionally lump patients into sub-groups based on predominance or not of tremor. Bronte-Stewart’s research
suggests that patients “display a heterogeneous set of motor profiles under the domains of bradykinesia and fatigue.”
From previous studies by Jankovic, et. al., we can expect to measure start time of postural tremor, latency of re-emergent
tremor, and to compare tremor between affected and less affected limbs.
Important image processing work by Segen offers an inexpensive and effective platform upon which to develop the MARS and
VRAMS software. The work of Segen, the success of Bronte-Stewart, and the perceived need inspired the design of the
MARS and VRAMS and motivated this study.
D. Research Design Methods
Thirty Parkinson’s Disease patients and thirty age-matched control subjects will be recruited as described above to perform the
tasks in the UPDRS, the Chopstick Test, and the Survey. All participants will be required to give informed consent in order to
participate in the study. The study must be approved by the local ethics committee. Participants will be asked to report to the
laboratory in the East Building on the campus of NJIT in Newark, New Jersey.
Abe, K, Asai, Y, Matsuo, Y, Nomura, T, Sato, S, Inoue, S, Mizukura, I, Sakoda, S, Classifying lower limb dynamics in
Parkinson’s disease, Brain Research Bulletin 61 (2003) 219-226.
Bronte-Stewart, H. MSE MD, Ding, L. MS, Alexander, C. BA, Zhou, Y. PhD, Moore, G.P. PhD, Quantitative Digitography
(QDG) – a sensitive measure of digital motor control in Idiopathic Parkinson’s Disease, Movement Disorders, 2000 (15) 36-47.
The study will be conducted as described in Preliminary Studies, above. The same steps and measurement instruments will be
used. The software is expected to be stable during this part of the study.
The MARS architecture will consist of: two video cameras, a computer, and image processing software. Additional equipment
will include: black velvet lap cloth, table cloth, and chopsticks. Image data will be collected at a rate of 60 half frames per
second. Control subjects in one motor skills test tapped their finger at a mean rate of 4 strikes per second per finger striking
475 times in 60 seconds.
To analyze test results, for comparison of performance within patients on and off medication, two tests will be used. For data
that is expected to be normally distributed, a paired T-test will be used to assess the statistical significance of any differences in
the mean. Else, a Wilcoxon Signed Rank Test will be used. To compare the difference in performance between patients (on or
off medication) and control subjects, an independent T-test will be used to determine the statistical significance of the
difference in the mean duration or mean interval.
E. Human Subjects
Subjects will be recruited as described above, from the community, from Parkinson’s Disease Clinics, and from the neurology
clinic at UMDNJ. All study participants will be required to give informed consent. Before human participation, the local
ethics committee must approve the study. Participant privacy will be maintained. Identification numbers will be assigned to
each participant and the study doctor will have the key. The records will be kept locked in the Biomedical Engineering Lab in
the East Building, 6th floor. The subjects will be recruited from the community and from appropriate clinical situations.
Control subjects must have no evidence of neurological disease. Parkinson’s Disease patients must have a score on the
UPDRS. Anyone for whom the tasks on the test are painful or too difficult will be excluded from the study. Pregnant women,
and prisoners will be excluded from the study. Children will not be recruited for the study because there are few children who
have been diagnosed with Parkinson’s Disease.
F. Vertebrate Animals
No animals will participate in this study.
G. Literature Cited
SUMMARY OF PROPOSED PROJECT IN SCIENTIFIC TERMS:
Automated Objective Metrics and Instruments for Motor Assessment, Therapy, and Rehabilitation
How will the study be analyzed?
Objectives will be analyzed to determine outcome of the study.
For objective one, video will be examined using image processing. Movements will be measured.
The measurements will be recorded and reported.
For objective two, results from the image processing evaluation will be organized into a scoring
instrument. The goal is to be able to reliably measure the five possible scores for the UPDRS test.
The researchers would like to increase the possible scores reliably measured to more than five for
each movement category.
For objective three, scores from the image processing instrument will be compared with the
Parkinson’s patient records as a means to calibrate the tests to the existing gold standard.
WHAT ARE THE POTENTIAL BENEFITS TO SUBJECTS OR OTHERS?
There are no known benefits to the subjects by participating in the study. However, if the
new measurement technique is demonstrated to be sensitive and specific, patients with
Parkinson’s Disease or other motor disorders may benefit as a result of clinical
application of this testing technique and its applications to assessment, diagnosis,
treatment, therapy, and rehabilitation.
Patients with Parkinson’s Disease experience dynamic changes in their motor control –
they wait for “on” conditions so they can enjoy more control of their motor skills.
Testing techniques that are administered infrequently or which don’t represent the total
experience of the patient may not offer the best representative picture of the person’s
motor condition. With more refined and incremental objective metrics, it may be
possible to precisely determine objective performance metrics, to take samples more
often, to detect refractory symptoms more quickly, or to determine disease progression,
and as a result, to fine tune treatment including adjusting drug dosage and administration
periods and frequency, and or to adjust electrode stimulation patterns and periods for
DBS, thus improving quality of life for Parkinson’s Disease patients. Affordable
automated objective techniques for measuring motor performance calibrated with
existing test methods may offer the opportunity to maximally manage incremental
changes in patient performance and to offer quick intervention for improved quality of
life. Methodology and instruments may be included in rehabilitation environments
including but not limited to: clinical, tele-rehabilitation, community, etc. Including
affordable automated rating techniques in the Virtual Reality rehabilitation environment
will increase the potential for dynamic feedback and the integration of that feedback into
the rehabilitation applications. Clinical intervention may be expanded to include
increased frequency of drug administration, better resolution of administration of deep
brain stimulation, etc. and ultimately, better overall patient motor management.
Methodology and instruments may be included in research environments to assist in
providing objective motion analysis of various neurological states and interventions.