CTSA Biostatistics/Epidemiology/Research Design (BERD)
Key Function Committee (KFC) Meeting
November 20-21, 2008
CTSA BERD KFC Face-to-Face Meeting
Attendees: S. Zeger, Johns Hopkins (Chair); D. Dixon, NIAID (NIH Coordinator); I. Obrams, NCRR
(NIH Coordinator); P. Wakim, NIDA (NIH Coordinator); C. Ahn, UT-Southwestern; M. Banach, UCSF;
G. Beck, CWRU; L. Beckett, UC Davis; R. Bhore, UT-Southwestern; D. Bui, UCSF; H. Cabral, Boston;
C. Chaisson, Boston; R. Chang, Northwestern; E. Collier, NCRR; D. Couper, North Carolina; A.
Cucchiara, UPenn; D. del Junco, UT-Houston; D. DeMets, Wisconsin; R. Dominik, North Carolina; J.
Dziura, Yale; S. Fernandez, Ohio State; J. Fine, North Carolina; T. Greene, Utah; M. Griswold, Johns
Hopkins; F. Harrell, Vanderbilt; N. Geller, NHLBI; M. Gezmu, NIAID; E. Harris, NIDCR; P. Heagerty,
Univ. of Washington; N. Hills, UCSF; L. Hindorff, NHGRI; G. Howard, Alabama-Birmingham; S. Hui,
Indiana; S. Hurwitz, Harvard; D. Jarjoura, Ohio State; L. Johnson, NCCAM; M. Kim, Einstein; J.
Kittelson, Colorado Denver; M. Kutner, Emory; P. Lavori, Stanford; M. Lindstrom, Wisconsin; Y.
Lokhnygina, Duke; G. McCabe, Indiana; P. Miller, Wash U.; M. Mori, OHSU; C. Moy, NINDS; J.
Murphy, Colorado Denver; J. Myles, Michigan; D. Oakes, Rochester; C. Patterson (Moore), Pittsburgh;
S. Perkins, Indiana; B. Pollock, UT-San Antonio; A. Quarshie, Emory; M.H. Rahbar, UT-Houston; D.
Rubio, Pittsburgh; J. Stephens, Ohio State; P. Stewart, North Carolina; B. Talbot, NCRR; R. Thisted,
Chicago; S. Thurston, Rochester; H. Tiwari, Alabama-Birmingham; J. Torner, Iowa; J. Ware, Harvard;
K. Wittkowski, Rockefeller
Day 1: Thursday, November 20, 2008
I. Introduction to CTSA Governance and Key Function Committees
Drs. Zeger and Obrams welcomed BERD KFC members to the meeting. Dr. Obrams provided brief
descriptions about the overall CTSA mission, governance and organizational structure. She also
described the four CTSA strategic goals and subgoals (listed below), which evolved as a result of a
strategic planning process.
A. Enhancing national clinical and translational research capability
1. Clinical research management
2. Research infrastructure
3. Phenotyping – human and preclinical models
B. Enhancing the training and career development of clinical and translational scientists
C. Enhancing consortium-wide collaborations
1. Social networking
2. Inventory of resources
3. Data sharing
D. Enhancing the health of our communities and the nation
1. Community engagement
2. Public health policy
CTSA BERD KFC Face-to-Face November 20-21, 2008 1
II. Panel Presentations by 2006 and 2007 Cohort Member Volunteers
A. Dr. Michael Griswold, Johns Hopkins University
Dr. Griswold described the experience at Johns Hopkins University. He shared that the three
components of the grant are service, innovation and education. He also discussed what has
worked, what has not worked as well as inter/intra-CTSA collaborations.
1. What Works
a. Biostatistics Clinic: Free one-hour biostatistics clinic in a coffee lounge
b. Advertisements: Used large plasma screens throughout the campus to spread the
c. Generating a Wiki: Increased communication within the biostatistics core.
d. Translational Forums: Allowed for thesis practice and feedback from a collegial
e. Open Courseware: Though this has been in place since 2006 before the CTSA
grant, this has been very successful.
2. What Did Not Work
a. Supply: 4.5 FTEs with an infinite demand
b. Tracking Effort and Outcomes
c. Administration – Centralized vs. Decentralized: How do you incentivize to get
people to work together?
d. Research and Translational Career Paths
B. Dr. Doris Rubio, University of Pittsburgh
Dr. Rubio described that the core is structured knowing that there is no way to supply all demand.
Therefore, they collaborate with the different departments for Statistics, Biostatistics and
Epidemiology to be able to offer more services for investigators. Since she is the director of a data
center, she is able to offer statistical assistance and provide consultation for data management.
Since they cannot meet all demand, her institution works with the investigator to make sure they
are clear about what services they can provide.
1. What Works
a. Guest speakers who conduct training for statisticians
b. Pairing investigators with biostatisticians based on focus area and expertise
2. What Did Not Work
a. Supply: 2 graduate student researchers, 1 full-time Masters, 3 Ph.D. (1.5 FTE)
b. Striking the balance on how much work to do for the investigator
C. Dr. Tomi Mori, Oregon Health and Science University
Dr. Mori described the mission, aims, vision and values of the Oregon Clinical and Translational
Research Institute (OCTRI). Dr. Mori expressed that the demands at OCTRI far exceed the
1. What Works
a. Strategic and business planning: led to setting more realistic goals
b. Emphasis on collaboration and pre-award support (for Ks and first-time R01s)
c. Targeted outreach with partnering institution
d. Drop-in consulting sessions
2. What Did Not Work
a. Strategic and business planning: time-consuming and labor-intensive
b. Ambitious goals and objectives that do not match resources
CTSA BERD KFC Face-to-Face November 20-21, 2008 2
c. Front door requirement can be a barrier
d. Enforcing strict business model for service center – discouraged some investigators
e. Establishing complicated eligibility criteria for free services – tough to define what is
a clinical and translational research project
f. Complex tracking system – tough to obtain faculty adoption
3. Inter/Intra CTSA Collaborations
a. Intra - Collaborations with Biomedical Informatics, Community Engagement and
other centers and institutes at OHSU
b. Inter - West Coast CTSA Consortium which resulted in sharing benchmarking data
and educational materials
1. Assistance for Junior Investigators: The expectation for the majority of institutions is
that if a grant is awarded to collaborators, then they are expected to reimburse for
2. Biostatistics Staffing Models: Most institutions have a small amount of time from a large
number of BERD staff, while at a few institutions a small number of people offer
significant collaborative time.
3. Definition of a CTSA Investigator: Most institutions felt that if an investigator has a
study that was reviewed by a scientific advisory committee, then that investigator is a
E. Advice for New Cohort
1. Be realistic about program goals and objectives given your resources
2. Implement a simple, easy-to-use tracking system
3. Remember key measures of success and let them guide your institution where your
resources should go
4. Hire a professional web programmer
III. Discussion by 2008 Cohort Members
During the discussion, two key themes arose. First, since Biostatistics is central to the discipline of
translation, it should be in the center of the CTSA. Second, the majority of members from the new
cohort indicated that they are more interested in collaborative efforts rather than in providing
consultations. Members also identified a set of common aims and challenges.
1. Conduct biostatistical methods research
2. Train junior biostatisticians
3. Facilitate Biostatistics graduate programs
4. Amass statistical tools and content on CTSA Wiki
5. Leverage eCollab
6. Have distributed group of federated biostatisticians in basic science
1. Adjust to increasing demand by creating online courses and posted case studies
2. Connect with Informatics and other disciplines
3. Track time effort
4. Evaluation criteria
CTSA BERD KFC Face-to-Face November 20-21, 2008 3
IV. BERD Evaluation Group Discussion
Participants reviewed the latest iteration of the Evaluation Guidelines document. Dr. Rubio described
that the document can be treated as a guide for members to track services and collaborations at their
institutions and that it is a living document. She shared that the effort was initiated as a result of a
discussion at the 2007 CTSA BERD Annual Meeting in which members felt strongly about playing an
integral role in setting metrics.
Following a review of the document’s key operating principles, members then provided suggestions to
the existing sections.
A. Section 1 – Collaborative Activities
B. Section 2 – Grants Submitted and Grants Funded
C. Section 3 – Protocols Developed and Reviewed
D. Section 4 – Abstracts/Manuscripts Submitted and Accepted
E. Section 5 – New Methodology Developed
F. Section 6 – Educational Activities, Number of Courses and Students
G. Additional Survey?
Day 2: Friday, November 21, 2008
V. Introduction to Full BERD Meeting
Dr. Hayward noted that it has been two years since the CTSA program started and briefly reviewed the
process by which the CTSA Consortium Steering Committee took to select priorities and arrive at the
four strategic goals. He advised the BERD KFC to continue to focus on what they do best, as well as
think about major need areas, barriers that can be overcome and whether there are achievements to
share with the rest of the consortium. Dr. Wittkowski noted that Biostatistics is not mentioned in the
CTSA Strategic Plan document; however, Informatics is listed multiple times. Dr. Obrams urged
members to think about the highest priority for the CTSA Consortium and what is most achievable for
the BERD KFC.
VI. Informatics Strategic Planning
Dr. Johnson provided a brief description of the Informatics KFC’s strategic planning process. She
noted that the group initially identified a number of potential initiatives that were narrowed through an
in-depth web poll and discussion. As a result of the process, the Informatics KFC decided upon three
priority projects, all of which align with the PI strategic goals.
VII. Group Reports
A. Online Resources and Education: Dr. Banach reviewed the strategic goals, aims and specific
plans for CTSpedia. She stated that prior to extending the project to the BERD KFC, UCSF
worked with an information architect. To date, the Online Resources and Education has
launched a number of tools and aims to have more online resources from CTSA institutions.
Also, she reported that feedback was largely positive from a survey, which contained 271
CTSA BERD KFC Face-to-Face November 20-21, 2008 4
B. Evaluation: Dr. Rubio provided a brief summary of progress to date, and this includes but is
not limited to selecting common areas to evaluate, identifying areas for common measures
and discussing best practices. As a result of the previous day’s meeting, Dr. Rubio cited
stories (i.e. applying meaning to the numbers being collected), mentoring (i.e. working
collaboratively with the investigator), and contribution to the national effort as additional areas
for operationalization. She intends to post the updated version of the Evaluation Guidelines
on CTSA Wiki.
VIII. Demos from CTSA and NIH Institutions
A. Dr. Sally Thurston, University of Rochester
Dr. Thurston provided a demonstration of Rochester’s CTSI consulting service, which showed
the different types of consults offered, as well as how to schedule and track consults.
B. Dr. Michael Griswold, Johns Hopkins University
Dr. Griswold presented a snapshot of eCollab, which essentially takes all elements of the
research process for a project and disseminates that information to a webpage. He noted that
this is a great teaching tool and if developed into vignettes, then it helps with the effort to create
reproducible research. Dr. Griswold asked BERD KFC members to provide feedback on what to
add and what to alter. He also shared that Johns Hopkins University would like to partner with
other CTSAs when the Distance Education group has the beta version up and running in two to
C. Dr. Paul Wakim, NIDA
Dr. Wakim offered a demonstration on NIDA’s Clinical Trials Network Public Data Share
(www.ctndatashare.org). He stated that raw data (ASCII and SAS Transport files) is made
available to the public after either the primary acceptance of a paper or 18 months of data lock,
allowing for further analysis.
D. Dr. John Kittelson, University of Colorado Denver
Dr. Kittelson presented a brief demonstration of a detailed investigator and time-tracking system
that the University of Colorado Denver uses for its BERD program.
E. Dr. Frank Harrell, Vanderbilt University
Dr. Harrell gave a brief description about Vanderbilt’s TWiki (http://biostat.mc.vanderbilt.edu) and
F. Mr. David Bui, UCSF
Mr. Bui provided a presentation on UCSF’s consult tracking database. The UCSF CTSI BREAD
features include but are not limited to consultant time logging, billing and reporting; billing module
that produces recharge journals for upload to UCSF Online Financial System; and project
tracking and reporting.
IX. CTSA Consortium Strategic Goals
Dr. Packer spoke briefly on the CTSA Consortium strategic goals and discussed that if the CTSA
defines clinical research as a discipline, then methodological excellence needs to be a core value.
He suggested that the group needs to enhance its advocacy efforts within the individual CTSAs. In
response to this suggestion, Drs. Zeger, Obrams, Harrell, Pollock and DeMets will draft a brief
statement on the central role of BERD collaborators to CTSA investigators that will be sent to the
CTSA Steering Committee.
CTSA BERD KFC Face-to-Face November 20-21, 2008 5
X. Reports from Breakout Sessions
A. Statistical and Consulting Tools
a. Form subgroups of Online Resources (and Education) Task Force for
important sub-areas of tools development. Members willing to actively
collaborative or take a lead role include Drs. Harrell, Griswold, Rahbar,
Thisted and Hills.
b. Devote resources to tool development for impact
a. Develop tools to help collaboration for investigators (e.g. develop questions,
thinking, data prior to first meeting, vignettes for good collaborations) and
statisticians (e.g. repository, analytics, clinics)
b. Develop and share content for general distribution (e.g. good statistical
practice, free biostatistics “clinic” organization and experiences, models for
training students, mechanisms to get inserted earlier in the research design
c. Advertisement of where tools and contents are housed
3. Specific Tools
a. Reproducible research tools and methods
b. Teaching tools for younger investigators and statisticians
c. Clinics as a tool
d. Tracking tools
e. Tools for advertising
B. Administrative Aspects of Managing Cores
a. Prioritization of investigators
b. Managing demand
c. Funding support
e. Best model for time spent
2. Task Force to Address Objectives
a. Dr. Banach at UCSF volunteered to be in charge of a potential task force.
She will set up the above items at CTSpedia and will ask the BERD KFC
members to provide opinions.
C. CTSA Educational Components for Biostatistics: How to Train Collaborative Researchers
a. Educate not just trainees, but the leadership of CTSA
b. Do not teach biostatistics just as a separate discipline; become involved in
the research project and teach to junior colleagues
c. Have consulting conversation with fellow methodologists (e.g.
epidemiologists, bioinformaticians, graduate students)
d. Shift discourse from individuals to ideas to deal with recurring problems of
phenotyping (e.g. molecular biomarkers, data warehouse for genetics data,
inferences from medical record databases, collecting data without design)
CTSA BERD KFC Face-to-Face November 20-21, 2008 6
e. Collaborate with Informatics KFC to gain recognition that there are
overlapping problems which require joint solutions
a. Principles and best practices for drawing valid conclusions in defining
phenotypes. Members willing to actively collaborate or take a lead role
include Drs. del Junco and Lavori.
b. Principles and best practices for drawing valid conclusions in making
inferences from large epidemiological databases. Members willing to
actively collaborate or take a lead role include Drs. Kittelson and
c. Case studies and biostatistics resources for innovative ways to educate
biostatisticians and other clinical and translational researchers (e.g. web
resources, papers and videos, documenting success). Members willing to
actively collaborate or take a lead role include Mr. Bui.
D. Data Management and Informatics
a. Workgroup crossing KFCs on topic
b. Develop a supplement to jointly develop and test data management and
study execution tools for small studies not currently served
a. Present data management standards to PIs
b. Assess projects currently available (e.g. Human Studies Database Project,
CDISC/ADaM, WISDOM, REDCap)
c. Develop supplement with several KFCs: how best practices more
efficiently manage studies, meta study generator, help design case report
forms and collect data to improve data utility throughout life cycle.
Members willing to actively collaborate or take lead role include Drs. Pollock, Wittkowski, Miller
E. Methodological Research in Translational Science
a. Initiate and organize leadership and staffing to disseminate new
methodology and foster discussion on unsolved problems on CTSpedia
b. Pursue activities to identify methodological problems with particular
relevance to clinical and translational research
2. Short-Term Objectives
a. Implement and build the CTSpedia resources within twelve months
b. Continue discussions with biostatistician and methodological priorities
3. Long-Term Objectives
a. RFA to encourage methodological research relevant to Clinical and
Translational online journal
F. Training and Job Market Issues
CTSA BERD KFC Face-to-Face November 20-21, 2008 7
Two overarching goals arose from this breakout session. The first involves “minding the gap” of
supply and demand. The second involves becoming a 60-institution lobby group to minding the
gap. To address the first goal of minding the gap, the group suggested a three-pronged
1. Three-Pronged Approach To Minding the Gap
a. Make ‘em or Find ‘em (build a supply, e.g. NHLBI’s summer training for
b. Train ‘em (expand the training slots, particularly Masters’ level).
c. Feed ‘em (increase CTSA-related methodology grants).
2. Short-Term Objectives
a. Booster Training
b. Get 7 sites (currently 3) to get funded for summer training program
c. Launch a training program for graduate students that is broader than
3. Long-Term Objectives
a. Consolidate training grants across institutes
b. Broaden training grants for Ph.D. and non-Ph.D.
c. Develop internship for graduate students
d. Partner with ASA to educate high school students about statistical careers
Members willing to take a lead role include Drs. Kutner, DeMets, Howard and Cabral.
Members willing to actively collaborate include Drs. Greene, Moore, Bhore, Quarshie,
Couper, Bui, Beckett, Joskow, Gezmu, Geller, Obrams and Wakim.
G. Protocol Review: Who Does It and How To Be Involved
a. Survey of protocol and scientific review activities at each CTSA
organization and how biostatistics review is conducted (i.e. principle, what
to review, when to review, etc.)
b. Best practices (i.e. BERD involvement in protocol development stage and
how it is done)
c. Protocol/proposal development tools and strategies (lead: Dr. Perkins)
XI. Discussion on Top Three BERD KFC Priorities
In a brief discussion of potential BERD objectives that were outlined in the breakout sessions,
members overwhelmingly supported the following areas, which will serve as priority objectives for the
A. Create Shared Access to Reproducible Research Methods and Tools
B. Mind The Gap
C. Develop Online Journal for Methodological Research in Translational Science
XII. Action Items
# Action Items Owner Due Date
1 Draft brief statement on the central role of Drs. Zeger, 1/7/09
BERD collaborators to CTSA investigators to be Obrams, Harrell,
sent to the CTSA Steering Committee Pollock, DeMets
2 Send URLs for open courseware to BERD Dr. Griswold 12/19/08
CTSA BERD KFC Face-to-Face November 20-21, 2008 8
3 Use the survey in Evaluation Guidelines in the Dr. Ware TBD
startup of evaluation activities at Harvard CTSA
4 Post most recent version of Evaluation Dr. Rubio 12/18/08
5 Send feedback re: eCollab to Dr. Griswold BERD Members 12/1/08
6 Set up objectives listed in Administrative Dr. Banach TBD
Aspects of Managing Cores breakout session at
CTSpedia and ask the BERD KFC members to
CTSA BERD KFC Face-to-Face November 20-21, 2008 9