Public perception of public health research priorities
Work package 9
Slovak Public Health Association, European Public Health Alliance
Gabriel Gulis, Lara Garrido Herrero
August 31st 2007
Table of Contents
Executive Summary .......................................................................................................1
1.0 Rationale and background..................................................................................3
2.1 Phase one: developing the questionnaire ...................................................... 5
2.2 Phase two: preliminary assessment.............................................................. 9
3.0 Results and analysis .........................................................................................11
4.0 Discussion / Interpretations..............................................................................25
5.0 Reflections and recommendations for the future .............................................25
Appendix 1- The questionnaire....................................................................................27
This report reflects the results of the work carried out by SAVEZ and EPHA from
2004 until 2007. Both organizations are partners in the project SPHERE
(Strengthening Public Health Research in Europe), lead by the Faculty of the Royal
College of Physicians of the United Kingdom and funded by the 6th Framework
Programme of the European Commission.
The aim of the work carried out by SAVEZ and EPHA was to undertake a wide
consultation with representatives of the public across Europe on their perspectives and
priorities for public health research.
Public health research in understanding of this paper comprises strategic and
behavioral research. Public health research, as public health itself aims to be inter-
sectoral. A contemporary challenge for public health research is to integrate the
capabilities of different academic disciplines to address policies for health on one
side, and improve implementation of research results on other side. In spite of
standard public health research institutions such as national institutes and universities,
non-governmental organizations are entering public health research field during last
The term non-governmental organization (NGO) is used in a variety of ways all over
the world and, depending on the context in which it is used, can refer to many
different types of organizations. Often, NGOs bring together people with different
expertise but a shared interest to act on a common good. This is also the case in the
Public Health field. However, most of public health related NGOs focus rather on
field work, practice, advocacy, and at some extent policy development. During the
last decades this is under change and NGO are more often entering the filed of public
health research as well.
The work carried out by SAVEZ and EPHA centred mainly in creating a
questionnaire, sending it out and collecting responses to gather the views of European
NGOs about research in public health. In order to carry out this main activity, EPHA
and SAVEZ also attended and organised consultations with selected groups (for
example: meeting in the European Parliament in July 2005, workshop in Bratislava in
April 2007) and with the rest of the project partners (meeting in Montreaux in
Out of approximately 1,863 NGOs contacted through the questionnaire, SAVEZ
received 80 filled questionnaires. The recipients included mainly EPHA members
(around 100) who are active NGOs working on public health and research and EPHA
contacts in Eastern European Countries and European Institutions.
The responses to the different questions have been combined to provide useful
information regarding the participation of the respondents in international public
health research. The main findings are:
⋅ Respondents were NGOs that had been operational for a few years.
⋅ There was a balance between NGOs working on broad public health issues and
those working on single health determinants or diseases.
⋅ Not many of the NGOs did carry out research themselves.
⋅ Most of the NGOs were members of one or more international network or
organisation, and were involved in international projects.
⋅ Although NGOs are aware about the Framework Research Programme and the
Public Health Programme of the European Commission, they did not show so
much knowledge about other funding resources for public health research.
⋅ NGOs identified a long list for potential priorities for public health research in
Europe, and they actually showed to be quite objective given the fact that the
areas they identified as in need for more research were not only the areas on
which they were currently working on or researching on.
European NGOs are dynamic and active, both in their composition and in the
priorities they focus on. Although this is positive, it makes difficult the assessment of
this exercise (in particular the questionnaire). We can say we have got good
geographical balance, NGOs working on very varied public health topics, and a
comprehensive list of priorities by country of origin (see tables 14 and 15).
The project activities have been presented in several fora as an example of a platform
for partnership between the academic community working in public health and civil
society organisations advocating for health and increased participation of European
citizens in decision-making processes. This cooperation aslo helped feeding the
project results into policy making.
1.0 Rationale and background
Public health research concerns with population health, determinants of health, health
systems research, health promotion, environmental health, health protection, disease
prevention and research in other fields. It complements biomedical research, which is
concerned with biomedical mechanisms of disease and medical treatments. In contrast
to laboratory or clinical level of biomedical research public health research is
undertaken on population level. The Global Forum for Health Research defines three
main categories of health research such as strategic research, biomedical research and
behavioral research1. Public health research in understanding of this paper comprises
strategic and behavioral research. Public health research, as public health itself aims
to be inter-sectoral, transboundary cooperative research. A contemporary challenge
for public health research is to integrate the capabilities of different academic
disciplines to address policies for health on one side, and improve implementation of
research results on other side. In spite of standard public health research institutions
such as national public health research institutes, universities, public and private
sectoral research institutes non-governmental organizations are entering public health
research field during last decades.
The term non-governmental organization (NGO) is used in a variety of ways all
over the world and, depending on the context in which it is used, can refer to many
different types of organizations. In its broadest sense, a non-governmental
organization is one that is not directly part of the structure of government. Often,
NGOs bring together people with different expertise but a shared interest to act on a
common good. The public health area is for long time not an exception, for example,
the history of the American Public Health Association goes back to 1872. However,
most of public health related NGOs focus themselves to public health field work,
practice, advocacy, and at some extent policy development. During last decades this is
under change and NGO are more often entering the field of public health research as
There is also a crucial role for a fruitful interaction between the scientific community
and policy-makers, which respects the ability of science to demonstrate the
implications and opportunities of policies and to ensure that public health research is
applied. Incentives for this will not come through commercial markets, but may be
enhanced by public debate.
It is a fact that there is a lack of sources of information about the participation of
NGOs in public health research in Europe. In fact, the lack of references justifies this
Other work packages of the study have produced a substantial account of the literature
in public health research, made assessments of the quality and range, European
collaboration, implications and uptake by policy bodies at international, national and
local level, and offer comparisons from practice outside Europe. These findings will
be unique and of relevance to a wide range of audiences.
1 Ghaffar A, de Francisco A, Matlin S: The Combined Approach Matrix; A priority setting
tool for health research, Global Forum for Health Research, 2004
The Strengthening Public Health Research in Europe (SPHERE) project2 aims to gather
information and produce knowledge on the state of public health research in Europe. The
focus of this project, therefore, is on how national governments and European institutions can
support public health research that is relevant to, and used by, public policy-makers. A strong
focus has been pointed to NGOs and their view of recent and future public health research
priorities in Europe. The presented paper brings results of this work package.
The collection of information on NGO engagement in public health research was
carried out in a systematic way. The methodology was agreed by the partners at the
beginning of the project. All work packages followed the same methodology in order
to make the results comparable. All partners agreed that there would be two phases on
the methodology: deveoping of the questionnaire and preliminary assessment of
It is worth explaining here some of the changes of experts for this work package. Due
to the rapid changing nature of of a non-governmental organisation, the European
Public Health Alliance experienced changes in the staffing during the execution of the
project. Tamsin Rose was the expert appointed to this project from the beginning, and
she carried out most of the awareness raising and advocacy activities. Tamsin left
EPHA in December 2005, therefore it was necessary to appoint a new expert, with
similar skills, that could carry out the tasks assigned to EPHA. That person was
Roxana Radulescu, who worked for EPHA from January 2006 until December 2006
and was involved mainly in the designing of the questionnaire, its distribution and the
collection of responses. Finally, the expert appointed by EPHA to contribute to phase
two of the methodology, and mainly to prepare this report, was Lara Garrido-Herrero
(from January 2007 until the end of the project).
Information and advocacy activities at EU level is one of the main areas of work of
EPHA. This meant that EPHA was in a privileged position to make a direct link
between the activities of the SPHERE project and EU policy making activities.
Amongst these advocacy activities EPHA seeks to identify windows of opportunity
for influencing EU policies and to mobilise the relevant actors in order to achieve the
desired changes on policies.
The SPHERE project started at a time when the 7th Framework Programme was
beginning to be drafted. One of the objectives of the SPHERE project was to provide
a retrospective account of European research in public health and also a prospective
about the relevance to future research commissioning, eg. in the the 7th Framework
In early 2005, the EPHA expert (Tamsin Rose) identified an opportunity to organise a
briefing for Members of the European Parliament about public health research under
the draft proposal for the 7th Framework Programme. Preparations for this briefing
started: identifying the needs and positioning of MEPs, analysing the Commission
2 Clarke A, McCarthy M, Alvarez-Dardet C, Sogoric S, Groenewegen P, Groot W, Delnoij
D: New directions in European public health research: report of a workshop, J Epidemiol.
Community Health 2007; 61; 194-197
draft proposal and identifying next steps for MEPs. The EPHA expert spent a
considerable amount of time on this activity, which included also all the logistic
arrangements. The briefing took place on 6 July 2005, for more information see:
The conclusions from this meeting identified support from key Members of the
European Parliament in making public health research a higher priority in the
Framework Programme and in involving civil society organisations more in research
A concrete follow up action was to organise a workshop with EPHA members on 13
December 2005, with a double aim: to agree on the main needs of health NGOs
regarding public health research under the new 7th Framework Programme and to
make them familiar with the SPHERE project. A presentation by the coordinators of
the SPHERE project was delivered during this workshop.
The organisation and follow up of this workshop was one of the main activities of the
EPHA expert for the months of September, October, November and December 2005.
The next step from this workshop was to send to the members of the European
Parliament a letter with suggestions for amendments to the draft Commission
proposal. This was done in early January 2006.
2.1 Phase one: developing the questionnaire
The project proposal established the start of the work at the end of 2004 with the
kicking off partners meeting to be held during the EUPHA annual conference in Oslo.
Although the contract was not signed until after that meeting and the pre-financing did
not arrive until 2005, we will consider this first meeting as the start of the project
SAVEZ participated in the workshop of 2004, where the questionnaire was discussed
for the first time amongst project partners. This discussion was important because it
clafied the objectives and target audience of each work package, as well as explaining
to the project partners the expectations of the European Commission, funder of the
After this meeting, SAVEZ and EPHA maintained contact through emails and
telephone calls, agreed on the workload, list of tasks and the time line, and started
discussing the outline of the questionnaire.
The next milestone was the open discussion of the structure of the questionnaire at the
partners meeting during the EUPHA conference at the end of 2005 in Graz.
The discussion took place between SAVEZ and the partners involved in the other
work packages that involved consulting the national ministries and the national public
health associations. It was agreed that the questionnaires would be adapted to the
different audiences instead of using the same model for the three work packages. It
was agreed that for this work package the questionnaire should be presented as easy
and simple as possible, to target people and organisations that were not experts in
public health research. SAVEZ presented the outline of the questionnaire as it had
been agreed with Tamsin Rose (EPHA's expert at that time) through email and
telephone calls. The rest of the project partners acknowledged the outline and the
organisation of the tasks for the work package.
In September 2005 Gabriel Gulis (SAVEZ) and Tamsin Rose (EPHA) met for the
first time. This meeting was very important as this was the first time that SAVEZ and
EPHA worked together in an European project.
The first aim of the meeting was to identify the skills and knowledge from both
partners. The second aim was to build trust between the two partners for the work that
was to be carried out during the following three years.
Gabriel Gulis from SAVEZ presented its experience in public health research, health
promotion and public health, cancer and environmental epidemiology; he also
explained his extensive knowledge about the situation of public health research in the
new EU Member States.
The expert from EPHA, Tamsin Rose, explained EPHA's strengths as the voice of the
non governmental and non for profit health community in Europe, with an
understanding of the needs and nature of the health NGOs and with a comprehensive
database of contacts that would be put at service for this project.
The next milestone in the preparation of the questionnaire, was January 2006 when it
was agreed at a telephone conference between Gabriel Gulis (SAVEZ) and Roxana
Radulescu (the new expert from EPHA) that the first step would be to identify and
contact non-governmental organisations across Europe.
At the time of the preparations of the project proposal, EPHA had a database of
around 100 member organisations. This database is updated twice a year by the EPHA
staff with new memberships, and a considerable amount of time is dedicated to this
It was agreed that the questionnaire would be sent by electronic mail to the EPHA
database. The follow up would be also by electronic mail, with no physical meetings
or individual interviews planned with the respondents.
Roughly at the same time that the SPHERE project started, EPHA embarked in the
creation of an ambitious database of health NGOs in the European continent, with a
strong focus on NGOs based in Eastern European Countries.
According to EPHA's records, 15% is the average response rate when consulting
EPHA members by email. This is not particularly high. An explanation for this is that
health NGOs in Europe tend to be weak and fragmented. They often rely on
volunteers and part-time staff, they have one or two priorities, and they are often not
available to fill in questionnaires or respond to surveys.
Therefore, it became clear that if we wanted to have a significant number of
respondents, the questionnaire should be distributed no only to EPHA members but to
a wider audience of health NGOs in Europe.
This was all explained to SAVEZ, who supported the idea of broadening the
recipients of the questionnaire. Moreover, SAVEZ sent to EPHA the contacts of more
health NGOs working in Eastern European Countries, specially in Slovakia, Armenia
The mailing lists that were chosen for distribution listed and explained below:
⋅ CEE contacts (NGOs working on health and environment in new member states
and neighbouring countries): 1,334 contacts as of February 2006.
⋅ EPHA newsletter mailing list: this included mainly NGOs but also any other
individual interested on receiving EPHA's monthly newsletter. It also included
academics, government officials, Commission officials, politicians, etc. Both
SAVEZ and EPHA knew that these kind of organisations were not the targeted
audience, but they considered that they could act as multipliers, ie. they could
forward the questionnaire to other NGOs (and indeed they were kindly asked to
do so in the email they received). This mailing list contained 1,584 contacts as
of February 2006.
⋅ EPHA members: these are all EPHA members, NGOs and non-for profit
organisations working for health in Europe. This list contained 222 contacts as
of February 2006.
⋅ CAM platform: these are European NGOs working on complementary and
alternative medicine. Some of them were also EPHA members, but not all of
them. We considered them relevant because they have a very particular interest
on European public health research. This list contained 72 contacts as of
⋅ European Health Policy Forum: the members of the EHPF are European
networks of organisations working for health in Europe. The EHPF was set up
by the European Commission (DG Sanco) as a forum for NGOs and the
Commission. It includes NGOs, professional bodies, academics, etc. Some of
them are EPHA members, but not all of them (some of them only work on
health care or on health services, but they have an interest on public health
research). This list contained 47 contacts as of February 2006.
⋅ Children conference: this mailing list was created in November 2005 when
EPHA organised a conference on EU policies that impact children. Many NGOs
and regional authorities participated. It contained 103 members.
⋅ EEN members: This mailing list contained NGOs working on environment and
health, they are the members of the former EPHA Environment Network (now
“Health and Environment Alliance” - HEAL). It contained 72 contacts as of
⋅ Green 10: these are the 10 biggest environmental NGOs with a representation in
Brussels. It contains 10 contacts.
Total number of recipients: 3,444
Estimated number of NGOs in these lists: 1,863
At the same time that the recipients of the questionnaire were identified, SAVEZ
finalised the questionnaire took place by SAVEZ. The EPHA expert (Roxana
Radulescu) acted as the “tester” of the questionnaire.
Gabriel Gulis, from SAVEZ but based in Denmark, also consulted his colleagues in
Slovakia to contrasts opinions about the questionnaires. The colleagues gave useful
feedback, particularly on the question about international experience (see annex I for
the questionnaire), and how to define and measure it.
Roxana had worked with health NGOs for several years and she was aware about the
language and the understanding of these organisations. She suggested that the
questions about the NGOs activities on public health research should be accompanied
by examples, because the term “public health research” was not understood in the
same way in different countries. This was included in the questionnaire.
Also, in order to obtain the attention of the recipients, it was agreed to mention
prominently the objectives of the project and the source of funding.
The questionnaire was finalised and agreed in February 2006 (see Annex I). The first
mailing was sent to the recipients on 16 February 2006.
Regarding ethical issues, EPHA follows the Belgium national legislation (according
to European legislation) regarding data protection. The information in EPHA's
mailing lists is not used for other purpose than for communicating between EPHA and
the recipients; EPHA does not lend, sell or publish the contacts in its mailing lists.
Those people that figure in EPHA's database have full access at any time to the
information stored in them and they can modify or cancel their subscription to these.
The results of the survey have been used in a confidential manner to obtain statistical
data and general conclusions and recommendations.
It was also in February 2006 that the EPHA expert (Roxana) organised a meeting with
one of the key Members of the European Parliament that had attended the July 2005
briefing. The aim of the meeting was to present an update of the conclusions of the
December 2005 workshop with EPHA members and to present the activities of the
A report of the work up to that date was prepared jointly by Gabriel and Roxana and
sent to the coordinators by March 2006. This report included the preliminary
assessment of the results of the survey.
Also in March 2006 the EPHA expert was invited to an event organised by the
Madariaga Foundation in Brussels to discuss the priorities of the 7th Framework
Programme and in particular the view of the public health NGOs on public health
research. The activities of the SPHERE project were also presented during this
There was a second mailing (reminder) sent to the same mailing lists in April 2006
with deadline to respond by 15 May 2006.
The EPHA expert also contributed to the work of other work packages by compiling
and sending a list of contacts in the ministries of health of the Eastern European
Countries to the project coordinators in May 2006.
All activities carried out by SAVEZ and EPHA were done with regards to the
principle of gender balance. The events organised by EPHA in the Parliament and
with EPHA members included a balance of speakers, as well as participants.
Recipients of the questionnaire included organisations working on breast cancer and
prostat cancer, and minorities including ethnically-related diseases and sexual
2.2 Phase two: preliminary assessment
We have described in the previous section the emails that were sent to the different
mailing lists, the meetings with Parliamentarians and relevant actors on public health
and research in Europe and the reports prepared by the project partners for the project
coordinators, which included the preliminary assessment.
The mailing of April 2006 included also the preliminary results and invited those
organisations that had responded to the questionnaire to provide feedback on the
Following this mailing, SAVEZ received more answers to the questionnaire from
organisations that had not repsonded yet, but they did not receive any specific
comments aiming to correct the results of the preliminary assessment.
The content of the feedback was rather questions for clarification of the concepts used
in the questionnaire.
Several NGOs from the new Member States asked about the concept used to define an
NGO. We believe that this reflects a particular situation in the new Member States
where University researchers often set up NGOs to carry out local projects or to
advocate for healthier public policies of relevance to their community. In fact, some
of the answers on behalf of NGOs were sent from email addresses of Universities.
SAVEZ answered to these questions saying that the survey aimed at NGOs in the way
we have described in the chapter of “Rationale and background” of this report.
At the end of 2006, a partners' meeting took place during the EUPHA annual
conference in Montreaux. SAVEZ and EPHA presented the results of the work of the
completed phase one and the start of phase two.
For the next months, Gabriel Gulis spent considerable time organising and analysing
the data collected through the questionnaire, and presenting it in the format of section
“Results and analysis” of this report.
In the first half of 2007, a partners meeting took place in Dusseldorf. The final
assessments (phase two) of the different packages were discussed.
It was noted that the partners in charge of collecting information from national
ministries received plenty of feedback from the participants of the survey. The
feedback aimed to clarify the preliminary assessment in what concerned the situation
of their own country.
This was due to the nature of the role of the representatives of national ministries,
who have to be careful presenting the official position of the country, and therefore
had to check the results of the preliminary assessment with their own country's
position and even compare it with other countries' results.
This was not the case for our work package, where NGO representatives often express
their own opinion drawn from an individual's experience or from consultations with
their colleagues and NGO members, and do not necessarily compare it with the results
of other countries or groups of NGOs.
Besides, the objective of this work package was less to offer a result of public health
research per country and more to find out what was the average situation across the
European Union and the involvement of NGOs in international public health research.
However, it was clear that feedback from the participants in the survey to the
preliminary assessment was necessary for the purpose of the overall project, and it
would not be through more mailings that this would be achieved.
At the end of 2006 EPHA announced to its members and partners the organisation of
two-days international conference on the topic of health and the enlargement, with the
posibility of organising workshops and paralell sessions to present topics related to
the overall theme. The conference was organised as the closing event to a three year
project on capacity building in the new Member States partially funded by the Public
Health Programme of the European Commission. The participants of the conference
were NGOs, health care professionals, policy makers, researchers and academics
working in public health related issues in Europe, and in particular, in the new
Taking advantage of this opportunity, the SPHERE project coordinators suggested to
EPHA and SAVEZ a paralell workshop during this conference to present the
preliminary assessment and to gather feedback from NGOs and policy makers in a
more effective way.
The workshop included two presentations by the following project partners:
⋅ Mark Mc Carthy and Gabrielle Harvey: Faculty Public Health, United
Kingdom; “Comparisons between the EU 15 and Accession Countries”
⋅ Gabriel Gulis: University of Southern Denmark / Unit of Public Health,
Denmark; “Public health research priorities in Europe by eyes of NGO’s”
The participants gave useful feedback to the project partners, although the comments
did not aim to change the results of the preliminary assessment, rather to complete it.
Another positive outcome was that more NGOs filled in the questionnaire and these
extra responses were included in the preparation of the final assessment.
3.0 Results and analysis
In total, 80 questionnaires were received from 80 different NGOs in 28 European
countries. It is very hard to express a response rate due to different numbers of
contacted NGOs. Taking in account only NGOs in EPHA membership (150) then the
response rate is quit high, about 53 %. On other side, if we take the whole number of
about 1,863 contacted NGOs then the response rate is minimal.
In spite of the respondents being mainly European non-governmental organizations,
one response came from Brazil and a couple of responses claimed that the
organization had no NGO status anymore. These responses were excluded from the
analysis. The remaining responses were introduced in a simple and easy to manage
database using Excel. Among them also three NGOs who claimed to be an NGO, but
not a public health NGO, and six who consider themselves a public health NGO, but
not public health research NGO. This report includes detailed description and
interpretation of the results of the survey.
The responses have been collected by SAVEZ, who also analysed the statistical data.
The data collection and analysis process was completed in May 2007; the preliminary
results were presented at project group meeting in Montreux (November 2006) and at
the EPHA conference in Bratislava (April 2007), and this report presents the final
As well as the work carried out jointly by SAVEZ and EPHA with the questionnaire,
EPHA has sought to add value to SPHERE by raising its visibility though its many
advocacy activities at EU level. The project has been presented in several fora as an
example of a platform for partnership between the academic community working in
public health and civil society organisations advocating for health and increased
participation of European citizens decision-making processes.
Respondents came from all around Europe. The following bullets present the
geographic distribution of received responses by countries, which is then presented on
Figure 1 in map format.
⋅ EU27 (15+10+2)
⋅ UK 5, Belgium 5, Germany 3, the Netherlands 3, Greece 3, Austria 3, Ireland 3,
Italy 2, Sweden 2, Luxembourg 1, Spain - EC 1 - 31
⋅ Cyprus 6, Slovenia 4, Czech Republic 3, Hungary 3, Poland 3, Slovakia 3,
Estonia 2, Latvia 1, Lithuania 1, Malta 1 - 27
⋅ Non EU
⋅ Macedonia 5, Armenia 2, Turkey 2, Switzerland 1, Croatia 1, Norway 1, Serbia
and Montenegro 1 – 13
Figure 1: Distribution of responses by country
Analysis of the responses
The questionnaire (see Annex I) aimed to obtain the opinion of NGOs, considered as
representatives of the public, about public health research in Europe. The questions
were divided in 4 sections:
⋅ General information about the NGO and its experience in international public
⋅ Awareness of the NGO about international public health possibilities
⋅ Experience of the NGO in national, regional and international public health
⋅ Issues that the NGO considered necessary to be addressed by public health
The responses to the different questions have been combined to provide useful
information regarding NGOs participation in international public health research.
According to the responses to question 1.c, we can separate the participants in
different groups depending on how long their NGO has been active for (the time-
frame chosen is “since the creation of the NGO until 2005”). This is shown in Table
The mean duration of existence for the NGOs that participated in the survey is 18.27
years (with a standard deviation of 23.32 years). The value of the median was 14
years signalizing that the mean is slightly shifted toward higher values by couple of
older, long time existing NGOs.
Table 1: Duration of existence of NGOs
Number of years of establishment (up to 2005) Number of respondents falling under this category
>= 21 14
Under question 1.b, we gathered relevant information regarding the area of work of
the NGO and also their working methods. The answer was left quite open, which
means that some respondents only included information about their areas of work,
while others also included information about their methods of work, and a few did not
included any information at all.
Table 2 organizes the number of respondents in different groups, depending on the
working methods that the NGOs use in their practice. Only those respondents that
answered this question are included. It is worth noticing that only four of the
respondents listed research as one of the main work method.
Table 2: Working methods of NGOs
Number of respondents
falling under this category
Advocacy, networking, field work 12
Health promotion activities (health and human rights
Provision of services (care of people with disabilities,
chronic illness, etc)
Awareness raising campaigns 2
Financial support (donors, provision of grants) 1
Standard setting 1
Looking at working method by membership in EU (separately EU15 and EU10) and
non-EU country origin NGOs, we found no statistical significant difference in
working methods (P=0.5563)
Table 3 describes in more detail the areas of work of the NGOs that took part in the
survey. We can see that there is a balance between NGOs that work on broad health
issues (such as population heath and environment and health) and those who work on
single health issues. The exception is the cluster of respondents that are grouped under
Attention Deficit Hyperactivity Disorder (ADHD).
Again, only those respondents who answered this question are included in the analysis.
Table 3: Work areas of NGOs
Number of respondents falling under this
Area of work of the NGO
Population health (health promotion, prevention) 9
Environment and health issues 6
ADHD (Attention Deficit Hyperactivity Disorder) 6
Health and social care 6
Mental health 4
Breast cancer, cancer 4
Complementary and alternative medicine 3
Alcohol and drug policy (and other addictions such as
Tobacco control 2
Medical education 2
Injury prevention 2
Human rights 2
Lifestyle, physical activity 2
Health promotion, education 2
Sexual and reproductive heath 1
Cardiovascular diseases 1
Hospice and palliative care 1
Nursing, midwifery 1
Pharmaceutical products, access to medicines 1
Grant provision 1
Safety and security 1
There is no statistical significant difference by belonging of the county of origin of a
NGO to one of the three groups (P= 0.7627).
Questions 1.d and 1.e aimed to collect information about the international experience
of NGOs in general and their experience with public health research work.
Regarding question 1.d, about 52.2% of responding NGOs (n=37) claimed that they
had experience working at international level, 47.8% claimed no international
Respondents reported having international experience on the following areas:
consultancy services, collaborative projects, organizing and participating in
international workshops, and attending meetings abroad.
Some of those who did not answer question 1.d reported being members of
international networks of NGOs, though they did not consider this to be significant
international experience. This might signalize a kind of passive membership with no
active working in a network or collaborative research.
Figure 2 shows information about the international experience in terms of
membership to international networks of NGOs. Some of the respondents had a
significant number of developed partnerships, for example one of the respondents
claimed membership in 10 international networks of NGOs. In general, 51 NGOs
reported membership in more than one international network or organization. 20
NGOs say that they were not at all members of any international organization.
Number of international networks of which the respondents claim
number of NGOs
0 1 2 3 4 5 6 7 8 10
number of networks
There is no statistical significant difference in memberships in networks by belonging
of the country of origin into one of the three groups of countries (P=0.424).
Combining the information gathered from question 1.c, 1.d and 1.e, we aim to present
the relationship between having international experience and being a consolidated
NGO. We have observed that there is no clear trend that supports the idea that more
established NGOs have more international experience, and the explanation could be
that most of the newest or less consolidated NGOs (specially in the new EU member
states) are aware about the importance of international work and the importance to be
a member of international networks in order to gain recognition at national level
(recognition as reliable stakeholders by their government or Parliament, and increased
attraction of independent funding).
Table 4: Membership in international networks by duration of existence of a NGO
Membership of international networks
Years of establishment 0 1 2 3 4 5 6 7 8 10 TOTAL
0-5 6 2 4 0 1 1 1 1 0 0 16
6-10 3 1 2 3 1 1 0 0 0 0 11
11-15 1 3 5 2 0 4 0 0 0 0 15
16-20 4 0 1 0 1 2 1 0 1 0 10
21-25 0 2 0 1 1 0 0 0 0 0 4
The information gathered from the answers to question 1.e about the participation in
international project work completes the conclusions drawn from Table 4. Figure3
shows participation of NGOs in international projects; 37 NGOs reported
participation in at least one international project. Most of the NGOs are involved in
one or two international projects, only one NGO claimed to be involved in 8
international projects and another one was involved in 9 projects.
Participation in international projects
number of NGOs
0 1 2 3 4 8 9
number of projects
Under question 2.c, respondents were asked about their awareness of funding
resources for public health research other than the European Commission Framework
Programme (FP) for research or the Public Health Programme (PHP). Figure 4 shows
a summary of these answers. The first conclusion is that there is a limited amount of
respondents that were aware of the existence of other funding sources.
Awareness of international public health research funding sources
ber of NGOs
Not available answer
Aware, but never used them
20 Aware, but does not know the details
other international funding sources PHP and FP
More than 50% of the respondents claimed knowledge of either the European
Commission’s Framework Programmes for research or the Public Health Programme.
The answers to question 1.e (general experience of the respondents on international
public health) are gathered in table 5, which shows that when asked about general
experience (without specifying if it is research experience or not) the respondents are
more or less equally divided.
Table 5: General public health experience
Number of respondents falling under this category
Not available answer 12
Asking specifically for public health research experience brought higher percentage of
respondents claiming experience at international level, as seen on table 6.
Table 6: Experience with public health research
Experience with PH research Number of respondents falling under this category
Not available answer 5
Yes in PH, but not research 1
Yes, but only individual members 1
The contrast of results might be explained by the understanding of ”research” in
different languages. Out of 37 NGO’s who claimed experience, 29 also claimed
participation in international public health research. All of this participation is within
either the FP programs or the public health action plan of the EC.
The breakdown of experience in public health research projects in relation with the
consolidation of an NGO is presented on table 7. The statistical significance is not
very high, which can be explained by the fact that NGOs (specially those in the new
EU Member States) have received quite a lot of training and capacity building to
access EU funds in the past 3-5 years. It is also a fact that most NGOs rely on public
funding (to avoid conflict of interests) therefore they actively look for partnerships.
Table 7: Public health research experience by duration of existence
Not involved in
Duration of Not involved Involved in projects Involved in projects
N.A. projects, but funds TOTAL
existence in projects for a while only recently
0-5 1 10 0 5 1 17
6-10 0 6 0 7 0 13
11-15 0 9 0 6 0 15
16-20 1 4 0 5 0 10
21-25 0 4 0 0 0 4
Questions 3.a and 3.b aimed at gathering information about the level of work of the
NGO (national, regional or international). The responses are shown in Figure 5:
Distribution of NGOs by geographic area of work
Figure 5: Distribution of NGO's by geographic area of work
The main objective of this questionnaire was to find out about the opinions of NGOs
regarding the priorities for public health research, and this was achieved through
questions 4.a and 4.b. Respondents expressed opinions in both national (regional) and
international public health research priorities. The answers are summarized in tables 8
(national, regional) and 9 (international).
In most cases, the respondent listed more than one priority. Tables 12 and 13 keep the
priorities in the same order in which respondents listed them (I, II, II). We have not
ranked the priorities, therefore all of the topics issued are considered priorities, but
those on the top of the list are not necessary the most important priorities.
Table 8: PH research needs at national and regional level
I priority II priority III priority Total
Public health, population health 10 1 0 11
Environment and health 6 0 0 6
ADHD 5 0 0 5
Obesity, nutrition 2 3 0 5
Tobacco 3 0 1 4
Ageing 3 0 1 4
Cancer 1 3 0 4
Awareness research 3 1 0 4
Economic, social determinants of health 0 3 1 4
Mental health 3 0 0 3
Injury prevention 2 0 1 3
Smaller size research 3 0 0 3
Sexual and reproductive health 0 0 2 2
Hospital care, palliative care, post-hospital and
0 2 0 2
Pharmaceutical products 1 1 0 2
Behavioral health 2 0 0 2
Health care systems, reforms, finances, access 1 1 0 2
Private sector collaboration 1 1 0 2
Child safety, violence 1 0 1 2
DALY, burden of disease 0 1 1 2
Health education, promotion 2 0 0 2
Drug use, injecting drug use 1 0 0 1
Diabetes 0 1 0 1
Medical education 1 0 0 1
Cardiovascular disease 0 1 0 1
Meningitis 1 0 0 1
Human rights 0 0 1 1
Rheumatism 1 0 0 1
Allergies 1 0 0 1
Rare diseases 1 0 0 1
HIV/AIDS 0 1 0 1
Rural health 0 1 0 1
Anemia 0 1 0 1
National health surveys 1 0 0 1
Health inequalities 1 0 0 1
Scientific writing 0 1 0 1
Efficiency of Chinese medicine 1 0 0 1
Asthma 0 1 0 1
Physical activity 0 1 0 1
Cost effectiveness of prevention, treatment 0 0 1 1
Community resilience 1 0 0 1
Patient registry 1 0 0 1
Quality improvement 0 1 0 1
Health and social care 0 1 0 1
Nursing, midwifery 0 1 0 1
Table 9: PH research needs - international
I Priority II Priority III Priority Total
Public health, population health 6 2 1 9
Health care systems, reforms, financing, 0 1 6
Obesity, nutrition 2 4 0 6
Environment and health 5 0 0 5
Mental health 3 1 1 5
Tobacco 3 2 0 5
Smaller size research 4 0 0 4
Awareness research 4 0 0 4
Ageing 3 0 1 4
Lifestyle 2 2 0 4
Cost effectiveness of prevention, treatment 3 0 1 4
Implementation research 2 1 0 3
Sexual and reproductive health 2 0 1 3
Health and social care 2 1 0 3
Injury prevention 2 0 0 2
ADHD 2 0 0 2
Health education, promotion 1 1 0 2
Medical education 0 2 0 2
Cancer 0 2 0 2
Scientific writing 0 2 0 2
Economic, social determinants of health 0 1 1 2
Cardiovascular disease 1 0 0 1
Pharmaceutical products 1 0 0 1
Rheumatism 1 0 0 1
Efficiency of Chinese medicine 1 0 0 1
Based on WHO recommendations 1 0 0 1
Data collection systems 1 0 0 1
Support to South East Europe 1 0 0 1
Drug use, injecting drug use 0 1 0 1
Rural health 0 1 0 1
Asthma 0 1 0 1
Allergy 0 0 1 1
Human rights 0 0 1 1
Child safety, violence 0 0 1 1
Nursing, midwifery 0 1 0 1
There is a high level of comparability in national (regional) and international public
health research priorities as seen by NGOs. It seems that priorities tend to be similar
on different governance levels.
Both in case of national and international priorities we did a statistical test for
significance of the difference by three main country groups and found no significance.
P-values were 0.2757 and 0.3558 respectively, for national and international
priorities. We also checked for correlation between work area of an NGO and the
stated priorities to assess level of objectivity. Using Pearsson correlation coefficient
statistic, correlation coefficients of 0.2094 and 0.2557 were calculated comparing
priorities and work area of NGO. This shows quit a good level of objectivity; NGOs
do not tend to focus on their own area only.
Combining the answers to question number 1.c and question number 4.b, we can
compare the priorities claimed at international level with the consolidation of the
NGO. This is shown in table 10.
Table 10: PH research needs – international by duration of existence
0-5 6-10 11-15 16-20 21-25 >25
Public health, population health 2 2 1 1 0 0
Mental health 0 0 1 1 0 1
Sexual and reproductive health 0 0 1 1 0 0
Tobacco 0 1 2 0 0 0
Cardiovascular disease 0 0 1 0 0 0
Injury prevention 1 0 0 1 0 0
Environmental heath 0 2 1 1 0 0
Aging 2 0 0 0 0 0
ADHD 1 0 0 1 0 0
Health and social care 1 0 0 0 0 1
Pharmacy 0 0 0 0 1 0
Rheumatism 0 0 0 0 1 0
Efficiency of Chinese medicine 0 0 1 0 0 0
Smaller size research 1 1 1 1 0 0
Lifestyle 0 1 0 0 0 1
Health care systems, reforms, financing, 0 0
0 1 3 1
Based on WHO recommendations 0 0 0 0 1 0
Implementation research 1 0 0 0 0 1
Cost effectiveness of prevention, treatment 1 0 0 1 0 1
Data collection systems 0 0 0 1 0 0
Support to South East Europe 1 0 0 0 0 0
Obesity, nutrition 1 0 0 0 0 1
Health education, promotion 1 0 0 0 0 0
Awareness research 1 1 0 0 1 1
Tables 11 and 12 present public health priorities of NGOs by country of origin.
Table 11: National and regional public health research needs by countries
Table 12:International public health research needs by countries
4.0 Discussion / Interpretations
We have tried to include as much as possible our discussions and interpretations in the
previous chapter. However, as a final thought, we would like to add that according to the
work that we have carried out for this work package, we conclude that many of the
priorities for public health research could not be considered as a research priority (for
example the scientific writing issue, or support for smaller size research, etc…).
This conclusion re-emphasize the need to deal with public health training in context of
Europe. Comparing with other work packages, especially with survey among
governmental units (ministries responsible for research) and national public health
associations, a low level of compliance of priorities is seen by three major public health
5.0 Reflections and recommendations for the future
NGOs are characterised for being vibrant, active and dynamic organizations. In the
presented research one of the hardest points is therefore to assess its representative-ness.
The usual measure of questionnaire surveys, response rate, is hard to apply due to the fact
of the constant change. However, looking at geographic distribution we get quite a nice
coverage of countries from Armenia to Ireland and from Cyprus to Norway. This,
together with statistically proved fact of lack of significant differences among EU 15, EU
10 and non-EU countries suggest a good level of representative-ness of our results.
Responses show significant international experience of NGOs. Networking, collaborative
project work, international consultancy, and other means are claimed by NGOs as means
of international work.
Work area of NGOs seems to be often mono-thematic focusing on a single public health
issue. In contrary they presented a rather broad range of priorities both on national and
international level. The low level of correlation between work area and priority shows a
high level of objectivity; NGOs do not tend to focus only on their own area while looking
into future and to needs of population.
As explained under the methodology, the project activities have been presented in several
fora as an example of a platform for partnership between the academic community
working in public health and civil society organisations advocating for health and
increased participation of European citizens decision-making processes.
This meant that the project results feed straight into policy making. We can see that by
looking at the newly adopted FP7, which includes reference to the participation of civil
society organizations in research and a first call for proposals (issued in December 2006,
deadline for submission 19th April 2007 for the first call and 18th September 2007 for the
second call) with a wide range of public health issues of interest to NGOs. Apart from the
specific public health issues addressed by the Programme, the FP7 Cooperation
Programme includes “collaborative research” as one of the cross-cutting issues,
supporting the active collaboration of research teams from all sectors, including
international European interest organizations and civil society organizations.
e&call_id=10#infopack ). In fact, the public health priorities of the FP7 for 2007 do
coincide with the public health research themes that have been highlighted as important
by the NGOs respondent to the questionnaire.
We suggest as the next steps for the advocacy activities: press work to publicize the
preliminary assessment, organize an event in Brussels (end of 2007 or in 2008) to present
further SPHERE project results, maybe linking it to a review of the implementation of
FP7 (how the actual FP7 legislation is being implemented in practice, is it answering to
the needs of society? can we say we have learned the lessons from FP6?), which is
something MEPs might be interested on as they were directly involved in the drafting of
the FP7 as it stands now.
Another suggestion to improve public health research in Europe and its relevance for
society and policy making is to carry out capacity building for NGOs to take part in
transboundary cooperative research. This would provide NGOs with the tools and
knowledge to participate in international public health research projects, and it would
allow them to bring the point of view of society and citizens into ongoing research.
A final point that was mentioned by the respondents and that we have been able to
discuss with other project parnters in our meeting of 2007 in Dusseldorf, is the need to
support transboundary cooperative research: a smaller size research than the usual large
scale projects funded by the Framework Programme of the European Commission. These
large scale projects usually support projects with at least five partners from European
countries that bring a broad geographical balance (north, south, east, west), whereas
researchers sometimes would prefer partnerships of two or three organisations from
countries that are closer to each other or that belong to a single region of Europe. This
could be achieved by modifying the evaluation criteria to allow projects with less than
Appendix 1- The questionnaire
Strengthening public health research in Europe
Questionnaire on Public Perception
The European Public Health Association (EUPHA), through the Faculty of Public Health,
(United Kingdom), has proposed a project SPHERE (Strengthening Public Health in the
European Region) that will describe public health research at the European level,
including support by national governments, and advise how it can be strengthened and
most effectively integrated into European health policy.
The project is financially supported by the European Commission, DG Research
You can get more information about the project at its website
At present on level of the European Union there are two major research funding schemes in
Europe. The “Framework 6” program directed by the Research and Development Directorate of
the European Commission aims to support basic research and establish a European Integrated
Research Area. Public health topics are included in this major research program and if not yet
familiar, you can find more information about it at the following web site:
The second major public health resource under European Commission is the public health
program of the EC directed by Directorate of public health and consumer protection. This
program aims to fund more applied research, project dealing with implementation of new
knowledge into practice. You can get more information about this program at the following
The SPHERE study will be undertaken by a consortium of European public health
experts from several EU Member States. We would like to ask for your collaboration to a
part of the study that is undertaken by European Public Health Alliance (EPHA) and the
Slovak Public Health Association (SAVEZ). Its objective is to obtain information from
non-governmental organizations as representatives of public about public health research
in Europe. We would be grateful if you could fill in the attached questionnaire and return
it up to February 20, 2006 to Gabriel Gulis by E-mail (email@example.com).
Thank you for your participation!
Gabriel Gulis Roxana Radulescu
Name and address of your NGO
Area of work NGO
Duration of existence
Experience with international work (collaboration with international organizations,
partnership in international projects etc.)
Experience with international public health research work (public health research in this
sense is considered as a non-clinical medicine research tackling major problems of
population health. The above mentioned research programs describe more in depth what
could be considered as public health research)
Awareness about international public health research possibilities before this questionnaire
Have you ever heard before receiving this questionnaire about Framework 6 and the
Public Health Program of the EU? Please describe.
Have you ever been asked to participate in an international project funded by major EU
programs in field of public health? Please describe.
Have you ever heard or been contacted in relation to other international public health
research funding resources? Please describe
Experience with national, regional and international public health research
a. Have you ever been a partner within a national or regional public health research
project? Please describe
b. Have you ever been a partner within an international public health research project?
Please describe what would be, from your perspective, the needs for public health research on
national/regional and international level
National and/or Regional
Thank you for your time and willingness to collaborate!