Annex 3 Participant comments WHO/EDM-UNICEF Technical Briefing Seminar 15-19 March 2004 This long unsorted listing is primarily intended as feedback for presenters and organisers. Quotes are verbatim and a number of positive but non discriminatory comments have been left out of the list of evaluations if they had previously been said e.g. good presentation, very interesting, very good presentation, Great!, excellent information/discussions, very well presented, very informative information, Quite good, Good, Excellent day, etc Monday 15 March 1) 25 Years of Essential Drugs/Medicines- Impact & Challenges Very enthusiastic Good overview For all sessions it is important that there is room for questions and interaction Update the presentation Should have emphasised the way forward If only relevant papers could be distributed earlier (It is unclear what earlier means; before the seminar or at the start of the session? Copies for this session were provided at the onset of the presentation) Sets the scene for the briefing 2) National Drug Policies Good overview, good group discussion I learnt how important it is to have a national drug policy Participants to be encouraged to give views on situations in their countries, probably a pre seminar summary of the situation in their country would be helpful It is useful how Richard Laing cites country examples More information needed on country implementation of National Drug Policies. I would like to see more updated information/data regarding country situations Excellent overview but as is overview, is short: More coverage of the components of the National Drug Policy Nice with the emphasis on stakeholders: more of this 3) Update 2004 WHO Model List of Essential Medicines: Selection, evidence-based approach Some repetition from session 1 Has improved both process and outcome –CONGRATULATIONS Very useful description of the Expert Committee process Generated many questions and discussion and sharing. A lot of repeated slides from previous lecture (session 1) The process was interesting to hear about 4) WHO EMLibrary/ Model Formulary / EDM Document Center, Give short handout to remind people (where was this participant when the handout was distributed?) Very helpful, good to learn of document availability. CDR is great. Annex 3 Presentation good, but less clear/visible from the back of the room: Make it more interactive at the end of the day An impressive information tool I have a new knowledge about WHO website A little confusing. Use less abbreviations Excellent resource even though still work in progress. An incredibly useful resource. This is something I’ve been hoping for for a long time and even thought about seeking funding to develop! Highly impressive with the importance of the centre. There may it be a supplementary useful information in the price database such as suppliers international certificates (i.e. ISO 9000), kind of packing (bulk, blister etc it implies on the final costs) (difficult to read) Very interesting useful, could be taught at a computer facility for students to visit different sites and access different documents A lot of information and good advertisement for the site and is very useful as one browses the WHO homepage, may miss the EMLib. Highly relevant & useful for daily work Tuesday 16 March 2004 5) Medicine/Drug regulation and quality assurance Next time workshop plenary sessions should be incorporated to allow the sharing of experiences from various parts of the world so as to make issues more practical The written presentation (handout?)would benefit from being more detailed A bit more interaction in small groups would be good to keep it a bit more moving Whole day very interesting, but also very much information. Pay attention to interactions. Presentations (handouts?) beforehand are really very helpful Very useful explanation of originator v multisource manufacturers. Including country examples in the presentations would be helpful. Maybe I was sleepy and not following these two lectures well but I’m thinking that SK’s presentation was more general and should possibly have preceded LR’s. Given dark and warm room, the continuous lecture was trying. LR: Excellent. Difficult topic made very accessible. SK Good talk. Good synthesis of WHO work and position. Please give handouts before talk. SK referenced numerous QSM documents without giving the actual titles – frustrating –perhaps provide handout with document titles. 6) Assessment/strengthening regulatory capacity & assessment Summary document in print should be made available instead of referring to download on the internet Professional Associations should be involved to work side by side to the Drug Regulatory Authority Some elements were too detailed (EW presentation) A bit more interaction in small groups would be good to keep it a bit more moving Could use more practical examples As it is a key element in the quality of the pharmaceutical system in a country, may be more time should be spent on the presentation. Nice introduction by VR with helpful presentation visually of how regulatory mechanisms cannot be identical given varying country circumstances. VR Good talk, interested and interesting. EW talk a little to long slow. Black arrows on blue background not visible projected or printed. Could use references on Drug Information Annex 3 7) Pre-qualification of essential medicines WHO should try to assist those countries that do not have a Drug Regulatory Authority or are not able to establish any due to lack of personnel Define prequalification more comprehensively if possible A bit more interaction in small groups would be good to keep it a bit more moving Great, would have liked to delve deeper into reasons against prequalification Session should be moved from after lunch time to a better appropriate moment when people are more “aggressive” A critical presentation Very important topic, very well presented. Good use of time (including skipping slides to met time). Actually would have liked more time for this topic. 8) Medicine procurement, supply and distribution Not enough discussion More detailed written material Good to do the case studies Use of case study very positive Very interesting debate and country case study Good to include this presentation. Although it repeats some information, it ties together many topics covered in previous presentations with a practical applications slant. Very good talk, case study useful, made material real. Please add speakers names to evaluation forms (noted) Wednesday 17 March 2004 9) Access to essential medicines: franework It is important when asks for people’s opinions, to use these opinions and inputs Duplication of presentation done before (session 8?) Good use of participation & input from the class. Good to repeat some of the concepts such as EML stemming from STGs. Good graphical depictions of access issues. Interactive and good presentation Highlighted some issues which I’ve never thought about with regard to access to drugs Discussion at the beginning good, good way to start the day. Good adaptation of talk in real time to skip previously discussed material Very clear, more examples of access issues 10) HIV/AIDS More practical cases (even though I know it has just started) Very useful orientation to AMDS. Given all the questions about this new initiative, the approach employed to address them all was very effective. Good presentation. More time for discussion needed Has brought up many matters/areas which I have no experience or knowledge of, and hence has been very educational and enlightening to me Very useful talk – 3 x 5 so important. Needed more time. Excellent job in the time allotted Need more time for this to discuss issues Annex 3 11) Malaria Nice opening approach engaging the participants. Therapeutic efficacy of SP slide is a bit confusing. Perhaps there is a clearer way to present this data. Would have been helpful to obtain more info M2S2 initiative. I understand it is new but this is the perfect venue to provide us with info on it. This was a missed opportunity I feel: And response to questioning was not complete Good presentation. More time for discussion needed. Presentation (handout) in advance useful Big issue and short session Has brought up many matters/areas which I have no experience or knowledge of, and hence has been very educational and enlightening to me. The publication of the quality tests of different antimalarials in Africa is just mind boggling Great talk – very – timely - useful to get overview of ACTs and associated problems. Speak a little louder 12) Tuberculosis: Too short Too short. TB always gets too little attention RL knows this topic so well, that its always a good presentation. Sorry he got short time – but it was the right presentation to cut Good but rather concise and too little time for discussion Big issue and short session Need enough time to discuss TB medicines Not enough time to hear more on this topic Insufficient time for talk; handled time shortage well A pity that it was shortened The time was so restricted Would be interesting to know more about WHO strategies for HIV/AIDS and TB 13) Medicine prices: WHO/NGO Price Survey Methodology I think it is an important but (delicated?) approach Its important to discus better the methodological details to enforce reliability and understanding of validity Very interesting , also more time needed for policy part Good to have objectives of presentation presented at the start of LL’s & AC’s presentations. Interesting method of presented by LL. Although the Kuwait information is interesting the way it was presented was not concise or clearly understandable. Given so much preliminary info available, would it be best to highlight a couple of points only and leave sufficient time for final speaker and questions. Although it was really interesting to hear about KT’s and LL’s work, I think it might have been more beneficial to hear more from AC, particularly the policy info. KT & LL raised great issues, but perhaps shorter, tighter presentations by them with more analysis + discussion lead by AC might have been better. Very interesting presentations Interesting topic to be discussed further Synopses of studies generated much interest and the study methodology can be should be used by countries to help them do their own price studies . results used to drive policies. Really important topic! Good to see data 14) Survey of procurement in Church-based facilities Nice to have different perspective Work in progress I was not clear on the research questions. Presentation was rather vague Presentation of the study only, should be accelerated to get data as soon as possible Annex 3 Generally informative – never heard of FBO, could perhaps elaborate on this especially on their roles and function/work generally Interesting but would like to have had more details Look forward to results Thursday 18 March 2004 General Comment: It would be useful to get EDM phone list at the beginning of the week. Also need to include those who presented to us but not on the list i.e. SL and CO 15) WHO strategy for working with and in countries (SA – GF) Although the organisation of entities at high level is rather ethereal in most cases, it’s still useful to be exposed to the attempt to get organised. Good to hear SA present this info, though it seemed like a lot of words with little substance. GF needs to increase his volume. I was lucky to be at the front today. Very good presentation by GF Very interesting and comprehensive debate Until now the least inspiring session, though highly relevant. Too much information, not enough time for discussion. Identify clearly, key information and problems. For all sessions do not put too much information on the slide/sheet – difficult to follow and later reread Too much detail on structures: more” vivid” presentations would be welcomed Lot of words where is the action? Both talks could be made more relevant: SA Please explain abbreviations: CSU, PME; GMG, PRP, HRS, CBF…… GF Please speak louder Quite diplomatic talk 16) EDM strategy for working with countries- The Africa region: 3 x 5 initiative Very very useful to learn about EDM’s efforts to assess ARV availability. I have a problem with presentation of data for country B. Probably best not to present data which is invalid. Availability indicator requires a physical inventory, so if this was not done, then say so. But “No records” or even “No data” is really not clear depiction of situation Excellent presentation: waiting for the results of the study. Excellent debate Work in progress clearly, but interesting and highly relevant Would have loved more on going discussions Excellent talk – evidence based. Current “Down to ground” debate very important I guess when research is not concluded the most important aspect to be stressed is the pro and limitations of the methodological approach (not very clear this statement) 17) Indicators on pharmaceutical situations Good review Very good useful presentation Good talks good speakers. Useful tool with good explanation with evidence collected to demonstrate the” real world” 18) Globalisation, international trade agreements, Doha, Cancun Very helpful Very good presentation Excellent presentation by CO Very good useful presentation Annex 3 Very clear exposure on a cloudy matter Important issue; case study would be interesting Speaker spoke too quickly It is heavy going but explained well. CO has added to my understanding Excellent stand in! Very much and complete information; so enough time is essential: Excellent sheets. Great that CO managed to stand in for GV. More time for this topic – very important but complicated Way to much information within one hour, not at all clear for non lawyers or someone without previous background Very good job covering someone else’s talk. Very very important. Some aspects weren’t defined in talk….. Well presented, group discussion would have been useful. Not fully exhausted in my opinion, appears to have been rushed Friday 19 March 2004 19) Pharmacovigilance Nice overview. Needs more specific tools/instruments for implementing such a scheme Give more information on the ways to report and highlight the importance of doctors having to report to pharmacist for adverse reactions (antagonism between MD and pharmacists) Nice to put names to presentation subjects. Useful info shared Excellent talk. Began with basics and built up. Good synthesis of very important material. Good presentation style. Clear presentation, good introduction to range of materials and support approaches by WHO Good sheets, would have been helpful to have them in advance. Case study would make the presentation more concrete. Exciting! Would be interesting to see research done with reporting by healthcare professional…should/could Pharmacovigilance be included in main curricula in health schools? As it applies to traditional medicine has not been highlighted Show more examples to put things into context and perspective. 20) Promoting rational medicine use Excellent presentation. The following could be added * reimbursement as a tool for improving RUD & the importance of reference pricing both for synonyms and analogue drugs for RUD Very good interactive session. Well done! Next seminar should include more time for this topic; PRDU for prescribers and PRDU in the community More time for this It was well presented and has been beneficial Move this session from the last one in the agenda to another time in the week. Show evidence, if any, that rational drug use might not be so good in developed countries I like KH’s interactive style to present the key points in her presentation. Important, but just too many facts. Could this talk be split into 2 parts on two days. Many charts not readable in black and white. Not as clear on tools and approaches that could be used and drawn upon. Too much to cover in too short a time – needs longer session and possibly earlier in the agenda of the week – RDU is a critical element in whether change happens. Interaction is good, but questions were not always clear to me! Probably because I am not really working in this field. Very good integration of studies to illuminate the issue. Stimulating and interactive Annex 3 Very interesting! Very good and clear presentation style. Good examples Very loaded information [loaded in the sense of too much, not biased I hope!? RG] Suggestion: Few selected (2) of good cases and (2) bad cases across the globe would be beneficial Excellent great and interesting Well balanced in the realities and advances made in this area. Excellent talk. More time to be allocated for this topic Very important well presented session. This session should have been given more time for review and discussion. 21) General Comments on the Seminar An excellent training seminar – it is just a pity that time is so limited. The most outstanding was the prequalification programme and the RUD presentation Very welcome and interesting seminar. The more interactive the sessions, the more interesting. Some presentations were “dull” and therefore not interesting (e.g. session 15) Good reading material and handouts. You could hand out some reading material for reading before each day. RL is an excellent and well informed facilitator. And Robin has a very good sense of humour! (Thanks. Glad you noticed. It makes it all worth while! RG). Thanks a lot for being given the chance to join this seminar. Extremely good/helpful getting all the presentations on the CD-ROM!!! (could, though, have been announced on day 1)[Yes it could have been announced on day 1 but there was only a 50/50 chance that it would be available by 12.00 on Friday. It is foolish to contract/announce something one may not be able to deliver. However there is next time and the comment is noted .RG] Very nice with all the “high level” presentations making the workshop/briefing very good. Good to meet so many of the EDM team. On the whole the seminar is good but why not rotate it? i.e. one time in Geneva and another time in South Africa or Ghana [The answer to that is quite simply money. The seminar incurs a minimum of expenditure for PAR; coffee & reception ($US 1,366) and functions with staff already in Geneva, It’s run on shoe string as the saying goes. As you will note on the flyer/publicity PAR is not in a position to support travel ,board and lodging or any other form of aid for participants. An official WHO meeting costs between $US 80,000 - 90,000 for a team or department. Another element is that it gives a chance for participants to meet not only EDM staff but also people from other HQ departments or clusters i.e. networking possibilities. RG] This very interesting and comprehensive seminar needs to be held in French or at least to involve French speaking African countries [This has been brought up before and will be brought up again. I am not so sure whoever mentioned this realises the practical aspects and implications of such a suggestion. RG] Would be helpful to get finalized participant list early so we can make notes on it. [ You would be surprised at the discrepancies between completed application forms and the final participant list. Participants are asked to check the list and there are often changes resulting from typos (here) but also participants changing what they wish to be on the record. Noted however, we can try for next time. RG] Also, please provide EDM staff list with telephone numbers on 1st day to help understand EDM organisation + to be able to find everyone easily. [ Point taken, I slipped up this year. RG] Love the idea of getting CD with all the presentations. [Noted RG] Also glad that almost all docs are available on-line. Good to mention regularly the” library” for EDM . How luck you are to have TM. He is a very helpful member of the EDM team. An excellent seminar. Thanks. Provide board to list all URLs mentioned in class; provide World map [The location of Iceland and Brunei where not know by some participants RG] It would be useful for certain sessions to have some preliminary readings [handouts?]In the patent session [Session 18] perhaps have us read the 6 page briefing paper the night before. Please be very careful with abbreviations and acronyms [Could not agree more RG]. PEPFAR used on Monday was Annex 3 defined on Thursday. Post one sheet of paper on side wall and collect during the week? Handout at beginning of week? I was told that this seminar is very important to EDM – that all staff are here this week. Tell that in the beginning. [Correct for previous seminars but not for this one: request for staff travel ban for this week was asked for but was refused by previous Director on the grounds that it would curtail staff travel. It would have meant for 2004 bans or partial bans for two Technical Briefing Seminars (TBS), the Executive Board (EB), World Health Assembly(WHA) and Meeting of Interested Parties (MIP) RG]. Allocate 1 hour at the end to brainstorm how to take this information home and begin to use ideas discussed during the seminar. I really got a lot from this valuable course, and I really from now on will encourage everybody to participate in it because it put me in the right direction towards WHO strategy In general very balanced in terms of subjects addressed. Variation in presentations: some people have a tendency to give all the information available and not key messages. The presence of participants was not always used optimally with so much experience from different settings. Level of participation different: how to use everybody, be inviting and not get caught in the trap of using the more vocal people. I felt very welcome by WHO and the participants all felt very free to speak up. Thanks. Very good overview of issues in EDM as always – too short time for interactions between participants – maybe a structured evening programme with short presentations by participants instead of the dinner evening to improve discussion between participants. For the dinner find another venue with a polite staff - and give the little participation of people from developing countries – maybe in a centre like the John Knox it might be possible to organise a less expensive opportunity [Agreed: RL will find a better place and organise it next time RG]. Very committed team of speakers. Very creative and convincing Master of Ceremonies MC( = RL) It was great to have the opportunity of participating. I thank a lot WHO for funding me [ Not EDM] It’s very expensive and few people have this opportunity each year. I think it would be stressed the commitment to replicate and divulgate the information we got here. For the monitoring and evaluation experiences shown here the methodologies’ limitations, advantages and cost should be emphasised – to use the results for policy making the data must be reliable and feasible. THANKS VERY MUCH I think Indonesia’s experience in procurement and distribution of medicines in the public sector will be beneficial to colleagues in other countries. Elaborate further the topic Development and implementation of national drug policies. Need time to discuss further too certain topics There is need for country presentations so as to share best practices and make presentations relevant. Accommodate group work Promote interaction from the start. Decrease length of presentations. Increase discussion and case study Please explain abbreviation…only point of concern The Seminar has been a valuable source of precise and relevant information for me. I hope to be able to provide part of it – especially regarding regulatory matters – to the German Association of Drug Regulatory affairs. Thank you very much for this valuable opportunity to get a profound back ground and so much more interesting perspectives to work in the future. The experience was great, as an overview, congratulations. Could be interesting to introduce case studies and to have a panel of discussions of some experience (selected) with advice from the staff and participants to improve it. Could be used more PBL [Problem Based Learning] learning strategies with more opportunity for sharing experiences The Seminar provided good information about different areas in the pharmaceutical sector. The Seminar helped me a lot to think, plan and do well. It has really put me on the right track which is great benefit to my country. In general the programme was overloaded this not allowing thorough discussion. Excellent course. RL is an inspiration and manages to make the course even better. I am very motivated and happy to have received a broad view on essential drugs and put the access component into perspective. Constructive criticism: short presentation from participants on situation of essential drugs in their respective countries? The course should have lasted at least for 2 weeks!!!! Annex 3 Technical Briefing seminar was very technical indeed but enormous amount of information transferred in a short time. More time for questions needed. Very good interactions between participants and seminar staff and presenters. I think this is a very good seminar. The topics were relevant and the presenters were all experienced people. I would however be happy if traditional medicine can be included as this is becoming one of the areas being promoted by WHO The whole course has been enlightening and has brought up more issues I never thought of. It has also highlighted many of my shortcomings, in terms of knowledge of all the good work of WHO & their ongoing campaigns. Can WHO ensure that those in the MOH who have more power over policy directions to attend this technical briefing? Although I am also in regulation but I cannot as effectively as the panel of experts, impress upon them the importance of some of the issues discussed and brought up this week. In general a very good seminar with excellent information that would lead to help improve drug supply. Such seminar should be continued once or twice yearly. Thank you very much.