VIEWS: 6 PAGES: 13 POSTED ON: 9/12/2012
2012 Inaugural “Digital Pathology Showcase” Draft – Version 3 for Public Comment Objective The primary goals of this event are to provide the audience with a good sense of the state-of-the-art in WSI-based digital pathology and provide visibility into the viability and readiness of digital WSI technology for routine use in pathology: “is it ready for prime time?” To accomplish that, a set of pathology cases will be provided by the organizers and reviewed live by practicing pathologists. Participating vendors will provide their equipment to the pathologists to showcase the capabilities of their WSI technology. Systems will be subjectively and informally judged by the audience, observing the time it takes and the quality of the diagnostic recommendation based on the WSI reviews. The inaugural event will exclude issues such as workflow optimizations via LIS integration and advanced Computer Assisted Diagnostics (CAD). The case reviews will be done based on manual review of the WSI and using basic annotation capabilities. In future years, we do expect this to grow and include some of the excluded aspects. Logistics We are anticipating 3 vendors participating in the 2012 event with duration of approximately 3 hours. The order of presentation will be pre-determined so that each vendor will present first, second and third of the three scenarios proposed. We plan to interleave the review of the three types of pathology cases so that a subjective side-by-side comparison will be possible for the audience. We are considering some overlap of the scanning for the second case scenario (multi slide biopsy with a focal cancer) to ensure the even can be conducted in a timely manner. We will make that determination once we do have confirmation of the three participating vendors. Cases Each vendor will be given a set of representative cases in advance of the event so that it is known what type of cases will be expected to be reviewed during the event. Except for the first case, those cases will be replaced prior to the event to ensure the pathologists do not know the diagnostic outcome of the case but will have to perform a “live” review. The three sets of cases will be: 1. Frozen section: 1 slide Slides will be dry and scanned as well as reviewed “live”. The total time to scan, transfer, and diagnose the slides “live” will be observed by the audience. 2. Biopsy with a focal cancer: 6 to 9 slides Case will consist of 2 or 3 blocks and 3 slides per block, H&E only, various cases of infiltrating breast. Prostate or colon carcinomas with the cancer being present it at least one of the blocks. The case shared ahead of the event and the one being reviewed during the event will not be the same. New slides will be provided shortly prior to the session so that there is enough time to scan but no access to the images by the pathologist, thus ensuring a “live” review of the case. The time it takes the pathologist to come to their diagnosis will be observed by the audience. 3. Immunohistochemical Quantitative Interpretation: 1 slide Case will consist of one immunohistochemical stain of a breast biopsy for ER, PR or Her2Neu interpretation. The case shared ahead of the event and the one being reviewed during the event will not be the same. New slides will be provided shortly prior to the session so that there is enough time to scan but no access to the images by the pathologists, thus ensuring a “live” review to determine the finding in terms of the biomarker examined.. Rules The organizers of the event are setting forth a set of rules to make this a valuable comparison. In the following we list the guidance for the vendors on what is expected as well as permitted: Demonstration setup is required to include scanner as well as needed workstation and server(s) Demonstration to be conducted by practicing pathologist(s) and a pathology technician (histotechnologist, IT or scanner personnel or other non-vendor personnel. No vendor employees are allowed to take place in the actual demonstrations. Software needs to be a commercially released version, no Alpha or Beta software allowed. Hardware equipment needs to reflect a common commercial deployment (e.g. do not replace standard hard disks with solid state drives to boost performance beyond what customers can afford in a real deployment) Prefetching or pre-caching of cases will not be allowed and network utilization will be monitored Additional rules might need to be put in place as we refine the draft of this document. Setup The setup will include scanners, client workstations, and possibly server infrastructure, depending on the deployment needs of each vendor. Vendors are asked to setup the equipment the evening before the event and will be assisted with respect to the network setup. The network bandwidth between the scanners and the respective storage unit (server) will vary based on each case review. For the Frozen section, a 2Mbit/sec network bandwidth and 100ms network latency will be applied while for the other two cases 20Mbit/sec network bandwidth and 30ms network latency will be applied. This is to reflect reasonable network conditions for a remote read of a frozen section as well as regular reads throughout a hospital network. If selected vendor(s) requires a higher degree of network performance that will be considered and all vendors will have access to any upgrades from this specification that occur. A total of three projection screens will be available. Screen #1 will display a view of the respective scanner during the “live” scanning of the frozen section scanning and subsequently a view of the keyboard + mouse of the pathologist reviewing each case. Screen #2 and #3 will show a view of the two screens of the workstation the pathologist will use to read a case. This setup will allow the audience to fully comprehend the effort and assess ease of use of reviewing each case. Appendix Suggested room layout: (This can be altered to fit the available space) 3 en # Scre Screen #2 Podium 1 en # Scre ... ... Suggested network setup: Servers of vendors Clients of vendors #1 … #3 #1 … #6 Apposite Linktropy Mini2 (WAN emulator): does bandwidth up to 100Mbps and latency. ET Inc. Bandwidth Manager: limit bandwidth on the ... LAN up to 1Gbps. ... WAN latency emulator and LAN bandwidth simulator ... Scanners of vendors #1 … #3 Comments from Vendor Interviews (Draft from MB – some not approved/verified for release) Interviews with Industry Partners Mark Newberger (MN), Apollo Telemedicine, Inc June 6th, 2011 Interviewer – Mike Becich (MB) Summary of Mark’s Newberger’s comments: 1) Overall MN thought the idea of a digital pathology showcase as proposed in 4/20 Draft is a good idea 2) He also thought the 18-20 month lead time for the showcase in fall 2012 was a good idea as well 3) He did admit that he initially thought we were planning to have the showcase this fall with a 4 to 6 month lead time 4) He had issues with companies like Apollo (which does workflow software and is scanner agnostic) would have to partner with one or two other companies to compete in the showcase. 5) He didn’t feel that the current draft would yield more than two or three vendors who could do end to end (scanning, processing/display and diagnostic workflow) digital pathology 6) He favored breaking the showcase into segments (scanning, processing/display and diagnostic workflow) and having different vendors for each of the segments 7) MN suggested that having a “training” program for the vendors on the technical set up a day before the 2012 meeting was a good idea 8) MN and MB suggested that the technical specification for the showcase set up be made available for testing at the company sites. MB suggested that Pittsburgh would offer a “test” setup in Pittsburgh at least 11 months before the 2012 meeting to let companies decide if they want to participate (test cases and technical set up). 9) MN agreed that offering the first six vendors who committed to the showcase was a good idea but was concerned you might not get six the first year. Vendors might be concerned how they would “show” in this type of open showcase. 10) MN also thought the showcase would provide good marketing potential to their customers as well as new customers. 11) MN agreed the “open” sharing of the whitepaper and the forum discussion as an industry panel at Pathology Informatics 2011 was a good idea. 12) MB closed the meeting by asking MN if he would mind sharing his comments on the showcase with the broad pathology and vendor community and he agreed. Interviews with Industry Partners Ole Eichhorn (OE), Aperio June 10th, 2011 Interviewer – Mike Becich (MB) Summary of Ole Eichhorn’s comments: 1) Overall OE thought the timing and planning for the industry panel in 2011 and the digital pathology showcase in 2012 was a good plan 2) OE stated that Aperio considered being involved in the European Scanner Contest in 2010 but decided not to participate because of the “closed” nature of the planning and evaluation 3) OE felt the current plan for the 2012 Digital Pathology Showcase was a starting point but suggested more complex activities (e.g. quantitative analysis and algorithms) might be considered as well. 4) OE stated that Aperio (as leader in the field) was interested in further dialogue on the format of the showcase in 2012 and would circulate the document to others in Aperio. . Interviews with Industry Partners John Wellbank (JW) and Jose Costanon (JC), Phillips June 17th, 2011 Interviewer – Mike Becich (MB) Summary of John Wellbank’s and Jose Costanon’s comments: 1) Both JW and JC liked the idea of the industry panel in 2011 and the digital pathology showcase in 2012 but raised some concerns. 2) JC felt that Phillip’s would like to see higher volume of slides scanned included in the showcase. When MB told them that there were plans for 10 to 20 slides for the second (biopsy) and third (resection) case this did not fit the “higher volume” scenario. MB assured JC that this would be considered and discussed in the industry panel in 2011 and as part of the “open” comments from the community. JW added that Phillips would need scanning in 20 and 40x as well as more volume if possible. Phillips does not scan in 20x equivalency so that test is not possible. Therefore we need at least some of this test to scan resolutions equivalent to 40x for true time comparisons 3) JW was concerned that the network bandwidth and latency listed in the current (attached) draft of the plan were not sufficient from Phillip’s perspective. MB reassured both JC and JW that network bandwidth and latency will be provided to the highest specification requested by the industry partners. MB asked JW/JC to provide a network specification and that MB would make sure this was part of the plan. 4) JW felt the bar might be set “too low” for the first meeting and would provide additional feedback from others at Phillips on the case scenarios. 5) JW was concerned that there would be no time for vendors to test the setup for the Industry Showcase. MB reassured JW that a “pilot setup” (simulating the setup for 2012) would be available in Pittsburgh for at least 11 months in advance of the showcase and made available for scheduling to industry partners. MB also stated that the specification for the Industry Showcase would be made available to all interested industry partners in November 2011 should an industry partner want to replicate the environment in the company headquarters. 6) Overall JW and JC thought that the timing (over 18 months of lead time) and consensus building with industry partners would ensure a successful showcase. Interviews with Industry Partners Philippe Nore (PN), Ventana July 5th, 2011 Interviewers – Mike Becich (MB) and Bruce Friedman (BF) Summary of Philipe Nore’s comments: 1) Don’t want a contest that “compares” vendor on only one dimension (perhaps speed of scanning or image quality). 2) Don’t want to promote instruments that are “skewed” on the basis of any one factor (e.g. a “contest” specific set of criteria). 3) Don’t want to end up pushing company production of instruments and software to “win the contest”. The focus should be on how to meet the needs of customers. 4) Question – Will image analysis be a part of the first showcase? Ventana would like to see a IHC or ISH image analysis as part of the first showcase. PN felt many of the initial digital pathology solutions use this today, so why not showcase this to customers today. 5) Will the event will have time for six vendors demos (may be too many). 6) PN thought that even having 4 vendors would be hard to keep audience’s attention through the entire event and should limit time for each exercise. 7) Ideally for any event you would want to be vendor #1 or 2 but not 3 or 4 (much less 5 or 6) in the current setup due to “repetitive/redundant” nature of the tasks 8) Commentary – reduce the time for each serial demo by vendor. 9) PN thought by 2012 everyone would have a image analysis algorithm. 10) PN thought 20 minutes per vendor should be considered – based on his experience with customers. 11) PN asked what “live mode” means. Does this mean scanning occurs on stage. In the Ventana system the slide viewing is dynamic robotic so scanning a frozen section is not required once mounted on their hardware. Ventana would like the flexibility to view the slide without scanning for the frozen section exercise. 12) No FDA approval for primary diagnostics so exercise # 2 and # 3 may be out of question. They need to check with their regulatory lawyers to make sure this event is misperceived as promoting non-FDA approved use of devices. 13) User training is an issue and need this to be addressed. Who will run the demos? How will Pathology Informatics deal with this – told PN about the 3 solutions we propose for this including: a. Availability of a “mock” setup in Pittsburgh b. Availability of a “specification” for the setup and encouraging replication of the setup at the vendors headquarters or at a key clients location. c. Training experience on the “mock” system in Pittsburgh will be shared with users/vendors. 14) Ventana would require their own scanner, image server and application server loaded in advance. Can’t share with others. Will follow up with PN on the specs. 15) Separate fee for vendor participation in the Industry Showcase was PN’s recommendation. Rob Liepse (RL) and Stephane Willaert (SW), Barco July 20th, 2012 Interviewer – Mike Becich (MB) Summary of Rob Liepse’s and Stephane’s Willaert (SW) comments: 1) SW commented that the European contest was only focused on the scanner. SW felt that the “end to end” solution including software, servers, etc... to be included in the showcase was a good focus for the 2012 event. 2) SW asked if there would be opportunities to have multiple vendors partner for the various partners for the component technologies. 3) SW questioned what role did I envision for Barco in the Industry Showcase and it seemed Barco’s role would be in the display of scanned slides. 4) SW stated in the Berlin scanner contest their role was as the display partner for all the vendors that participated. 5) SW stated that Barco is working with all the vendors in the digital pathology space. 6) RL mentioned that they have a very large footprints in digital mammography and PACS displays. They have been a repeat vendor at RSNA for years and participated in IHE (Integrating Healthcare Enterprise). They have extensive experience in primary diagnosis in the mammography space. 7) RL mentioned that Barco, worlds only approved display for TOMO (tomosynthesis – mammography tomography – have 510K approval) and have been very involved in standards efforts and moving this technology to the masses. 8) SW thought that the focus of the proposed Industry Showcase was appropriate but that it would make a number of the vendors nervous due to the lack of industry focus on “end to end” solutions. SW felt this would move the industry forward but it may be ambitious. 9) SW thought the technical focus on bandwidth support for the demo was a good one and that not enough studies are looking at the effect of the network on the viewing experience of the diagnosing pathologist. Interviews with Industry Partners Doug Giszczynski (DG) and Donal O’Shea (DO), Leica July 26th, 2011 Interviewer – Mike Becich (MB) and Bruce Friedman (BF) Summary of Leica’s comments: 1) DO - Leica participated in the European event and was please that they won awards for their products at the event. 2) DO and DG thought the European event was well communicated and went well but questioned the value of some metrics. 3) DO thought the document was clear and the scenarios and technical setup were useful. 4) DO thought the document needed more details on the metrics and quantification of the measures of the event. 5) DO wanted clarification on the scenarios and the timing (wasn’t clear). MB explained the scenarios and order of the vendors would be shuffled. BF very much liked MB’s idea of having the companies work different scenarios sequentially: vendor #1 works scenario #2; followed by vendor #2 works scenario #1, etc. No vendor then is in an unfavorable position in terms of the order of presentation. However, this causes problems if only one vendor is on the stage at any time with the rest in the bullpen. Lot's of confusing set-up time on the stage. Could get tedious. It seems to me that the only way this would work if all three (or four) vendors are continuously positioned on the stage at the same time with the "spotlight" on each as they go through each scenario. Then, only the switches need to be reset for each vendor. 6) DG what is the output of the showcase....need to finalize. MB stated that this would be proposed in the next draft of the document. Frozen section scenario would have 3 metrics - 7) DO queried on who would run the showcase and MB stated that a pathologist and a technical assistant who used the system would be running the company’s system and not vendor staff. 8) BF asked about the second event was canceled in Europe. DO did now know why Helsinki event was canceled. 9) DO thought the “end to end” scenarios and focus on the workflow was an appropriate one. The focus on how the digital pathology wholistically fits into their laboratory is a good and clean goal. 10) DO thought the technical setup was robust and adequate. Leica wanted gigabit between the image server and the viewing software. Include the make, model and specification of . 11) DO mentioned the potential of include a wireless broadcast mode during the showcase (very interesting). BF really liked this idea having the images available wirelessly for the audiences with iPads and smartphones. 12) DG asked for a full day instead of 8 to 10 hours to get the work pods for the showcase setup and testing. There will need to be able to do this in “private” to protect company intellectual property (IP). Sturdy tables should be available so that vendors can set up in a private area and then have their setup “wheeled” into place. BF thought that we should consider having set-up the previous afternoon/evening and the morning of the showcase. The showcase would then start at 1:00 on the workshop day and extend to perhaps 5:00 for a total of four hours (with perhaps a break). This would allow registrants for the event to fly in to Chicago that morning. 13) BF thought the event should be in the afternoon to allow for same day travel for participants and to give more time for setup. BO agreed that this would be ideal. 14) BO thought having a 5 to 10 minute time for the vendor to speak during the opening of the event perhaps in “dead-time” (during scanning perhaps). DG agreed but thought this would be better at the end of the event. BF thought a closing remarks or panel Q&A would be useful at the end of the workshop. I think perhaps that we should have a wrap-up at the end with the thee reps at a table at the end to answer questions and wrap-up. Given the three hour time limit, I think that this may be difficult with four vendors. 15) BO thought the company’s selected pathologist would be a valuable spokesman and should be drawn into discussion and commentary. 16) BO thought we should be clear on what will be published and where it would be available and to whom. 17) DG thought approaching media and medical societies (many housed in Chicago) about the event would be useful. 18) DG wanted to make sure this was fair to all participants and having more details on the output of the event was critical. 19) BF thought we should attempt to videotape the entire session and make this available on the web site. Documenting the entire event would be of value to the participating companies. 20) BF commented on the potential name of the Vendor Showcase and taking into consideration the emphasis on end-to-end demonstrations and the Leica emphasis on workflow, I think that we should refer to this event going forward as: Vendor Showcase: End-to-End Digital Pathology Workflow. Interviews with Industry Partners John Fonte (JF), Metasystems July 26th, 2011 Interviewers – Mike Becich (MB) and Bruce Friedman (BF) Summary of John Fonte’s comments: 1) Overall JF thought the idea of a digital pathology showcase as proposed in current Draft is a good idea, especially the “open” and “consensus building” for the showcase. 2) JF thinks the focus on end to end solutions from scanning, to display and diagnose is a good one for the industry. 3) JF stated Metasystems has a slightly different take on systems looking at FISH, ISH, fluorescence and molecular testing on whole slides and this can be done remotely by matching an identified area on light microscope images to areas for analysis at a remote site without pathologist having to re-identify areas in the special testing suite. 4) JF would like to see this (see #3 above) as a case scenario. 5) JF despite the company’s focus as stated above, they think the showcase is an important event and they (Metasystems) supported the European scanner event as a participant. 6) JF thought that having pathologists and technologists running the demos was risky but that generally valuable. 7) JF was pleased with the technical setup and lead time we promised provide. 8) JF was going to circulate the documents who were more involved in the European contest. Interviews with Industry Partners Rajiv Enand (RE), Omnyx, LLC August 13th, 2011 Interviewers were Mike Becich (MB) and Bruce Friedman (BF) 1) RE sounds like a fantastic event and put together nice ideas which will help industry and pathologists alike. 2) Sept 2012 would be a great time for an event like this in terms of market acceptance 3) Wanted to show commercial development if possible. 4) RE asked about venue – Hyatt Grand in Chicago 5) BF reiterated about having color commentary on the stage during the event. 6) RE wanted to know if this was a “contest” or a demonstration. BF and MB reiterated that this would be a “showcase” more like a demo but emphasizing end to end integrated workflow. 7) MB told about fall backs – specs, demo set up, etc... 8) RE felt that videotaping was of little commercial value to vendors. Marketing value is how many participants in the venue. 9) RE had questions on marketing the Industry Showcase – simulcast on web would be a good idea. Vendors like Omnyx would help to promote the event. 10) MB committed that there would be nothing running against this event. 11) RE commented on the amount of time the companies would have to spend to prepare for this so everything Path Info did to drive both attendance in person and on-line would be very important to get participation. 12) RE thought this is an important event to encourage adoption and provide this as a valuable service to the pathology community. 13) RE said this would be of great interest to Omnyx. 14) RE thought a “judges choice” award would be a great idea. 15) RE agreed that image quality would be a useful metric worthy of an award or recognition. 16) RE thought that a narrative assessment by judges that were quotable document published as proceedings in Journal of Pathology Informatics. 17) RE thought we should try to get unbiased judges who would use the technology.
Pages to are hidden for
"Digital Pathology"Please download to view full document