Digital Pathology by Glmz5fX

VIEWS: 6 PAGES: 13

									                   2012 Inaugural “Digital Pathology Showcase”

                         Draft – Version 3 for Public Comment

Objective
The primary goals of this event are to provide the audience with a good sense of the state-of-the-art in
WSI-based digital pathology and provide visibility into the viability and readiness of digital WSI
technology for routine use in pathology: “is it ready for prime time?” To accomplish that, a set of
pathology cases will be provided by the organizers and reviewed live by practicing pathologists.
Participating vendors will provide their equipment to the pathologists to showcase the capabilities of
their WSI technology. Systems will be subjectively and informally judged by the audience, observing the
time it takes and the quality of the diagnostic recommendation based on the WSI reviews.

The inaugural event will exclude issues such as workflow optimizations via LIS integration and advanced
Computer Assisted Diagnostics (CAD). The case reviews will be done based on manual review of the WSI
and using basic annotation capabilities. In future years, we do expect this to grow and include some of
the excluded aspects.


Logistics
We are anticipating 3 vendors participating in the 2012 event with duration of approximately 3 hours.
The order of presentation will be pre-determined so that each vendor will present first, second and third
of the three scenarios proposed. We plan to interleave the review of the three types of pathology cases
so that a subjective side-by-side comparison will be possible for the audience. We are considering some
overlap of the scanning for the second case scenario (multi slide biopsy with a focal cancer) to ensure
the even can be conducted in a timely manner. We will make that determination once we do have
confirmation of the three participating vendors.


Cases
Each vendor will be given a set of representative cases in advance of the event so that it is known what
type of cases will be expected to be reviewed during the event. Except for the first case, those cases will
be replaced prior to the event to ensure the pathologists do not know the diagnostic outcome of the
case but will have to perform a “live” review. The three sets of cases will be:

    1. Frozen section: 1 slide
       Slides will be dry and scanned as well as reviewed “live”. The total time to scan, transfer, and
       diagnose the slides “live” will be observed by the audience.

    2.   Biopsy with a focal cancer: 6 to 9 slides
        Case will consist of 2 or 3 blocks and 3 slides per block, H&E only, various cases of infiltrating
        breast. Prostate or colon carcinomas with the cancer being present it at least one of the blocks.
        The case shared ahead of the event and the one being reviewed during the event will not be the
        same. New slides will be provided shortly prior to the session so that there is enough time to scan
        but no access to the images by the pathologist, thus ensuring a “live” review of the case. The
        time it takes the pathologist to come to their diagnosis will be observed by the audience.

    3. Immunohistochemical Quantitative Interpretation: 1 slide
       Case will consist of one immunohistochemical stain of a breast biopsy for ER, PR or Her2Neu
       interpretation. The case shared ahead of the event and the one being reviewed during the event
       will not be the same. New slides will be provided shortly prior to the session so that there is
       enough time to scan but no access to the images by the pathologists, thus ensuring a “live”
       review to determine the finding in terms of the biomarker examined..


Rules
The organizers of the event are setting forth a set of rules to make this a valuable comparison. In the
following we list the guidance for the vendors on what is expected as well as permitted:

       Demonstration setup is required to include scanner as well as needed workstation and server(s)
       Demonstration to be conducted by practicing pathologist(s) and a pathology technician
        (histotechnologist, IT or scanner personnel or other non-vendor personnel. No vendor
        employees are allowed to take place in the actual demonstrations.
       Software needs to be a commercially released version, no Alpha or Beta software allowed.
       Hardware equipment needs to reflect a common commercial deployment (e.g. do not replace
        standard hard disks with solid state drives to boost performance beyond what customers can
        afford in a real deployment)
       Prefetching or pre-caching of cases will not be allowed and network utilization will be monitored

Additional rules might need to be put in place as we refine the draft of this document.


Setup
The setup will include scanners, client workstations, and possibly server infrastructure, depending on
the deployment needs of each vendor. Vendors are asked to setup the equipment the evening before
the event and will be assisted with respect to the network setup.

The network bandwidth between the scanners and the respective storage unit (server) will vary based
on each case review. For the Frozen section, a 2Mbit/sec network bandwidth and 100ms network
latency will be applied while for the other two cases 20Mbit/sec network bandwidth and 30ms network
latency will be applied. This is to reflect reasonable network conditions for a remote read of a frozen
section as well as regular reads throughout a hospital network. If selected vendor(s) requires a higher
degree of network performance that will be considered and all vendors will have access to any upgrades
from this specification that occur.

A total of three projection screens will be available. Screen #1 will display a view of the respective
scanner during the “live” scanning of the frozen section scanning and subsequently a view of the
keyboard + mouse of the pathologist reviewing each case. Screen #2 and #3 will show a view of the two
screens of the workstation the pathologist will use to read a case. This setup will allow the audience to
fully comprehend the effort and assess ease of use of reviewing each case.


Appendix
Suggested room layout: (This can be altered to fit the available space)
                                      3
                                  en #
                              Scre
                  Screen #2




                                           Podium
                                       1
                                  en #
                              Scre
              ...




                                                    ...
Suggested network setup:
          Servers of vendors                                                                 Clients of vendors
               #1 … #3                                                                            #1 … #6
                                        Apposite Linktropy Mini2 (WAN emulator): does
                                         bandwidth up to 100Mbps and latency.

                                        ET Inc. Bandwidth Manager: limit bandwidth on the
             ...                         LAN up to 1Gbps.




                                                                                                       ...
                                                    WAN latency emulator
                                                            and
                                                   LAN bandwidth simulator




                                   ...

                    Scanners of vendors #1 … #3
     Comments from Vendor Interviews (Draft from MB – some not approved/verified for release)

                                   Interviews with Industry Partners

Mark Newberger (MN), Apollo Telemedicine, Inc
June 6th, 2011
Interviewer – Mike Becich (MB)

Summary of Mark’s Newberger’s comments:
   1) Overall MN thought the idea of a digital pathology showcase as proposed in 4/20 Draft is a good
       idea
   2) He also thought the 18-20 month lead time for the showcase in fall 2012 was a good idea as well
   3) He did admit that he initially thought we were planning to have the showcase this fall with a 4 to
       6 month lead time
   4) He had issues with companies like Apollo (which does workflow software and is scanner
       agnostic) would have to partner with one or two other companies to compete in the showcase.
   5) He didn’t feel that the current draft would yield more than two or three vendors who could do
       end to end (scanning, processing/display and diagnostic workflow) digital pathology
   6) He favored breaking the showcase into segments (scanning, processing/display and diagnostic
       workflow) and having different vendors for each of the segments
   7) MN suggested that having a “training” program for the vendors on the technical set up a day
       before the 2012 meeting was a good idea
   8) MN and MB suggested that the technical specification for the showcase set up be made
       available for testing at the company sites. MB suggested that Pittsburgh would offer a “test”
       setup in Pittsburgh at least 11 months before the 2012 meeting to let companies decide if they
       want to participate (test cases and technical set up).
   9) MN agreed that offering the first six vendors who committed to the showcase was a good idea
       but was concerned you might not get six the first year. Vendors might be concerned how they
       would “show” in this type of open showcase.
   10) MN also thought the showcase would provide good marketing potential to their customers as
       well as new customers.
   11) MN agreed the “open” sharing of the whitepaper and the forum discussion as an industry panel
       at Pathology Informatics 2011 was a good idea.
   12) MB closed the meeting by asking MN if he would mind sharing his comments on the showcase
       with the broad pathology and vendor community and he agreed.
                                   Interviews with Industry Partners

Ole Eichhorn (OE), Aperio
June 10th, 2011
Interviewer – Mike Becich (MB)

Summary of Ole Eichhorn’s comments:
   1) Overall OE thought the timing and planning for the industry panel in 2011 and the digital
      pathology showcase in 2012 was a good plan
   2) OE stated that Aperio considered being involved in the European Scanner Contest in 2010 but
      decided not to participate because of the “closed” nature of the planning and evaluation
   3) OE felt the current plan for the 2012 Digital Pathology Showcase was a starting point but
      suggested more complex activities (e.g. quantitative analysis and algorithms) might be
      considered as well.
   4) OE stated that Aperio (as leader in the field) was interested in further dialogue on the format of
      the showcase in 2012 and would circulate the document to others in Aperio.
      .
                                   Interviews with Industry Partners

John Wellbank (JW) and Jose Costanon (JC), Phillips
June 17th, 2011
Interviewer – Mike Becich (MB)

Summary of John Wellbank’s and Jose Costanon’s comments:
   1) Both JW and JC liked the idea of the industry panel in 2011 and the digital pathology showcase
      in 2012 but raised some concerns.
   2) JC felt that Phillip’s would like to see higher volume of slides scanned included in the showcase.
      When MB told them that there were plans for 10 to 20 slides for the second (biopsy) and third
      (resection) case this did not fit the “higher volume” scenario. MB assured JC that this would be
      considered and discussed in the industry panel in 2011 and as part of the “open” comments
      from the community. JW added that Phillips would need scanning in 20 and 40x as well as more
      volume if possible. Phillips does not scan in 20x equivalency so that test is not possible.
      Therefore we need at least some of this test to scan resolutions equivalent to 40x for true time
      comparisons
   3) JW was concerned that the network bandwidth and latency listed in the current (attached) draft
      of the plan were not sufficient from Phillip’s perspective. MB reassured both JC and JW that
      network bandwidth and latency will be provided to the highest specification requested by the
      industry partners. MB asked JW/JC to provide a network specification and that MB would make
      sure this was part of the plan.
   4) JW felt the bar might be set “too low” for the first meeting and would provide additional
      feedback from others at Phillips on the case scenarios.
   5) JW was concerned that there would be no time for vendors to test the setup for the Industry
      Showcase. MB reassured JW that a “pilot setup” (simulating the setup for 2012) would be
      available in Pittsburgh for at least 11 months in advance of the showcase and made available for
      scheduling to industry partners. MB also stated that the specification for the Industry Showcase
      would be made available to all interested industry partners in November 2011 should an
      industry partner want to replicate the environment in the company headquarters.
   6) Overall JW and JC thought that the timing (over 18 months of lead time) and consensus building
      with industry partners would ensure a successful showcase.
                                    Interviews with Industry Partners

 Philippe Nore (PN), Ventana
July 5th, 2011
Interviewers – Mike Becich (MB) and Bruce Friedman (BF)

Summary of Philipe Nore’s comments:
   1) Don’t want a contest that “compares” vendor on only one dimension (perhaps speed of
       scanning or image quality).
   2) Don’t want to promote instruments that are “skewed” on the basis of any one factor (e.g. a
       “contest” specific set of criteria).
   3) Don’t want to end up pushing company production of instruments and software to “win the
       contest”. The focus should be on how to meet the needs of customers.
   4) Question – Will image analysis be a part of the first showcase? Ventana would like to see a IHC
       or ISH image analysis as part of the first showcase. PN felt many of the initial digital pathology
       solutions use this today, so why not showcase this to customers today.
   5) Will the event will have time for six vendors demos (may be too many).
   6) PN thought that even having 4 vendors would be hard to keep audience’s attention through the
       entire event and should limit time for each exercise.
   7) Ideally for any event you would want to be vendor #1 or 2 but not 3 or 4 (much less 5 or 6) in
       the current setup due to “repetitive/redundant” nature of the tasks
   8) Commentary – reduce the time for each serial demo by vendor.
   9) PN thought by 2012 everyone would have a image analysis algorithm.
   10) PN thought 20 minutes per vendor should be considered – based on his experience with
       customers.
   11) PN asked what “live mode” means. Does this mean scanning occurs on stage. In the Ventana
       system the slide viewing is dynamic robotic so scanning a frozen section is not required once
       mounted on their hardware. Ventana would like the flexibility to view the slide without
       scanning for the frozen section exercise.
   12) No FDA approval for primary diagnostics so exercise # 2 and # 3 may be out of question. They
       need to check with their regulatory lawyers to make sure this event is misperceived as
       promoting non-FDA approved use of devices.
   13) User training is an issue and need this to be addressed. Who will run the demos? How will
       Pathology Informatics deal with this – told PN about the 3 solutions we propose for this
       including:
            a. Availability of a “mock” setup in Pittsburgh
            b. Availability of a “specification” for the setup and encouraging replication of the setup at
               the vendors headquarters or at a key clients location.
            c. Training experience on the “mock” system in Pittsburgh will be shared with
               users/vendors.
   14) Ventana would require their own scanner, image server and application server loaded in
       advance. Can’t share with others. Will follow up with PN on the specs.
   15) Separate fee for vendor participation in the Industry Showcase was PN’s recommendation.
Rob Liepse (RL) and Stephane Willaert (SW), Barco
July 20th, 2012
Interviewer – Mike Becich (MB)

Summary of Rob Liepse’s and Stephane’s Willaert (SW) comments:
1) SW commented that the European contest was only focused on the scanner. SW felt that the “end
   to end” solution including software, servers, etc... to be included in the showcase was a good focus
   for the 2012 event.
2) SW asked if there would be opportunities to have multiple vendors partner for the various partners
   for the component technologies.
3) SW questioned what role did I envision for Barco in the Industry Showcase and it seemed Barco’s
   role would be in the display of scanned slides.
4) SW stated in the Berlin scanner contest their role was as the display partner for all the vendors that
   participated.
5) SW stated that Barco is working with all the vendors in the digital pathology space.
6) RL mentioned that they have a very large footprints in digital mammography and PACS displays.
   They have been a repeat vendor at RSNA for years and participated in IHE (Integrating Healthcare
   Enterprise). They have extensive experience in primary diagnosis in the mammography space.
7) RL mentioned that Barco, worlds only approved display for TOMO (tomosynthesis – mammography
   tomography – have 510K approval) and have been very involved in standards efforts and moving
   this technology to the masses.
8) SW thought that the focus of the proposed Industry Showcase was appropriate but that it would
   make a number of the vendors nervous due to the lack of industry focus on “end to end” solutions.
   SW felt this would move the industry forward but it may be ambitious.
9) SW thought the technical focus on bandwidth support for the demo was a good one and that not
   enough studies are looking at the effect of the network on the viewing experience of the diagnosing
   pathologist.
                                   Interviews with Industry Partners

Doug Giszczynski (DG) and Donal O’Shea (DO), Leica
July 26th, 2011
Interviewer – Mike Becich (MB) and Bruce Friedman (BF)

Summary of Leica’s comments:
   1) DO - Leica participated in the European event and was please that they won awards for their
       products at the event.
   2) DO and DG thought the European event was well communicated and went well but questioned
       the value of some metrics.
   3) DO thought the document was clear and the scenarios and technical setup were useful.
   4) DO thought the document needed more details on the metrics and quantification of the
       measures of the event.
   5) DO wanted clarification on the scenarios and the timing (wasn’t clear). MB explained the
       scenarios and order of the vendors would be shuffled. BF very much liked MB’s idea of having
       the companies work different scenarios sequentially: vendor #1 works scenario #2; followed by
       vendor #2 works scenario #1, etc. No vendor then is in an unfavorable position in terms of the
       order of presentation. However, this causes problems if only one vendor is on the stage at any
       time with the rest in the bullpen. Lot's of confusing set-up time on the stage. Could get tedious.
       It seems to me that the only way this would work if all three (or four) vendors are continuously
       positioned on the stage at the same time with the "spotlight" on each as they go through each
       scenario. Then, only the switches need to be reset for each vendor.
   6) DG what is the output of the showcase....need to finalize. MB stated that this would be
       proposed in the next draft of the document. Frozen section scenario would have 3 metrics -
   7) DO queried on who would run the showcase and MB stated that a pathologist and a technical
       assistant who used the system would be running the company’s system and not vendor staff.
   8) BF asked about the second event was canceled in Europe. DO did now know why Helsinki event
       was canceled.
   9) DO thought the “end to end” scenarios and focus on the workflow was an appropriate one. The
       focus on how the digital pathology wholistically fits into their laboratory is a good and clean
       goal.
   10) DO thought the technical setup was robust and adequate. Leica wanted gigabit between the
       image server and the viewing software. Include the make, model and specification of .
   11) DO mentioned the potential of include a wireless broadcast mode during the showcase (very
       interesting). BF really liked this idea having the images available wirelessly for the audiences
       with iPads and smartphones.
   12) DG asked for a full day instead of 8 to 10 hours to get the work pods for the showcase setup and
       testing. There will need to be able to do this in “private” to protect company intellectual
       property (IP). Sturdy tables should be available so that vendors can set up in a private area and
       then have their setup “wheeled” into place. BF thought that we should consider having set-up
       the previous afternoon/evening and the morning of the showcase. The showcase would then
       start at 1:00 on the workshop day and extend to perhaps 5:00 for a total of four hours (with
       perhaps a break). This would allow registrants for the event to fly in to Chicago that morning.
   13) BF thought the event should be in the afternoon to allow for same day travel for participants
       and to give more time for setup. BO agreed that this would be ideal.
   14) BO thought having a 5 to 10 minute time for the vendor to speak during the opening of the
       event perhaps in “dead-time” (during scanning perhaps). DG agreed but thought this would be
      better at the end of the event. BF thought a closing remarks or panel Q&A would be useful at
      the end of the workshop. I think perhaps that we should have a wrap-up at the end with the
      thee reps at a table at the end to answer questions and wrap-up. Given the three hour time
      limit, I think that this may be difficult with four vendors.
15)   BO thought the company’s selected pathologist would be a valuable spokesman and should be
      drawn into discussion and commentary.
16)   BO thought we should be clear on what will be published and where it would be available and to
      whom.
17)   DG thought approaching media and medical societies (many housed in Chicago) about the event
      would be useful.
18)   DG wanted to make sure this was fair to all participants and having more details on the output
      of the event was critical.
19)   BF thought we should attempt to videotape the entire session and make this available on the
      web site. Documenting the entire event would be of value to the participating companies.
20)   BF commented on the potential name of the Vendor Showcase and taking into consideration the
      emphasis on end-to-end demonstrations and the Leica emphasis on workflow, I think that we
      should refer to this event going forward as: Vendor Showcase: End-to-End Digital Pathology
      Workflow.
                                   Interviews with Industry Partners

John Fonte (JF), Metasystems
July 26th, 2011
Interviewers – Mike Becich (MB) and Bruce Friedman (BF)

Summary of John Fonte’s comments:
   1) Overall JF thought the idea of a digital pathology showcase as proposed in current Draft is a
      good idea, especially the “open” and “consensus building” for the showcase.
   2) JF thinks the focus on end to end solutions from scanning, to display and diagnose is a good one
      for the industry.
   3) JF stated Metasystems has a slightly different take on systems looking at FISH, ISH, fluorescence
      and molecular testing on whole slides and this can be done remotely by matching an identified
      area on light microscope images to areas for analysis at a remote site without pathologist having
      to re-identify areas in the special testing suite.
   4) JF would like to see this (see #3 above) as a case scenario.
   5) JF despite the company’s focus as stated above, they think the showcase is an important event
      and they (Metasystems) supported the European scanner event as a participant.
   6) JF thought that having pathologists and technologists running the demos was risky but that
      generally valuable.
   7) JF was pleased with the technical setup and lead time we promised provide.
   8) JF was going to circulate the documents who were more involved in the European contest.
                                 Interviews with Industry Partners


Rajiv Enand (RE), Omnyx, LLC
August 13th, 2011
Interviewers were Mike Becich (MB) and Bruce Friedman (BF)

   1) RE sounds like a fantastic event and put together nice ideas which will help industry and
       pathologists alike.
   2) Sept 2012 would be a great time for an event like this in terms of market acceptance
   3) Wanted to show commercial development if possible.
   4) RE asked about venue – Hyatt Grand in Chicago
   5) BF reiterated about having color commentary on the stage during the event.
   6) RE wanted to know if this was a “contest” or a demonstration. BF and MB reiterated that this
       would be a “showcase” more like a demo but emphasizing end to end integrated workflow.
   7) MB told about fall backs – specs, demo set up, etc...
   8) RE felt that videotaping was of little commercial value to vendors. Marketing value is how many
       participants in the venue.
   9) RE had questions on marketing the Industry Showcase – simulcast on web would be a good idea.
       Vendors like Omnyx would help to promote the event.
   10) MB committed that there would be nothing running against this event.
   11) RE commented on the amount of time the companies would have to spend to prepare for this
       so everything Path Info did to drive both attendance in person and on-line would be very
       important to get participation.
   12) RE thought this is an important event to encourage adoption and provide this as a valuable
       service to the pathology community.
   13) RE said this would be of great interest to Omnyx.
   14) RE thought a “judges choice” award would be a great idea.
   15) RE agreed that image quality would be a useful metric worthy of an award or recognition.
   16) RE thought that a narrative assessment by judges that were quotable document published as
       proceedings in Journal of Pathology Informatics.
   17) RE thought we should try to get unbiased judges who would use the technology.

								
To top