Human-in-the-Loop AI | Risk-Adapted Screening

“Swarm is exactly how I’d want a good pub trivia team to operate like.”

Cheers to this tweet from Andrew Murphy about Stanford’s new “hive mind” AI study.


Imaging Wire Sponsors

  • Carestream – Focused on delivering innovation that is life changing – for patients, customers, employees, communities and other stakeholders
  • Focused Ultrasound Foundation – Accelerating the development and adoption of focused ultrasound
  • Medmo – Helping underinsured Americans save on medical scans by connecting them to imaging providers with unfilled schedule time
  • Nuance – AI and cloud-powered technology solutions to help radiologists stay focused, move quickly, and work smarter
  • Pocus Systems – A new Point of Care Ultrasound startup, combining a team of POCUS veterans with next-generation genuine AI technology to disrupt the industry
  • Qure.ai – Making healthcare more accessible by applying deep learning to radiology imaging

The Imaging Wire



Human-in-the-Loop AI

Stanford University and Unanimous AI published the results of their latest study into the human-in-the-loop (HITL) AI process (here’s their last one), finding that combining AI with a team of human radiologists operating as a ‘hive mind” may indeed capitalize on both groups’ advantages and avoid their shortcomings. Here are some details:

HITL AI – This approach leverages the advantages of AI (i.e. rapid automated detection), while relying on humans to focus on areas where an algorithm has low confidence or may be prone to bias, creating “a collective super intelligence.”

The Study – The study had two groups of radiologists (Group A = 7, Group B = 6) estimate the probability of pneumonia on 50 chest X-rays, doing so individually and as a swarm. The study also had Stanford’s CheXNet and CheXMax deep learning models estimate the probability of pneumonia on the same chest X-rays. Lastly the team combined the radiologists via the swarm platform and the deep learning models to review the images using a human-in-the-loop AI process.

The Results – There’s a lot to compare in this study (humans vs. AI, individuals vs. swarm, AI vs. human swarm, groups A vs. B, etc.) and it’s worth reviewing out for complete details. The main takeaway is that when Groups A & B were combined as a single swarm and augmented with the CheXmax model, their estimates were more accurate (91% vs. 35%-84%) and had greater sensitivity (0.875 vs. 0.519-0.900) and specificity (0.933 vs. 0.519-0.933) than the majority of the other approaches.

Implications – This research team is bullish on HITL AI, suggesting that its ability to bring the best out of humans and AI “has broad implications” for future AI deployment and implementation strategies. Even if using a team of 7 radiologists (or 13) to augment AI doesn’t seem all that feasible to some.



Risk-Adapted Screening

A study of over 5 million Swedish women provided new insights into when women should begin breast cancer screening depending on their age and risk profile.

The Study – The study analyzed 1958-2015 data from 5,099,172 women born after 1932, using age and history of breast cancer among first and second-degree relatives to identify recommended screening ages.

The Results – 118,953 (2.3%) of the women were diagnosed with primary invasive breast cancer, including 102,751 women who didn’t have a family history of breast cancer at the time of diagnosis (55.9 mean age). However, data on patients who did have a family history of cancer revealed that these patients would have benefitted from screening as much as 14 years earlier than standard guidelines. For example, the researchers suggest that a woman with a history of cancer among two or more first-degree relatives and one or more second-degree relatives should begin screening at 28 years old (vs. 37yrs for one 2nd degree relative or 41yrs for patients with no family history).

Implications – The authors criticized current guidelines for not sufficiently using family history when recommending when to begin screening. They also called for more research into whether screening should be delayed for women with low-risk profiles and research into the costs associated with early screening for women with higher risk profiles.


The Wire

  • A breakthrough from Duke researchers may help expand optical coherence tomography (OCT) beyond ophthalmology applications, starting with a new portable and low-cost system for image-guided minimally invasive joint surgery. The researchers used a 4mm rigid borescope (a thin tube of lenses) to deliver enough infrared light to the surgical area of a pig’s knee to perform OCT, while remaining small enough to fit inside the body.
  • The National Council on Radiation Protection and Measurements (NCRP) revealed that radiology industry initiatives helped drive a 15% to 20% drop in diagnostic and interventional medical radiation doses between 2006 and 2016 (vs. a six-fold increase between the early 1980s and 2006). Although doses fell across most modalities, CT grew its share of collective medical imaging doses (from 50% to 63%) as CT volume increased by 20% and CT dosage per person remained stable.
  • Life Image and data firm Graticule launched their new GLIMPS service (Graticule Life Image Machine Parsed Set), allowing biopharma and AI companies to subscribe to Life Image’s de-identified patient level data using the AWS Data Exchange. GLIMPS is intended for for product development and validation, including using longitudinal data to identify and research disease markers, using control data for machine learning studies or model validation, and to explore new approaches or techniques.
  • A new paper in the Journal of Digital Imaging detailed how a UPenn team adopted an automated workflow to import outside studies from CDs to their PACS in a minute (linking them with an internal accession number and exam code), reducing CD-associated manual labor and making studies available on their PACS in nearly real time. As a result of the new automated image-exchange workflow, the UPenn team was able to reassign six of their nine FTE employees to other roles.
  • Fujifilm made a big move in its other imaging business, paying $2.3 billion to acquire Xerox’s 25% stake in Fuji Xerox, fundamentally changing Fujifilm’s largest business unit and its relationship with its largest customer (Xerox is a major Fuji Xerox OEM customer). Although Fuji Xerox is pretty far away from Fujifilm’s medical imaging business, the deal’s opportunities (FX is leaner/nimbler, no longer tethered to Carl Icahn, has new business opportunities) and threats (Fujifilm just spend $2.3b on print in 2019, Xerox now a less dependable OEM client) could have an organization-wide impact.
  • Research from a Wayne State and Henry Ford team found that adopting a “CTA for All” emergency stroke imaging protocol that combines CT Angiography, non-contrast CT, and an initial assessment for all acute ischemic stroke (AIS) patients within 24hrs (vs. 6hrs) had a wide range of advantages. The study compared AIS patient outcomes one year before (n=388) and after (n=515) adopting the protocol, finding that the protocol lead to more patients undergoing CTA (91% vs. 61%) and CTA w/ non-contrast CT (78% vs. 35%), shorter ED arrival to CTA times (29 vs. 43 minutes), more LVO detections (166 / 32% vs. 96 / 25%), more mechanical thrombectomy procedures (108 / 21% vs. 68 / 18%), and more patients discharged with favorable outcomes (53% vs. 37%).
  • British researchers adapted their earlier Magnetoencephalography (MEG) helmet scanners to create a new bike helmet style system for pediatric brain scans, calling it an important step towards understanding childhood brain development and researching certain neurological and mental health conditions (e.g. epilepsy and autism). The new design addresses the body movement issues that have traditionally hindered pediatric brain imaging by using new light-weight quantum sensors that reduced the MEG scanners from 1,100 lb. to 1.1 lb. and allows placement much closer to the young patients’ heads due to significantly reduced cooling requirements.
  • Thomas Jefferson researchers found that bedside optic nerve ultrasound is a valuable way to diagnose increased intracranial pressure without the cost, accessibility, and invasiveness disadvantages of current techniques (e.g. CT, lumbar puncture, intracranial drains). The researchers reviewed 71 studies involving 4,551 patients, finding that optic nerve ultrasound achieved “excellent accuracy” detecting increased intracranial pressure among children and adults regardless of cause (TBI or not) and type of clinical specialist (e.g. neurosurgeon, radiologist, emergency medicine clinician, neurologist, or critical care specialist), recommending that optic nerve ultrasound be added as a standard test.
  • Surprise billing is back in the news after a JAMA Internal Medicine article found that markups for anesthesiology services (a common surprise billing specialty) increased at a much faster rate than emergency medicine between 2012 and 2016 (+32% vs. 28%) and a separate Families USA poll found that 44% of U.S. voters either received a surprise bill or have an immediate family member who received a surprise medical bill.
  • A pair of UPenn radiologists published another AI hype-busting paper in Academic Radiology, suggesting that as the reality of AI becomes clearer, we’ll come to realize that AI’s value as a triage tool could be “substantial” in poorer regions with few radiologists but only “incremental” in developed countries. AI’s “incremental” value proposition in countries like the U.S. would focus on triage-based emergency imaging prioritization, but may still face challenges due to false positives/negatives and would require randomized control trials to achieve full trust/adoption.
  • Cardiac imaging startup, ElectroSonix, licensed a series of University of Arizona acoustoelectric cardiac imaging (ACI) patents that they suggest could improve cardiac ablation accuracy for arrhythmia treatment. ACI emits ultrasound waves that interreact with the heart’s electrical currents to produce a map of electrical activity, creating “precise, real-time data before, during and after cardiac ablation procedures.” The team also believes ACI could be used to map “nearly all bodily functions that rely on electrical signals,” allowing other applications.

The Resource Wire

  • Qure.ai is ahead of the AI pack at RSNA 2019 with four research presentations on 1) Evaluating radiology AI models; 2) Comparing AI-based TB screening and bacteriological tests; 3) Segmenting and measuring ventricular and cranial vault volumes; and 4) How clinical context improves AI performance for cranial fracture detection.
  • POCUS Systems is approved as a Veteran Owned Business with the US Government Office of Veterans Business Development, paving the way for partnerships with the federal healthcare delivery systems.
  • By partnering with Medmo, imaging centers can keep their schedules full and their equipment busy. Here’s where to learn more and get started.

Leave a Reply

Basic HTML is allowed. Your email address will not be published.

Subscribe to this comment feed via RSS

Get every issue of The Imaging Wire

Delivered right to your inbox. We will never sell or share your information.