# Reason Like a Radiologist

Structured thinking first, writing the report last.

Canonical: https://astro-blog-bice-chi.vercel.app/blog/reason-like-radiologist

Edition: en · Language: en · Revision: 1

Archived: 2025-07-23

Updated: 2026-09-21

The archive date records this article’s first appearance in this repository; its original publication date is unknown.

Identifiable source figures and tables were re-extracted from pinned paper versions. Original attachment files remain unavailable; uncertain matches are explicitly marked. Selected source and metric descriptions were corrected.

Updated   2026-09-21

### Reason Like a Radiologist: Chain-of-Thought and Reinforcement Learning for Verifiable Report Generation

Published Apr 2025

URL [https://arxiv.org/pdf/2504.18453](https://arxiv.org/pdf/2504.18453)

Tags

RRGX-rayRL

## [Takeaways](https://astro-blog-bice-chi.vercel.app/blog/reason-like-radiologist#takeaways)

*   intuitive approach : instead of direct **X-ray**\-to-report, leverage **CoT** to get structured thinking right.
    
*   extension is **LIMITED** : fixed workflow with a small set of predefined labels and anatomical regions; does not demonstrate generalization to other modalities.
    

## [Methods](https://astro-blog-bice-chi.vercel.app/blog/reason-like-radiologist#methods)

Three-stage method:

1.  **Medical Concept Learning (MCL)**, a SFT with rewriting a report into a structured **CoT** (Finding, Disease Category, Anatomical Region)
    
    1.  **Disease Category** = 14 clinical labels derived from **CheXpert** (Irvin et al., 2019):
        
        pneumonia, fracture, consolidation, cardiomegaly, enlarged cardiomediastinum, no finding, pleural other, pneumothorax, atelectasis, support devices, edema, pleural effusion, lung lesion, and lung opacity
        
    2.  **Anatomical Region** = 12 areas based on prior work ([**STREAM**: Yang et al., 2025](https://github.com/yangyan22/STREAM)):
        
        abdomen, cardiac silhouette, left apical zone, left hilar structures, left lung, mediastinum, right apical zone, right hilar structures, right lung, whole lung, spine, and trachea
        
    3.  Example: Findings (“Lungs are low in volume”) → Disease (Atelectasis, via intermediate concepts like collapse and mediastinal shift) → Anatomy (Whole Lung)
        
2.  **Spatially Verifiable Reinforcement (SVR)**, a **RL** (**GRPO**) that enforces that the location also gives an accurate bbox prediction.
    
    1.  Interestingly, it only rewards with **IOU** and format reward, but **NO** “correctness” reward.
    2.  I guess the goal of this stage is to reinforce the structured parser and accurate bbox.
3.  **Report adaptor**, a **LORA** that makes an **LLM** “talk” the same way as a report, instead of in a structured language like **MCL**.
    
    Only **LORA** on the language part works the best.
    

![BoxMed-RL adaptation strategies](https://astro-blog-bice-chi.vercel.app/assets/blog/paper-reading/2504.18453v1-p12-table-5.webp)

Source: [2504.18453v1, Table 5, PDF p. 12](https://arxiv.org/pdf/2504.18453v1#page=12). Re-extracted from the cited paper for this discussion; the original attachment is unavailable.

## [Data](https://astro-blog-bice-chi.vercel.app/blog/reason-like-radiologist#data)

For stages 1 and 2:

1.  **MS-CX** ([https://arxiv.org/abs/2204.09817](https://arxiv.org/abs/2204.09817)): **MIMIC-CXR** ([https://physionet.org/content/mimic-cxr-jpg/2.0.0/](https://physionet.org/content/mimic-cxr-jpg/2.0.0/)), includes 1,162 image–sentence pairs with annotated bounding boxes and a corresponding medical phrase.
    
2.  **LATTE-CXR** ([https://physionet.org/content/latte-cxr/1.0.0/](https://physionet.org/content/latte-cxr/1.0.0/)): built from **MIMIC-CXR** and **REFLACX**, includes 13,751 verified bounding box annotations aligned with radiological findings
    

For stage 3, Language **LORA**:

1.  **MIMIC-CXR** comprises 377,110 chest **X-ray** images paired with 227,835 radiology reports
    
2.  **IU X-Ray** ([https://openi.nlm.nih.gov/](https://openi.nlm.nih.gov/)): 7,470 images and 3,955 reports
    

## [Results](https://astro-blog-bice-chi.vercel.app/blog/reason-like-radiologist#results)

![BoxMed-RL component ablation results](https://astro-blog-bice-chi.vercel.app/assets/blog/paper-reading/2504.18453v1-p8-table-4.webp)

Source: [2504.18453v1, Table 4, PDF p. 8](https://arxiv.org/pdf/2504.18453v1#page=8). Re-extracted from the cited paper for this discussion; the original attachment is unavailable.

Interestingly, **LORA** alone works pretty well.

Considering **LORA** has the most data, it makes sense: 230k vs ~15k.

Achieved **CE** metrics with **F1** score of 0.412 on **MIMIC-CXR** and 0.610 on **IU X-ray**.

![BoxMed-RL per-label CheXpert metrics](https://astro-blog-bice-chi.vercel.app/assets/blog/paper-reading/2504.18453v1-p7-table-3.webp)

Source: [2504.18453v1, Table 3, PDF p. 7](https://arxiv.org/pdf/2504.18453v1#page=7). Re-extracted from the cited paper for this discussion; the original attachment is unavailable.

This seems low, but is actually ranked top-tier in each benchmark.

*   According to a concurrent paper, **Medical Report Generation Is A Multi-label Classification Problem** ([https://arxiv.org/pdf/2409.00250](https://arxiv.org/pdf/2409.00250)), training a **ResNet50** for label classification on these datasets with poor precision contributes to a very long-tailed distribution.

![ResNet multi-label classification on IU-Xray and MIMIC-CXR](https://astro-blog-bice-chi.vercel.app/assets/blog/paper-reading/2409.00250v1-p7-table-ii.webp)

Source: [2409.00250v1, Table II, PDF p. 7](https://arxiv.org/pdf/2409.00250v1#page=7). Re-extracted from the cited paper for this discussion; the original attachment is unavailable.

*   I previously noted a GPT-supplied estimate of human consistency on **MIMIC-CXR**, but could not identify a supporting study. I have removed the numerical estimate; it should not be used to interpret these benchmark scores.