About: EMIXER: End-to-end Multimodal X-ray Generation via Self-supervision

Facets (new session)
Description
Metadata
Settings
- owl:sameAs
- Inference Rule:

About: EMIXER: End-to-end Multimodal X-ray Generation via Self-supervision Goto Sponge NotDistinct Permalink

An Entity of Type : schema:ScholarlyArticle, within Data Space : wasabi.inria.fr associated with source document(s)

Attributes	Values
type	Academic Article research paper schema:ScholarlyArticle
isDefinedBy	Covid-on-the-Web dataset
has title	EMIXER: End-to-end Multimodal X-ray Generation via Self-supervision
Creator	Biswal, Siddharth Koyejo, Sanmi Pyrros, Ayis Siddiqui, Nasir Sun, Jimeng Tech, Georgia Zhuang, Peiye
Source	ArXiv
abstract	Deep generative models have enabled the automated synthesis of high-quality data for diverse applications. However, the most effective generative models are specialized to data from a single domain (e.g., images or text). Real-world applications such as healthcare require multi-modal data from multiple domains (e.g., both images and corresponding text), which are difficult to acquire due to limited availability and privacy concerns and are much harder to synthesize. To tackle this joint synthesis challenge, we propose an End-to-end MultImodal X-ray genERative model (EMIXER) for jointly synthesizing x-ray images and corresponding free-text reports, all conditional on diagnosis labels. EMIXER is an conditional generative adversarial model by 1) generating an image based on a label, 2) encoding the image to a hidden embedding, 3) producing the corresponding text via a hierarchical decoder from the image embedding, and 4) a joint discriminator for assessing both the image and the corresponding text. EMIXER also enables self-supervision to leverage vast amount of unlabeled data. Extensive experiments with real X-ray reports data illustrate how data augmentation using synthesized multimodal samples can improve the performance of a variety of supervised tasks including COVID-19 X-ray classification with very limited samples. The quality of generated images and reports are also confirmed by radiologists. We quantitatively show that EMIXER generated synthetic datasets can augment X-ray image classification, report generation models to achieve 5.94% and 6.9% improvement on models trained only on real data samples. Taken together, our results highlight the promise of state of generative models to advance clinical machine learning.
has issue date	2020-07-10(xsd:dateTime)
has license	arxiv
sha1sum (hex)	0c3612f08814b86a5e44dfd1fe549acb7a1c1801
resource representing a document's title	EMIXER: End-to-end Multimodal X-ray Generation via Self-supervision
resource representing a document's body	covid:0c3612f08814b86a5e44dfd1fe549acb7a1c1801#body_text
is schema:about of	named entity 'datasets' named entity 'machine learning' named entity 'variety' named entity 'image' named entity 'data' named entity 'encoding' named entity 'Multimodal' named entity 'supervised' named entity 'data' named entity 'limited' named entity 'images' named entity 'confirmed' named entity 'free-text' named entity 'X-ray' named entity 'data' named entity 'illustrate' named entity 'applications' named entity 'experiments' named entity 'text' named entity 'assessing' named entity 'advance' named entity 'x-ray images' named entity 'genERative model' named entity 'chest X-rays' named entity 'Generative adversarial networks' named entity 'X-ray image' named entity 'Neural Network' named entity 'training data' named entity 'X-ray image' named entity 'CNN' named entity 'X-ray' named entity 'synthetic data' named entity 'cardiac silhouette' named entity 'neural network' named entity 'X-ray images' named entity 'convolution' named entity 'X-ray images' named entity 'image captioning' named entity 'X-ray' named entity 'radiograph' named entity 'pneumonia, viral' named entity 'X-ray' named entity 'pneumonia' named entity 'one-hot' named entity 'variational autoencoders' named entity 'X-ray images' named entity 'GAN' named entity 'GAN' named entity 'pooling layer' named entity 'softmax' named entity 'hemidiaphragm' named entity 'X-ray images' named entity 'X-ray images' named entity 'X-ray images' named entity 'clinical data' named entity 'COVID-19' named entity 'synthetic data' named entity 'COVID-19' named entity 'X-rays' named entity 'X-ray image' named entity 'LSTM' named entity 'X-ray images' named entity 'X-ray images' named entity 'algorithm' named entity 'imaging studies' named entity 'LSTM' named entity 'feature vector' named entity 'X-ray' named entity 'Report Generator' named entity 'x-ray' named entity 'Autoregressive Models' named entity 'angle of rotation' named entity 'medical imaging' named entity 'classification model' named entity 'labeled data' named entity 'algorithm' named entity 'radiograph' named entity 'natural language processing' named entity 'Data sharing' named entity 'colon' named entity 'X-ray image'

Faceted Search & Find service v1.13.91 as of Mar 24 2020

Alternative Linked Data Documents: Sponger | ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3229 as of Jul 10 2020, on Linux (x86_64-pc-linux-gnu), Single-Server Edition (94 GB total memory)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2024 OpenLink Software