Effects of personal identifier resynthesis on clinical text de-identification

Yeniterzi, Reyyan and Aberdeen, John and Bayer, Samuel and Wellner, Ben and Hirschman, Lynette and Malin, Bradley (2010) Effects of personal identifier resynthesis on clinical text de-identification. Journal of the American Medical Informatics Association, 17 (2). pp. 159-168. ISSN 1067-5027

Full text not available from this repository. (Request a copy)


Objective De-identified medical records are critical to biomedical research. Text de-identification software exists, including "resynthesis" components that replace real identifiers with synthetic identifiers. The goal of this research is to evaluate the effectiveness and examine possible bias introduced by resynthesis on de-identification software. Design We evaluated the open-source MITRE Identification Scrubber Toolkit, which includes a resynthesis capability, with clinical text from Vanderbilt University Medical Center patient records. We investigated four record classes from over 500 patients' files, including laboratory reports, medication orders, discharge summaries and clinical notes. We trained and tested the de-identification tool on real and resynthesized records. Measurements We measured performance in terms of precision, recall, F-measure and accuracy for the detection of protected health identifiers as designated by the HIPAA Safe Harbor Rule. Results The de-identification tool was trained and tested on a collection of real and resynthesized Vanderbilt records. Results for training and testing on the real records were 0.990 accuracy and 0.960 F-measure. The results improved when trained and tested on resynthesized records with 0.998 accuracy and 0.980 F-measure but deteriorated moderately when trained on real records and tested on resynthesized records with 0.989 accuracy 0.862 F-measure. Moreover, the results declined significantly when trained on resynthesized records and tested on real records with 0.942 accuracy and 0.728 F-measure. Conclusion The de-identification tool achieves high accuracy when training and test sets are homogeneous lie, both real or resynthesized records). The resynthesis component regularizes the data to make them less "realistic," resulting in loss of performance particularly when training on resynthesized data and testing on real data.
Item Type: Article
Divisions: Faculty of Engineering and Natural Sciences
Depositing User: Reyyan Yeniterzi
Date Deposited: 31 Mar 2010 11:54
Last Modified: 24 Jul 2019 16:21
URI: https://research.sabanciuniv.edu/id/eprint/13876

Actions (login required)

View Item
View Item