Reverse image search for scientific data within and beyond the visible spectrum

Flavio H.D. Araujo, Romuere R.V. Silva, Fatima N.S. Medeiros, Dilworth D.Parkinson, Alexander Hexemer, Claudia M.Carneiro, Daniela M.Ushizima

ScienceDirect
November 1, 2018

Abstract: The explosion in the rate, quality and diversity of image acquisition instruments has propelled the development of expert systems to organize and query image collections more efficiently. Recommendation systems that handle scientific images are rare, particularly if records lack metadata. This paper introduces new strategies to enable fast searches and image ranking from large pictorial datasets with or without labels. The main contribution is the development of pyCBIR, a deep neural network software to search scientific images by content. This tool exploits convolutional layers with locality sensitivity hashing for querying images across domains through a user-friendly interface. Our results report image searches over databases ranging from thousands to millions of samples. We test pyCBIR search capabilities using three convNets against four scientific datasets, including samples from cell microscopy, microtomography, atomic diffraction patterns, and materials photographs to demonstrate 95% accurate recommendations in most cases. Furthermore, all scientific data collections are released. 

The preprint version of the paper was published in Expert Systems with Applications on March 12, 2018:
Reverse image search for scientific data within and beyond the visible spectrum
Flavio H.D. Araujo, Romuere R.V. Silva, Fatima N.S. Medeiros, Dilworth D.Parkinson, Alexander Hexemer, Claudia M.Carneiro, Daniela M.Ushizima
​March 12, 2018 | Expert Systems with Applications



Featured Fellows

Daniela Ushizima

Computational Research Division, Lawrence Berkeley National Lab

Flavio Araujo

LBNL
Visiting Scholar

Romuere Silva

LBNL
Visiting Scholar