Kanishka Bhaduri

Member since: Sep 24, 2010, Mission Critical Technologies Inc

Scalable, Asynchronous, Distributed Eigen-Monitoring of Astronomy Data Streams

Shared by Kanishka Bhaduri, updated on May 05, 2011


Author(s) :
Kanishka Bhaduri, Kamalika Das, K. Borne, C. Giannella, T. Mahule, H. Kargupta

In this paper, we develop a distributed algorithm for monitoring the principal components (PCs) for next generation
of astronomy petascale data pipelines such as the Large Synoptic Survey Telescopes (LSST). This telescope will take repeated
images of the night sky every 20 s, thereby generating 30 terabytes of calibrated imagery every night that will need to be
co-analyzed with other astronomical data stored at different locations around the world. Event detection, classification, and
isolation in such data sets may provide useful insights to unique astronomical phenomenon displaying astrophysically significant
variations: quasars, supernovae, variable stars, and potentially hazardous asteroids. However, performing such data mining tasks
is a challenging problem for such high-throughput distributed data streams. In this paper, we propose a highly scalable and
distributed asynchronous algorithm for monitoring the PCs of such dynamic data streams and discuss a prototype web-based
system PADMINI (Peer-to-Peer Astronomy Data Mining) which implements this algorithm for use by the astronomers. We
demonstrate the algorithm on a large set of distributed astronomical data to accomplish well-known astronomy tasks such as
measuring variations in the fundamental plane of galaxy parameters. The proposed algorithm is provably correct (i.e., converges
to the correct PCs without centralizing any data) and can seamlessly handle changes to the data or the network. Real experiments
performed on Sloan Digital Sky Survey (SDSS) catalogue data show the effectiveness of the algorithm.

show more info
Publication Name
Scalable, Asynchronous, Distributed Eigen-Monitoring of Astronomy Data Streams
Publication Location
Statistical Analysis and Data Mining Journal
Year Published


528.3 KB 134 downloads


Add New Comment

Kanishka's Projects (4)

Need help?

Visit our help center