DIC2FBA: Distributed Incremental Clustering with Closeness Factor Based Algorithm for Analysis of Smart Meter Data

Abstract

Due to increased civilization, smart cities and advent of technology, lots of buildings including commercials, residentials and other types are populating in numbers in the recent past. The electricity consumption is also affecting due to increased occupancy in these buildings. To analyse the electricity consumption patterns technology is utmost useful. This analysis will be useful for consumers and electricity generation units too to know about consumption and future requirements of electricity. Incremental clustering algorithm is the best choice to handle ever increasing data. In this research work, in the first phase the electricity consumption data was extracted from smart meter images and then in the second phase the data was taken from extracted .csv files merging data from various sources together. This research proposes Distributed Incremental Clustering with Closeness Factor Based Algorithm (DIC2FBA), to update load patterns without overall daily load curve clustering. The proposed DIC2FBA has used Amazon Web Service(AWS) and Microsoft Azure HDInsight service. The AWS EC2 instance, AWS S3 bucket, and HdInsight, which clusters data from multiple sites in iterative and incremental mode. The DIC2FBA first extracts load patterns from new data and then intergrades the existed load patterns with the new ones. Further, we have compared the findings achieved using the DIC2FBA with IK means based on time, features, silhouette score, and Davis Bouldin index which indicate that our method can provide an efficient response for electricity consumption patterns analysis to end consumers via smart meters.