Resource Utilization Performance of Complex Workflows on the Public Cloud: A Simulation-Based Approach

Abstract

"Public cloud computing has emerged as a popular computing platform. To benefit from its pay-as-you go billing model, it is important to allocate enough public cloud resources to execute complex scientific workflows promptly in an environment consisting of various virtual machines. By provisioning more resources than required, applications will finish in a short amount of time, resource provisioning costs may significantly increase. In practice, cloud users employ cost-benefit analysis to determine the optimal resource provisioning strategy or general guidelines, or they use optimization-based methods to predict application execution time and proactively allocate public cloud resources to run complex workflows. The effectiveness of these methods can be greatly influenced by the burden associated with accessing cloud resources due to high costs, which can impact the accuracy of prediction models, application complexity, as well as the available budget. Therefore, previous studies have indicated the performance of these strategies. This research contributes by presenting a simulation tool that employs empirical rules for self-allocating public cloud resources to execute complex workflows. The suggested simulation platform (WorkFlowSim) includes specific features that assist in estimating the time needed for workflow execution. We have approved three workflows, each with different structures and sizes. One of these types contains real data from previous operations on the Microsoft Azure cloud, which helps ensure results and gain more credibility. The main goal of the study is to normalize a platform to accurately and efficiently determine cloud resource usage predictions. Therefore, as an achievement verified by experiments the minimum squared error (MSE) of the estimated runtime has calculated to be 454.1161, the minimal predict value when the resource count is 12, and is 100% consistent with the actual execution."