Construção de uma base de dados multi-cloud e aplicação de aprendizado de máquina para análise comparativa de recursos de computação em nuvem

Abstract

Comparing virtual machine costs across multi-cloud environments is a complex task due to the heterogeneity of catalogs, naming conventions, pricing models, and APIs provided by different vendors. This fragmentation makes it difficult to identify technically equivalent instances and to objectively evaluate the cost-benefit ratio among offerings from Amazon Web Services, Microsoft Azure, and Google Cloud Platform, for instance. This paper presents an architecture for collecting, normalizing, and analyzing virtual machine instance prices, combining data engineering and machine learning to support cloud resource comparison and selection decisions. The developed solution features a multi-cloud ETL pipeline capable of extracting public pricing and technical specification data, transforming heterogeneous structures into a common schema, and consolidating them into a unified relational database. The final dataset comprised 34,000 records of On-Demand and Spot instances, with approximately half of these used in the modeling experiments. Two complementary approaches were evaluated using this subset. The first employs clustering techniques to identify similar technical profiles among instances. The second uses regression models to estimate the hourly price of virtual machines. In the clustering phase, K-Means and Gaussian Mixture Model algorithms were compared, with K-Means selected for its superior grouping quality. In the regression phase, Linear Regression, Random Forest, and XGBoost models were evaluated, with XGBoost showing the best marginal performance. Based on the estimates produced by the selected regression model, a cost-benefit indicator was defined, relating an instance's observed price to the price predicted for its configuration. This indicator allows for the assessment of whether an instance is priced potentially below, near, or above the pricing standard learned from the catalog, complementing analyses based solely on absolute price. The integration of clustering, regression, and the cost-benefit indicator was demonstrated through conceptual scenarios focused on analyzing price alignment and identifying equivalent alternatives across providers. Catalog standardization, combined with the clustering of technical profiles and predictive price modeling, proved capable of supporting comparative analyses in multi-cloud environments.