Processamento eficiente de junção espacial em ambiente paralelo e distribuído baseado em Spatialhadoop
Abstract
The huge volume of spatial data generated and made available in recent years from
different sources, such as remote sensing, smart phones, space telescopes, and
satellites, has motivated researchers and practitioners around the world to find out a way
to process efficiently this huge volume of spatial data. Systems based on the MapReduce
programming paradigm, such as Hadoop, have proven to be an efficient framework for
processing huge volumes of data in many applications. However, Hadoop has showed
not to be adequate in native support for spatial data due to its central structure is not
aware of the spatial characteristics of such data. The solution to this problem gave rise to
SpatialHadoop, which is a Hadoop extension with native support for spatial data.
However, SpatialHadoop does not enable to jointly allocate related spatial data and also
does not take into account any characteristics of the data in the process of task scheduler
for processing on the nodes of a cluster of computers. Given this scenario, this PhD
dissertation aims to propose new strategies to improve the performance of the processing
of the spatial join operations for huge volumes of data using SpatialHadoop. For this
purpose, the proposed solutions explore the joint allocation of related spatial data and the
scheduling strategy of MapReduce for related spatial data also allocated in a jointly form.
The efficient data access is an essential step in achieving better performance during
query processing. Therefore, the proposed solutions allow the reduction of network traffic
and I/O operations to the disk and consequently improve the performance of spatial join
processing by using SpatialHadoop. By means of experimental evaluations, it was
possible to show that the novel data allocation policies and scheduling tasks actually
improve the total processing time of the spatial join operations. The performance gain
varied from 14.7% to 23.6% if compared to the baseline proposed by CoS-HDFS and
varied from 8.3% to 65% if compared to the native support of SpatialHadoop.