A Scalable MapReduce Similarity Join Algorithm Using LSH - Université d'Orléans Accéder directement au contenu
Rapport (Rapport De Recherche) Année : 2021

A Scalable MapReduce Similarity Join Algorithm Using LSH

Résumé

Similarity Joins are recognized to be among the most useful data processing and analysis operations. A similarity join is used to retrieve all data pairs whose distances are smaller than a predened threshold λ. In this paper, we introduce the MRS-join algorithm to perform similarity joins on large trajectories datasets. The MapReduce model and a randomized LSH (Local Sensitive Hashing) keys redistribution approach are used to balance load among processing nodes while reducing communications and computations to almost all relevant data by using distributed histograms. A cost analysis of MRS-join algorithm shows that our approach is insensitive to data skew and guarantees perfect balancing properties, in large scale systems, during all stages of similarity join computations. These performances have been conrmed by a series of experiments using the Fréchet distance on large datasets of trajectories from real world and synthetic data benchmarks.
Fichier principal
Vignette du fichier
main.pdf (792.42 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03276756 , version 1 (02-07-2021)
hal-03276756 , version 2 (02-09-2021)

Identifiants

  • HAL Id : hal-03276756 , version 2

Citer

Sébastien Rivault, Mostafa Bamha, Sébastien Limet, Sophie Robert. A Scalable MapReduce Similarity Join Algorithm Using LSH. [Research Report] LIFO, Université d'Orléans, INSA Centre Val de Loire. 2021. ⟨hal-03276756v2⟩
128 Consultations
176 Téléchargements

Partager

Gmail Facebook X LinkedIn More