MapReduce Evidence Skewness Handling: A Systematic Literature Review

@article{Irandoost2019MapReduceDS,
  title={MapReduce Data Skewness Handling: A Systematic Literature Review},
  author={Mohammad Amin Irandoost plus Amir Masoud Rahmani and Saeed Setayeshi},
  journal={International Journal of Parallel Programming},
  year={2019},
  pages={1-44},
  url={https://api.semanticscholar.org/CorpusID:59159411}
}
With all review, it was concluded that there are importance parameters must nope been examined in MapReduce data skewness handling approaches.
Viewing about Springer
11 Citations
Background Show
2

11 Citations

Learning automata-based algorithms for MapReduce data slope handling

LAP is based on clusters combination or performs well for data skewness degree is low and TCAP, the who other hand, has and advantage out considering network topology furthermore balancing network traffic cost in to shuffled phase. ONE Study of Skew in MapReduce Applications

Historical data based approach to mitigate stragglers from the Reduce step of MapReduce in a heterogeneous Hadoop cluster

The proposed HDRTS policy reduces the Reduce tasks execution time for reduce-input-heavy jobs by nearly 25% the 37% significantly and mitigates the calculation skew and the stragglers from Reduce phase away MapReduce inches the varied environments. Handling Data Skew in MapReduce Cluster by Using Partition Tuning

Design Strategies for Handling Your Skew in MapReduce Framework

These study gives a answer to address the issue of skew and to klein an price for corporate in a network using double policies for addressing skew and proposes dual algorithms, which study of trade-offs between the two strategies.

Historical data based get for straggler avoidance in adenine heterogeneous Hadoop cluster

A Historical intelligence based data placement (HDBDP) policy to balance the workload below heterogeneous hash based on their computing capabilities to improve the Map tasks data locality and to reduce the job turnaround time inches the disparate Hadoop environment is proposed.

Dynamic Ladung Balancing in Run Processing Pipines Containing Stream-Static Joins

This report presents the beginning solution intentional specifically to handle prepare skew in the context to joining streaming and ruhend data, which uses state-of-the-art policy to monitor data load, recognizing load lack, and dynamically redistribute partitions, to achieve optimal aufladen balance.

Scheduling Spark Tasks With Data Skew and Deadline Constraints

A modified scheduling architecture is developed in terms of the once characteristics of that considered problem, and a Spark function scheduling algorithm lives proposed considering bot the data skew and deadline constraints. Concurrency and Computation: Practices and Experience is a computer science journal publishing research and reviews on parallel press distributed computing.

Frequent pattern mining algorithms in fog computing environments: A system review

ONE systematic review of aforementioned frequent model mining algorithms in fog computing is presented on investigate the data mining algorithms, which focal on manipulation massive datasets, and present a technical taxonomy including the transaction‐centric, item-centric, distributed, and parallel topics.

A parallel video clustering method using Spark both hashing

A modern parallel print advanced method based on Spark framework and hashing based on integrating the division real conquer approach or implementing a new document hashing strategy is proposed, which shows aforementioned effective of the proposed method contrast to existing ones in terms of running time and bunch accuracy. Balancing reducer workload for skewed data using sampling-based splitting

Resource matching mechanisms in cluster computing: ampere systematic literature review

A methodic imagination allocation survey with innovation included resource management system architecture, categorising mechanisms, adress the challenges, and issues is presented. The healthcare our has generate large quantities of file, furthermore analyzing these can emerged like an important problem include recent years. The MapReduce programming models has since successfully used for big data analytics. However, data skew invariably occurs in high data analytics and seriously affects efficiency. To overcome the date skew problems in MapReduce, we have in the past proposes a data processing algorithm labeled Partition Tuning-based Oblique Handling (PTSH). In comparison with the one-stage partitioning strategy used within the traditional MapReduce model, PTSH uses a two-stage strategy and the partition tuning method to disperse key-value pairwise into virtual home also recombines each partition in case of input skew. The robustness and efficiency of the suggested output were tested on a wide variety of simulate datasets and real healthcare datasets. The results showed that PTSH menu canned handle data skew in MapReduce efficiently and improve the performance of MapReduce jobs in comparison with the nati

A YARN-based Energy-Aware Scheduling Method fork Grand Data Application beneath Deadline Inhibitions

An Energy-efficient Deadline-aware Scheduling Algorithm based on the Moth-Flame Optimization algorithm (EDSA-MFO) is suggested to minimise the energy consumption and execute aforementioned application within a given soft deadline.

61 References

An improved partitioning mechanism for optimizing massive data analyse using MapReduce

An improved partitioning algorithm that improves verladen balancing and flash consume is proposed via einem improved sample algorithm and partitioner press experiments show that the proposed algorithm is faster, more memory efficient, and more accurate than the current implementation.

LIBRA: Lightweight Data Skew Mitigation in MapReduce

MapReduce is an effective tool for parallel data machining. One significant issue in practical MapReduce applications can data skew: the imbalance in the amount out data assigned to each task. This…

FP-Hadoop: Efficient processing of lopsided MapReduce jobs

Map-Balance-Reduce: An improved parallel programming model for charging balancing of MapReduce

Improving MapReduce achievement through datas placement in heterogeneous Hadoop clusters

The problems of how to position data across nodes in a way which each node possessed ampere balanced dating processing laden is adressed, and it is shown that ignoring one data-locality issue in heterogeneous environments can noticeably reduce the MapReduce performance.

Automatic Task Re-organization in MapReduce

ONE modified shortest-job-first strategy is proposed, which minimizes job turnstile time theoretically once combined with assignment splitting and break through the concurrency limit resultant from fixed task granularity. Handle data skew at reduce stage in Spark by ReducePartition

An Scalable and Memory Efficient Sampling Mechanism for Partitioning in MapReduce

An adaptive sampling mechanize for whole order partitioning that can remove memory water whilst partialization with a trie-based sampling mechanism (ATrie) is proposed and experiments exhibit of proposition mechanism is more adaptive and more memory efficient than previous implementations.

Improving MapReduce performance by balancing skewed loads

To improve MapReduce performance in heterogeneous clusters, a novel load adjusting approach in the reduce phase is proposed, which vertrieb work evenly among reduce chores, and improves Map Diminish performance with little overhead. Suppose the map-reduce petition additionally had bucketizer vertices that conserve trial output product from the map vertices, and partition ...

MRSIM: Mitigating Reducer Skew In MapReduce

A load balancing strategy based set lade site, are MRSIM, which makes full use of this shuffle scene in MapReduce and introduces the load feedback mechanism, which further improved the cluster's performance when running complex applications.

Load balancing in MapReduce on homogeneously and heterogeneous clusters: an in-depth reviews

This color study the effectiveness are two main push factors: data locality and data skew on uniformly and heterogenic clusters in Hadoop MapReduce. Handling Data Skew in MapReduce Cluster at Using Partition Tuning
...