Using BIG DATA implementations onto Software Defined Networking

Using BIG DATA implementations onto Software Defined Networking
Mohammed Najm abeer Tariq Mohamed Qasim
DEPT. of comput.engineering DEPT.of comp.science DEPT.of comp.science Technology
Technology University Technology University Technology University
Baghdad-Iraq Baghdad-Iraq Baghdad-Iraq
mustafamna@Yahoo abeer28003@Yahoo mohammad.qassim2002@gmail
Abstract
An elastic , effective, activety or intelligent ,graceful networking architecture layout be desired to make
processing massive data. next to that ,existent network architectures be considerably incapable for
cleatting the huge data. massive data thrusts network exchequers into border it consequence with in
network overcrowding ,needy achievement, then permicious employer exprtises. this offered the current
state-of-the-art research affronts ,potential solutions into huge data networking notion. More specifically,
present the state of networking problems into massive data connected intrequirements,capacity,running ,
data manipulating also will introduce the architectures of MapReduce , Hadoop paradigm within research
requirements, fabric networks and software defined networks which utilizized into making today’s idly
growing digital world and compare and contrast into identify relevant drawbacks and solutions.
Keywords:Big Data; MapReduce; Hadoop; SDN;
1. INTRODUCTION
at present times, implementations be fermantly growing a rate such data be mountng within the
world.Legal Studies be displayed for 2020 the world be growing 50 times a sizes of data had into 2011,
whom currently 1.8 zetta bytes or 1.8 trillion gigabytes of data [18]. A natural cause into a acute mounting
into data making a storage along years merely decline into fees from a store.
IT industry making a fees from store become low then implementations be able from provision data
exponential rates. That will get an affront from making present network buildings in what way running
then execute these massive data to be used to utilizing information [1, 5 - 6, 16]. numerous huge data
implementations action or requirements work in a good-time. implementations required into make,
storage then operates a big quantities from information whom decrease big accord from quantity ,need
for a network. When looking at data from a networking perspective, multi-level regions be required to be
reconnoiter which contain network formatting , parallel structures then huge data progressing algorithms,
the data recovery then confidentially problemss [13]. The topic massive data as yet afresh rousing parts
from research among the Information Technology staff , being demand on extremely regard into the years
onto derive applying onto network theory,huge data being described as any aggregate or data-in-motion
and many of its applications have real-time requires into making effective especially in an
education/research environment [8]. the industry, common implementations utilize to massive data contain
for analyzing the data to come up with efficient conclusions in implementation domains like astrophysics,
particle physics (e.g., CERN research), biological science (e.g., healthcare), geological science, social
networking (e.g., Facebook), trend prediction (e.g., google flu), optimizing route delivery(e.g., UPS package
delivery, fuel tracking) [13].
International Journal of Computer Science and Information Security (IJCSIS),
Vol. 15, No. 8, August 2017
139 https://sites.google.com/site/ijcsis/
ISSN 1947-5500

A typical organization has a limited network infrastructure and resources which able be catching that sizes
into traffic flows whom cause standard services (e.g., Email, Web browsing, video streaming) being
strained. these making come down network execution affecting bandwidth and exposing hardware
requirements from tools like a firewall progressing be making over whelmed [13]. This paper presents a
comprehensive survey on the network theory of big data. This work startsby introducing the concept of big
data and networking theory by giving some background information on the state of networking issues related
to capacity, management and data processing in Section II. In Section III briefly presents the architecture of
Big Data which includes the MapReduce paradigm and Hadoop distributed architecture, fabric network
infrastructure and software defined networks (SDN). We present architectures, designs techniques using
theorized solutions to handle today’s idly growing digital world and compareand contrast them to identify
relevant problems and solutions. Finally, we discuss, evaluate, and state conclusions on the analysis of the
defined Big Data networking concepts in Section IV.
2. BIG DATA NEEDS
this introduces making network, a different data types, data flow needs and
implementations from massive Data.
2.1. meet the requirements for the massive Data Network
Huge data requires a good execution request from a networks buildings implementations that network
needs being to extra efficient, flexible then have some form into implementations knowledgs. Huge data
network architecture making be deals with a distributed architecture so as to planning , making
accessibility from distributed resources when together making in parallel on a single job [7]. Massive data
implementations needs starts a progressing big sizes from the information makes a bigger as data
propagated towards network. Table 1 shows the comparison of traditional and massive data kinds 11].
2.2.Needs from different data kind
Features into massie data kinds being so diverse of the classic data patterns. Huge data
implementations which needs a good -time realization needs the big data sets being defective of a little
growing volums onto progressing . The system can not able to executed onto perfect online transaction
processing , while utilizing the common SQL analysis models [11]. Massive data requires a further agile
flat horizontally scaled data base environment being be able to catching differents unstructured datakinds
within complicated , unknown data relationships amongst each other in distributed network architecture. a
little older data bases modules making efforted to making a mutate onto building a NewSQL, unless
generality have abandoned SQL for new models such as NoSQL [8]. These new databases making difficult or
Vol. 15, No. 8, Augus 2017
ISSN 1947-5500

hard to use huge data’s analytical devices ,responsibilities. Figure 1 shows the throughput comparison of
RDMS and No SQL data base over the volume of data.
2.3. Network Flow needs
network infrastructure a long the years has been adapted in a 3-tier architecture contains computing,
storage, networking. the trends of network traffic is used to as “north-south” which means it usually
starts at the end user then goes down the integrated stack to the server than to the database. With the
needed of massive data in networks, the traffic pattern requires to change. Massive data structure
depending on a horizontal scale ,utilizes a distributed approach among the nodes, the traffic demands
between the server and storage nodes are much higher than the typical data flow between servers and end
users. Data is no longer just being created by external sources, generated from a number of devices and
applications. This type of traffic flow is referred to as machine-to-machine/virtual machine network traffic or
“east-west” and it needs to be supported when creating a high-execution scalable huge data network.
3. BIG DATA ARCHITECTURE
we present architecture and research challenges of MapReduce and Hadoop [2,3,4]that are used to huge
data manipulating. introducint a fabric network technology [19] utilized into massive data infrastructure
over within characteristics ,requirements. briefly introduced software defined networks (SDN), that more
popular onto huge data execution.
3.1. Hadoop and MapReduce
MapReduce a core component of the Hadoop Apache software framework , a kind from a programming
module whicht used to implemented into different onto languages (e.g., Java, C++) which utilizing to
progressing massive data [17]. these kinde from the software device be applications into
smallerfragments or blocks which are then sent out to nodes in a cluster or map. It uses a map function that
will able to filter, sort, and distribute jobs to various nodes and also uses a reduce function to collect the
results from those jobs so they can resolved into a single value to be used for efficientanalysis (Figure 2 )
The MapReduce consists of a job tracker, task trackers, and sometimes a job history server [23].The job
Vol. 15, No. 8, Augus 2017
ISSN 1947-5500

tracker is used as the master node that is in charge of managing resources and jobs. The task tracker is used
to be deployed to each node in order to run the map and help with some of thecluster task load. The job
history server is used to track finished jobs and can be deployed as an independent function. MapReduce
operates in parallel across vast cluster sizes, while jobs can bedivided across many different servers [17].
MapReduce has fault-tolerance where each nodesends status updates to the master node, who can re-assign
jobs to functioning nodes in cases of node failure.
On the other hand, Hadoop is a management software framework that plays an important role inbig data
analytics. Hadoop is capable of cataloging, managing, distributing, and queryingunstructured large data sets
rapidly across many nodes within a distributed network environment.It uses a Hadoop distributed file storage
system (HDFS) for storage that divides data into blockswhich is distributed and stored on multiple nodes. In
order to process the data, Hadoop usesMapReduce to break down data for the nodes to process and sort in
parallel (Figure 3). The mapprocedure is responsible for filtering and sorting and the reduce procedure
focuses on summarytasks. It supports high speed transfer rates and is capable of resilient uninterrupted
operation insituations when there is node failure [20]. The infrastructure divides up the nodes into groups
orracks. Hadoop is an excellent framework for applications using large search engines such asGoogle
Together Hadoop and MapReduce provide the current popular choice for implementation of bigdata
infrastructure [10]. A typical Hadoop network structure consists of a slave (data node, tasktracker), master
(name node, job tracker) and a client [9]. The client is basically the user interfaceor query engine. The data
nodes are used as storage for the data that contain smaller databasessystems and are horizontally
distributed across the network. The task tracker is used to process the broken down fragments of the task
that has been distributed to a node. The name node maintains a location index of all the other nodes found
in the network, so it knows where the specific data is located in which data node. The job tracker is the
software job tracking mechanism that is used to transfer and aggregate request search queries (tasks)
through to the tasktracker nodes so the end user can perform information analysis on the result.
Vol. 15, No. 8, Augus 2017
ISSN 1947-5500

3.2. Making a Network substructue onto massive Data
Atopology depending on a concatenation of switches Known as a leafs which take the access layer and
these switches be moer meshed onto spine switches directly building a “fabric” network (Figures7 - 8).
Every access-layer switch be one hop away onto each other, basically decreasng a likelihood of bottlenecks
then reducing the time latency [22].
A leaf-spine topology can function in layer 2 and 3 that leads switching, routing between the leaf and spine
layer. Fabric is said into grown connectivity ,flexibility in networks which executet dynamic virtual
environments , prepared for a converged scalability ofdevices. The fabric network topology is defined as
“when network nodes are connected into each other using one or many network switches” [19] .
Vol. 15, No. 8, Augus 2017
ISSN 1947-5500

Fabric networks are based on spreading the flow of network traffic across many physical links,which
outputs an increase in higher throughput, bandwidth and performance over typicalbroadcast networks [15].
A flatter converged network is simpler and cheaper to install, while growing traffic efficiency ,decreasing
congestion. Fabric networks have the benefit upon a tree topology because they use layer 2 network paths
which help with load balancing and reducingredundancy [21]. A fabrics convergence feature allows
multiprotocol communication andtransparency extending to all locations in the network under a single
environment. This allows theimplementation of consistent services and a greater network intelligence
management policy thatcan provide efficient and secure communication [21].
different characteristces into fabric above a structural tree based topology. first into a fabric
topology depending on staffing a point-to-point conexion between nodes utiliziing a single hop distance in
relation to the switching process, that greatly decreases latency in the network. Second, may be tolerable to
perform a switching fabric utilizing virtualization that permit to diverse networking combinations into
perform as one then support to enhance switching execution. likely into make a pool of virtual switches
which decrease manual work from the executivor.Third, via having a flat architecture of a single extended
environment, it is easier supplier extenination into multi uses areas of the network, when makinging a data
center fabric topology[15]. Expansion and enhancement done easily by creating virtual domains or by
making point-to-point cconexions.
3.3. Software Defined Networking substructure
SDN [22, 26] were progressing into a network arrangement to make network administrators runs
manipulating planes. The massive Data connections uses to run one to many connections of client to
databases to supply services and manage a high end implementation s so as to load massive amounts of
dynamic unstructured data that traditional networks hard to cure and treat. meaning time a data-planes
tools link and react on a standard patterns. detaching the control plane and data plane (packet
forwarding) make changes who time locked system to an open , action it imaginable to disposition
network tools working. detachs to 3-layers: application, control and infrastructure/data (Figure 9).
Vol. 15, No. 8, Augus 2017
ISSN 1947-5500

With a view to performance employ SDN architecture at supplying programmable network communicate
one be fit run these practicability ,employment about a network by software programs. right away
running planes grants an originative trend to leverage adjust the control plane to tool up a elastic and
adaptable topology which be used to work with multi uses networked applications and services. Open
Networking Foundation supplies the mostly gets the SDN:
“Software-Defined Networking becomming the network building wherever the control plane separated of
forwarding plane , then fair to programmable” [22]. Figure 10 explain a structure about massive data
which programming operations , huge data assure, programming , scheduling and triggering.
3.3.1. parcticabilities about SDN
goal from Software Defined Network making the SW which has been used to manage the network [27],
to layer of abstraction the goal to use an elastic network running system to mode huge Data demands
performance and flexible. The Switches at most forwarding tools s controlling routing decisions which gain
processed through central controller.
A concept from the programmable networks , decoupled data and control plan exposes a rule to whom a
SDN controller performance treats. A control plane connects the channel utilized to replacing indexes
(messages) amongst logical network tools for forwarding and running . make the most of the available
products of SDN set it up with a North Bound plane who employers utilized a programmability by an
(API)onto re-attampt the real time statics , service utilizing . data plane also known as the forward plane
which all the information set out to switched in a network which traffic evaluted by then measured to
service utilizing. Utilizing these network in an application layer for whom custom-made applications
utilized.
The Virtualized networks [27] utilized for getting an elastic and resillient . then clients beginning thrust
accordingly you utilized a service to holder virtual network iunderlying that utilizing the Virtual Private
Server (VMs) to build the customized topologies simplifying the dynamic network re-arrangment utilizing
the centralized control plane executed within a software.
Vol. 15, No. 8, Augus 2017
ISSN 1947-5500

3.3.2. Traffic-Aware Architecture
A Northbound API connecting SDN controller utilizes the additional implementation to relocating.
Open Flow be used into conform the callbration from a network tool programming , arrangement in the
midst controllers , switches. operation Open Flow contains of 2 - rules that identify the fields, running
the packet headers then the instructions make to competitions. Through crossing controller forwarding a
switch rule then path from a packet. Effectiveness the controller supples established in advance identify
the principles which be executed over the static or dynamic SDN control software. the administrator will
decide to use a flow to traffic utilizing models, applications , cloud re-resources. During the Open Flow
considered a strongly techniques which existence utilized the operations SDN control plane data which
has deiciency while taking into consideration a real-world hardware requirements. to be further
essential flow a confiqurations which it has suggested by Curtis[28] the authors of Networking to a huge
Data moduler Curtis scanning the delineation Devo Flow to “devolve control by cloning rules whenever
flow built utilizing the wild cards.” work done in [13] scaling the overhead by Open Flow so that possess
further enhancement to flow running to massive Data scanning applications networking utilizies local jobs
to switches owns local re-locate decisions does not need to contact a controller. Figure 11 shows SDN
controller and flow table connections to Open Flow.
Vol. 15, No. 8, Augus 2017
ISSN 1947-5500

3.3.3. Application-Aware Architecture
Aim of massive data networks being permanently enhancing effectiveness onto a networks during
network arrangements , programmability. doing this impotance from network infrastructure be aware of
implementation deficiencies capable of service its requires. A technigues onto service implementation
requires to utilize switches pre-rendered flow information into control plane so as to locate , arrange
scheduled flows. the flows implement-level information and disorder be contain at the transport layer
from huge Data implementations. that participate multiple data transfers called as shuffles from mappers
and flow tables connecting into specific sorting of flows. enhancement the flows both hardware and
application requirements deficiency to orchestrate a correlation for seamless reconfiguration of resources
and upper-layer needs. Due to weak implementations by Hadoop in heterogeneous clusters optimizers have
been proposed. OF Scheduler runs MapReduce jobs, asses network traffic, and load-balances traffic on the
links decreasing the time , takes jobs to finish based on the demand of MapReduce tasks to enhance the
execution [29]. Heavy loaded links are off loaded by preference of load balancing flows especially duplicated
jobs, larger flows can affect global cost so offloading happens to large flows to maximize rendering showing
and prevent the re-scheduling [13].
REFERENCES
[1] Han, Jing, et al. “Survey on NoSQL database”. Pervasive computing and applications (ICPCA), 20116th
international conference on. IEEE, 2011..
[2] Lam, Chuck. Hadoop in action. Manning Publications Co., 2010.
[3] M. R. Jam, L. M. Khanli, M. S. Javan and M. K. Akbari, “A survey on security of Hadoop”, 2014
4thInternational Conference on Computer and Knowledge Engineering (ICCKE), Mashhad, 2014, pp.716-721.
[4 Borthakur, Dhruba. “The hadoop distributed file system: Architecture and design”, Hadoop ProjectWebsite
11, No. 21, 2007.
[5] X. W. Chen and X. Lin, “Big Data Deep Learning: Challenges and Perspectives,” in IEEE Access,vol. 2, pp.
514-525, 2014.
[6] Sagiroglu, Seref, and DuyguSinanc. “Big data: A review.” Collaboration Technologies and Systems(CTS),
2013 International Conference on. IEEE, 2013.
[7] Bushong, Michael. “Six considerations for big data networks”, searchsdn.techtarget.com, Accessed Web
on March 2016
[8] Kvernvik Tor and Matti Mona. “Applying big-data technologies to network architecture”, EricssonReview,
2012
[9] Guohui Wang, T.S. Eugene Ng, and AneesShaikh, “Programming your network at run-time for bigdata
applications”, In Proceedings of the first workshop on Hot topics in software defined networks (HotSDN '12).
ACM, New York, USA, pp. 103-108, 2012
[10] Idiro Analytics, “Hadoop”, http://idiro.com/about-idiro/our-technology/ Accessed Web in May 2016
[11] Introduction to big data: infrastructure and networking considerations, Juniper Networks White
Paper2012.
[12] Jain, Raj. “Networking issues for big data”, Class Lecture, Washington University in St. Louis 2013
[13] Lin Xiaodong, MisicJelena, ShenXuemin, and Yu Shui, “Networking for big data”, Boca Raton:CRC Press,
2015 Print.
Vol. 15, No. 8, Augus 2017
ISSN 1947-5500

[14] Bertolucci, Jeff. “Big data development challenges: talent, cost, time”. Informationweek.com. Web.Aug
18, 2012.
[15] Borovick, Lucinda and Villars and L. Richard, “The critical role of the network in big dataapplications”.
Cisco White Paper, April, 2012.
[16] Chen, Min, Shiwen Mao, and Yunhao Liu. "Big data: a survey." Mobile Networks and Applications19.2
(2014): 171-209.
[17] Dean, Jeffrey, and Sanjay Ghemawat. "MapReduce: simplified data processing on large
clusters."Communications of the ACM 51.1 (2008): 107-113.
[18] Mearin, Lucas. “World’s data will grow by 50x in next decade, IDS study predicts”.
Computerworld.com, Web Accessed on May 2016.
[19] Eneboe, Michael K., and Andrew D. Hospodor. “Isochronous switched fabric network”, U.S. Patent
[20] Jeffrey Dean and Sanjay Ghemawat, “MapReduce: simplified data processing on large clusters”, TheACM
Communication Magazine, Vol. 51, Issue. 1, pp. 107-113, January 2008
[21] Borovick Lucinda and Villars L. Richard, “The critical role of the network in big data applications”.Cisco
White Paper, April 2012.
[22] Banks, Ethan. “Data center network design moves from tree to leaf”.
Search data center.tech target.com, Packet Pushers Interactive, Accessed on June 2016
[23] Jeffrey Dean and Sanjay Ghemawat, “MapReduce: simplified data processing on large clusters”. TheACM
Communication Magazine, Volume 51, Issue 1, pp. 107-113, January 2008.
[24] IBM, “Bringing big data to th enterprise”,https://www01.ibm.com/software/in/data/bigdata/ Accessed
Web on June 2016.
[25] McDougall, Richard. “Is your cloud ready for big data?” Strata Conference – Co-presented by O’Really
Cloudera, New York, Oct 28- 30, 2013
[26] Open Network Foundation, “Software-defined networking: The new norm for networks,” White Paper,
April 2012
[27] Open Networking Foundation, “SDN Architecture Overview”, White Paper V1.0, December 2013.
[28] Curtis, A. R., Mogul, J. C., Tourrilhes, J., Yalagandula, P., Sharma, P., and Banerjee, S., “Devoflow:Scaling
flow management for high-performance networks”, ACM SIGCOMM Computer Communication Review,
41(4): 254-265, July 2011
[29] Zhao Li, Yao Shen, Bin Yao, MinyiGuo, “OF Scheduler: A Dynamic network Optimizer for MapReduce
Heterogenous Cluster”, International Journal of Parallel Programming, Volume43, Issue 3, pp. 472-488, June
2015.
Vol. 15, No. 8, Augus 2017
ISSN 1947-5500

Using BIG DATA implementations onto Software Defined Networking

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Similar to Using BIG DATA implementations onto Software Defined Networking

Similar to Using BIG DATA implementations onto Software Defined Networking (20)

Recently uploaded

Recently uploaded (20)

Using BIG DATA implementations onto Software Defined Networking