This document discusses a methodology for monitoring internet traffic and detecting anomalous behavior. It begins by noting the challenges of understanding vast quantities of internet traffic data due to the diversity of applications and services. Recent cyber attacks have made it important to develop techniques to analyze communication patterns in traffic data for network security purposes.
The proposed methodology uses data mining and entropy-based techniques to build behavior profiles of internet backbone traffic. It involves clustering traffic based on communication patterns, automatically classifying behaviors, and modeling structures for analysis. The methodology is validated using data sets from internet core links. It aims to automatically discover significant behaviors, provide interpretations, and quickly identify anomalous events like scanning or denial of service attacks.
How to Troubleshoot Apps for the Modern Connected Worker
Anomaly Detection in Internet Traffic Using Data Mining
1. INTERNET TRAFFICMONITORING
FOR ANOMALOUS BEHAVIOUR
DETECTION:ABSTRACT:As the internet continues grow in size and
complexity the challenge of effectively provisioning
managing and securing it has become inextricability
line to a deep understanding of internet traffic.
Although there has been sufficient process in
instrumenting data collection system for high
speed network at the core of the internet
developing a comprehensive understanding of
collected data remains a daunting task . This is due
to the vast quantities of data and the wide diversity
of end host application and services found in
internet traffic.
Recent spates of cyber attacks and frequent
emergence application affecting internet traffic.
Dynamics have made it imperative to develop
effective techniques that can extract and make
sense of significant communication patterns from
internet traffic data for use network operation and
security management. In this pattern we present
general methodology for building comprehensive
behavior profiles of internet backbone traffic in
terms of communication patterns of end –hosts
services. relying on data mining and entry-based
techniques ,the methodology consist of significant
cluster extraction ,automatic behaviors
classification and structural modeling for in
depth interpretive analysis.
example observations may review the effects of
event such as a network failure and operational
failure or a security incident on network traffic.
There are several other uses of network
monitoring equal in Quos estimation
bandwidth planning etc but in routine network
monitoring the interest on events. if there are
not event of interest network manage will
probably not want to ”Look” at the traffic .the
traffic data such cases is destined for archiving
from here it would probably be backed up on
off line media or disconnected .
Present monitoring system don’t have
mechanism or detecting event of interest .so it
appears that operator will either at will the
traffic mechanically .we use data event from
wide area network examine the utility and
effectiveness of approach. The process of
mechanical event detection heavily
Depend on the availability and accuracy of data
but in standard monitoring environment there
is life guarantee for these two factors .to erase
the availability and accuracy of the data we
purpose the deployment of multiline data
collectors at geographically and network.
Topologically separated points .we has carried
out experiment on wide area network and
have existing the combined how the quality of
the data can be raid
Availability and accuracy of that can be
increased using the collection of redundancy.
CHAPTER-1
INTRODUCTION ABOUT IN PAPER:Network traffic monitoring is important aspects
of network management and securing .for
In this paper we present a general methodology
for building comprehensive behavior profiles of
internet backbone traffic in terms of
communication patterns of end –host and
services. Relying on mining and entropy based
2. techniques, the methodology consists of
automatic behavior analysis .we validate the
methodology using due set from core of the
internet.
methodology using due set from core of the
internet.
LITERATURE REVIEWS:-
SYSTEM STUDY:-
Recent spates of cyber attacks and frequent
emergence or applications and affecting
internet traffic dynamics made it imperative to
develop effective techniques that can extract
and make sense of significant communication
patterns from internet traffic data for use in
network operation and security management.
The system study phase analyze the problem of
existing systems defines the objective to be
attained by solution and evaluates various of
solution alternatives.
The process of mechanical event detection
heavily depend on the availability and accuracy
of data but in standard monitoring
environment there is life guarantee for these
two factors .to erase the availability and
accuracy of the data we purpose the
deployment of multiline data collectors at
geographically and network, topologically
separated points. We have carried out
experiment on wide area network and have
existing the combined how the quality of the
data can be raised. How the availability and
accuracy of that can be increased using the
collection of redundancy.
In this paper we present a general methodology
for building comprehensive behavior profiles of
internet backbone traffic in terms of
communication patterns of end –host and
services. Relying on mining and entropy based
techniques, the methodology consists of
automatic behavior analysis .we validate the
methodology using due set from and entropy
based techniques, the methodology consists of
automatic behavior analysis .we validate the
CHAPTER -2
EXISTING SYSTEM:Recent spates of cyber attacks emergence of
applications affecting internet traffic dynamics have
made imperative to develop effective techniques
that can make sense of significant communication
patterns from internet traffic data for use in
network operation and security management
.network monitoring is alone performed using many
tool like snort .many web portals establishing
without data mining technique will need to serious
problem while number of user increase.
SIMPLE NETWORK MANAGEMENT PROTOCOL(SNMP)
DISADVANTAGE OF EXISTING SYSTEM:As the internet continues grow in size and
complexity the challenge of effecting provisioning,
managing and security. It has be inextricably liked
3. to deep understanding of internet traffic .although
there has been significant progress in instrumenting
data collection for high speed network all the core
of the internet, developing a comprehensive
understanding of the collected data remains a
daunting task this is due to the vast techniques of
data and wide diversity of end hosts, applications
and services found in internet traffic.
to all the remaining clusters to find out anomaly
behavior .
ADVNATAGE OF PROPOSED SYSTEM:-
There is processing need for techniques that can
extract underlying structures and significant
communication patterns from internet traffic data
for use in network operation s and security
management.
The methodology for profiling internet backbone
traffic that 1) not only automatically but 2)
discovers significant behaviors of interest from
massive traffic data but 3) also provides a possible
interpretation of these behaviors and quickly
identifying anomalous events with a significant
amount of traffic . e.g. Large scale scanning
activities worm outbreaks and denial of service of
tasks.
PURPOSED SYSTEMS:-
PROBLEM DEFINITION:-
in this purposed systems we use packet header
tracker collected on internet backbone links in fire
–ISP what are aggregated into flow based on the
well known the source IP address source port
,destination port and protocol fields. Since our goal
is to traffic in terms of communication patter ns we
start with the essential four dimension feature
space.
Recent monitoring systems don’t have
mechanism of detecting events of interest .so it
appears that the operator will either look at all
the traffic to detect events of internet or will not
look at the traffic all in our work we attempt to
mechanically detect event of interest and draw
operator attention to these events .we use data
from wide area network to examine the utility and
effectiveness of the approach. But in standard
monitoring environment there is little guarantee for
these two factors. To raise the availability and
accuracy of the data in purpose the deployment of
multiple data collections at geographically and
network topologically separated point.
Using four dimensional feature space we extract
clusters of significance along each dimensions
where each cluster consists of flows with the same
feature value in said dimension .this leads to four
collection of interesting clusters.
The first two represent a collection of host
behaviors while the last two represent collection
of service behavior .in extracting cluster
significance instead uses a fixed threshold based
on volume adopt an entropy based approach that
cells interesting illustrates based on underlying
feature value distribution in the fixed dimension
.imitatively clusters with feature value that are
distinct in terms of distribution are considered
significant and extracted the process is repeated
CHAPTER-3 :SYSTEM ANALYSIS:The analysis of a problem that will try to solve with
an information system .it describes what a system
should do?
PACKAGE SELECTED:-
4. The package selected to develop the project JDk
1.5 and win cap tool. the selected package have
more advanced feature .as the system is to be
develop in networking domain .we had preferred
java2 standard edition .the supports all class
libraries. Window XP with all features is selected as
the development (operating system) area to install
and develop the system in java platform.
required design, develop, implement and
test. The project, the resource to analyze is
employees’ time and
SRS. Teams of
three members are involved in the entire
SDLC. Lifecycle except the testing phase .the
testing phase guided by manual tester
before the hosting the application in the
server space.
Time analyzed to complete this project
approximately two months with 4hrson
daily basis except week ends .SRS is
prepared and provided as per the URS.
Window XP with professional offers a no. of
features unavailable in the home edition including:
•
•
•
•
•
•
•
The ability to become part of windows
server domain a group of computers that
are remotely managed by one or more
central servers.
Remote desktop server which allows a PC to
be operated by another window XP user
over a local area network or internet.
Offline file and folders which allow to PC to
automatically store a copy of files from
another network computer and work with
while disconnect from network.
Centralized administration features,
including group, policies, automatic
software installation and maintains room
user profiles and remote installation
services (RIS).
Internet information services (IIS),
Microsoft HTTP and FTP server.
Support for two physical central processing
units (CPU).
Windows management instrumentation
control (WMIC) .WMIC is a command line
tool designed to parse WMI information
retrieval about system by using Keyword
(aliases).
RESOURCE REQUIRED:Planning and analyzes the resources is also
one of the major part of the SDLC to
complete he has given time. In this we need
analyze the availability of resources that are
FEASIABILTY STUDY:
The feasibility determine whether the
solution is achievable, given the
organization resources constraints by
performing feasibility study the scope of the
system will defined completely.
Most computers systems are develop to satisfy is
known user requirement this means that the first
event in the life cycle of system is usually task of
studying whether it is feasible to computerize a
system under consideration or not. Once the
decision is made report is forwarded and is known
as feasibility report. The feasibility is studied under
the three contexts.
a)
b)
c)
A)
Technical feasibility
Economic feasibility
Operational feasibility
TECHNICAL FEASIBLITY:What resources are available for given
developer system? Is the problem worth
solving? in proposed system technical
feasibility centre on the existing computer
system and what extent it can support the
purposed system .therefore now we need
to install the software existing system for
this project and operation of this system
requires knowledge about window XP
5. window professional ellipse and JDK 1.3,
the assistance would be easily available.
Even though these technical requirements
are needed to implementing system code is
generated and compiled. The executable
code of project is sufficient to application
hence the proposed system is feasible.
B) ECHONOMICAL FEASIBLITY:Economic feasibility is used for evaluating
the effectiveness of a candidate system .the
procedure to determine the cost
benefits/saving that are accepted from a
candidate system and compare with the
cost. If the cost is less and benefit is high
then decision made to design and
implement. The system regarding the
maintains, since the source code will be
with company and small necessary changes
can be done with minimum maintains cost
involve in it. The organization has to spend
amount of technology as it is not
computerized the present system
performance is high when compared to the
previous system. So for the organization the
cost factor is acceptable so it is
economically feasible.
If installed will certainly beneficial since the
will be reduction in manual work and
increase in the speed of work there by
increasing the profit of company and saving
time. As the purposed system as JPCAP is
free download tool since the system is
economically feasible.
C) OPERATIONAL FEASIBLITY:Network traffic profiling and monitoring
system is many developed to monitor the
made is network this is done by using JPCAP
tool .the system should include feature like
• Extract the parameter from the client
network.
• Monitor the parameter in the list view
•
Analyze the anomaly packets.
The main problem developing a new
system is getting acceptance and the
co operation from the users are
reluctant to operate on a new system
.the software being developed is more
interactive with the developing
system .it is instantaneous , moreover
even a new period can operation, the
system and easily execute the system.
So it is operationally feasible.
User network diagram
CHAPTER-4
SYSTEM DESIGN:
In this design phase of SDLC both logical and
physical design specification for the system solution
are produced modules are:
1) METWORK DESCRIPTION
2) PACKET ANALYSIS
3) PACKET ANALYSIS
6. 4) GRAPHICAL INTERFACE
Module description:Network Monitor Packet Capture:
This feature provides the faculty of capture
network packet. This packet will be parsed
and the packet header detail will be listed in
table the packet can be stored in serialized
formats. This packet can be store in file
retrieved later for viewing and analysis.
When packet come up with a new for
creating network if often takes security
community a while determine the method
used .in aircraft‘s black box is used to
analyze the default of a crash .we believe a
similar capability is needed for network.
Being able to quickly learn how attack work
can will shorten the effective useful lifetime
of the attack.
PACKET FILTERING:The captured packet can be filtered to
display according to the packet type the
packet can be filtered by protocol type
TCP(transmission control protocol
),ARP(address resolution
protocol),UDP(user datagram
protocol),ICMP(internet control message
protocol) and IGMP(internet group
management protocol).
ADVANTAGE:
• easy to install
Packet filter make use of current
network router therefore
implementing a packet filter
security system is typically thus
network security software.
• support high speed
• With simple network configuration,
packet filter can be fast since there
is direction connection between
internal user-end external hosts
data can be transmitted at host
speed.
• make s security transparent to end
–users
Because packet filters work at the level of
the network router, filtering is transparent
to end user that makes uses client
application much easier.
DISADVANTAGE:• leave data susceptible to exposure:With packet filter user connect directly
network to network. Direct connection
leave data susceptible to exposure such as a
user address from the data stream network
security can be compromised.
• offer little flexibility
Creating complex access rates with packet
file can be different with segments local
area network to configure rule set for user
with different access privileges.
• maintain no state related
communication
Packet filter make decision based on
individual packet and not on the “context”
of the traffic this will not provide good
security as can be seen from the ex. In case
of packet filter either we need to open all
ports greater than some number (1023) or
else the FTP will fail.
• offers no user base authentication
Packet filters are restricted to design or
granting access based on source or
destination address ports. There is no way
for packet filter to authentication
information community from specific user.
PACKET ANALYSIS:The detailed packet information is displaced
below:
7. •
•
•
•
Build customized capture and
display filters
Tap into local network
communication
Graph traffic network pattern to
visualize the data flowing
across your network.
Build states and report to help you
better explain technical
network information to
non-technical users.
GRAPHICAL INTERFACE:A graphical interface (GUI) is type of user
interface which allows people to internet
with electronics device such as computers.
hand held devices such as MP3 players
portable media players or gaming devices
household application and office
equipment .a GUI offers graphical icons
and visual indicators as opposed to text
based interfaces type command labels or
text navigation to fully represent the
information and action available to user.
The action is usually performed through
direct manipulation of the
graphical interface.
We have implemented an easy to use
window build graphics user
interface.
Special Feature of Language Utility
Introduction to java:J2se is collection of java programs API (Application
programming interface) that is very useful l many
java platform programs. It is derived from one of
the most programming language known as a
“java”&one of the three basic edition of java
known as java standard edition bring used for
writing applet &other web based applications.
J2se platform has been developed under the
java umbrella &primarily used for writing applets
&other java based applications .It is mostly used
for individual computers .Applet is type of fast
working subroutine of java that is independent
platform but work within other frame works .It
is minimum application that performs a variety
of functions large &small ordering &dynamic
within framework of larger application.
J2SE provides the facility to user to see flash
moves or hear audio files by clicking on web
page link. As the user clicks pages goes into the
browser environment &begins the process of
launching application-within an application to
play requested video or sound application. So
many online games are being developed on
J2SE.java Beans can also developed by using
j2SE.
About Swing Design:Project swing is the part of the java function
classes (JFC)s/w that implements a set of GUI
components with pluggable look &feel. Project
swing is implemented entirely in the java
program language & is based on the JDK 1.1
lightweight via framework.
The pluggable look & feel lets you design a
single set GVI components that can
automatically have look & feel of any OS
platform (ms Window, Solaris,& MAC into)
Project swing component is include both 100%
pure java certified versions of the existing
AWT components set (Button ,Scrollbar ,List,
Table ,checkbox Textfield, Textarea)
Plus a rich set of higher level components
(such as tree, view, list box & tabbed panes)
ABOUT JCAP TOOL:-
8. JCAP is open source library for it
Capturing and sending network packet from
java application.
Provides facilities to:
*Capture row packet live from the wire.
•
Save captured packet to an
offline file read
capture packet
from the offline fail.
•
Automatically (for Ethernet,
IPV4, IPV6,
ARP/RARP, TCP,
UDP and ICMPV4.
•
Send raw packet to the
network JCAP is based on libpcap/Win cap
is implemented in c and java. JCAP has
been tested on Microsoft windows
(982001XPvistaLINUX (fedora, udanta),
Mac OS X (drawing. Free BSP and Solaris.
Kinds of application to be developed
using JCAP .JCAP can be used to develop
Many kinds of network application are
including:
a) Network and protocol analyzes
b) Traffic triggers.
c) Traffic generators
d) User level bridge and router
e) Network scanners
f) Security tools.
Schedulers and personal firewalls.
Improved Performance:The performance of both client & server application
have been significantly improved in J2SE 5.0.
Monitoring and manageability:J2SE 5.0 bring s advanced monitoring and
manageability framework into the java virtual
machine for java platform (JVM).you can use your
exiting management consoles with industry
standard JMX &SNMP protocols to monitor a JVM
&even detect low memory conditions. The JDK
release provides demo called Jconsole. If lets you
evaluate the benefits in the monitoring the JVM and
see how can exceed your availability matrices.
New Look and Fell:The java platform contains already pluggable look
and fell frame work the addition of the new ocean
look and fell enables cross platform application to
switch between ocean and native operating system
look and fell without the need to rebuild or
recompile them.
Reduced Startup Time:-
WHAT JCAP CANN’T DO?
JPCAP captures and sends packet
independency from the host protocol.
This means
The JPCAP doesn’t block filter or
manipulate the traffic generated by other
programs
On the same machine. It simply “shift”
the packet that transit on the wire
therefore
If doesn’t provide appropriate support for
application like traffic shaper Quos
You haven’t started a desktop java application in
the last few years .you may be in for a pleasant
surprise. The introduction of class (in combination
without streamline option) has been saved nearly
30% off the startup time for some application.
Great 64-bit Performance:The J@SE 5.0 64 bit JVM delivered record results
with AMD64/operation CPU and SUSE LINUX
enterprise edition 8.0, SLES 8.0 . in addition the 32
bit version of JRE can run side by side under the
9. same 64 -bit OS for use with exiting 32 –bit web
browsers.
Performance ergonomics:The JVM is none self configuring and self tuning on
server classes machines .a server class machine with
two more CPU and at least 2GB of memory. The
server based performance ergonomics kicked in by
right sizing both the memory required and class of
optimizations needed for longer lived applications.
This has resulted in 80% improvement on one
application server benchmark without changing line
of code or supplying any runtime options.
Reduced Development Time:Integrated development (IDEs) have tried to make
developers little easier with auto completion &
wizards for common tasks J2SE 5.0 new language
feature for further streamline development
whether you use an IDE or hand code in a text
editor.
Reduced Need for Developer Coding:Many for java language changes reduce the
amount t of code a developer has to write .the
following figure quantifies the reduction in
comparison to J2SEs 1.4.2 . to take real life
example one open source application server uses
over 2,00 iterant by substituting the new
enhanced for loop .the code work would be
reduced by up to 4,000 characters.
A network interface object contains some
information about corresponding network interface
such as its name description, IP & MAC addresses
and data link and description.
Open Network Interface:After obtaining the list of network interfaces and
choose .which network interface to picture packet
from interface by using JPCaptor.openDvice ()
method. The following piece of code illustrates how
to open network interface
Capture Packet from the Network
Interface:After obtaining the instance of JPCaptor, you can
capture packet from the interface there is major
approaches to capture packet using a JPcaptor
instance using callback method and capturing
packet one by one.
Then call either JPcaptor.processPacket () or
JPcaptor.openPacket () method to start capturing
using the callback method. When calling process
packet () or loop packet () method also specify the
number of packet to capture before the methods
returns.
Then specify -1 to continue capturing packets
infinitely .the two methods for callback
.ProcessPacket () and LoopPacket () are very similar.
Usually might want to use ProcessPacket () because
it supports timeout and non blocking mode, while
Packet ().does not.
Obtain the List of Network Interfaces:Capturing Packet One by One:To capture packets from a network ,the first thing
you have to do list to obtain the list of network
interfaces on your machine .to do so JPCAP
provides JPCaptor.getDeviceList() method .it returns
an array of network interface objects.
Using callback method is little key bit tricky because
you don’t know when the callback method is called
JPCAP. if you don’t want to use callback method
also capture packets using the
JPcaptorCaptor.getpacket()method simple returns a
10. captured packet have to callget.packget() method
multiple times to capture consecutive packets.
Set Capturing Filter:In JCAP set a filter so that JCAP doesn’t capture
wanted packet. The filter expression “IP and TCP”
keep only the packet that are both IPV4 and TCP
and deliver them to the application “.by properly
setting a filter and reduce the number of packet
examine and thus can improve the performance
of your application.
Save Captured Packet into a File:To save captured packet into a binary file so that
later review then using JPCAP or other application
.when supports reading to TCP dump format file.
To save captured packet first need to open a file by
calling JPcaptor .open file () method with an
instance of JPcaptor which is used to capture
packets and string filename. After obtained an
instance of JP captor through open file () method, to
save capture packet using JPcaptor. Write packet ()
method .after saved all the packet to call JPcaptor
writer. close () method to close the opened file.
Read saved packet from file in JPCAP read the
packet saved using JPcap writer by opening the file
using JP captor. Open file () method. Similar to
JPcaptor. Open Device () method JPcap captor.
Open files () method also returns an instance of
JPcaptor classes. so use the same ways described in
capture packet from the network interface section
to read packet from the file. Send packet to the
network using JPCAPS it is need to obtain an
instance of Jcapsender. Opendevice () or
JPcaptor.getcap. sener () instance methods.
After obtaining an instance of Jcapsender passes an
instance of packet class to JPcap sender .send
Packet () method.
Introduction to Eclipse Tool:Eclipse is an extensible open source IDE (Integrated
development environment).the project was
originally launched in Nov 2001.when IBM donated
$40 million worth of source code from web sphere
studio workbench and formed the eclipse
consortium to manage the continued development
or the tool.
The state goals of eclipse are “to develop or robust
full featured commercial quality industry platform
for the development to highly integrated tools” to
that end the eclipse consortium has been focused
on three major projects.
1.the eclipse project is responsible for developing
the eclipse IDE workbench the platform hosting
eclipse tools, the java development tools (JDT) and
plug In Development Environment(IDE) used to
extend the platform.
2. The eclipse tools project is focused on creating
best of bread tools for the eclipse platform current
subprojects include a COBAL IDE a C/C++, IDE and
EMF modeling tool.
3. The eclipse technology project focuses on
technology research in combination and education
using the eclipse platform.
The eclipse platform when combined with IDE
offers many features you did not expect from a
commercial quality IDE a syntax highlighting editor
,incremental code compilation a thread aware
source level debugger class navigator a file project
manager interfaces to standard source control
system such as CVS and clear case.
Eclipse also include a number of unique factors
such as a code refactoring ,automate code update
installs(via the update manager),task list and
support for unit testing with joint and integration
with Jakarta build tool.
11. Despite large no. of standard features eclipse is
different from traditional IDEs is a number of
fundamental ways. Perhaps the most interesting
feature eclipse is that is completely platform and
language neutral .in addition to the electric mix of
languages supported by the eclipse consortium
(Java, C& C++). There are also projects underway to
add support for languages as diverse as python,
Eiffel & Ruby &C# to eclipse.
Platform-wise the eclipse consortium provides prebuilt binaries for windows, Linux, Solaris, HP-UX,
AIX, QNX and MAC OS XP. Much of the interest in
eclipse centre around the plug in architecture and
rich .APIs provided by the pug in development
,environment for extending eclipse adding support
for a new type of editor viewer programming
language is remarkably easy ,given the well
designed API and rich building blocks that eclipse
provides with hundred plug in development
project in progress ,industry giants like IBM,HP and
rational(just award by IBM) providing resources
and design heavy weight lake Erich gamma helping
to guide the process the future indeed looks bright
for eclipse
ARCHITECTURAL DESIGN:Architecture diagram shows the relationship
between different components of systems the
diagram is very important to understand the overall
concept of system.
RESULT:Test case are created manually in ms Excel
sheet for the bugs in each module
&validated again using waterfall model.
ARCHITECTRUAL DESIGN
12. other latency .we also discuss event detection
with these statics applying for network
management. We plan to study following as a
future work. We will estimate the accuracy of
detectors of indications of event .we shall also
evaluate the suitability of the traffic models to
detect the event .we shall investigate there are
of event classification .for example the
relationship between indices.
SUBMITTED:GYAN PRAKASH
(E-mail:prakashgyan90@yahoo.com)
MITHLESH KUMAR
(E-mail:-prabhatk02@gmail.com )
BRANCH:-CSSE
Vinayaka Missions University
CHAPTER:5
CONCLUSION:in this paper ,we are introduce our monitoring
and analysis activities about monitoring
activities .we shows our environment in the
local network about analysis activities we show
our monitoring items one is traffic volume and
AARUPADAI VEEDU INSTITUTE OF
TECHNOLOGY PAYANOOR, CHENNAI
TAMILNADU (INDIA)