Eating the elephant
•Transferir como PPTX, PDF•
1 gostou•70 visualizações
In the fast paced world of information security, analysts are tasked to perform seemingly and often improbable feats of data analysis, and produce actionable results. Actionable could mean, things to block, collect, or be-on-the-look-out (BOLO). Data sources can range from data obtained on-line, device log files, PCAPs, and miscellaneous CSV files to name a few. Seldom does the data align properly, and it could be missing vital contextual information. On the surface, the various data sets may not appear to have any relationship. Not to mention, the small problem, the information totals are in the millions. As if your day was not already turbulent. When, where, why and how do we begin to make sense of the madness? The answer lies in in the solution to this question: How do we eat an elephant? One byte at a time. During this talk, we will analyze real data, discuss and apply various methods, data frameworks and tools coupled with the Python programming language to tackle this obstacle. Along the way identifying methods to triage and process the data. Unearthing the golden nuggets of information that can and will help the analyst better defend the network, and find the proverbial needle in a haystack. It’s time to make the data work for the analyst instead of the analyst being held at its mercy. Even the simplest data sets can yield promising results, we just need to know what to ask, how to look, and enrich the data.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Semelhante a Eating the elephant
Semelhante a Eating the elephant (20)
Último
Último (20)
Eating the elephant
- 1. Eating the Elephant Leveraging Data Analytics to Tackle Everyday Security Tasks and Provide Actionable Intelligence
- 2. Agenda • What is Eating the Elephant? • Actionable Intelligence: A Matter of Perspective • Case Study: The C4 URL List • Eating The Elephant • Data Enrichment and Organization • 21 Questions: The Data Interview • Actionable Findings • Recommendations • Q&A
- 3. What is Eating the Elephant? Eating the elephant is the process of systematically solving a problem one small portion at a time. How do you eat an elephant? One byte at a time.
- 4. Actionable Intelligence: A Matter of Perspective Data and or information that can: • Enhance or close the gap on another topic or initiative • React; create policy or change • Provide greater insight and or understanding
- 5. Background Story Background: Sometime before the end of 2016 a list of presumably vulnerable or open IoT camera/equipment was posted online. This list contains the URLs of 24,400 devices. Scenario: Your boss/CEO/Big Cheese asks you to take action on the items on this list to protect the organization/customers/people/things that matter.
- 6. What you might be thinking… What the #%*$! am I supposed to do with this?
- 7. Eat the Elephant Data Affirmation: You work the data, the data does not work you. • Add enrichment to the URL listing • Free Association Questions: build a baseline of questions to answer • Tie the data into other datasets to add value and context • Let the data talk to you • Small Steps = Big Gains
- 8. C4 URL List Sample Where do we go from here?
- 9. Possible Options • Solution #1: Block IP addresses associated with URLs • Impact: Increased device overhead, may not be considered practical • Solution #2: Block domains associated with URLs • Impact: similar to solution 1, however some blocking or monitoring may prove to be beneficial. • Solution #3: Close your eyes until the problem goes away • Impact: Might get funny looks, boss thinks ITSec is like TV or movies • Solution #4: Eat the Elephant • Impact: Via problem 0wn4g3, deliver actionable information
- 10. The Data: The Good and The Good-ish The Good • Multiple commonalities in URI structure • Source Ports • IP Addresses The Good-ish • IoT CCTV/IP Camera List • Volume (very large list)
- 12. Data Enrichment: Geolocation & Industry Vertical Identify the following traits: • Top originating countries • Top originating providers • Countries and vertical trends • Targeted industry
- 13. Data Enrichment: Virus Total & Detux** • Identify malware names and families • Identify targeted networks and net blocks • Identify binary detectability • Identify what’s not being found or reported
- 14. Data Enrichment: Network Packet (PCAP) Trends • Identify scanned or targeted ports • Identify established communications and protocols • Identify protocol anomalies
- 15. Data Enrichment: Camera Metadata (Server Headers & Index) • Identify configuration trends • Identify static header content • Identify configuration URLs • Identify camera vendors
- 16. Data Organization • Data subsets stored in individual data frames • Aggregated (overlapped) data stored in a matrix
- 17. Data Organization: The Matrix
- 18. 21 Questions: The Data Interview Preparing data questions is like conducting an interview. • Ask the right questions, and you can find gold • Ask the wrong questions and success can be hindered • Possible Questions: • Are there any identifiable vulnerabilities in the camera? • Are there any identifiable network traits? • What vendors are affected?
- 19. Actionable Findings: Observations and Discoveries • Industry Verticals • government, retail, municipal, education, internet service, and telecommunications • Camera Server Versions • 28 Versions • Netwave IP Camera, most identified • JAWS/1.0, most prolific • Countries • 27 Countries • United States and Viet Nam, top 2 • Listening Ports • 20 different configurations • Ports 86 and 60001, top 2 • Binary Targeting • 220,262 IP addresses • 180 Countries • 10,580 Organizations • 51 Samples
- 21. Actionable Findings: Port Comm. & Targeting
- 23. Actionable Findings: Camera Server Versions
- 24. Actionable Findings: Affected Vendors
- 26. Recommendations • Enrich your data as much as possible • Graphs and other visuals are your friends • Don’t be afraid to be wrong • Sometimes a cigar is just a cigar • Chase the rabbit
- 27. Special thanks Special thanks to Detux.org and the Indian Honeynet Project for providing me increased API access to conduct my research. All of you, for attending this talk.
- 28. Contact Info • Twitter: @feedbrain • Email: rrcave@n00dle.org / rrcave@gmail.com • Website: http://www.n00dle.org
Notas do Editor
- By day I work as a Research Analyst for NTT Security, on the Security Engineering Research Team (SERT) looking into malware communications, covert channels, malware, etc. By night I am known as the Daddy Princess by my daughter, forever cast in the role Anna never Elsa. One day I will have my day is the true princess.
- The process of Eating the Elephant is unique to everyone; techniques and applications that work for me, may not necessarily apply to you. The goal of ETE is to turn one big seemingly impossible problem into smaller, manageable byte size portions, that are consumed until the elephant is all gone. Nom-Nom-Nom-Nom-Nom
- Threat Intel is a broad and ambiguous topic. Couple this with cyber and the situation can get a little complicated, though not as complicated as say asking a tomato and a banana to create an apple. There are still challenges, especially depending on your background and perspective. To simplify these challenges and hopefully get all the kids to play nice in the sandbox, Actionable Intelligence in its simplest form can be thought of as described in this slide. Some may disagree and say “no, we need X”, which is correct from their perspective. I am not asking you and the other kids to be BFFs while playing in the sandbox, but to agree on some fundamentals that are needed to keep everyone happy during recess. As you work together, those other perspectives that each of you as individuals bring to the table will find their places as the analysis progresses.
- Every analysis no matter how big or small, whether it’s a single IP address, company name, URL or list of 1000 domains, has a trigger-analysis-scenario reaction (Everybody stop what your doing, this is an immediate priority!). Usually these situations start very broad. How do we make this random information or feed work for us? Often several other questions need to be answered to fully tackle and address this problem. At times there can be a disconnect between the analyst and the person asking the question. The requester, usually someone in a position of power, may be thinking “This is all the information you need,” and in a Jean-Luc Picard voice they command “make it so”. As the analyst it will often be your responsibility to create context around the question and data. That process begins with data enrichment, which is covered later in this presentation. You are the requester’s expert; they have heard about this kind of stuff all the time. It happens on TV and movies all the time. I mention TV and movies in jest, but the reality is many people do feel what is portrayed in TVs and movies is factual.
- A presentation would not be a presentation without a well placed and meaningful cat meme.
- Say this mantra every time you hit a data analytical road block: “I work the data, the data does not work me”. Think it, believe it, do it. Just because the data was given to you in a certain format does not mean you have to work within its walls of limitation. Most likely, the data was presented in that format because it was intended only for dissemination. It’s up to you to make this work; not everyone is inclined or permitted to share their work. Now that the data is working for you, kick through the walls and build something great. It should be thanking you for setting it free.
- This seemingly innocuous list of URLs is actually a gold mine of potential information. Since IP/CCTV and a list of other IoT devices are essentially webservers, their natural verbose information disclosure speaks volumes of how they are configured, and in some cases their intended purpose. All of the URLs were checked by using an open source tool I developed called Incognito, which is used for downloading malware samples via TOR or designated proxy servers. It was repurposed for this task of checking the URLs, and downloaded only the index pages of the base URLs. For example: http://194.214.236.241:80/ was accessed instead of http://194.214.236.241:80/ IMAGE.JPG?COUNTER Incognito stores all information returned by the camera when accessing the index page. Depending on the webserver configuration on the camera, sometimes the bare minimum response is returned. C4 Header Sample {'content-length': '5205', 'url': 'http://2.104.32.106:80', 'ipaddress': 0, 'age': '1', 'server': 'Boa/0.94.13', 'connection': 'close', 'cache-control': 'private', 'date': 'Thu, 12 Jan 2017 00:08:54 GMT', 'content-type': 'text/html’
- ETE is about Problem 0wn4g3. You have been given a seemingly impossible task (queue Mission Impossible theme song). You step up to the plate, hit the mission accept button, role up your sleeves and game on. This is what separates the men from the boys, and girls from the Disney Princesses. Some of the tasks you face will have a clear win and victorious outcome, others may venture down a path that only lead to be stalled. Both are victories because you learned something, anything. No matter how small in significance, that information or knowledge gained could one day be the missing piece of the puzzle that uncovers the smoking gun.
- Data is never really bad or insignificant, it just may not be applicable or provide value in the context it was presented and/or implemented at the current time .
- Brainstorming: let the ideas flow. How can this data be enriched? What paths and rabbit holes can we follow to add value and context to the original data? Free Association (what are the first thoughts you have on topic ) can be a very useful analytical tool; it can help to uncover overlaps and perspectives that may not be inherently obvious at the time, and can also be used to help formulate data interview questions.
- Adding geolocation information to a dataset can add a great deal of context to your data. Trends among IP addresses, organizations, and providers can be unmasked. Couple this with industry verticals more and more clusters, patterns, and other juicy tidbits of information are waiting for you. At the very least, I recommend adding geolocation information to any dataset when possible. The level of detail regarding that information is a matter of personal preference and requirements, however for most analyses simply having the country and provider or ISP data is extremely useful. While there are several companies that sell subscriptions to geolocation feeds, though the costs range dramatically depending on the services provided, there are also open source and free options available.
- Malware names and families are important to know, because it gives you a baseline of characteristics of what to look for. You can confirm with your AV and IPS vendor to know if these are being detected. For me, binaries that have a low or no detect rate are particularly interesting. In addition to being a protocol junkie, strange binary files are another fascination for me. As I mentioned during the talk, items that are tagged by the major AV players, aka common household names, are what we expect and assume will show up. But what about a binary that is not detected as malicious by your favorite vendor, but is identified by a less common, or off-brand vendor? This can turn into a game of “Guess Who’s Right” with your host, Chuckles McMann. Does Off Brand X have a secret sauce; or did my go-to product get it right? I have seen things like this that happen with compressed Go-lang ELF binaries. Once you remove the compression, the item is detectable. At the very least, I make a note to take another look at the structures of the binary in an attempt to identify what is happening. ** For those that are unaware http://www.detux.org is malware sandbox site run by the Indian Honeynet Project. Their data is indexed very well, allowing for searches on protocols, ports, domains, and strings to name a few. Currently their focus appears to be centered around Linux or ELF based malware, but it’s not uncommon to find the occasional script, TARBALL, or EXE that was left behind.
- Ahh, the heart of my data addiction: packets, packets, packets. Packet capture (PCAP) files are an invaluable asset for an analyst. They are not the most friendly of files to work with, but once you know how speak to them, they can be very accommodating to most requests. But whatever you do, never, ever, under any circumstance feed them after midnight or get them wet. In a perfect world where vendors and developers (even the evil ones) follow the standards for programing network communications, we can follow the RFC and protocol specifications to quickly ascertain the purpose of why a certain implementation was used. John Travolta in a $10,000 dollar suit will appear before you proclaiming “The best hackers in the world can do this in 60 seconds”. You need to do it in 30 seconds, and when you find the “evil flag”, the code will turn red. Sadly, good developers and especially the evil ones, may not follow industry standards all the time. Many deceptive traits will be introduced to make your job exponentially difficult and/or serve a purpose only known to the developer. If you need an example of this, look at the current state of IoT. “Excuse me developer person, why did you code a hidden backdoor into the firmware?” I know our jobs are hard, but you are a she-hulk data smashing Disney Enforcer Princess (don’t let the pink dress fool you, she will wr3k your $h*t!), teamed up with Maj0r B@d4zz; difficult is a walk in the park, impossible means “I might have to sling more than 10 lines of code.” You both came to Eat the Elephant, and together proclaim, WE HUNGRY! Identifying traits among the traffic is a great starting point, and as we learn later, it can tell you what, where and how to look for trends in your network or customer data. If that’s not actionable, then the word needs to be redefined.
- Server headers are incredibly useful, but do not always paint the clearest or most accurate picture. Just like most any banner on a system, these can be changed. Even those modifications can provide additional context and intent. Since it is only natural for people to want recognition of their product and or marketing, The metadata in an index file can be very telling. Potentially you could uncover configuration settings, location information, URLs, model numbers, or other product branding.
- Data organization is unique to the individual; me personally, I like to separate most components into individual data sets or collections if I am working in Mongo. At some time during analysis the data will be combined to form a matrix to identify any overlaps. These will often be the focus of, but not the only aspect of analysis. Keeping things separate allows me more opportunity to focus on specific groups of data or on the off chance I accidently modify something, which can happen since I get hit on the head a lot (remember, married), I can start over. It is also helpful to keep things separate when creating graphs or other charts. Over time you will find what best works for you, it could be something completely different.
- At its simplest interpretation the matrix can give a great actionable overview of the data. Scenario: You are asked to prepare a briefing on TechnoWind S.A., a long time customer that has been experiencing spikes in data usage. Your company is currently monitoring both their firewall, and IPS systems, which monitor their core and user LANs. The security devices are not reporting any irregularities. TechnoWind is very security minded, and tries to stay current with the latest patches, updates, and conducts quarterly security audits in addition to semi annual user security awareness training. Based on the above information you deliver a briefing similar to the following: On February 7, 2017 detux.org in affiliation with the Indian Honeynet Project, analyzed an ARM based Linux executable (ELF) binary, triangulated to the malware family known as Mirai. The binary in question was tied to IP address 201.221.0.6. Based on the server configuration it’s listed as a Netwave IP Camera listening on TCP port 90. The binary in question has a total of 190,898 associated IP address. Given the high number of associated addresses, the traffic is most likely outbound and possibly the immediate cause of the spike in bandwidth usage. All of this information was deduced despite not all of the information being mapped. To further add to the brief, IPS signatures could be developed based on the PCAPs. If the device is unknown to the client, a quick scanner could be used to identify other cameras on the network.
- Typically, I start out with a few basic questions to get the lay of the land. Who is the top in the following categories: ipaddress, country, and ISP/provider? If PCAP information is available, source ports, and destination ports will also be factored into the initial stats. From there, various other data questions can be formulated based on the initial data. For instance: Do the events occur at specific times or intervals? Are there any identifiable vulnerabilities? What are the relationships between the binaries? What is the focus of the binaries? What campaign or initiative are the binaries associated with? Configuration Similarities ?
- Graphs and other visuals are great for illustrating points and clusters or areas of concern, and typically managers love visuals. The above graph shows the majority of cameras in the telecommunications and internet provider verticals. This should not be a surprise. The parts of the graph that peek my interest are the outliers; the groups that have small concentrations. Think of these clusters as the people that go to a party and chill by the wall talking to no one. You kind of wondering what they’re doing and why they are there if they are not going to join in on the fun. Interesting things can be learned from studying the outliers.
- In the above picture all of the yellow arrows are pointing to certain repeated characteristics observed within the various datasets. At least three of the binaries have established communications via TCP port 7547 At least four of the binaries are targeting TCP port 1221 At least three of the binaries have non-standard ICMP traffic (traffic not associated with port unreachable messages) At least one binary has non-IPv4 traffic All of this information can be used to identify affected and potentially vulnerable hosts/IoT devices
- We have narrowed down the binary list from 51 to 6 that can be analyzed for immediate response actions. Packet numbers are identified to uncover domains, ICMP, and non-IPv4 traffic. The established connection packets can be analyzed for payloads and other connect specifics for signature creation.
- Lastly, it is possible to identify product numbers based on content in the index pages. There are also lots of other interesting tidbits that will be covered in the final report.
- Bad Joke: The creators of the malware are secret Trekkies bent on world domination. This is an example of a less than optimal graph of all the port to country relationships. Often simpler graphs yield the most results. But, play around with charts and graphs until one or more are found that help explain your data. In the event a graph is not easily understandable, but essential to your findings, explain to your readers how to interpret the information, and why.