SlideShare uma empresa Scribd logo
1 de 16
Baixar para ler offline
Fuzzing with complexities
Introduction
Now before moving on the specifics of this paper I would introduce a little bit about software testing
and its limitations. I would try to describe fuzzing first before moving on with the case study.

We all have been a witness to major threats in the past years and I guess no one could forget names
like ‘Conficker’ (1), ‘Stuxnet’ (2) and ‘Aurora Project’ (3). All these malware had a unique delivery
system which was based on exploiting the host operating system and further talking control of the
OS for quite talked about purposes. These threats are always there and only thing we expect to
achieve is that, we find vulnerability before a bad guy do and do something about it.

Software companies spend a lot of their time and money in making their product more stable, more
reliable and more secure. But what these incidents prove us that whatever they are doing, there is
always a change to find a bug in a process.

Microsoft has even eliminated the use of several known functions which has historically been proven
to cause buggy code. For example since Vista Microsoft has made sure that functions like strcpy,
sprintf etc. are eliminated at the Software development lifecycle (SDL). Thus normal static analysis
technique used to find the ‘lowest hanging fruit’ would not work if we are working to find bugs in
these windows applications.

In fact all major vendors have realized the importance of having a secure SDL and importance of
testing in their product (which also has cost benefits as compared to releasing and developing
patch). Microsoft and adobe has adopted fuzzing into their SDL cycle thus highlighting the
importance of this practice.




                                   Figure 1: Microsoft Simplified SDL (4)




While referring back to security conferences in the past, significant work has been done in between
2005 – 2006 (5). This period showed that security researchers and hackers are increasing their
efforts to better themselves in the art of fuzzing and finding bugs and producing exploit using this
technique.
Until many vendors were compelled to release their monopoly on popular binary and network
formats by European Union or in some case voluntary disclosure, this field was majorly focused on
Black box testing, meaning the hacker had very little or in some cases no knowledge about the
protocol they are trying to fuzz.

Black-box fuzzing focuses on input format—ignoring the tested software target. While being efficient
and allowing reuse of the same test tools across different tested targets that share the same data
formats, this method misses significant code paths. These code paths depend on configuration
options or specific complex conditions that are governed by application logic. (5)

Even as Microsoft and other vendors do talk up fuzzing seriously, developing multiple protocol
fuzzers for various line of product. There is no doubt that their fuzzer is uber cool and is able to find
most of the bugs, some bugs remains and the security industry is gladly finding them and making
living out of it.

Google and Firefox have a policy of rewarding any researcher who comes up with a bug or a
resulting exploit. Other such rewards on submitting a bug is funded by various IPS companies like
iSight Partners, Tipping point etc. They buy your bug and then work towards a responsible disclosure
while negotiating with the vendor.

In the initial years fuzzing was never looked up by academia as it was considered to be naïve among
the peers, but soon this process gained momentum and attention of many researchers. This simple
technique is proven to produce result faster and cheaper than any other process known. More
information about fuzzing would follow in the following discussion on fuzzing.


Software Testing
Software testing is any activity aimed at evaluating an attribute or capability of a program or system
and determining that it meets its required results (6). Although crucial to software quality and widely
deployed by programmers and testers, software testing still remains an art, due to limited
understanding of the principles of software. The difficulty in software testing stems from the
complexity of software: we cannot completely test a program with moderate complexity. Testing is
more than just debugging. The purpose of testing can be quality assurance, verification and
validation, or reliability estimation. Testing can be used as a generic metric as well. Correctness
testing and reliability testing are two major areas of testing. Software testing is a trade-off between
budget, time and quality. (7)

Unlike most physical systems, most of the defects in software are design errors, not manufacturing
defects. Software does not suffer from corrosion, wear-and-tear -- generally it will not change until
upgrades, or until obsolescence. So once the software is shipped, the design defects -- or bugs -- will
be buried in and remain latent until activation. (7)
With time it was realised that as fixing bug is quite expensive and there is a need to optimize this
process. A huge interest in this field has led to many innovations both from academia and
companies. Many companies completely dedicated in producing software testing techniques, and
automation has grown in recent years. Looking at these innovations it would be unfair to describe
software testing in a few words. My motive here is to introduce this process as a concept if a reader
want more details on this topic I encourage him to read books like “A Practitioner's Guide to
Software Test Design” and others.

Code Coverage
Code coverage is one of the most important metrics used to decide on the completeness of the test
cases. This metrics gives us the relationship between the test conducted and instructions executed
with the application. The more the instructions executed the better the code coverage is. It is the
quantitative measure of instructions executed among the maximum possible instructions with a
particular application.

Of course this metrics can be further broken down into more detailed metrics. Major among them
are: (8)

       Function coverage - Has each function (or subroutine) in the program been called?
       Statement coverage - Has each node in the program been executed?
       Decision coverage - Has every edge in the program been executed? For instance, have the
        requirements of each branch of each control structure (such as in IF and CASE statements)
        been met as well as not met?
       Condition coverage - Has each Boolean sub-expression evaluated both to true and false?
       Condition coverage - Both decision and condition coverage should be satisfied.

Code coverage is also used to discard cases that do have low code coverage or they cannot be used
to increase the code coverage.




                                  Figure 2: An example of Code coverage




There is a specific class of software specifically designed to monitor the code coverage of the
application under observation. Based on type of application we can have either source code
coverage analyser or binary code coverage analyser etc.
It is quite clear that this metric is both intuitive and based on testing properties; give us specific
enough information about the test and its completeness. Some advance code coverage analysis tool
can even give us information that what should we do to improve our test case. But remember it will
only give us a metric of completeness of the test not the metrics of quality or reliability or in any kind
of significance on the application itself.

Also remember that coverage analysis is one of many testing techniques; you should not rely on it
alone.


                                                Test needed to find
                                                       bugs


                                                        Tests
                                                     needed for
                                                      coverage




                   Figure 3: Shows that even on a good coverage some bugs would still be left alone




Blackbox Testing
While testing software, one has only one of three situations in his hand. Firstly, I would have no
knowledge of the inner working of the software, neither of the protocol or kind of input expected,
this situation is rightly named as Black-box Testing. While testing software with no knowledge in
hand would not be very fruitful, but this strategy has worked well in the past. Some of the features
of black box testing are described pictorially below with self-expiatory terms.




                                                          Expected Input

             Black box
              testing                                   Boundary values


                                                           Illegal values



                                            Figure 4: Blackbox Testing
Grey box Testing
The other situation can be that users have knowledge of internal data structure and algorithms for
purpose of designing test cases, but the testing is done at blackbox level. Managing user input and
output are beyond the scope of this kind of testing, this testing condition falls under graybox testing.



White box Testing
Finally, information on internal data structure and algorithms is completely shared between the
product development team and the tester’s team. This kind of information can be used to test API’s,
Code Coverage, fault injection, Mutation of testing and many more. A basic testing setup is
described below.




                                                   Statement Testing

            White box
             testing                                 Decision Testing


                                                  Conditional Testing



                                        Figure 5: Whitebox testing



Fuzzing
Fuzzing is a kind of software testing technique which was until recently categorized inside black box
testing. To put simply, it is the technique in which invalid or mutated or malformed input is supplied
to application with only intention to find bugs the application.

The first person credit of working and formulating this technique is Barton Miller and his students
from University of Wisconsin-Madison in 1989. They described this technique as,

“Fuzz testing is a simple technique for feeding random input to applications. While random is a time-
honoured technique, our approach has three characteristics that, when taken together, make is
somewhat different from other approaches.

    1. Input is random: We do not use any model of program behaviour, application type, or system
       description. This is something called black box testing. In the command line studies, the
       random input was simply random ASCII characters streams. From our x-window study,
       Windows NT study and MAC OS study, the random input included cases that had only valid
       keyboard and mouse events.
2. Our reliability criteria are simple: if the application crashes or hung, it is considered to fail the
       test; otherwise, it passes. Note that the application does not have to respond in a sensible
       manner to the input and it can even quietly exit.

    3. As a result of the first two characteristics, fuzz testing can be automated to a high degree
       and results can be compared across applications, operating systems and vendors.” (9)

Now as we look back at this the early definition of fuzzing, a few things have changed. We
understand this process more clearly, more concrete theory is developed and it is proven to work
time and time again.

It is important to remember that any type of interaction of user supplied data either supplied by
network, file, registry, API etc. can be fuzzed.

Microsoft has described some key terms in fuzzing terms that one should know.

Term             Definition

Dumb fuzzing     Corruption of data packets randomly without awareness of data structure.

Smart fuzzing    Corruption of data packets with awareness of the data structure, such as encodings
                 (for example, base-64 encoding) and relations (checksums, bits indicating the
                 presence of some fields, fields indicating offsets or lengths of other fields).

Black-box        Sending of malformed data without actual verification of which code paths were hit
fuzzing          and which were not.

White-box        Sending of malformed data with verification that all target code paths were hit—
fuzzing          modifying software configuration and the fuzzed data to traverse all data validations
                 in the tested code.

Generation       Generation of fuzzed data automatically—not basing on any previous input.

Mutation         Corruption of valid data according to defect patterns, to produce fuzzed data.

Mutation         Well-formed buffer that represents an equivalence class of the input. The fuzzer
template         takes the mutation template as an input—producing a fuzzed buffer to be sent to
                 the tested software.

Code             Technology (such as that which is bundled in Microsoft Visual Studio 2005) that
coverage         allows inspection of which code paths were executed during testing. This is useful
                 for verification of test effectiveness and improvement of test coverage.

                                        Table 1: Fuzzing Taxonomy (5)

It is observed that fuzzing is most effective against application developed in C/C++, these languages
make the programmer responsible for memory management whereas managed code i.e. developed
in C#, Java etc. would yield bugs of a very different class. This does not mean that you can’t find bugs
in them. (5)
Fuzz metrics have been understood, process is divided into two types of fuzzer and further these
fuzzers have been separated into more focused modules which make them work more efficiently.


Hitting the target application with valid looking inputs would defiantly result in few anomalies. These
anomalies can be introduced into the system any kind of user input as previously discussed.

Some applications like Web Browser uses a lot of open source library and if any one of the libraries is
buggy and exploit can is made out of it, we will see more of the situations like that of ‘Aurora’.

Finally I would like to say that there is an important distinction between Fuzzing and other testing
activity. This distinction is the intent. A testing team knows a lot about the program and basically
test that whether a program is behaving as it is supposed to behave where as a security researcher
only care that his fuzzer crashes your tested application. Security researchers do not care about the
internal working of the program, he has one and only one purpose i.e. to crash the program or make
it behave differently while playing with it.

Fuzzer
There have a plethora of fuzzers available in market, both open source and closed.

I would like to make note of two python based fuzzing framework available in the open source
community that I use most extensively.

       PeachFuzzer - Peach is a SmartFuzzer that is capable of performing both generation and
        mutation based fuzzing (10).
       Sulley - Sulley is a fuzzer development and fuzz testing framework consisting of multiple
        extensible components. Sulley (IMHO) exceeds the capabilities of most previously published
        fuzzing technologies, commercial and public domain (11).

Peach is been improved day in and day out and it is the only other open source fuzzer that is
maintained apart from Metasploit fuzzer. Peach is written as primary data fuzzer, but as it open
source it can be extended to secondary and even nth-class fuzzer. Peach fuzzer is also used by adobe
in its testing of Adobe reader (12).

Sulley is not maintained but is as good as you can get when it comes to generation based fuzzing.

These fuzzers have separated the data generation or mutation process from process monitoring
technique which is again separated from control and even support distributed fuzzing (where a
fuzzer run on multiple machine). These are some of the reasons why these fuzzers are worth
mentioning.

A major commercial Fuzzer is Codenomicon (13). This is the child project of the very first fuzzer ever
developed called PROTOS.

If you want to check out other fuzzers available I recommend you to check this
http://packetstormsecurity.org/fuzzer/.
The Complexity
Jiantao Pan of Carnegie Mellon University has described the complexity revolving around bugs as (7)

“Software bugs will almost always exist in any software module with moderate size: not because
programmers are careless or irresponsible, but because the complexity of software is generally
intractable -- and humans have only limited ability to manage complexity. It is also true that for any
complex systems, design defects can never be completely ruled out.”

He also describes about the never ending race among security researchers who are finding bugs and
that of vendors and uses the terms, used by Boris Beizer in his book Software Testing Techniques, 2nd
Edition, Pesticide paradox to describe this race.

Pesticide Paradox – “Every method you use to prevent or find bugs leaves a residue of subtler bugs
against which those methods are ineffectual” (14).

In simple terms on can say that once you have found a bug and fixed it, you are now sure that the
input that earlier did trigger bug would not be able to crash the program. But you can be never sure
that pre-error test cases that the application passed in the first time would not come haunting.

In many of the fuzzer it is observed that test cases produced fails to achieve the basic packet
sanitation test, which prevents it from further testing. In all general purposes the system rejects
virtually all the randomly generated packets, thus defeating the purpose of testing the remaining
code effectively.

What this means is that fuzzers with improper understanding of the input type and structure would
fails often then they succeed. Intuitively the one with more knowledge about the input structure
would result in better fuzzers.

A study done by Microsoft on a 450 lines of code and then testing it with various fuzz combinations
to see the effective results that was produced is shown below :

Technique                                                Effort        Code coverage              Defects found

Black box + Dumb                                         10 min        50%                        25%

White box + Dumb                                         30 min        80%                        50%

Black box + Smart                                        2 hr          80%                        50%

White box + Smart                                        2.5 hr        99%                        100%

       Table 2: Analysis based on Effort in producing fuzzer and defects found correlated with kind of fuzzer (5)

Know what I believe, and the test shows that investing your time in right place would result in better
productivity with respect to kind of bugs we found and also the amount of bugs we found, further
restricting on the number of cases where input is out rightly rejected on the basis of basic sanitation
techniques of the programs input validation module.

If we want to go deep inside the program semantics and actually test the code of our choice or track
the data input we have no choice but to work with all the knowledge we have or could acquire about
the protocol. This involves reading and understanding of Data Format documentation of disclosed
file formats, network formats and many more. We can easily right such fuzzer that generates
credible input to the tested application if the stick to either an ASCII based or Binary based.

WHO #UnrealIRCd
JOIN #Support
PONG irc.fyremoon.net irc2.dynam.ac
STATS u
                            Table 3: Example of a pure Ascii Based Data format

ZIP local file header

Offset        Bytes      Description[5]

 0            4          Local file header signature = 0x04034b50

 4            2          Version needed to extract (minimum)

 6            2          General purpose bit flag

 8            2          Compression method

10            2          File last modification time

12            2          File last modification date

14            4          CRC-32

18            4          Compressed size

22            4          Uncompressed size

26            2          File name length (n)

28            2          Extra field length (m)

30            n          File name

30+n          m          Extra field

                          Table 4: An example of purely binary based data format

But what happens when the formats no longer sticks to one data format? What happens when our
data switches from one set of data format like ASCII to binary and then binary to ASCII again and to
add a cherry on top sections are encoded differently even the ascii portion can be encoded and even
imported from other binary or ASCII based formats. If you want more complication here is one more
to your bag, Can the code coverage be a correct metrics when analysing fuzzing input? The answer is
No, you would be lucky to even get 25% code coverage for any given input, this is because of high
modular structure of these programs, distributed among various binaries and not all binary load at a
given input.
Things are getting pretty complicated. Hmm!

%PDF-1.5
%µµµµ
1 0 obj
<</Type/Catalog/Pages 2 0 R/Lang(en-IN) /StructTreeRoot 54 0 R/MarkInfo<</Marked true>>>>
endobj
2 0 obj
<</Type/Pages/Count 7/Kids[ 3 0 R 23 0 R 29 0 R 38 0 R 47 0 R 49 0 R 51 0 R] >>
endobj
3 0 obj
<</Type/Page/Parent 2 0 R/Resources<</Font<</F1 5 0 R/F2 10 0 R/F3 12 0 R/F4 14 0 R/F5 16 0
R/F6 21 0 R>>/ProcSet[/PDF/Text/ImageB/ImageC/ImageI] >>/MediaBox[ 0 0 595.32 841.92]
/Contents 4 0 R/Group<</Type/Group/S/Transparency/CS/DeviceRGB>>/Tabs/S/StructParents
0>>
endobj
4 0 obj
<</Filter/FlateDecode/Length 5463>>
stream
xœ•[•Û¸’~•ÿ Gû
­ˆÝ€A                              I&9ÈbgÏì&»ó0³jµÜÖÄq÷Hêtz~ýÖ…”HY”ÅÁœ´e²X$ëúUÉÑë_£
Ÿ~zýË»•?GÉõuôöçwÑÛÏ/_¼þ                                            "™Å‰Ž>ï^¾QÿQ.ãDê(×i,Šèó×—
/’8)%±øÿ*K£înññ•ýóå‹ß•J’¤¼ÎáŸ2¿¾*ñßwþ¿ÅÛë+ú>¹¾*ðsÉŸeÆ•æ¹—ñ¿ÅþÞŽ+Òë+‘à
                          Table 5: Example of one such complex formats ie. PDF

We see these being used in every day applications like office documents, Adobe PDF, SMB protocols
and more. One cannot try to randomly fuzz these files as they have pretty good input validation
modules which prevent any dumb attempt to fuzz them. All these formats are considered as very
attractive formats within the security industry. But understanding even one format completely
would take some beautiful years out of once life. So is there a solution? My answer is yes there
“are” few solutions available still.




                                   Figure 6: Adobe Fuzzing Technique
What we have gathered until here is summarized here as we move ahead you will find answers to
these problems.

Complications
Code coverage is cannot be used as a metrics while testing test type of input streams
Data format consistency i.e. Either ASCII or Binary is not maintained
Multiple encoded levels exists which can be different within the same file
Multiple data format either binary or ASCII are embedded within these input stream
                                       Table 6: Discussed problems

Code Coverage fails for these applications
In all these cases we have to work with what we have in hand. We are going to use a combination of
the technique of taint analysis, protocol awareness and Code coverage. I call this as relative code
coverage metric.

Protocol awareness can be used as once we have all the information of a protocol that we could
have, we can intuitively say that the packet which contains the most number of tags or objects
would require more code to be covered with that module. Now this could be said that we cannot
guarantee the code coverage still because if we do not find a packet that contains all the tags or
object that our application which is usually the case, we would be not able to test all cases.

Testing all cases in one go was never the idea but multiple tests covering every tag is what will be
fruitful. It is intuitive that when all the tags are covered we would have near complete code
coverage. To further improve upon, one can use Taint analysis to find all the possible check and
balances and the further incorporate them in code. Read the ‘future’ section for more details.

Data format inconsistency
The other problem that we face is of data format inconsistency. One can easy write a fuzzer of either
and ASCII based packet or for binary based packet. But when these formats get together in a packet,
it becomes unnaturally difficult to write one.

The solution lays in visualizing and breaking problem in parts which we most comfortable in. We can
use the separate out the data generation capability from both ASCII and Binary format. Remember
here I have trying to separate out these capabilities not necessarily for fuzzing.

Multiple Encoding levels
As we have separated ASCII with Binary in the same format one can further add custom encoding in
each packet as one like. They will all fall back together when we combine them later. See the case
study for more clarification.

In a PDF file if we have a multiple font embedded inside the file we can make use of different
encoders for each such font as each is generated separately.

Multiple Files Embedded in a single packets
With separating of types we can further separate to a secondary level data production module.
What this means is that if a PDF file if we have a font and image embedded inside the file we can
actually write a different fuzzer for font and for an image and combines each of these result with the
PDF files in the manner similar to multiple encoding level problem.
Now is the right time to talk about the strategy that I have used when fuzzing one such format, PDF.
You will find different definition of these terms, but this is what I understand out of them. This
process is typically described in the terms of system under test and called for directed area with the
system, where as in my study I have taken it out of box and placed these conditions on Data packet
itself.



Attack Point Selection
The attach point selection is a simple process in which I have tried to specify a specific point within
the packet which needs to be tested. Now selection of these points depends a lot upon some
gathered intelligence of the system, including pervious vulnerabilities. As this eliminates a few attack
point as they have already been attacked before. For example if working on a simple PDF file which
contain a U3D file which is known to previously cause a vulnerability in Adobe reader one can say
this format is previously been tested primarily (after looking at the vulnerability) so a lot more
efforts would be required in finding a vulnerability next time. One can focus his time and energy in
finding other routes into the application which has still not been tested by security researchers.

Directed Fuzzing
Whenever a vulnerability is released it is released with a very few information. One such disclosure
example would be.

  Adobe Flash Player Multiple Tag JPEG Parsing Remote Code Execution Vulnerability

 -- Vulnerability Details:

 This vulnerability allows remote attackers to execute arbitrary code on vulnerable installations of
 Adobe Flash Player. User interaction is required in that a target must visit a malicious website.
 The specific flaw exists within the code for parsing embedded image data within SWF files. The
 DefineBits tag and several of its variations are prone to a parsing issue while handling JPEG data.
 Specifically, the vulnerability is due to decompression routines that do not validate image
 dimensions sufficiently before performing operations on heap memory. An attacker can exploit
 this vulnerability to execute arbitrary code under the context of the user running the browser.


                               Figure 7: An example of Vulnerability disclosure

Now you would like to get your hands on this kind of vulnerability. Getting this vulnerability is very
difficult task as we have only a specific area with a binary file to be fuzzed and with extensive inputs
mutation / generations. Looking at the above example here, of the kind of disclosure available we
would like to fuzz only a specific region as we have previously been aware that certain code might
contain some vulnerability somewhere.


Case Study: Attach Point selection
I have implemented a fuzzer using the Peach fuzzing framework, where I have decided to test PDF
files which have Open Type font embedded in them. Now I fuzz only the font part within the PDF by
using a secondary fuzzer which contains the complete file format of TTF.
I have done mutated fuzzing, so I had to choose a file that contains most of the tags (to achieve the
maximum relative code coverage) that are there in documentation. This can be easily done if we
have a good base of fonts already in our repository that was easy to get using torrents. I could not
find a single file that contains all the tags so I have to go for the one which contains the maximum
number of tags. I had few other files that contain tags that were not there in the maximum tag file,
thus my test case is complete with all the tags that were allowed in the font file.

This TTF font is arranged within PDF using Report Lab python API. Once the PDF is formed it is open
up inside Adobe Reader to which debugger is attached. There are few other libraries available for
you to actually merge the components with PDF some notable among them is iText and pyPdf. Even
if you do not have one such library available it is easy to embed on file with simple scripts one such
example is there on Feliam’s blog (15).




                     Figure 8: Data format analysis by 010 Editors custom template of TTF
After making this custom template and getting a thorough understanding of TTF file format I wrote a
fuzzer for it.




                                 Figure 9: Snippet of fuzzer for TTF in Peach PIT format

After running this fuzzer for just one day I had the following result.

   Microsoft (R) Windows Debugger Version 6.11.0001.404 X86
   Copyright (c) Microsoft Corporation. All rights reserved.
   ……
   CommandLine: C:PROGRA~1AdobeReader~1.0ReaderAcroRd32.exe C:FuzzFileFileFormatTTFfuzzedfile.pdf
   Symbol search path is:
   SRV*http://msdl.microsoft.com/download/symbols;symsrv*symsrv.dll*c:symbols*http://msdl.microsoft.com/download/symbol
   s
   Executable search path is:
   ModLoad: 00400000 00454000 AcroRd32Exe.exe
   ModLoad: 7c900000 7c9b0000 ntdll.dll
   ………..
   ………..
   (2e4.674): Access violation - code c0000005 (first chance)
   r
   eax=00000001 ebx=07d52594 ecx=04dd57ec edx=00000002 esi=04dd56ac edi=07dfbfe4
   eip=080111e1 esp=0012de48 ebp=0012de5c iopl=0             nv up ei pl nz na po nc
   cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000                efl=00010202
   *** ERROR: Symbol file could not be found. Defaulted to export symbols for
   C:PROGRA~1AdobeReader~1.0ReaderCoolType.dll -
   CoolType+0x111e1:
   080111e1 660fb64f23 movzx cx,byte ptr [edi+23h] ds:0023:07dfc007=??

                                              Figure 10: Result from fuzzing



And the following information were logged
Figure 11: Fuzzing Results were logged

For office application there are actually two ways in which you can do Fuzzing using this concept.
First you can use an Library to create office files embedded with test case of your choice (some
libraries test the cases sanctity before embedding, one can bypass this by changing the sanctity
checking code and recompiling the library. To know which section of code is doing that you can
always fall back to code coverage) and the other way would be using COM API of these Office
software. By using the second method you can test 2 things. First, is there a bug in loading resource
process and secondly is there a bug in processing the resource file.


Case Study: Directed Fuzzing
Once we are familiar with the process of creating files we can further brings thing down to tag level.
This means that if we want to only fuzz one particular tag in a file which would be further embedded
into some complicated data format, we just have to model that particular tag of the file in our
mutator and the rest of the procedure remain the same. For example, in the example given earlier if
one wants to fuzz DefineBits tag in JPEG file which would be embedded into a SWF file they have to
only write a mutator for this particular tag i.e. DefineBits Tag. Rest of the procedure remains the
same.

In my study I will reuse my above code developed in directed fuzzing and will fuzz only a particular
tag which is maxp in case of TTF fonts which would be further embedded into the PDF file.


Conclusion
With new functionalities being added into application with every release, new modules being added
and legacy code left few holes unplugged. One has the opportunity to find these bugs given a refined
and detailed process to fuzz. More knowledge of the protocol always yields into better fuzzers and
more bugs.


Bibliography
1. Conficker. Wikipedia. [Online] http://en.wikipedia.org/wiki/Conficker.

2. Stuxnet. Wikipedia. [Online] http://en.wikipedia.org/wiki/Stuxnet.

3. Operation Aurora. Wikipedia. [Online] http://en.wikipedia.org/wiki/Operation_Aurora.

4. SDL. Microsoft . [Online] http://www.microsoft.com/security/sdl/discover/default.aspx.
5. John Neystadt. Automated Penetration Testing with White-Box Fuzzing. Microsoft. [Online]
Microsoft Corporation, February 2008. http://msdn.microsoft.com/en-us/library/cc162782.aspx.

6. Hetzel, William C. The Complete Guide to Software Testing, 2nd ed. s.l. : Wellesley, Mass.
0894352423.

7. Jiantao Pan. Software Testing. Carnegie Mellon                     University.     [Online]     1999.
http://www.ece.cmu.edu/~koopman/des_s99/sw_testing/.

8. Code coverage. Wikipedia. [Online] http://en.wikipedia.org/wiki/Code_coverage.

9. Noam Rathaus, Gadi Evron. Fuzzing - What's that? Open source fuzzing tools. s.l. : Syngress, 2007.

10. Michael Eddington. Peach Fuzzing Platform. Peach Fuzzer. [Online] Deja Vu Security.
http://peachfuzzer.com/.

11. Pedram Amini. Sulley. Google Code. [Online] http://code.google.com/p/sulley/.

12. Kyle Randolph. Fuzzing Reader - Lessons learned. Adobe. [Online] Adobe Systems Incorporated,
1 December 2009. http://blogs.adobe.com/asset/2009/12/fuzzing_reader_-_lessons_learned.html.

13. Codenomicon. [Online] http://www.codenomicon.com/.

14. Kaul, Nihit. Pesticide Paradox. Blogs MSDN. [Online] Microsoft Corporation, Jul 2004.
http://blogs.msdn.com/b/nihitk/archive/2004/07/16/185836.aspx.

15. Taint-based Directed Whitebox Fuzzing. V. Ganesh, T. Leek, and M. Rinard. s.l. : Proceedings of
the 2009 IEEE 31st International Conference on Software Engineering, 2009.

16. Fuzz Testing. Wikipedia. [Online] http://en.wikipedia.org/wiki/Fuzzing.

17. The buzz about fuzzer. Info World. [Online] http://www.infoworld.com/d/developer-world/buzz-
about-fuzzers-089.

18. Software Testing. Wikipedia. [Online] http://en.wikipedia.org/wiki/Software_testing.

19. Quality Assurance. Wikipedia. [Online] http://en.wikipedia.org/wiki/Quality_assurance.

20. SDL. Wikipedia. [Online] http://en.wikipedia.org/wiki/Software_development_life_cycle.

21.         Paradox          of        the         pesticides.           Wikipedia.              [Online]
http://en.wikipedia.org/wiki/Paradox_of_the_pesticides.

22. Grammar-based Whitebox Fuzzing. Patrice Godefroid, Adam Kiezun, and Michael Y. Levin. s.l. :
Microsoft Research, 2007. MSR-TR-2007-154.

Mais conteúdo relacionado

Mais procurados

Manuel testing word
Manuel testing wordManuel testing word
Manuel testing wordVijay R
 
Testing survey by_directions
Testing survey by_directionsTesting survey by_directions
Testing survey by_directionsTao He
 
Defect effort prediction models in software
Defect effort prediction models in softwareDefect effort prediction models in software
Defect effort prediction models in softwareIAEME Publication
 
Testing terms & definitions
Testing terms & definitionsTesting terms & definitions
Testing terms & definitionsSachin MK
 
Đề thi mẫu 1(ISTQB)
Đề thi mẫu 1(ISTQB)Đề thi mẫu 1(ISTQB)
Đề thi mẫu 1(ISTQB)Jenny Nguyen
 
Manual testing real time questions by subbu
Manual testing real time questions by subbuManual testing real time questions by subbu
Manual testing real time questions by subbupalla subrahmanyam
 
Unit testing & TDD concepts with best practice guidelines.
Unit testing & TDD concepts with best practice guidelines.Unit testing & TDD concepts with best practice guidelines.
Unit testing & TDD concepts with best practice guidelines.Mohamed Taman
 
30 testing interview questions for experienced
30 testing interview questions for experienced30 testing interview questions for experienced
30 testing interview questions for experienceddilipambhore
 
Software testing for biginners
Software testing for biginnersSoftware testing for biginners
Software testing for biginnersSriman Eshwar
 
Unit Testing to Support Reusable for Component-Based Software Engineering
Unit Testing to Support Reusable for Component-Based Software EngineeringUnit Testing to Support Reusable for Component-Based Software Engineering
Unit Testing to Support Reusable for Component-Based Software Engineeringijtsrd
 
Testing a GPS application | Testbytes
Testing a GPS application | TestbytesTesting a GPS application | Testbytes
Testing a GPS application | TestbytesTestbytes
 
QUALITY METRICS OF TEST SUITES IN TESTDRIVEN DESIGNED APPLICATIONS
QUALITY METRICS OF TEST SUITES IN TESTDRIVEN DESIGNED APPLICATIONSQUALITY METRICS OF TEST SUITES IN TESTDRIVEN DESIGNED APPLICATIONS
QUALITY METRICS OF TEST SUITES IN TESTDRIVEN DESIGNED APPLICATIONSijseajournal
 
Automation frameworks
Automation frameworksAutomation frameworks
Automation frameworksVishwanath KC
 
Unit testing - what is its importance
Unit testing - what is its importanceUnit testing - what is its importance
Unit testing - what is its importanceTestingXperts
 

Mais procurados (17)

Manuel testing word
Manuel testing wordManuel testing word
Manuel testing word
 
Testing survey by_directions
Testing survey by_directionsTesting survey by_directions
Testing survey by_directions
 
Defect effort prediction models in software
Defect effort prediction models in softwareDefect effort prediction models in software
Defect effort prediction models in software
 
Testing terms & definitions
Testing terms & definitionsTesting terms & definitions
Testing terms & definitions
 
Block 1 ms-034 unit-1
Block 1 ms-034 unit-1Block 1 ms-034 unit-1
Block 1 ms-034 unit-1
 
Đề thi mẫu 1(ISTQB)
Đề thi mẫu 1(ISTQB)Đề thi mẫu 1(ISTQB)
Đề thi mẫu 1(ISTQB)
 
Manual testing real time questions by subbu
Manual testing real time questions by subbuManual testing real time questions by subbu
Manual testing real time questions by subbu
 
Unit testing & TDD concepts with best practice guidelines.
Unit testing & TDD concepts with best practice guidelines.Unit testing & TDD concepts with best practice guidelines.
Unit testing & TDD concepts with best practice guidelines.
 
TestDrivenDeveloment
TestDrivenDevelomentTestDrivenDeveloment
TestDrivenDeveloment
 
30 testing interview questions for experienced
30 testing interview questions for experienced30 testing interview questions for experienced
30 testing interview questions for experienced
 
Software testing for biginners
Software testing for biginnersSoftware testing for biginners
Software testing for biginners
 
Ensuring code quality
Ensuring code qualityEnsuring code quality
Ensuring code quality
 
Unit Testing to Support Reusable for Component-Based Software Engineering
Unit Testing to Support Reusable for Component-Based Software EngineeringUnit Testing to Support Reusable for Component-Based Software Engineering
Unit Testing to Support Reusable for Component-Based Software Engineering
 
Testing a GPS application | Testbytes
Testing a GPS application | TestbytesTesting a GPS application | Testbytes
Testing a GPS application | Testbytes
 
QUALITY METRICS OF TEST SUITES IN TESTDRIVEN DESIGNED APPLICATIONS
QUALITY METRICS OF TEST SUITES IN TESTDRIVEN DESIGNED APPLICATIONSQUALITY METRICS OF TEST SUITES IN TESTDRIVEN DESIGNED APPLICATIONS
QUALITY METRICS OF TEST SUITES IN TESTDRIVEN DESIGNED APPLICATIONS
 
Automation frameworks
Automation frameworksAutomation frameworks
Automation frameworks
 
Unit testing - what is its importance
Unit testing - what is its importanceUnit testing - what is its importance
Unit testing - what is its importance
 

Destaque

Memoria anual Escuela Mpal. de Fútbol de Chipiona Temp.11 12
Memoria anual Escuela Mpal. de Fútbol de Chipiona Temp.11 12Memoria anual Escuela Mpal. de Fútbol de Chipiona Temp.11 12
Memoria anual Escuela Mpal. de Fútbol de Chipiona Temp.11 12escuelafutbolchipiona
 
Badge Pass Brochures
Badge Pass BrochuresBadge Pass Brochures
Badge Pass Brochuresdfspratlin
 
SMi Group's 20th annual European Pharmaceutical Pricing & Reimbursement confe...
SMi Group's 20th annual European Pharmaceutical Pricing & Reimbursement confe...SMi Group's 20th annual European Pharmaceutical Pricing & Reimbursement confe...
SMi Group's 20th annual European Pharmaceutical Pricing & Reimbursement confe...Dale Butler
 
Dossier pedago casa_bernarda_4e-3e_def2-2
Dossier pedago casa_bernarda_4e-3e_def2-2Dossier pedago casa_bernarda_4e-3e_def2-2
Dossier pedago casa_bernarda_4e-3e_def2-2Akkioui01
 
Red Ant: Digital Strategy Whitepaper 2011
Red Ant: Digital Strategy Whitepaper 2011Red Ant: Digital Strategy Whitepaper 2011
Red Ant: Digital Strategy Whitepaper 2011Brian Crotty
 
Publicaciones libro 02 actas ii seminario docente 2006 varios autores
Publicaciones libro 02 actas ii seminario docente 2006 varios autoresPublicaciones libro 02 actas ii seminario docente 2006 varios autores
Publicaciones libro 02 actas ii seminario docente 2006 varios autoresLiteratura y Tradición
 
KURIER & OMV :: Education Needs Innovation (“Bildung Braucht Innovation”)
KURIER & OMV :: Education Needs Innovation (“Bildung Braucht Innovation”)KURIER & OMV :: Education Needs Innovation (“Bildung Braucht Innovation”)
KURIER & OMV :: Education Needs Innovation (“Bildung Braucht Innovation”)George Nimeh
 
Sales Strategy: 27 Sales Campaigns to Scale Revenue - Ken Krogue
Sales Strategy: 27 Sales Campaigns to Scale Revenue - Ken KrogueSales Strategy: 27 Sales Campaigns to Scale Revenue - Ken Krogue
Sales Strategy: 27 Sales Campaigns to Scale Revenue - Ken KrogueInsideSales.com
 
Las minas de mercurio de almadén
Las minas de mercurio de almadénLas minas de mercurio de almadén
Las minas de mercurio de almadénYoblanso
 
Crcs short_brochure_revision monroe
Crcs  short_brochure_revision monroeCrcs  short_brochure_revision monroe
Crcs short_brochure_revision monroeEugenio Aburto AIA-E
 
Dossier (historias reales) Engrata
Dossier (historias reales) EngrataDossier (historias reales) Engrata
Dossier (historias reales) EngrataJose Angel Gomis
 
Communication part consists of client module SCALANCE W747-1RR.
Communication part consists of client module SCALANCE W747-1RR.Communication part consists of client module SCALANCE W747-1RR.
Communication part consists of client module SCALANCE W747-1RR.James George
 
Dossier del programa de Formación en Supervisión y Coaching Sistémicos 2014
Dossier del programa de Formación en Supervisión y Coaching Sistémicos 2014Dossier del programa de Formación en Supervisión y Coaching Sistémicos 2014
Dossier del programa de Formación en Supervisión y Coaching Sistémicos 2014Nestor Sangroniz Akarregi
 

Destaque (20)

Die Welt von Falkensteiner – Therme & Golf Hotel Bad Waltersdorf
Die Welt von Falkensteiner – Therme & Golf Hotel Bad WaltersdorfDie Welt von Falkensteiner – Therme & Golf Hotel Bad Waltersdorf
Die Welt von Falkensteiner – Therme & Golf Hotel Bad Waltersdorf
 
Memoria anual Escuela Mpal. de Fútbol de Chipiona Temp.11 12
Memoria anual Escuela Mpal. de Fútbol de Chipiona Temp.11 12Memoria anual Escuela Mpal. de Fútbol de Chipiona Temp.11 12
Memoria anual Escuela Mpal. de Fútbol de Chipiona Temp.11 12
 
Badge Pass Brochures
Badge Pass BrochuresBadge Pass Brochures
Badge Pass Brochures
 
SMi Group's 20th annual European Pharmaceutical Pricing & Reimbursement confe...
SMi Group's 20th annual European Pharmaceutical Pricing & Reimbursement confe...SMi Group's 20th annual European Pharmaceutical Pricing & Reimbursement confe...
SMi Group's 20th annual European Pharmaceutical Pricing & Reimbursement confe...
 
Neotion
NeotionNeotion
Neotion
 
Dossier pedago casa_bernarda_4e-3e_def2-2
Dossier pedago casa_bernarda_4e-3e_def2-2Dossier pedago casa_bernarda_4e-3e_def2-2
Dossier pedago casa_bernarda_4e-3e_def2-2
 
Red Ant: Digital Strategy Whitepaper 2011
Red Ant: Digital Strategy Whitepaper 2011Red Ant: Digital Strategy Whitepaper 2011
Red Ant: Digital Strategy Whitepaper 2011
 
Publicaciones libro 02 actas ii seminario docente 2006 varios autores
Publicaciones libro 02 actas ii seminario docente 2006 varios autoresPublicaciones libro 02 actas ii seminario docente 2006 varios autores
Publicaciones libro 02 actas ii seminario docente 2006 varios autores
 
KURIER & OMV :: Education Needs Innovation (“Bildung Braucht Innovation”)
KURIER & OMV :: Education Needs Innovation (“Bildung Braucht Innovation”)KURIER & OMV :: Education Needs Innovation (“Bildung Braucht Innovation”)
KURIER & OMV :: Education Needs Innovation (“Bildung Braucht Innovation”)
 
Sales Strategy: 27 Sales Campaigns to Scale Revenue - Ken Krogue
Sales Strategy: 27 Sales Campaigns to Scale Revenue - Ken KrogueSales Strategy: 27 Sales Campaigns to Scale Revenue - Ken Krogue
Sales Strategy: 27 Sales Campaigns to Scale Revenue - Ken Krogue
 
Ingles (15) copia
Ingles (15)   copiaIngles (15)   copia
Ingles (15) copia
 
Tubo de rayos x
 Tubo de rayos x Tubo de rayos x
Tubo de rayos x
 
I twin technology
I twin technologyI twin technology
I twin technology
 
Arte realismo 2
Arte realismo 2Arte realismo 2
Arte realismo 2
 
Las minas de mercurio de almadén
Las minas de mercurio de almadénLas minas de mercurio de almadén
Las minas de mercurio de almadén
 
Crcs short_brochure_revision monroe
Crcs  short_brochure_revision monroeCrcs  short_brochure_revision monroe
Crcs short_brochure_revision monroe
 
Dossier (historias reales) Engrata
Dossier (historias reales) EngrataDossier (historias reales) Engrata
Dossier (historias reales) Engrata
 
Reassignment committee meeting - June 25, 201 june 25
Reassignment committee meeting - June 25, 201  june 25Reassignment committee meeting - June 25, 201  june 25
Reassignment committee meeting - June 25, 201 june 25
 
Communication part consists of client module SCALANCE W747-1RR.
Communication part consists of client module SCALANCE W747-1RR.Communication part consists of client module SCALANCE W747-1RR.
Communication part consists of client module SCALANCE W747-1RR.
 
Dossier del programa de Formación en Supervisión y Coaching Sistémicos 2014
Dossier del programa de Formación en Supervisión y Coaching Sistémicos 2014Dossier del programa de Formación en Supervisión y Coaching Sistémicos 2014
Dossier del programa de Formación en Supervisión y Coaching Sistémicos 2014
 

Semelhante a nullcon 2011 - Fuzzing with Complexities

Semelhante a nullcon 2011 - Fuzzing with Complexities (20)

Software testing methods
Software testing methodsSoftware testing methods
Software testing methods
 
stm f.pdf
stm f.pdfstm f.pdf
stm f.pdf
 
Software testing
Software testing   Software testing
Software testing
 
Testing fundamentals
Testing fundamentalsTesting fundamentals
Testing fundamentals
 
unittesting-190620114546 (1).pptx document
unittesting-190620114546 (1).pptx documentunittesting-190620114546 (1).pptx document
unittesting-190620114546 (1).pptx document
 
Testing
Testing Testing
Testing
 
Software coding and testing
Software coding and testingSoftware coding and testing
Software coding and testing
 
Software testing strategies
Software testing strategiesSoftware testing strategies
Software testing strategies
 
Software testing
Software testingSoftware testing
Software testing
 
STM-UNIT-1.pptx
STM-UNIT-1.pptxSTM-UNIT-1.pptx
STM-UNIT-1.pptx
 
Software Testing Principal
Software Testing PrincipalSoftware Testing Principal
Software Testing Principal
 
Software techniques
Software techniquesSoftware techniques
Software techniques
 
Why Software Testing is Crucial in Software Development_.pdf
Why Software Testing is Crucial in Software Development_.pdfWhy Software Testing is Crucial in Software Development_.pdf
Why Software Testing is Crucial in Software Development_.pdf
 
EFFECTIVE TEST CASE DESING: A REVIEW
EFFECTIVE TEST CASE DESING: A REVIEWEFFECTIVE TEST CASE DESING: A REVIEW
EFFECTIVE TEST CASE DESING: A REVIEW
 
A COMPOSITION ON SOFTWARE TESTING
A COMPOSITION ON SOFTWARE TESTINGA COMPOSITION ON SOFTWARE TESTING
A COMPOSITION ON SOFTWARE TESTING
 
Software engg unit 4
Software engg unit 4 Software engg unit 4
Software engg unit 4
 
Testing concepts
Testing conceptsTesting concepts
Testing concepts
 
Software_testing Unit 1 bca V.pdf
Software_testing Unit 1 bca V.pdfSoftware_testing Unit 1 bca V.pdf
Software_testing Unit 1 bca V.pdf
 
Software testing
Software testingSoftware testing
Software testing
 
Why is software testing important
Why is software testing important Why is software testing important
Why is software testing important
 

Mais de n|u - The Open Security Community

Gibson 101 -quick_introduction_to_hacking_mainframes_in_2020_null_infosec_gir...
Gibson 101 -quick_introduction_to_hacking_mainframes_in_2020_null_infosec_gir...Gibson 101 -quick_introduction_to_hacking_mainframes_in_2020_null_infosec_gir...
Gibson 101 -quick_introduction_to_hacking_mainframes_in_2020_null_infosec_gir...n|u - The Open Security Community
 

Mais de n|u - The Open Security Community (20)

Hardware security testing 101 (Null - Delhi Chapter)
Hardware security testing 101 (Null - Delhi Chapter)Hardware security testing 101 (Null - Delhi Chapter)
Hardware security testing 101 (Null - Delhi Chapter)
 
Osint primer
Osint primerOsint primer
Osint primer
 
SSRF exploit the trust relationship
SSRF exploit the trust relationshipSSRF exploit the trust relationship
SSRF exploit the trust relationship
 
Nmap basics
Nmap basicsNmap basics
Nmap basics
 
Metasploit primary
Metasploit primaryMetasploit primary
Metasploit primary
 
Api security-testing
Api security-testingApi security-testing
Api security-testing
 
Introduction to TLS 1.3
Introduction to TLS 1.3Introduction to TLS 1.3
Introduction to TLS 1.3
 
Gibson 101 -quick_introduction_to_hacking_mainframes_in_2020_null_infosec_gir...
Gibson 101 -quick_introduction_to_hacking_mainframes_in_2020_null_infosec_gir...Gibson 101 -quick_introduction_to_hacking_mainframes_in_2020_null_infosec_gir...
Gibson 101 -quick_introduction_to_hacking_mainframes_in_2020_null_infosec_gir...
 
Talking About SSRF,CRLF
Talking About SSRF,CRLFTalking About SSRF,CRLF
Talking About SSRF,CRLF
 
Building active directory lab for red teaming
Building active directory lab for red teamingBuilding active directory lab for red teaming
Building active directory lab for red teaming
 
Owning a company through their logs
Owning a company through their logsOwning a company through their logs
Owning a company through their logs
 
Introduction to shodan
Introduction to shodanIntroduction to shodan
Introduction to shodan
 
Cloud security
Cloud security Cloud security
Cloud security
 
Detecting persistence in windows
Detecting persistence in windowsDetecting persistence in windows
Detecting persistence in windows
 
Frida - Objection Tool Usage
Frida - Objection Tool UsageFrida - Objection Tool Usage
Frida - Objection Tool Usage
 
OSQuery - Monitoring System Process
OSQuery - Monitoring System ProcessOSQuery - Monitoring System Process
OSQuery - Monitoring System Process
 
DevSecOps Jenkins Pipeline -Security
DevSecOps Jenkins Pipeline -SecurityDevSecOps Jenkins Pipeline -Security
DevSecOps Jenkins Pipeline -Security
 
Extensible markup language attacks
Extensible markup language attacksExtensible markup language attacks
Extensible markup language attacks
 
Linux for hackers
Linux for hackersLinux for hackers
Linux for hackers
 
Android Pentesting
Android PentestingAndroid Pentesting
Android Pentesting
 

Último

Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Paola De la Torre
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...gurkirankumar98700
 

Último (20)

Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
 

nullcon 2011 - Fuzzing with Complexities

  • 1. Fuzzing with complexities Introduction Now before moving on the specifics of this paper I would introduce a little bit about software testing and its limitations. I would try to describe fuzzing first before moving on with the case study. We all have been a witness to major threats in the past years and I guess no one could forget names like ‘Conficker’ (1), ‘Stuxnet’ (2) and ‘Aurora Project’ (3). All these malware had a unique delivery system which was based on exploiting the host operating system and further talking control of the OS for quite talked about purposes. These threats are always there and only thing we expect to achieve is that, we find vulnerability before a bad guy do and do something about it. Software companies spend a lot of their time and money in making their product more stable, more reliable and more secure. But what these incidents prove us that whatever they are doing, there is always a change to find a bug in a process. Microsoft has even eliminated the use of several known functions which has historically been proven to cause buggy code. For example since Vista Microsoft has made sure that functions like strcpy, sprintf etc. are eliminated at the Software development lifecycle (SDL). Thus normal static analysis technique used to find the ‘lowest hanging fruit’ would not work if we are working to find bugs in these windows applications. In fact all major vendors have realized the importance of having a secure SDL and importance of testing in their product (which also has cost benefits as compared to releasing and developing patch). Microsoft and adobe has adopted fuzzing into their SDL cycle thus highlighting the importance of this practice. Figure 1: Microsoft Simplified SDL (4) While referring back to security conferences in the past, significant work has been done in between 2005 – 2006 (5). This period showed that security researchers and hackers are increasing their efforts to better themselves in the art of fuzzing and finding bugs and producing exploit using this technique.
  • 2. Until many vendors were compelled to release their monopoly on popular binary and network formats by European Union or in some case voluntary disclosure, this field was majorly focused on Black box testing, meaning the hacker had very little or in some cases no knowledge about the protocol they are trying to fuzz. Black-box fuzzing focuses on input format—ignoring the tested software target. While being efficient and allowing reuse of the same test tools across different tested targets that share the same data formats, this method misses significant code paths. These code paths depend on configuration options or specific complex conditions that are governed by application logic. (5) Even as Microsoft and other vendors do talk up fuzzing seriously, developing multiple protocol fuzzers for various line of product. There is no doubt that their fuzzer is uber cool and is able to find most of the bugs, some bugs remains and the security industry is gladly finding them and making living out of it. Google and Firefox have a policy of rewarding any researcher who comes up with a bug or a resulting exploit. Other such rewards on submitting a bug is funded by various IPS companies like iSight Partners, Tipping point etc. They buy your bug and then work towards a responsible disclosure while negotiating with the vendor. In the initial years fuzzing was never looked up by academia as it was considered to be naïve among the peers, but soon this process gained momentum and attention of many researchers. This simple technique is proven to produce result faster and cheaper than any other process known. More information about fuzzing would follow in the following discussion on fuzzing. Software Testing Software testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results (6). Although crucial to software quality and widely deployed by programmers and testers, software testing still remains an art, due to limited understanding of the principles of software. The difficulty in software testing stems from the complexity of software: we cannot completely test a program with moderate complexity. Testing is more than just debugging. The purpose of testing can be quality assurance, verification and validation, or reliability estimation. Testing can be used as a generic metric as well. Correctness testing and reliability testing are two major areas of testing. Software testing is a trade-off between budget, time and quality. (7) Unlike most physical systems, most of the defects in software are design errors, not manufacturing defects. Software does not suffer from corrosion, wear-and-tear -- generally it will not change until upgrades, or until obsolescence. So once the software is shipped, the design defects -- or bugs -- will be buried in and remain latent until activation. (7)
  • 3. With time it was realised that as fixing bug is quite expensive and there is a need to optimize this process. A huge interest in this field has led to many innovations both from academia and companies. Many companies completely dedicated in producing software testing techniques, and automation has grown in recent years. Looking at these innovations it would be unfair to describe software testing in a few words. My motive here is to introduce this process as a concept if a reader want more details on this topic I encourage him to read books like “A Practitioner's Guide to Software Test Design” and others. Code Coverage Code coverage is one of the most important metrics used to decide on the completeness of the test cases. This metrics gives us the relationship between the test conducted and instructions executed with the application. The more the instructions executed the better the code coverage is. It is the quantitative measure of instructions executed among the maximum possible instructions with a particular application. Of course this metrics can be further broken down into more detailed metrics. Major among them are: (8)  Function coverage - Has each function (or subroutine) in the program been called?  Statement coverage - Has each node in the program been executed?  Decision coverage - Has every edge in the program been executed? For instance, have the requirements of each branch of each control structure (such as in IF and CASE statements) been met as well as not met?  Condition coverage - Has each Boolean sub-expression evaluated both to true and false?  Condition coverage - Both decision and condition coverage should be satisfied. Code coverage is also used to discard cases that do have low code coverage or they cannot be used to increase the code coverage. Figure 2: An example of Code coverage There is a specific class of software specifically designed to monitor the code coverage of the application under observation. Based on type of application we can have either source code coverage analyser or binary code coverage analyser etc.
  • 4. It is quite clear that this metric is both intuitive and based on testing properties; give us specific enough information about the test and its completeness. Some advance code coverage analysis tool can even give us information that what should we do to improve our test case. But remember it will only give us a metric of completeness of the test not the metrics of quality or reliability or in any kind of significance on the application itself. Also remember that coverage analysis is one of many testing techniques; you should not rely on it alone. Test needed to find bugs Tests needed for coverage Figure 3: Shows that even on a good coverage some bugs would still be left alone Blackbox Testing While testing software, one has only one of three situations in his hand. Firstly, I would have no knowledge of the inner working of the software, neither of the protocol or kind of input expected, this situation is rightly named as Black-box Testing. While testing software with no knowledge in hand would not be very fruitful, but this strategy has worked well in the past. Some of the features of black box testing are described pictorially below with self-expiatory terms. Expected Input Black box testing Boundary values Illegal values Figure 4: Blackbox Testing
  • 5. Grey box Testing The other situation can be that users have knowledge of internal data structure and algorithms for purpose of designing test cases, but the testing is done at blackbox level. Managing user input and output are beyond the scope of this kind of testing, this testing condition falls under graybox testing. White box Testing Finally, information on internal data structure and algorithms is completely shared between the product development team and the tester’s team. This kind of information can be used to test API’s, Code Coverage, fault injection, Mutation of testing and many more. A basic testing setup is described below. Statement Testing White box testing Decision Testing Conditional Testing Figure 5: Whitebox testing Fuzzing Fuzzing is a kind of software testing technique which was until recently categorized inside black box testing. To put simply, it is the technique in which invalid or mutated or malformed input is supplied to application with only intention to find bugs the application. The first person credit of working and formulating this technique is Barton Miller and his students from University of Wisconsin-Madison in 1989. They described this technique as, “Fuzz testing is a simple technique for feeding random input to applications. While random is a time- honoured technique, our approach has three characteristics that, when taken together, make is somewhat different from other approaches. 1. Input is random: We do not use any model of program behaviour, application type, or system description. This is something called black box testing. In the command line studies, the random input was simply random ASCII characters streams. From our x-window study, Windows NT study and MAC OS study, the random input included cases that had only valid keyboard and mouse events.
  • 6. 2. Our reliability criteria are simple: if the application crashes or hung, it is considered to fail the test; otherwise, it passes. Note that the application does not have to respond in a sensible manner to the input and it can even quietly exit. 3. As a result of the first two characteristics, fuzz testing can be automated to a high degree and results can be compared across applications, operating systems and vendors.” (9) Now as we look back at this the early definition of fuzzing, a few things have changed. We understand this process more clearly, more concrete theory is developed and it is proven to work time and time again. It is important to remember that any type of interaction of user supplied data either supplied by network, file, registry, API etc. can be fuzzed. Microsoft has described some key terms in fuzzing terms that one should know. Term Definition Dumb fuzzing Corruption of data packets randomly without awareness of data structure. Smart fuzzing Corruption of data packets with awareness of the data structure, such as encodings (for example, base-64 encoding) and relations (checksums, bits indicating the presence of some fields, fields indicating offsets or lengths of other fields). Black-box Sending of malformed data without actual verification of which code paths were hit fuzzing and which were not. White-box Sending of malformed data with verification that all target code paths were hit— fuzzing modifying software configuration and the fuzzed data to traverse all data validations in the tested code. Generation Generation of fuzzed data automatically—not basing on any previous input. Mutation Corruption of valid data according to defect patterns, to produce fuzzed data. Mutation Well-formed buffer that represents an equivalence class of the input. The fuzzer template takes the mutation template as an input—producing a fuzzed buffer to be sent to the tested software. Code Technology (such as that which is bundled in Microsoft Visual Studio 2005) that coverage allows inspection of which code paths were executed during testing. This is useful for verification of test effectiveness and improvement of test coverage. Table 1: Fuzzing Taxonomy (5) It is observed that fuzzing is most effective against application developed in C/C++, these languages make the programmer responsible for memory management whereas managed code i.e. developed in C#, Java etc. would yield bugs of a very different class. This does not mean that you can’t find bugs in them. (5)
  • 7. Fuzz metrics have been understood, process is divided into two types of fuzzer and further these fuzzers have been separated into more focused modules which make them work more efficiently. Hitting the target application with valid looking inputs would defiantly result in few anomalies. These anomalies can be introduced into the system any kind of user input as previously discussed. Some applications like Web Browser uses a lot of open source library and if any one of the libraries is buggy and exploit can is made out of it, we will see more of the situations like that of ‘Aurora’. Finally I would like to say that there is an important distinction between Fuzzing and other testing activity. This distinction is the intent. A testing team knows a lot about the program and basically test that whether a program is behaving as it is supposed to behave where as a security researcher only care that his fuzzer crashes your tested application. Security researchers do not care about the internal working of the program, he has one and only one purpose i.e. to crash the program or make it behave differently while playing with it. Fuzzer There have a plethora of fuzzers available in market, both open source and closed. I would like to make note of two python based fuzzing framework available in the open source community that I use most extensively.  PeachFuzzer - Peach is a SmartFuzzer that is capable of performing both generation and mutation based fuzzing (10).  Sulley - Sulley is a fuzzer development and fuzz testing framework consisting of multiple extensible components. Sulley (IMHO) exceeds the capabilities of most previously published fuzzing technologies, commercial and public domain (11). Peach is been improved day in and day out and it is the only other open source fuzzer that is maintained apart from Metasploit fuzzer. Peach is written as primary data fuzzer, but as it open source it can be extended to secondary and even nth-class fuzzer. Peach fuzzer is also used by adobe in its testing of Adobe reader (12). Sulley is not maintained but is as good as you can get when it comes to generation based fuzzing. These fuzzers have separated the data generation or mutation process from process monitoring technique which is again separated from control and even support distributed fuzzing (where a fuzzer run on multiple machine). These are some of the reasons why these fuzzers are worth mentioning. A major commercial Fuzzer is Codenomicon (13). This is the child project of the very first fuzzer ever developed called PROTOS. If you want to check out other fuzzers available I recommend you to check this http://packetstormsecurity.org/fuzzer/.
  • 8. The Complexity Jiantao Pan of Carnegie Mellon University has described the complexity revolving around bugs as (7) “Software bugs will almost always exist in any software module with moderate size: not because programmers are careless or irresponsible, but because the complexity of software is generally intractable -- and humans have only limited ability to manage complexity. It is also true that for any complex systems, design defects can never be completely ruled out.” He also describes about the never ending race among security researchers who are finding bugs and that of vendors and uses the terms, used by Boris Beizer in his book Software Testing Techniques, 2nd Edition, Pesticide paradox to describe this race. Pesticide Paradox – “Every method you use to prevent or find bugs leaves a residue of subtler bugs against which those methods are ineffectual” (14). In simple terms on can say that once you have found a bug and fixed it, you are now sure that the input that earlier did trigger bug would not be able to crash the program. But you can be never sure that pre-error test cases that the application passed in the first time would not come haunting. In many of the fuzzer it is observed that test cases produced fails to achieve the basic packet sanitation test, which prevents it from further testing. In all general purposes the system rejects virtually all the randomly generated packets, thus defeating the purpose of testing the remaining code effectively. What this means is that fuzzers with improper understanding of the input type and structure would fails often then they succeed. Intuitively the one with more knowledge about the input structure would result in better fuzzers. A study done by Microsoft on a 450 lines of code and then testing it with various fuzz combinations to see the effective results that was produced is shown below : Technique Effort Code coverage Defects found Black box + Dumb 10 min 50% 25% White box + Dumb 30 min 80% 50% Black box + Smart 2 hr 80% 50% White box + Smart 2.5 hr 99% 100% Table 2: Analysis based on Effort in producing fuzzer and defects found correlated with kind of fuzzer (5) Know what I believe, and the test shows that investing your time in right place would result in better productivity with respect to kind of bugs we found and also the amount of bugs we found, further restricting on the number of cases where input is out rightly rejected on the basis of basic sanitation techniques of the programs input validation module. If we want to go deep inside the program semantics and actually test the code of our choice or track the data input we have no choice but to work with all the knowledge we have or could acquire about
  • 9. the protocol. This involves reading and understanding of Data Format documentation of disclosed file formats, network formats and many more. We can easily right such fuzzer that generates credible input to the tested application if the stick to either an ASCII based or Binary based. WHO #UnrealIRCd JOIN #Support PONG irc.fyremoon.net irc2.dynam.ac STATS u Table 3: Example of a pure Ascii Based Data format ZIP local file header Offset Bytes Description[5]  0 4 Local file header signature = 0x04034b50  4 2 Version needed to extract (minimum)  6 2 General purpose bit flag  8 2 Compression method 10 2 File last modification time 12 2 File last modification date 14 4 CRC-32 18 4 Compressed size 22 4 Uncompressed size 26 2 File name length (n) 28 2 Extra field length (m) 30 n File name 30+n m Extra field Table 4: An example of purely binary based data format But what happens when the formats no longer sticks to one data format? What happens when our data switches from one set of data format like ASCII to binary and then binary to ASCII again and to add a cherry on top sections are encoded differently even the ascii portion can be encoded and even imported from other binary or ASCII based formats. If you want more complication here is one more to your bag, Can the code coverage be a correct metrics when analysing fuzzing input? The answer is No, you would be lucky to even get 25% code coverage for any given input, this is because of high modular structure of these programs, distributed among various binaries and not all binary load at a given input.
  • 10. Things are getting pretty complicated. Hmm! %PDF-1.5 %µµµµ 1 0 obj <</Type/Catalog/Pages 2 0 R/Lang(en-IN) /StructTreeRoot 54 0 R/MarkInfo<</Marked true>>>> endobj 2 0 obj <</Type/Pages/Count 7/Kids[ 3 0 R 23 0 R 29 0 R 38 0 R 47 0 R 49 0 R 51 0 R] >> endobj 3 0 obj <</Type/Page/Parent 2 0 R/Resources<</Font<</F1 5 0 R/F2 10 0 R/F3 12 0 R/F4 14 0 R/F5 16 0 R/F6 21 0 R>>/ProcSet[/PDF/Text/ImageB/ImageC/ImageI] >>/MediaBox[ 0 0 595.32 841.92] /Contents 4 0 R/Group<</Type/Group/S/Transparency/CS/DeviceRGB>>/Tabs/S/StructParents 0>> endobj 4 0 obj <</Filter/FlateDecode/Length 5463>> stream xœ•[•Û¸’~•ÿ Gû ­ˆÝ€A I&9ÈbgÏì&»ó0³jµÜÖÄq÷Hêtz~ýÖ…”HY”ÅÁœ´e²X$ëúUÉÑë_£ Ÿ~zýË»•?GÉõuôöçwÑÛÏ/_¼þ "™Å‰Ž>ï^¾QÿQ.ãDê(×i,Šèó×— /’8)%±øÿ*K£înññ•ýóå‹ß•J’¤¼ÎáŸ2¿¾*ñßwþ¿ÅÛë+ú>¹¾*ðsÉŸeÆ•æ¹—ñ¿ÅþÞŽ+Òë+‘à Table 5: Example of one such complex formats ie. PDF We see these being used in every day applications like office documents, Adobe PDF, SMB protocols and more. One cannot try to randomly fuzz these files as they have pretty good input validation modules which prevent any dumb attempt to fuzz them. All these formats are considered as very attractive formats within the security industry. But understanding even one format completely would take some beautiful years out of once life. So is there a solution? My answer is yes there “are” few solutions available still. Figure 6: Adobe Fuzzing Technique
  • 11. What we have gathered until here is summarized here as we move ahead you will find answers to these problems. Complications Code coverage is cannot be used as a metrics while testing test type of input streams Data format consistency i.e. Either ASCII or Binary is not maintained Multiple encoded levels exists which can be different within the same file Multiple data format either binary or ASCII are embedded within these input stream Table 6: Discussed problems Code Coverage fails for these applications In all these cases we have to work with what we have in hand. We are going to use a combination of the technique of taint analysis, protocol awareness and Code coverage. I call this as relative code coverage metric. Protocol awareness can be used as once we have all the information of a protocol that we could have, we can intuitively say that the packet which contains the most number of tags or objects would require more code to be covered with that module. Now this could be said that we cannot guarantee the code coverage still because if we do not find a packet that contains all the tags or object that our application which is usually the case, we would be not able to test all cases. Testing all cases in one go was never the idea but multiple tests covering every tag is what will be fruitful. It is intuitive that when all the tags are covered we would have near complete code coverage. To further improve upon, one can use Taint analysis to find all the possible check and balances and the further incorporate them in code. Read the ‘future’ section for more details. Data format inconsistency The other problem that we face is of data format inconsistency. One can easy write a fuzzer of either and ASCII based packet or for binary based packet. But when these formats get together in a packet, it becomes unnaturally difficult to write one. The solution lays in visualizing and breaking problem in parts which we most comfortable in. We can use the separate out the data generation capability from both ASCII and Binary format. Remember here I have trying to separate out these capabilities not necessarily for fuzzing. Multiple Encoding levels As we have separated ASCII with Binary in the same format one can further add custom encoding in each packet as one like. They will all fall back together when we combine them later. See the case study for more clarification. In a PDF file if we have a multiple font embedded inside the file we can make use of different encoders for each such font as each is generated separately. Multiple Files Embedded in a single packets With separating of types we can further separate to a secondary level data production module. What this means is that if a PDF file if we have a font and image embedded inside the file we can actually write a different fuzzer for font and for an image and combines each of these result with the PDF files in the manner similar to multiple encoding level problem.
  • 12. Now is the right time to talk about the strategy that I have used when fuzzing one such format, PDF. You will find different definition of these terms, but this is what I understand out of them. This process is typically described in the terms of system under test and called for directed area with the system, where as in my study I have taken it out of box and placed these conditions on Data packet itself. Attack Point Selection The attach point selection is a simple process in which I have tried to specify a specific point within the packet which needs to be tested. Now selection of these points depends a lot upon some gathered intelligence of the system, including pervious vulnerabilities. As this eliminates a few attack point as they have already been attacked before. For example if working on a simple PDF file which contain a U3D file which is known to previously cause a vulnerability in Adobe reader one can say this format is previously been tested primarily (after looking at the vulnerability) so a lot more efforts would be required in finding a vulnerability next time. One can focus his time and energy in finding other routes into the application which has still not been tested by security researchers. Directed Fuzzing Whenever a vulnerability is released it is released with a very few information. One such disclosure example would be. Adobe Flash Player Multiple Tag JPEG Parsing Remote Code Execution Vulnerability -- Vulnerability Details: This vulnerability allows remote attackers to execute arbitrary code on vulnerable installations of Adobe Flash Player. User interaction is required in that a target must visit a malicious website. The specific flaw exists within the code for parsing embedded image data within SWF files. The DefineBits tag and several of its variations are prone to a parsing issue while handling JPEG data. Specifically, the vulnerability is due to decompression routines that do not validate image dimensions sufficiently before performing operations on heap memory. An attacker can exploit this vulnerability to execute arbitrary code under the context of the user running the browser. Figure 7: An example of Vulnerability disclosure Now you would like to get your hands on this kind of vulnerability. Getting this vulnerability is very difficult task as we have only a specific area with a binary file to be fuzzed and with extensive inputs mutation / generations. Looking at the above example here, of the kind of disclosure available we would like to fuzz only a specific region as we have previously been aware that certain code might contain some vulnerability somewhere. Case Study: Attach Point selection I have implemented a fuzzer using the Peach fuzzing framework, where I have decided to test PDF files which have Open Type font embedded in them. Now I fuzz only the font part within the PDF by using a secondary fuzzer which contains the complete file format of TTF.
  • 13. I have done mutated fuzzing, so I had to choose a file that contains most of the tags (to achieve the maximum relative code coverage) that are there in documentation. This can be easily done if we have a good base of fonts already in our repository that was easy to get using torrents. I could not find a single file that contains all the tags so I have to go for the one which contains the maximum number of tags. I had few other files that contain tags that were not there in the maximum tag file, thus my test case is complete with all the tags that were allowed in the font file. This TTF font is arranged within PDF using Report Lab python API. Once the PDF is formed it is open up inside Adobe Reader to which debugger is attached. There are few other libraries available for you to actually merge the components with PDF some notable among them is iText and pyPdf. Even if you do not have one such library available it is easy to embed on file with simple scripts one such example is there on Feliam’s blog (15). Figure 8: Data format analysis by 010 Editors custom template of TTF
  • 14. After making this custom template and getting a thorough understanding of TTF file format I wrote a fuzzer for it. Figure 9: Snippet of fuzzer for TTF in Peach PIT format After running this fuzzer for just one day I had the following result. Microsoft (R) Windows Debugger Version 6.11.0001.404 X86 Copyright (c) Microsoft Corporation. All rights reserved. …… CommandLine: C:PROGRA~1AdobeReader~1.0ReaderAcroRd32.exe C:FuzzFileFileFormatTTFfuzzedfile.pdf Symbol search path is: SRV*http://msdl.microsoft.com/download/symbols;symsrv*symsrv.dll*c:symbols*http://msdl.microsoft.com/download/symbol s Executable search path is: ModLoad: 00400000 00454000 AcroRd32Exe.exe ModLoad: 7c900000 7c9b0000 ntdll.dll ……….. ……….. (2e4.674): Access violation - code c0000005 (first chance) r eax=00000001 ebx=07d52594 ecx=04dd57ec edx=00000002 esi=04dd56ac edi=07dfbfe4 eip=080111e1 esp=0012de48 ebp=0012de5c iopl=0 nv up ei pl nz na po nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010202 *** ERROR: Symbol file could not be found. Defaulted to export symbols for C:PROGRA~1AdobeReader~1.0ReaderCoolType.dll - CoolType+0x111e1: 080111e1 660fb64f23 movzx cx,byte ptr [edi+23h] ds:0023:07dfc007=?? Figure 10: Result from fuzzing And the following information were logged
  • 15. Figure 11: Fuzzing Results were logged For office application there are actually two ways in which you can do Fuzzing using this concept. First you can use an Library to create office files embedded with test case of your choice (some libraries test the cases sanctity before embedding, one can bypass this by changing the sanctity checking code and recompiling the library. To know which section of code is doing that you can always fall back to code coverage) and the other way would be using COM API of these Office software. By using the second method you can test 2 things. First, is there a bug in loading resource process and secondly is there a bug in processing the resource file. Case Study: Directed Fuzzing Once we are familiar with the process of creating files we can further brings thing down to tag level. This means that if we want to only fuzz one particular tag in a file which would be further embedded into some complicated data format, we just have to model that particular tag of the file in our mutator and the rest of the procedure remain the same. For example, in the example given earlier if one wants to fuzz DefineBits tag in JPEG file which would be embedded into a SWF file they have to only write a mutator for this particular tag i.e. DefineBits Tag. Rest of the procedure remains the same. In my study I will reuse my above code developed in directed fuzzing and will fuzz only a particular tag which is maxp in case of TTF fonts which would be further embedded into the PDF file. Conclusion With new functionalities being added into application with every release, new modules being added and legacy code left few holes unplugged. One has the opportunity to find these bugs given a refined and detailed process to fuzz. More knowledge of the protocol always yields into better fuzzers and more bugs. Bibliography 1. Conficker. Wikipedia. [Online] http://en.wikipedia.org/wiki/Conficker. 2. Stuxnet. Wikipedia. [Online] http://en.wikipedia.org/wiki/Stuxnet. 3. Operation Aurora. Wikipedia. [Online] http://en.wikipedia.org/wiki/Operation_Aurora. 4. SDL. Microsoft . [Online] http://www.microsoft.com/security/sdl/discover/default.aspx.
  • 16. 5. John Neystadt. Automated Penetration Testing with White-Box Fuzzing. Microsoft. [Online] Microsoft Corporation, February 2008. http://msdn.microsoft.com/en-us/library/cc162782.aspx. 6. Hetzel, William C. The Complete Guide to Software Testing, 2nd ed. s.l. : Wellesley, Mass. 0894352423. 7. Jiantao Pan. Software Testing. Carnegie Mellon University. [Online] 1999. http://www.ece.cmu.edu/~koopman/des_s99/sw_testing/. 8. Code coverage. Wikipedia. [Online] http://en.wikipedia.org/wiki/Code_coverage. 9. Noam Rathaus, Gadi Evron. Fuzzing - What's that? Open source fuzzing tools. s.l. : Syngress, 2007. 10. Michael Eddington. Peach Fuzzing Platform. Peach Fuzzer. [Online] Deja Vu Security. http://peachfuzzer.com/. 11. Pedram Amini. Sulley. Google Code. [Online] http://code.google.com/p/sulley/. 12. Kyle Randolph. Fuzzing Reader - Lessons learned. Adobe. [Online] Adobe Systems Incorporated, 1 December 2009. http://blogs.adobe.com/asset/2009/12/fuzzing_reader_-_lessons_learned.html. 13. Codenomicon. [Online] http://www.codenomicon.com/. 14. Kaul, Nihit. Pesticide Paradox. Blogs MSDN. [Online] Microsoft Corporation, Jul 2004. http://blogs.msdn.com/b/nihitk/archive/2004/07/16/185836.aspx. 15. Taint-based Directed Whitebox Fuzzing. V. Ganesh, T. Leek, and M. Rinard. s.l. : Proceedings of the 2009 IEEE 31st International Conference on Software Engineering, 2009. 16. Fuzz Testing. Wikipedia. [Online] http://en.wikipedia.org/wiki/Fuzzing. 17. The buzz about fuzzer. Info World. [Online] http://www.infoworld.com/d/developer-world/buzz- about-fuzzers-089. 18. Software Testing. Wikipedia. [Online] http://en.wikipedia.org/wiki/Software_testing. 19. Quality Assurance. Wikipedia. [Online] http://en.wikipedia.org/wiki/Quality_assurance. 20. SDL. Wikipedia. [Online] http://en.wikipedia.org/wiki/Software_development_life_cycle. 21. Paradox of the pesticides. Wikipedia. [Online] http://en.wikipedia.org/wiki/Paradox_of_the_pesticides. 22. Grammar-based Whitebox Fuzzing. Patrice Godefroid, Adam Kiezun, and Michael Y. Levin. s.l. : Microsoft Research, 2007. MSR-TR-2007-154.