1. WHI TE PAPE R
False Alarms and the
Hidden Cost of Ownership of
Inaccurate Database
Activity Monitoring Systems
Contents
2 Summary
2 The Right Call
2 The Path of a False Alert
3 The Cost of the False Positives
4 The Effect of the False Positives
4 Improving Security Performance –
Lowering Costs and Reducing Risk
4 About Secerno Database Activity
Monitoring
4 Conclusion

2. Summary The Path of a False Alert
We have all witnessed the steady flow of stories of If we are to understand the cost of even a single false
data breaches from around the world. Data loss positive alert, then we need to follow its flow and the
has become the new security nightmare, as the effects it has. Consider this imaginary but typical
database – the prime repository of all of the sensitive scenario:
information which ultimately defines a company’s
value – has come blinking into the spotlight for the It is mid-afternoon and…
first time. Previously felt to be impregnable, buried
as it was within the depths of the organisation, it has 1 Your Database Activity Monitoring system sees
now become perceived as one of the most vulnerable a statement which it considers to be anomalous.
elements in the architecture. Databases are prey Perhaps it contains a keyword that might indicate
to criminal organisations (often working through malicious activity. It triggers an alert.
your own employees) and frighteningly subject to
embarrassing accidental losses by staff flaunting, or 2 One millisecond later, your security event
being ignorant of, your own carefully crafted security management system receives the alert.
policies.
3 Around a minute later, the alert finds its way into an
Over the course of the last decade, consumers’ email batch sent to the first line support team.
concerns grew as the stories hit the press each week,
so government agencies responded with more and
more regulation, heaping on the compliance burden So far, so good. The alert has cost no money and
but not necessarily helping actual data security. The has found its way to the inbox of a person.
market duly responded and we see now the growth
of new solutions to address the new auditing and the Of course, the query itself will have executed
security needs. and the resulting data may well already be with
the requestor, unless you have your system in
Database Activity Monitoring (DAM) is one of these blocking mode.
new market sectors. It is growing rapidly – doubling in
size annually as companies adopt solutions to protect
their key data assets. Purchasing decisions are being 4 The front line support analyst sees the alert arrive
made based on notional functional requirements and and opens it up. Having read the information
the ticket price of the solutions on offer. But what does supplied, he concludes that he is not sufficiently
it really cost to protect your data effectively? What qualified to judge whether the database query
are the key cost drivers which need to be taken into statement in question is malicious or not. It does not
account? look like a text book SQL injection attack – he has
seen enough of those before – but the DAM system
This paper seeks to address these questions with a obviously responded for a reason. He does a little
focus on the particular aspect of Database Activity research then decides to play safe, and escalates
Monitoring system ownership costs: the impact on your the event on to the Database Administrator currently
organisation of choosing a system prone to generating on call. Let us say this took half an hour.
false alerts – the cost of “Crying Wolf”.
5 Another fifteen minutes later, the on-call DBA
finishes his previous task and opens the message
The Right Call from support. He decides to deal with this
immediately. The database request certainly looks
The first wave of anomaly detection in application legal and is not obviously malicious. The next step
behaviour has depended on the tried, tested and partly requires application knowledge to answer the
trusted approaches originally developed for host and question: “Does the query fit within the context of the
network intrusion prevention in previous decades. The company’s applications?” Fifteen minutes later the
approaches are probabilistic – looking for elements DBA decides to escalate the alert to the Application
in each query which MIGHT indicate anomalous Support Group for advice on source code.
behaviour – and are based on detecting known threats
using signatures.
It is now one hour since the original alert was
In technology terms, these approaches represented triggered. It has been through two people’s
the industry’s best bet – they were known to have their hands. The data resulting from the query is in a
weaknesses but they were all there was available, spreadsheet being sliced, diced and acted upon.
and some protection is better than none of course.
However, there is a fundamental problem with these
approximate approaches which in practical terms 6 The Application Support Group specialist looks at the
renders them ineffective in security terms: they alert information for a few minutes then passes the
are prone to generating far too many false positive case over to their resident DBA.
responses. False alarms drive operators to distraction,
inflate the true cost of ownership and eventually lead to 7 The Application Support Group DBA is a little
the devices being tuned down or disabled. perplexed. It sort of looks reasonable to him. Sure,
the use of this particular query structure is rather
A new generation of DAM solutions has become inelegant, but it is legal SQL. The “Our-town Credit
available now, alongside these first entrants. So how Union” subject of the query looks fair enough. They
do you weigh up the cost of owning these against the are a new customer, but presumably legitimate. He
older products – how much will they save you? Hence, calls up the application source code. And, just to be on
the question we seek to answer now is: what is the cost the safe side, as this is a security alert, he escalates
of generating false positive alerts? the issue to his manager as “under investigation”. This
all takes him an hour. He eventually sends the query
2

3. information back to the Application Support Group 10 A further email to the IT administrators over there
specialist to confirm his conclusion that the query eventually confirms that the query originated from
was legally generated from their own applications. a hot desk in room 6.2.13. There is no-one there, of
course, as the query was issued yesterday. But the
8 The DBA’s manager sends an email at the end of log shows who had been there. They are called and
the day to the security team with alert details. they eventually confirm their legitimate use of the
application and query data.
We are now three hours into the event. The 11 The Security Team mark the incident ticket as
Security Group has the alert for investigation, closed, more than 24 hours after the query had
though the message is on its way back to triggered the alert.
front line support that the query was probably
generated legally, by the company’s own 12 The incident response team looks at the policy
applications. description on the DAM system and realise that
the “Union” element in the customer’s name
had triggered a signature used for checking for
9 The IP address originating the query came from potentially malicious UNION instructions in SQL.
within the company, so the Security Analyst receiving They disable the rule concerned.
the query contacts the Operations Group. The next
morning, they confirm that the query originated in the A day after the false alert was generated and
Sales Department in Building 6. the incident is closed, while the results data
from the query are on a memory stick on the
other side of the country. No harm was done in
this case.
The Path of a False Alert
12 Day 2
Time elapsed
DATA
26 hours
20 SECONDS SQL STATEMENT
Rule disabled
Time elapsed
30 minutes
Day 1
DAM DBA
SQL STATEMENT EMAIL ALERT SIEM EMAIL ALERT EMAIL
SYSTEM
1 2 3 4
1st Line Support Team
Time elapsed Time elapsed
1 millisecond 1 minute
Day 1 Day 1 Day 1
Time elapsed
60 minutes
Day 1
Incident Closed
Time elapsed
Operations Team IT Admin Team 140 minutes
5
EMAIL
EMAIL
Day 1
Time elapsed
9 Day 2
Time elapsed
Day 2
Time elapsed
10 Day 2
Time elapsed
7
3 hours and 18 hours and 20 hours 20 hours and
15 minutes 30 minutes Room 15 minutes
Email Query Building Number Email Query
“Which building did
it come from?”
Name “Which room?” Day 1
Time elapsed
6
125 minutes
EMAIL ALERT ASG MANAGER 7 DBA
DETAILS EMAIL
End User Security Team Application Support Group
Day 1
Time elapsed
10 Time elapsed
23 hours
3 hours
Incident Closed
8
Day 2
11 Day 2
Time elapsed
25 hours
The Cost of the False Positives
So what happens now? There have been ten such false Ten such alerts have been generated during the
positive alerts from the system in the intervening time it took to clear this one, building up a potential
24 hours, all worthy of investigation. The DAM system response cost of $12k/day, just on false positives
has seen 15 million SQL statements flow past in that alone. Within two weeks, you will have spent more
time, so a false alarm rate appears low, at first glance, chasing false alarms than you did on the box itself.
at below one per million. But consider the cost to the
organisation: Following the same logic, ten full-time equivalent
staff would be required just to respond to this
A minimum of eight people have been involved system alone.
in investigating this process and at least one
person-day was used to resolve and document The appliance that cost $50k to buy is costing
the issue fully. We can estimate the full cost to the hundreds of thousands of dollars to run.
organisation at around $1,200.
3

4. The Effect of the False Positives
No company would tolerate this, of course. No The older technologies described above use regular
company could. So the specialist managing the DAM expression matching – an approach which relies on
system continued to disable the signatures and rules simply spotting known strings and patterns in SQL
generating false alerts. The false alarm rate dropped statements. As we have seen in the scenario we
a little temporarily, each time, but then crept up again considered, this simplistic method is prone to producing
as new business systems came online. Eventually, the false positive alerts, as there is little or no appreciation
response team simply disabled alerting, leaving the of the context of the keyword within its statement.
box in place just for producing monthly reports and
to satisfy regulatory requirements demanding they A new approach has been developed by Secerno; the
have such a system installed. They had been facing an SynoptiQ Analysis Engine at the heart of Secerno.SQL.
impossible dilemma: commit enormous resource to SynoptiQ enables Secerno.SQL systems to analyse
chasing false alerts or switch off the security aspects each database query statement in full, understanding
of the DAM system, leaving the database exposed. They the intent of the whole interaction, rather than just
had little choice but to switch off the alerts and adopt checking for keywords irrespective of their context.
the “Ostrich Position”. This approach delivers error-free database activity
monitoring and blocking. It is the only solution able to
The biggest cost hidden in the scenario hit the company do this.
several weeks later, when an employee started using
the company’s own applications to retrieve credit card The Secerno.SQL approach has fully automated
and identity information on their entire customer base. analysis and clustering, with intelligence naturally built
The partly-disabled DAM system was at least helpful into the system. To put it simply:
in analysing how he managed to do this, and he was
subsequently caught and successfully prosecuted. The older solutions use simple processes and rely
The company was forced to spend more than a million on intelligent humans to carry out difficult security
dollars on upgraded security measures, forensic and analyses
penetration testing, recruiting new staff as well as
customer and investor PR, but the company’s share Secerno uses intelligent processes which mean
price and market reputation never fully recovered the that humans can make simple security decisions
resulting dip. easily and quickly
Independent reviews and customer case studies have
The Improving Security Performance – Lowering Costs shown that the result of this breakthrough approach is
and Reducing Risks a zero defect rate on alerting – to the degree that the
majority of Secerno customers run their systems in
There is another way, one which avoids the hidden blocking mode, further reducing the cost of responding
overheads associated with false positive alerts; to alerts, and guaranteeing the highest level of data
allowing monitoring, and even blocking, to remain in security.
place – delivering true security.
The full semantic analysis of each and every SQL Conclusion
statement, carried out by a Secerno.SQL database
activity monitoring and blocking system – powered Database Activity Monitoring is an essential tool for
by Secerno’s patented SynoptiQ technology - would protecting sensitive data assets in database systems
have shown that the query was consistent with normal from losses to external and internal sources – but it
application behaviour. No alert would have been must be accurate. It is a fast-growing market precisely
generated. The system would sit there quietly, waiting because of the value of the data protected and the huge
for a real incident. cost of a potential data breach.
If you have absolute confidence in your system and it However, unless they can operate with a zero-error
issues an alert, then you can be completely sure that an rate, then the cost of alerting anomalous behaviour
anomalous statement was on the wire. These systems becomes prohibitive (not to mention the ability to block
are so quiet when operational, that many customers becomes impactical) and the security of your sensitive
operate them in blocking mode. Thus any statement data assets gets thrown out of the window.
that triggers a response can be stopped before it
reaches the database, before it can do any damage.
This lowers the response cost and protects your data. ZERO FALSE POSITIVES
About Secerno
Secerno.SQL is Secerno’s award-winning family of database activity grammatical clustering and machine-learning. SynoptiQ analyses
monitoring and database security solutions - uniquely available all database traffic to automatically fingerprint the true intent of all
as either hardware or virtual appliances. Secerno.SQL protects database requests. This enables organisations to see and prove with
data at the point at which it is accessed and delivers the highest unprecedented granular analysis exactly how their data is accessed
levels of protection against internal and external threats, optimises and changed.
compliance auditing and delivers the ability to improve the security
and efficiency of applications. SynoptiQ automatically clusters database interactions with others of
similar intent; highlighting areas of concern such as authenticated
At the core of all products is Secerno’s patent-pending SynoptiQ users abusing their privileges, attackers masquerading as
technology, based on breakthrough research into efficient authenticated users or any other form of SQL injection attack.
Code: WP0810_FALSALRM
Web: www.secerno.com Email: enquiries@secerno.com Copyright © Secerno Ltd
4