CallQ scope and user specification summary

1
Author and Developer: Chris Morton
Scope
And
User Specification
Summary

2
Overview ......................................................................................................................................................................................................5
User Roles.....................................................................................................................................................................................................5
Workflow......................................................................................................................................................................................................6
Quality Assessment ......................................................................................................................................................................................7
Quality Assessment Scorecard Selection Form........................................................................................................................................7
Call Details Capture Form........................................................................................................................................................................7
Scorecard Details Capture Form..............................................................................................................................................................8
Compliance Questions........................................................................................................................................................................8
Weighted Questions...........................................................................................................................................................................9
Quality Assessment Population ..........................................................................................................................................................9
Notepad, Paraphrases and Listwords .................................................................................................................................................9
Notepad.........................................................................................................................................................................................9
Paraphrases.................................................................................................................................................................................10
Listwords .....................................................................................................................................................................................10
Other Quality Assessment Capture Forms.............................................................................................................................................10
Client Notebook................................................................................................................................................................................10
Flag Call ............................................................................................................................................................................................11
Training And Development...............................................................................................................................................................11
Calculators And Tools .......................................................................................................................................................................11
Ratings..............................................................................................................................................................................................12
Quality Assessment Summary Form......................................................................................................................................................12
Scorecard Builder .......................................................................................................................................................................................13
Select an Existing Scorecard or Create a New Scorecard.......................................................................................................................13
Scorecard Questions..............................................................................................................................................................................13
Scorecard Words ...................................................................................................................................................................................15
Scorecard Settings.................................................................................................................................................................................15
Scorecard Reports .................................................................................................................................................................................16
Scorecard Category Question Order......................................................................................................................................................16
Attributes and Wordcloud Reports ............................................................................................................................................................17
Dashboard..................................................................................................................................................................................................20
Bar Graphs/Linear Graphs .....................................................................................................................................................................21
Total Compliant Percent/Average Quality Score ..............................................................................................................................21
Total Non-Compliant Calls/Total Compliant (%) Calls.......................................................................................................................22

3
Total Trifactor Distribution Percentage of Quality Scores ................................................................................................................22
Cumulative Evaluated Calls with Quality Scores <= 85% and Compliance Fails ................................................................................22
Cumulative Total Evaluated Calls with Quality Scores Below 85%....................................................................................................23
Total Failed Evaluated Calls with Quality Scores above 95% ............................................................................................................23
Total Evaluated Calls per day............................................................................................................................................................24
Cumulative Total Evaluated Calls......................................................................................................................................................24
Average Handling Time (Seconds) ....................................................................................................................................................24
Average № Calls Per QA Agent Per Day............................................................................................................................................25
Overall Total Cumulative № Evaluated Calls Per Dialler Agent.........................................................................................................25
Overall Total Cumulative № Evaluated Calls Compliance/Non Compliance ratio Per Dialler Agent.................................................26
Overall Total Trifactor Distribution Percentage per Dialler Agent ....................................................................................................26
Overall Total Cumulative № Evaluated Calls Per Dialler Team .........................................................................................................26
Overall Total Cumulative № Evaluated Calls Per QA Agent ..............................................................................................................27
Pie Graphs and Area Graphs..................................................................................................................................................................27
Percent Certainty..............................................................................................................................................................................27
Formula Reference (Sample Certainty)........................................................................................................................................27
Compliant/Ave Quality Scores..........................................................................................................................................................28
Quality Markdowns ..........................................................................................................................................................................28
Compliance Markdowns...................................................................................................................................................................29
Agent Ranking Data...............................................................................................................................................................................29
Top 5 Agents and Bottom 5 Agents Formula Reference ..............................................................................................................29
Date Range............................................................................................................................................................................................30
Wordclouds ...........................................................................................................................................................................................30
Feedback ...............................................................................................................................................................................................31
Configuration Manager ..............................................................................................................................................................................32
Team/Agent/Scorecard Configuration...................................................................................................................................................32
Client/Dashboard Configuration............................................................................................................................................................33
Client User Configuration ......................................................................................................................................................................34
Manager User Configuration .................................................................................................................................................................35
QA User Configuration...........................................................................................................................................................................36
Scorecard Configuration........................................................................................................................................................................36
Filename Decoder ......................................................................................................................................................................................37
Calculators..................................................................................................................................................................................................38

4
Stats Model Calculator – Flash ..............................................................................................................................................................38
Percent to be Audited.......................................................................................................................................................................38
Sample Certainty ..............................................................................................................................................................................38
Sample Certainty Comparison ..........................................................................................................................................................39
Stat Model Assessment Calculator ........................................................................................................................................................39
Trend Analysis Tool ....................................................................................................................................................................................40
Combinator – Select Ordinal Column ....................................................................................................................................................40
Combinator – Select Comparison Columns ...........................................................................................................................................41
Combinator – Triplets.......................................................................................................................................................................41
Combinator – Quads.........................................................................................................................................................................42
Trend Findings.......................................................................................................................................................................................42
Dynamic Import..........................................................................................................................................................................................43
Import Pre-Analysis ...............................................................................................................................................................................43

5
OVERVIEW
This document serves to summarise all intended functionality related to the CallQ Infinity Quality Assurance System as commissioned by
Call Quality to meet the business needs described to the developer.
The CallQ Infinity system is designed to meet the following core functionality requirements:
1. Enable the user to define and publish customised scorecards to be processed by agents in order to evaluate customer
interactions
2. Collate and report on data collected in customised scorecards for the purposes of qualitative and quantitative analysis of
Compliance Pass/Fail Data and numerically weighted Scorecard Questionnaire Data to aid the management to make correct
and timely decisions pertaining to a specific campaign, as would be deduced from the cited reports
3. Real time and chronologic graphical/numerical dashboard display of collated data
4. Facilitate a user friendly and feature rich graphical user interface to replace existing quality assurance procedures specific to
Call Qualitys’ internal business models and processes
5. Hierarchical organisation of human resources related to internal operation of Call Quality in order to create work flow
efficiencies and correct dissemination of collected data to relevant people of both client and internal data consumers
CallQ Infinity a generic and dynamic software system that is developed to replace existing quality assurance and customer interaction
evaluation procedures with a standardized, accurate and consistent methodology. The purpose of CallQ is to provide high quality and
reliable data that clients and executives can refer to, to support decision making of senior executives both internally within CCL and
externally by our clients, relating to interactions of call centre agents and customers of the subscribed client or campaign.
USER ROLES
In order to achieve the required functionality a number of user roles have been defined on the system. User roles serve to allow correct
access permissions to certain functionality within the system and to reflect the human resources hierarchy within Call Quality to achieve a
structured quality assessment environment; this ensures that quality assessment procedures are adhered to thereby ensuring that data
published by Call Quality is reliable and consistent. Examine the table below for descriptions of the roles defined in CallQ.
Role Description Access Notes
Administrator Native WordPress role allows
for full access to the entire
frontend system, including the
WordPress Dashboard
Full access, cannot capture
quality assurance data that is
reflected in reporting
functionality
Recommended that this role is
limited to only senior technical
staff. Administrators must have
a configured login and can only
be added by the WordPress
native user ‘admin’.
Client Client subscribers to the CallQ
system can view data as
represented on the client
dashboard. Navigation is
limited to the Client Dashboard
only and is exclusively a data
consumer role. A Client will be
able to login via a VPN
connection from a remote
location
Assigned Client Dashboards
only. Cannot generate
documents using the document
export functionality. No
navigational options.
Clients must have a configured
login.
Campaign Manager Campaign Manager Users can
be configured for senior staff
members that are responsible
for the daily operations on the
dialling floor.
Access is restricted to
Dashboard Access and cannot
generate documents from the
CallQ document export
functionality. Dashboard
functionality includes access to
all scorecard data that are
assigned to the campaign
manager.
Campaign Managers are
assigned an automatically
generated CallQ login.
Configuration Manager Configuration Manager Users
are of key importance
concerning the addition of new
users, teams, definition of and
visibility of new/existing
scorecards, assignment of
scorecards to teams and to
specific QA Agent Users.
Full Access to the Configuration
Manager interface, Scorecard
Builder and List Manager
Configuration Manager users
should be assigned to only
senior staff members who are
responsible at a high level for
all human resources (besides
Administrators) and the
definition and publication of
scorecards. Configuration

6
Additionally Configuration
Manager users can define
which Dashboards a Client User
has access to.
Managers are assigned an
automatically generated CallQ
login.
Data Manager Data Manager Users have
access to the reporting
functionality and the scorecard
building functionality of CallQ.
Full Access to Attribute and
Wordcloud Reporting, All
Dashboards (all campaigns
including document exports),
Trend Analysis Tool, Dynamic
Import
Data Managers can access all
campaign data within the
system, therefore must be in a
trusted position in order to
ensure that data is distributed
only to relevant data
consumers. Data Managers can
use CallQ to provide data to be
further analysed by MI Analysts.
Data Managers are assigned an
login.
Director Director Users are similar to
Campaign Manager Users,
however can use the document
export features to export data
from the dashboards they have
access to.
Dashboard functionality
includes access to all scorecard
data that are assigned to the
Director. Full Access to Trend
Analysis Tool and Dynamic
Import Access
Director Users are assigned an
login.
QA Agent The QA Agent users on the
system make up the majority of
users of the CallQ users. QA
Agent users are assigned to a
Team as defined in CallQ, and
can be assigned scorecards
either individually or inherit the
scorecards already configured
for all users in a specified team.
QA Agents have access
restricted to the Quality
Assessment Functionality only
and are the only users who can
capture data relating to the
quality assessments according
to the scorecards assigned to
the user.
QA Agents are assigned an
login.
WORKFLOW
The workflow captured in CallQ includes the population of user defined scorecards specific to any campaign by QA Agents. This data is
then collated and reported on using two main reporting tools, namely the Attributes and Wordcloud Reports and the Dashboard.
The business model of CallQ corresponds to the Call Quality quality assessment procedure, whereby QA Agents listen to calls and evaluate
customer/agent interactions against a set of questions formulated to identify what interactions are most relevant when determining the
interaction outcome, be it favourable or unfavourable. After a number of assessments have been conducted against a set of questions
specific to the campaign, a data set is analysed to reveal key performance indicators that are most relevant to the interaction outcome.
The data is represented graphically and in the form of downloadable documents that can be compiled ad hoc from one of the following
data reporting features:
Dashboard
Attribute and Wordcloud Reporting
Trend Analysis Tool
From the data reporting, one can derive which factors, according to the set of questions specific to the campaign, which agent and
customer behaviours are most influential when determining a particular outcome. Once conclusions are drawn from analysis, senior
company executives can use the information to support or refute certain decisions regarding campaign performance. Such information
that can support executive decisions from the conclusions drawn from CallQ include and are not limited to:
Identification of Dialler Agents who are not performing well enough
Identification of Dialler Agents who are exceptionally good at their jobs
Identification of Dialler Agent behaviours areas that need improvement
Identification of Customer behaviours that determine a particular call outcome
Identification of relationships of scorecard questions with regard to a certain interaction outcome
Quality and Compliance metrics concerning the efficacy of internal training
Data that can support Speculative Analysis of marketing activity efficacy
Data that can support Speculative Analysis concerning consumer behaviours
Data that can support Speculative Analysis of future trending and current market behaviours

7
CallQ is programmed to allow the objective and automated analysis of data captured by the QA Agents; the scientific approach allows a
report data consumer to be confident in the report findings, in order to give the management decision makers the necessary information
to make informed judgements concerning aspects of day to day business operations.
QUALITY ASSESSMENT
The Quality Assessment Capture Form facilitates the process whereby QA Agents can evaluate a customer interaction against a Scorecard
defined for a particular campaign. Scorecards are assigned to a particular user or team, from which the QA Agent can select and proceed
to populate and submit to the database. From all the user roles in CallQ, only QA Agents can capture data that can be used to report on.
Please note that these screenshots show that an administrator user is logged in, and therefore the data that is being captured is not added
to the databases, since the populated form is not submitted by a QA user.
QUALITY ASSESSMENT SCORECARD SELECTION FORM
The screenshot on the right is the first step in the
Quality Assessment Process. This form allows the
QA Agent to select the scorecard relevant to the
customer interaction that they are about to
evaluate.
CALL DETAILS CAPTURE FORM
The Call Details Capture form includes fields that
are used to enter the most basic data related to
the customer interaction, including details such
as Dialler Agent Name, Team Name and CLI. This
form is a common form for all Quality
Assessments, collecting information to uniquely
identify the customer interaction on the
database.
To ensure the highest possible quality of data
capture, including cleanliness and consistency of
the captured data the form includes validated
fields that check the type of data that that is
being captured against what is expected to be
captured. For example the CLI can only accept a
correctly entered phone number, which does not
contain letters; similarly where name fields are
entered, numbers are not allowed. Additionally
CallQ ensures the correct casing of person names
is entered by automatically correcting syntax errors upon entry of erroneous data. To ensure a higher accuracy and consistency of
captured data, special dropdown lists are included to allow the user to first select the correct value, and if it does not exist in the list, then
add it manually.
To optionally match the unique quality assessment to a specific customer interaction, the Call Details Capture form allow the user to
upload a recordings of the interaction, so that the customer interaction can be matched against the quality assessment, for the purposes
of ‘dip-checking’ and evaluated call; this ensures that the internal quality procedures of Call Quality are being adhered to.

8
SCORECARD DETAILS CAPTURE FORM
The Scorecard Details
Capture Form displays
the specific
questionnaire details
according to the
Scorecard created on
the Scorecard Builder.
This form includes a
navigational ‘tree’ of
all questions on the
scorecard. In most
cases the questions
are distinctly of two
kinds: Compliance
Questions and
Weighted Questions.
COMPLIANCE QUESTIONS
Compliance Questions represent a series of questions that must all be affirmative in order for a compliance pass to be evaluated. If a
single Compliance question is not answered correctly, then the customer interaction will fail the Compliance Evaluation section of the
quality assessment.

9
WEIGHTED QUESTIONS
Weighted Questions are associated with a numerical ‘points’ value. Upon selecting a correct answer for a weighted question, the weighted
points are accrued in the total Quality Score of the particular assessment, which upon submitting the questionnaire to the database is
saved as a total Quality Score as well as a percentage value of the total maximum Quality Score for a particular scorecard.
Weighted Questions are grouped into categories as specified in the Scorecard Builder, creating an organised and topically related question
tree for orderly and scientific navigation of the scorecard question set.
QUALITY ASSESSMENT POPULATION
To populate the form the QA Agent is required to navigate the question tree and select a
question, where upon the question is displayed and the agent input can be entered into the
inputs displayed, as defined for that particular question. The inputs that can be displayed for
each question can differ, and the following inputs can be entered:
Radio Buttons
Dropdown Lists
Multi-select Lists
Check Boxes
Single Line Text Inputs
Text Area Fields
Calendar Date Selection
Upon capturing information and clicking save, the answers for the question are showed in a
quality assessment history window, to help the QA Agent view a summary of the quality
assessment as they proceed to populate the form. After all appropriate questions have been
answered by the QA Agent, the agent proceeds to submit the form to the database, and the
QA Agent can continue to move on to the next quality assessment.
NOTEPAD, PARAPHRASES AND LISTWORDS
NOTEPAD
The functionality enabled by Notepad allows a
QA Agent to edit free text in text area sections
that will not be captured to the database, to
give the QA Agent an area to edit their
commentary to a suitable standard which can be
accepted into the database. Such editing may
include the correction of spelling and grammar
or the truncation of text to make it as concise as
possible.

10
PARAPHRASES
Paraphrases represent the essence of a particular phrase that could be said in a number of ways, include the verbatim expression as
shown in the left most listbox of the paraphrase. For example, referring to the screenshots, the phrase ‘I enjoy playing Eurochance’ shows
(2) in the right box. In this example this may mean that the QA Agent has gleaned from the customer interaction that the customer has
said a paraphrase that has equivalent meaning twice during the call. This may mean the customer said ‘I love playing Eurochance’ and
‘Eurochance is fun to play’ which in this case are both paraphrases of ‘I enjoy playing Eurochance’.
The value that this adds to the quality assessment procedure is that calls can be grouped by the essence of the communication, adding an
additional layer to the customer interaction reporting to be able to better identify what kind of things customers might say that relate to a
particular interaction outcome. The definition of paraphrases may also include phrases said by the Dialler Agent, which may also influence
the outcome.
Paraphrases are defined during the definition of the scorecard on the Scorecard Builder, as a targeted phrase collection, intentionally to
pre-emptively distinguish what customers might be saying. The quantitative result can be used to analyse the frequency of certain
‘communication essence’ as related to how this affects a certain interaction outcome.
LISTWORDS
The functionality of Listwords enables the QA Agent to capture verbatim words mentioned either by the Dialler Agent or by a customer.
This serves the purpose of identifying certain words specific to the clients’ product or service, or the customer interactions that are being
assessed. For example by using Listwords it is possible to quantify how often the clients competitor product, brand or service is being
mentioned. Specifically for example, if a client campaign is specifically trying to promote ‘Product X’ (from the client ‘Vendor A’), yet the
direct competitor to this product is ‘Product Y’ (from ‘Vendor B’), one can ascertain that customers are mentioning Product Y on average
of 3 time per customer interaction when the outcome of the customer interaction promoting Product X leads to a particular outcome (e.g.
a non-sale). These data, once reported on, may help determine certain customer behaviours in relation to Vendor A’s marketing activities.
OTHER QUALITY ASSESSMENT CAPTURE FORMS
To ensure that the correct volume of information is captured, the Quality Assessment Form includes the following additional forms for
capturing data that adds value to a particular quality assessment:
CLIENT NOTEBOOK
The Client Notebook is a free text field to provide
the client with important notes that the client
should be aware of as captured by the QA Agent.

11
FLAG CALL
The Flag Call Form enables a QA Agent to flag an exceptionally good or bad call, and to bring
this to the attention of persons whom are configured to receive an email of this alert. This
form allows for emergency interventions to be taken when appropriate, or alternatively
allows a collection of useful training resources to be collected.
TRAINING AND DEVELOPMENT
To capture freetext comments particularly to do
with improving Dialler Agent performance, for
the purposes assisting with training and
development of Dialler Agents this form enables
the capture of three specific types of feedback
that can be used by Dialler Agent Coaches.
CALCULATORS AND TOOLS
In certain instances it is necessary for QA Agents
to calculate and convert numbers. To allow this
in a single interface, without having to depend
on external software the Calculators And Tools
form assists QA Agents to do their job. In the
current CallQ system a simple calculator, a
minutes to seconds conversion tool and a
seconds to minutes conversion tool has been
included in this form.

12
RATINGS
The capture of certain ratings is important in
terms of base metrics to be analysed by reporting.
The ratings form enables a QA Agent to rate the
Dialler Agent by the following metrics:
Overall Rating
Speech Clarity Rating
Product Knowledge Rating
Customer Rapport Rating
Total Talk Time
Total Hold Time
Number of Hold Intervals
The collection of such metrics may help identify
which agents are not adequately performing
according to the rating system, and may
additionally reveal how certain Dialler Agent
behaviours in regards to hold intervals and total
hold times may or may not influence a certain
campaign.
QUALITY ASSESSMENT SUMMARY FORM
Once a quality assessment is submitted, the
summary of the quality assessment capturing
process is displayed before re-selecting the next
Scorecard Quality assessment. The Quality
Assessment Summary Form gives the QA Agent
some feedback, enhancing the user experience
of the QA Agent, by allowing the QA Agent a
chance to review and amend their answers
before moving on to the next quality
assessment.
If necessary to amend the capture of the
previous quality assessment, the QA Agent can
use the browser back button to load the
previous form and recapture as required.

13
SCORECARD BUILDER
The Scorecard Builder Form is the starting point from which a quality assessment can be initiated. The Scorecard Builder includes an
intuitive interface whereby a scorecard can be authored by a Configuration Manager or an Administrator.
SELECT AN EXISTING SCORECARD OR CREATE A NEW SCORECARD
The Scorecard Builder
facilitates the creation of a
new scorecard or the
modification of an existing
one. The Configuration
Manager can enter a new scorecard name or
select an existing scorecard from the dropdown
list. The Scorecard Builder loads the Scorecard
Questions Form and the Configuration Manager
can proceed to define a scorecard by adding a
new question using the green plus icon, or remove a question by clicking the red cross icon.
SCORECARD QUESTIONS
The Scorecard Questions Form includes a template question construction interface. The
Configuration Manager must first populate the following fields:
The Question Text field is limited to 150 characters of alphanumeric (including
punctuation) text.
From Category Field the Configuration Manager can
select from the list of Categories or add a new category
(50 characters of alphanumeric text). To create a
Compliance Question the user must either ‘Compliance’ or ‘Critical Fail’ from the dropdown. To create a
Weighted Question any other category can be selected or entered.
The Question Weighting represents the points
value of a Weighted Question. The user may
select a value from 0-20, the sum of all the
weighting from Weighted Questions is the
total points value of the Scorecard. When defining a question, the
Configuration Manager must assign appropriate question weightings according
to the relevance of the question in the context of the quality assessment for a
particular campaign.
All questions are indexed by default, and should the Client requirements
include a specific tracking of a particular KPI, the Index for Analysis checkbox should be selected.
In order to temporarily hide a from the Scorecard in the Quality Assessment Form the show question
checkbox can be unchecked.
To create the answer template for the scorecard question, icons
representing the various answer input types are single clicked and display
in the question preview.
To accommodate a dynamic interface, the Scorecard Question Form
facilitates the construction of an answer template in a WYSIWIG style way,
so that the scorecard author can view and test the questions as the
scorecard is built, with the Question Preview. The Question Preview is an approximation of what the QA
Agent can see on the published scorecard.

14
Certain pre-defined dropdown lists exist, including a year, month, day and time selection. All other dropdown selections can be defined
using the popup that displays when the list icon is clicked. From the list popup the user can select how to represent the list in the
scorecard, which can be one of the following:
Radio Buttons
Dropdown Lists
Multi-select Lists
Check Boxes
To define the values for a list a comma separated list of values can be entered. The comma
separate values will be displayed as options according to the type of list selected.
When the single line text icon is selected a dialogue displays that allows the user to select a
validated text type according to what is required by the question. The different options create
a corresponding single line text field in the Question Preview. The validation on the text boxes
ensures that only data conforming to the type of text box can be captured, helping achieve
data cleanliness and correct data capture as required.

15
SCORECARD WORDS
To populate the Listwords and Paraphrases for a
specific scorecard this can be done in the
Scorecard Words form. This involves compiling a
comma separated list of words or phrases and
entering the data into the appropriate sections.
Note that phrases cannot contain commas, since
this would result in the phrase being spilt where
the comma exists.
Listwords should represent a list of ‘targeted
words’ that the QA Agent is tasked to listen for
and count. The Quality Assessment Listword
Counter allows the QA Agent to easily identify
and count the frequency of specific words by
clicking on the corresponding word in the
Listword Counter.
Similarly the paraphrases should consist of a list of ‘targeted phrases’ that are synonymous to an expression that either the Dialler Agent
or customer may say. The QA Agent can use the Paraphrase counter in the Quality Assessment page to count the frequency of
synonymous paraphrases as the customer interaction is assessed.
SCORECARD SETTINGS
The Scorecard Settings allow a Configuration
user to upload a Scorecard Logo that will be
displayed on the Quality Assessment Form in
order to create a graphic identity for the
scorecard. Additionally a scorecard description
can be edit that serves as a guide for the QA
Agent. The Scorecard Description should contain
instructions and information concerning the
particular quality assessment related to the
scorecard.
Call flag email addresses are also configured
here, as a comma separated list. People who are
likely to appreciate real-time email feedback of
customer interactions should be listed in here.

16
SCORECARD REPORTS
From the Scorecard Reports Form the
Configuration Manager can set the schedule of
automatic reporting. The Scheduled Automatic
Reporting works in the background to generate
reports as scheduled to increase the speed of
other reporting functionality.
By including a comma separated list of Report
Subscriber email addresses, the subscribers are
automatically sent an alert email when a
scheduled report completes successfully,
prompting the user to login to a specified url to
view the compiled report.
SCORECARD CATEGORY QUESTION ORDER
To accommodate the required functionality for
displaying the scorecard question on the Quality
Assessment in the correct order, the Scorecard
Category Order Form enables a Configuration
Manager to order questions and categories into
the order required by the QA Agents.
Questions are grouped together by categories,
the categories can be ordered into place by
dragging and dropping the Category Texts
accordingly.
Similarly the questions belonging to each
category can be dragged and dropped into the
correct order. The order in which the questions
are displayed in
the Question Tree
is set here, and is
important in terms
of ensuring a
logical order for
the QA Agents to
capture the data
for each quality
assessment.

17
ATTRIBUTES AND WORDCLOUD REPORTS
The Attributes and Wordcloud Reports generate a collated report of all question answer pairs for a particular scorecard. The first page of
the Attributes and Wordcloud Reports includes a number of selection criteria to generate a collation according to the filter criteria
selected.
The filter fields which can be selected include the following:
Scorecard – the scorecard for the collated report to compile data from
Dialler Campaign – the Dialler Campaign which the QA Agents have captured on a specified Scorecard
Dialler Team – the Dialler Team as captured by the QA Agents on a specified Scorecard
Dialler Agent – the Dialler Agent as captured by the QA Agents on a specified Scorecard
QA Team – the QA Team of QA Agents who captured against a specified Scorecard
QA Agent – the QA Agent of who captured against a particular
CF Outcome – a flag to filter either Compliance Fails or Compliance passes for a specified Scorecard
Min Percent – a filter to collate records above a particular Quality Score Percent for a specified Scorecard
Max Percent – a filter to collate records below a particular Quality Score Percent for a specified Scorecard
Assessment Start Date – the earliest date from which to compile data for a specified Scorecard
Assessment End Date – the latest date from which to compile data for a specified Scorecard
Should non-matching filter criteria be selected (for example if a certain QA Team does not capture against a particular Scorecard) the
report will prompt the user that no data exists for the selected filter criteria.

18
The Attributes and Wordcloud reports can be accessed by Data
Managers and Administrators.
By selecting the ‘Do Word Cloud’ check box, Wordclouds for
questions that have free text fields will be analysed and
Wordclouds generated for each qualifying question.
The Attributes and Wordcloud Report also consists of a dialogue
that can be used to define a custom colours to use when colour
coding the Wordcloud Report. Each word is coloured according to
its frequency and the corresponding colour selected.
The next form in the Attributes and Wordcloud Reports allows a
user to customize the fields to be displayed in the collated report.
By default all questions are displayed and checked for a particular
scorecard. Unchecking a question will not show that field in the
scorecard. The user may order the column order of the collated
report by drag and dropping the questions into the order
required by the user.
The result of the Attributes and Wordclouds Report is a downloadable .csv file that can be opened in a spread sheet program such as
Microsoft Excel.
The downloadable file is automatically named with a unique filename which can be entered into the Filename decoder to obtain
information pertaining to the files creation.
The .csv file includes all the data that has been
captured for a particular scorecard which meets
the filter criteria as selected in the first and
second pages of the Attributes and Wordcloud
reports. The file can be further analysed by data
analyst staff to reveal the information that is
being sought.
The Wordcloud report reflects the frequency of
unique words that occur in freetext fields as
captured by QA Agents for questions that have
freetext fields in the answer template.
Depending on the number of occurrences of a
particular word, more frequent words show in the report are larger in size and are coloured in darker shades and the inverse applies for
words that occur less frequently. The Wordcloud Report form includes the Wordcloud Bar that allows the user to edit the colour theme of
the reported words and order them according to the number of occurrences and alphabetically. The Wordcloud bar also allows a user to
select a particular wordcloud for export as PowerPoint, Word or Excel files which become available for download.

20
DASHBOARD
The Dashboard is an important feature of CallQ and is the primary graphical interface for data visualisation of the quality assessment
process as captured for a specific scorecard.
The Dashboard includes several features that can be used to accurately ascertain the current performance of a quality assessment
process, including certain KPI’s which are useful for determining the efficacy of certain executive level decisions.
The Dashboard is accessible to the following roles in CallQ:
Administrator – Full access to export functionality (All Scorecards)
Client – No Access to export functionality (Assigned Scorecards)
Campaign Manager – No access to export functionality (Assigned Scorecards)
Data Manager – Full access to export functionality (All Scorecards)
Director – No access to export functionality (Assigned Scorecards)
The first stage of loading the Dashboard includes selecting a scorecard from the specified list, as determined by the assignment of
applicable scorecards as configured by a Configuration Manager.
The Dashboard does not conform to the rest of the standard CallQ navigation or layout by design. There is no navigation menu to guide
the user to any of the other pages, and specifically for Client user, no navigation is possible beyond the Dashboard Page.

21
The Dashboard shows the synopsis of data related to a selected Scorecard quality assessment over a period
of time. The initial load page displays data representing up to 50 days of collated data. The data visualization
attempts to show graphically performance metrics of the selected Scorecard.
The Navigation options for the Dashboard page include the Campaign Overview (Up to 50 days of Scorecard
Data), Date Range (Configurable Date Range), Wordclouds and Feedback.
BAR GRAPHS/LINEAR GRAPHS
The Bar Graphs and Linear Data
represented on the dashboard are
displayed in an interactive interface that
must be navigated to show all data
visualisations. The graphics are generated
as the user navigates via the arrows or
alternatively via the menu for the section of the page. Each graph represents data that is
extracted from the Collated Scorecard using a particular query to calculate the metrics.
Upon clicking the graphic in a certain region the graphic is enlarged and the corresponding
tabular data is shown, as represented by the graph.
The enlarged display graph can be selected for
export by clicking the office export item,
adding the graphic and the data to a
compilation of data from the dashboard,
made available for export into PowerPoint,
Word or Excel.
TOTAL COMPLIANT PERCENT/AVERAGE QUALITY SCORE
The Total Compliant Percent/Average Quality Score Linear Graph
displays the calculated totals of customer interactions that are
compliant and the average quality scores on a daily basis. This
particular graph includes functionality that allows the user to zoom
into a particular date range to see the data in a detailed display. The
graph also includes the ability to scroll the date range of the data
displayed, as shown below.

22
TOTAL NON-COMPLIANT CALLS/TOTAL COMPLIANT (%) CALLS
Total Non-Compliant/Total Compliant Calls displays a
stacked bar graph of the percent ratio of calls that are
compliant against the calls that are non-compliant on a
day by day basis. Compliant calls represent all calls
(customer interactions) where the QA Agents have
passed all compliance questions according to the
Scorecard for the particular quality assessment. Non-
Compliant calls represent customer interactions in which
the QA agents have failed one or more compliance questions for the Scorecard for the quality assessment.
The value of this graph helps identify ‘bad days’ and ‘good days’ for a particular quality assessment allocation of calls. The dates on this
graph indicate the date of evaluation that the quality assessments where processed by the QA Teams.
TOTAL TRIFACTOR DISTRIBUTION PERCENTAGE OF QUALITY SCORES
The Total Trifactor Distribution Percentage of Quality
Scores shows a stacked bar chart representing how the
distribution of quality scores varies over a period of time on
a day to day basis. The Trifactors are represented in three
colours:
Green represents the percentage proportion of customer interactions that have been evaluated with Quality Scores between
95 and 100% (including 95%)
Yellow represents the percentage proportion of customer interactions that have been evaluated with Quality Scores of 85 up to
95% (not including calls with 85% or 95% Quality Scores)
Red represents the percentage proportion of customer interactions that have been evaluated with Quality Scores less than 85%
(including 85%)
The value of this graph is that it allows the user to see how well the customer interactions are performing in terms of overall quality over
time. The dates on this graph indicate the date of evaluation that the quality assessments where processed by the QA Teams.
CUMULATIVE EVALUATED CALLS WITH QUALITY SCORES <= 85% AND COMPLIANCE FAILS
The Cumulative Evaluate Calls with Quality Scores <=
85% and Compliance Fails represent the total
percentage of calls with scores below 85% and calls
with Compliance fails in relation to the total number
of evaluated calls over a period of time on a day to
day basis, as a compound line graph. This is an
important graph since spikes or dips in this graph will
indicate a significant change in quality of the
customer interactions being assessed.

23
One can see that in this particular graph that the percent values plateau, which is indicating that the quality is stabilising. From this plateau
the data analysts may determine a baseline proportion of what is acceptable in terms of Quality Scores and Compliance failures. The dates
on this graph indicate the date of evaluation that the quality assessments where processed by the QA Teams.
CUMULATIVE TOTAL EVALUATED CALLS WITH QUALITY SCORES BELOW 85%
The Cumulative Total Evaluated Calls with Quality
Scores below 85% shows a bar graph of the total
number of calls that have been evaluated with
Quality Scores equal to or below 85% over a period of
time on a day to day basis. Sharp rises in this graph
would indicate a sudden drop in quality. The number
displayed in the bars of the graph is the total sum of
calls that have been evaluated with a Quality Score of
below 85%. As time progresses this number will increase as more customer interactions are processed, the last number (bar) in the graph
always represents the overall total number of calls with low quality scores.
The dates on this graph indicate the date of evaluation that the quality assessments where processed by the QA Teams.
TOTAL FAILED EVALUATED CALLS WITH QUALITY SCORES ABOVE 95%
The Total Failed Evaluated Calls with Quality Scores
above 95% graph represents a cumulative total of calls
with high quality, above or equal to 95% that were also
compliance fails. These customer interactions would
have failed on a Compliance markdown yet have
achieved the required quality, therefore the calls that
belong to this data set are of particular interest in terms
of the quality assessment procedure.
The quantification and the identification of this data is important since these calls may have been
erroneously failed or may indicate that the Dialler Agents that are involved in these customer
interactions are not trained properly, or may not be conforming with instructions of the Client
brief. Also the customer interactions of this data set may represent potential follow ups for the
Dialler agents to revisit. The dates on this graph indicate the date of evaluation that the quality
assessments where processed by the QA Teams.

24
TOTAL EVALUATED CALLS PER DAY
The Total Evaluated Calls Per Day Graph shows how
many calls where evaluated on a given day, on a day by
day basis. The value of this KPI will help identify how
well the QA Team are working in relation to the
prescribed target for the day. As more QA Agents are
assigned to the quality assessment for a specified
Scorecard the total evaluated number of customer
interactions should increase, similarly the inverse
applies when QA Agents are removed from the specified Scorecard. A decrease or increase in total number of calls evaluated per day
could also offer insight into how motivated a particular staff contingent assigned to a specific Scorecard are.
CUMULATIVE TOTAL EVALUATED CALLS
The Cumulative Total Evaluated Calls shows a bar
graph indicating the total number of calls evaluated
over a period of time, on a day by day basis. This KPI
indicates the gross productivity of a quality
assessment and is important in terms of determining
the certainty of the results of the other KPI’s
displayed. In relation to overall quality assessment
targets it can help provide information about how
close the QA Agents are to achieving the number of customer interactions required by the client in order to achieve an estimate on when
a certain target allocation will be reached.
AVERAGE HANDLING TIME (SECONDS)
The Average Handling Time graph shows the
average time in seconds used to evaluate a call.
Since customer interaction vary greatly in time it is
important to display the average time spent on
quality assessments. By establishing and
monitoring average handling times the user can
easily see on the data visualisation where
insufficient or excessive handling times are
evident; this may indicate that the quality assessments are not being carried out properly or that the customer interactions are either
increasing or decreasing in duration as the campaign ages. By establishing a average handling time baseline, management can allocate
sufficient staff resources for a particular campaign, determine the estimated cost of a quality assessment activity and ensure that a
consistent quality standard is adhered to in terms of how much time a staff member should be spending on a quality assessment of a
customer interaction.

25
AVERAGE № CALLS PER QA AGENT PER DAY
The Average Number of Calls Per QA Agent Per Day
graph displays a stacked bar graph of the average
number of quality assessments that are being carried
out for the specified scorecard, compared with the
number of QA Agents that have been capturing for a
particular day.
Ideally the average number of calls a day should be
constant, since each individual QA Agent should be processing the same number of calls per day. In this stacked bar chart, the red
indicates the total number of QA Agents that have submitted data, and the green indicates the average number of assessments per agent.
OVERALL TOTAL CUMULATIVE № EVALUATED CALLS PER DIALLER AGENT
The Overall Cumulative Number of Evaluated Calls Per
Agent bar graph shows the number of customer
interactions evaluated for each Dialler Agent Recorded
on the system. Ideally all Dialler Agents should be
equally evaluated in the beginning of a quality
assurance against a particular Scorecard. As Dialler
Agents according to the trifactors calculated for each
agent, the quality assessment can target certain
customer interactions belonging to agents to who are underperforming or are performing exceptionally well, and over time the
distribution of overall total for evaluated customer interactions will be intentionally skewed according to internal business protocol.
The labels of each Dialler Agent displayed correspond to the number of quality assessments evaluated for the agent.

26
OVERALL TOTAL CUMULATIVE № EVALUATED CALLS COMPLIANCE/NON COMPLIANCE RATIO PER DIALLER
AGENT
The Overall Total Cumulative Number of Evaluated
Calls Compliance/Non-Compliance Ratio Per Dialler
Agent stacked bar graphs show the percentage
proportion of customer interactions evaluations that
passed and failed the Compliance Questions for the
labelled Dialler Agent. The proportion of
Failed/Passed customer interactions can be used to
identify agents whom are not sufficiently trained or
are not adhering to compliance dictated by the Campaign Manager or Client. Additionally it allows interventions to be taken should a
Dialler Agent consistently fail on the compliance criteria of the Scorecard, helping deliver a higher quality outcome for our clients. The red
proportion represents failed evaluations; the green portion represents passed evaluations.
The labels of each Dialler Agent displayed correspond to the proportion of Failed/Passed call interactions evaluated for the agent.
OVERALL TOTAL TRIFACTOR DISTRIBUTION PERCENTAGE PER DIALLER AGENT
The Overall Total Trifactor Distribution Percentage Per
Dialler Agent displays a stacked bar graph that show
how each individual Dialler Agent quality scores are
distributed in terms of quality score as determined by
the Trifactor Criteria:
Green represents the percentage proportion of customer interactions that have been evaluated with Quality Scores between
95 and 100% (including 95%)
Yellow represents the percentage proportion of customer interactions that have been evaluated with Quality Scores of 85 up to
95% (not including calls with 85% or 95% Quality Scores)
Red represents the percentage proportion of customer interactions that have been evaluated with Quality Scores less than 85%
(including 85%)
Once can deduce that the agents with higher proportions of green are producing higher quality customer interactions, and similarly
Dialler Agents with higher proportions of red are not producing customer interactions that meet the minimum standards expected.
The labels of each Dialler Agent displayed correspond to the proportion of Failed/Passed call interactions evaluated for the agent.
OVERALL TOTAL CUMULATIVE № EVALUATED CALLS PER DIALLER TEAM
The Overall Total Number Evaluated Calls per Dialler Team bar graph shows the numerical distribution of calls
evaluated for each Dialler Team as recorded by the QA Agents. Ideally upon beginning a quality assurance campaign,
all teams should receive the same number of evaluations, and as time goes by, and customer interactions from the
Dialler Teams can be identified against Quality and Compliance standards, by using the other KPI’s displayed on the
Dashboard (In particular Compliant/Ave Quality Scores (Area Graph)). The internal quality assessment protocol can be

27
enforced and specific customer interactions from particular teams can then be targeted and the number of each Dialler Team’s evaluated
customer interaction can be adjusted accordingly. The image here represents that two teams have received more attention by QA Agents
than the other 4 Dialler Teams.
OVERALL TOTAL CUMULATIVE № EVALUATED CALLS PER QA AGENT
The Overall Cumulative Number of Evaluated Calls Per QA Agent displays a bar graph of the number of quality assessments
under taken by each individual QA Agent who has been allocated onto the Scorecard. This graph clearly shows that most of
the calls assessed in this scorecard have been performed by a particular QA Agent. Ideally the graph should indicate that all
QA Agents are processing the same number of customer interactions.
PIE GRAPHS AND AREA GRAPHS
PERCENT CERTAINTY
Since most quality assessment campaigns can consist of several hundred thousand (or million) customer
interactions it is not possible to evaluate each individual customer interaction, therefore a way of sampling the
total customer interaction pool is necessary to evaluate calls. A statistically accepted model has been included in
the CallQ code to calculate exactly the number of customer interactions required to be evaluated in order to
achieve a prescribed percentage of certainty. Call Quality prescribed that a minimum of 95% certainty needs to
be delivered in order to give our clients information that can be used accurately to support executive decisions.
The image on the left shows the current number of evaluations (sample size) that have been assessed and the
percent certainty (confidence level) of the results of the rest of the dashboard. One can see that from 10000 allocated customer
interactions, 571 quality assessments have been done, which is calculated as 100% confidence of the results on the dashboard. The
minimum number of assessments that need to be done to achieve a 95% confidence result for 10000 interactions is 370. For more
information about these calculations please refer to the Calculators section in this document.
The formulas necessary to calculate the required sample sizes to achieve the minimum 95% percent certainty are listed below:
FORMULA REFERENCE (SAMPLE CERTAINTY)
(confidence) x = Z(c/100)2
r(100-r)
(sample size) n = N x/((N-1)E2
+ x)
(margin of error) E = Sqrt[(
N -
n)
x/n(N-1)]
where:
N is the population size
r is the fraction of responses
Z(c/100) is the critical value for the confidence level c

28
COMPLIANT/AVE QUALITY SCORES
The Compliant/Ave Quality Scores area graph shows the distribution of Compliance Passes (percent of total
evaluations) and Average Quality Scores as distinguished by each team. This graph easily helps with the
identification as whole of how teams are performing in regards to overall performance. To view the
numerical values more clearly the user can click on the image, resulting in a zoomed popup window,
displaying the values in human readable format, as illustrated below.
QUALITY MARKDOWNS
The Quality Markdowns pie graph shows the top 10 Quality
Questions that have been marked by the QA Agent as fails for
the particular question. For example if the Scorecard has 40
questions that can be marked as ‘Yes’ or ‘No’ , the top most
questions marked as ‘No’ will be shown on this graph as a
percentage total of the top 10 ten marked down Quality
Questions. The user can click to image to show a zoom window,
clearly showing the associated values represented the total ‘No’
answers for each question. The user can also use the navigation menu or arrows to scroll into each question in detail to see the break
down of each questions answers as captured by the QA Agents.
From the data represented under Quality Markdowns,
one can deduce which quality aspects of the Dialler
Agent customer interactions need most attention,
helping direct training resources to the area’s most
needing improvement for the purposes of improving
overall dialler campaign customer interactions.

29
COMPLIANCE MARKDOWNS
The Compliance Markdowns pie graph shows the top 10
Compliance Markdowns for the selected Scorecard. In this
example one can see 4 questions displayed, and this can be
attributed to the fact that only 4 Compliance Questions exist or
that from the total pool of compliance questions, only 4 have
been marked with ‘No’ answers, so in this case only the top 4
Compliance Questions are represented. Navigation of the specific question breakdowns can
be done using the arrows or menu. Additionally the user may click on the image to show a
zoomed in view of the data with human readable values.
From the data represented under Compliance Markdowns,
one can deduce which compliance aspects of the Dialler
Agent customer interactions need most attention, helping
direct training resources to the area’s most needing
improvement for the purposes of maintaining an acceptable
compliance as prescribed by the Client.
AGENT RANKING DATA
The Agent Ranking Data rates agents according to the top 5 Dialler Agents and the bottom 5 Dialler Agents as captured by the QA Agents.
The importance of this statistic must be considered in regards to the Overall Cumulative Number of Evaluated Calls Per Agent bar graph,
since the results will alter significantly if as Dialler Agents are targeted more specifically.
This statistic ranks Dialler Agents who have at least 4 quality evaluations in the CallQ system, specific to the Scorecard. At least 10 agents
who have 4 evaluations each need to exist in the system for the ranking system to display any data (minimum of 40 evaluated customer
interactions). The data represented is entered into a formula to generate a ‘Quality Factor’ score, then ranked accordingly, considering tie
positions if applicable. The general formula used to calculate the top 5 and bottom 5 is explained below.
TOP 5 AGENTS AND BOTTOM 5 AGENTS FORMULA REFERENCE
Dialler Agents are ranked according to their ‘Quality Factor’ as can be derived from the following formulas:
passrate = total number of compliance passes/total number evaluated customer interactions
Quality Factor = (passrate*average quality score)/(total number evaluated customer interactions-total number of compliance passes) +0.95

30
DATE RANGE
Upon loading the Dashboard shows by default the last 50 days of activity for the selected scorecard, or
approximately the last two months of working days. In order to allow a further drill down of data according
the date of customer interactions evaluated, the
Dashboard interface includes a date range dialog.
The dialog includes 7 buttons to easily drill down
into the periods represented by those dates,
however the Assessment Start Date and
Assessment End Date can be selected by using the
calendar that displays when the user clicks the
input field, as in the illustration.
Should no data exist for a particular date range
query then the user is advised to change the date
range. When a certain date range is applicable
the dashboard is generated with all the data
represented in the default 50 day report albeit as
a chronologically applicable dataset (Bar Graphs,
Pie and Area Graphs, Agent Ranking Data,
Wordclouds).
WORDCLOUDS
The Dashboard includes a navigation option that
displays automatically generated Wordclouds
that are derived from question answer
templates that contain inputs for freetext
capture by the QA Agents. The Wordclouds
represent the most common words captured in
the free text of the QA Agents assessments. (See
section on Attributes and Wordcloud Reports)

31
FEEDBACK
The Dashboard includes a section for a user to communicate
internally with the Call Quality staffs that are nominated to
receive communication from the Dashboard. The client cannot
see the name of the nominated recipient of communication,
however can view the seniority of the nominated recipient
(reflected as the role they perform on CallQ).
The Dashboard Feedback interface allows a
data consumer to type a message, upload
attachments and request a new question to
be submitted for the selected Scorecard.
File attachments can include any of the
following extensions, limited by a total of 5
attachments: doc,docx,xls,xlsx,ppt,pptx,jpg,
bmp,gif,swf,png,zip,rar,csv,txt,rtf,pdf,tif,tiff,
mp3,wav,ogg,wma,mpeg,avi,wmv,flv,mp4,m
ov
The value of feedback is apparent since it
allows a clear line of communication to
nominated recipients of the communication,
enabling an anonymous and streamlined
communication route to most effectively
deal with day to day running of the customer
interaction campaign as required by the Client.
The Dashboard enables a user to edit and
submit a new question for evaluation,
specific to the selected Scorecard for
which the data is shown for. The interface
is exactly the same in regards to the
Scorecard Builder – Scorecard Questions
sections. Upon submitting the proposed
new question, the data is emailed to a
nominated Data Manager, to be captured
on the Scorecard Builder. It is important to
note that new questions are not
automatically added to the scorecard,
since QA Agent training might be
applicable, and the question may need to
be edited for spelling, correct answer template publication etc. If a new scorecard is selected, it is attached as a file to the email that is
sent to the QA Agent, with a filename that resembles ‘5177A575C7E.callq’. The file is a proprietary format that can only be opened in
CallQ.

32
CONFIGURATION MANAGER
The Configuration Manager is an important feature of CallQ since it allows for the management of the Users, Teams and Scorecards. The
business model captured by CallQ involves a hierarchical role (user) structure, each role having access to different parts of the application
thereby ensuring that the only the correctly assigned users have access to only the part of the application that they need to do their job.
The Configuration Manager consists of seven different sections as illustrated below:
TEAM/AGENT/SCORECARD CONFIGURATION
The Team/Agent/Scorecard Configuration serves the purpose of mapping applicable Scorecards to their denoted teams and users. The
form allows any of the available scorecards to
be assigned to any QA Team that has been
configured in the system.
The form consists of a list of configured teams,
as a menu button; upon clicking the Team Name
the Available Scorecards and Assigned
Scorecards can be seen. The Available
Scorecards are listed according to the
Scorecards that have been configured as visible
on the Scorecard Configuration Form (also part
of the Configuration Manager).
If Scorecards have already been assigned to the
QA Team, the Assigned Scorecards are visible in
this list. Assigned Scorecards are available to the
team members of the applicable team, either to
be assigned individually or to the whole QA
team, by checking the radio button for each
user as listed in the Team Members section of
the form, specific to the team.
In order for the Configuration Manager to
assign/remove Scorecards, either from the
Available Scorecards list to/from the Assigned
Scorecards list, or from the Assigned Scorecards
list to/from and individual user, the
Configuration Manager is required to double
click the text in the applicable box. If the ‘All
Team Members’ radio button is selected, then
the Assigned Scorecard will be assigned to all
team members in that team. If a team member
is transferred to another team, all scorecards
need to be re-assigned accordingly.
The tool bar for each Team includes an icon that allow the Configuration Manager to clear all individual and team scorecard assignments,
and an icon that allows the configuration manager to deselect the currently selected team member. The team members displayed on the
Team Member List will always be QA Agent users who are currently employed.

33
CLIENT/DASHBOARD CONFIGURATION
The Client/Dashboard Configuration form
facilitates the creation of a new Client Entity on
CallQ. The Client represents an important data
consumer of the Dashboard; Client Users
configured for the Client inherit the access
privileges given to the Client, in regards to
which dashboards are assigned to each Client.
CallQ caters for a Client to have several
Dashboards at their disposal, fulfilling the
business requirement stipulated that a Clients
customer interactions may need to be assessed
using a number of different scorecards since
each Client may have at least one product or
customer service campaigns running
concurrently.
In order to assign a dashboard to a specific
Client (access is cascaded to Client Users
configured for the Client), the Configuration
Manager must double click the Dashboard title
in the list of Available Dashboards to add it to
the Assigned Dashboards list. Similarly to
remove access to a Dashboard, the
Configuration Manager must double click the
Dashboard title in the Assigned Dashboards.
The Client/Dashboard Configuration includes the following fields to be populated:
Client Name - the name of the company
Contact Name – the name of the person who is most relevant for communication from Call Quality (this is not a user on the
system necessarily)
Contact Email – the email of the primary contact person
Contact Phone – the primary phone number for the contact person
Client Country – for internal statistical purposes
Website – for internal information purposes
Industry – for internal statistical purposes
Color Theme – the colour theme that the clients Dashboard is themed by, usually representing company colours of the client

34
CLIENT USER CONFIGURATION
The Client User Configuration Form facilitates
the creation of new Client Users for a particular
Client. Client Users inherit the permissions of
the Client they are configured for; additionally
Client users have no access to any of the other
pages in CallQ. Should a Client user try to
navigate off the page by manually altering the
URL, they are automatically redirected to the
‘Select Scorecard’ dialog of the Dashboard.
Once a Client User is added on to the system,
the Client User can be seen on the front end,
with the associated data for that user.
CallQ automatically generates a login name for
client users, with the general formula
‘firstname_lastname*_number+’. A number is
appended to the automatically generated
username should the username already exist in
the WordPress users.
The Client User Configuration includes the following fields to be populated:
Password – password the user will login with, the user can change their own password on the system. The password is securely
stored in the database using native MD5 encryption.
Organisational Level – useful to determine what level of the Client Company hierarchy a specific user belongs to.
Title – indicates gender, marital status or education
Firstname
Lastname
Photo – a file of the user image that can be used to identify clients graphically
Client – the Client to which a specific user belongs, and from which the user inherits access permissions to the Assigned
Dashboards for the client.
Contact Number
Email Address – must be a unique email address in the WordPress installation, an email is sent to this address when the user is
configured on the system, for the Client User records (login details).
Notes – any notes about a particular client that may be useful when corresponding with the client or communicating via phone.

35
MANAGER USER CONFIGURATION
The Manager User Configuration Form allows
the Configuration Manager to create users that
perform management roles within CallQ. Each
role has specific access to the application as
documented in the User Roles section of this
document. Management Roles that can be
inherited when creating a new user include:
Campaign Manager
Configuration Manager
Data Manager
Director
The CallQ login for Manager Users is generated
automatically. For Directors the username
conforms to the general format
‘firstname_lastname*_number+’, for Campaign
Managers, Configuration Managers and Data
Managers the username conforms to the
general format ‘CALLQ####’ where # represents
a 4 digit number, if less than 1000, with zero
placed holders for the three left most numbers.
The following fields must be populated when a new Manager User is created:
Password - password the user will login with, the user can change their own password on the system. The password is securely
Organisational Level – The level of seniority of users within the CCL/Call Quality staff compliment
First Name
Last Name
CCL Login – the login of the user on the CCL LAN Domain (or employee code)
Photo – a file of the user image that can be used to identify users graphically
User Type – what type of management user is the new user
Team Name - the team that the Manager User belongs to, if the Team Name is not already in dropdown list, the Team can be
added as the user is created
Contact Number – the phone number that the staff member can be contacted by
configured on the system, for the Manager User records (login details)
Notes – any notes about a particular user that may be useful when corresponding with the user or communicating via phone

36
QA USER CONFIGURATION
The QA User (Agent) Configuration form allows a
Configuration Manager to add a QA Agent to the
WordPress installation. QA Agents are unique on
the system since QA Agents are the only users
that can capture valid Scorecard Data that can
be reported on.
When a new QA Agent is created on the System
the QA Agent is assigned to a QA Team, and
inherits scorecards that have been assigned to
all team members in that team if applicable. For
QA Agents an automatically generated
username is created for login, conforming to the
format ‘CALLQ####’ where # represents a 4 digit
number, if less than 1000, with zero placed
holders for the three left most numbers.
The following fields must be populated when a new QA Agent user is created:
Password - password the user will login with, the user can change their own password on the system. The password is securely
Organisational Level – The level of seniority of users within the CCL/Call Quality staff compliment
First Name
Last Name
CCL Login – the login of the user on the CCL LAN Domain (or employee code)
Photo – a file of the user image that can be used to identify users graphically
Team Name - the team that the QA Agent user belongs to, if the Team Name is not already in dropdown list, the Team can be
added as the user is created
Contact Number – the phone number that the staff member can be contacted by
configured on the system, for the QA Agent records (login details)
Notes – any notes about a particular user that may be useful when corresponding with the user or communicating via phone
SCORECARD CONFIGURATION
The Scorecard Configuration form allows the
Configuration Manager to hide and show
Scorecards/Dashboards configured on the
system. If the Scorecards box is unchecked this
indicates that the specific Scorecard is not
available to any user on the CallQ installation,
accept for Configuration Managers and
Administrators. The feature ensures that only
the most up to date and relevant scorecards can
be made available for QA Agents and other
users (including users whom are authorised on
the Dashboard and Attribute and Wordcloud
Report), verifying that the activities undertaken
in CallQ are currently applicable to the Call
Quality business objectives at the time.

37
FILENAME DECODER
CallQ produces a number of different types of exports for various different User Roles in the system. In order to guarantee that all files are
unique in system, each file named by default with an encoded file name, uniquely generated for the file.
CallQ also recommends a human readable filename, these file names are longer and provide a description of the contents of the file,
however may be impractical for many users. In order to allow a user to recover useful information from an encoded filename, the
filename decoder shows information including the following:
System Recommended File Name – a longer descriptive version of the contents of the file
Page – the page of CallQ on which the file was generated
Link – a link to the file as stored on the server (for data redundancy)
Created Time – the time the file was created
User – the user who created the file
Filter – if applicable, the dynamically generated filter that was applied when the file was created (applies to Attributes and
Wordcloud Reports)
Number of Records – if applicable, the number of records in the file (applies to .csv and .xlsx files only)

38
CALCULATORS
STATS MODEL CALCULATOR – FLASH
The Stats Model Calculator – Flash page in CallQ displays a number of flash calculators for calculating various critical numbers relating to
sample certainty, volumes and staff compliment. The calculators on this page are compiled as Flash Applets (.swf) and the browser must
be updated with the Adobe Flash Plugin in order to be viewed.
PERCENT TO BE AUDITED
The Percent to be Audited Calculator calculates
the percent of the total sample required to
achieve the entered certainty, which is 95% by
default.
The calculation allows for two inputs for total
allocation size, namely sales calls and non-sales
calls and calculates the actual number of
required sample size of each stream of input and
the percent representation of those numbers
from the total allocation size in the analogue
gauges on the right.
Additionally the calculator calculates the number
of required QA Agents in order to process the given sample size according to the call length and the Average handling time. The number
represented here is the sum of the QA Agent required for Sales calls and Non-Sales calls.
SAMPLE CERTAINTY
The Sample Certainty Calculator calculates what
certainty can be achieved from a specified
sample size from a total allocation size.
For more information about the mathematics of
the calculations, please refer to the Formula
Reference (Sample Certainty) section of this
document.
Additionally the calculator calculates the number
of required QA Agents in order to process the
given sample size according to the call length and
the Average handling time. The number
represented here is the sum of the QA Agent
required for Sales calls and Non-Sales calls.

39
SAMPLE CERTAINTY COMPARISON
The Sample Certainty Comparison Calculator
calculates the current sample certainty of a
specified number of samples, compared with
required number of samples in order to achieve a
95% certainty.
The calculator also calculates the number of QA
Agents required to perform assessments in
relation to the campaign complexity and
interaction length as Low, Medium and High
buttons.
The animated dial on the right of the calculator shows the Sample Certainty as entered compared to the prescribed 95% certainty for Call
Quality assessments.
STAT MODEL ASSESSMENT CALCULATOR
The Stat Model Assessment Calculator is a
JavaScript calculator that does all volume and
certainty calculations needed to ensure that
quality standards are adhered to in terms of
correct numbers of assessments being used. This
calculator is the most comprehensive calculator
necessary for the internal business processes at
Call Quality.
In addition to the basic functionality need to
calculate certainties and volumes, the revealed
‘Recommended Values’ section allows the user
fine tune the calculated sample size considering
margin of error, population size, confidence
percent and response percent.
The calculator also allows for entry of detailed
parameters related to the calculation of staff
allocation required to perform assessments. The
calculation includes fields for entry of the
following:
Agent Performance Split
QA Daily Productive Hours
Average Sales Call Length
Average Non-Sales Call Length
QA AHT (Average Handling Time)
QA Working Days (Week)
The estimate of the number of QA Agents
required is also broken down into a high,
medium and low performers (referring to QA
Agents productivity) and how many calls should
be allocated to each class of QA Agent.
For more information about the mathematics of the calculations, please refer to the Formula Reference (Sample Certainty) section of this
document.

40
TREND ANALYSIS TOOL
The Trend Analysis Tool is a tool for extracting the meaning of data and representing it as human readable text that can be quoted in other
written text. The Trend Analysis Tool is a powerful means of analysing data for trends. The Trend Analysis tool can be used either for ad
hoc analysis of data from a Data Catalog (refer to Dynamic Import section of this document) or data collated from a specific Scorecard. The
tool involves a brute force analysis algorithm that calculates all possible permutations of a specific data set (up to 5 columns at a time),
and numerically evaluates the frequency of each permutation in a given data set, allowing for the extraction of the most significant
permutations to be identified as influencing the result of a selected ‘ordinal column’.
The brute force calculation currently uses large amounts of memory and increases disk activity and is restricted for use to the
Administrator, Director and Data Manager roles in CallQ.
COMBINATOR – SELECT ORDINAL COLUMN
The Combinator requires the user to select the
‘Ordinal Column’ that a trend needs to be
calculated for. Typically the Ordinal Column
should be the information the user is seeking to
answer a question about. For example, if one
wants to find the most significant influencing
factors that relate to ‘Call Outcome’, one would
select that column.
The columns displayed for selection of an
ordinal column are defined by their distinctness
of the data in those columns. The columns that
can be selected for an ordinal column contain up
to ten different values, and columns with free
text are ignored.
For example, using the ‘Call Outcome’ column in
this data set, this column has only 6 possible
values, as can be seen in the image below.
The data can be analysed for specific outcomes
by checking/unchecking the values represented for the column. A significant performance advantage can be achieved by analysing only
the necessary data.

41
COMBINATOR – SELECT COMPARISON COLUMNS
Once an ordinal column is selected the user is required to select columns to compare the ordinal values against, in order to calculate all
possible permutations for the selected columns. Although all columns can be selected for analysis it is highly recommended that the
fewest possible columns are selected for analysis. For the user to select the correct columns that may relate most significantly to the
ordinal column selected, the user should have an in depth knowledge of the data being analysed.
The Comparison columns incudes a section for
selecting ‘triplets’ and ‘quads’ by checking the
appropriate checkbox, to refine the trend
extracting methodology to examine 4 or 5
columns respectively.
If required the user may examine the results in
detail by unchecking the ‘Do not show workings’
check box, which will result in a tabular graphical
display of all permutations identified and the
frequency of those permutations in the data set
examined.
Upon selecting the Comparison Columns the user
can use a dialog such as the one below to specify
which values to analyse for trending. By default
all values in each column are selected, except for
blank or null values.
Significant performance gains can be made
depending on how the value options are
configured and the fewer values that are selected
the better the performance will be.
One must keep in mind when comparing the
comparison values that significant trending for
values that are not checked will not show in any
of the analytic results, therefore the results must be appreciated against the original dataset mindful that other trends may exist that are
not displayed.
By default, if triplets or quads are not selected, the trend analysis algorithm will only compare two columns against each value in the
selected ordinal column. By calculating all permutations of all paired columns selected on the Comparison Columns form and comparing
them against all selected values of the ordinal column, the frequency of each permutation is counted in the original data source and
tabulated into a trend extraction.
COMBINATOR – TRIPLETS
Similar to the default pair comparison, the triplet comparison evaluates trends by calculating all permutations of the comparison columns
selected (and the values selected for those columns), in sets of three, and comparing against the ordinal column. The frequency of each
permutation is counted and tabulated for trend extraction. In order for triplet trending data to be displayed at least 3 columns must be
selected from the Comparison Columns.

42
One must note the processor and memory overhead incurred when processing triplet trending is significantly greater than it would be for
pair trending in a specific data set.
COMBINATOR – QUADS
Using the same methodology as pair and triplet trending, quad trending reveals trends involving permutations of 4 columns against the
selected values of the ordinal column. Similarly it must be noted that the processor and memory overhead is significantly greater than that
of the pair and triplet trend extractions. The user should be aware that analysing trending can take a long time. In order for quad trending
data to be displayed at least 4 columns must be selected from the Comparison Columns.
TREND FINDINGS
The results from the Combinator show the
trends identified as human readable text in plain
English, an optionally as tabular data. If either or
both of the ‘Do Triplets’ or ‘Do Quads’
checkboxes have been checked and the
minimum respective number of columns have
been selected the trending is displayed
respectively, according to algorithmic extraction.
The trends are shown in the Pair, Triplet and
Quad findings sections respectively. The output
displays all trending information that has a
minimum frequency of 2.5% of the total data
set, displaying the top three most significant
results in respect to the ordinal column values
selected.
If no trending information is compiled by the
algorithm, then where applicable the text ‘No
Trends Found’ will be displayed.
The image displayed here shows the fully
expanded version of the Trend Analysis tool
results; the interface conveniently includes
collapsible sections to facilitate a more user
friendly interaction by allowing the user to only
view the results they are interested in.

43
DYNAMIC IMPORT
The Dynamic Import feature of CallQ allows a Data Manager or Administrator user to upload a .csv file for ad hoc analysis and data
cleansing operations.
The Dynamic Import
feature works by
using a special
algorithm to
recognise what type
of data is in each
column of the
uploaded .csv file,
assigning a ‘Data
Expression’ to the
column, and applying
data cleansing rules
specific for the type of
data in each column. Upon import a table is created in the database, with column data types corresponding to native MySQL data types.
The Data Expression algorithm evaluates the data using regular expressions, differentiating between fields such as person names, dates,
long text and short text. In order to standardize in disparate data (for example data that contains dates in various formats), the cleansing
algorithm includes features that standardize date/times, remove trailing/leading/extra spaces, apply correct casing to person names,
replace empty text with null values and correct decimal numbers that have commas to have points instead. The underlying database
supports Unicode encoding for text fields (UTF-8 collations) where applicable, supporting characters that are not found in the English
alphabet.
IMPORT PRE-ANALYSIS
The import pre-analysis applies a ‘top-values’
analysis to show the top 10 most frequent
values in qualifying columns of the imported
CSV file.
The top 10 most frequent values are shown in
order of the frequency, displaying the actual
count of the particular value, and the percent of
the total those values represent of the total row
count in the .csv file. The graphic representation
is shown as a pie graph.
The result of the pre-analysis also includes
summary of the import details and the
corresponding table in the database. The Table
Data Summary displays information as listed
below:
Column Name (the original name of the
column in the csv file)
Serial Name (the internal system name of
the column)
System Data Type (MySQL data type)
Data Expression (name, email, integer,
decimal, phone number, website, text
select, short text, long text)
Unique Value Count
Non-blank Value Count
Blank Count
Index Type (none, reference (clustered
index), text search (full text index))

44
The Data Catalog produced by the Dynamic Import can be appended to by importing another .csv file that exactly matches the format
(column names, number of columns and data expression) original Data Catalog produced in the initial data import.

CallQ scope and user specification summary

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (16)

Destaque

Destaque (8)

Semelhante a CallQ scope and user specification summary

Semelhante a CallQ scope and user specification summary (20)

Mais de MakeNET

Mais de MakeNET (11)

Último

Último (20)

CallQ scope and user specification summary