1. FEATURES, FUNCTIONS, ARCHITECTURE
OF BIGDATAREVEALED FOR DATA
PROTECTION, COMPLIANCE AND
DATA MANAGEMENT ESPECIALLY FOR
GDPR AND CCPA
- A SINGLE BIG DATA APPLICATION, (APACHE/HDFS, AMAZON HDFS AND AWS S3,
AND MICROSOFT AZURE) BUILT TO MANAGE BIG DATA AND ALL DATA.
- DELIVERING REPEATABLE, COLLABORATIVE, AUTOMATED, AND ACCURATE RESULTS
FOR ALL INDUSTRIES TO MEET REGULATORY COMPLIANCES SUCH AS GDPR, CCPA,
HIPAA, AND OTHERS, NOW AND THOSE THAT WILL APPEAR IN THE NEAR FUTURE.
- DATA PRIVACY AND PROTECTION USING THE LATEST TECHNOLOGIES &
LANGUAGES.

2. DATA PROTECTION IMPACT ASSESSMENTS – THE ABILITY TO FIRST UNDERSTAND
YOUR CORPORATE DATA ASSETS, CREATE META DATA CATALOGS TO NAVIGATE
AND REMEDIATE A COMPANIES DATA ASSETS, CREATE A CORE FOUNDATION
FOR THE REST OF DATA COMPLIANCE INCLUDING REMEDIATION AND AS
NEEDED FOR COMPLIANCE AND ANALYTICS TO AVOID REGULATORY
HEADACHES. HERE IS HOW BIGDATAREVEALED DELIVERS:
• BigDataRevealed (BDR) is delivered with an extensive pattern recognition engine
and library that users are able to augment. Contains patterns of Personal
information, Regulated Information or any pattern the company desires to search.
BDR will also search for static values such as an individual’s name, email, phone …
• Once this process has completed, BDR will display results allowing staff to
collaborate and select Remediation measures for legacy and Big Data environments
via Encryption and other security measures.
• This Information will become the core nucleus for most of the following features,
functions, and ability to deliver the mass requirements of the Regulatory world.

3. OUR MANY YEARS OF EXPERIENCE HAVE TAUGHT US TO NEVER INTERFERE
OR RUN LABOR INTENSIVE PROCESSES AGAINST OPERATIONAL SYSTEMS,
ESPECIALLY IF IT NEGATIVELY EFFECTS THE CUSTOMERS EXPERIENCE WITH
YOUR COMPANY. TO MINIMIZE THE IMPACT ON YOUR OPERATING SYSTEMS,
AND TO FACILITATE CERTAIN DIFFICULT REGULATORY REQUIREMENTS THAT
CANNOT BE MET ACROSS VARYING DATABASES AND OPERATING SYSTEMS,
BDR HAS ARCHITECTED AN ECOSYSTEM BUILT IN APACHE HADOOP THAT IS
QUICK TO INGEST DATA, ABLE TO ACCURATELY STORE FOLDERS AND GROUP
THEM IF DESIRED, PROTECT THE DATA WHILE INGESTING DATA STREAMS, AND
MOST IMPORTANTLY, ABLE TO DELIVER ON THE MORE DIFFICULT
REGULATORY REQUIREMENTS AND DO SO WITH MUCH GREATER ACCURACY
AND TIMELINESS.
• Metadata created above will be used to deliver Compliance results that your Company and its
Customers need.
• BDR can become the new Central Repository for your Companies complete Data Assets including;
• Legacy Data
• OCR
• Office, PDF and .pst email Data
• Biometrics Data
• And most any other forms of data

4. DATA PROTECTION IMPACT ASSESSMENTS – RESULTS FOR EVALUATION OF
REMEDIATIONS SUCH AS:
• Determine, by viewing Metadata, which exposed Personal Data columns
should be immediately Encrypted, Sequestered, or even eliminated.
• Allow collaboration and communication between stakeholders, data
management team members, Executive levels, DPO, ETL and others regarding
their opinions and reasons for the data to be;
• Available for production, point of sale or ongoing analytics
• If Point of Sale or Customer Service applications need to access encrypted consumers PII,
BigDataRevealed offers APIs (Spring framework) to decrypt a single customer’s data for
display and then re-encrypt any additions or changes made before updating the databases.
Your production applications are never interrupted or degraded and your database is secure.
• Only a single customer’s data is exposed during this process.
• Share and integrate the BigDataRevealed MetaData Catalog with your Companies
existing metadata systems.

5. OTHER IMPORTANT VALUES OF THE BIGDATAREVEALED METADATA
CATALOGS FOR THE PROTECTION OF PII IN YOUR LEGACY SYSTEMS.
• The ability of BDR Metadata to store all Files, Columns and Rows coupled with the
type of run and action performed allows for:
• The ETL, Data Management, DPO, Compliance team and others to determine what legacy
data may need to be:
• Deleted
• Encrypted
• Sequestered
• Evaluated in more detail
• Reported to Consumers that requested how their data is being used and what data of theirs you
have stored and why
• BigDataRevealed then allows these processes to be performed using its library of
200 Spring Framework API’s. These API’s can be stand alone, within operational and
production applications, or most anywhere they are needed. They are all written in
Spark / Java for flexibility, speed, scalability and again ease of use through the
Spring Framework.

6. OTHER IMPORTANT VALUES OF THE BIGDATAREVEALED INGESTION OF
LEGACY DATA AS WELL AS IOT AND STREAMING DATA.
• The ability of the BDR Spark Java Ingestion API, it also reads and processes IoT,
Social Media and other forms of streaming data and can:
• Determine PII Data, on the fly
• Process or notify other systems of the PII
• Determine PII through its pattern recognition and NLP Engines and based on earlier user
parameters and requests;
• Encrypt this PII data before it resides in data at rest or in other systems
• Delete this PII data if it matches a Customer that has Opted-Out
• Creates the metadata of its findings for later use
• Best determine file folder names for the files and …
• Do not allow re-inclusions of PII data into your data ecosystem for a customer that has
‘Opted-Out’. While you process IoT, Third Party data streams, or any other streams of data
assets, BDR can isolate information for these customers and keep their data from re-entering
your environment, which would be a compliance violation.

7. THE REASON BIGDATAREVEALED USES HBASE IS PRIMARILY FOR CONTINUAL
IOT AND STREAMING DATA TO ALLOW CONTINUAL READING OF THE
RESULTS. HDFS DOES NOT ALLOW READING OF SUCH RESULTS TILL A JOB IS
COMPLETED. WE ALSO USE HBASE FOR QUICK RESPONSE TO OUR
SUMMARIZED CHARTS, GRAPHS AND GAUGES.
• HBase adds speed and flexibility for BDR queries and drills into data for a
more granulated view.
• Hbase along with MySQL is easier for storing various summaries of our
processes and provides faster and easier retrieval, especially from third
party applications that may not read HDFS efficiently or at all.

8. THE ‘SEARCH CUSTOMER FEATURE’ SHOWS HOW BIGDATAREVEALED CALLS
THE SPRING FRAMEWORK API’S TO FIRST SEARCH FOR CUSTOMER DATA
STORED IN NON ENCRYPTED FORMAT. IF IT DOES NOT FIND A MATCH IT WILL
THEN FIND THE SECURED ENCRYPTION KEY, ENCRYPT THE INPUT DATA AND
SEARCH FOR THE CUSTOMER DATA USING ENCRYPTED VALUES. IF IT THEN
FINDS THE DATA, IT WILL TEMPORARILY DECRYPT THE DATA, PASS IT TO THE
OPERATIONAL SYSTEMS FOR NORMAL UNINTERRUPTED PROCESSING AND
THEN MAKE SURE THE DATA IS RE-ENCRYPTED ONCE AGAIN AND DELETE ALL
UNENCRYPTED DATA FROM MEMORY. THIS IS USEFUL AND IN MOST CASES
NECESSARY FOR CONTINUAL UNINTERRUPTED POINT OF SALE, CUSTOMER
SERVICE AND OTHER CUSTOMER RELATED OPERATIONS, PRODUCTION
SYSTEMS AND FOR DATA SCIENTISTS AND DATA MANAGEMENT PEOPLE TO
GET TEMPORARY ACCESS TO ENCRYPTED DATA AS NEEDED AND AS THEY
HAVE AUTHORITY TOO.
POINT OF SALE – WE CAN ACCEPT NON-ENCRYPTED DATA FROM A
CUSTOMER AND SEARCH FOR IT’S ENCRYPTED VALUE IN DATABASES. WE
THEN ARE ABLE TO DECRYPT THE RECORDS FOUND AND RETURN THE
INFORMATION TO THE POINT OF SALE OR CUSTOMER SERVICE APPLICATION.

9. AFTER VARIOUS PROCESSES HAVE BEEN RUN, SUCH AS THE DISCOVERY OF
PII, DETERMINING WHICH DATA FIELDS NEED REMEDIATION, SETTING UP
PROCESSES FOR INGESTION OF IOT AND SOCIAL MEDIA STREAMS,
BIGDATAREVEALED HAS A SYSTEM TO ASSIGN TASKS TO INDIVIDUALS AND
TO MONITOR THE PROGRESS OF THOSE TASKS VIA INTERACTIVE NOTE
GENERATION.
• The results of these tasks can be shared at the file levels within the
BigDataRevealed Metadata Catalog interface to support collaborative efforts.
• The assignee’s can request additional assistance, reject the task and even assign
the task to another individual.

10. BIGDATAREVEALED PROVIDES MANY TYPES OF PROCESSES THAT RUN
AGAINST A VARIETY OF DATA ASSETS AND DATA TYPES. LETS LOOK AT SOME
OF THESE.
• BigDataRevealed runs its Discovery of PII Patterns, fixed values, indirect identifiers,
encrypted data for these and many other file types of data:
• Hadoop HDFS, Hbase
• Amazon AWS S3
• Data At Rest
• Semi-Structured Data
• Streaming Data from Company, Social Media, Third parties and …
• Office Documents, PDF’s, OCR, Text, .PST email files and many others ,,,
• BigDataRevealed has it’s own OCR capability for PII Discovery and other processes.
• BigDataRevealed has its own Biometrics Engine to search for Facial Recognition,
Object Recognition, Finger Prints and much more …. Using the same open source
OpenCV as Google.

11. BIGDATAREVEALED PROVIDES A VARIETY OF PROCESSES THAT RUN AGAINST
A VARIETY OF DATA AND DATA TYPES. LET’S LOOK AT SOME OF THESE RUN
PROCESSES:
• BigDataRevealed runs its Discovery of PII Patterns and fixed values
• BigDataRevealed run a process called Quick Business Classification that:
• Identifies all User Selected PII Patterns and lists for each Column in the File:
• How may times and what percentage of a file’s columns contained each of the Discovered
Patterns.
• Allows the User to Drill down into the data to the HDFS system Single source of record to view
the surrounding data to identify the context of the PII and how it is used and if it is a false
positive or not
• Data Discovery Delivers
• See Next Page

12. BIGDATAREVEALED PROVIDES A VARIETY OF PROCESSES THAT RUN AGAINST
A VARIETY OF DATA AND DATA TYPES. LET’S LOOK AT SOME OF THESE RUN
PROCESSES:
• Data Discovery Delivers:
• Stores a list of all the unique data patterns and values in a file
• Allows for the other processes to have the information needed to accurately drill to the system source of record
• Show various data profiling stats such as;
• Duplicates
• Zeros/nulls
• Percentage of the file containing each pattern
• Minimum, Maximum and average of numeric values and …
• Outlier Discovery
• Based on the mean and average values of numeric data:
• Find within the user selected ranges what values are considered Outliers for:
• Anti Money Laundering
• Parts about to break down
• ….
• Indirect identifiers – See next page

13. BIGDATAREVEALED PROVIDES A VARIETY OF PROCESSES THAT RUN AGAINST
A VARIETY OF DATA AND DATA TYPES. LET’S LOOK AT SOME OF THESE RUN
PROCESSES:
• Indirect identifiers:
• BDR discovers groups of fields that by themselves do not identify an individual, or very small
group of individuals, but when grouped together will identify a person or small group of
people. Such as Zip Code, Date of Birth and Gender…
• Processing, Discovery and Searching of .PST Outlook email files:
• Drill and search the results after Pattern or Value discovery
• Search the same for and view the same for email attachments
• ….

14. BIGDATAREVEALED ALLOWS FOR THE USERS TO EASILY SETUP AND RUN ALL
THE AVAILABLE PROCESSES THAT HAVE BEEN DISCUSSED IN THIS
POWERPOINT. THE USERS CAN ALSO;
• Add to, or modify the pattern search library
• Add any values they desire to be searched for, particularly to process a Citizens Right of
Erasure request (Right to be forgotten).
• The ability to download and view the BigDataRevealed Metadata Catalogues, or to
disseminate to others and integrate into other metadata tools.
• Use the Data Scientists Workbench or the new more executive style dashboard
• …

15. BIGDATAREVEALED OFFERS A MEANS FOR CITIZENS TO CREATE THEIR OWN
NEW AND UNIQUE IDENTITIES FOR THE PURPOSE OF INTERACTIVE REQUEST
PROCESSING:
• Request to delete, add or modify Consent of the use of the Citizens PII
• Request the Citizens Right of Erasure
• Request one of the many Regulatory Rights of Citizens found in both GDPR and CCPA
such as;
• What data of mine do you have
• How are you using my data
• To whom have you disclosed my data
• Allow for bilateral communications between the Citizen and the Company
• Using the Citizens PII submitted with their request, BDR automates the Discovery process of
their PII and thereby expedites the entire process.
• …

16. BIGDATAREVEALED OFFERS A MEANS FOR CITIZENS TO CREATE THEIR OWN
NEW AND UNIQUE IDENTITIES FOR THE PURPOSE OF INTERACTIVE REQUEST
PROCESSING:
• Allow a Company to build a central processing site for all Citizens and companies to
communicate and fulfill required Regulations. Thereby making it easier for the Citizen to
make requests of many companies, and for many companies to have one location to pick
up all their Citizen requests.
• Reach us at 847-440-4439