SlideShare a Scribd company logo
1 of 16
Conquer Reporting By Scaling Out SQL Server




                        January 16, 2013
My Contact Information
Anthony Sammartino MCITP(BI), PMP®
Vice President
TekPartners Business Intelligence Solutions
(954) 282-1798
asammartino@tekpartners.com

Twitter: @SQLAnt

Linkedin: http://www.linkedin.com/in/apsammartino

Blog: www.sqlant.me

Web: www.tekpartnersbi.com
About Me
• Started working with SQL Server 7.0 in 1999


• 14 Data Warehouse / BI Projects in 13 Years


• 11 Years SQL Server 2000, 2005, 2008(R2), 2012


• Live in South Florida with my wife and two daughters


• Fan of Miami and Philadelphia Sports teams, all things technology


• Solutions Architect, Project Manager, Technical Lead
  MCP(SQL Server), MCTS(BI), MCITP(BI), PMP®
Session Overview
1. Case for Scaling Out SQL Server for Reporting
2. Choosing a Method to Scale Out SQL Server
3. Architecture with Reporting Scaled Out
4. Replication Server Roles
5. Replication Agents
6. Distribution Database
7. Publications, Articles, Subscriptions
8. Partitioning
9. References for Replication
10. Questions
The "Situation"
Problem Statement:
A company named (House Widgets Inc.) has a main application that sells
widgets online. Every time the Sales staff runs reports to see how many
widgets were sold the application slows, and even times out.
Sales staff complains their reports are slow. Users complain application is slow.

Background:
• DBA team has tuned the reports (T-SQL) through execution plans.
• They also verified its using the correct indexes (Seeks).
• They even resorted to dirty reads using isolation level and nolocks in the code.
• Deemed the data is just to large and too busy. Thousands of users are trying to access
   it, change it at the same time.
• Solid State Disks were even thrown at it. Helped for a short time.

Resolution:
• Time to think about separating the databases for the application and reporting.
• Many ways to scale out SQL Server for reporting.
• Many variables to think about.
Choosing a Method to Scale Out
Requirements
• Requirement is to have real time reporting.
• Requirement is to only report on half of the db tables.

Transactional Replication…Why?
• Transactional Replication happens in near real time.
• Once setup, transactions flow.

Other types of Replication
• Snapshot: Point in time. For example once a day at 9am.
   Not Near real time.
• Merge: Near real time but ideal for situations where two dbs
   need to be kept in sync with writes in two locations.
   Example: Bank with two branches in NY, FL.

Database Mirroring
• Mirroring is very easy to setup and maintain.
• It’s a whole database or nothing solution.
• Point in time using database snapshots.

Always On Availability Groups *NEW SQL Server 2012
• Near real time transactions.
• It’s a whole database or nothing solution.
• Enterprise Edition needed $$$ on both Source and Replica.

Service Broker
• Near real time transactions.
• Harder to setup and maintain, more suited for transforming data.
Architecture with Reporting Scaled Out to a Replica
                         Application (HouseWidgets.com)




       SQL Server (SQLVM03)                          SQL Server (SQLVM03SUBSCRIBER)
             Publisher                                    Distributor & Subscriber



           Application                                                   Application DB-
              DB                                          Distribution      Replica
                                     Transactional
                                      Replication                                  Read Only
Replication Server Roles
Publisher:
The publisher is the source SQL Server that contains database objects that you
wish to replicate. In this case SQLVM03.


Distributor:
The distributor is the server that you wish to setup to “distribute” transactions
from the publisher to the subscriber server.


Subscriber:
The Subscriber is the server that hosts the replica database and where the
reporting will be directed from the application. In this case SQLVM03Subscriber
Secret Agents…Man
• Replication creates a number of Agents.

• These Agents run as jobs scheduled under SQL
  Server Agent.

• SQL Server Agent must be turned on.

• Recommend restarting options be checked, and
  configure SQL Server Agent with a service account
  with access to both machines.

• Replication agents can be administered from SQL
  Server Replication Monitor and SQL Server
  Management Studio.
More Secret Agents
Snapshot Agent
• Used with all types of replication.
• It prepares schema and initial data files of published tables and other objects.
• Records information about synchronization in the distribution database.
• The Snapshot Agent runs at the Distributor.

Log Reader Agent
• Used with transactional replication.
• Moves transactions marked for replication from the Publisher to the
  distribution database. Each database published using transactional replication
  has its own Log Reader Agent.

Distribution Agent
• Used with snapshot replication and transactional replication.
• It applies the initial snapshot to the Subscriber and moves transactions held
  in the distribution database to Subscribers.
• The Distribution Agent runs at either the Distributor for
  push subscriptions or at the Subscriber for pull subscriptions.
More Secret Agents
Merge Agent
• Used with merge replication.
• It applies the initial snapshot to the Subscriber and moves and reconciles
  incremental data changes that occur.
• Each merge subscription has its own Merge Agent that connects to both the
  Publisher and the Subscriber and updates both.
• The Merge Agent runs at either the Distributor for push subscriptions or the
  Subscriber for pull subscriptions.

Replication Maintenance Jobs
Replication has a number of maintenance jobs that perform scheduled and
on-demand maintenance.
Distribution Database
Distribution DB
The distribution database holds all of the transactions from the publisher
database and applies them to the subscriber.

Performance Tip
As a general rule of thumb for smaller publisher database
implementations small in size, small number of concurrent users, small number
of transactions, place the distributor and subscriber on one server separate
from the publisher.

For larger database implementations large in size, large number of concurrent
users, large number of transactions, place the distributor on its own server and
the subscriber on its own server, both separate from the publisher.

Always separate the distributor and subscriber from the publisher.
Your goal is to take load off the publisher with this architectural solution.
Publications, Articles, Subscriptions
Publications
A Publication is what is created on the Publisher server and stores articles that
will be replicated.

Tip: One table per publication, especially if they are large. Re-snapshot.


Articles
Tables, Stored Procedures, Views , Indexed Views, User Defined Functions.



Subscriptions
A Subscriber is what is created on the Subscriber server to receive all of the
transactions of the Publication.
Partitioning Large Tables – How to deal?
• Partitioning Schemes can be propogated from the Publisher
  but don’t have to.


• They can also be created a the subscriber entirely different better for
  reporting.


• If you want to archive the Publishing database but keep the subscriber
  around you can get creative with Partition swapping.


• Replication is scalable. I have replicated a 6 billion record table with good
  performance. A lot of partitions 
Replication References
http://www.replicationanswers.com

Paul Ibison’s website full of good replication…well answers.

http://www.sqlsoldier.com

Robert Davis’s website and if you use the search box you can find some good stuff
on replication

Twitter: #sqlhelp
Questions
Thank you for your time today and hope you found this educational

Anthony Sammartino MCITP(BI), PMP®
Vice President
TekPartners Business Intelligence Solutions
(954) 282-1798
asammartino@tekpartners.com

Twitter: @SQLAnt

Linkedin: http://www.linkedin.com/in/apsammartino

Blog: www.sqlant.me

Web: www.tekpartnersbi.com

More Related Content

What's hot

Tableau Architecture
Tableau ArchitectureTableau Architecture
Tableau ArchitectureVivek Mohan
 
SSAS, MDX , Cube understanding, Browsing and Tools information
SSAS, MDX , Cube understanding, Browsing and Tools information SSAS, MDX , Cube understanding, Browsing and Tools information
SSAS, MDX , Cube understanding, Browsing and Tools information Vishal Pawar
 
Business Intelligence in Excel 2013
Business Intelligence in Excel 2013Business Intelligence in Excel 2013
Business Intelligence in Excel 2013Stéphane Fréchette
 
Self-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerViewSelf-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerViewIvan Donev
 
Tableau free tutorial
Tableau free tutorialTableau free tutorial
Tableau free tutorialtekslate1
 
Real-world BISM in SQL Server 2012 SSAS
Real-world BISM in SQL Server 2012 SSASReal-world BISM in SQL Server 2012 SSAS
Real-world BISM in SQL Server 2012 SSASLynn Langit
 
PowerPivot and PowerQuery
PowerPivot and PowerQueryPowerPivot and PowerQuery
PowerPivot and PowerQueryin4400
 
Deploy PowerPivot Enterprise Wide
Deploy PowerPivot Enterprise WideDeploy PowerPivot Enterprise Wide
Deploy PowerPivot Enterprise Widewww.panorama.com
 
A Gentle Introduction to Microsoft SSAS
A Gentle Introduction to Microsoft SSASA Gentle Introduction to Microsoft SSAS
A Gentle Introduction to Microsoft SSASJohn Paredes
 
Self-Service Business Intelligence with Power BI
Self-Service Business Intelligence with Power BISelf-Service Business Intelligence with Power BI
Self-Service Business Intelligence with Power BITheresa Lubelski
 
Sql developer resume
Sql developer resumeSql developer resume
Sql developer resumeguest9db49b8
 
Power BI Report Server & Office Online Server
Power BI Report Server & Office Online ServerPower BI Report Server & Office Online Server
Power BI Report Server & Office Online ServerIsabelle Van Campenhoudt
 
SQL server 2016 New Features
SQL server 2016 New FeaturesSQL server 2016 New Features
SQL server 2016 New Featuresaminmesbahi
 
A Primer To Sybase Iq Development July 13
A Primer To Sybase Iq Development July 13A Primer To Sybase Iq Development July 13
A Primer To Sybase Iq Development July 13sparkwan
 
Powerpivot prototyping
Powerpivot prototypingPowerpivot prototyping
Powerpivot prototypingdivyaww
 
Basic introduction to power query
Basic introduction to power queryBasic introduction to power query
Basic introduction to power querydisha parmar
 

What's hot (20)

Tableau Architecture
Tableau ArchitectureTableau Architecture
Tableau Architecture
 
SSAS, MDX , Cube understanding, Browsing and Tools information
SSAS, MDX , Cube understanding, Browsing and Tools information SSAS, MDX , Cube understanding, Browsing and Tools information
SSAS, MDX , Cube understanding, Browsing and Tools information
 
Business Intelligence in Excel 2013
Business Intelligence in Excel 2013Business Intelligence in Excel 2013
Business Intelligence in Excel 2013
 
Self-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerViewSelf-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerView
 
Tableau free tutorial
Tableau free tutorialTableau free tutorial
Tableau free tutorial
 
Real-world BISM in SQL Server 2012 SSAS
Real-world BISM in SQL Server 2012 SSASReal-world BISM in SQL Server 2012 SSAS
Real-world BISM in SQL Server 2012 SSAS
 
PowerPivot and PowerQuery
PowerPivot and PowerQueryPowerPivot and PowerQuery
PowerPivot and PowerQuery
 
Deploy PowerPivot Enterprise Wide
Deploy PowerPivot Enterprise WideDeploy PowerPivot Enterprise Wide
Deploy PowerPivot Enterprise Wide
 
A Gentle Introduction to Microsoft SSAS
A Gentle Introduction to Microsoft SSASA Gentle Introduction to Microsoft SSAS
A Gentle Introduction to Microsoft SSAS
 
VishalWalujwarCV
VishalWalujwarCVVishalWalujwarCV
VishalWalujwarCV
 
Self-Service Business Intelligence with Power BI
Self-Service Business Intelligence with Power BISelf-Service Business Intelligence with Power BI
Self-Service Business Intelligence with Power BI
 
Machine Learning in SQL Server 2019
Machine Learning in SQL Server 2019Machine Learning in SQL Server 2019
Machine Learning in SQL Server 2019
 
Sql developer resume
Sql developer resumeSql developer resume
Sql developer resume
 
Power BI Report Server & Office Online Server
Power BI Report Server & Office Online ServerPower BI Report Server & Office Online Server
Power BI Report Server & Office Online Server
 
SQL server 2016 New Features
SQL server 2016 New FeaturesSQL server 2016 New Features
SQL server 2016 New Features
 
A Primer To Sybase Iq Development July 13
A Primer To Sybase Iq Development July 13A Primer To Sybase Iq Development July 13
A Primer To Sybase Iq Development July 13
 
SSAS Tabular model importance and uses
SSAS  Tabular model importance and usesSSAS  Tabular model importance and uses
SSAS Tabular model importance and uses
 
Powerpivot prototyping
Powerpivot prototypingPowerpivot prototyping
Powerpivot prototyping
 
Basic introduction to power query
Basic introduction to power queryBasic introduction to power query
Basic introduction to power query
 
Sql Portfolio
Sql PortfolioSql Portfolio
Sql Portfolio
 

Similar to Conquer Reporting by Scaling Out SQL Server

Case Study For Replication For PCMS
Case Study For Replication For PCMSCase Study For Replication For PCMS
Case Study For Replication For PCMSShahzad
 
Sql disaster recovery
Sql disaster recoverySql disaster recovery
Sql disaster recoverySqlperfomance
 
KoprowskiT_SQLSat409_MaintenancePlansForBeginners
KoprowskiT_SQLSat409_MaintenancePlansForBeginnersKoprowskiT_SQLSat409_MaintenancePlansForBeginners
KoprowskiT_SQLSat409_MaintenancePlansForBeginnersTobias Koprowski
 
KoprowskiT_SQLSaturday409_MaintenancePlansForBeginners
KoprowskiT_SQLSaturday409_MaintenancePlansForBeginnersKoprowskiT_SQLSaturday409_MaintenancePlansForBeginners
KoprowskiT_SQLSaturday409_MaintenancePlansForBeginnersTobias Koprowski
 
UNIT3 DBMS.pptx operation nd management of data base
UNIT3 DBMS.pptx operation nd management of data baseUNIT3 DBMS.pptx operation nd management of data base
UNIT3 DBMS.pptx operation nd management of data baseshindhe1098cv
 
Stay productive_while_slicing_up_the_monolith
Stay productive_while_slicing_up_the_monolithStay productive_while_slicing_up_the_monolith
Stay productive_while_slicing_up_the_monolithMarkus Eisele
 
Virtualizing Tier One Applications - Varrow
Virtualizing Tier One Applications - VarrowVirtualizing Tier One Applications - Varrow
Virtualizing Tier One Applications - VarrowAndrew Miller
 
Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...
Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...
Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...Andrew Miller
 
Boosting the Performance of your Rails Apps
Boosting the Performance of your Rails AppsBoosting the Performance of your Rails Apps
Boosting the Performance of your Rails AppsMatt Kuklinski
 
Database Virtualization: The Next Wave of Big Data
Database Virtualization: The Next Wave of Big DataDatabase Virtualization: The Next Wave of Big Data
Database Virtualization: The Next Wave of Big Dataexponential-inc
 
Marketing Automation at Scale: How Marketo Solved Key Data Management Challen...
Marketing Automation at Scale: How Marketo Solved Key Data Management Challen...Marketing Automation at Scale: How Marketo Solved Key Data Management Challen...
Marketing Automation at Scale: How Marketo Solved Key Data Management Challen...Continuent
 
SOA with Zend Framework
SOA with Zend FrameworkSOA with Zend Framework
SOA with Zend FrameworkMike Willbanks
 
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018 Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018 Antonios Chatzipavlis
 
Migrate a successful transactional database to azure
Migrate a successful transactional database to azureMigrate a successful transactional database to azure
Migrate a successful transactional database to azureIke Ellis
 
Introduction to Microservices
Introduction to MicroservicesIntroduction to Microservices
Introduction to MicroservicesMahmoudZidan41
 
Sql server replication step by step
Sql server replication step by stepSql server replication step by step
Sql server replication step by steplaonap166
 
DBA, LEVEL III TTLM Monitoring and Administering Database.docx
DBA, LEVEL III TTLM Monitoring and Administering Database.docxDBA, LEVEL III TTLM Monitoring and Administering Database.docx
DBA, LEVEL III TTLM Monitoring and Administering Database.docxseifusisay06
 
Building and Deploying Large Scale SSRS using Lessons Learned from Customer D...
Building and Deploying Large Scale SSRS using Lessons Learned from Customer D...Building and Deploying Large Scale SSRS using Lessons Learned from Customer D...
Building and Deploying Large Scale SSRS using Lessons Learned from Customer D...Denny Lee
 

Similar to Conquer Reporting by Scaling Out SQL Server (20)

Case Study For Replication For PCMS
Case Study For Replication For PCMSCase Study For Replication For PCMS
Case Study For Replication For PCMS
 
Sql disaster recovery
Sql disaster recoverySql disaster recovery
Sql disaster recovery
 
KoprowskiT_SQLSat409_MaintenancePlansForBeginners
KoprowskiT_SQLSat409_MaintenancePlansForBeginnersKoprowskiT_SQLSat409_MaintenancePlansForBeginners
KoprowskiT_SQLSat409_MaintenancePlansForBeginners
 
KoprowskiT_SQLSaturday409_MaintenancePlansForBeginners
KoprowskiT_SQLSaturday409_MaintenancePlansForBeginnersKoprowskiT_SQLSaturday409_MaintenancePlansForBeginners
KoprowskiT_SQLSaturday409_MaintenancePlansForBeginners
 
UNIT3 DBMS.pptx operation nd management of data base
UNIT3 DBMS.pptx operation nd management of data baseUNIT3 DBMS.pptx operation nd management of data base
UNIT3 DBMS.pptx operation nd management of data base
 
Stay productive_while_slicing_up_the_monolith
Stay productive_while_slicing_up_the_monolithStay productive_while_slicing_up_the_monolith
Stay productive_while_slicing_up_the_monolith
 
Virtualizing Tier One Applications - Varrow
Virtualizing Tier One Applications - VarrowVirtualizing Tier One Applications - Varrow
Virtualizing Tier One Applications - Varrow
 
Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...
Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...
Varrow Q4 Lunch & Learn Presentation - Virtualizing Business Critical Applica...
 
Boosting the Performance of your Rails Apps
Boosting the Performance of your Rails AppsBoosting the Performance of your Rails Apps
Boosting the Performance of your Rails Apps
 
Database Virtualization: The Next Wave of Big Data
Database Virtualization: The Next Wave of Big DataDatabase Virtualization: The Next Wave of Big Data
Database Virtualization: The Next Wave of Big Data
 
Marketing Automation at Scale: How Marketo Solved Key Data Management Challen...
Marketing Automation at Scale: How Marketo Solved Key Data Management Challen...Marketing Automation at Scale: How Marketo Solved Key Data Management Challen...
Marketing Automation at Scale: How Marketo Solved Key Data Management Challen...
 
SOA with Zend Framework
SOA with Zend FrameworkSOA with Zend Framework
SOA with Zend Framework
 
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018 Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
Azure SQL Database for the SQL Server DBA - Azure Bootcamp Athens 2018
 
Microservices vs monolithics betabeers
Microservices vs monolithics   betabeersMicroservices vs monolithics   betabeers
Microservices vs monolithics betabeers
 
Database Management System - 2a
Database Management System - 2aDatabase Management System - 2a
Database Management System - 2a
 
Migrate a successful transactional database to azure
Migrate a successful transactional database to azureMigrate a successful transactional database to azure
Migrate a successful transactional database to azure
 
Introduction to Microservices
Introduction to MicroservicesIntroduction to Microservices
Introduction to Microservices
 
Sql server replication step by step
Sql server replication step by stepSql server replication step by step
Sql server replication step by step
 
DBA, LEVEL III TTLM Monitoring and Administering Database.docx
DBA, LEVEL III TTLM Monitoring and Administering Database.docxDBA, LEVEL III TTLM Monitoring and Administering Database.docx
DBA, LEVEL III TTLM Monitoring and Administering Database.docx
 
Building and Deploying Large Scale SSRS using Lessons Learned from Customer D...
Building and Deploying Large Scale SSRS using Lessons Learned from Customer D...Building and Deploying Large Scale SSRS using Lessons Learned from Customer D...
Building and Deploying Large Scale SSRS using Lessons Learned from Customer D...
 

Recently uploaded

Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdfChristopherTHyatt
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 

Recently uploaded (20)

Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdf
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 

Conquer Reporting by Scaling Out SQL Server

  • 1. Conquer Reporting By Scaling Out SQL Server January 16, 2013
  • 2. My Contact Information Anthony Sammartino MCITP(BI), PMP® Vice President TekPartners Business Intelligence Solutions (954) 282-1798 asammartino@tekpartners.com Twitter: @SQLAnt Linkedin: http://www.linkedin.com/in/apsammartino Blog: www.sqlant.me Web: www.tekpartnersbi.com
  • 3. About Me • Started working with SQL Server 7.0 in 1999 • 14 Data Warehouse / BI Projects in 13 Years • 11 Years SQL Server 2000, 2005, 2008(R2), 2012 • Live in South Florida with my wife and two daughters • Fan of Miami and Philadelphia Sports teams, all things technology • Solutions Architect, Project Manager, Technical Lead MCP(SQL Server), MCTS(BI), MCITP(BI), PMP®
  • 4. Session Overview 1. Case for Scaling Out SQL Server for Reporting 2. Choosing a Method to Scale Out SQL Server 3. Architecture with Reporting Scaled Out 4. Replication Server Roles 5. Replication Agents 6. Distribution Database 7. Publications, Articles, Subscriptions 8. Partitioning 9. References for Replication 10. Questions
  • 5. The "Situation" Problem Statement: A company named (House Widgets Inc.) has a main application that sells widgets online. Every time the Sales staff runs reports to see how many widgets were sold the application slows, and even times out. Sales staff complains their reports are slow. Users complain application is slow. Background: • DBA team has tuned the reports (T-SQL) through execution plans. • They also verified its using the correct indexes (Seeks). • They even resorted to dirty reads using isolation level and nolocks in the code. • Deemed the data is just to large and too busy. Thousands of users are trying to access it, change it at the same time. • Solid State Disks were even thrown at it. Helped for a short time. Resolution: • Time to think about separating the databases for the application and reporting. • Many ways to scale out SQL Server for reporting. • Many variables to think about.
  • 6. Choosing a Method to Scale Out Requirements • Requirement is to have real time reporting. • Requirement is to only report on half of the db tables. Transactional Replication…Why? • Transactional Replication happens in near real time. • Once setup, transactions flow. Other types of Replication • Snapshot: Point in time. For example once a day at 9am. Not Near real time. • Merge: Near real time but ideal for situations where two dbs need to be kept in sync with writes in two locations. Example: Bank with two branches in NY, FL. Database Mirroring • Mirroring is very easy to setup and maintain. • It’s a whole database or nothing solution. • Point in time using database snapshots. Always On Availability Groups *NEW SQL Server 2012 • Near real time transactions. • It’s a whole database or nothing solution. • Enterprise Edition needed $$$ on both Source and Replica. Service Broker • Near real time transactions. • Harder to setup and maintain, more suited for transforming data.
  • 7. Architecture with Reporting Scaled Out to a Replica Application (HouseWidgets.com) SQL Server (SQLVM03) SQL Server (SQLVM03SUBSCRIBER) Publisher Distributor & Subscriber Application Application DB- DB Distribution Replica Transactional Replication Read Only
  • 8. Replication Server Roles Publisher: The publisher is the source SQL Server that contains database objects that you wish to replicate. In this case SQLVM03. Distributor: The distributor is the server that you wish to setup to “distribute” transactions from the publisher to the subscriber server. Subscriber: The Subscriber is the server that hosts the replica database and where the reporting will be directed from the application. In this case SQLVM03Subscriber
  • 9. Secret Agents…Man • Replication creates a number of Agents. • These Agents run as jobs scheduled under SQL Server Agent. • SQL Server Agent must be turned on. • Recommend restarting options be checked, and configure SQL Server Agent with a service account with access to both machines. • Replication agents can be administered from SQL Server Replication Monitor and SQL Server Management Studio.
  • 10. More Secret Agents Snapshot Agent • Used with all types of replication. • It prepares schema and initial data files of published tables and other objects. • Records information about synchronization in the distribution database. • The Snapshot Agent runs at the Distributor. Log Reader Agent • Used with transactional replication. • Moves transactions marked for replication from the Publisher to the distribution database. Each database published using transactional replication has its own Log Reader Agent. Distribution Agent • Used with snapshot replication and transactional replication. • It applies the initial snapshot to the Subscriber and moves transactions held in the distribution database to Subscribers. • The Distribution Agent runs at either the Distributor for push subscriptions or at the Subscriber for pull subscriptions.
  • 11. More Secret Agents Merge Agent • Used with merge replication. • It applies the initial snapshot to the Subscriber and moves and reconciles incremental data changes that occur. • Each merge subscription has its own Merge Agent that connects to both the Publisher and the Subscriber and updates both. • The Merge Agent runs at either the Distributor for push subscriptions or the Subscriber for pull subscriptions. Replication Maintenance Jobs Replication has a number of maintenance jobs that perform scheduled and on-demand maintenance.
  • 12. Distribution Database Distribution DB The distribution database holds all of the transactions from the publisher database and applies them to the subscriber. Performance Tip As a general rule of thumb for smaller publisher database implementations small in size, small number of concurrent users, small number of transactions, place the distributor and subscriber on one server separate from the publisher. For larger database implementations large in size, large number of concurrent users, large number of transactions, place the distributor on its own server and the subscriber on its own server, both separate from the publisher. Always separate the distributor and subscriber from the publisher. Your goal is to take load off the publisher with this architectural solution.
  • 13. Publications, Articles, Subscriptions Publications A Publication is what is created on the Publisher server and stores articles that will be replicated. Tip: One table per publication, especially if they are large. Re-snapshot. Articles Tables, Stored Procedures, Views , Indexed Views, User Defined Functions. Subscriptions A Subscriber is what is created on the Subscriber server to receive all of the transactions of the Publication.
  • 14. Partitioning Large Tables – How to deal? • Partitioning Schemes can be propogated from the Publisher but don’t have to. • They can also be created a the subscriber entirely different better for reporting. • If you want to archive the Publishing database but keep the subscriber around you can get creative with Partition swapping. • Replication is scalable. I have replicated a 6 billion record table with good performance. A lot of partitions 
  • 15. Replication References http://www.replicationanswers.com Paul Ibison’s website full of good replication…well answers. http://www.sqlsoldier.com Robert Davis’s website and if you use the search box you can find some good stuff on replication Twitter: #sqlhelp
  • 16. Questions Thank you for your time today and hope you found this educational Anthony Sammartino MCITP(BI), PMP® Vice President TekPartners Business Intelligence Solutions (954) 282-1798 asammartino@tekpartners.com Twitter: @SQLAnt Linkedin: http://www.linkedin.com/in/apsammartino Blog: www.sqlant.me Web: www.tekpartnersbi.com