This document summarizes a presentation about Microsoft's Log Analytics SaaS service, which uses Apache Solr. It discusses the challenges of supporting a multi-tenant service at scale, including bottlenecks in Solr Cloud and performance issues with wide queries. The presentation describes Microsoft's approach to addressing these challenges through workload management across Solr clusters, centralized configuration, and querying cold storage clusters to improve query performance. It concludes by discussing next steps to further optimize Solr for the log analytics scenario.

1. O C T O B E R 1 1 - 1 4 , 2 0 1 6 • B O S T O N , M A

2. Multi-Tenant Log Analytics SaaS Service using SOLR
Chirag Gupta (Principal Software Engineer)
Srivatsan Parthasarathy (Partner Software Engineer)
Microsoft Corp.

3. Solutions
Search & Analytics
Portal/API
Microsoft Log Analytics

4. Real-time search Patching
Configuration Change Tracking
Linux Solutions
Monitoring App Solutions
Community Gallery
Security Audit Data Protection Dashboards
Integrations/Collectors Log Analytics Service
Solution Platform
Microso' Log Analy0cs Solu0on Pla4orm

5. 5
02
DEMO
Microsoft Log Analytics
Aka OMS (Operations Management Suite)

6. 6
02
• 50K+
customers
with
10M+
shards
Usage Statistics
• 20PB
searchable
documents
• 188
million
queries
per
week

7. 7
02
Challenges
• SOLR
Cloud
boDlenecks
• MulE-­‐tenant
support
• Performance
issues
with
wide
queries

8. 8
02
SOLR Cloud

9. IngesEon
Azure File Shares
SOLR Clusters
SOLR
Query
Cluster
State
Microso' Log Analy0cs Cloud
Gateway
SOLR
Query
Merger

10. SOLR
Cluster
Query
Inges0on
Predictable Load
Single Customer/
Workload
SOLR Workloads
Light Load
Moderate Load
Heavy Load
Long tail of large and small customers
Growing/shrinking without warning

11. IngesEon
SOLR Clusters
Azure File Shares
Cold
Cluster1
SOLR
Query
Microso' Log Analy0cs Workload Management
Cluster
State
Gateway
SOLR
Query
Merger
Management

12. SOLR
Cluster
Wide Range Query
SOLR Query

13. Query
Cold
Cluster1
SOLR
Hot SOLR Clusters
Azure File Shares
Cold
Cluster1
SOLR
Cold Cluster
Microso' Log Analy0cs Cold Query
Cluster
State
Gateway
SOLR
Query
Merger

14. 14
02
Recap
• Used
external
durable
storage
vs
replicaEon
• Central
scoped
config
with
ApplicaEon
Load
Balancing
vs
Zookeeper
• Shard
Lifecycle
Management
as
a
separate
component
• Cold
Storage
based
Queries

15. 15
02
Next Steps
• Further
opEmize
SOLR
for
Log
AnalyEcs
Scenario
• IngesEon
pipeline
• AnalyEcs
queries

16. 16
02
Q/A

17. 17
02
Thank You
Check out Microsoft Log Analytics @
http://microsoft.com/oms
Blogs @ https://blogs.technet.microsoft.com/msoms/