See conference video - http://www.lucidimagination.com/devzone/events/conferences/revolution/2011

1. Practical Search
in the Cloud
(sneak preview)
Martijn Koster, Lucid Imagination
martijn.koster@lucidimagination.com
Wednesday, 25 May 2011

2. My Background
 Martijn Koster
 Sr. Software Engineer at Lucid Imagination
• Search in early Web – WebCrawler
• Customer web services – Excite Mail
• Proxies for Mobile – Danger
2
Wednesday, 25 May 2011

3. What I Will Cover
 A Solr based Cloud Service
 The need
 How
 Demo
 Wrap up
3
Wednesday, 25 May 2011

4. Why? – Getting Started is Hard
 Many new concepts
 Complex software configuration
 System Administration overhead
4
Wednesday, 25 May 2011

5. Search as a Service
 A few clicks to get started
 Full Power of LucidWorks and Solr
 Offload operations
 Expertise, tools and processes are transferable
to your own deployments.
5
Wednesday, 25 May 2011

6. Usage Model
Users
Web UI
LWE JVM
Lucidworks
LWE JVM
Solr
User
Code
Direct Access
6
Wednesday, 25 May 2011

7. Architecture
3rd party billing Provisioning
Users Service (Rails)
Web UI
(Rails) Heroku
Provisioner
Heroku
(Ruby)
User
Code Direct Access
(Proxy with URL
rewriting) LWE JVM
LWE JVM
LWE JVM
SQL
Control Monitoring
(RDS)
(Chef) (Zabbix)
7
Wednesday, 25 May 2011

8. Deployment Automation
 AMI
 Chef
 Custom
8
Wednesday, 25 May 2011

9. Scaling
 Heroku
 Direct Access
 Search Servers
 Provisioning/DB
 Monitoring and Control
9
Wednesday, 25 May 2011

10. Direct Access Example
 The Cloudy Direct Access Solr interface
• Simple submission
• Simple search
First, copy SOLR access URL:
SOLR="https://access.cloudy.io/…/solr"
11
Wednesday, 25 May 2011

11. Example - Add
$ cat > /tmp/doc.xml <<EOM
<add><doc>
<field name="id">testdoc_LR2011</field>
<field name="text_en">LR rocks!</field>
</doc></add>
EOM
curl $SOLR/my_news/update?commit=true
-H "Content-Type: text/xml"
--data-binary @/tmp/doc.xml
12
Wednesday, 25 May 2011

12. Example - Search
curl $SOLR'/my_news/lucid?
q=news&role=DEFAULT&wt=json'
curl $SOLR'/my_news/select?
wt=json&q=news&start=0&rows=10'
13
Wednesday, 25 May 2011

13. Example – Lucidworks API
• The Cloudy Direct Access Lucid API interface
API="http://access.demo.cloudy.io/…/api”
curl $API/collections
[
{"name":"lwelogs","instance_dir":"lwelogs”
{"name":"collection1","instance_dir":"collection1_0"},
{"name":"my_news","instance_dir":"my_news_1"}
]
14
Wednesday, 25 May 2011

14. Live Demo
 Sam Pierson, Director of Product Development
 The Web User Interface
• Administration UI improvements
• Log indexing and graphing
 API access
• Ruby bindings for the LucidWorks REST API
• http://rubygems.org/gems/lucid_works
10
Wednesday, 25 May 2011

15. Lessons
 Cloud technologies speed up development
 Automate everything
 Avoid lock-in
 Dogfooding
16
Wednesday, 25 May 2011

16. Wrap Up
 Search as a Service
 Built in the Cloud, with open source
 Full Lucidworks Enterprise and Solr
 Powered by Lucid Imagination
17
Wednesday, 25 May 2011

17. When
 Early Access Program: July
• Interested? cloud@lucidimagination.com
18
Wednesday, 25 May 2011

18. Contact
 Martijn Koster
• martijn.koster@lucidimagination.com
• http://www.lucidimagination.com/
• twitter.com/makuk66
19
Wednesday, 25 May 2011