Search engine

1. Full Text Search Engine

2. What is Sphinx?
Sphinx is a standalone software package provides fast and relevant full-
text search functionality to client applications. It was specially designed to
integrate well with SQL databases storing the data, and to be easily
accessed by scripting languages. However, Sphinx does not depend on
nor require any specific database to function.
On a live database of nearly 300,000 rows of five indexed columns, where
each column contains about 15 words, Sphinx can yield a result for an
"any of these words" search in 1/100th of a second (on a 2-GHz AMD
Opteron processor with 1 GB of RAM running Debian Linux® Sarge).

3. Key features of Sphinx
 It can index any data you can represent as a string.
 It can index the same data in different ways. With multiple indexes, each





tuned for a specific purpose, you can choose the most appropriate index
to optimize search results.
It can associate attributes with each piece of indexed data. You can then
use one or more of the attributes to further filter search results.
It supports morphology, so a search for the word "cats" also finds the
root word "cat."
You can distribute a Sphinx index among many machines, providing
failover.
It can create indexes of word prefixes of arbitrary length and indexes of
infix substrings of varying lengths. For instance, a part number may be
10 characters wide. The prefix index would match against all possible
substrings anchored at the start of the string. The infix index would
match substrings anywhere within the string.
You can run it as a storage engine within MySQL V5.

4. Sphinx has three components: an index generator, a search engine, and a
command-line search utility:
 The index generator is called indexer. It queries your database, indexes
each column in each row of the result, and ties each index entry to the row's
primary key.
 The search engine is a daemon called searchd. The daemon receives
search terms and other parameters, scours one or more indices, and returns
a result. If a match is made, searchd returns an array of primary keys. Given
those keys, an application can run a query against the associated database
to find the complete records that comprise the match. Searchd
communicates to applications through a socket connection on port 3312.
 The handy search utility lets you conduct searches from the command line
without writing code. If searchd returns a match, search queries the
database and displays the rows in the match set. The search utility is useful
for debugging your Sphinx configuration and performing impromptu
searches.

5. The indexer will query the database based on the config parameter
sql_query
E.G. of use
source src1
{
sql_query
=
SELECT
id, group_id, UNIX_TIMESTAMP(date_added) AS date_added,
title, content
FROM documents
sql_query_info
= SELECT * FROM documents WHERE id=$id
}
index test1
{
source
path
= @CONFDIR@/data/test1
charset_type
}
= src1
= utf-8

6. indexer
{
mem_limit
= 246M
}
searchd
{
listen
listen
= 9306:mysql41
log
= @CONFDIR@/log/searchd.log
query_log
= @CONFDIR@/log/query.log
pid_file
= @CONFDIR@/log/searchd.pid
max_matches
= 1000
binlog_path
}
= 9312
= @CONFDIR@/data

7. How we utilise Sphinx Search
Sphinx
Index‟s all
product
User search‟s
term (bra)
Results
gathered from
Sphinx
Results saved
to MySQL
database
2ND User
search’s
term (bra)
Result are
gather MySQL
database

8. How we should utilise Sphinx
User
search
Sphinx
generates
results e.g.
product
ID‟s
Results are
displayed
to User
Query
MySQL
database
by ID‟s

9. On Agent Provocateur we currently we store our result from Sphinx to the
following table which I presume is same with all our clients.
MySQL Tables
 search_refine_result
 search_results
Sphinx search‟s term‟s based on specific set of fields on the products,
rangesty, ranges, styles, sizes, colours, and menu tables which is
defined by the index query on the Sphinx configuration file.
Configuration directory : (client_dir)/scripts/search/sphinx.conf

10. API usage e.g.
<?php
include sphinxapi.php;
$sphinx = new sphinxClient();
$sphinx->Query(„@prodgroup “GRP101”‟);
Get‟s all product in the product under the product groupid 101

11. Is a Search engine necessary?
Website implementing search capabilities with high traffic volumes a
search engine is diffidently necessary, some searches may be more
specialized than the database can perform, or a search may be so
complicated that the required SQL JOINs are simply too slow to keep up
with a high volume of search queries.
Our usage of Sphinx is very limited due to the search result being restored
back into the MySQL database. Sphinx is more than capable enough to
manage all search queries. It‟s certainly necessary to incorporate a search
engine on our clients website however we will need to implement the
search engine in a more flexible and expandable way easily deployable to
new and old clients.