Search is broken It can be fixed

1. Semantic Search
Do you know what I mean?
Collaboration for Life

2. Search is broken
1. Search engines rely heavily on words and too little on concepts
2. Search engines only use geography to limit linguistic scope
3. Seach engines ignore the linguistic scope variables of industry,
organisation, product line, scientific discipline, project
4. No formal notion of semantic equivalence
Collaboration for Life

3. Words vs concepts
“software architecture”
words are
labels
“solution architecture”
“persistent datastore”
“database”
“integration”
“program”
“program code”
“software”
“structural &
dynamic models”
“architectural design”
“architecture”
“structural &
dynamic models”
“architectural design”
concepts contain
semantic links
“architecture”
“program”
“program code”
“software”
“product”
“application”
Collaboration for Life

4. Mapping between labels and semantics
depends on linguistic scope!
“solution architecture”
“solution architecture”
Organisation A
Organisation B
“persistent datastore”
“database”
“persistent datastore”
“database”
“integration”
“structural &
dynamic models”
“structural &
dynamic models”
“architectural design”
“architectural design”
“architecture”
“architecture”
≠
“program”
“program code”
“program”
“program code”
“software”
“software”
“product”
“application”
“application”
Collaboration for Life

5. Semantic equivalence
“software”
“architecture”
=
“software”
“architecture”
“software architecture”
“database”
“integration”
“solution”
“solution architecture”
=
“software”
“architecture”
=
“software”
“architecture”
“product”
“application”
Collaboration for Life

6. Fixing search
1. Interact with user to confirm translation of words into concepts
2. Fully take into account the significance of linguistic scope, in
particular the variables of industry, organisation, product line,
scientific discipline, project
3. Introduce a formal notion of semantic equivalence
to simplify search specification and execution
PS: The W3C Semantic Web standards are not the solution
Collaboration for Life

7. Example 1
Semantic Seach
Collaboration for Life

8. 1. System: Ask the user for one or two concepts
?
Matches: ∞
?
Collaboration for Life

9. 2. User: Nominate two concepts to reduce the search space
Matches: < 10,000
car
Jorn Bettin
Collaboration for Life

10. 3. System: Suggest statistically related 2nd level concepts
Matches: < 10,000
car
?
Jorn Bettin
Collaboration for Life

11. 4. User: Select a concept to reduce the search space
Matches:
< 100
car
owner
Jorn Bettin
Collaboration for Life

12. 5. System: Suggest statistically related concepts
?
car
owner
Jorn Bettin
Collaboration for Life
Matches:
< 100

13. 6. User: Select a concept to reduce the search space
Matches:
< 10
car
owner
Victoria
Jorn Bettin
Collaboration for Life

14. 7. System: Suggest statistically related 2nd level concepts
Matches:
< 10
?
car
owner
Victoria
Jorn Bettin
Collaboration for Life

15. 8. User: Select a concept to reduce the search space
Matches:
<3
Volkswagen
car
owner
Victoria
Jorn Bettin
Collaboration for Life

16. 9. System: Suggest statistically related 3rd level concepts
Matches:
<3
Volkswagen
?
car
owner
Victoria
Jorn Bettin
Collaboration for Life

17. 10. User: Select a concept to reduce the search space
Matches:
1
Volkswagen
Golf
car
owner
Victoria
Jorn Bettin
Collaboration for Life

18. Example 2
Semantic Seach
Collaboration for Life

19. What if the name of the owner is highly ambiguous?
Matches:
<3
Volkswagen
Golf
car
owner
Victoria
John Smith
Collaboration for Life

20. 11. System: Suggest statistically related 2nd level concepts
Matches:
<3
Volkswagen
Golf
car
owner
Victoria
John Smith
Collaboration for Life
?

21. 12. User: Select a concept to reduce the search space
Australia
Volkswagen
Golf
car
owner
Victoria
John Smith
Collaboration for Life
Matches:
1

22. Example 3
Semantic Seach
Collaboration for Life

23. 1. System: Ask the user for one or two concepts
?
Matches: ∞
?
Collaboration for Life

24. 2. User: Nominate one concept to reduce the search space
car
Collaboration for Life
Matches: < 100,000,000,000

25. 3. System: Suggest statistically related concepts
Matches: < 100,000,000,000
car
?
Collaboration for Life

26. 4. User: Select or nominate a concept to reduce the search space
Matches:
< 100
car
ABC 123
Collaboration for Life

27. 5. System: Suggest statistically related 2nd level concepts
Matches:
< 100
car
?
ABC 123
Collaboration for Life

28. 6. User: Select a concept to reduce the search space
Matches:
< 10
car
number plate
ABC 123
Collaboration for Life

29. 7. System: Suggest statistically related concepts
Matches:
< 10
?
car
number plate
ABC 123
Collaboration for Life

30. 8. User: Select a concept to reduce the search space
Matches:
1
car
number plate
Victoria
ABC 123
Collaboration for Life

31. Example 4
Semantic Seach
Collaboration for Life

32. 1. System: Ask the user for one or two concepts
Matches: ∞
?
?
Collaboration for Life

33. 2. User: Nominate two concepts to reduce the search space
Matches: < 1,000,000
architecture
software
Collaboration for Life

34. 3. System: Suggest statistically related concepts
?
architecture
software
Collaboration for Life
Matches: < 1,000,000

35. 4. User: Select or nominate a concept to reduce the search space
Melbourne
Matches:
< 1,000
architecture
software
Collaboration for Life

36. 5. User: Select or nominate concept to enlarge the search space
Sydney
Melbourne
architecture
software
Collaboration for Life
Matches:
< 3,000

37. 6. System: Suggest statistically related concepts
Sydney
Melbourne
?
architecture
software
Collaboration for Life
Matches:
< 3,000

38. 7. User: Select a concept to reduce the search space
Sydney
product development
Melbourne
architecture
software
Collaboration for Life
Matches:
< 1,500

39. 8. User: Connect concepts to reduce the search space
Sydney
product development
Melbourne
Matches:
< 1,000
architecture
software
Collaboration for Life

40. 9. User: Connect concepts to reduce the search space
Sydney
product development
Melbourne
architecture
software
Collaboration for Life
Matches:
< 300

41. 10. System: Suggest statistically related 2nd level concepts
Sydney
product development
?
Melbourne
architecture
software
Collaboration for Life
Matches:
< 300

42. 11. User: Select a 2nd level concept to reduce the search space
Sydney
product development
Melbourne
architecture
Web
software
Collaboration for Life
Matches:
< 200

43. 12. System: Suggest statistically related 2nd level concepts
Sydney
product development
Melbourne
architecture
Web
software
?
Collaboration for Life
Matches:
< 200

44. 13. User: Select a 2nd level concept to reduce the search space
Sydney
product development
Melbourne
architecture
Web
software
mobile
Collaboration for Life
Matches:
< 100

45. 14. System: Suggest statistically related concepts
?
product development
Sydney
Melbourne
architecture
Web
software
mobile
Collaboration for Life
Matches:
< 100

46. 15. User: Select a concept to reduce the search space
permanent
Sydney
product development
Melbourne
architecture
Web
software
mobile
Collaboration for Life
Matches:
< 50

47. 16. System: Suggest statistically related concepts
permanent
Sydney
product development
architecture
Web
Melbourne
?
software
mobile
Collaboration for Life
Matches:
< 50

48. 17. User: Select a concept to reduce the search space
permanent
Sydney
product development
architecture
Web
Melbourne
senior
software
mobile
Collaboration for Life
Matches:
< 20

49. 18. System: Suggest statistically related 2nd level concepts
permanent
Sydney
product development
architecture
Web
Melbourne
senior
software
?
mobile
Collaboration for Life
Matches:
< 20

50. 19. User: Select a 2nd level concept to reduce the search space
permanent
Sydney
product development
architecture
Web
Melbourne
senior
software
semantic search
mobile
Collaboration for Life
Matches:
1

51. Semantic search can be fixed
Cell Platform
http://s23m.org/S23M/cell-platform.html
Collaboration for Life

52. The latest book on Domain Engineering
Domain Engineering is of considerable practical
significance, as it provides methods and techniques that
help reduce time-to-market, development costs, and
project risks on one hand, and helps improve system
quality and performance on a consistent basis on the other.
•
The most comprehensive and up-to-date work on
domain engineering
•
Covers all important technological aspects, including
software product lines, domain-specific languages, and
conceptual modeling
•
Introduces novel approaches and techniques, and
includes a wealth of pointers for further research
•
ISBN 978-3-642-36653-6, published 2013
http://www.springer.com/computer/swe/book/978-3-642-36653-6
Collaboration for Life

53. Semantic Search
S23M knows what you mean!
S23M
info @ s23m.com
Nothing beats capturing the knowledge flow
of leading domain experts to co-create
organisations & systems that are
understandable by future generations of
humans & software tools.
Collaboration for Life