A case study of Nestoria. Discuss the difference between a simple mash-up and a business that runs on mash-ups. Also touch on some challenges we have faced.

1. this space left intentionally blank

2. why perl is awesome and
all other languages suck
Mike Astle
flamebait@lokku.com

3. from mash-up to business
Mike Astle - Lokku Ltd.
Google Developer Day
31 May 2007 - London

4. i like questions

6. Milestones
• Start dev April 2006
• Launch London June 2006
• Launch UK Sept 2006
• Launch Spain May 2007

9. Numbers
• In the UK, have 850,000 listings
refreshed continually
• Boils down to about 500,000 properties
• 100,000+ users per month
• 4 programmers
• No dedicated operations staff

10. Houses on a map…so what?

11. this is harder than it looks

12. The Plan
• Talk about some of the technical
decisions we made at Nestoria
• Go over a few challenges that we have
faced
• Hopefully there are lessons to learn
from our experiences

13. first, you design

14. Things to worry about
• Architecture
• Services vs. APIs
• How to scale?
• i18n
• Future products

15. then you choose a platform

17. MS vs. Everything Else

18. Java?

19. That Leaves LAMP

20. PHP?

21. Ruby?
LAMR?

22. Python?

23. Perl

24. Perl
Utility Belt == CPAN

25. Apache

26. MySQL

27. GNU/Linux
(Debian)

28. now choose a
development environment

29. Source Control
• SVN…duh
• Does anybody use CVS anymore?
• Love the trac

31. Coders
• A few veterans
• As many interns / junior coders as fit in
the room

32. Highly Collaborative
(but not XP or Agile)

33. Emphasis
• Collective decision-making
• Individual responsibility for deadlines
• Many small success rather than few
large projects

36. Why write tests?
• Find bugs, of course
• Increase confidence to make changes
to unfamiliar systems or libraries
• Promote good design / expose bad
design

37. Design
• Informal process (no UML, minimal
design patterns)
• Concentrate on the things that will really
hurt - e.g. i18n
• Not everything has to be an object

38. Documentation
• POD for interfaces
• twiki for systems
• Schedule regular refreshers

39. Outsource
• If you can spec it faster than you can
write it, consider farming it out
• Many, many site full of hungry coders
• Have had good results with
scriptlance.com but there are lots of
others out there

40. how to deploy?

43. Virtualization?
• Sounds like voodoo to me
• Extra layer to worry about
• Easy enough to do everything in user
space

44. Wigwam
• 12 subdomains
• Live data
• 22 services
• Many playpens per box
• Releases several times a week

45. big ISP vs. small ISP
• Bigger - more consistent
• Smaller - more personal
• Bigger - discrete choices
• Smaller - continuous advice
• We went small

46. Hardware
• Know your downtime limits
• Be reasonable!
• Load balancing - expensive and
complex - we do it with DNS through a
vendor
• Have a failure plan

47. The
Hawkeye
Principle

48. but is anybody using it?

49. Logging
• Need your own, and it needs to be
flexible
• Use the free tools as a sanity check

50. Reporting
• An important area of expertise, not an
annoying afterthought
• Glossary
• Every mean needs a variance
• A/B Testing (a.k.a Bucket Testing) is
hard to do right

51. Also…
• Build tools for ad hoc queries, they
>will< get used
• Keep an audit trail of changes
• Effective tools for drilling down on
sessions allows non-techs to help
debugging

52. problems we have solved

54. http://www.nestoria.co.uk/covent-
garden/flat/buy/bedrooms-2/maxprice-
1000000/minprice-250000/bathrooms-2

64. • Alderly Edge => Alderley Edge
• Bayswater London => Bayswater
• Berkhampsted => Berkhamsted
• Birminham => Birmingham
• Bletchingly => Bletchingley
• Broxbourne Station => Broxbourne
• Cambuslank => Cambuslang
• Canary Warf => Canary Wharf
• Claphamjuction => Clapham Junction
• Covent Garden London => Covent
Garden

65. Geocoding
• Transform address into lat/long
coordinates
• 1,000+ calls a minute
• Very high data variance (few cache hits)
• Royal Mail data for UK
• Teleatlas data for Spain

66. • Albert Villas, Gilbert Mews, LEIGHTON BUZZARD,
• Bedfordshire, LU7 1NF
• Lake Lock Drive, Wakefield West Yorkshire
• Church Lane, London
• C ELMINGTON ROAD, LONDON
• Fountain Road (Flat 2), Edgbaston
• London SE18, UK
• Lee, London, UK
• Free Trade Wharf, 340 The Highway,
Wapping/Limehouse London
• Stunning 3 Storey Contemporary Barn Conversion In
The Popular Village Of Utkinton
• A.4-1 GREAT NORTHERN TOWER,WATSON
STREET,CITY CENTRE,
• MANCHESTER GREATER MANCHESTER
• 15, ASTLEY HOUSE, LONDON

67. (((^[BEGLMNS][1-9]d?) | (^W[2-9] ) | ( ^( A[BL] | B[ABDHLNRST] |
C[ABFHMORTVW] | D[ADEGHLNTY] | E[HNX] | F[KY] | G[LUY] | H[ADGPRSUX] |
I[GMPV] | JE | K[ATWY] | L[ADELNSU] | M[EKL] | N[EGNPRW] | O[LX] |
P[AEHLOR] | R[GHM] | S[AEGKL-PRSTWY] | T[ADFNQRSW] | UB | W[ADFNRSV] |
YO | ZE ) dd?) | (^W1[A-HJKSTUW0-9]) | (( (^WC[1-2]) | (^EC[1-4]) |
(^SW1) ) [ABEHMNPRVWXY] ) ) (s*)? ([0-9][ABD-HJLNP-UW-Z]{2}))

69. what next?

70. Future Projects
• Expose more data via API (e.g.
historical aggregate data)
• Implicit personalisation (the system
learns from users)
• More international expansion
• Systemic testing to improve user
experience

71. like what you see?

76. http://www.lokku.com/jobs

77. thanks!
astle@lokku.com
http://www.lokku.com
http://www.nestoria.co.uk
http://www.nestoria.es
http://www.lokku.com/lpw06_geocoding/
http://www.lokku.com/lpw06_wigwam/