Groovy & Grails eXchange 2012 - Building an e-commerce business with gr8 technologies in Latin America
•
7 likes•3,002 views
Presentation given in the Groovy & Grails exchange 2012 at London.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Groovy & Grails eXchange 2012 - Building an e-commerce business with gr8 technologies in Latin America
Similar to Groovy & Grails eXchange 2012 - Building an e-commerce business with gr8 technologies in Latin America (20)
More from Domingo Suarez Torres
More from Domingo Suarez Torres (20)
Recently uploaded
Recently uploaded (20)
Groovy & Grails eXchange 2012 - Building an e-commerce business with gr8 technologies in Latin America
- 1. Building an e-commerce business with gr8 technologies in Latin America Groovy & Grails eXchange London 14th December 2012 Domingo Suarez Torres @domix lunes, 29 de abril de 13
- 2. Agenda • About the Business • Version 1 • Version 2 • Lessons learned lunes, 29 de abril de 13
- 3. About me • Java & Grails developer from México City • grails.org.mx Mexican G&G community Founder • My team won the second place in the Grails48 hackaton. • TrollBoard. Kanban Board for GitHub Issues • http://trollboard.rs.af.cm lunes, 29 de abril de 13
- 4. clickOnero • Started in 2010 as a Daily Deals website. • German/Swiss investors • Late 2011 acquired by Mexican investors • Today it is a 100% Mexican company with a pure ecommerce model lunes, 29 de abril de 13
- 5. clickOnero 1.0 Grails, Terracotta, RabbitMQ & mySQL lunes, 29 de abril de 13
- 6. clickOnero 1.0 • CodeName: CoolDeals • Typical Grails application • MySql,Tomcat, ehcache (Terracotta), RabbitMQ • Built in 2 months by 4 devs from Germany, UK, Greece and Mexico lunes, 29 de abril de 13
- 7. Caching • Prevent excessive database access • ehcache, backed with Terracotta • Stored in cache • Domain classes • HTML pages & fragments • JSON, XML documents lunes, 29 de abril de 13
- 8. Long running code • RabbitMQ • Messaging with AMQP • Asynchronous processing • Mainly to use STMP services. Millions emails sent daily • Financial reports (long complex queries) lunes, 29 de abril de 13
- 9. CoolDeals Deployment http://www.springstar.com lunes, 29 de abril de 13
- 10. CoolDeals Deployment http://www.springstar.com lunes, 29 de abril de 13
- 11. CoolDeals Deployment http://www.springstar.com lunes, 29 de abril de 13
- 12. Dev Team @ Mexico • Very hard to find Grails developers • Java developers -> Grails developers (4) • It’s easy to do! • New Grails developers excited about new possibilities and simplicity of the language and frameworks lunes, 29 de abril de 13
- 13. Very Bad stuff • Test cases only for critical pieces • checkout, user registration, mailing list subscription • Very low code coverage. • No Continuos integration • Internal backend was a mess, a lot of code in controllers • Bad GORM queries lunes, 29 de abril de 13
- 14. Enters JavaMelody • The goal of JavaMelody is to monitor Java or Java EE application servers in QA and production environments. • It is a tool to measure and calculate statistics on real operation of an application depending on the usage of the application by users. • ¡Grails Plugin! lunes, 29 de abril de 13
- 15. lunes, 29 de abril de 13
- 16. JDBC Queries lunes, 29 de abril de 13
- 17. JavaMelody • It helps us improve our app! lunes, 29 de abril de 13
- 18. More Plugins • RabbitMQ • Spring Security • Rendering • Quartz • Feeds • Export • Recaptcha • AWS • Joda Time • Java Melody lunes, 29 de abril de 13
- 19. Hardware lunes, 29 de abril de 13
- 20. Boxes • 4 WebServers • DBServer • Physical Firewall • LoadBalancer lunes, 29 de abril de 13
- 21. Numbers • 200k-300k visits daily • 1.5M > page views daily • 80K concurrent users • 20K-50K new users daily • 1000-3000 purchases daily • 2.5M registered users lunes, 29 de abril de 13
- 22. Logstalgia-httpd log viz lunes, 29 de abril de 13
- 23. “Ecommerce Company of the year in Mexico” lunes, 29 de abril de 13
- 24. Lessons learned • Cache is critical to scale • The content in our app is ‘static’, few changes, sometimes no changes. • If the content is ‘static’, why do we always execute code to render that content, even if the content is cached? lunes, 29 de abril de 13
- 25. clickOnero 2.0 Single Page Application (chaplin, backbone) lunes, 29 de abril de 13
- 26. Motivation • December 2011 clickOnero was acquired by Mexican investors • The company started to change. • The new CEO asked to develop a brand new store to sell different items like: clothes, shoes, trips, services (hair cuts, massages, cinema tickets, etc) lunes, 29 de abril de 13
- 27. New markets • clickOnero acquired other e-commerce startups to increase the market • Miplan.com: was a e-commerce company specialized in offering national and international trips. 500k users • PezUrbano is a daily deals website from Brazil, merging the two platforms is on the works. 1.5M users • Very close to 5M users lunes, 29 de abril de 13
- 28. Improvements • Split the complete system in several components • API: Grails app for exchange information • Admin. Grails application for internal use • Hipstore.A totally static HTML application built with Chaplin. Not a Grails app.The store for the users lunes, 29 de abril de 13
- 29. lunes, 29 de abril de 13
- 30. Grails Apps • API • Typical Grails application, with no GSPs. • Speaks only JSON • Used by partners (remote services) and HipStore • Admin • Typical Grails application lunes, 29 de abril de 13
- 31. Development enhancements • Increased test cases in both Grails apps • Spock • Jasmine for JavaScript code • Introduced Jenkins • Jenkins Jobs to deploy automatically to QA & Production environments • Bash shell scripts lunes, 29 de abril de 13
- 32. HipStore • Static HTML application built with Chaplin • Chaplin is an architecture for JavaScript applications using the Backbone.js library lunes, 29 de abril de 13
- 33. HipStore • Developed in CoffeeScript • Uses PushState • RequireJS (AMD Support) • HandleBars (Template Engine) • JQuery • Underscore • Twitter Bootstrap • Build and packaged with Jake lunes, 29 de abril de 13
- 34. HipStore • Single Page Application • Chaplin consumes JSON from the API to render the store items. • Uses PushState to update the URL in the browser.Very useful for bookmarking and social media sharing, even for SEO. lunes, 29 de abril de 13
- 35. Sample URLs • http://{server}/store/ • http://{server}/store/{category} • http://{server}/store/{category}/{campaign} • http://{server}/store/{category}/{campaign}/{item} • Above urls valid for the Chaplin router, but invalid for the browser, because them doesn’t exist. If you try to use it, you’ll get a 404 lunes, 29 de abril de 13
- 36. HashBang • http://{server}/store/#!/{category} • Not very friendly with search engines. • Google will try something like: • http://{server}/store?_escaped_fragment_= • Please read: https://developers.google.com/ webmasters/ajax-crawling/ lunes, 29 de abril de 13
- 37. Our approach • Write to disk all the possible links in HipStore. Crazy? • We use ZombieJS to navigate the website and then write to disk the generated HTML • Put those static files (HTML) in the webserver document root • The best cache ever lunes, 29 de abril de 13
- 38. Why do this? • When the user visit our website, the webserver will respond with completely static HTML, CSS, JavaScript files • Very fast • The user never hits the Tomcats, we reduce the load in the app servers. • The webserver always responds very fast lunes, 29 de abril de 13
- 39. ZombieJS lunes, 29 de abril de 13
- 40. HTML harvesting • The Node app receives the JSON message • Navigates to HipStore with ZombieJS • Executes the JavaScript (Chaplin app) • Extracts the generated HTML with jQuery • Saves to disk in the web server document root lunes, 29 de abril de 13
- 41. Things to consider • When an item in the Store is modified, we need to regenerate the appropriate HTML file only once. • Then all the users will receive the same file • The user only hits the Tomcat when really need it (CheckOut, Registration) • When the user click in some action in the app, the interaction is handled by Chaplin controller if the browser supports JavaScript • Remember in our website the content is almost static? lunes, 29 de abril de 13
- 42. Results • The load in our Web Servers was reduce a lot. • The load in the database reduced a lot. • The users can share the links. • Store becomes very search engine friendly lunes, 29 de abril de 13
- 43. Some stats lunes, 29 de abril de 13
- 44. Thanks! • Q&A lunes, 29 de abril de 13
Editor's Notes
- Hi my name is Domingo Suarez, I'm from Mexico City, and this afternoon I'm going to talk about my experience building the largest e-commerce website in Mexico with grails. I will talk about what worked well, what didn't, some lessons learned and so on. I only want to share my experience in my context
- Before I start, let me tell you a little about my background. I've been using Grails since late 2007, my first website built with grails went live in 2008. For all my professional career I've been a java developer, mostly on the server side. In Mexico I founded the Groovy/Grails user group. In the Grails48 hackaton my team won the second place for developing a kanban board for github issues. Let’s get started...
- For the last two years I've been working at clickOnero, an ecommerce company with a national presence in Mexico. Here is the story, In 2010 a friend of mine asked me if I knew an available Grails developer for hire. In Mexico is very hard to find Grails developers, very hard... At the end I accepted the job and joined clickOnero, a company with Swiss investors. The clickOnero investors made more investments around the world with the same business model.
- In he beginning, The clickOnero's business model was to sell coupons for daily deals. The CEO asked us to develop an in-house solution for this.
- We built the solution in 2 months, with a team from all over the world, 4 developers from: Germany, UK, Greece and Mexico. I went to Germany to work together with them and then came back to Mexico to deploy and launch the company. The first version was built using the common Grails artifacts, controllers, gsps, taglibs, domain classes and services. This was deployed to tomcat with MySQL as database. When we started to build cooldeals, we made some crucial technical decisions.
- For example, to cache heavily with ehcache, backed with terracotta in production.
- From the very beginning we had some requirements related to massive emails campaigns and chose RabbitMQ for long running operations like sending emails.
- We went live in Mexico the first week of September 2010, one week later we launched the website in Argentina, both websites with the same branding and the same platform. Two weeks later, the Dubai version went live with a different branding but same platform. The platform was built with multi currency and multilingual support, for 2 different continents and 6 different countries (the Dubai version runs in 4 different countries in the Middle East)a
- Meanwhile in Mexico, I help put together the dev team. I hired 4 local Java developers and converted them to Groovy/Grails developers, it was really easy to do. The developers were very excited about the new possibilities and simplicity of the language and frameworks.
- But, we made bad things. In the same application, we built the website for the customers and the administration backend. We wrote only test cases for the critical pieces (checkout, user registration). The internal backend was really a mess!
- We started to have issues. Slow response times. We didn’t know where the problem was We used JavaMelody to detect bottlenecks and long running code. It was really helpful to fixed the issues we had.
- JavaMelody generates this graphs, where you can see problem in your app. Memory problems, jdbc problem, threads
- This is an example of jdbc queries. You can see the hit count, mean time response, max, min Of course you can use this data to identify bad code in your app
- As an IT department, we were responsible for the hardware infrastructure as well. No sysadmin and no dba in the company, only 5 developers. The hardware we had was 4 app servers(tomcat) and one database(MySQL) server. Here is a bit more technical detail on these servers: With this infrastructure we were able to support the company's growth. By late 2011 we had 2 million users, with 50k concurrent users. Show logstalgia visualization
- As an IT department, we were responsible for the hardware infrastructure as well. No sysadmin and no dba in the company, only 5 developers. The hardware we had was 4 app servers(tomcat) and one database(MySQL) server. Here is a bit more technical detail on these servers: With this infrastructure we were able to support the company's growth. By late 2011 we had 2 million users, with 50k concurrent users. Show logstalgia visualization
- The company worked very well for several months, in fact the last year we won the prize for the best Internet company in Mexico, a prize granted for the Mexican Internet Association (AMIPCI)
- At December 2011, the company was acquired by Mexican investors and the company started to change to improve the business performance.
- At this time the investors acquired other e-commerce startups to increase the clickOnero market:-MiPlan.com was a ecommerce company specialized in offering national and international trips. When this company was acquired, the clickOnero user base increased to 3 Million users-PezUrbano is a daily deals website from Brazil, merging the two platforms is on the works. The clickOnero's user base will increase by 1.5 million users
- We divided all the system in several components:-API, Grails application only for data exchange. This app exposes http services for getting data with basic auth http. In this app, we built the checkout, stock, user registration, mailing list subscription services.-Hipstore. A totally static HTML application built with Chaplin, an abstraction over backbone, with a templating engine. This app is served from the web server, with gzip and cached in the client with etag if available. This app consumes the API services-Admin. Grails application for internal use, the production team uses this app to create items campaigns, the customer service department uses this app for customer support. The sales team uses this app for tracking sales
- The admin apps users made some changes in the database items, when the change is committed to the database, the admin app sends a JSON message to a RabbitMQ A node app listen that message and with ZombieJS navigate the
- We saw the users spend more time in the store, we reduced the bounce rate and guess what, We went down from 4 to 2 app servers. The users didn't notice the reduction in hardware. (show the google analytics image)