With RabbitMQ, Redis, and Groovy on Grails, Mercado Libre has a set of open source capabilities and a surrounding community to help them improve development lifecycles and scale both applications and data services, ultimately improving the customer experience. As they look to future growth, they have a solid foundation with a cloud-based architecture they can rely on. To learn more, visit pivotal.io/platform-as-a-service/pivotal-cf.

1. MercadoLibre (NASDAQ: MELI) is eBay’s Latin American partner, the largest e-commerce ecosystem in Latin America, and comScore’s 8th largest e-retailer in the world. A Fortune “Fastest-Growing Company” for three years, 2013 showed net revenues and gross merchandise volume up by 50%, total payment volume up 66%, and registered users up 22% to 99.5 million. Headquartered in Argentina, they operate across thirteen countries, and allow people to list items, sell, buy, pay, and collect payments online. They also provide payment solutions, digital advertising, and web stores.
As a digital business, years of growth began to impact the original architecture, and analysts forecasted more growth in the future. In 2010, they began transforming the original, monolithic e-commerce system into a next-generation, decoupled architecture to provide independence to internal teams while scaling applications, data, and data centers. They also expanded the core API layer which provides access to users, items, orders, and more.
CHALLENGE
Two things drove Mercado Libre to change—speed to market and scaling past 20 million requests per minute and 4GB of bandwidth per second.
User Experience Suffers from Slow Release Cycles and Intertwined Development
Applications must scale to achieve solid user experience and customer satisfaction. Matias Waisgold, Technical Lead for the core API, explained, “Over time, our services and applications became entangled. This caused problems. Often, one team’s release impacted other team’s code. Our release frequency slowed, and we couldn’t improve user experience fast enough. So, we set a vision—an enterprise service bus with decentralized, decoupled applications that support an agile process.”
Revenue at Risk with Traditional Databases, Tight Coupling, and Single Data Centers
Mercado Libre’s application generates revenue. So, scale and uptime are critical. “As our user-base grew, we needed a multi-
AT-A-GLANCE
Challenges
•
Improving time to market and user experience
•
Scaling past 20 million requests per minute and 4GB bandwidth per second
•
Reliably supporting revenue while removing traditional database bottlenecks
•
Connecting micro-services with an ESB architecture
Solution
•
RabbitMQ
•
Redis
•
Groovy on Grails
Key Benefits
•
Improved development agility, time to market, and user experience
•
Scaling data stores and applications across languages with an ESB
•
Open source solutions and support communities
CASE STUDY
Mercado Libre
IMPROVING USER EXPERIENCE, DEVELOPMENT AGILITY, AND SCALING TRADITIONAL DBS
OVERVIEW
“ Our ESB, built largely with RabbitMQ, Groovy on Grails, and Redis, has a big impact on development and operations. We can scale individual apps and data services instead of wrestling with one big monolithic structure where we would be bound by the slowest part. Asynchronous and independent components also allow our teams to develop and deploy independently with less governance and coordination. Together, these three tools help us get to market faster, improve our competitiveness, make a better user experience, and ultimately impact revenue.”
—Matias Waisgold, Technical Lead, Mercado Libre
pivotal.io

2. Pivotal is a registered trademark or trademark of Pivotal Software, Inc. in the United States and other countries. All other trademarks used herein are the property of their respective owners. © Copyright 2014 Pivotal Software, Inc.
All rights reserved. Published in the USA. PVTL-CS-367-06/14
At Pivotal our mission is to enable customers to build a new class of applications, leveraging big and fast data, and do all of this with the power of cloud independence.
Uniting selected technology, people and programs from EMC and VMware, the following products and services are now part of Pivotal: Greenplum, Cloud Foundry, Spring,
GemFire and other products from the VMware vFabric Suite, Cetas and Pivotal Labs.
CASE STUDY MERCADO LIBRE
Pivotal 3495 Deer Creek Road Palo Alto, CA 94304 pivotal.io
datacenter architecture to avoid failures and help scale. We
knew our Oracle database wouldn’t support the volume of
reads and writes, and both users and usage were growing across
geographies. We designed a hybrid cloud with reads/writes in
one location and reads from other data centers. We also needed
a cache layer to take load off the database. Lastly, we wanted to
support a distributed computing model that allowed us to scale
specific services or components,” said Waisgold.
SOLUTION
Delivering Faster Releases and Scaling with RabbitMQ, Redis,
and Groovy
Largely an open source shop, the Mercado Libre team built a
roadmap. At the application layer, they broke the application into
micro-services and transformed each into an open, API-based
platform connected by an enterprise service bus (ESB).
At the data layer, they designed the main data center to support
reads and writes with 100% data consistency. The team relaxed
data consistency rules for reads in the other data centers and
used Oracle’s, Java-based Golden Gate technology to replicate
across data centers and feed changes to RabbitMQ within a single
data center. RabbitMQ messages to a refetching service written
in Groovy on Grails that updates a Memcached cluster and cost
effectively scales the RDBMS. The refetchers also publish to a
system called Big Queue that uses Redis for in-memory storage.
Big Queue stores events as separate topics within each datacenter.
These are consumed asynchronously by other apps in parallel. It
supports an HTTP-based API, is highly available, and consistent
within a data center, ultimately delivering the ESB messages to
dozens of services.
By taking this approach, components independently scale
to address bottlenecks. Each application now operates
asynchronously and uses the ESB to listen for relevant events
from other apps like a new listing or an update to an item. For
example, data changes pass through the bus to notify and update
Elastic search, and the developer APIs for items updates both
Oracle and RabbitMQ directly, leading to cache updates and ESB
events. The decentralized architecture allows each business unit
and development team to operate with autonomy. As long as
shared APIs are respected, developers can deploy specific app
updates or scale components as needed. Users of the site still see
a seamless, single experience made up of dozens of apps each
controlled by a separate team.
BUSINESS BENEFITS
Improved User Experience, Time to Market, and
Development Agility
“RabbitMQ is certainly very easy to understand, use, and
implement. However, our ESB, built largely with RabbitMQ,
Groovy on Grails, and Redis, has a bigger impact on development
and operations teams. We can scale individual apps and data
services instead of wrestling with one big monolithic structure
where we would be bound by the slowest part. Asynchronous
and independent components also allow our teams to develop
and deploy independently with less governance and coordination.
Groovy helps us accelerate the time we spend coding and testing.
Redis stores key-values of big data in memory for the real-time
use cases across our ESB. Together, these three tools help us get
to market faster, improve our competitiveness, make a better user
experience, and ultimately impact revenue,” said Waisgold.
Scaling Data Stores and Applications across Languages
Waisgold explained, “RabbitMQ is lightweight, supports the open
standard AMQP protocol, and helps to scale our existing RDBMS
by feeding the cache and Big Queue through Groovy, ultimately
fanning out to other apps. Across 18 queues that are fed by
Oracle or external APIs, RabbitMQ processes spikes up to 1000
messages per second and 2000 acknowledgements per second
on a 4 machine cluster with 16 GB of RAM and 8 cores each.
We see 0.188-second transactions writing to RabbitMQ, which
is super fast. With external developer APIs for items, we write
to RabbitMQ directly from our webservers without degrading
performance. RabbitMQ gives us a simple, reliable way to scale on
pools of shared infrastructure or components and makes it easy
for any app to talk to Java, Groovy on Grails, Ruby, or virtually any
other language we might use. Similarly, the Redis data store works
with all modern development languages, and this gives us tons of
flexibility for downstream apps.”
CONCLUSION
With RabbitMQ, Redis, and Groovy on Grails, Mercado Libre has
a set of open source capabilities and a surrounding community
to help them improve development lifecycles and scale both
applications and data services, ultimately improving the customer
experience. As they look to future growth, they have a solid
foundation with a cloud-based architecture they can rely on.
LEARN MORE
To learn more about our products, services and solutions, visit us
at pivotal.io.