How a Hackathon Approach Juices Innovation on Big Data Applications for Thomson Reuters
How a Hackathon Approach Juices Innovation on Big Data
Applications for Thomson Reuters
Transcript of a Briefings Direct podcast on how information giant Thomson Reuters leveraged a
"hackathon" to spur new ideas among its developers.
Listen to the podcast. Find it on iTunes. Sponsor: HP
Dana Gardner: Hello, and welcome to the next edition of the HP Discover Podcast Series. I'm
Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator for this
ongoing discussion on IT innovation and how it's making an impact on people’s
lives.
Once again, we're focusing on how companies are adapting to the new style of
IT to improve IT performance, gain new insights and deliver better user
experiences, as well as better overall business results.
This time we're coming to you directly from the HP Discover 2014 Conference
in Barcelona to learn directly from IT and business leaders alike how big data changes
everything, for IT, for businesses, and governments, as well as for you and me.
Bringing human understanding to the cloud
Sign up for IDOL on Demand
Helping developers build a new class of apps
Our next innovation interview explores the use of a "hackathon" approach to unlock creativity in
the search for better use of data for analytics. We will hear how Thomson Reuters in London
sought to foster innovation and derive more value from its vast trove of business and market
data.
The result: A worldwide virtual hackathon, that brought together developers and data scientists to
uncover new applications, visualizations, and services to make data actionable and impactful.
To learn more about getting developers on board the big-data analysis gravy train, please join me
in welcoming our guest. We're here with Chris Blatchford, Director of Platform
Technology in the IT organization at Thomson Reuters in London. Welcome,
Chris.
Chris Blatchford: Hi. Good to be here.
Gardner: Tell us about Thomson Reuters, the organization, for those who don't know.
Gardner
Blatchford: Thomson Reuters is the world's leading source of intelligent information. We
provide data across the finance, legal, news, IP, and science, tax, and accounting industries
through product and service offerings, combining industry expertise with innovative technology.
Gardner: It’s hard to think of an organization where data and analysis is more important. It’s so
core to your very mission.
Blatchford: Absolutely. We take data from a variety of sources. We have our own original data,
third party sources, open data sources, and augmented information, as well as all of the original
content we generate on a daily basis For example, our journalists in the field provide original
news content to us directly from all over the globe. We also have third-party licensed data that
we further enrich and distribute to our clients through a variety of tools and services
Gardner: And therein lies the next trick, what to do with the data once you have it. About this
hackathon, how did you come up upon that as an idea to foster innovation?
Big, Open, Linked Data
Blatchford: One of our big projects or programs of work currently is, as everyone else is
doing, big data. We have an initiative called BOLD, which is Big, Open, Linked Data, headed up
by Dan Bennett. The idea behind the project is to take all of the data that we
ingest and host within Thomson Reuters, all of those various sources that I just
explained, stream all of that into a central repository, cleanse the data, centralize
it, extract meaningful information, and subsequently expose it to the rest of the
businesses for use in their specific industry applications.
As well as creating a central data lake of content, we also need to provide the
tools and services that allow businesses to access the content; here we have both
developed our own software and licensed existing tools.
So, we could demonstrate that we could build big-data tools using our internal expertise, and we
could demonstrate that we could plug in third-party specific applications that could perform
analysis on that data. What we hadn’t proved was that we could plug in third-party technology
enterprise platforms in order to leverage our data and to innovate across that data, and that’s
where HP came in.
HP was already engaged with us in a number of areas, and I got to speaking with their Big Data
Group around their big data solutions. IDOL OnDemand came up. This is now part of the Haven
OnDemand platform. We saw some synergies there between what we were doing with the big-
data platform and what they could offer us in terms of their IDOL OnDemand API’s. That’s
where the good stuff started.
Gardner: Software developers, from the very beginning, have had a challenge of knowing their
craft, but not knowing necessarily what their end users want them to do with that craft. So the
Blatchford
challenge -- whether it’s in a data environment, a transactional environment or interface, or
gaming -- has often been how to get the requirements of what you're up to into the minds of the
developers in a way that they can work with. How did the hackathon contribute to solving that?
Blatchford: That’s a really good question. That’s actually one of the biggest challenges big data
has in general. We approach big data in one of two ways. You have very specific use cases, for
example, consider a lawyer working on a particular case for a client, it would be useful for them
to analyze prior cases with similar elements. If they are able to extract entities and relevant
attributes, they may be able to understand the case final decision, or perhaps glean information
that is relevant to their current case.
Then you have the other approach, which is much more about exploration, discovering new
insights, trends, and patterns. That’s similar to the the approach we wanted to take with the
hackathon -- provide the data and the tools to our developers for them just to go and play with
the data.
We didn’t necessarily want to give them any requirements around specific products or services. It
was just, "Look, here is a cool platform with some really cool APIs and some capabilities. Here
is some nice juicy data. Tell us what we should be doing? What can we come up with from your
perspective on the world?"
A lot of the time, these engineers are overlooked. They're not necessarily the most extroverted of
people by the nature of what they do and so they miss chances, they miss opportunities, and
that’s something we really wanted to change.
Gardner: It’s fascinating the way to get developers to do what you want them to do is to give
them no requirements.
Interesting end products
Blatchford: Indeed. That can result in some interesting end products. But, by and large, our
engineers are more commercially savvy than most, hence we can generally rely on them to
produce something that will be compelling to the business. Many of our developers have side
projects and personal development projects they work on outside of the realms of their job
requirement. We should be encouraging this sort of behavior.
Gardner: So what did you get when you gave them no requirements, what happened?
Blatchford: We had 25 teams that submitted their ideas. We boiled that down to 7 finalists based
upon a set of preliminary criteria, and out of those 7, we decided upon our first-, second-, and
third-place winners. Those three end results were actually taken, or are currently going through a
product review, to potentially be implemented into our product lines.
The overall winner was an innovative UI design for mobile devices, allowing users to better
navigate our content on tablets and phones. There was a sentiment analysis tool, that allowed
users to paste in news stories or any news content source on the web and extract sentiment from
that news story.
And the other was more of an internally focused, administrative exploration tool, that allowed us
to more intuitively navigate our own data, which perhaps doesn’t initially seem as exciting as the
other two, but is actually a hugely useful application for us.
Gardner: Now, how does IDOL OnDemand come to play in this? IDOL is the ability to take any
kind of information, for the most part, apply a variety of different services to it, and then create
analysis as a service. How did that play into the hackathon? How did the developers use that?
Bringing human understanding to the cloud
Sign up for IDOL on Demand
Helping developers build a new class of apps
Blatchford: Initially the developers looked at the original 50-plus APIs that IDOL OnDemand
provides, and you have everything in there from facial recognition, to OCR, to text analytics, to
indexing, all sorts of cool stuff. Those, in themselves, provided sufficient capabilities to produce
some compelling applications, but our developers also utilized Thomson Reuters API’s and
resources to further augment the IDOL platform.
This was very important, as it demonstrated that not only could we plug in an Enterprise
analytics tool into our data, but also that it would fit well with our own capabilities.
Gardner: And HP Big Data also had a role in this. How did that provide value?
Five-day effort
Blatchford: The expertise. We should remember we stood this hackathon up from inception to
completion in a little over one month, and that’s I think pretty impressive by any measure.
The actual hackathon lasted for five days. We gave the participants a week to get familiar with
the APIs, but they really didn’t need that long because the documentation behind the APIs on
IDOL OnDemand and the kind of "try it now" functionality it has was amazing. This is what the
engineers and the developers were telling me. That’s not my own words.
The Big Data Group was able to stand this whole thing up within a month, a huge amount of
effort on HP’s side that we never really saw. That ultimately resulted in a hugely successful
virtual global hackathon. This wasn’t a physical hackathon. This was a purely virtual hackathon
the world over.
Gardner: HP has been very close to developers for many years, with many tools, leading tools
in the market for developers. They're familiar with the hackathon approach. It sounds like HP
might have a business in hackathons as a service. You're proving the point here.
For the benefit of our listeners, if someone else out there was interested in applying the same
approach, a hackathon as a way of creating innovation, of sparking new thoughts, light bulbs
going off in people's heads, or bringing together cultures that perhaps hadn't meshed well in the
past, what would you advise them?
Blatchford: That’s a big one. First and foremost, the reason we were successful is because we
had a motivated, willing partner in HP. They were able to put the full might of their resources
and technology capabilities behind this event, and that along side our own efforts ultimately
resulted in the events success.
That aside, you absolutely need to get the buy-in of the senior executives within an organization,
get them to invest into the idea of something as open as a hackathon. A lot of hackathons are
quite focused on a specific requirement. We took the opposite approach. We said, "Look,
developers, engineers, go out there and do whatever you want. Try to be as innovative in your
approach as possible."
Typically, that approach is not seen as cost effective, businesses like to have defined use cases,
but sometimes that can strangle innovation. Sometimes we need to loosen the reins a little.
There are also a lot of logistical checks that can help. Ensure you have clear criteria around
hackathon team size and members, event objectives, rules, timeframes and so on. Having these
defined up front makes the whole event run much smoother.
We ran the organization of the event a little like an Agile project, with regular stand-ups and
check-ins. We also stood up a dedicated internal intranet site with all the information above.
Finally, we set-up user accounts on the IDOL platform early on, so the participants could
familiarize themselves with the technology.
Winning combination
Gardner: Yeah, it really sounds like a winning combination: the hackathon model, big data as
the resource to innovate on, and then IDOL OnDemand with 50 tools to apply to that. It’s a very
rich combination.
Blatchford: That’s exactly right. The richness in the data was definitely a big part of this. You
don’t need millions of rows of data. We provided 60,000 records of legal documents and we had
about the same in patents and news content. You don’t need vast amounts of data, but you need
quality data.
Then you also need a quality platform as well. In this case IDOL OnDemand.The third piece is
what’s in their heads. That really was the successful formula.
Gardner: I have to ask. Of course, the pride in doing a good job goes a long way, but were there
any other incentives; a new car, for example, for the winning hackathon application of the day?
Blatchford: Yeah, we offered a 1960s Mini Cooper to the winners. No, we didn't. We did offer
other incentives. There were three main incentives. The first one, and the most important one in
my view, and I think in everyone’s view, was exposure to senior executives within the
organization. Not just face time, but promotion of the individual within the organization. We
wanted this to be about personal growth as much as it was about producing new applications.
Going back to trying to leverage your resources and give them opportunities to shine, that’s
really important. That’s one of the things the hackathon really fostered -- exposing our talented
engineers and product managers, ensuring they are appreciated for the work they do.
We also provided an Amazon voucher incentive, and HP offered some of their tablets to the
winners. So it was quite a strong winning set.
Gardner: Sounds like a very successful endeavor others out there might be interested in
emulating, so I am glad we could find out more about it.
I'm afraid we're going to have to leave it there. We've been learning about how Thomson Reuters
in London sought to foster innovation and derive more value from its vast trove of business and
market data, and how they used a virtual worldwide hackathon approach to bring together
developers and data scientists to uncover new applications and services.
So a big thank you to our guest. We've been joined here with Chris Blatchford, the Director of
Platform Technology in the IT organization at Thomson Reuters in London. Thank you, Chris.
Blatchford: Thanks.
Bringing human understanding to the cloud
Sign up for IDOL on Demand
Helping developers build a new class of apps
Gardner: And I'd like to thank our audience as well for joining us for this special new style of
IT discussion coming to you directly from the HP Discover 2014 Conference in Barcelona.
We've explored solid evidence from early adopters of how big data changes everything, for IT,
for businesses and governments, as well as for you and me.
I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host for this ongoing series of
HP-sponsored discussions. Thanks again for listening, and come back next time.
Listen to the podcast. Find it on iTunes. Sponsor: HP
Transcript of a Briefings Direct podcast on how information giant Thomson Reuters leveraged a
"hackathon" to spur new ideas among its developers. Copyright Interarbor Solutions, LLC,
2005-2015. All rights reserved.
You may also be interested in:
•
How Waste Management Builds a Powerful Services Contiunuum Across Operations,
Infrastructure, Development, and IT Processes
•
GSN Games hits top prize using big data to uncover deep insights into gamer preferences
•
Hybrid cloud models demand more infrastructure standardization, says global service
provider Steria
•
Service providers gain new levels of actionable customer intelligence from big data
analytics
•
How UK data solutions developer Systems Mechanics uses HP Vertica for BI, streaming
and data analysis
•
Advanced cloud service automation eases application delivery for global service provider
NNIT
•
HP network management heightens performance while reducing total costs for Nordic
telco TDC
•
How Capgemini's UK financial services unit helps clients manage risk using big data
analysis
•
Perfecto Mobile goes to cloud-based testing so developers can build the best apps faster
•
Software security pays off: How Heartland Payment Systems gains steep ROI via
software assurance tools and methods
•
HP ART documentation and readiness tools bring better user experiences to Nordic IT
solutions provider EVRY