Anyone can build a loosely affiliated, unstructured crowd - a mob. The key to successfully employing a crowdsourcing model in a b2b/professional services type space is to advance beyond the realm of a ‘mob’ to create an engaged, interactive community of diverse and skilled professionals. With the help of reputation and compensation systems, community recruitment and engagement, public profiles and social media, crowdsourcing has the potential to take the services industry to new heights. Using real-world examples, Johnston will dispel the most common myths about crowdsourcing; explain why it doesn’t mean the end to in-house staffs; and reveal why it is NOT just another marketing buzz word.

1. Online Communities
And How They‘re Changing the Way
Work Gets Done
Matt Johnston | Chief Marketing Officer
matt@utest.com | @matjohnston

2. Agenda
• Session Intro
• Crowdsourcing 101
• Secret Sauce
• Where It Fits (And Where It Doesn‟t)
• Myths & Misconceptions
• Q&A

3. What‘s Crowdsourcing?
“The act of taking a job or service traditionally
performed by a designated person or team
and sourcing it to a large group of people in
the form of an open call.”
- Jeff Howe,
Wired journalist & author of Crowdsourcing

4. Crowdsourcing Across Industries

5. BUT crowds often look (and act)
like unruly mobs

6. And mobs don‘t work in every category
Example:
Delivering a skilled service at an enterprise level
of predictability and efficiency requires an orderly
“community” capable of consistently producing the
desired results

7. So how do you turn this –

8. …. into this?

9. The Secret Sauce of Crowdsourcing

10. 4 Crowdsourcing Building Blocks
1. Community profiling
– Technical: OS, browser, mobile devices, carriers
– Geographic: City, country, languages
– Demographic: Age, gender, education, hobbies
2. Community ratings & micro-ratings
– By testing type
– By industry
3. Highly precise matching
– Between each project & each tester
4. Services layer for enterprise clients
– Maximize throughput
– Minimize overhead

11. How Does This Work In Testing?
Recruit, train & incentivize testers:
• Two forms of compensation in crowdsourcing
1. Monetary
– Paid for approved bugs, test cases, usability surveys
2. Reputation
– Robust tester rating system and badges
– Customers can mark testers as „favorites‟
• Other forms of community engagement
o EG: uTest University (all community-driven)
– Training & certification
– Career advice
– Networking with peers
– Product reviews

12. Community Management: (Hard) Lessons Learned
Scaling Requires:
• Rating & reputation systems that drive desired behavior
• Matching that sets people up to succeed
• Creating tools & programs to serve the masses
- Best proof point: Two years ago, uTest community was 3x with 10x
revenue… yet total inquiries in 2011 are now lower
• Embracing FUBU concepts, such as:
- Community vetting community: uTest Sandbox
- Community teaching community: Crash Courses
- Community managing community: Test Team Leads
- Community policing community: Forums Moderators

13. Where Crowdsourcing Fits
(And Where It Doesn‘t)

14. Tenet 1: Know Thy Task

15. Tenet 2: Know Thyself
• Is your culture highly cautious & risk-averse?
• Are you in a highly regulated industry?
– Defense industry
– PII and PHI
• Do you have an appetite for innovation?
• Are you centrally organized or decentralized?
– Sourcing
– IT & IS
– Engineering

16. Tenet 3: Know Thy Partner
Selection criteria are vital:
• Referenceable customer successes
– By company size
– By industry
• Ability to adapt to your legacy systems and processes
• Ability to satisfy legal requirements
– IP protection
– NDA
• Ability to do the job
– Consistent
– Predictable
– Professional

17. Tenet 4: Know Thy Trends
(Enterprises that already leverage crowdsourcing)

18. Myths & Misconceptions

19. Myth #1
Less In-House Staff, Less Overhead
TRUTH:
• Complement & scale “as needed”…not replace
• Only employees will fully understand strategy,
company position, internal processes
BEST PRACTICES:
• Build strengths around employees‘ core competencies
• Where does the crowd ‗fits‘ into your team?
– Fresh eyes, fresh perspectives, and fresh ideas
– Rote or repetitive tasks

20. Case Study
How Google Taps Into The Crowd…
To Launch Great Apps
BACKGROUND:
• Google tests 15+ software products via crowdsourcing
- Web products
- Desktop clients
- Mobile applications
LESSONS from a BRAND LEADER:
• Pick the right projects
– Public-facing apps with diverse user bases
• Rethink ‗outside the box‘ in maximizing team expertise
• Rethink ‗team‘ to maximize flexibility and scale
• Save overhead on testing that might otherwise be cost-prohibitive

21. Myth #2
…One Less Project to Manage—Yeah!
TRUTH:
• Crowdsourcing (when executed well) should NOT require more time
to manage than internal team… should make life easier, not harder
• But, it still requires effective, detailed communications
BEST PRACTICES:
• Assign an internal project manager
– Secure executive buy-in
– Ensure flow of information
– Track deadlines, progress
• Hand-off clearly defined ―pieces‖ of a project

22. Myth #3
One crowdsourcing vendor should be all I need.
TRUTH:
• Some crowdsourcing firms do many, different tasks well within
similar scope
– Crowdflower
– Samasource
– TaskRabbit
• Crowdsourcing boom = specialization
BEST PRACTICE:
• Match the work to the vendor
– A crowd vendor produced great web design for you…
– But…should that same company be your „go-to‟ for advertising vs.Trada?

23. Myth #4
But we need to own the results
TRUTH:
• Intellectual property (IP) is a legit issue, esp. for the new & innovative
– Copywriting
– Design
– Animation
– Code writing
• IP policies vary within crowdsourcing orgs
BEST PRACTICES:
• Establish the ground rules: What‘s their IP policy?
– How is it communicated to the crowd?
– How is it enforced?

24. Myth #5
Tell the crowd? But we‘re in stealth mode!
TRUTH:
• Confidentiality is a serious matter with material consequence.
• Crowdsourcing is often used for pre-launch products
BEST PRACTICES:
• Ask upfront about preventative measures
– Precautions… NDAs? What else?
– Is the policy all-inclusive? e.g. incl. social media, message boards,
etc.
– What are the consequences for breaches?

25. Key Takeaways
• The nature of “work” and “jobs” is changing
• Crowdsourcing has already permeated the enterprise
• Moving upstream into more technical, sophisticated work
• Engineering
• Testing
• Advertising
• Marketing
• And like any other trend
- There are highly proficient actors… and posers
- There are times to leverage it… and not
- Growing in momentum… and based on VC funding, that won‟t change
- The enterprises that figure out how/where/when to deploy it, will win

26. Questions?
Answers.