What does it take to win the Kaggle/Yandex competition

•Transferir como PPTX, PDF•

6 gostaram•4,614 visualizações

Kenji LEFEVRE

A feedback on how we won Kaggle/Yandex competition

Tecnologia Negócios

WHAT DOES IT TAKE TO WIN THE
KAGGLE/YANDEX COMPETITION

Christophe Bourguignat
Kenji Lefèvre-Hasegawa
Paul Masurel @Dataiku
Matthieu Scordia @Dataiku

OUTLINE OF THE TALK

• Review of the Kaggle/Yandex Challenge
• How we worked (team work & tools)
• The winning model

GOAL Re-rank URLs returned by Yandex according to
the personal preferences of the users
url1

url3

url2

url2
GOAL

url3

url1

url4

url4

ML CHALLENGE Predict user’s pertinence

for urls and rerank result set accordingly
The Kaggle/Yandex challenge

GIVEN
• 30 days logs test: 3 days, train: 27 days
• Users historic queries, clicks & dwell-times
Q

Q

Q

Q

• Test session prior activity queries, clicks & dwell-times
Test session :

SIZE
• 15Gb size
The Kaggle/Yandex challenge

Q

Q

T

?

QUALITY METRIC
• One query test / user on the last 3 days
• NDCG metric penalize error of pertinence on top ranked
urls

• No A/B test
url1
url2

OK

BAD

url4
url3

Kaggle
The Kaggle/Yandex challenge

Prediction

Another ranking

TEAM DATAIKU SCIENCE STUDIO / KAGGLE

•
•
•
•

Christophe Bourguignat Engineer, Data enthusiastic
Kenji Lefèvre-Hasegawa Ph.D. math, new to ML
Paul Masurel Software Engineer @dataiku
Matthieu Scordia Data Scientist @dataiku
First meeting : October16th 2013

How we worked (Team work & tools)

WE’VE USED
•
•
•
•
•

Related papers (mainly Microsoft’s)
12 core, 64 Gb
Python scikit-learn
Dataiku Science Studio
Java Ranklib

How we worked (Team work & tools)

DATAIKU SCIENCE STUDIO
Features & labels

Features

Labels

Split train & validation

Original train

LEARNING
Team members
work independantly

FEATURES CONSTRUCTION
Team members work
independantly
DATA DRIVEN
COMPUTATION
How we worked (Team work & tools)

HOW MUCH WORK ?
• 960+ emails
• 360+ features
• 50+ ideas grid tuned (300+ models fitted)
• Server heavily loaded the last 3 weeks
• 56 kaggle submissions
• 196 teams, 264 players, 3570 submissions

How we worked (Team work & tools)

2014-01-01

1st

Future top 2 & 3
enter race

1 week

3rd

1 week

1st

5th

Top 10

Top 25

1/2 month

1 week

PROBLEM ANALYSIS
Query

Result Set
• Rank
• URL Snippet Quality
• URL is skipped, clicked or missed

CLICK
Reading URL
• URL & Domain pertinence with dwell-time

The winning model

FEATURES
Features :
• Rank
• User habits, query specificity (entropy, frequency,…)
• Snippet pertinence
• Missed, Skipped, Clicked
• URL & Domain Pertinence
Declinaison of
& Clicked
• Probability, Stimuli freq., Mean Reciprocal Rank (MRR)
• For each user : historic & previous activity in test session &
aggregate
• For all user
• Declined for all queries & same query
The winning model

MODELS
• Random Forest (predict proba)
+ maximize E(NDCG)
Kaggle/Yandex Top 1
then 3rd

• Lambda MART (Gradient Boosting Tree
optimized for NDCG) WINS !
The winning model

Mais conteúdo relacionado

Semelhante a What does it take to win the Kaggle/Yandex competition

Emptying Your Cup an Agile Primer Emptying Your Cup an Agile Primer is a introductory overview of Agile project management presented by Bruce Nix an experience Agile coach and project manager. Presenter Bruce Nix is an Agile Coach and Sr. Project Manager with Lokion (www.lokion.com), a digital interactive agency that specializes in ecommerce, enterprise governance and digital strategy. Bruce has been actively engaged in agile adoptions for over 7 years and has worked diligently at influencing enterprises to take more lean and agile approaches to product delivery and has the scars and psychology bills to prove it.

Emptying Your Cup an Agile Primer

Todd Shelton

Agile testing experiments

Baiju Joseph

Come hear how we used agile development and Sencha tools to meet our design requirements, to create a system that is highly configurable, flexible, and exceeds the expectations of our customers. Learn how to use an adaptive/responsive design to be able to support two very different types of users, with a single application, and using the same set of libraries. Most importantly, learn how to create a system that even your most important and demanding users will find value and actually want to use. By assembling some of the best programmers and DBA developers in the world, we have been able to create a best-in-class, fully functional, scalable and highly configurable system, while maintaining an amazingly easy-to-use interface.

SenchaCon 2016: Creating a Flexible and Usable Industry Specific Solution - D...

Sencha

All organisations aspiring agile don’t have a significant level of automated checks available. Deciding to start adding automation on top of legacy code is a long road to travel and while on route, releases will rely on more manual testing. Meanwhile, automating builds to a single click deploy already enables delivering. This talk tells a story of going into mindset of continuous delivery of continuous releases to production without the level of test automation often seen as a key practice. You will learn from the changes we went through moving from monthly (scrum) to continuous (kanban) releases as we started delivering value in production in small incremental changes. The talk will help you see one route to a happy agile team with satisfied customers, where the recipe is built more around collaboration and smart exploratory testing than test automation to deliver a steady stream value.

Continuous Delivery without Significant Test Automation

Maaret Pyhäjärvi

During my last 10+ years of agile testing journey we have tried out few experiments and innovations. Some of them worked really well and some of them failed. Some of the experiments which yielded good results are in the areas of test automation infrastructure, Test Huddles, thinning the developer-tester boundary, usage of mind maps, theme based group tests etc...Usage of A/B testing also yielded good results in some of the projects.

My Experiments In Agile Testing in Yahoo.pptx

Baiju Joseph

Play with Kaggle

Matthieu Scordia

Presented by Ricardo Merizalde, Software Development Manager, Backcountry.com The intent of this presentation is to describe an implementation of an open source framework for e-commerce sites. The presentation will focus on Oracle ATG but can be extended to other platforms. First, a brief introduction on what CommerceSearch is (an open source integration and framework for eCommerce sites). Review challenges we had with FAST Impulse. Review the main search artifacts merchandisers can manage through CommerceSearch. Review how CommerceSearch integrates with ATG to deploy changes in near real time. Review CommerceSearch integration test framework and automated test framework (Selenium). Finally, summarize the benefits we got by moving from FAST to Solr.

Moving from fast to solr on atg

lucenerevolution

CommerceSearch: Moving from FAST to Solr on ATG

lucenerevolution

Agile Testing

danielbilling

20121213 qa introduction smileryang

netdbncku

Jay Trimble - NASA Ames Research Center I will present my teams experience in going from traditional development to agile user centered development using participatory design methods with my group at NASA Ames Research Center. We reduced our delivery cycle from six months to three-weeks, we increased product quality, decreased cost and increased the engagement and enjoyment of our own team and the customer. I'll present the key principles and lessons learned, the role of the customer, the users and our own team of developers and designers.

User Centered Agile Development at NASA - One Groups Path to Better Software

Balanced Team

User centered agile dev balanced team 2013

Jay Trimble

Agile by KD

Karl Dickman

Agile by KD

Karl Dickman

Journey of Migrating Millions of Queries on The Cloud

takezoe

SCQAA-SF Meeting on May 21 2014

Sujit Ghosh

Resume

Akansha Khanna

Scalability and performance for e commerce

Ecommerce Solution Provider SysIQ

Effective Scrum

Sándor Zolta Székely Sipos

Java certification, as an additional form of formal confirmation of knowledge and skills, represents a competitive advantage, both for individual and for the company. However, it can also represent a loss of valuable time and non-negligible financial cost. In this session we will try to provide answers to various questions about the certification, explain typical certification path, and explain the advantages and differences of certain certificates. We will try to help with instructions and tips about the easiest way to approach certification and how to take and renew certificates. Finally, we will present how to get to the highest levels of certification and explain what could be the benefit in your professional life, and whether the effort is worth it.

Java Certification by HUJAK - 2015-05-12 - at JavaCro'15 conference

HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association

Semelhante a What does it take to win the Kaggle/Yandex competition (20)

Emptying Your Cup an Agile Primer

Agile testing experiments

SenchaCon 2016: Creating a Flexible and Usable Industry Specific Solution - D...

Continuous Delivery without Significant Test Automation

My Experiments In Agile Testing in Yahoo.pptx

Play with Kaggle

Moving from fast to solr on atg

CommerceSearch: Moving from FAST to Solr on ATG

Agile Testing

20121213 qa introduction smileryang

User Centered Agile Development at NASA - One Groups Path to Better Software

User centered agile dev balanced team 2013

Agile by KD

Journey of Migrating Millions of Queries on The Cloud

SCQAA-SF Meeting on May 21 2014

Resume

Scalability and performance for e commerce

Effective Scrum

Java Certification by HUJAK - 2015-05-12 - at JavaCro'15 conference

Último

Tata AIG General Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Slack Application Development 101 Slides

praypatel2

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

What is a good lead in your organisation? Which leads are priority? What happens to leads? When sales and marketing give different answers to these questions, or perhaps aren't sure of the answers at all, frustrations build and opportunities are left on the table. Join us for an illuminating session with Cian McLoughlin, HubSpot Principal Customer Success Manager, as we look at that crucial piece of the customer journey in which leads are transferred from marketing to sales.

04-2024-HHUG-Sales-and-Marketing-Alignment.pptx

HampshireHUG

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Presentation on how to chat with PDF using ChatGPT code interpreter

naman860154

Choosing the right accounts payable services provider is a strategic decision that can significantly impact your business's financial performance and operational efficiency. By considering factors such as expertise, range of services, technology infrastructure, scalability, cost, and reputation, businesses can make informed decisions and select a provider that aligns with their unique needs and objectives. Partnering with the right provider can streamline accounts payable processes, drive cost savings, and position your business for long-term success. https://katprotech.com/accounts-payable-and-purchase-order-automation/

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

Katpro Technologies

Microsoft's Threat Matrix for Kubernetes helps organizations understand the attack surface a Kubernetes deployment introduces to their environments. This ensures that adequate detections and mitigations are in place. By covering over 40 different attacker techniques, defenders can learn about Kubernetes-specific mitigations and controls to deploy to their environments. In this session, we will explore the MS-TA9013 Host Path Mount technique, which is commonly used by attackers to perform privilege escalation in a Kubernetes cluster. Attendees will learn how attackers and defenders can: * Escape the container's host volume mount to gain persistence on an underlying node * Move laterally from the underlying node into the customer's cloud environment * Analyze Kubernetes audit logs to detect pods deployed with a hostPath mount * Deploy an admission controller that prevents new pods from using a hostPath mount

Breaking the Kubernetes Kill Chain: Host Path Mount

Puma Security, LLC

CNv6 Instructor Chapter 6 Quality of Service

giselly40

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Advantages of Hiring UIUX Design Service Providers for Your Business

Pixlogix Infotech

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

Enterprise Knowledge’s Urmi Majumder, Principal Data Architecture Consultant, and Fernando Aguilar Islas, Senior Data Science Consultant, presented "Driving Behavioral Change for Information Management through Data-Driven Green Strategy" on March 27, 2024 at Enterprise Data World (EDW) in Orlando, Florida. In this presentation, Urmi and Fernando discussed a case study describing how the information management division in a large supply chain organization drove user behavior change through awareness of the carbon footprint of their duplicated and near-duplicated content, identified via advanced data analytics. Check out their presentation to gain valuable perspectives on utilizing data-driven strategies to influence positive behavioral shifts and support sustainability initiatives within your organization. In this session, participants gained answers to the following questions: - What is a Green Information Management (IM) Strategy, and why should you have one? - How can Artificial Intelligence (AI) and Machine Learning (ML) support your Green IM Strategy through content deduplication? - How can an organization use insights into their data to influence employee behavior for IM? - How can you reap additional benefits from content reduction that go beyond Green IM?

Driving Behavioral Change for Information Management through Data-Driven Gree...

Enterprise Knowledge

What are drone anti-jamming systems? The drone anti-jamming systems and anti-spoof technology protect against interference, jamming, and spoofing of the UAVs. To protect their security, countries are beginning to research drone anti-jamming systems, also known as drone strike weapons. The anti-jam and anti-spoof technology protects against interference, jamming and spoofing. A drone strike weapon is a drone attack weapon that can attack and destroy enemy drones. So what is so unique about this amazing system?

What Are The Drone Anti-jamming Systems Technology?

Antenna Manufacturer Coco

What does it take to win the Kaggle/Yandex competition

1. WHAT DOES IT TAKE TO WIN THE KAGGLE/YANDEX COMPETITION Christophe Bourguignat Kenji Lefèvre-Hasegawa Paul Masurel @Dataiku Matthieu Scordia @Dataiku

2. OUTLINE OF THE TALK • Review of the Kaggle/Yandex Challenge • How we worked (team work & tools) • The winning model

3. GOAL Re-rank URLs returned by Yandex according to the personal preferences of the users url1 url3 url2 url2 GOAL url3 url1 url4 url4 ML CHALLENGE Predict user’s pertinence for urls and rerank result set accordingly The Kaggle/Yandex challenge

4. GIVEN • 30 days logs test: 3 days, train: 27 days • Users historic queries, clicks & dwell-times Q Q Q Q • Test session prior activity queries, clicks & dwell-times Test session : SIZE • 15Gb size The Kaggle/Yandex challenge Q Q T ?

5. QUALITY METRIC • One query test / user on the last 3 days • NDCG metric penalize error of pertinence on top ranked urls • No A/B test url1 url2 OK BAD url4 url3 Kaggle The Kaggle/Yandex challenge Prediction Another ranking

6. TEAM DATAIKU SCIENCE STUDIO / KAGGLE • • • • Christophe Bourguignat Engineer, Data enthusiastic Kenji Lefèvre-Hasegawa Ph.D. math, new to ML Paul Masurel Software Engineer @dataiku Matthieu Scordia Data Scientist @dataiku First meeting : October16th 2013 How we worked (Team work & tools)

7. WE’VE USED • • • • • Related papers (mainly Microsoft’s) 12 core, 64 Gb Python scikit-learn Dataiku Science Studio Java Ranklib How we worked (Team work & tools)

8. DATAIKU SCIENCE STUDIO Features & labels Features Labels Split train & validation Original train LEARNING Team members work independantly FEATURES CONSTRUCTION Team members work independantly DATA DRIVEN COMPUTATION How we worked (Team work & tools)

9. HOW MUCH WORK ? • 960+ emails • 360+ features • 50+ ideas grid tuned (300+ models fitted) • Server heavily loaded the last 3 weeks • 56 kaggle submissions • 196 teams, 264 players, 3570 submissions How we worked (Team work & tools) 2014-01-01 1st Future top 2 & 3 enter race 1 week 3rd 1 week 1st 5th Top 10 Top 25 1/2 month 1 week

10. PROBLEM ANALYSIS Query Result Set • Rank • URL Snippet Quality • URL is skipped, clicked or missed CLICK Reading URL • URL & Domain pertinence with dwell-time The winning model

11. FEATURES Features : • Rank • User habits, query specificity (entropy, frequency,…) • Snippet pertinence • Missed, Skipped, Clicked • URL & Domain Pertinence Declinaison of & Clicked • Probability, Stimuli freq., Mean Reciprocal Rank (MRR) • For each user : historic & previous activity in test session & aggregate • For all user • Declined for all queries & same query The winning model

12. MODELS • Random Forest (predict proba) + maximize E(NDCG) Kaggle/Yandex Top 1 then 3rd • Lambda MART (Gradient Boosting Tree optimized for NDCG) WINS ! The winning model

13. QUESTIONS ? Thank you !

What does it take to win the Kaggle/Yandex competition

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a What does it take to win the Kaggle/Yandex competition

Semelhante a What does it take to win the Kaggle/Yandex competition (20)

Último

Último (20)

What does it take to win the Kaggle/Yandex competition