Implementation challenges in Big Data - Dr. Nilesh Karnik

•

0 gostou•1,236 visualizações

In todays competitive environment companies are faced with different types of challenges. Implementation of Big Data is one of them. Dr. Nilesh Karnik takes us through some of them.

Tecnologia Negócios

Aureus Claims Solution

Implementation
Challenges in Big Data
Footer Option 2
Analytics
• Dr. Nilesh N. Karnik

Copyright 2013 RESTRICTED CIRCULATION

What we will discuss

The Challenge of BIG Data

ADVANCED Analytics
SOLUTIONS in the Pipeline

Copyright 2013 RESTRICTED CIRCULATION

2

Big Data : Distributed Processing

Aureus Claims Solution

!
Footer Option 2

OLD IDEA
Copyright 2013 RESTRICTED CIRCULATION

NEW IDEA
3

EXAMPLE 1: Task of storing books on a shelf

Aureus Claims Solution

Footer Option 2
Image source Flickr. Image copyright belongs with original artist.

Simple, right?
Copyright 2013 RESTRICTED CIRCULATION

5

EXAMPLE 1: Task of storing books on a shelf

Aureus Claims Solution

And now?

Footer Option 2

Image source Flickr. Image copyright belongs with original artist.
Copyright 2013 RESTRICTED CIRCULATION

6

Aureus Claims Solution

Footer Option 2

Image source Flickr. Image copyright belongs with original artist.
Copyright 2013 RESTRICTED CIRCULATION

7

EXAMPLE 2 : Summarizing a Report

Aureus Claims Solution

SUMMER PROJECT
REPORT

Footer Option 2

Simple, right?
Copyright 2013 RESTRICTED CIRCULATION

8

EXAMPLE 2 : Summarizing a Report

Aureus Claims Solution

Footer Option 2

And now?
Copyright 2013 RESTRICTED CIRCULATION

9

EXAMPLE 3 : Baking a Cake

Simple, right?

Aureus Claims Solution

And now?

Footer Option 2

Image source PINTEREST. Image copyright belongs with original artist.
Copyright 2013 RESTRICTED CIRCULATION

10

Advanced Analytics
•

Well developed tool set for “small data” environment

•

Aureus Claims Solution

Challenges in Big Data environment

Footer Option 2

Copyright 2013 RESTRICTED CIRCULATION

11

Advanced Analytics: MapReduce Difficulties

Aureus Claims Solution

Footer Option 2
ITERATIVE
Image source Flickr. Image copyright belongs with original artist.
Copyright 2013 RESTRICTED CIRCULATION

12

Advanced Analytics: MapReduce Difficulties

Aureus Claims Solution

Footer Option 2
INCREMENTAL PROCESSING REQUIRES RESTART
Image source Flickr. Image copyright belongs with original artist.
Copyright 2013 RESTRICTED CIRCULATION

13

Advanced Analytics: MapReduce Difficulties

Aureus Claims Solution

Footer Option 2

BATCH LEARNING SCANS ALL DATA IN ONE GO
Copyright 2013 RESTRICTED CIRCULATION

14

Some Solutions Data Scientists are working on

Aureus Claims Solution

New frameworks
• E.g., HaLoop*, PrIter# (Extensions of Hadoop)
• Percolator$ (Proprietary Google framework)

Footer Option 2

* Y. Bu, B. Howe, M. Balazinska, and M. Ernst, “HaLoop: Efficient iterative data processing on large clusters”, VLDB, 2010.
# Y. Zhang, Q. Gao, L. Gao and C. Wang, “PrIter: A distributed framework for prioritized iterative computations”, SoCC, 2011.
$ D. Peng and F. Dabek, “Large-scale incremental processing using distributed transactions and notifications”, OSDI, 2010

Copyright 2013 RESTRICTED CIRCULATION

15

Some Solutions Data Scientists are working on

Aureus Claims Solution

Smarter algorithms / Different
implementations
• Random forest
• Parallelized Stochastic Gradient
Descent

Footer Option 2

Copyright 2013 RESTRICTED CIRCULATION

16

@nilesh_karnik
Nilesh@aureusanalytics.com
Copyright 2013 RESTRICTED CIRCULATION

17

Aureus Claims Solution

ThankOption 2
Footer You!
SINGAPORE

INDIA

Aureus Analytics Pte. Ltd.
17, Phillip Street,
#05-01, Grand Building
Singapore (048695)

Aureus Analytics Pvt. Ltd.
706, Powai Plaza
Hiranandani Gardens, Powai
Mumbai – 400076

info@aureusanalytics.com
Copyright 2013 RESTRICTED CIRCULATION

www.aureusanalytics.com

Mais conteúdo relacionado

Destaque

Point of Decision Analytics in Insurance

Aureus Analytics

Work Samples - ASB Digital Savvy Workforce (2013)

Arturo Pelayo

Building Healthier Communities: TEDMED 2016

Luminary Labs

Life@SlideShare

Amit Ranjan

TIME: 2015 Person of the Year

guimera

Net Promoter Score Pitfalls to Avoid

Aureus Analytics

Digital Portfolios

Silvia Rosenthal Tolisano

Culture is something we take pride in at LinkedIn. As the collective personality of our organization, it sets us apart, defines who we are and shapes what we aspire to be. Hundreds of companies have defined their unique cultures on SlideShare as part of the Culture Code campaign. We thought it was important for LinkedIn to join in this effort; we want everyone, including our current and our future employees, to know exactly what it’s like to work here.

LinkedIn’s Culture of Transformation

Pat Wadors

Destaque (8)

Point of Decision Analytics in Insurance

Work Samples - ASB Digital Savvy Workforce (2013)

Building Healthier Communities: TEDMED 2016

Life@SlideShare

TIME: 2015 Person of the Year

Net Promoter Score Pitfalls to Avoid

Digital Portfolios

LinkedIn’s Culture of Transformation

Mais de Aureus Analytics

Machine learning and artificial intelligence have great potential in helping insurance companies improve their internal processes such as underwriting, cross selling, fraud and claims prevention which directly have an impact on the consumer experience. In this presentation Dr. Karnik shares his experience in building models for prediction of early claims and potentially fraudulent claims. These are based on working with insurance carriers of all sizes.

Using Machine Learning to Find a needle in a haystack Aureus Analytics

Aureus Analytics

Big Data and Analytics Trends 2018

Aureus Analytics

Improving Marketing Campaigns

Aureus Analytics

Improvement in Persistency Score for Large Insurer

Aureus Analytics

The Art Of Net Promoter Score

Aureus Analytics

Big data-analytics-trends-2016

Aureus Analytics

Statistical model infographic.compressed (2)

Aureus Analytics

Infographic: Big Data and Analytics trends for 2015

Aureus Analytics

Infographic: Steps to Building a Statistical Model

Aureus Analytics

Ten Commandments of Statistical Modelling - Infographic

Aureus Analytics

67 years of Steering Innovation

Aureus Analytics

Demystifying A Data Scientist

Aureus Analytics

FIFA World Cup 2014 in Numbers

Aureus Analytics

Big Data and Analytics Trends in 2014

Aureus Analytics

Mais de Aureus Analytics (14)

Using Machine Learning to Find a needle in a haystack Aureus Analytics

Big Data and Analytics Trends 2018

Improving Marketing Campaigns

Improvement in Persistency Score for Large Insurer

The Art Of Net Promoter Score

Big data-analytics-trends-2016

Statistical model infographic.compressed (2)

Infographic: Big Data and Analytics trends for 2015

Infographic: Steps to Building a Statistical Model

Ten Commandments of Statistical Modelling - Infographic

67 years of Steering Innovation

Demystifying A Data Scientist

FIFA World Cup 2014 in Numbers

Big Data and Analytics Trends in 2014

Último

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

This presentation explores the impact of HTML injection attacks on web applications, detailing how attackers exploit vulnerabilities to inject malicious code into web pages. Learn about the potential consequences of such attacks and discover effective mitigation strategies to protect your web applications from HTML injection vulnerabilities. for more information visit https://bostoninstituteofanalytics.org/category/cyber-security-ethical-hacking/

HTML Injection Attacks: Impact and Mitigation Strategies

Boston Institute of Analytics

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar. In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

Partners Life - Insurer Innovation Award 2024

The Digital Insurer

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

Tata AIG General Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

💉💊+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHABI}}+971581248768 +971581248768 Mtp-Kit (500MG) Prices » Dubai [(+971581248768**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Maya Whatsapp +971581248768 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971581248768''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971581248768' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Cl

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@

A Principled Technologies deployment guide Conclusion Deploying VMware Cloud Foundation 5.1 on next gen Dell PowerEdge servers brings together critical virtualization capabilities and high-performing hardware infrastructure. Relying on our hands-on experience, this deployment guide offers a comprehensive roadmap that can guide your organization through the seamless integration of advanced VMware cloud solutions with the performance and reliability of Dell PowerEdge servers. In addition to the deployment efficiency, the Cloud Foundation 5.1 and PowerEdge solution delivered strong performance while running a MySQL database workload. By leveraging VMware Cloud Foundation 5.1 and PowerEdge servers, you could help your organization embrace cloud computing with confidence, potentially unlocking a new level of agility, scalability, and efficiency in your data center operations.

Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...

Principled Technologies

Manulife - Insurer Innovation Award 2024

The Digital Insurer

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

Implementation challenges in Big Data - Dr. Nilesh Karnik

4. EXAMPLE 1: Task of storing books on a shelf Aureus Claims Solution Footer Option 2 Image source Flickr. Image copyright belongs with original artist. Simple, right? Copyright 2013 RESTRICTED CIRCULATION 5

5. EXAMPLE 1: Task of storing books on a shelf Aureus Claims Solution And now? Footer Option 2 Image source Flickr. Image copyright belongs with original artist. Copyright 2013 RESTRICTED CIRCULATION 6

9. EXAMPLE 3 : Baking a Cake Simple, right? Aureus Claims Solution And now? Footer Option 2 Image source PINTEREST. Image copyright belongs with original artist. Copyright 2013 RESTRICTED CIRCULATION 10

11. Advanced Analytics: MapReduce Difficulties Aureus Claims Solution Footer Option 2 ITERATIVE Image source Flickr. Image copyright belongs with original artist. Copyright 2013 RESTRICTED CIRCULATION 12

12. Advanced Analytics: MapReduce Difficulties Aureus Claims Solution Footer Option 2 INCREMENTAL PROCESSING REQUIRES RESTART Image source Flickr. Image copyright belongs with original artist. Copyright 2013 RESTRICTED CIRCULATION 13

14. Some Solutions Data Scientists are working on Aureus Claims Solution New frameworks • E.g., HaLoop*, PrIter# (Extensions of Hadoop) • Percolator$ (Proprietary Google framework) Footer Option 2 * Y. Bu, B. Howe, M. Balazinska, and M. Ernst, “HaLoop: Efficient iterative data processing on large clusters”, VLDB, 2010. # Y. Zhang, Q. Gao, L. Gao and C. Wang, “PrIter: A distributed framework for prioritized iterative computations”, SoCC, 2011. $ D. Peng and F. Dabek, “Large-scale incremental processing using distributed transactions and notifications”, OSDI, 2010 Copyright 2013 RESTRICTED CIRCULATION 15

15. Some Solutions Data Scientists are working on Aureus Claims Solution Smarter algorithms / Different implementations • Random forest • Parallelized Stochastic Gradient Descent Footer Option 2 Copyright 2013 RESTRICTED CIRCULATION 16

17. Aureus Claims Solution ThankOption 2 Footer You! SINGAPORE INDIA Aureus Analytics Pte. Ltd. 17, Phillip Street, #05-01, Grand Building Singapore (048695) Aureus Analytics Pvt. Ltd. 706, Powai Plaza Hiranandani Gardens, Powai Mumbai – 400076 info@aureusanalytics.com Copyright 2013 RESTRICTED CIRCULATION www.aureusanalytics.com

Implementation challenges in Big Data - Dr. Nilesh Karnik

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (8)

Mais de Aureus Analytics

Mais de Aureus Analytics (14)

Último

Último (20)

Implementation challenges in Big Data - Dr. Nilesh Karnik