SlideShare uma empresa Scribd logo
1 de 64
Baixar para ler offline
1
ex.ipadic




            2
✓
✓
✓
-
-
-
✓“   ”
✓

         3
I like Masashi Sada.→I/like/Masashi/Sada/




               →
    →




                                            4
ipadic          NAIST-jdic Unidic



→   /       -               -
    /           -
        /           -           -       -
            /           -           -       -
    /           -           -
        /           -
        /

                                                    5
/



→       /
    /           -               -
        /           -
            /           -
                /           -
    /           -               -
        /           -




                                    6
→       /           -       -   -
            /           -
    /           -
    /           -
    /           -
    /           -
        /           -
        /           -
        /           -
        /


                                    7
(            )

→    /   -
    /
    l/
    ∋/
    →/   -
    →/   -
     /   -



                 8
(   )

→   /           -
    /       -
    /
    /       -
        /       -
        /       -
    /       -
            /       -



                                9
•
•

•




    10
http://chasen.org/~daiti-m/paper/jfssa2009segment.pdf



•

• n-gram                    /                       Pitman-Yor   /
    Gibbs                            /                  ...



                                                                     11
12
13
14
15
Web
• Web




•
    MeCab   TRIE



                   16
17
•



•   Web




•         (´ ω `)



                    18
✓
✓
✓
-
-
-
✓“   ”
✓

         19
20
21
22
23
24
25
26
27
28
29
30
✓
✓
✓
-
-
-
✓“   ”
✓

         31
Google IME    Web




“       ”   Twitter




                        32
2003:
http://2boy.org/~yuta/publications/nl158-slides.pdf

                                                      33
×
×




    34
35
36
37
•
•
•




    38
.....   .....   ......


                         39
.....   .....   ......


                         40
41
↓




    42
Hadoop
      n-gram
 NameNode×             DataNode×

100          (Latin)               40   50
25       (         )               5    10




                                             43
•




•   n-gram




             44
Lingua::LanguageGuesser


http://gensen.dl.itc.u-tokyo.ac.jp/LanguageGuesser/LanguageGuesser_demo_ja.html




“just do it.”

”‫“ﻋﻴﺪ ﺍﳊﺐ‬



                                                                                  45
Unicode




Twitter TimeZone
                   46
TF-IDF
TF-IDF




         47
REST API
                        Streaming API

          Streaming API(sample API
http://apiwiki.twitter.com/Streaming-API-Documentation
                 firehose API
1/25      1/30




                                                         48
Google
Java          API


http://code.google.com/p/google-api-translate-java/




                                                      49
✓
✓
✓
-
-
-
✓“   ”
✓

         50
51
http://chinese.dsturgeon.net/text.pl?node=1081&if=gb



                                                       52
......



※




             53
http://www.aozora.gr.jp/index_pages/person1095.html



                                                      54
......

12




              55
http://www.sainet.or.jp/~eshibuya/


                                     56
......




25




              57
http://www.nhk.or.jp/namara03-blog/



                                      58
.....




        59
•




•   NG




         60
✓
✓
✓
-
-
-
✓“   ”
✓

         61
✓

✓

✓




    62
✓

    ※
        URL


✓ Ameba       (´ ω `)




                        63
64

Mais conteúdo relacionado

Semelhante a おひろめ会〜教師なしワード抽出

DevLOVE ターミナル勉強会 zsh + screen
DevLOVE ターミナル勉強会 zsh + screenDevLOVE ターミナル勉強会 zsh + screen
DevLOVE ターミナル勉強会 zsh + screenYozo SATO
 
snapshot ではじめる ローカライズ検証
snapshot ではじめる ローカライズ検証snapshot ではじめる ローカライズ検証
snapshot ではじめる ローカライズ検証moko ist
 
🐲 Here be Stacktraces — Flink SQL for Non-Java Developers
🐲 Here be Stacktraces — Flink SQL for Non-Java Developers🐲 Here be Stacktraces — Flink SQL for Non-Java Developers
🐲 Here be Stacktraces — Flink SQL for Non-Java DevelopersHostedbyConfluent
 
Ruby ecosystem applied to agile project
Ruby ecosystem applied to agile projectRuby ecosystem applied to agile project
Ruby ecosystem applied to agile projectKyosuke MOROHASHI
 
לבלוג או לא לבלוג? הבלוג בשירות המתרגם והעורך
לבלוג או לא לבלוג? הבלוג בשירות המתרגם והעורךלבלוג או לא לבלוג? הבלוג בשירות המתרגם והעורך
לבלוג או לא לבלוג? הבלוג בשירות המתרגם והעורךYael Sela-Shapiro
 
PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...
PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...
PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...Mark Wong
 
Search and Replacement Techniques in Emacs: avy, swiper, multiple-cursor, ag,...
Search and Replacement Techniques in Emacs: avy, swiper, multiple-cursor, ag,...Search and Replacement Techniques in Emacs: avy, swiper, multiple-cursor, ag,...
Search and Replacement Techniques in Emacs: avy, swiper, multiple-cursor, ag,...Kazuki Yoshida
 

Semelhante a おひろめ会〜教師なしワード抽出 (9)

DevLOVE ターミナル勉強会 zsh + screen
DevLOVE ターミナル勉強会 zsh + screenDevLOVE ターミナル勉強会 zsh + screen
DevLOVE ターミナル勉強会 zsh + screen
 
snapshot ではじめる ローカライズ検証
snapshot ではじめる ローカライズ検証snapshot ではじめる ローカライズ検証
snapshot ではじめる ローカライズ検証
 
🐲 Here be Stacktraces — Flink SQL for Non-Java Developers
🐲 Here be Stacktraces — Flink SQL for Non-Java Developers🐲 Here be Stacktraces — Flink SQL for Non-Java Developers
🐲 Here be Stacktraces — Flink SQL for Non-Java Developers
 
Ruby ecosystem applied to agile project
Ruby ecosystem applied to agile projectRuby ecosystem applied to agile project
Ruby ecosystem applied to agile project
 
לבלוג או לא לבלוג? הבלוג בשירות המתרגם והעורך
לבלוג או לא לבלוג? הבלוג בשירות המתרגם והעורךלבלוג או לא לבלוג? הבלוג בשירות המתרגם והעורך
לבלוג או לא לבלוג? הבלוג בשירות המתרגם והעורך
 
MateriApps LIVE! の設定
MateriApps LIVE! の設定MateriApps LIVE! の設定
MateriApps LIVE! の設定
 
MateriApps LIVE! の設定
MateriApps LIVE! の設定MateriApps LIVE! の設定
MateriApps LIVE! の設定
 
PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...
PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...
PostgreSQL Portland Performance Practice Project - Database Test 2 Filesystem...
 
Search and Replacement Techniques in Emacs: avy, swiper, multiple-cursor, ag,...
Search and Replacement Techniques in Emacs: avy, swiper, multiple-cursor, ag,...Search and Replacement Techniques in Emacs: avy, swiper, multiple-cursor, ag,...
Search and Replacement Techniques in Emacs: avy, swiper, multiple-cursor, ag,...
 

Mais de moai kids

中国最新ニュースアプリ事情
中国最新ニュースアプリ事情中国最新ニュースアプリ事情
中国最新ニュースアプリ事情moai kids
 
FluentdとRedshiftの素敵な関係
FluentdとRedshiftの素敵な関係FluentdとRedshiftの素敵な関係
FluentdとRedshiftの素敵な関係moai kids
 
Twitterのsnowflakeについて
TwitterのsnowflakeについてTwitterのsnowflakeについて
Twitterのsnowflakeについてmoai kids
 
Programming Hive Reading #4
Programming Hive Reading #4Programming Hive Reading #4
Programming Hive Reading #4moai kids
 
Programming Hive Reading #3
Programming Hive Reading #3Programming Hive Reading #3
Programming Hive Reading #3moai kids
 
"Programming Hive" Reading #1
"Programming Hive" Reading #1"Programming Hive" Reading #1
"Programming Hive" Reading #1moai kids
 
Casual Compression on MongoDB
Casual Compression on MongoDBCasual Compression on MongoDB
Casual Compression on MongoDBmoai kids
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDBmoai kids
 
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきましたHadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきましたmoai kids
 
HBase本輪読会資料(11章)
HBase本輪読会資料(11章)HBase本輪読会資料(11章)
HBase本輪読会資料(11章)moai kids
 
snappyについて
snappyについてsnappyについて
snappyについてmoai kids
 
第四回月次セミナー(公開版)
第四回月次セミナー(公開版)第四回月次セミナー(公開版)
第四回月次セミナー(公開版)moai kids
 
第三回月次セミナー(公開版)
第三回月次セミナー(公開版)第三回月次セミナー(公開版)
第三回月次セミナー(公開版)moai kids
 
Pythonで自然言語処理
Pythonで自然言語処理Pythonで自然言語処理
Pythonで自然言語処理moai kids
 
HandlerSocket plugin Client for Javaとそれを用いたベンチマーク
HandlerSocket plugin Client for Javaとそれを用いたベンチマークHandlerSocket plugin Client for Javaとそれを用いたベンチマーク
HandlerSocket plugin Client for Javaとそれを用いたベンチマークmoai kids
 
Yammer試用レポート(公開版)
Yammer試用レポート(公開版)Yammer試用レポート(公開版)
Yammer試用レポート(公開版)moai kids
 
掲示板時間軸コーパスを用いたワードトレンド解析(公開版)
掲示板時間軸コーパスを用いたワードトレンド解析(公開版)掲示板時間軸コーパスを用いたワードトレンド解析(公開版)
掲示板時間軸コーパスを用いたワードトレンド解析(公開版)moai kids
 
中国と私(仮題)
中国と私(仮題)中国と私(仮題)
中国と私(仮題)moai kids
 
不自然言語処理コンテストLT資料
不自然言語処理コンテストLT資料不自然言語処理コンテストLT資料
不自然言語処理コンテストLT資料moai kids
 
n-gramコーパスを用いた類義語自動獲得手法について
n-gramコーパスを用いた類義語自動獲得手法についてn-gramコーパスを用いた類義語自動獲得手法について
n-gramコーパスを用いた類義語自動獲得手法についてmoai kids
 

Mais de moai kids (20)

中国最新ニュースアプリ事情
中国最新ニュースアプリ事情中国最新ニュースアプリ事情
中国最新ニュースアプリ事情
 
FluentdとRedshiftの素敵な関係
FluentdとRedshiftの素敵な関係FluentdとRedshiftの素敵な関係
FluentdとRedshiftの素敵な関係
 
Twitterのsnowflakeについて
TwitterのsnowflakeについてTwitterのsnowflakeについて
Twitterのsnowflakeについて
 
Programming Hive Reading #4
Programming Hive Reading #4Programming Hive Reading #4
Programming Hive Reading #4
 
Programming Hive Reading #3
Programming Hive Reading #3Programming Hive Reading #3
Programming Hive Reading #3
 
"Programming Hive" Reading #1
"Programming Hive" Reading #1"Programming Hive" Reading #1
"Programming Hive" Reading #1
 
Casual Compression on MongoDB
Casual Compression on MongoDBCasual Compression on MongoDB
Casual Compression on MongoDB
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
Hadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきましたHadoop Conference Japan 2011 Fallに行ってきました
Hadoop Conference Japan 2011 Fallに行ってきました
 
HBase本輪読会資料(11章)
HBase本輪読会資料(11章)HBase本輪読会資料(11章)
HBase本輪読会資料(11章)
 
snappyについて
snappyについてsnappyについて
snappyについて
 
第四回月次セミナー(公開版)
第四回月次セミナー(公開版)第四回月次セミナー(公開版)
第四回月次セミナー(公開版)
 
第三回月次セミナー(公開版)
第三回月次セミナー(公開版)第三回月次セミナー(公開版)
第三回月次セミナー(公開版)
 
Pythonで自然言語処理
Pythonで自然言語処理Pythonで自然言語処理
Pythonで自然言語処理
 
HandlerSocket plugin Client for Javaとそれを用いたベンチマーク
HandlerSocket plugin Client for Javaとそれを用いたベンチマークHandlerSocket plugin Client for Javaとそれを用いたベンチマーク
HandlerSocket plugin Client for Javaとそれを用いたベンチマーク
 
Yammer試用レポート(公開版)
Yammer試用レポート(公開版)Yammer試用レポート(公開版)
Yammer試用レポート(公開版)
 
掲示板時間軸コーパスを用いたワードトレンド解析(公開版)
掲示板時間軸コーパスを用いたワードトレンド解析(公開版)掲示板時間軸コーパスを用いたワードトレンド解析(公開版)
掲示板時間軸コーパスを用いたワードトレンド解析(公開版)
 
中国と私(仮題)
中国と私(仮題)中国と私(仮題)
中国と私(仮題)
 
不自然言語処理コンテストLT資料
不自然言語処理コンテストLT資料不自然言語処理コンテストLT資料
不自然言語処理コンテストLT資料
 
n-gramコーパスを用いた類義語自動獲得手法について
n-gramコーパスを用いた類義語自動獲得手法についてn-gramコーパスを用いた類義語自動獲得手法について
n-gramコーパスを用いた類義語自動獲得手法について
 

Último

Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhisoniya singh
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?XfilesPro
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAndikSusilo4
 

Último (20)

Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & Application
 

おひろめ会〜教師なしワード抽出