open source virtual singer(or 'v'ocaloid)

1. Virtual singer 徴音梅林 & LINNE
CC-BY-NC

2. Slide author
●
張正一 (Chou Shouichi)/ MGdesigner
●
Paul Liu and me organize 徴音梅林開発委員會
● Wikimedia.tw: member of board of directors
(and direct tech development )
● A programmer
● A musician (Jazz ukulele, DTM)
● Shoichi.chou@gmail.com

3. Everyone knows her

4. Powered by
Yamaha
Vocaloid2 engine

5. So
Why
a FOSS 'v'ocaloid?

6. If you buy an instrument
You can
play any song,
Do anything.

7. play

8. By teeth

9. Break !

10. burn

11. In any Vocaloid product EULA
You didn't get whole rights
●
no “anti-society” （反社会） works
(so,What works are “anti-society”?)
● Trademarks protection (images, keywords)
(ex: 'Vocaloid' ,' 初音ミク ',' 初音ミク ''s image)

14. No using Miku images=not popular
musicians are
controlled
No freedom

15. Be ruled
● Using a Gibson guitar,you are its master.
● Using Vocaloid products, You are their slave.

16. INDIE DIE

17. UTAU ？

18. A free vocaloid-like
●
DIY a 『 vocaloid 』
● Programs: editor(frontend)+resampler+wavtool
● data: vocal DB - oto.ini + wav samples
● Vocal DB is an open spec ,many people DIY

19. vocaloid programs working flow
1.Editor: compose the melody(many notes)
2.Resampler: modulate a sample to Specified
pitch,or other parameters (velocity...).
3.Wavtool: combine these modulated wavs
Finally,we get a song vocal wav file,and mix into a
song

20. but
● Charge no fee,not freedom
● Default resampler work badly
● DB bad international support (S-JIS)
● Oto.ini no implementing ini comments “;“
● UTAU always auto sort oto.ini (hard collaboration)
● Hard UI control
● Not open source
● Its development is very private

21. And you know ...
Yamaha owns many super
patents

22. A nice “free vocaloid“
Is
Very difficult

23. During 2011-2012
One day, Paul Liu talked to me
● New Algorithm, 'World' better than Vocaloid2
●
Author: 日本山梨大學 Doctor 森勢将雅
● Patent free
● EFB-GW(Synthesizer) for UTAU
● Open source(old version GPL,newer is BSD)
● https://github.com/mmorise/World
●
During Dec,Dr. 森勢 'll do another great upgrade

24. How good is World algorithm?
● very awesome 'autotune'
(original official test is a realtime Karaoke autotune for 音痴 s. )
● Modulate a sample to any pitch without distortion
(Keep F0 well)
(Vocaloid2 can't ,so Miku need 3 different range versions of each sample)
● Very fast ,no need to pre-preapre frequency tables
(Just do it real time)
● If X86, Even works good on older machines(maybe
on ARM)

25. Ok Let's do it!

26. Finally
we made her...

27. Listen...
Hear “MAMA” cover

28. 徵音梅林
「徵音」 :Ancient Chinese,Japanese
Pentatonic scale note. (Do Re Mi Sol La)
宮（きゅう）、商（しょう）、 角（かく）、 徴（
Also means We 'recruit' a voice actor(and
also a Jazz singer) from Internet
「梅林」： Merlin(super wizard)
林： Linux

29. http://projectmeilin.github.io/

30. Project Meilin Features
● CC-BY
● Utau compatible
● Professional recording(in studio)
● Src:24bits 48000hz wavs
●
VCV 連続音 、 VC 単独音
(V - Vowel c - Consonant)
● Recorded: Japanese,Mandarin(Taiwan style)

31. How good? A test
● Commercial Miku VS. open content Meilin
● V2 Miku each sample recorded high,middle,low
versions
VS.
Meilin each sample just record 1
version.

32. Listen to the
comparing video…
( song: 歌い手様総合テスト , Start from 0:44)
Especially check super low pitch and super
high pitch if is distorted （失真） ?

33. fact
Miku DB:
● 1gb+
● Only Japanese
Meilin DB:
● 627mb
● Japanese+Mardarin
●
Mardarlin DB is 3 倍
of JP DB

34. thank to Dr. 森勢
Without his effort and kindness,
a good FOSS virtual singer is
imposible

35. 2 more
Special features

36. 1: 14 Special effects
Defined in oto.ini
● 3 breath : br1,br2,br3 ( ex:Miku only have these breath. )
● Spanish 'R' rolling: trill
● Cough: cough
● Cry,dry tears:drytears
● Blownose: blownose
● Sucking: suck
●
sigh( 嘆 ):sgn1,sgn2,sgn3,sgn4
● Whistle :whsl
● clean throat: clnt

37. 2: 日本方言 possible
●
EX: 円唇母音'う' in 関西弁 (video)
● in Mandarin ,there is the same 'u'
● Just borrow what we recorded.
● also can borrow other Mandarin samples for
synthesizing 方言 or some foreign languages.
(ex: 1 or 2 foreign lyrics in a Japanese song)

38. 'v'ocaloid also can do speech
synthesis
Better than traditional speech synthesis
● Accent(= pitch,velocity,rhythm,speed) controllable
● Could do many emotion(melody lines) : cry,angry...
●
TTS,story telling,emotional ' うかがか' possible
● Some tests which I have done by Miku: 1,2,3,4 based on my scale
algorithm. 'Auto render' possible,but….
● If use Vocaloid to do this,you need to beg YAMAHA for opening API. But
our software stack are open source. She could do more than singing.

39. How made?

40. Recorded in a pro studio

41. Thanks to sponsor 阿怪 (Aguai),my master
(A famous POP song producer in TW.)

42. About the vocal
●
Her name is 羅竺 (Lo Chu).
● We choose her voice from 20 girls from on internet.
● She is a singer in a JAZZ / anime cover song band.
● Also vocal acting trained.
● Japanese accent not bad.
●
Japanese friend ATsushi 發音指導

43. But very hard work
Japanese recording need 3~4 hours.
But
Intact Madarin(possibility on math ,then minus
repeated samples by Phonology)
Madarin recording needs days.

44. The final day

46. LINNE platform
● We defined the FOSS 'v'ocaloid stack
● Of course opensource
● Compatible with Utau DB (but UTF-8)
● resampler+wavtool+editor(interface)+DB
-making tools
● May include 'hardware'

47. Hardware Ex: Doll robot

48. Our Oto.ini DB spec
● You can use ';' for comments
● Editors programs shouldn't resort the file
● UTF-8
● IPA based (International Phonetic Alphabet)
● By IPA,Different languages could use common
pronunciation samples
(no more re-recording again, keep the DB size smaller, more storage efficiency )

49. Engine (now is xvsqExec ,may need to
be changed)
Jcadencii
Linne-editor (in dev)
(song editor,front end)
Wavtool-pl
(GPL wavtool)
tn_fnds_yc (gpl)
(resampler,EFB-GW variant )
World lib
Other programs in the future
ex: linne-TTS

50. The chart may need evolution.

51. Problem now: the editor(frontend)
● Cadencii is written by .net with binding too
many Windows native calls
● Jcadencii is very slow (Cadencii java port)
● Upstream dev stopped. We also give it up.
● Another open Utau frontend:
http://fluidvocalsynth.weebly.com/ (also .Net)

52. ●linne-editor(frontend)
● https://github.com/marty1885/linne-editor
● In very earily development

53. fact
We don't have enough manpower
about interface coding…
● When normal users edit, still need wine+Utau
● Similar to early Linux dev in Minix >_<

54. Wish more people join
Make LINNE
More complete!

55. 徴音梅林開発委員会
● Open source community
● OSS programmers,musicians,a
physicist,Phonologists,artists...
● Members are international(TW,JP)

56. welcome
● Official Site
● Github: https://github.com/ProjectMeilin/
● Slack (tech talk): https://meilin.slack.com/
(email me for invitation : shoichi.chou@gmail.com )
● FB fan page
● FB group (more about DB making and musician)
● Youtube channel

57. Q&A
Listen another songs? :P
Hackasong.JP
Umbrella カバー
END