Outline:
a. MediaPlayer Subsystem
b. Related Files
c. MediaPlayer Frame of Playing Flow
-StageFright and AwesomePlayer Relatin
-AwesomePlayer Frame and Playing Flow
d. Simple Playing Implement
2. Confidential
Outline
MediaPlayer Subsystem
Related Files
MediaPlayer Frame of Playing Flow
─ StageFright and AwesomePlayer Relatin
─ AwesomePlayer Frame and Playing Flow
Simple Playing Implement
8. Confidential
MediaPlayer Frame of Playing Flow
Ref: http://blog.csdn.net/tx3344/article/details/8084912
SurfaceFlinger
AudioFlinger
AwesomePlayer Frame and Playing Flow
Ref: http://blog.csdn.net/tx3344/article/details/8109185
9. Confidential
Simple Playing Implement
//For example:
//Simple Playing Implement:
MediaPlayer mp = new MediaPlayer();
mp.setDataSource(PATH_TO_FILE);
mp.prepare();
mp.start();
Ref: http://developer.android.com/reference/android/media/MediaPlayer.html
MediaPlayer State Diagram
10. Confidential
Simple Playing Implement
//For example:
//Simple Playing Implement:
MediaPlayer mp = new MediaPlayer();
mp.setDataSource(PATH_TO_FILE);
mp.prepareAsync();
mp.start();
11. Confidential
New MediaPlayer
Applications
Applications Framework
• Initiation and Setup
Media Player
Application
android_media_MediaPlayer
MediaPlayer.java
android_media_MediaPlayer_native_init()
android_media_MediaPlayer_native_setup()
Static{
System.loadLibrary("media_jni");
native_init();
}
static void android_media_MediaPlayer_native_init (JNIEnv *env){
clazz = env->FindClass("android/media/MediaPlayer");
:
fields.post_event = env->GetStaticMethodID(clazz, "postEventFromNative",
"(Ljava/lang/Object;IIILjava/lang/Object;)V");
:
fields.context = env->GetFieldID(clazz, "mNativeContext", "I");
:
}
static void android_media_MediaPlayer_native_setup (JNIEnv *env, jobject thiz, jobject weak_this)
{
:
sp<MediaPlayer> mp = new MediaPlayer();
// create new listener and give it to MediaPlayer
sp<JNIMediaPlayerListener> listener = new JNIMediaPlayerListener(env, thiz, weak_this);
mp->setListener(listener);
// Stow our new C++ MediaPlayer in an opaque field in the Java object.
setMediaPlayer(env, thiz, mp);
}
static sp<MediaPlayer> setMediaPlayer (JNIEnv* env, jobject thiz, …)
{
Mutex::Autolock l(sLock);
:
env->SetIntField(thiz, fields.context, (int)player.get());
:
}
12. Confidential
Simple Playing Implement
//For example:
//Simple Playing Implement:
MediaPlayer mp = new MediaPlayer();
mp.setDataSource(PATH_TO_FILE);
mp.prepareAsync();
mp.start();
14. Confidential
Media Set Data Source Flow (setDataSource)
Get Player Type
Create Player
Instance
Player Set Data
Source
Using file type to get player type
get real player instance
• Load software and hardware codec
• Register default sniffers
• Create AwesomeEvents of onVideoEvent, onStreamDone, onBufferingUpdate,
onVideoLagUpdate, onThumbnailEvent, onCheckAudioStatus and
onAudioTearDownEvent.
• setNotifyCallback
Using real player to set data source
• new FileSource(fd, …) insert to MediaExtractor::Create()
─ Sniff DataSource and get mine type
─ Get extractor with mime type
• Separate video (sp<MediaSource> mVideoTrack) and
audio (sp<MediaSource> mVideoSource) track
15. Confidential
Media Set Data Source Flow (setDataSource)
Using file type to get player type
get real player instance
• Load software and hardware codec
• Register default sniffers
• Create AwesomeEvents of onVideoEvent, onStreamDone, onBufferingUpdate,
onVideoLagUpdate, onThumbnailEvent, onCheckAudioStatus and
onAudioTearDownEvent.
• setNotifyCallback
Using real player to set data source
• new FileSource(fd, …) insert to MediaExtractor::Create()
─ Sniff DataSource and get mine type
─ Get extractor with mime type
• Separate video (sp<MediaSource> mVideoTrack) and
audio (sp<MediaSource> mVideoSource) track
Get Player Type
Create Player
Instance
Player Set Data
Source
16. Confidential
Libraries
Media Set Data Source Flow (setDataSource)
Using file type to get player type
MediaPlayerFactory.cpp
-factory: IFactory*
-sFactoryMap
NuPlayerFactory
+createPlayer(): return new NuPlayerDriver
SonivoxPlayerFactory
+createPlayer(): return new MidiFile
TestPlayerFactory
+createPlayer(): return new TestPlayerStub
StagefrightPlayerFactory
+createPlayer(): return new StagefrightPlayer
(libmedia.so)
MediaPlayer
player->setDataSource
MediaPlayerFactory
IMediaPlayer
Media Player
Application
android_media_MediaPlayerMediaPlayer.java
Applications
Applications Framework
MediaPlayerService::
Client
a. playerType:getPlayerType()
MediaPlayerFactory:
StagefrightPlayerFactory
createPlayer()
return new StagefrightPlayer();
setDataSource()
android_media_MediaPlayer_setDataSourceFD()
setDataSource()
b. sp<MediaPlayerBase>:
createPlayer(playerType)
a. getPlayerType:
1. sFactoryMap add all player factory while MediaPlayerService
started.
2. scoreFactory function using compare file
to get corresponding player type:
a. NuPlayerFactory
if(!strncasecmp("http://", url, 7)
|| !strncasecmp("https://", url, 8)
|| !strncasecmp("file://", url, 7))
b. SonivoxPlayerFactory
if (!strncasecmp(url + start, FILE_EXTS[i], len))
static const char* const FILE_EXTS[] = { ".mid",
".midi",
".smf",
".xmf",
".mxmf",
".imy",
".rtttl",
".rtx",
".ota" };
c. StagefrightPlayerFactory
1. compare raw data header if is equal to 0x5367674f
2. Default Player Type
17. Confidential
Media Set Data Source Flow (setDataSource)
Get Player Type
Using file type to get player type
get real player instance
• Load software and hardware codec
• Register default sniffers
• Create AwesomeEvents of onVideoEvent, onStreamDone, onBufferingUpdate,
onVideoLagUpdate, onThumbnailEvent, onCheckAudioStatus and
onAudioTearDownEvent.
• setNotifyCallback
Using real player to set data source
• new FileSource(fd, …) insert to MediaExtractor::Create()
─ Sniff DataSource and get mine type
─ Get extractor with mime type
• Separate video (sp<MediaSource> mVideoTrack) and
audio (sp<MediaSource> mVideoSource) track
Get Player Type
Create Player
Instance
Player Set Data
Source
18. Confidential
Media Set Data Source Flow (setDataSource)
Media Player
Application
Applications
android_media_MediaPlayer
Applications Framework
MediaPlayer.java
Libraries
• Load software and hardware codec
StagefrightPlayerMediaPlayerFactory
new StagefrightPlayer()
connect()
OMXClient
OMX
OMXMaster
OMX()
mMaster(new OMXMaster)
Software CodecHardware Codec
SoftOMXPlugin
AwesomePlayer
new AwesomePlayer()
libstagefrighthw.so
20. Confidential
Media Set Data Source Flow (setDataSource)
Media Player
Application
Applications
android_media_MediaPlayer
Applications Framework
MediaPlayer.java
Libraries
StagefrightPlayerMediaPlayerFactory
new StagefrightPlayer() new AwesomePlayer()
AwesomePlayer
mVideoEvent
= new AwesomeEvent(this, &AwesomePlayer::onVideoEvent);
mStreamDoneEvent
= new AwesomeEvent(this, &AwesomePlayer::onStreamDone);
mBufferingEvent
= new AwesomeEvent(this, &AwesomePlayer::onBufferingUpdate);
mVideoLagEvent
= new AwesomeEvent(this, &AwesomePlayer::onVideoLagUpdate);
mThumbnailEvent
= new AwesomeEvent(this, &AwesomePlayer::onThumbnailEvent);
• Create AwesomeEvents of onVideoEvent …
Ref: http://blog.csdn.net/tx3344/article/details/8053106
21. Confidential
Applications
Applications Framework
Libraries
• set Notify Callback (before new MediaPlayer()) : AP -> HAL
APP : setOn##Listener(m##Listener);
MediaPlayer.java : setOn##Listener(listener){mOn##Listener=listener}
Media Player
Application
android_media_MediaPlayer_native_init(){
:
fields.post_event = env->GetStaticMethodID(clazz, "postEventFromNative",
"(Ljava/lang/Object;IIILjava/lang/Object;)V");
:
}
android_media_MediaPlayer
MediaPlayer.java
Static{
System.loadLibrary("media_jni");
native_init();
}
Media Set Data Source Flow (setDataSource)
26. Confidential
Media Set Data Source Flow (setDataSource)
Using file type to get player type
get real player instance
• Load software and hardware codec
• Register default sniffers
• Create AwesomeEvents of onVideoEvent, onStreamDone, onBufferingUpdate,
onVideoLagUpdate, onThumbnailEvent, onCheckAudioStatus and
onAudioTearDownEvent.
• setNotifyCallback
Using real player to set data source
• new FileSource(fd, …) insert to MediaExtractor::Create()
─ Sniff DataSource and get mine type
─ Get extractor with mime type
• Separate video (sp<MediaSource> mVideoTrack) and
audio (sp<MediaSource> mVideoSource) track
Get Player TypeGet Player Type
Create Player
Instance
Player Set Data
Source
27. Confidential
Media Set Data Source Flow (setDataSource)
AwesomePlayer.cpp
setDataSource_l(dataSource)
MediaExtractor.cpp
Create(dataSource)
if (!strcasecmp(mime, MEDIA_MIMETYPE_CONTAINER_MPEG4)
|| !strcasecmp(mime, "audio/mp4")) {
ret = new MPEG4Extractor(source);
:
:
return ret;
DataSource.cpp
sniff(mimeType,…)
setDataSource_l(extractor)
extractor
Using Real Player Set Data Source
• MediaExtractor::Create(const sp<DataSource> &source,...)
─ Sniff DataSource and get mine type
Read at filesource
Compabile ?
Write mine type
─ Get extractor with mime type
28. Confidential
Media Set Data Source Flow (setDataSource)
─ Sniff DataSource and get mine type
Read at filesource
Compabile ?
Write mine type
─ Get extractor with mime type
Compabile ?
Read at filesource: Ex: Golden Flower mp4
Write mine type
29. Confidential
Media Set Data Source Flow (setDataSource)
Using Real Player Set Data Source
• Separate video and audio track
Video track
Audio track
30. Confidential
Simple Playing Implement
//For example:
//Simple Playing Implement:
MediaPlayer mp = new MediaPlayer();
mp.setDataSource(PATH_TO_FILE);
mp.prepareAsync();
mp.start();
37. Confidential
Video Audio Synchronization
Playing point
Audio Samples
Video Samples
Start Point
The Time in AudioPlayer Real Playing
The Time in Audi Real Playing out
Playback Time Line:
Playing point
latenessUs (VideoPlayer Lateness Time)