技术背景
大牛直播SDK自2015年发布RTSP、RTMP直播播放模块,迭代从未停止,SmartPlayer功能强大、性能强劲、高稳定、超低延迟、超低资源占用。无需赘述,全自研内核,行业内一致认可的跨平台RTSP、RTMP直播播放器。本文以Android平台为例,介绍下如何集成RTSP、RTMP播放模块。
技术对接
系统要求
- SDK支持Android5.1及以上版本;
- 支持的CPU架构:armv7, arm64, x86, x86_64。
准备工作
- 确保SmartPlayerJniV2.java放到com.daniulive.smartplayer包名下(可在其他包名下调用);
- Smartavengine.jar加入到工程;
- 拷贝SmartPlayerV2\app\src\main\jniLibs\armeabi-v7a、 SmartPlayerV2\app\src\main\jniLibs\arm64-v8a、SmartPlayerV2\app\src\main\jniLibs\x86和SmartPlayerV2\app\src\main\jniLibs\x86_64 下 libSmartPlayer.so到工程;
- AndroidManifast.xml添加相关权限:
- 如需集成到自己系统测试,请用大牛直播SDK的app name,授权版按照授权app name正常使用即可;
- 如何改app-name,strings.xml做以下修改:
接口设计
Android RTSP|RTMP播放端SDK接口详解 |
调用描述 |
接口 |
接口描述 |
最先调用,如成功返回播放实例 |
SmartPlayerOpen |
player初始化,设置上下文信息,返回player句柄 |
Event回调 |
SetSmartPlayerEventCallbackV2 |
设置event callback |
硬解码设置(H.264) |
SetSmartPlayerVideoHWDecoder |
设置是否用H.264硬解码播放,如硬解码不支持,自动适配到软解码 |
硬解码设置(H.265) |
SetSmartPlayerVideoHevcHWDecoder |
设置是否用H.265硬解码播放,如硬解码不支持,自动适配到软解码 |
视频画面
填充模式 |
SmartPlayerSetRenderScaleMode |
设置视频画面的填充模式,如填充整个view、等比例填充view,如不设置,默认填充整个view |
设置SurfaceView模式下render类型 |
SmartPlayerSetSurfaceRenderFormat |
设置SurfaceView模式下(NTRenderer.CreateRenderer第二个参数传false的情况),render类型
0: RGB565格式,如不设置,默认此模式; 1: ARGB8888格式 |
设置SurfaceView模式下抗锯齿效果 |
SmartPlayerSetSurfaceAntiAlias |
设置SurfaceView模式下(NTRenderer.CreateRenderer第二个参数传false的情况),抗锯齿效果,注意:抗锯齿模式开启后,可能会影像性能,请慎用 |
设置播放的surface |
SmartPlayerSetSurface |
设置播放的surface,如果为null,则播放纯音频 |
设置视频硬解码下Mediacodec自行绘制模式 |
SmartPlayerSetHWRenderMode |
此种模式下,硬解码兼容性和效率更好,回调YUV/RGB、快照和图像等比例缩放功能将不可用 |
更新硬解码surface |
SmartPlayerUpdateHWRenderSurface |
设置更新硬解码surface |
音频回调 |
YUV/RGB |
SmartPlayerSetExternalRender |
提供解码后YUV/RGB数据接口,供用户自己render或进一步处理(如视频分析) |
Audio |
SmartPlayerSetExternalAudioOutput |
回调audio数据到上层(供二次处理之用) |
audio输出类型 |
SmartPlayerSetAudioOutputType |
如果use_audiotrack设置为0,将会自动选择输出设备,如果设置为1,使用audiotrack模式,一对一回音消除模式下,请选用audiotrack模式 |
Video输出类型 |
NTRenderer.CreateRenderer(上层demo内) |
第二个参数,如果是true,用openGLES绘制,false则用默认surfaceView |
播放模式 |
缓冲时间设置 |
SmartPlayerSetBuffer |
设置播放端缓存数据buffer,单位:毫秒,如不需buffer,设置为0 |
首屏秒开 |
SmartPlayerSetFastStartup |
设置快速启动后,如果CDN缓存GOP,实现首屏秒开 |
低延迟模式 |
SmartPlayerSetLowLatencyMode |
针对类似于直播娃娃机等期待超低延迟的使用场景,超低延迟播放模式下,延迟可达到200~400ms |
快速切换URL |
SmartPlayerSwitchPlaybackUrl |
快速切换播放url,快速切换时,只换播放source部分,适用于不同数据流之间,快速切换(如娃娃机双摄像头切换或高低分辨率流切换) |
RTSP TCP/UDP模式设置 |
SmartPlayerSetRTSPTcpMode |
设置RTSP TCP/UDP模式,如不设置,默认UDP模式 |
RTSP超时时间设置 |
SmartPlayerSetRTSPTimeout |
设置RTSP超时时间,timeout单位为秒,必须大于0 |
设置RTSP TCP/UDP自动切换 |
SmartPlayerSetRTSPAutoSwitchTcpUdp |
对于RTSP来说,有些可能支持rtp over udp方式,有些可能支持使用rtp over tcp方式
为了方便使用,有些场景下可以开启自动尝试切换开关, 打开后如果udp无法播放,sdk会自动尝试tcp, 如果tcp方式播放不了,sdk会自动尝试udp. |
设置RTSP用户名和密码 |
SetRTSPAuthenticationInfo |
如果RTSP URL已包含用户名和密码, 此接口设置的用户名和密码将无效. 就是说要用这个接口设置的用户名和密码去做认证, RTSP URL不能包含用户名和密码. |
实时静音 |
SmartPlayerSetMute |
实时静音 |
设置播放音量 |
SmartPlayerSetAudioVolume |
播放端音量实时调节,范围[0,100],0时为静音,100为原始流数据最大音量 |
设置是否禁用 Enhanced
RTMP |
DisableEnhancedRTMP |
disable enhanced RTMP, SDK默认是开启enhanced RTMP的 |
实时截图 |
CaptureImage |
支持JPEG和PNG两种格式 |
视频镜像旋转 |
旋转 |
SmartPlayerSetRotation |
设置顺时针旋转, 注意除了0度之外, 其他角度都会额外消耗性能,当前支持 0度,90度, 180度, 270度 旋转 |
水平反转 |
SmartPlayerSetFlipHorizontal |
设置视频水平反转 |
垂直反转 |
SmartPlayerSetFlipVertical |
设置视频垂直反转 |
设置URL |
SmartPlayerSetUrl |
设置需要播放或录像的RTMP/RTSP url |
开始播放 |
SmartPlayerStartPlay |
开始播放RTSP/RTMP流 |
停止播放 |
SmartPlayerStopPlay |
停止播放RTSP/RTMP流 |
关闭播放实例 |
SmartPlayerClose |
结束时必须调用close接口释放资源 |
功能支持
- 音频:AAC/Speex(RTMP)/PCMA/PCMU;
- 视频:H.264、H.265;
- 播放协议:RTSP|RTMP;
- 支持纯音频、纯视频、音视频播放;
- 支持多实例播放;
- 支持软解码,特定机型硬解码;
- 支持RTSP TCP、UDP模式设置;
- 支持RTSP TCP、UDP模式自动切换;
- 支持RTSP超时时间设置,单位:秒;
- 支持buffer时间设置,单位:毫秒;
- 支持超低延迟模式;
- 支持断网自动重连、视频追赶,支持buffer状态等回调;
- 支持视频view实时旋转(0° 90° 180° 270°);
- 支持视频view水平反转、垂直反转;
- 支持Surfaceview/OpenGL ES/TextureView绘制;
- 支持视频画面填充模式设置;
- 音频支持AudioTrack、OpenSL ES模式;
- 支持jpeg、png实时截图;
- 支持实时音量调节;
- 支持解码前音视频数据回调;
- 支持解码后YUV/RGB数据回调;
- 支持Enhanced RTMP;
- 支持扩展录像功能;
- 支持Android 5.1及以上版本。
接口调用详解
本文以大牛直播SDK Android平台SmartPlayerV2为例,播放之前,设置初始化参数配置(软解还是硬解、buffer time等)和需要播放的RTSP或RTMP URL,点开始播放即可。
onCreate()时,先new SmartPlayerJniV2():
开始播放、停止播放实现,开始播放的时候,调用InitAndSetConfig(),完成常规参数初始化,然后调用仅播放相关的其他接口。
由于RTSP、RTMP播放模块,除了常规的直播播放外,也可能录像、或者实时拉流转发到RTMP服务器或轻量级RTSP服务,所以,和录像、转发相关的播放端基础参数配置,放到InitAndSetConfig()实现:
EventHandle播放端事件回调处理,是底层状态反馈非常重要的媒介,除了网络状态、buffering状态回调外、还有录像状态、快照状态等回调:
如果RTSP、RTMP流需要录像:
其中,录像参数配置选项设置如下,除了下面演示接口外,还可以设置仅录视频或音频:
如需播放过程中实时截图:
如需对视频view做水平、垂直翻转或旋转:
onDestroy() 的时候,停掉播放、录像、释放播放端实例句柄:
以上是大概的流程,如果需要播放多实例,可以做个简单的封装,多实例效果如下:
编辑
LibPlayerWrapper.java参考封装代码如下,如需额外功能,只要按照设计框架,添加进去即可:
package com.daniulive.smartplayer;
import android.content.Context;
import android.util.Log;
import android.view.Surface;
import android.view.SurfaceView;
import android.view.View;
import com.eventhandle.NTSmartEventCallbackV2;
import java.lang.ref.WeakReference;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
public class LibPlayerWrapper {
private static String TAG = "NTLogLibPlayerW";
private static final int OK = 0;
private WeakReference<Context> context_;
private final ReadWriteLock rw_lock_ = new ReentrantReadWriteLock(true);
private final java.util.concurrent.locks.Lock write_lock_ = rw_lock_.writeLock();
private final java.util.concurrent.locks.Lock read_lock_ = rw_lock_.readLock();
private SmartPlayerJniV2 lib_player_;
private volatile long native_handle_;
private View view_;
private volatile boolean is_playing_;
private volatile boolean is_recording_;
private WeakReference<EventListener> event_listener_;
public LibPlayerWrapper(SmartPlayerJniV2 lib_player, Context context, EventListener listener) {
if (!empty())
throw new IllegalStateException("it is not empty");
if (null == lib_player)
throw new NullPointerException("lib_player is null");
this.lib_player_ = lib_player;
if (context != null)
this.context_ = new WeakReference<>(context);
if (listener == null ) {
this.event_listener_ = null;
}
else {
this.event_listener_ = new WeakReference<>(listener);
}
}
private void clear_all_playing_flags() {
this.is_playing_ = false;
this.is_recording_ = false;
}
public void set(long handle) {
if (!empty())
throw new IllegalStateException("it is not empty");
write_lock_.lock();
try {
clear_all_playing_flags();
this.native_handle_ = handle;
} finally {
write_lock_.unlock();
}
Log.i(TAG, "set native_handle:" + handle);
}
public void SetView(View view) {
Log.i(TAG, "SetView: " + view);
this.view_ = view;
}
@Override
protected void finalize() throws Throwable {
try {
if (check_native_handle()) {
if(is_playing()) {
lib_player_.SmartPlayerStopPlay(get());
this.is_playing_ = false;
}
if(is_recording()) {
lib_player_.SmartPlayerStopRecorder(get());
this.is_recording_ = false;
}
lib_player_.SmartPlayerClose(this.native_handle_);
Log.i(TAG, "finalize close handle:" + this.native_handle_);
this.native_handle_ = 0;
}
}catch (Exception e) {
}
super.finalize();
}
public void release() {
if (empty())
return;
if(is_playing())
StopPlayer();
if (is_recording())
StopRecorder();
long handle;
write_lock_.lock();
try {
handle = this.native_handle_;
this.native_handle_ = 0;
clear_all_playing_flags();
} finally {
write_lock_.unlock();
}
if (lib_player_ != null && handle != 0)
lib_player_.SmartPlayerClose(handle);
}
public boolean try_release() {
if (empty())
return false;
if (is_player_running()) {
Log.i(TAG, "try_release it is running, native_handle:" + get());
return false;
}
long handle;
write_lock_.lock();
try {
if (is_player_running())
return false;
handle = this.native_handle_;
this.native_handle_ = 0;
} finally {
write_lock_.unlock();
}
if (lib_player_ != null && handle != 0)
lib_player_.SmartPlayerClose(handle);
return true;
}
public final boolean empty() { return 0 == this.native_handle_; }
public final long get() { return this.native_handle_; }
public View get_view() {return this.view_;}
public final boolean check_native_handle() {
return this.lib_player_ != null && this.native_handle_ != 0;
}
public final boolean is_playing() { return is_playing_; }
public final boolean is_recording() { return is_recording_; }
public final boolean is_player_running() { return is_playing_ || is_recording_; }
private boolean isValidRtspOrRtmpUrl(String url) {
if (url == null || url.isEmpty()) {
return false;
}
return url.trim().startsWith("rtsp://") || url.startsWith("rtmp://");
}
private EventListener getListener() {
if ( this.event_listener_ == null )
return null;
return this.event_listener_.get();
}
protected final Context application_context() {
if (null == context_)
return null;
return context_.get();
}
public boolean OpenPlayerHandle(String playback_url, int play_buffer, int is_using_tcp) {
if (check_native_handle())
return true;
if(!isValidRtspOrRtmpUrl(playback_url))
return false;
long handle = lib_player_.SmartPlayerOpen(application_context());
if (0==handle) {
Log.e(TAG, "sdk open failed!");
return false;
}
lib_player_.SetSmartPlayerEventCallbackV2(handle, new EventHandleV2());
lib_player_.SmartPlayerSetBuffer(handle, play_buffer);
lib_player_.SmartPlayerSetReportDownloadSpeed(handle, 1, 4);
boolean isFastStartup = true;
lib_player_.SmartPlayerSetFastStartup(handle, isFastStartup ? 1 : 0);
int rtsp_timeout = 10;
lib_player_.SmartPlayerSetRTSPTimeout(handle, rtsp_timeout);
int is_auto_switch_tcp_udp = 1;
lib_player_.SmartPlayerSetRTSPAutoSwitchTcpUdp(handle, is_auto_switch_tcp_udp);
lib_player_.SmartPlayerSaveImageFlag(handle, 1);
lib_player_.SmartPlayerSetRTSPTcpMode(handle, is_using_tcp);
lib_player_.DisableEnhancedRTMP(handle, 0);
lib_player_.SmartPlayerSetUrl(handle, playback_url);
set(handle);
return true;
}
private void SetPlayerParam(boolean is_hardware_decoder, boolean is_enable_hardware_render_mode, boolean is_mute)
{
Surface surface = null;
int surface_codec_media_color_format = 0;
if (view_ != null && view_ instanceof SurfaceView && ((SurfaceView) view_).getHolder() != null)
surface = ((SurfaceView) view_).getHolder().getSurface();
lib_player_.SetSurface(get(), surface, surface_codec_media_color_format, 0, 0);
lib_player_.SmartPlayerSetRenderScaleMode(get(), 1);
if (is_hardware_decoder && is_enable_hardware_render_mode) {
lib_player_.SmartPlayerSetHWRenderMode(get(), 1);
}
lib_player_.SmartPlayerSetAudioOutputType(get(), 1);
lib_player_.SmartPlayerSetMute(get(), is_mute ? 1 : 0);
if (is_hardware_decoder) {
int isSupportHevcHwDecoder = lib_player_.SetSmartPlayerVideoHevcHWDecoder(get(), 1);
int isSupportH264HwDecoder = lib_player_.SetSmartPlayerVideoHWDecoder(get(), 1);
Log.i(TAG, "isSupportH264HwDecoder: " + isSupportH264HwDecoder + ", isSupportHevcHwDecoder: " + isSupportHevcHwDecoder);
}
boolean isLowLatency = true;
lib_player_.SmartPlayerSetLowLatencyMode(get(), isLowLatency ? 1 : 0);
boolean is_flip_vertical = false;
lib_player_.SmartPlayerSetFlipVertical(get(), is_flip_vertical ? 1 : 0);
boolean is_flip_horizontal = false;
lib_player_.SmartPlayerSetFlipHorizontal(get(), is_flip_horizontal ? 1 : 0);
int rotate_degrees = 0;
lib_player_.SmartPlayerSetRotation(get(), rotate_degrees);
int curAudioVolume = 100;
lib_player_.SmartPlayerSetAudioVolume(get(), curAudioVolume);
}
class EventHandleV2 implements NTSmartEventCallbackV2 {
@Override
public void onNTSmartEventCallbackV2(long handle, int id, long param1,
long param2, String param3, String param4, Object param5) {
if(event_listener_.get() != null)
{
event_listener_.get().onPlayerEventCallback(handle, id, param1, param2, param3, param4, param5);
}
}
}
public boolean SetMute(boolean is_mute) {
if (!check_native_handle())
return false;
return OK == lib_player_.SmartPlayerSetMute(get(), is_mute? 1 : 0);
}
public boolean SetInputAudioVolume(int volume) {
if (!check_native_handle())
return false;
return OK == lib_player_.SmartPlayerSetAudioVolume(get(), volume);
}
public boolean CaptureImage(int compress_format, int quality, String file_name, String user_data_string) {
if (!check_native_handle())
return false;
return OK == lib_player_.CaptureImage(get(), compress_format, quality, file_name, user_data_string);
}
public boolean StartPlayer(boolean is_hardware_decoder, boolean is_enable_hardware_render_mode, boolean is_mute) {
if (is_playing()) {
Log.e(TAG, "already playing, native_handle:" + get());
return false;
}
SetPlayerParam(is_hardware_decoder, is_enable_hardware_render_mode, is_mute);
int ret = lib_player_.SmartPlayerStartPlay(get());
if (ret != OK) {
Log.e(TAG, "call StartPlay failed, native_handle:" + get() + ", ret:" + ret);
return false;
}
write_lock_.lock();
try {
this.is_playing_ = true;
} finally {
write_lock_.unlock();
}
Log.i(TAG, "call StartPlayer OK, native_handle:" + get());
return true;
}
public boolean StopPlayer() {
if (!check_native_handle())
return false;
if (!is_playing()) {
Log.w(TAG, "it's not playing, native_handle:" + get());
return false;
}
boolean is_need_call = false;
write_lock_.lock();
try {
if (this.is_playing_) {
this.is_playing_ = false;
is_need_call = true;
}
} finally {
write_lock_.unlock();
}
if (is_need_call)
lib_player_.SmartPlayerStopPlay(get());
return true;
}
public boolean ConfigRecorderParam(String rec_dir, int file_max_size, int is_transcode_aac,
int is_record_video, int is_record_audio) {
if(!check_native_handle())
return false;
if (null == rec_dir || rec_dir.isEmpty())
return false;
int ret = lib_player_.SmartPlayerCreateFileDirectory(rec_dir);
if (ret != 0) {
Log.e(TAG, "Create record dir failed, path:" + rec_dir);
return false;
}
if (lib_player_.SmartPlayerSetRecorderDirectory(get(), rec_dir) != 0) {
Log.e(TAG, "Set record dir failed , path:" + rec_dir);
return false;
}
if (lib_player_.SmartPlayerSetRecorderFileMaxSize(get(),file_max_size) != 0) {
Log.e(TAG, "SmartPlayerSetRecorderFileMaxSize failed.");
return false;
}
lib_player_.SmartPlayerSetRecorderAudioTranscodeAAC(get(), is_transcode_aac);
lib_player_.SmartPlayerSetRecorderVideo(get(), is_record_video);
lib_player_.SmartPlayerSetRecorderAudio(get(), is_record_audio);
return true;
}
public boolean StartRecorder() {
if (is_recording()) {
Log.e(TAG, "already recording, native_handle:" + get());
return false;
}
int ret = lib_player_.SmartPlayerStartRecorder(get());
if (ret != OK) {
Log.e(TAG, "call SmartPlayerStartRecorder failed, native_handle:" + get() + ", ret:" + ret);
return false;
}
write_lock_.lock();
try {
this.is_recording_ = true;
} finally {
write_lock_.unlock();
}
Log.i(TAG, "call SmartPlayerStartRecorder OK, native_handle:" + get());
return true;
}
public boolean StopRecorder() {
if (!check_native_handle())
return false;
if (!is_recording()) {
Log.w(TAG, "it's not recording, native_handle:" + get());
return false;
}
boolean is_need_call = false;
write_lock_.lock();
try {
if (this.is_recording_) {
this.is_recording_ = false;
is_need_call = true;
}
} finally {
write_lock_.unlock();
}
if (is_need_call)
lib_player_.SmartPlayerStopRecorder(get());
return true;
}
private static boolean is_null_or_empty(String val) {
return null == val || val.isEmpty();
}
}
总结
以上是Android平台RTSP、RTMP直播播放模块对接说明,在此之前,我们针对SmartPlayer做过一些技术方面的探讨,从低延迟、音视频同步处理、多实例实现、解码效率、性能占用、解码后数据对接、实时截图、录像、网络抖动处理等各个维度,做过相关的技术分享。感兴趣的开发者,可以单独跟我们探讨。