iOS平台RTSP|RTMP直播播放器技术接入说明

 技术背景

大牛直播SDK自2015年发布RTSP、RTMP直播播放模块,迭代从未停止,SmartPlayer功能强大、性能强劲、高稳定、超低延迟、超低资源占用。无需赘述,全自研内核,行业内一致认可的跨平台RTSP、RTMP直播播放器。本文以iOS平台为例,介绍下如何集成RTSP、RTMP播放模块。

技术对接

 系统要求

  • SDK支持iOS 9.0及以上版本;
  • 支持的CPU架构:arm64(真机调试)。

准备工作

  • 相关库:libSmartPlayerSDK.a
  • 相关头文件:
    1. nt_common_media_define.h(如需转发或第三方数据对接)
    2. nt_event_define.h
    3. SmartPlayerSDK.h
  • 如集需要引入的framework
    1. libbz.tbd
    2. Libbz2.tbd
    3. libiconv.tbd
    4. libstdc++.tbd
    5. Libc++.tbd
    6. Accelerate.framework
    7. AssetsLibrary.framework
    8. AudioToolBox.framework
    9. AVFoundation.framework
    10. CoreMedia.framework
    11. Foundation.framework
    12. GLKit.framework
    13. OpenGLES.framework
    14. UIKit.framework
    15. VideoToolBox.framework
  • 如需集成到自己系统测试,请用大牛直播SDK的app name:

Info.plist–>右键Open As–>Source Code

添加或者编辑

<key>CFBundleName</key>
<string>SmartiOSPlayer</string>

  • 快照添加到“照片”权限:

Info.plist–>右键Open As–>Source Code 添加:

<key>NSPhotoLibraryUsageDescription</key>
<string>1</string>

  • 如需后台播放音频(添加后台播放权限):

功能支持

iOS端,RTMP|RTSP直播播放,我们设计实现的功能如下:

  • 音频:AAC/PCMA/PCMU/SPEEX(RTMP);
  • 视频:H.264;
  • 播放协议:RTMP或RTSP;
  • 支持纯音频、纯视频、音视频播放;
  • 支持多实例播放;
  • 支持网络状态、buffer状态等回调;
  • [RTSP协议]支持RTSP TCP/UDP模式设置;
  • [RTSP协议]支持RTSP TCP、UDP模式自动切换;
  • [RTSP协议]支持RTSP超时时间设置,单位:秒;
  • [RTSP协议]支持上报RTSP 401事件,如URL携带鉴权信息,会自动处理;
  • 支持buffer time设置;
  • 支持实时静音、取消静音;
  • 支持首屏秒开功能(需服务器缓存GOP);
  • 支持超低延迟模式; 断网自动重连,支持视频追赶;
  • 支持视频view实时旋转(0° 90° 180° 270°);
  • 支持视频view水平反转、垂直反转;
  • 支持图像等比例缩放绘制;
  • 支持实时快照;
  • 支持实时音量调节;
  • 支持YUV数据回调;
  • 支持H.264|H.265数据回调;
  • 支持AAC/SPEEX/PCMA/PCMU数据回调;
  • 支持RTMP扩展H.265播放(Enhanced RTMP);
  • 支持扩展录像功能;
  • 支持Unity3D接口;
  • 支持H.264扩展SEI接收模块;
  • 支持iOS 9.0及以上版本。

播放模块接口详解

iOS播放端SDK接口详解

调用描述 接口 接口描述
最先调用,创建播放实例,如成功返回player实例 SmartPlayerInitPlayer 初始化,创建player实例,此接口请第一个调用
Event回调 SmartPlayerDelegate 设置event callback,上层由handleSmartPlayerEvent处理
软、硬解码设置 SmartPlayerSetVideoDecoderMode 设置是否用硬解码播放,如硬解码不支持,自动适配到软解码

0: 软解码;

 1: 硬解码.

创建播放view SmartPlayerCreatePlayView x y width height 指定播放位置
设置播放view SmartPlayerSetPlayView 设置播放view到底层SDK
释放播放view SmartPlayeReleasePlayView 释放播放view
视频回调 设置YUV回调 SmartPlayerSetYuvBlock 设置拉流时,视频YUV数据回调
YUV回调 PlayerYuvDataBlock 提供解码后YUV/RGB数据接口,供用户自己render或进一步处理(如视频分析)
播放模式 缓冲时间设置 SmartPlayerSetBuffer 设置播放端缓存数据buffer,单位:毫秒,如不需buffer,设置为0
首屏秒开 SmartPlayerSetFastStartup 设置快速启动后,如果CDN缓存GOP,实现首屏秒开
低延迟模式 SmartPlayerSetLowLatencyMode 针对类似于直播娃娃机等期待超低延迟的使用场景,超低延迟播放模式下,延迟可达到200~400ms
快速切换URL SmartPlayerSwitchPlaybackUrl 快速切换播放url,快速切换时,只换播放source部分,适用于不同数据流之间,快速切换
RTSP TCP/UDP模式设置 SmartPlayerSetRTSPTcpMode 设置RTSP TCP/UDP模式,如不设置,默认UDP模式
RTSP超时时间设置 SmartPlayerSetRTSPTimeout 设置RTSP超时时间,timeout单位为秒,必须大于0
设置RTSP TCP/UDP自动切换 SmartPlayerSetRTSPAutoSwitchTcpUdp 对于RTSP来说,有些可能支持rtp over udp方式,有些可能支持使用rtp over tcp方式

为了方便使用,有些场景下可以开启自动尝试切换开关, 打开后如果udp无法播放,sdk会自动尝试tcp, 如果tcp方式播放不了,sdk会自动尝试udp.

实时静音 SmartPlayerSetMute 实时静音
设置播放音量 SmartPlayerSetAudioVolume 播放端音量实时调节,范围[0,100],0时为静音,100为原始流数据最大音量
视频镜像旋转 旋转 SmartPlayerSetRotation 设置顺时针旋转, 注意除了0度之外, 其他角度都会额外消耗性能,当前支持 0度,90度, 180度, 270度 旋转
水平反转 SmartPlayerSetFlipHorizontal 设置视频水平反转
垂直反转 SmartPlayerSetFlipVertical 设置视频垂直反转
设置URL SmartPlayerSetPlayURL 设置播放或录像的url
开始播放 SmartPlayerStart 开始播放RTSP/RTMP流
停止播放 SmartPlayerStop 停止播放RTSP/RTMP流
销毁播放实例 SmartPlayerUnInitPlayer 结束时必须调用close接口释放资源

录像模块接口详解

如需录像,录像相关的接口如下:

iOS播放端录像SDK接口详解

调用描述 接口 接口描述
录像设置 设置录像目录 SmartPlayerSetRecorderDirectory 设置录像文件目录
设置录像文件大小 SmartPlayerSetRecorderFileMaxSize 设置每个录像文件的大小,比如100M,超过这个大小后,会自动生成下一个录像文件
音频转码 SmartPlayerSetRecorderAudioTranscodeAAC 设置录像时音频转AAC编码的开关

aac比较通用,sdk增加其他音频编码(比如speex, pcmu, pcma等)转aac的功能.

录制视频 SmartPlayerSetRecorderVideo 设置是否录视频,默认的话,如果视频源有视频就录,没有就不录, 但有些场景下可能不想录制视频,只想录音频,所以增加个开关
录制音频 SmartPlayerSetRecorderAudio 设置是否录音频,默认的话,如果视频源有音频就录,没有就不录, 但有些场景下可能不想录制音频,只想录视频,所以增加个开关
开始录像 SmartPlayerStartRecorder 开始录像
停止录像 SmartPlayerStopRecorder 停止录像

Event回调详解

由于iOS播放录像SDK和播放端SDK可组合使用,相关Event同步更新在iOS播放端SDK(如下图):

iOS播放端SDK Event回调说明

事件ID 事件描述
EVENT_DANIULIVE_ERC_PLAYER_STARTED 开始播放
EVENT_DANIULIVE_ERC_PLAYER_CONNECTING 播放端连接中
EVENT_DANIULIVE_ERC_PLAYER_CONNECTION_FAILED 播放端连接失败
EVENT_DANIULIVE_ERC_PLAYER_CONNECTED 播放端连接成功
EVENT_DANIULIVE_ERC_PLAYER_DISCONNECTED 播放端连接断开
EVENT_DANIULIVE_ERC_PLAYER_STOP 停止播放
EVENT_DANIULIVE_ERC_PLAYER_RESOLUTION_INFO 返回视频宽、高信息
EVENT_DANIULIVE_ERC_PLAYER_NO_MEDIADATA_RECEIVED 收不到媒体数据(可能是URL错误)
EVENT_DANIULIVE_ERC_PLAYER_SWITCH_URL 快速切换URL
EVENT_DANIULIVE_ERC_PLAYER_RECORDER_START_NEW_FILE 开始一个新的录像文件(param3返回包含录像路径在内的录像文件名)
EVENT_DANIULIVE_ERC_PLAYER_ONE_RECORDER_FILE_FINISHED 已生成一个录像文件(param3返回包含录像路径在内的录像文件名)
EVENT_DANIULIVE_ERC_PLAYER_CAPTURE_IMAGE 播放端实时快照
EVENT_DANIULIVE_ERC_PLAYER_START_BUFFERING 开始缓冲数据
EVENT_DANIULIVE_ERC_PLAYER_BUFFERING 缓冲中(param1参数

会返回缓冲百分比)

EVENT_DANIULIVE_ERC_PLAYER_STOP_BUFFERING 停止缓冲数据
EVENT_DANIULIVE_ERC_PLAYER_DOWNLOAD_SPEED 返回当前RTSP/RTMP流实时下载速度
EVENT_DANIULIVE_ERC_PLAYER_RTSP_STATUS_CODE RTSP收到错误码,可能是用户名、密码不对

逻辑调用

先说开始播放:

//
//  ViewController.m
//  SmartiOSPlayerV2
//
//  Author: daniusdk.com
//  WeChat: xinsheng120
//  Created by daniulive on 2016/01/03.
//
- (void)playBtn:(UIButton *)button {
    
    NSLog(@"playBtn only++");
    
    button.selected = !button.selected;
    
    if (button.selected)
    {
        if(is_playing_)
            return;
        
        [self InitPlayer];
        
        //如需处理回调的用户数据+++++++++
        __weak __typeof(self) weakSelf = self;
        
        _smart_player_sdk.spUserDataCallBack = ^(int data_type, unsigned char *data, unsigned int size, unsigned long long timestamp, unsigned long long reserve1, long long reserve2, unsigned char *reserve3)
        {
            [weakSelf OnUserDataCallBack:data_type data:data size:size timestamp:timestamp reserve1:reserve1 reserve2:reserve2 reserve3:reserve3];
        };
        
        Boolean enableUserDataCallback = YES;
        [_smart_player_sdk SmartPlayerSetUserDataCallback:enableUserDataCallback];
         //如需处理回调的用户数据---------
        
        if(![self StartPlayer])
        {
            NSLog(@"Call StartPlayer failed..");
        }
        
        [playbackButton setTitle:@"停止播放" forState:UIControlStateNormal];
        
        is_playing_ = YES;
    }
    else
    {
        if ( !is_playing_ )
            return;
        
        [self StopPlayer];
        
        if(!is_recording_)
        {
            [self UnInitPlayer];
        }
        
        [playbackButton setTitle:@"开始播放" forState:UIControlStateNormal];
        
        is_mute_ = NO;
        [muteButton setTitle:@"实时静音" forState:UIControlStateNormal];
        
        is_playing_ = NO;
    }
}

其中,InitPlayer实现如下:

-(bool)InitPlayer
{
    NSLog(@"InitPlayer++");
    
    if(is_inited_player_)
    {
        NSLog(@"InitPlayer: has inited before..");
        return true;
    }
    
    //NSString* in_cid = @"";
    //NSString* in_key = @"";
    
    //[SmartPlayerSDK SmartPlayerSetSDKClientKey:in_cid in_key:in_key reserve1:0 reserve2:nil];
    
    _smart_player_sdk = [[SmartPlayerSDK alloc] init];
    
    if (_smart_player_sdk ==nil ) {
        NSLog(@"SmartPlayerSDK init failed..");
        return false;
    }
    
    if (playback_url_.length == 0) {
        NSLog(@"playback url is nil..");
        return false;
    }
    
    if (_smart_player_sdk.delegate == nil)
    {
        _smart_player_sdk.delegate = self;
        NSLog(@"SmartPlayerSDK _player.delegate:%@", _smart_player_sdk);
    }
    
    NSInteger initRet = [_smart_player_sdk SmartPlayerInitPlayer];
    if ( initRet != DANIULIVE_RETURN_OK )
    {
        NSLog(@"SmartPlayerSDK call SmartPlayerInitPlayer failed, ret=%ld", (long)initRet);
        return false;
    }
    
    [_smart_player_sdk SmartPlayerSetPlayURL:playback_url_];
    //[self try_set_rtsp_url:playback_url_];
    
    //超低延迟模式设置
    [_smart_player_sdk SmartPlayerSetLowLatencyMode:(NSInteger)is_low_latency_mode_];
    
    //buffer time设置
    if(buffer_time_ >= 0)
    {
        [_smart_player_sdk SmartPlayerSetBuffer:buffer_time_];
    }
    
    //快速启动模式设置
    [_smart_player_sdk SmartPlayerSetFastStartup:(NSInteger)is_fast_startup_];
    
    NSLog(@"[SmartPlayerV2]is_fast_startup_:%d, buffer_time_:%ld", is_fast_startup_, (long)buffer_time_);
    
    //RTSP TCP还是UDP模式
    [_smart_player_sdk SmartPlayerSetRTSPTcpMode:is_rtsp_tcp_mode_];
 
    //设置RTSP超时时间
    NSInteger rtsp_timeout = 10;
    [_smart_player_sdk SmartPlayerSetRTSPTimeout:rtsp_timeout];
    
    //设置RTSP TCP/UDP自动切换
    NSInteger is_tcp_udp_auto_switch = 1;
    [_smart_player_sdk SmartPlayerSetRTSPAutoSwitchTcpUdp:is_tcp_udp_auto_switch];
    
    //快照设置 如需快照 参数传1
    [_smart_player_sdk SmartPlayerSaveImageFlag:save_image_flag_];
    
    //如需查看实时流量信息,可打开以下接口
    NSInteger is_report = 1;
    NSInteger report_interval = 3;
    [_smart_player_sdk SmartPlayerSetReportDownloadSpeed:is_report report_interval:report_interval];
    
    //录像端音频,是否转AAC后保存
    NSInteger is_transcode = 1;
    [_smart_player_sdk SmartPlayerSetRecorderAudioTranscodeAAC:is_transcode];
    
    //录制MP4文件 是否录制视频
    NSInteger is_record_video = 1;
    [_smart_player_sdk SmartPlayerSetRecorderVideo:is_record_video];
    
    //录制MP4文件 是否录制音频
    NSInteger is_record_audio = 1;
    [_smart_player_sdk SmartPlayerSetRecorderAudio:is_record_audio];
    
    
    is_inited_player_ = YES;
    
    NSLog(@"InitPlayer--");
    return true;
}

停止播放StopPlayer实现如下:

-(bool)StopPlayer
{
    NSLog(@"StopPlayer++");
    
    if (_smart_player_sdk != nil)
    {
        [_smart_player_sdk SmartPlayerStop];
    }
    
    if (!is_audio_only_) {
        if (_glView != nil) {
            [_glView removeFromSuperview];
            [SmartPlayerSDK SmartPlayeReleasePlayView:(__bridge void *)(_glView)];
            _glView = nil;
        }
    }
    
    NSLog(@"StopPlayer--");
    return true;
}

UnInitPlayer实现如下:

-(bool)UnInitPlayer
{
    NSLog(@"UnInitPlayer++");
    
    if (_smart_player_sdk != nil)
    {
        [_smart_player_sdk SmartPlayerUnInitPlayer];
        
        if (_smart_player_sdk.delegate != nil)
        {
            _smart_player_sdk.delegate = nil;
        }
        
        _smart_player_sdk = nil;
    }
    
    is_inited_player_ = NO;
    
    NSLog(@"UnInitPlayer--");
    return true;
}

实时录像:

- (void)RecorderBtn:(UIButton *)button {
    
    NSLog(@"record Stream only++");
    
    button.selected = !button.selected;
    
    if (button.selected)
    {
        if(is_recording_)
            return;
        
        [self InitPlayer];
        
        //设置录像目录
        NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
        NSString *recorderDir = [paths objectAtIndex:0];
        
        if([_smart_player_sdk SmartPlayerSetRecorderDirectory:recorderDir] != DANIULIVE_RETURN_OK)
        {
            NSLog(@"Call SmartPlayerSetRecorderDirectory failed..");
        }
        
        //每个录像文件大小
        NSInteger size = 200;
        if([_smart_player_sdk SmartPlayerSetRecorderFileMaxSize:size] != DANIULIVE_RETURN_OK)
        {
            NSLog(@"Call SmartPlayerSetRecorderFileMaxSize failed..");
        }
        
        [_smart_player_sdk SmartPlayerStartRecorder];
        [recButton setTitle:@"停止录像" forState:UIControlStateNormal];
        
        is_recording_ = YES;
    }
    else
    {
        [_smart_player_sdk SmartPlayerStopRecorder];
        [recButton setTitle:@"开始录像" forState:UIControlStateNormal];
        
        if(!is_playing_)
        {
            [self UnInitPlayer];
        }
        
        is_recording_ = NO;
    }
}

实时快照:

- (void)SaveImageBtn:(UIButton *)button {
    if ( _smart_player_sdk != nil )
    {
        //设置快照目录
        NSLog(@"[SaveImageBtn] path++");
        
        NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
        NSString *saveImageDir = [paths objectAtIndex:0];
        
        NSLog(@"[SaveImageBtn] path: %@", saveImageDir);
        
        NSString* symbol = @"/";
        
        NSString* png = @".png";
        
        // 1.创建时间
        NSDate *datenow = [NSDate date];
        // 2.创建时间格式化
        NSDateFormatter *formatter = [[NSDateFormatter alloc] init];
        // 3.指定格式
        formatter.dateFormat = @"yyyyMMdd_HHmmss";
        // 4.格式化时间
        NSString *timeSp = [formatter stringFromDate:datenow];
        
        NSString* image_name =  [saveImageDir stringByAppendingString:symbol];
        
        image_name = [image_name stringByAppendingString:timeSp];
        
        image_name = [image_name stringByAppendingString:png];
        
        NSLog(@"[SaveImageBtn] image_name: %@", image_name);
        
        [_smart_player_sdk SmartPlayerSaveCurImage:image_name];
    }
}

Event回调处理如下:

- (NSInteger) handleSmartPlayerEvent:(NSInteger)nID param1:(unsigned long long)param1 param2:(unsigned long long)param2 param3:(NSString*)param3 param4:(NSString*)param4 pObj:(void *)pObj;
{
    NSString* player_event = @"";
    NSString* lable = @"";
    
    if (nID == EVENT_DANIULIVE_ERC_PLAYER_STARTED) {
        player_event = @"[event]开始播放..";
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTING)
    {
        player_event = @"[event]连接中..";
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTION_FAILED)
    {
        player_event = @"[event]连接失败..";
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTED)
    {
        player_event = @"[event]已连接..";
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_DISCONNECTED)
    {
        player_event = @"[event]断开连接..";
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_STOP)
    {
        player_event = @"[event]停止播放..";
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_RESOLUTION_INFO)
    {
        NSString *str_w = [NSString stringWithFormat:@"%ld", (long)param1];
        NSString *str_h = [NSString stringWithFormat:@"%ld", (long)param2];
        
        lable = @"[event]视频解码分辨率信息: ";
        player_event = [lable stringByAppendingFormat:@"%@*%@", str_w, str_h];
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_NO_MEDIADATA_RECEIVED)
    {
        player_event = @"[event]收不到RTMP数据..";
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_SWITCH_URL)
    {
        player_event = @"[event]快速切换url..";
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CAPTURE_IMAGE)
    {
        if ((int)param1 == 0)
        {
            NSLog(@"[event]快照成功: %@", param3);
            lable = @"[event]快照成功:";
            player_event = [lable stringByAppendingFormat:@"%@", param3];
            
            tmp_path_ = param3;
            
            image_path_ = [ UIImage imageNamed:param3];
            
            UIImageWriteToSavedPhotosAlbum(image_path_, self, @selector(image:didFinishSavingWithError:contextInfo:), NULL);
        }
        else
        {
            lable = @"[event]快照失败";
            player_event = [lable stringByAppendingFormat:@"%@", param3];
        }
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_RECORDER_START_NEW_FILE)
    {
        lable = @"[event]录像写入新文件..文件名:";
        player_event = [lable stringByAppendingFormat:@"%@", param3];
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_ONE_RECORDER_FILE_FINISHED)
    {
        lable = @"一个录像文件完成..文件名:";
        player_event = [lable stringByAppendingFormat:@"%@", param3];
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_START_BUFFERING)
    {
        //NSLog(@"[event]开始buffer..");
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_BUFFERING)
    {
        NSLog(@"[event]buffer百分比: %lld", param1);
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_STOP_BUFFERING)
    {
        //NSLog(@"[event]停止buffer..");
    }
    else if (nID == EVENT_DANIULIVE_ERC_PLAYER_DOWNLOAD_SPEED)
    {
        NSInteger speed_kbps = (NSInteger)param1*8/1000;
        NSInteger speed_KBs = (NSInteger)param1/1024;
        
        lable = @"[event]download speed :";
        player_event = [lable stringByAppendingFormat:@"%ld kbps - %ld KB/s", (long)speed_kbps, (long)speed_KBs];
    }
    else if(nID == EVENT_DANIULIVE_ERC_PLAYER_RTSP_STATUS_CODE)
    {
        lable = @"[event]RTSP status code received:";
        player_event = [lable stringByAppendingFormat:@"%ld", (long)param1];
        
        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            dispatch_async(dispatch_get_main_queue(), ^{
                UIAlertController *aleView=[UIAlertController alertControllerWithTitle:@"RTSP错误状态" message:player_event preferredStyle:UIAlertControllerStyleAlert];
                UIAlertAction *action_ok=[UIAlertAction actionWithTitle:@"确定" style:UIAlertActionStyleCancel handler:nil];
                [aleView addAction:action_ok];
                
                [self presentViewController:aleView animated:YES completion:nil];
            });
        });
    }
    else if(nID == EVENT_DANIULIVE_ERC_PLAYER_NEED_KEY)
    {
        player_event = @"[event]RTMP加密流,请设置播放需要的Key..";
    }
    else if(nID == EVENT_DANIULIVE_ERC_PLAYER_KEY_ERROR)
    {
        player_event = @"[event]RTMP加密流,Key错误,请重新设置..";
    }
    else
        NSLog(@"[event]nID:%lx", (long)nID);
    
    NSString* player_event_tag = @"当前状态:";
    NSString* event = [player_event_tag stringByAppendingFormat:@"%@", player_event];
    
    if ( player_event.length != 0)
    {
        NSLog(@"%@", event);
    }
    
    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            dispatch_async(dispatch_get_main_queue(), ^{
                self.textPlayerEventLabel.text = event;
            });
    });
    
    return 0;
}

总结

iOS平台RTSP、RTMP直播播放模块,延迟低、资源占有少,性能优异。由于设备和系统比较单一,优先考虑硬解码,除了基础播放外,我们还实现了实时快照、实时录像、实时回调YUV数据、实时音量调节等,实际体验下来,iOS平台RTMP和RTSP,可以轻松毫秒级。

[iOS]大牛直播SDK基于unity3d平台的rtmp/rtsp直播播放端SDK接口说明

2.1 demo说明

  • SmartU3diOSPlayer: 大牛直播SDK Unity3D iOS RTMP/RTSP直播播放端工程。

2.2 功能说明

标准接口:

  • 音频:AAC/PCMA/PCMU/SPEEX(RTMP);
  • 视频:H.264/H.265(RTSP);
  • 播放协议:RTMP/RTSP;
  • 支持RTSP TCP/UDP模式切换;
  • 支持RTSP timeout设置;
  • 支持RTSP TCP/UDP模式自动切换;
  • 支持纯音频、纯视频、音视频播放;
  • 支持秒开模式;
  • 音视频多种render机制;
  • 支持buffer设置;
  • 真正靠谱的超低延迟;
  • 支持多实例播放;
  • 支持播放url快速切换;
  • 断网自动重连,支持视频追赶;
  • 支持视频video实时旋转、水平反转、垂直反转。

增值接口:

  • 同时支持rtsp、rtmp播放;
  • 播放过程中,实时静音、取消静音;
  • 播放端回调YUV,供unity3d调用完成绘制;
  • 实时快照;
  • 实时录像。

2.3 集成说明

拷贝以下文件,到Assets–>Plugins–>iOS目录:

相关头文件和调用说明,参见:SmartPlayeriOSMono.cs

Unity3D工程下,File–>Build Settings,Platform选择iOS,然后点击build,设置目录,生成xcode工程:

生成后的xcode工程,添加以下依赖库:

  • 相关库:libSmartPlayerSDK.a
  • 引入以下依赖framework
    • libz.tbd
    • libbz2.tbd
    • libiconv.tbd
    • libstdc++.tbd
    • libc++.tbd
    • Accelerate.framework
    • AssetsLibrary.framework
    • AudioToolBox.framework
    • AVFoundation.framework
    • CoreMedia.framework
    • Foundation.framework
    • GLKit.framework
    • OpenGLES.framework
    • UIKit.framework
    • VideoToolBox.framework
  • 如需集成到自己系统测试,请用大牛直播的app name:

Info.plist–>右键Open As–>Source Code

添加或者编辑

<key>CFBundleName</key>

<string>SmartiOSPlayer</string>

  • 快照添加到“照片”权限:

Info.plist–>右键Open As–>Source Code

添加

<key>NSPhotoLibraryUsageDescription</key>
<string>1</string>

  • 导出后的xcode工程,如编译不过,参考以下设置: Library Search Paths:$(SRCROOT)/Libraries

2.4 调用时序(V2)

  1. 【最先调用】NT_U3D_Init:player初始化,目前预留;
  2. 【获得player句柄】NT_U3D_Open,设置上下文信息,返回player句柄;
  3. 【设置GameObject】NT_U3D_Set_Game_Object,注册Game Object,用于消息传递;
  4. 【设置硬解码】NT_U3D_SetVideoDecoderMode,设置是否用硬解码播放,如硬解码不支持,自动适配到软解码;
  5. 【缓冲设置】NT_U3D_SetBuffer,设置播放端缓存数据buffer,以毫秒(ms)为单位,如超低延迟模式下,不需buffer数据,设置为0;
  6. 【RTSP TCP/UDP设置】NT_U3D_SetRTSPTcpMode,设置TCP/UDP播放模式,注意:此接口仅用于RTSP;
  7. 【实时静音-可实时调用】NT_U3D_SetMute,设置播放过程中,实时静音/取消静音;
  8. 【快速启动】NT_U3D_SetFastStartup,Set fast startup(快速启动),设置快速启动后,如果CDN缓存GOP,daniulive player可快速出帧;
  9. 【低延迟模式】NT_U3D_SetPlayerLowLatencyMode,针对类似于直播娃娃机等期待超低延迟的使用场景,超低延迟播放模式下,延迟甚至可达到200~400ms;
  10. 【视频显示角度设置-可实时调用】NT_U3D_SetRotation,针对类似于安防摄像头或其他设备出来的图像倒置现象,支持视频播放view顺时针旋转, 当前支持 0度,90度, 180度, 270度 旋转,注意除了0度之外, 其他角度都会额外消耗性能;
  11. 【下载速度回调设置】NT_U3D_SetReportDownloadSpeed,设置下载速度上报, 默认不上报下载速度;
  12. 【快照设置】NT_U3D_SetSaveImageFlag(),设置是否需要在播放或录像过程中快照;
  13. 【快照-录像或播放后,可随时调用】NT_U3D_SaveCurImage,播放过程中,根据设置路径和文件名,实时快照;
  14. 【快速切换url-可实时调用】NT_U3D_SwitchPlaybackUrl,快速切换播放url,快速切换时,只换播放source部分,适用于不同数据流之间,快速切换(如娃娃机双摄像头切换或高低分辨率流切换);
  15. 【录像设置】NT_U3D_CreateFileDirectory,创建文件路径,注意:iOS只提供接口,未提供具体实现;
  16. 【录像设置】NT_U3D_SetRecorderDirectory,设置文件路径;
  17. 【录像设置】NT_U3D_SetRecorderFileMaxSize,设置每个录像文件最大size,以兆(M)为单位,范围(5M~500M);
  18. 【设置播放或录像URL】NT_U3D_SetUrl,设置播放/录像url;
  19. 【播放】NT_U3D_StartPlay,开始播放;
  20. 【播放】NT_U3D_GetVideoFrame,获取底层回调的YUV数据;
  21. 【播放】NT_U3D_StopPlay,停止播放;
  22. 【录像】NT_U3D_StartRecorder,开始录像;
  23. 【录像】NT_U3D_StopRecorder,停止录像;
  24. 【关闭】NT_U3D_Close, 关闭播放器实例;
  25. 【最后调用】NT_U3D_UnInit,UnInit Player,最后调用。

2.5 Event回调

/// <summary>

/// android 传递过来 code

/// </summary>

/// <param name=”code”></param>

public void onNTSmartEvent(string param)

{

if (!param.Contains(“,”))

{

Debug.Log(“[onNTSmartEvent] android传递参数错误”);

return;

}

string[] strs = param.Split(‘,’);

string player_handle =strs[0];

string code = strs[1];

string param1 = strs[2];

string param2 = strs[3];

string param3 = strs[4];

string param4 = strs[5];

Debug.Log(“[onNTSmartEvent] code: 0x” + Convert.ToString(Convert.ToInt32(code), 16));

switch (Convert.ToInt32(code))

{

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_STARTED:

Debug.Log(“开始。。”);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_CONNECTING:

Debug.Log(“连接中。。”);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_CONNECTION_FAILED:

Debug.Log(“连接失败。。”);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_CONNECTED:

Debug.Log(“连接成功。。”);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_DISCONNECTED:

Debug.Log(“连接断开。。”);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_STOP:

Debug.Log(“停止播放。。”);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_RESOLUTION_INFO:

Debug.Log(“分辨率信息: width: ” + Convert.ToInt32(param1) + “, height: ” + Convert.ToInt32(param2));

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_NO_MEDIADATA_RECEIVED:

Debug.Log(“收不到媒体数据,可能是url错误。。”);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_SWITCH_URL:

Debug.Log(“切换播放URL。。”);

break;

 

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_CAPTURE_IMAGE:

Debug.Log(“快照: ” + param1 + ” 路径:” + param3);

 

if (Convert.ToInt32(param1) == 0)

{

Debug.Log(“截取快照成功。.”);

}

else

{

Debug.Log(“截取快照失败。.”);

}

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_RECORDER_START_NEW_FILE:

Debug.Log(“[record]开始一个新的录像文件 : ” + param3);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_ONE_RECORDER_FILE_FINISHED:

Debug.Log(“[record]已生成一个录像文件 : ” + param3);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_START_BUFFERING:

Debug.Log(“Start_Buffering”);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_BUFFERING:

Debug.Log(“Buffering: ” + Convert.ToInt32(param1));

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_STOP_BUFFERING:

Debug.Log(“Stop_Buffering”);

break;

case EVENTID.EVENT_DANIULIVE_ERC_PLAYER_DOWNLOAD_SPEED:

Debug.Log(“download_speed:” + param1 + “Byte/s” + “, ”

+ (Convert.ToInt32(param1) * 8 / 1000) + “kbps” + “, ” + (Convert.ToInt32(param1) / 1024)

+ “KB/s”);

break;

}

}

更多资料,可以访问 https://github.com/daniulive/SmarterStreaming or http://www.daniulive.com/