awin/Obsidian-Main
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

vault backup: 2025-03-04 11:12:06

Browse Source
This commit is contained in:
Awin Huang
2025-03-04 11:12:06 +08:00
parent fd1b61cd3b
commit d1e51bfd2f
144 changed files with 0 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

303
-20.01. Programming/FFMPEG/FFMpeg.md Normal file
View File

@@ -0,0 +1,303 @@
## Initialization
```cpp
av_register_all();
avformat_network_init();
```
## Command line
利用FFMpeg來轉檔
```
ffmpeg -i <SOURCE.mp4> -t 10 -s <W>x<H> -pix_fmt <OUTPUT_FORMAT> <OUTPUT_FILENAME>
```
- `-i`：來源檔名
- `-t`：影片長度（秒）
- `-s`：影片大小，長x寬
- `-pix_fmt`：輸出的影像格式
example:
```
ffmpeg -i test.mp4 -t 10 -s 240x128 -pix_fmt yuv420p out240x128.yuv
```
## Open file
使用`avformat_open_input()`來打開檔案。
```cpp
AVFormatContext* pFormatContext = NULL;
char* path = "E:\\download\\Incredible Patagonia_1920x1080.mp4";
AVDictionary* pDict = NULL;
int result = avformat_open_input(&pFormatContext, path, NULL, &pDict);
```
若是開啟失敗`result`會是一個非0的值。可以用`av_strerror()`來印出詳細的錯誤訊息。例：
```cpp
char errorMsg[1024] = { 0 };
av_strerror(result, errorMsg, sizeof(errorMsg) - 1);
printf("Open file %s error, code = %d, message = %s\n", path, result, errorMsg);
```
### Get file information
用`avformat_find_stream_info()`來把檔案的video/audio相關訊息放到`AVFormatContext`裡面。例：
```cpp
result = avformat_find_stream_info(pFormatContext, NULL);
```
也可以用
```cpp
av_dump_format(pFormatContext, 0, path, 0);
```
來直接印出相關訊息，不過這方法是Ffmpeg內定的，如果要印出自己想要的內容，就必須將`AVFormatContext`裡面的`AVStream`一一撈出來：
```cpp
for (int i = 0; i < pFormatContext->nb_streams; i++) {
AVStream* avStream = pFormatContext->streams[i];
if (avStream->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) {
printf("Stream[%d] is a audio stream.\n", i);
printf(" Sample rate: %d\n", avStream->codecpar->sample_rate);
printf(" Format : %d\n", avStream->codecpar->format);
printf(" Channels : %d\n", avStream->codecpar->channels);
printf(" FPS : %d/%d\n", avStream->avg_frame_rate.num, avStream->avg_frame_rate.den);
printf(" Frame Size : %d\n", avStream->codecpar->frame_size);
printf(" Codec ID : %d\n", avStream->codecpar->codec_id);
} else if (avStream->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) {
printf("Stream[%d] is a video stream.\n", i);
printf(" Width : %d\n", avStream->codecpar->width);
printf(" Height: %d\n", avStream->codecpar->height);
printf(" FPS : %d/%d\n", avStream->avg_frame_rate.num, avStream->avg_frame_rate.den);
printf(" Codec ID: %d\n", avStream->codecpar->codec_id);
}
}
```
這個for-loop可以找出檔案裡面的video stream index與audio stream index。
另一個找出video stream index與audio stream index的方法：
```cpp
int videoStream = av_find_best_stream(pFormatContext, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
int audioStream = av_find_best_stream(pFormatContext, AVMEDIA_TYPE_AUDIO, -1, -1, NULL, 0);
printf("Found best video stream index: %d\n", videoStream);
printf("Found best audio stream index: %d\n", audioStream);
```
## Demux
### 重要結構
#### AVPackt
##### 重要的member：
- `AVBufferRef *buf`：buffer的reference count，基本上不會用到。
- `int64_t pts`：顯示順序
- `int64_t dts`：decode順序
- `uint8_t *data`：實際存data的buffer
- `int size`：上面那個buffer的大小
##### 操作function
- `AVPacket* av_packet_alloc()`：產生一個兂的AVPacket
- `AVPacket* av_packet_clone(const AVPacket*)`：複製一個AVPacket
- `int av_packet_ref(AVPacket*, const AVPacket*)`：增加reference count
- `av_packet_unref(AVPacket*)`：減少reference count
- `void av_packet_free(AVPacket**)`：釋放AVPacket
- `void av_init_packet(AVPacket*)`：初始化AVPacket
- `int av_packet_from_data(AVPacket*, uint8_t*, int size)`：從自己的data
- `av_seek_frame`：
- `AVFormatContext*`
- `int stream_index`：要seek的stream
- `int64_t timestamp`：timestamp
- `int flag`：seek的方法
- `#define AVSEEK_FLAG_BACKWARD 1`：往後找
- `#define AVSEEK_FLAG_BYTE 2`：以byte為單位，跳到timestamp相對應的byte
- `#define AVSEEK_FLAG_ANY 4`：跳到任意frame，不一定要key frame
- `#define AVSEEK_FLAG_FRAME 8`：依照frame number來找
## Decode
### 重要結構
#### AVCodecContext
##### 重要的member
- `int thread_count`：執行續的數量
- `AVRational time_base`
##### 操作function
- `avcodec_register_all()`：註冊所有的decoder。
- `AVCodec* avcodec_find_decoder(enum AVCodecID)`：用codec ID來找到相對應的codec instance
- `AVCodec* avcodec_find_decoder_by_name(const char* name)`：用codec的名子來找到相對應的codec instance
- `AVCodecContext* avcodec_alloc_context3(const AVCodec*)`：用codec來建立`AVCodecContext`，codec只是用來解碼，解碼之後的相關資料要放在`AVCodecContext`裡面。
- `void avcodec_free_context(AVCodecContext**)`：釋放`AVCodecContext`。
- `int avcodec_open2(AVCodecContext*, const AVCodec*, AVDictionary*)`
- 開始解碼
- `AVDictionary`的選項請看`libavcodec/options_table.h`
#### AVFrame
##### 重要的member
- `uint8_t* data[AV_NUM_DATA_POINTERS]`：存放這個frame的資料
- `int linesize[AV_NUM_DATA_POINTERS]`：存放指向各個plane的pointer
- Plane mode
- For video
- 0：Y
- 1：U
- 2：V
- 3：NULL
- For audio
- 0：Left channel
- 1：Right channel
- 2：NULL
- Non plane mode
- For video
- 0：RGB RGB RGB
- 1：NULL
- `int width`：For video, frame width
- `int height`：For video, frame height
- `int nb_samples`：For audio
- `int64_t pts`：這個frame的pts
- `int64_t pkt_dts`：Packet的dts
- `int sample_rate`：For audio，sampling rate
- `uint64_t channel_layout`：For audio
- `int channels`：For audio
- `int format`：For video it's AVPixelFormat, for audio it's AVSampleFormat
##### 操作function
- `AVFrame* av_frame_alloc()`：
- `void av_frame_free(AVFrame*)`：
- `AVFrame* av_frame_clone(const AVFrame*)`：
- `int av_frame_ref(AVFrame* dst, const AVFrame* src)`：
- `void av_frame_unref(AVFrame*)`：
- `int avcodec_send_packet(AVCodecContext*, const AVPacket*)`
- `int avcodec_receive_frame(AVCodecContext*, AVFrame*)`
## Frame scaling
##### 操作function
- `sws_getContext`
- `sws_getCachedContext`
```
SwsContext* sws_getCachedContext(
SwsContext*,
int srcW, int srcH, enum AVPixelFormat srcFormat,
int dstW, int dstH, enum AVPixelFormat dstFormat,
int flags,
SwsFilter srcFilter,
SwsFilter dstFilter,
const double* param)
```
- `sws_scale`
```
int sws_scale(
SwsContext*,
const uint8_t* const srcSlice[], // 來源buffer的pointer, 如果是plane mode，要丟srcSlice[3]
const int srcStride[], // linesize
int srcSliceY,
int srcSliceH, // image height
uint8_t* const dst[], // 目標buffer的pointer, 如果是plane mode，要丟dst[3]
const int dstStride[]) // linesize
```
- `void sws_freeContext(SwsContext*)`
## Audio resampling
- `SwrContext* swr_alloc()`
- `swr_alloc_set_opts`
```cpp
SwrContext* swr_alloc_set_opts(
SwrContext*,
int64_t out_ch_layout, // Output
AVSampleFormat outSampleFormat, // Output
int outSampleRate, // Output
int64_t in_ch_layout, // Input
AVSampleFormat inSampleFormat, // Input
int inSampleRate, // Input
int log_offset, // 一般丟0
void* log_ctx) // 一般丟0
```
- `int swr_init(SwrContext*)`
- `void swr_free(SwrContext**)`
- `swr_convert`
```cpp
int swt_convert(
SwrContext*,
uint8_t** outData,
int outCount, // Sample count for one channel
uint8_t** inData,
int inCount) // Sample count for one channel
```
## Audio 播放
### 會用到的class
#### QAudioFormat
- `setSampleRate`
- `setSampleSize`
- `setChannelCount`
- `setCodec("audio/pcm")`
- `setByteOrder(QAudioFormat::LittleEndian)`
- `setSampleType(QAudioFormat::UnSignedInt)`
#### QAudioOutput
- Constuctor參數就是一個`QAudioFormat`
- `QIODevice* start()`
- `suspend()`
- `resume()`
- `bufferSize()`
- `byteFree()`
- `periodSize()`
#### QIODevice
- `qint64 write(const char* data, qint64 len)`
#### QBuffer
```cpp
QBuffer buffer;
if (buffer.open(QIODevice::ReadWrite)) {
qInfo() << "Buffer Opened!";
QByteArray data("Hello World");
for (int i = 0; i < 5; i++) {
buffer.write(data);
buffer.write("\r\n");
}
//File and device access you may need to flush the data to the device
//buffer.flush()
//Move to the first position
buffer.seek(0);
qInfo() << buffer.readAll(); // retuen a QVariant
qInfo() << "closing the buffer";
//ALWAYS close your device!
buffer.close();
}
```
### Example
```cpp
QAudioFormat audioFormat;
audioFormat.setSampleRate(44100);
audioFormat.setSampleSize(16);
audioFormat.setChannelCount(2);
audioFormat.setCodec("audio/pcm");
audioFormat.setByteOrder(QAudioFormat::LittleEndian);
audioFormat.setSampleType(QAudioFormat::UnSignedInt);
QAudioOutput* audioOutput = new QAudioOutput(audioFormat);
QIODevice* ioDevice = audioOutput->start();
int size = audioOutput->periodSize();
char* buffer = new char[size];
FILE *pFile = fopen("E:\\download\\out.pcm", "rb");
while (!feof(pFile)) {
if (audioOutput->bytesFree() < size) {
QThread::msleep(1);
}
qint64 len = fread(buffer, 1, size, pFile);
if (len <= 0) {
break;
}
ioDevice->write(buffer, len);
}
fclose(pFile);
delete buffer;
buffer = NULL;
```
### 使用FFmpeg decode audio
- [音视频开发之旅（60) -调试分析FFmpeg (解封装部分的）常用结构体](https://zhuanlan.zhihu.com/p/441055685)
## 參考資料
- [mpv.io](https://mpv.io/)
- [《FFmpeg基础知识》](https://opensourcelibs.com/lib/zhangfangtaozft-ffmpeg)
- [ffmpeg - 专题 - 简书](https://www.jianshu.com/c/21a6fafc8ee3)
- [FFmpeg打开输入文件 · FFmpeg原理](https://ffmpeg.xianwaizhiyin.net/api-ffmpeg/input.html)
- [FFplay视频同步分析 - 掘金](https://juejin.cn/post/7156903872789741581)