awin/Obsidian-Main
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

vault backup: 2022-09-30 21:53:03

Browse Source 
Affected files:
.obsidian/workspace
03. 專注Study/Android/ADB 取得 APK 的 icon.md
03. 專注Study/Android/ADB.md
03. 專注Study/Android/AOSP.md
03. 專注Study/Android/Android programming.md
03. 專注Study/Android/AudioTrack.md
03. 專注Study/Android/Ktor.md
03. 專注Study/Android/Service.md
03. 專注Study/Android/Tools.md
03. 專注Study/Android/UI.md
03. 專注Study/C++/C++17.md
03. 專注Study/C++/Class template.md
03. 專注Study/C++/GCC.md
03. 專注Study/C++/Structured binding declaration.md
03. 專注Study/C++/for_each.md
03. 專注Study/C++/lambda.md
03. 專注Study/C++/lvalue.md
03. 專注Study/C++/move operator.md
03. 專注Study/C++/rvalue.md
03. 專注Study/C++/智慧指標.md
03. 專注Study/RxKotlin/20200207 - Study RxKotlin.md
04. Programming/COM/20210726 - COM Interface.md
04. Programming/DB/MySQL.md
04. Programming/DB/sqlite.md
04. Programming/Design Pattern.md
04. Programming/FFMPEG/00. Introduction.md
04. Programming/FFMPEG/01. Setup.md
04. Programming/FFMPEG/FFMpeg.md
04. Programming/Flask.md
04. Programming/Kotlin/class.md
04. Programming/Kotlin/run, let, with, also 和 apply.md
04. Programming/Media Foundation/20210604 - Windows media foundation.md
04. Programming/OpenCV.md
04. Programming/OpenGL.md
04. Programming/Python/argparse.ArgumentParser.md
04. Programming/Python/decorator.md
04. Programming/Python/logging.md
04. Programming/Python/opencv.md
04. Programming/Python/subprocess.md
04. Programming/Python/threading.md
04. Programming/Python/tkinter.md
04. Programming/Python/檢測工具.md
04. Programming/QT/Dropdown button.md
04. Programming/QT/QVariant.md
04. Programming/QT/Qt.md
04. Programming/UML.md
04. Programming/演算法.md
05. 資料收集/99. templates/blogHeader.md
05. 資料收集/99. templates/date.md
05. 資料收集/99. templates/front matter.md
05. 資料收集/99. templates/note.md
05. 資料收集/99. templates/table.md
05. 資料收集/99. templates/thisWeek.md
05. 資料收集/99. templates/日記.md
05. 資料收集/99. templates/讀書筆記.md
05. 資料收集/Linux/CLI/cut.md
05. 資料收集/Linux/CLI/scp.md
05. 資料收集/Linux/CLI/timedatectl.md
05. 資料收集/Linux/Programming.md
05. 資料收集/Linux/Ubuntu.md
05. 資料收集/Tool Setup/Hardware/RaspberryPi.md
05. 資料收集/Tool Setup/Software/Chrome.md
05. 資料收集/Tool Setup/Software/Obisidian.md
05. 資料收集/Tool Setup/Software/SublimeText.md
05. 資料收集/Tool Setup/Software/VirtualBox.md
05. 資料收集/Tool Setup/Software/Visual Studio Code.md
05. 資料收集/Tool Setup/Software/Windows Setup.md
05. 資料收集/Tool Setup/Software/Windows Terminal.md
05. 資料收集/Tool Setup/Software/freefilesync.md
05. 資料收集/Tool Setup/Software/vim.md
05. 資料收集/名言佳句.md
05. 資料收集/架站/Gitea.md
05. 資料收集/架站/HTTP Server/Apache.md
05. 資料收集/架站/HTTP Server/Nginx/Reverse Proxy(Layer4).md
05. 資料收集/架站/Pelican blog.md
05. 資料收集/架站/Proxmox VE.md
05. 資料收集/架站/SWAG Reverse proxy.md
05. 資料收集/架站/Storj.md
05. 資料收集/架站/Trojan.md
05. 資料收集/每週外食.md
05. 資料收集/科技/802.11.md
05. 資料收集/科技/HDR Sensor.md
05. 資料收集/科技/量子電腦.md
05. 資料收集/科技/鋰電池.md
05. 資料收集/興趣嗜好/RC/Traxxas Sledge.md
05. 資料收集/興趣嗜好/RC/好盈電變調整中立點.md
05. 資料收集/興趣嗜好/RC/差速器調教教學.md
05. 資料收集/興趣嗜好/模型/舊化作例.md
05. 資料收集/興趣嗜好/軍武/虎式.md
05. 資料收集/讀書筆記/20201201 - 學習如何學習.md
05. 資料收集/讀書筆記/20201218 - Kotlin權威2.0.md
05. 資料收集/讀書筆記/20201224 - 寫作是最好的自我投資.md
05. 資料收集/讀書筆記/20210119 - 中產悲歌.md
05. 資料收集/讀書筆記/20210220 - 最高學習法.md
05. 資料收集/讀書筆記/20210320 - 最高學以致用法.md
05. 資料收集/讀書筆記/20210406 - 精準購買.md
05. 資料收集/讀書筆記/20210723 - 高手學習.md
05. 資料收集/讀書筆記/20220526 - 深入淺出設計模式.md
05. 資料收集/讀書筆記/20220619 - 精確的力量.md
05. 資料收集/軟體工具/IPFS.md
05. 資料收集/軟體工具/MkDocs.md
05. 資料收集/軟體工具/Obsidian.md
05. 資料收集/軟體工具/docker.md
05. 資料收集/軟體工具/git/apply.md
05. 資料收集/軟體工具/git/submodule.md
05. 資料收集/軟體工具/youtube-dl.md
05. 資料收集/面試準備/技术面试最后反问面试官的话.md
This commit is contained in:
Awin Huang
2022-09-30 21:53:04 +08:00
parent 3fe30e50b8
commit 5e4265b783
107 changed files with 36 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
04. Programming/FFMPEG/00. Introduction.md Normal file
View File

@@ -0,0 +1,17 @@
- [FFmpeg](https://www.ffmpeg.org/)
- Open source
- For handling multimedia files and streams
- GPL/LGPS license
- Inspired by MPEG(Moving Picture Experts Group), FF means "Fast Foward"
- Libraries
- libavcodec: Contain all the encoders and decoders
- libavformat: Muxers and demuxers
- libavfilter: Filters that you can use on your video and audio according to your requirements
- libavdevice: Input and output devices
- libavutil
- libswscale
- libswresample
- Tools
- ffmpeg: Main transcoding engine
- ffplay: Minimum tool to play a video or audio
- ffprobe: For inspecting medias and extract valuable information

30
04. Programming/FFMPEG/01. Setup.md Normal file
View File

@@ -0,0 +1,30 @@
FFmpeg doesn't support any official builds for any operation systems. You can build the codes from source, or use the prebuilt packages.
## From source
Get source code from [snapshot](https://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2) or git(`git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg`).
## From prebuilt packages
### MacOS
Use homebrew to install FFMPEG package
```
brew install ffmpeg
```
### Ubuntu
```
sudo apt update ;\
sudo apt install ffmpeg
```
### Windows
Goto https://www.ffmpeg.org/download.html, and
![[Pasted image 20220607105807.png]]
And then browser jumps to https://www.gyan.dev/ffmpeg/builds/, click [release build](https://www.gyan.dev/ffmpeg/builds/#release-builds), download **release-full**.
![[Pasted image 20220607110211.png]]
Unzip the file, then put the folder to anywhere you like, e.g. `C:\libs`, so the full path of ffmpeg packages should looks like: `C:\libs\ffmpeg-5.0.1-full_build`
#### Add to system variable
Add the path of bin of ffmpeg packages to `PATH` system variable.
![[Pasted image 20220607110949.png]]

301
04. Programming/FFMPEG/FFMpeg.md Normal file
View File

@@ -0,0 +1,301 @@
## Initialization
```cpp
av_register_all();
avformat_network_init();
```
## Command line
利用FFMpeg來轉檔
```
ffmpeg -i <SOURCE.mp4> -t 10 -s <W>x<H> -pix_fmt <OUTPUT_FORMAT> <OUTPUT_FILENAME>
```
- `-i`：來源檔名
- `-t`：影片長度（秒）
- `-s`：影片大小，長x寬
- `-pix_fmt`：輸出的影像格式
example:
```
ffmpeg -i test.mp4 -t 10 -s 240x128 -pix_fmt yuv420p out240x128.yuv
```
## Open file
使用`avformat_open_input()`來打開檔案。
```cpp
AVFormatContext* pFormatContext = NULL;
char* path = "E:\\download\\Incredible Patagonia_1920x1080.mp4";
AVDictionary* pDict = NULL;
int result = avformat_open_input(&pFormatContext, path, NULL, &pDict);
```
若是開啟失敗`result`會是一個非0的值。可以用`av_strerror()`來印出詳細的錯誤訊息。例：
```cpp
char errorMsg[1024] = { 0 };
av_strerror(result, errorMsg, sizeof(errorMsg) - 1);
printf("Open file %s error, code = %d, message = %s\n", path, result, errorMsg);
```
### Get file information
用`avformat_find_stream_info()`來把檔案的video/audio相關訊息放到`AVFormatContext`裡面。例：
```cpp
result = avformat_find_stream_info(pFormatContext, NULL);
```
也可以用
```cpp
av_dump_format(pFormatContext, 0, path, 0);
```
來直接印出相關訊息，不過這方法是Ffmpeg內定的，如果要印出自己想要的內容，就必須將`AVFormatContext`裡面的`AVStream`一一撈出來：
```cpp
for (int i = 0; i < pFormatContext->nb_streams; i++) {
AVStream* avStream = pFormatContext->streams[i];
if (avStream->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) {
printf("Stream[%d] is a audio stream.\n", i);
printf(" Sample rate: %d\n", avStream->codecpar->sample_rate);
printf(" Format : %d\n", avStream->codecpar->format);
printf(" Channels : %d\n", avStream->codecpar->channels);
printf(" FPS : %d/%d\n", avStream->avg_frame_rate.num, avStream->avg_frame_rate.den);
printf(" Frame Size : %d\n", avStream->codecpar->frame_size);
printf(" Codec ID : %d\n", avStream->codecpar->codec_id);
} else if (avStream->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) {
printf("Stream[%d] is a video stream.\n", i);
printf(" Width : %d\n", avStream->codecpar->width);
printf(" Height: %d\n", avStream->codecpar->height);
printf(" FPS : %d/%d\n", avStream->avg_frame_rate.num, avStream->avg_frame_rate.den);
printf(" Codec ID: %d\n", avStream->codecpar->codec_id);
}
}
```
這個for-loop可以找出檔案裡面的video stream index與audio stream index。
另一個找出video stream index與audio stream index的方法：
```cpp
int videoStream = av_find_best_stream(pFormatContext, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
int audioStream = av_find_best_stream(pFormatContext, AVMEDIA_TYPE_AUDIO, -1, -1, NULL, 0);
printf("Found best video stream index: %d\n", videoStream);
printf("Found best audio stream index: %d\n", audioStream);
```
## Demux
### 重要結構
#### AVPackt
##### 重要的member：
- `AVBufferRef *buf`：buffer的reference count，基本上不會用到。
- `int64_t pts`：顯示順序
- `int64_t dts`：decode順序
- `uint8_t *data`：實際存data的buffer
- `int size`：上面那個buffer的大小
##### 操作function
- `AVPacket* av_packet_alloc()`：產生一個兂的AVPacket
- `AVPacket* av_packet_clone(const AVPacket*)`：複製一個AVPacket
- `int av_packet_ref(AVPacket*, const AVPacket*)`：增加reference count
- `av_packet_unref(AVPacket*)`：減少reference count
- `void av_packet_free(AVPacket**)`：釋放AVPacket
- `void av_init_packet(AVPacket*)`：初始化AVPacket
- `int av_packet_from_data(AVPacket*, uint8_t*, int size)`：從自己的data
- `av_seek_frame`：
- `AVFormatContext*`
- `int stream_index`：要seek的stream
- `int64_t timestamp`：timestamp
- `int flag`：seek的方法
- `#define AVSEEK_FLAG_BACKWARD 1`：往後找
- `#define AVSEEK_FLAG_BYTE 2`：以byte為單位，跳到timestamp相對應的byte
- `#define AVSEEK_FLAG_ANY 4`：跳到任意frame，不一定要key frame
- `#define AVSEEK_FLAG_FRAME 8`：依照frame number來找
## Decode
### 重要結構
#### AVCodecContext
##### 重要的member
- `int thread_count`：執行續的數量
- `AVRational time_base`
##### 操作function
- `avcodec_register_all()`：註冊所有的decoder。
- `AVCodec* avcodec_find_decoder(enum AVCodecID)`：用codec ID來找到相對應的codec instance
- `AVCodec* avcodec_find_decoder_by_name(const char* name)`：用codec的名子來找到相對應的codec instance
- `AVCodecContext* avcodec_alloc_context3(const AVCodec*)`：用codec來建立`AVCodecContext`，codec只是用來解碼，解碼之後的相關資料要放在`AVCodecContext`裡面。
- `void avcodec_free_context(AVCodecContext**)`：釋放`AVCodecContext`。
- `int avcodec_open2(AVCodecContext*, const AVCodec*, AVDictionary*)`
- 開始解碼
- `AVDictionary`的選項請看`libavcodec/options_table.h`
#### AVFrame
##### 重要的member
- `uint8_t* data[AV_NUM_DATA_POINTERS]`：存放這個frame的資料
- `int linesize[AV_NUM_DATA_POINTERS]`：存放指向各個plane的pointer
- Plane mode
- For video
- 0：Y
- 1：U
- 2：V
- 3：NULL
- For audio
- 0：Left channel
- 1：Right channel
- 2：NULL
- Non plane mode
- For video
- 0：RGB RGB RGB
- 1：NULL
- `int width`：For video, frame width
- `int height`：For video, frame height
- `int nb_samples`：For audio
- `int64_t pts`：這個frame的pts
- `int64_t pkt_dts`：Packet的dts
- `int sample_rate`：For audio，sampling rate
- `uint64_t channel_layout`：For audio
- `int channels`：For audio
- `int format`：For video it's AVPixelFormat, for audio it's AVSampleFormat
##### 操作function
- `AVFrame* av_frame_alloc()`：
- `void av_frame_free(AVFrame*)`：
- `AVFrame* av_frame_clone(const AVFrame*)`：
- `int av_frame_ref(AVFrame* dst, const AVFrame* src)`：
- `void av_frame_unref(AVFrame*)`：
- `int avcodec_send_packet(AVCodecContext*, const AVPacket*)`
- `int avcodec_receive_frame(AVCodecContext*, AVFrame*)`
## Frame scaling
##### 操作function
- `sws_getContext`
- `sws_getCachedContext`
```
SwsContext* sws_getCachedContext(
SwsContext*,
int srcW, int srcH, enum AVPixelFormat srcFormat,
int dstW, int dstH, enum AVPixelFormat dstFormat,
int flags,
SwsFilter srcFilter,
SwsFilter dstFilter,
const double* param)
```
- `sws_scale`
```
int sws_scale(
SwsContext*,
const uint8_t* const srcSlice[], // 來源buffer的pointer, 如果是plane mode，要丟srcSlice[3]
const int srcStride[], // linesize
int srcSliceY,
int srcSliceH, // image height
uint8_t* const dst[], // 目標buffer的pointer, 如果是plane mode，要丟dst[3]
const int dstStride[]) // linesize
```
- `void sws_freeContext(SwsContext*)`
## Audio resampling
- `SwrContext* swr_alloc()`
- `swr_alloc_set_opts`
```cpp
SwrContext* swr_alloc_set_opts(
SwrContext*,
int64_t out_ch_layout, // Output
AVSampleFormat outSampleFormat, // Output
int outSampleRate, // Output
int64_t in_ch_layout, // Input
AVSampleFormat inSampleFormat, // Input
int inSampleRate, // Input
int log_offset, // 一般丟0
void* log_ctx) // 一般丟0
```
- `int swr_init(SwrContext*)`
- `void swr_free(SwrContext**)`
- `swr_convert`
```cpp
int swt_convert(
SwrContext*,
uint8_t** outData,
int outCount, // Sample count for one channel
uint8_t** inData,
int inCount) // Sample count for one channel
```
## Audio 播放
### 會用到的class
#### QAudioFormat
- `setSampleRate`
- `setSampleSize`
- `setChannelCount`
- `setCodec("audio/pcm")`
- `setByteOrder(QAudioFormat::LittleEndian)`
- `setSampleType(QAudioFormat::UnSignedInt)`
#### QAudioOutput
- Constuctor參數就是一個`QAudioFormat`
- `QIODevice* start()`
- `suspend()`
- `resume()`
- `bufferSize()`
- `byteFree()`
- `periodSize()`
#### QIODevice
- `qint64 write(const char* data, qint64 len)`
#### QBuffer
```cpp
QBuffer buffer;
if (buffer.open(QIODevice::ReadWrite)) {
qInfo() << "Buffer Opened!";
QByteArray data("Hello World");
for (int i = 0; i < 5; i++) {
buffer.write(data);
buffer.write("\r\n");
}
//File and device access you may need to flush the data to the device
//buffer.flush()
//Move to the first position
buffer.seek(0);
qInfo() << buffer.readAll(); // retuen a QVariant
qInfo() << "closing the buffer";
//ALWAYS close your device!
buffer.close();
}
```
### Example
```cpp
QAudioFormat audioFormat;
audioFormat.setSampleRate(44100);
audioFormat.setSampleSize(16);
audioFormat.setChannelCount(2);
audioFormat.setCodec("audio/pcm");
audioFormat.setByteOrder(QAudioFormat::LittleEndian);
audioFormat.setSampleType(QAudioFormat::UnSignedInt);
QAudioOutput* audioOutput = new QAudioOutput(audioFormat);
QIODevice* ioDevice = audioOutput->start();
int size = audioOutput->periodSize();
char* buffer = new char[size];
FILE *pFile = fopen("E:\\download\\out.pcm", "rb");
while (!feof(pFile)) {
if (audioOutput->bytesFree() < size) {
QThread::msleep(1);
}
qint64 len = fread(buffer, 1, size, pFile);
if (len <= 0) {
break;
}
ioDevice->write(buffer, len);
}
fclose(pFile);
delete buffer;
buffer = NULL;
```
### 使用FFmpeg decode audio
- [音视频开发之旅（60) -调试分析FFmpeg (解封装部分的）常用结构体](https://zhuanlan.zhihu.com/p/441055685)
## 參考資料
- [mpv.io](https://mpv.io/)
- [《FFmpeg基础知识》](https://opensourcelibs.com/lib/zhangfangtaozft-ffmpeg)
- [ffmpeg - 专题 - 简书](https://www.jianshu.com/c/21a6fafc8ee3)