awin/Obsidian-Main
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

vault backup: 2025-03-04 11:17:00

Browse Source
This commit is contained in:
Awin Huang
2025-03-04 11:17:00 +08:00
parent d1e51bfd2f
commit ff12c4f4ca
161 changed files with 1 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
20.01. Programming/Python/Determine Python is running as a script or as a frozen exe.md Normal file
View File

@@ -0,0 +1,14 @@
```python
import os
import sys
config_name = 'myapp.cfg'
# determine if application is a script file or frozen exe
if getattr(sys, 'frozen', False):
application_path = os.path.dirname(sys.executable)
elif __file__:
application_path = os.path.dirname(__file__)
config_path = os.path.join(application_path, config_name)
```

34
20.01. Programming/Python/Plotly.md Normal file
View File

@@ -0,0 +1,34 @@
---
tags:
aliases:
date: 2024-11-10
time: 16:58:43
description:
---
**可以用來代替[Matplotlib](https://matplotlib.org/)**
Yes, `Matplotlib` is classic-it’s virtually the standard to go to when it comes to visualizing data in Python. But to be frank, it feels so much like trying to use an axe for delicate brain surgery, and its syntax? A little verbose, if we’re being honest. If you’re not creating highly customized visualizations, there are better options with a more straightforward syntax.
## Why [Matplotlib](https://matplotlib.org/) is Overrated:
**Clunky syntax**: Even simple charts take an amazingly large number of lines to plot sometimes.
**Outdated default style:** The default style is configurable, but it isn’t exactly inspiring-or, for that matter, particularly readable.
## What You Should Replace It With: Plotly
Where visualization cleanliness and interactivity matter, and definitely don’t want a pile of code, `Plotly` is great. This is especially useful when you have to share visuals fast or within presentations on the web.
```python
import plotly.express as px
df = px.data.iris()
fig = px.scatter(df, x="sepal_width", y="sepal_length", color="species")
fig.show()
```
With `Ploty`, you immediately get interactive graphs with great default visuals. The code is more concise and, by default, includes things like tooltips and zooming.
# 參考來源
- [5 Overrated Python Libraries (And What You Should Use Instead) | by Abdur Rahman | Nov, 2024 | Python in Plain English](https://python.plainenglish.io/5-overrated-python-libraries-and-what-you-should-use-instead-106bd9ded180)

1030
20.01. Programming/Python/Poetry.md Normal file
View File

File diff suppressed because it is too large Load Diff

34
20.01. Programming/Python/Polars.md Normal file
View File

@@ -0,0 +1,34 @@
---
tags:
aliases:
date: 2024-11-10
time: 16:57:31
description:
---
**可以用來代替[pandas](https://pandas.pydata.org/)**
Now, listen up-the thing is, `Pandas` is great at data exploration and for middle-sized datasets. But people just use it for everything, like it’s some magic solution that’s going to solve every problem in data, and quite frankly, it isn’t. Working with `Pandas` on huge datasets can turn your machine into a sputtering fan engine, and memory overhead just doesn’t make sense for some workflows.
## **Why [pandas](https://pandas.pydata.org/) Is Overrated:**
**Memory Usage:** As `Pandas` operates mainly in-memory, any operation on a large dataset will badly hit performance.
**Limited Scalability:** Scaling with `Pandas` isn’t easy. It was never designed for big data.
## What You Should Use Instead: Polars
`Polars` is an ultra-fast DataFrame library in Rust using Apache Arrow. Optimized for memory efficiency and multithreaded performance, this makes it perfect for when you want to crunch data without heating up your CPU.
```python
import polars as pl
df = pl.read_csv("big_data.csv")
filtered_df = df.filter(pl.col("value") > 50)
print(filtered_df)
```
**Why** `**Polars**`**?** It will process data that would bring `Pandas` to its knees, and it handles operations in a fraction of the time. Besides that, it also has lazy evaluation-meaning it is only computing what’s needed.
# 參考來源
- [5 Overrated Python Libraries (And What You Should Use Instead) | by Abdur Rahman | Nov, 2024 | Python in Plain English](https://python.plainenglish.io/5-overrated-python-libraries-and-what-you-should-use-instead-106bd9ded180)

45
20.01. Programming/Python/PyTorch.md Normal file
View File

@@ -0,0 +1,45 @@
---
tags:
aliases:
date: 2024-11-10
time: 17:00:12
description:
---
**可以用來代替[scikit-learn](https://scikit-learn.org/stable/)**
I know, `Scikit-Learn` isn’t supposed to be a deep learning library, but people use it as if it were. It is incredibly handy at quick prototyping and traditional machine learning models, but when it comes to neural networks, it’s just not in the same league as a library designed with tensors in mind.
## Why [scikit-learn](https://scikit-learn.org/stable/) is Overrated:
**No GPU Support:** Deep learning can be life-changing when training on GPUs. However, this is something that is not supported in `Scikit-Learn`.
**Not Optimized for Neural Networks:** `Scikit-learn` wasn’t designed for doing deep learning; using it this way is reactively assured poor results.
## What You Should Use Instead: PyTorch
`PyTorch` is more general and supports GPU. Hence, it’s perfect for deep learning projects. It’s Pythonic-this means for one coming from `Scikit-Learn`, it will feel natural, but with much more power.
import torch
import torch.nn as nn
import torch.optim as optim
# Define a simple model
```python
model = nn.Sequential(
nn.Linear(10, 5),
nn.ReLU(),
nn.Linear(5, 2)
)
```
# Define optimizer and loss
```python
optimizer = optim.SGD(model.parameters(), lr=0.01)
loss_fn = nn.CrossEntropyLoss()
```
If you’re serious about deep learning, you’ll want to use a library worked out for the task at hand-which will save you from such limitations and inefficiencies. You will fine tune models with `PyTorch` and leverage the GPUs to your heart’s content.
# 參考來源
- [5 Overrated Python Libraries (And What You Should Use Instead) | by Abdur Rahman | Nov, 2024 | Python in Plain English](https://python.plainenglish.io/5-overrated-python-libraries-and-what-you-should-use-instead-106bd9ded180)

173
20.01. Programming/Python/Write a Package.md Normal file
View File

@@ -0,0 +1,173 @@
---
tags:
aliases:
date: 2025-03-02
time: 20:53:47
description:
---
# Project Structure
```
mlpredictor/
│
├── mlpredictor/
│ ├── __init__.py
│ ├── model.py
│
├── tests/
│ ├── test_model.py
│
├── LICENSE
├── README.md
├── pyproject.toml
└── .gitignore
```
## Content of `setup.py`
```
from setuptools import setup
setup(
name='mypackage',
version='0.1',
packages=['mypackage'],
install_requires=['requests'])
```
## Content of `pyproject.toml`
```toml
[build-system]
requires = ["setuptools>=42", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "mlpredictor"
version = "0.1.0"
description = "A simple machine learning package using scikit-learn"
authors = [
{name = "Ebrahim", email = "ebimsv0501@gmail.com"}
]
license = {text = "MIT"}
readme = "README.md"
requires-python = ">=3.6"
dependencies = [
"scikit-learn>=1.0",
]
[project.urls]
"Homepage" = "https://github.com/ebimsv/mlpredictor"
```
- **[build-system]**: Specifies the build system requirements (i.e., using `setuptools` and `wheel`).
- **[project]**: Contains metadata about the package, like name, version, description, and dependencies.
## Content of `README.md`
<pre><code>
# MLPredictor
MLPredictor is a simple machine learning package that trains a RandomForest model using the Iris dataset and enables users to make predictions. The package is built using `scikit-learn` and is intended as a demonstration of packaging Python machine learning projects for distribution.
## Features
- Train a RandomForestClassifier on the Iris dataset.
- Make predictions on new data after training.
- Save and load trained models.
## Installation
You can install the package via **PyPI** or from **source**.
### Install from PyPI
```bash
pip install mlpredictor
```
### Install from Source (GitHub)
```bash
git clone https://github.com/ebimsv/mlpredictor.git
cd mlpredictor
pip install .
```
## Usage
After installation, you can use `MLPredictor` to train a model and make predictions.
### Example: Training and Making Predictions
```python
from mlpredictor import MLPredictor
# Initialize the predictor
predictor = MLPredictor()
# Train the model on the Iris dataset
predictor.train()
# Make a prediction on a sample input
sample_input = [5.1, 3.5, 1.4, 0.2]
prediction = predictor.predict(sample_input)
print(f"Predicted class: {prediction}")
```
</code></pre>
## Content of `.gitignore`
```
*.pyc
__pycache__/
*.pkl
dist/
build/
```
# Test
## Content of `tests/test_model.py`
```python
import pytest
from mlpredictor import MLPredictor
def test_train_and_predict():
model = MLPredictor()
model.train()
result = model.predict([5.1, 3.5, 1.4, 0.2])
assert len(result) == 1
if __name__ == "__main__":
pytest.main()
```
## Run test
```bash
pytest tests
```
# Install
## Test locally
```
pip install .
```
## Publish on PyPI
1. **Install 'Twine' and 'build'**:
```
pip install twine build
```
2. **Build the Package**:
```
python -m build
```
3. **Upload to PyPI**
```
twine upload dist/*
```
# 參考來源
- [Building Python Packages. A Comprehensive Guide to setup.py and… | by Ebrahim Mousavi | Medium](https://medium.com/@ebimsv/building-python-packages-07fbfbb959a9)

56
20.01. Programming/Python/argparse.ArgumentParser.md Normal file
View File

@@ -0,0 +1,56 @@
一個範例：
```python
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-u", "--update_tool", default=WIN_FW_UPDATER_PATH, help="The path of win_fw_updater.exe.")
parser.add_argument("-f", "--firmware", required=True, help="The path of ITB file.")
parser.add_argument("-w", "--waittime_update_firmware", default=600, type=int, help="Wait time for update the firmware.")
parser.add_argument("-g", "--ignore_check_file_path", action='store_true', help="Skip check the existence of file.")
args = parser.parse_args()
```
#### 要求user一定要設定的參數
使用`required=True`
例如：
```python
parser.add_argument("-f", "--firmware", required=True, help="The path of ITB file.")
```
如果使用者沒有下`-f`（或者`--firmware=XXX`）就會報錯，如下：
```bash
FwUpdateCheck.py: error: the following arguments are required: -f/--firmware
```
#### 有設定才會產生的參數
使用`action='store_true'`或`action='store_false'`
例如：
```python
parser.add_argument("-g", "--ignore_check_file_path", action='store_true', help="Skip check the existence of file.")
```
當使用者沒有設置`-g`時，`args.ignore_check_file_path`為`False`，當設置時，`args.ignore_check_file_path`為`True`。
#### 使用預設值
例如：
```python
parser.add_argument("-u", "--update_tool", default="C:\\tool.exe", help="The path of win_fw_updater.exe.")
```
用`default=<Something>`來設定參數的預設值，上面的例子中，`args.update_tool`的預設值為`C:\tool.exe`。
另外可以用`type=<Object type>`來指定預設值的型別。例如：
```python
parser.add_argument("-n", "--number", default=50, type=int, help="Assign a number")
```
上例中，`args.number`的預設值是50，型別是`int`，所以可以直接運算，不需要再經過`int(args.number)`這樣的轉換。
#### 限制使用者的選擇
Example:
```python
parser.add_argument('move', choices=['rock', 'paper', 'scissors'])
```
使用`choices=<list>`來限定輸入的選項，上例中，使用者只能輸入'rock'、'paper'、'scissors'這三個字串中的其中一個，否則會報錯：
```bash
error: argument move: invalid choice: 'fire' (choose from 'rock', 'paper', 'scissors')
```
-----
- https://docs.python.org/zh-tw/3/library/argparse.html

5
20.01. Programming/Python/choices().md Normal file
View File

@@ -0,0 +1,5 @@
從list中選出n個項目，有可能重複：
```python
import random
random.choices(seq, n)
```

48
20.01. Programming/Python/decorator.md Normal file
View File

@@ -0,0 +1,48 @@
## 在decorator內取得function的default argument與class member
```python
import sys
import inspect
from functools import wraps
def exampleDecorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
print(f"Decorator: call by {func.__name__}")
def get_default_args(func):
signature = inspect.signature(func)
return {
k: v.default for k, v in signature.parameters.items() if v.default is not inspect.Parameter.empty
}
## Get default
defaultKwargs = get_default_args(func)
defaultKwargs.update(kwargs)
print(f"Decorator: args = {args}, kwargs = {kwargs}, defaultKwargs = {defaultKwargs}")
objectInstance = args[0]
if hasattr(objectInstance, 'defaultArg1'):
print(f'objectInstance has defaultArg1, a.defaultArg1({type(objectInstance.defaultArg1)}) = {objectInstance.defaultArg1}')
if objectInstance.defaultArg1:
## Do something here
print("Decorator: some message...")
else:
print('objectInstance does not have defaultArg1')
return func(*args, **kwargs)
return wrapper
class ExampleClass():
def __init__(self, defaultArg1=True, defaultArg2="SomeString"):
self.defaultArg1 = defaultArg1
self.defaultArg2 = defaultArg2
print(f'self.defaultArg1 = {self.defaultArg1}, self.defaultArg2 = {self.defaultArg2}')
@exampleDecorator
def run(self, arg1=1, arg2=2):
print(f"ExampleClass.run(), arg1 = {arg1}, arg2 = {arg2}")
example = ExampleClass()
example.run()
```

36
20.01. Programming/Python/httpx.md Normal file
View File

@@ -0,0 +1,36 @@
---
tags:
aliases:
date: 2024-11-10
time: 16:54:12
description:
---
**可以用來代替[requests](https://pypi.org/project/requests/)**
## **Why [requests](https://pypi.org/project/requests/) is Overrated:**
**Blocking IO:** `Requests` is synchronous, which means each call waits for the previous call to finish. This is less than ideal when working with I/O-bound programs.
**Heavy:** It’s got loads of convenience baked in, but it does have a cost in terms of speed and memory footprint. Not a big deal on a simple script, but on larger systems this can be a resource hog.
## **What You Should Instead Use:** `httpx`
For parallel processing of requests, `httpx`provides a similar API but with asynchronous support. So, if you make many API calls, it’ll save you some time and resources because it will process those requests concurrently.
```python
import httpx
async def fetch_data(url):
async with httpx.AsyncClient() as client:
response = await client.get(url)
return response.json()
# Simple and non-blocking
data = fetch_data("https://api.example.com/data")
```
> **Pro Tip:** Asynchronous requests can reduce the processing time by a great amount if the task at hand is web scraping or ingesting data from somewhere.
# 參考來源
- [5 Overrated Python Libraries (And What You Should Use Instead) | by Abdur Rahman | Nov, 2024 | Python in Plain English](https://python.plainenglish.io/5-overrated-python-libraries-and-what-you-should-use-instead-106bd9ded180)

4
20.01. Programming/Python/log.md Normal file
View File

@@ -0,0 +1,4 @@
- [Python 中的 Log 利器：使用 logging 模組來整理 print 訊息 - zhung to be lazy…](https://zhung.com.tw/article/python%E4%B8%AD%E7%9A%84log%E5%88%A9%E5%99%A8-%E4%BD%BF%E7%94%A8logging%E6%A8%A1%E7%B5%84%E4%BE%86%E6%95%B4%E7%90%86print%E8%A8%8A%E6%81%AF/)\
- [[Python] logging 教學](https://zwindr.blogspot.com/2016/08/python-logging.html)
- [How can I color Python logging output? - Stack Overflow](https://stackoverflow.com/questions/384076/how-can-i-color-python-logging-output)
- [logging：如何使用 logging 紀錄事件 - Andrew Li](https://orcahmlee.github.io/python/python-logging/)

126
20.01. Programming/Python/logging.md Normal file
View File

@@ -0,0 +1,126 @@
### 準備
```
import logging
```
### logging level
| level | level number | funtion |
|:---------|:-------------|:---------------------|
| NOTSET | 0 | |
| DEBUG | 10 | `logging.debug()` |
| INFO | 20 | `logging.info()` |
| WARNING | 30 | `logging.warning()` |
| ERROR | 40 | `logging.error()` |
| CRITICAL | 50 | `logging.critical()` |
```
import logging
LOG_FORMAT = '%(asctime)s %(levelname)s: %(message)s'
LOG_FILENAME = 'C:\\RobotRun\\Output\\RobotRunDocUpdater.log'
logging.basicConfig(level=logging.INFO, filename=LOG_FILENAME, filemode='a', format=LOG_FORMAT)
logging.info('logging start')
```
### Print Exception
`logging` 模組也提供可以紀錄完整的堆疊追蹤 (stack traces)，若在 `logging.error()` 加上 `exc_info` 參數，並將該參數設為 `True`，就可以紀錄 Exception，如下：
```python
import logging
try:
x = 5 / 0
except:
logging.error("Catch an exception.", exc_info=True)
```
也可以使用`logging.exception("Catch an exception.")`，效果跟`logging.error("Catch an exception.", exc_info=True)`一樣。
### 自訂 logging 輸出格式
預設的訊息輸出格式只有 `levelname`、`name`、`message`，下面是其他相關的資訊：
| 格式化字串 | 說明 |
|:------------------|:---------------------------------------------------------------------|
| `%(asctime)s` | 日期時間, 格式為 `YYYY-MM-DD HH:mm:SS,ms`，例如：2018-12-13 17:20:30,567 |
| `%(filename)s` | 模組檔名 |
| `%(funcName)s` | 函數名稱 |
| `%(levelname)s` | 日誌的等級名稱 |
| `%(levelno)s` | 日誌的等級數值 |
| `%(lineno)d` | 呼叫日誌函數所在的行數 |
| `%(message)s` | 訊息 |
| `%(module)s` | 模組名稱 |
| `%(name)s` | logger 的名稱 |
| `%(pathname)s` | 檔案的完整路徑 (如果可用) |
| `%(process)d` | process ID (如果可用) |
| `%(thread)d` | 執行緒 ID (如果可用) |
| `%(threradName)s` | 執行緒名稱 |
例：
```python
FORMAT = '%(asctime)s %(levelname)s: %(message)s'
logging.basicConfig(level=logging.DEBUG, format=FORMAT)
logging.debug('debug message') --> 2018-12-13 17:40:34,604 DEBUG: debug message
```
### 儲存log
只要在 `logging.basicConfig()` 內的 `filename` 參數設定要儲存的日誌檔名，就可以將 logging 儲存：
```python
import logging
FORMAT = '%(asctime)s %(levelname)s: %(message)s'
logging.basicConfig(level=logging.DEBUG, filename='myLog.log', filemode='w', format=FORMAT)
logging.debug('debug message')
```
預設 `filemode` 參數是設為 `a`，代表 append (附加) 的意思，每次執行程式時，Logging 會將新的訊息加在舊的訊息後面，不會覆蓋舊的訊息。若要改成新訊息覆蓋就訊息，那可以將 `filemode` 參數設為 `w`，代表 write 的意思。
### 儲存log也輸出到console
`logging`有4個主要module：
- Logger：暴露了應用程式程式碼能直接使用的介面。
- Handler：將（記錄器產生的）日誌記錄傳送至合適的目的地。
- Filter：提供了更好的粒度控制，它可以決定輸出哪些日誌記錄。
- Formatter：指明瞭最終輸出中日誌記錄的佈局。
#### Handler
其中`Handlers`有以下幾類：
1. `logging.StreamHandler` -> 控制檯輸出
使用這個Handler可以向類似與`sys.stdout`或者`sys.stderr`的任何檔案物件(file object)輸出資訊。
它的建構函式是： `StreamHandler([strm])` 其中`strm`引數是一個檔案物件。預設是`sys.stderr`。
2. `logging.FileHandler` -> 檔案輸出
和StreamHandler類似，用於向一個檔案輸出日誌資訊。不過`FileHandler`會幫你開啟這個檔案。
它的建構函式是：`FileHandler(filename[,mode])` filename是檔名，必須指定一個檔名。 `mode`是檔案的開啟方式。預設是`'a'`，即新增到檔案末端。
3. `logging.handlers.RotatingFileHandler` -> 按照大小自動分割日誌檔案，一旦達到指定的大小重新生成檔案
這個Handler類似於上面的`FileHandler`，但是它可以管理檔案大小。當檔案達到一定大小之後，它會自動將當前日誌檔案改名，然後建立一個新的同名日誌檔案繼續輸出。比如日誌檔案是chat.log。當chat.log達到指定的大小之後，`RotatingFileHandler`自動把 檔案改名為chat.log.1。不過，如果chat.log.1已經存在，會先把chat.log.1重新命名為chat.log.2。
最後重新建立 chat.log，繼續輸出日誌資訊。它的建構函式是：`RotatingFileHandler(filename[, mode[, maxBytes[, backupCount]]])`，其中`filename`和`mode`兩個引數和FileHandler一樣。`maxBytes`用於指定日誌檔案的最大檔案大小。如果maxBytes為0，意味著日誌檔案可以無限大，這時上面描述的重新命名過程就不會發生。 `backupCount`用於指定保留的備份檔案的個數。比如，如果指定為2，當上面描述的重新命名過程發生時，原有的chat.log.2並不會被更名，而是被刪除。
4. `logging.handlers.TimedRotatingFileHandler` -> 按照時間自動分割日誌檔案
這個Handler和`RotatingFileHandler`類似，不過，它沒有通過判斷檔案大小來決定何時重新建立日誌檔案，而是間隔一定時間就自動建立新的日誌檔案。重新命名的過程與`RotatingFileHandler`類似，不過新的檔案不是附加數字，而是當前時間。它的建構函式是：`TimedRotatingFileHandler( filename [,when [,interval [,backupCount]]])`，其中`filename`引數和`backupCount`引數和`RotatingFileHandler`具有相同的意義。`interval`是時間間隔。 `when`引數是一個字串。表示時間間隔的單位，不區分大小寫。它有以下取值： S 秒 M 分 H 小時 D 天 W 每星期（`interval==0`時代表星期一） midnight 每天凌晨。
#### Formatters
Formatters預設的時間格式為`%Y-%m-%d %H:%M:%S`
#### Example
新增2個handler，一個輸出到螢幕上，一個寫到檔案裡。寫到檔案裡的那個handler必須是`logging.handlers.RotatingFileHandler`，超過1MB時會自動分割。
```python
import logging
import logging.handlers
logger = logging.getLogger(filename) # filename就是你要存log的檔名
shell_print = logging.StreamHandler() # 往螢幕上輸出
shell_print.setFormatter(format_str) # 設定螢幕上顯示的格式
file_print = logging.handlers.RotatingFileHandler(
filename=filename,
mode='a',
maxBytes=1024*1024,
backupCount=backCount,
encoding='utf-8')
file_print.setFormatter(format_str) # 設定檔案裡寫入的格式
logger.addHandler(sh) # 把物件加到logger裡
logger.addHandler(th)
```
-----
參考：
- [Python - 日誌 (logging) 模組](https://titangene.github.io/article/python-logging.html)
- [`logging` — Logging facility for Python](https://docs.python.org/3/library/logging.html#module-logging "logging: Flexible event logging system for applications.")

72
20.01. Programming/Python/matplotlib.md Normal file
View File

@@ -0,0 +1,72 @@
## 基本折線圖
給2個list，一個 x，一個 y
```python
plt.clf() # 把圖清掉，變空白
plt.plot(xList, yList)
```
## XY軸標籤
```python
plt.xlabel(
'Focus setting', # 標籤
fontsize=15, # 字型大小
labelpad=10, # 標籤留白
color='red', # 文字顏色
rotation=90, # 文字旋轉角度
fontweight='bold', # 粗體
)
```
## 不要顯示軸的刻線
```python
plt.gca().axes.get_xaxis().set_visible(False)
```
## 畫2張圖
```python
figure, axis = plt.subplots(2, 2)
```
用 `plt.subplots()` 來指定要畫幾張圖，第一個參數是要有幾個 row，第二個參數是要有幾個 column。
`axis` 會是一個 array，可以用類似座標的方式來控制你要的圖，例如：
```python
axis[0, 0].set_title("Sine Function")
axis[0, 1].set_title("Cosine Function")
```
`figure` 則是指外圍的大圖。
## 畫2條線
```python
plt.plot(x, y1, label='sine curve',color='b')
plt.plot(x, y2, label='cosine curve',color='r')
```
## 畫大圖
```python
figure(figsize=(12, 9), dpi=120)
```
`12`跟`9`指的是英吋，`dpi`是每英吋幾個點，所以就是`12*120`跟`9*120`，也就是`1440x1080`。
## 存檔
```python
plt.savefig(f'plot_{folder}.png')
```
## 註記（annotation）
```python
ax = plt.gca()
ax.annotate(
'local max', # 註記文字
xy=(xmax, ymax), # 點的座標
xytext=(xmax, ymax + 5), # 文字的座標
arrowprops=dict( # 箭頭的屬性
facecolor='black', # 顏色：黑色
shrink=0.05), #
)
```
官方說明：[matplotlib.axes.Axes.annotate — Matplotlib 3.7.1](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.annotate.html)
## 在X軸上畫一個範圍
```python
plt.gca().axvspan(startXPos, endXPos, alpha=0.2, color='red')
```

61
20.01. Programming/Python/opencv.md Normal file
View File

@@ -0,0 +1,61 @@
### 將camera包裝成class
```python
class CameraCv(object):
def __init__(self, videoSource=0):
self.videoSource = videoSource
self.camera = None
self.cameraWidth = 0
self.cameraHeight = 0
self.cameraPreviewThreadHandle = None
self.cameraPreviewThreadStopEvent = threading.Event()
self.lastframeRGB = None
self.latestFrame = None
def start(self):
print("Open Camera")
self.camera = cv2.VideoCapture(self.videoSource, cv2.CAP_DSHOW)
if not self.camera.isOpened():
raise ValueError("Unable to open video source {}".format(self.videoSource))
# Get video source width and height
self.cameraWidth = self.camera.get(cv2.CAP_PROP_FRAME_WIDTH)
self.cameraHeight = self.camera.get(cv2.CAP_PROP_FRAME_HEIGHT)
self.cameraPreviewThreadStopEvent.clear()
self.cameraPreviewThreadHandle = threading.Thread(target=self.collectFrame, daemon=True, args=())
self.cameraPreviewThreadHandle.start()
def stop(self):
print("Close Camera")
self.cameraPreviewThreadStopEvent.set()
if self.camera.isOpened():
self.camera.release()
cv2.destroyAllWindows()
def collectFrame(self):
while True:
ret, frame = self.camera.read()
if ret:
# Return a boolean success flag and the current frame converted to BGR
self.lastframeRGB = frame
self.latestFrame = ImageTk.PhotoImage(image=Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)))
if self.cameraPreviewThreadStopEvent.is_set():
break
time.sleep(0.016)
def draw(self, container):
if self.latestFrame is not None:
container.imgtk = self.latestFrame
container.configure(image=self.latestFrame)
def read(self):
return self.camera.read()
def getLastFrameRgb(self):
return self.lastframeRGB
def saveFrame(self, filepath):
cv2.imwrite(filepath, self.getLastFrameRgb())
```

7
20.01. Programming/Python/sample().md Normal file
View File

@@ -0,0 +1,7 @@
從一個list中，選出n個不重複的項目：
```python
import random
random.sample(seq, n)
```
不像 [[choices()]] 是會重複的。

35
20.01. Programming/Python/selectolax.md Normal file
View File

@@ -0,0 +1,35 @@
---
tags:
aliases:
date: 2024-11-10
time: 16:55:41
description:
---
**可以用來代替[Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)**
## **Why [Beautiful Soup Documentation](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) is Overrated:**
**Speed:** Not very fast, when the size of a document is very big.
**Thread blocking:** Much like `Requests` itself, it is not designed with async in mind, which certainly makes it ill-suited for scraping dynamic websites.
## **Instead What you should use:** `selectolax`
`selectolax` is a less famous library that uses `libxml2` for better performance and with less memory consumption.
```python
from selectolax.parser import HTMLParser
html_content = "<html><body><p>Test</p></body></html>"
tree = HTMLParser(html_content)
text = tree.css("p")[0].text()
print(text) # Output: Test
```
As it will turn out, by using `Selectolax`, you retain the same HTML parsing capabilities but with much-enhanced speed, making it ideal for web scraping tasks that are quite data-intensive.
> **“Do not fall in love with the tool; rather, fall in love with the outcome.” Choosing the proper tool is half the battle.**
# 參考來源
- [5 Overrated Python Libraries (And What You Should Use Instead) | by Abdur Rahman | Nov, 2024 | Python in Plain English](https://python.plainenglish.io/5-overrated-python-libraries-and-what-you-should-use-instead-106bd9ded180)

49
20.01. Programming/Python/subprocess.md Normal file
View File

@@ -0,0 +1,49 @@
### subprocess.Popen
```python
import subprocess
process = subprocess.Popen(['echo', 'More output'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
stdout, stderr
```
Input arguments is a list.
Notice `communicate()` will **block** until process was finished.
And the output string `stdout` and `stderr` is of type `byte`. You can convert the output to `string` by:
```python
new_string = stdout.decode('utf-8')
```
or use `universal_newlines=True` in `subprocess.Popen()`. Example:
```python
process = subprocess.Popen(['ping', '-c 4', 'python.org'],
stdout=subprocess.PIPE,
universal_newlines=True)
```
The `.poll()` will return the exit code of process. If process is still running. `.poll()` will return `None`. Example:
```python
process = subprocess.Popen(['ping', '-c 4', 'python.org'], stdout=subprocess.PIPE, universal_newlines=True)
while True:
output = process.stdout.readline()
print(output.strip())
# Do something else
return_code = process.poll()
if return_code is not None:
print('RETURN CODE', return_code)
# Process has finished, read rest of the output
for output in process.stdout.readlines():
print(output.strip())
break
```
-----
參考：
- [docs.python.org: `subprocess.Popen`](https://docs.python.org/3/library/subprocess.html#subprocess.Popen)
### subprocess.run
`subprocess.run()`跟`subprocess.Popen()`是一樣的行為，差別是`subprocess.run()`會在process執行完畢之後才return，也就是說流程會被block住。
`subprocess.run()`會回傳一個型別是`subprocess.CompletedProcess`的object.
-----
參考：
- [docs.python.org: _class_ `subprocess.CompletedProcess`](https://docs.python.org/3/library/subprocess.html#subprocess.CompletedProcess)

2
20.01. Programming/Python/threading.md Normal file
View File

@@ -0,0 +1,2 @@
- [Python 多執行緒 threading 模組平行化程式設計教學 - G. T. Wang](https://blog.gtwang.org/programming/python-threading-multithreaded-programming-tutorial/)
- [Python — 多線程. 介紹 | by Jease | Jease隨筆 | Medium](https://medium.com/jeasee%E9%9A%A8%E7%AD%86/python-%E5%A4%9A%E7%B7%9A%E7%A8%8B-eb36272e604b)

96
20.01. Programming/Python/tkinter.md Normal file
View File

@@ -0,0 +1,96 @@
### 把[[matplotlib]]包裝成獨立視窗
```python
class Plot2D(Frame):
def __init__(self, parent, dataCollector, **kwargs):
Frame.__init__(self, parent.mainWindow, **kwargs)
self.parent = parent
self.mainWindows = Toplevel(parent.mainWindow)
self.mainWindows.title("AF State")
self.figure = plt.Figure(figsize=(9,5), dpi=100)
self.figure.suptitle('AF value plot', fontsize=16)
self.ax = self.figure.add_subplot(111)
self.canvas = FigureCanvasTkAgg(self.figure, master=self.mainWindows)
self.canvas.get_tk_widget().pack(fill='both')
self.axline = None
self.dataCollector = dataCollector
self.dataCollector.start()
def close(self):
print("Plot2D close")
self.mainWindows.destroy()
self.dataCollector.stop()
self.dataCollector = None
def draw(self):
if self.dataCollector:
datax, datay = self.dataCollector.getPlotData()
self.ax.clear()
self.ax.set_xlabel('Last {} datas'.format(self.dataCollector.getDataLength()))
self.axline, = self.ax.plot(datax, datay)
self.canvas.draw()
def getWindow(self):
return self.mainWindows
def getLastData(self):
return self.dataCollector.getLastData()
```
其中這一行：
```python
self.mainWindows = Toplevel(parent.mainWindow)
```
是用來開一個新的視窗，其中的`parent.mainWindow`就是用`tk.TK()`所產生出來的root。
因為需要一直更新資料，所以需要的一個`DataCollector`來提供資料，`DataCollector`會提供畫圖需要的list，如：
```python
datax, datay = self.dataCollector.getPlotData()
```
`DataCollector`的定義如下：
```python
class AfStateCollector(threading.Thread):
def __init__(self, dataLength=100, pollingInterval=0.033):
threading.Thread.__init__(self)
self.dataLength = dataLength
self.pollingInterval = pollingInterval
self.stopEvent = threading.Event()
self.data = []
self.xdata = []
def run(self):
while True:
if self.stopEvent.is_set():
break
afValue = self.readAf()
self.data.append(afValue)
self.xdata.append(len(self.xdata))
if len(self.data) > self.dataLength:
self.data = self.data[-self.dataLength:]
self.xdata = list(range(self.dataLength))
# print(f'afValue = {afValue}')
time.sleep(self.pollingInterval)
print("AfStateCollector stopped.")
def readAf(self):
ReadTestXUreg_cmd = "lvreg testxu read 10"
ReadTestXUreg_cmd_process = subprocess.Popen(ReadTestXUreg_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
outstring, err = ReadTestXUreg_cmd_process.communicate()
outstring = outstring.strip().decode('utf-8')
outstring = int(outstring, 16)
outstring_H = (outstring & 0xFF00) / 256
outstring_L = outstring & 0xFF
outAFStat = int(outstring_L * 256 + outstring_H)
return outAFStat
```
- [Python GUI之tkinter視窗視窗教程大集合（看這篇就夠了） - IT閱讀](https://www.itread01.com/content/1547705544.html)
- [【Python】改善 VideoCapture 的影像延遲 | 夏恩的程式筆記 - 點部落](https://dotblogs.com.tw/shaynling/2017/12/28/091936)
- [Displaying a video feed with OpenCV and Tkinter - PyImageSearch](https://www.pyimagesearch.com/2016/05/30/displaying-a-video-feed-with-opencv-and-tkinter/)

96
20.01. Programming/Python/檢測工具.md Normal file
View File

@@ -0,0 +1,96 @@
## 單元測試
### [pytest](https://docs.pytest.org/en/7.1.x/)
Pytest 不僅可以幫助我們運行測試，還可以幫助我們配置如何運行它們、運行哪些文件等等……Pytest 有一個配置文件 `pytest.ini`，您可以在其中描述它的配置，例如哪個版本應該是 Pytest 或者哪些是測試文件，例如下列。
```ini
# pytet.ini
[pytest]
minversion = 6.0
addopts = -ra -q — cov=src — cov-report=html
python_files = test_*.py
```
### [tox](https://tox.wiki/en/latest/)
Tox 是一個通用的virtualenv管理和測試命令行工具。
使用不同的 Python 版本和解釋器檢查您的包是否正確安裝
在每個環境中運行您的測試，配置您選擇的測試工具
作為持續集成服務器的前端，大大減少樣板文件並合併 CI 和基於 shell 的測試。
Tox 也有它的配置文件。
```ini
[tox]
isolated_build = True
envlist = py{38}
[testenv]
usedevelop = true
deps = -r src/requirements_dev.txt
```
## 程式檢查工具
用來檢查程式是否符合coding style、PEP8之類的規範
### [pylint](https://github.com/PyCQA/pylint)
Pylint config: create `.pylintrc` file
```
[MESSAGES CONTROL]
disable=
missing-docstring,
too-few-public-methods[REPORTS]
output-format=colorized
files-output=no
reports=no
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
```
### [flake8](https://github.com/pycqa/flake8)
Flake8 config: create `.flake8` file
```
[flake8]
ignore = E203, E266, E501, W503, F403, F401, E402
max-line-length = 120
max-complexity = 18
select = B,C,E,F,W,T4,B9
exclude =
.git,
tests
```
### [mypy](http://www.mypy-lang.org/)
## Git hook
### pre-commit
Pre-commit 是一個創建 git hook的framework，以確保您的代碼編寫與您定義的代碼樣式相對應。
它會掃描您的原始碼並運行您將在預提交配置文件中定義的所有檢查器：`.pre-commit-config.yaml`
```
repos:
- repo: 'https://gitlab.com/pycqa/flake8'
rev: 3.8.2
hooks:
- id: flake8
name: Style Guide Enforcement (flake8)
args:
- '--max-line-length=120'
- repo: 'https://github.com/pre-commit/mirrors-mypy'
rev: v0.720
hooks:
- id: mypy
name: Optional Static Typing for Python (mypy)
```
## 漏洞檢查
### [SonarQube](https://www.sonarqube.org/)
有很多用於漏洞掃描的工具，但我們將看看[Sonarqube](https://www.sonarqube.org/)。Sonarqube 是用於代碼質量和安全掃描的開源強大工具，是該行業的領先工具之一。
更多在[官方文檔](https://docs.sonarqube.org/latest/)中。
您可以使用 Docker 映像設置本地 Sonarqube 服務器並定義`sonar-project.properties`
```
# must be unique in a given SonarQube instance
sonar.projectKey=python_app_blueprint
# --- optional properties ---
# defaults to project key
#sonar.projectName=My project
# defaults to 'not provided'
#sonar.projectVersion=1.0
# Path is relative to the sonar-project.properties file. Defaults to .
#sonar.sources=.
# Encoding of the source code. Default is default system encoding
#sonar.sourceEncoding=UTF-8
```