vault backup: 2025-03-04 11:17:00

20.01. Programming/Python/Polars.md

@@ -0,0 +1,34 @@
+---
+tags: 
+aliases: 
+date: 2024-11-10
+time: 16:57:31
+description:
+---
+
+**可以用來代替[pandas](https://pandas.pydata.org/)**
+
+Now, listen up-the thing is, `Pandas` is great at data exploration and for middle-sized datasets. But people just use it for everything, like it’s some magic solution that’s going to solve every problem in data, and quite frankly, it isn’t. Working with `Pandas` on huge datasets can turn your machine into a sputtering fan engine, and memory overhead just doesn’t make sense for some workflows.
+
+## **Why [pandas](https://pandas.pydata.org/) Is Overrated:**
+
+**Memory Usage:** As `Pandas` operates mainly in-memory, any operation on a large dataset will badly hit performance.
+
+**Limited Scalability:** Scaling with `Pandas` isn’t easy. It was never designed for big data.
+
+## What You Should Use Instead: Polars
+
+`Polars` is an ultra-fast DataFrame library in Rust using Apache Arrow. Optimized for memory efficiency and multithreaded performance, this makes it perfect for when you want to crunch data without heating up your CPU.
+
+```python
+import polars as pl  
+  
+df = pl.read_csv("big_data.csv")  
+filtered_df = df.filter(pl.col("value") > 50)  
+print(filtered_df)
+```
+
+**Why** `**Polars**`**?** It will process data that would bring `Pandas` to its knees, and it handles operations in a fraction of the time. Besides that, it also has lazy evaluation-meaning it is only computing what’s needed.
+
+# 參考來源
+- [5 Overrated Python Libraries (And What You Should Use Instead) | by Abdur Rahman | Nov, 2024 | Python in Plain English](https://python.plainenglish.io/5-overrated-python-libraries-and-what-you-should-use-instead-106bd9ded180)