We have to invite Shawn Mendes to read this post.
314 karma · joined September 28, 2018
ASF Member. Apache OpenDAL PMC Chair.
My VISION: access data freely across services by any method.
Please note that he mainly focuses on Sichuan cuisine, which is a bit spicy.
The Apache OpenDAL community is using PureGo and libffi to create Go bindings for OpenDAL.
- Multiple links in the same post make it difficult to organize discussions and thoughts.
- Monthly posts require extra effort to maintain and update, which didn't align with our initial goal of sharing more casually and recording our thoughts in real time.
- Modern static site generators support maintaining archive pages by month, so we no longer need to do it manually.
Hoping those ideas make sense to you.
For example:
use std::io;
use std::io::SeekFrom;
use futures::io::AsyncReadExt;
use opendal::Operator;
use opendal::Result;
async fn test(op: Operator) -> io::Result<()> {
let mut r = op
.reader("hello.txt")
.await?
// Only access range (0, 8*1024*1024 )
.into_futures_async_read(0..8*1024*1024)
.await?;
// Seek to 1024.
r.seek(SeekFrom::Start(1024)).await?;
let mut bs = Vec::new();
r.read_to_end(&mut bs).await?;
Ok(())
}The root cause is not about page alignment. In fact, all allocators are aligned.
The root cause is AMD CPU didn't implement FSRM correctly while copying data from 0x1000 * n ~ 0x1000 * n + 0x10.
> Other questions never answered, why did pyo3 add so much overhead? it was over half the difference between the two.
OpenDAL Python Binding v0.42 does have many place to improve, like we can alloc the buffer in advance or using `read_buf` into uninit vec. I skipped this part since they are not the root cause.