Stavros from TileDB, Inc. here: Arrow employs a columnar format to store objects like data frames. TileDB is also columnar (so you can sub-select attributes and perform analytics with very similar optimizations to Arrow), but the first-class citizens are (the more general) dense and sparse multi-dimensional arrays. Moreover, TileDB focuses on optimizing for the persistent storage backend, so that it can handle out-of-core analytics on massive datasets that cannot fit in main memory (e.g., genomics), or offer the same performance using less RAM (leading to cost savings in the cloud). Nevertheless, we are quite fond of Arrow, so we hope to work together at some point and integrate as seamlessly as possible.
What do you mean by out-of-core?
In-core algorithms require the entire array(s) in main memory to perform some computation. Out-of-core algorithms are typically block-based and stream the array blocks from persistent storage to main memory on demand, working on parts of the array(s) at a time, thus minimizing the memory requirements. If this is done asynchronously and carefully, for some CPU-bound algorithms you may be able to completely hide the storage-to-memory cost, thus saving memory without losing performance.
Arrow is a columnar format. TileDB does clustering across multiple dimensions based. Arrow is great when you request specific columns. TileDb is great when you want rectangular sections of the space (think getting tiles from a map).