Spark offers a fluent DSL with infix operators for data transforms, but after a company-imposed auto formatter got done mangling it I was only allowed to check in a wall of method chaining that none of my teammates would write.
things
.where($"key" === lit(key))
.select("value")
.as[String]
.repartition(N1, $"value")
.flatMap(new ThingIterator(_))
.repartition(N2)
.write
.parquet(
s"s3a://bucket/things/key=$key")
As for naming, I see stuff like “every method must take one arg whose name ends with Request” even when envelope or notification would make more sense.