If you're in AWS land you can use Kinesis firehose to S3, which is perfect for Spark. Way more straightforward than any Kafka solution.