BERT + LL would do just fine I imagine. There's the IMDB review dataset for sentiment analysis (50k reviews) which I imagine might be a fine dataset for transfer learning on if you can't find anything better.