This is a previous extension of his work to publicly release the FLAN collection: https://twitter.com/EnricoShippole/status/166175616624899686...
The dialog data is available on huggingface to download. It was processed at an extended context length of 8192. It contains relevant metadata such as Inputs, Targets, Task Source, and Task Name.
An additional FLAN Dialog submix dataset was also preprocessed for causal language modeling, fixing different encoding issues, and is available on huggingface to download: https://huggingface.co/datasets/conceptofmind/flan_dialog_su...
You can find the extended FLAN Zero-shot-only causal language modeling dataset on huggingface here: https://huggingface.co/datasets/conceptofmind/flan_dialog_zs...
You can find the extended FLAN Few-shot-only causal language modeling dataset on huggingface here:https://huggingface.co/datasets/conceptofmind/flan_dialog_fs...
Our work on an open reproduction of FLAN V2 and this conversational data release is all thanks to the generous sponsorship by CarperAI and StabilityAI.
You can find more information about CarperAI here: https://carper.ai/
And StabilityAI here: https://stability.ai/
A big thank you to Jason Phang from EleutherAI for helping build the dataset as well. You can find his Twitter here: https://twitter.com/zhansheng
You can find out more about Shayne Longrpe's work on FLAN here:https://twitter.com/ShayneRedford/status/1661734033720762368
You can find the original FLAN repository here: https://github.com/google-research/FLAN/tree/main/flan/v2#do...
Be sure to also read through Shayne Longpre's paper on the FLAN collection to get better insight into how the data was created:https://arxiv.org/abs/2301.13688
You can check out Shayne Longpre's new paper on building pre-training datasets here:https://twitter.com/ShayneRedford/status/1660670374206652419
This is not an official Google or StabilityAI product.
If you have any questions about the data be sure to reach out and ask! I will try to respond promptly.