curl cannot, AFAIK, do this. People usually suggest using xargs, which is a mediocre substitute because it waits for all the URLs to arrive before invoking curl, giving up any chance at parallelism between the command generating the URLs and the one downloading them.
echo '--url https://google.com/' | curl --config - ds@swann3:~# (for x in {1..100}; do sleep 0.1s; echo $x >&2; echo $x; done) | xargs -L5 echo
1
2
3
4
5
1 2 3 4 5
6
7
8
9
10
6 7 8 9 10
11
12
[... and so on ...]
If you replace echo with something that sleeps you'll see that the pipe doesn't stall waiting on xargs so the process producing the list can keep pushing new items to it as they are found:
ds@swann3:~# (for x in {1..100}; do sleep 0.1s; echo $x >&2; echo $x; done) | xargs --max-lines=5 ./echosleepecho
1
2
3
4
5
starting 1 2 3 4 5
6
7
8
9
10
11
12
13
14
done 1 2 3 4 5
starting 6 7 8 9 10
15
16
17
18
19
[... and so on until ...]
98
99
100
done 46 47 48 49 50
sleeping for 51 52 53 54 55
done 51 52 53 54 55
sleeping for 56 57 58 59 60
[... and so on until xarg's stdin is exhausted]
ds@swann3:~# (for x in {1..100}; do sleep 0.1s; echo $x >&2; echo $x; done) | xargs --max-lines=3 --max-procs=10 ./echosleepecho
1
2
3
sleeping for 1 2 3
4
5
6
sleeping for 4 5 6
7
8
9
sleeping for 7 8 9
10
11
12
sleeping for 10 11 12
done 1 2 3
13
14
15
sleeping for 13 14 15
done 4 5 6
16
[... and so on ...]
I'm not sure what you would do about error handling in all this though, more experimentation necessary there before I'd ever do this in production!
For an artificial example, change
(for x in {1..100}; do sleep 0.1s; echo $x >&2; echo $x; done) | xargs -L5 echo
(for x in {1..100}; do sleep 0.1s; echo $x >&2; echo $x; done) | sort | xargs -L5 echo
----
[1] An unnecessary sort in an ETL process causing the overall wall-clock time to increase significantly
You can specify multiple URLs on the same command in curl so using xargs in this way would do what you ask to an extent (the connection would be dropped and renegotiated at least between each batch) as long as you don't use any options that imply --max-lines=1.
With the --max-procs option you could be requesting multiple files at once which may improve performance over wget -i – though obviously take care doing this against a single site (with wget -i too for that matter) as this can be rather unfriendly (if requesting from multiple hosts this is moot, as is the multiple files-from-one-connection point).