15 karma · joined January 14, 2021
1). Finance is high dynamic. BloombergGPT retrains LLM using a mixed dataset of finance and general sources is too much expensive (1.3M hours). Lightweight adaptation is highly favorable.
2). Internet-scale finance data (timely updates using an automatic data curation pipeline) is critical. BloombergGPT has privileged data access and API access. A promising alternative is "democratizing Internet-scale finance data".
3). Another key technology is "RLHF (Reinforcement learning from human feedback)", which is missing in BloombergGPT. RLHF enables learning individual preferences (risk-aversion level, investing habits, personalized robo-advisor, etc.)
2). Internet-scale finance data (timely updates using an automatic data curation pipeline) is critical. BloombergGPT has privileged data access and API access. A promising alternative is "democratizing Internet-scale finance data".
3). Another key technology is "RLHF (Reinforcement learning from human feedback)", which is missing in BloombergGPT. RLHF enables learning individual preferences (risk-aversion level, investing habits, personalized robo-advisor, etc.)
FinRL can be found here: https://github.com/AI4Finance-Foundation
There is a previous discussion here: https://news.ycombinator.com/item?id=30819436
[1] Makoviychuk, Viktor, et al. "Isaac Gym: High-Performance GPU Based Physics Simulation For Robot Learning." Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). 2021.
At its release, people commented on Twitter that "it is the MNIST moment for reinforcement learning." And over the past year, I saw several follow-up works and tested NVIDIA's implementations.
For example, a demo by this blog
https://towardsdatascience.com/a-new-era-of-massively-parall...
The question is, does that technique help advance Reinforcement Learning, as expected?
Medium Blog at: https://medium.com/@zx2325/finrl-meta-from-market-environmen...
The ElegantRL library is featured with “elegant” in the following aspects:
Lightweight: core codes have less than 1,000 lines, e.g., tutorial.
Efficient: the performance is comparable with Ray RLlib.
Stable: more stable than Stable Baseline 3.
ElegantRL supports state-of-the-art DRL algorithms, including discrete and continuous ones, and provides user-friendly tutorials in Jupyter notebooks.
The ElegantRL implements DRL algorithms under the Actor-Critic framework, where an Agent (a.k.a, a DRL algorithm) consists of an Actor network and a Critic network. Due to the completeness and simplicity of code structure, users are able to easily customize their own agents.
In quantitative finance, automated trading is essentially making dynamic decisions, namely to decide where to trade, at what price, and what quantity, over a highlystochastic and complex stock market. Taking many complex financialfactors into account, DRL trading agents build a multi-factor model and provide algorithmic trading strategies, which are difficult for human traders
FinRL provides a framework that supports various markets, SOTA DRL algorithms, benchmarks of many quant finance tasks, live trading, etc.