Currently, Firefox has a system RAM leak for the GPU process. You need to periodically go to about:processes, scroll down to the GPU process, and close the GPU process with the X button on the right column.
1) Generate a profile performance with https://profiler.firefox.com/
2) Submit the link with your results to https://bugzilla.mozilla.org/