I mean, hell, the TRM (https://developer.arm.com/documentation/ddi0487/latest) for the CPU in a RPI4 is over 11k pages long. I don't think that's something you could reasonably expect to reimplement on your own.
There'd be nothing stopping you from implementing a simpler/reduced RISC-V processor though, with only the ISA extensions you want (or none at all).
A pretty fast softcore is gonna be like 500Mhz, some really optimized designs might hit close to 700Mhz on better FPGAs. Although when working with softcores you can a lot times get away with lower clocks since anything that would require a lot of cycles if written in software can a lot times be made as a block on the FPGA that your softcore just manages. Assuming your FPGA has enough area. Freeing up the softcore to do other things.
You can generally do quite a bit of parallelism, see the quad-core LiteX+VexRISCV solution at <https://antmicro.com/blog/2020/05/multicore-vex-in-litex/>.
But this isn't meant to be doom and gloom - the fact that you can buy a $200 board and go throw some verilog together and have it run at several hundred megahertz attached to a PCIe bus is phenomenal from a hobbyist perspective.
(The data from PCIe x4 @ 5 GT/s per lane can be carried by a 128 bit bus @ 125 MHz)
The RPi's CPUs operate at 1.5 GHz (1500 MHz) or higher, meaning there are 1.5 GHz signals being sent around inside the CPU. The logic in this FPGA under the most ideal theoretical conditions can only operate at around 400 MHz, and for a "real" design, much slower than that - hence my reference to VexRiscv. It's an extremely simple core by modern standards, clock-for-clock it's way slower than the ARM cores in the RPI, yet it's only capable of hitting 200 MHz in (this) FPGA.
To elaborate a bit:
An FPGA has a few major components, but from a logic perspective, the two to focus on here are the "slices" and the routing fabric. The slices of the FPGA implement user logic and are very simple. They contain a few LUTs (look up tables) which implement logic functions. In the case of the Xilinx 7 series architecture, these can be either 6 input 1 output logic functions or 5 input 2 output functions. Other FPGAs can and will be different. By "function", think logic gates. For every combination of inputs, is the output on or off? The logic blocks also contain a number of flip-flops to hold state. There is one for every LUT output, so twice the LUT count (on 7 series). There are a few fixed function components to improve performance of common logic types, such as the carry chain for an adder or multiplexers. Some FPGAs bundle hard-cores for various logic, many have hardware multipliers that can be used, more complex (and expensive) ones can even have CPU cores (Xilinx Zynq and Intel/Altera Cyclone V for instance).
There are thousands of these slices on an FPGA. The smallest member of the Artix family has 2,000 of them, the largest contains over 33,000. The big Kintex and Virtex parts can have hundred of thousands. In order to do anything complicated, you'll need to use many slices to implement logic, which are connected together through the logic fabric. You'll see things like "logic depth" which is the number of LUTs connected in series before connecting to a register. The greater the logic depth the slower the design. The shorter the logic depth, the longer the pipeline. Cue Netburst style concerns. If you don't have to worry about hazards, it's completely fine, if you do, it's a nightmare.
For instance, you could start with this development board: http://www.myirtech.com/list.asp?id=502
Not equivalent to the Pi 4, but still runs linux. (the Acorn CLE is a rebadged NiteFury)