This strategy is possible because p-values are themselves stochastic and a researcher will find one significant p-value for every 20 models that they run (at least on average).
p-hacking could also refer to pushing a p-value close to the significant cut-off (usually 0.05) by modifying the statistical model slightly until the desired result is achieved. This process usually involves the inclusion of control variables that are not really related to the outcome but that will change the standard errors/p-values.
Another way to p-hack is to drop specific observations until the desired p-value is reached. This process usually involves removing participants from a sample for a seemingly legitimate reason until the desired p-value is achieved. Usually identifying and eliminating a few high leverage observations is enough to change the significance level of a point estimate.
Multiple strategies to address p-hacking have been proposed and discussed. One of the most popular ones is pre-registration of research designs and models. The idea here is that a researcher would publish their research design and models before conducting the experiment and they will report only the results from the pre-registered models. This process eliminates the "fishing expedition" nature of p-hacking.
Other strategies involve better research designs that are not sensitive to model respecification. These are usually experimental and quasi-experimental methods that leverage an external source of variation (external to both the researcher and the studied system, like random assignment to conditions) to isolate the relationship between two variables.