Unprecedented progress in the deep learning field influenced many of different industries, including agriculture sector. The use of neural networks in agro-industrial activity in the task of recognizing cultivated crops and weed is a new direction with a history lower than 10 years. Dozens of new neural networks appear every year, but absence of any standards significantly complicates the understanding of the real situation of the use of the neural network in the agricultural sector. In the manuscript, we analyzed research over the past 10 years on the use of neural networks for the classification and tracking of crops and weeds in agriculture. We presented the analysis of the results of using various neural network algorithms for the task of classification and tracking. Finally, we made recommendations for the use of neural networks in the tasks of recognizing a cultivated object and weeds