There are many more sex hormones than estrogen and testosterone.
For example on the androgenic side there is DHT, androstenedione, DHEA, A5, etc. On the estrogenic side there is estetrol, estrodiol, estrone, etc.
DNA encoding in males and females also alters the quantity of non-sex related hormones like T3, T4, TSH in the thyroid for example.
Men and women are indeed different in metal characteristics, but it is often difficult to pinpoint one specific hormone as the cause.
Many hormones influence the release of other hormones. For example, testosterone is converted to estrogen in side of fat cells through a process called aromatization. So a male with too much testosterone, may end up with too much estrogen as well. DHT another androgenic hormone is also converted from testosterone, and testosterone itself is created out of fats in your blood stream.
Many of these hormones are signaling hormones for things like stress, libido, anxiety, etc. A combination of these is what makes up characteristics of a person's personality.