The formal definition of a byte is that it's the smallest addressable unit of memory. Think of a memory as a linear string of bits. A memory address points to a specific group of bits (say, 8 of them). If you add 1 to the address, the new address points to the group of bits immediately after the first group. The size of those bit groups is 1 byte.
In modern usage, "byte" has come to mean "a group of 8 bits", even in situations where there is no memory addressing. This is due to the overwhelming dominance of systems with 8-bit bytes. Another term for a group of 8 bits is "octet", which is used in e.g. the TCP standard.
Words are a bit fuzzier. One way to think of a word is that it's the largest number of bits acted on in a single operation without any special handling. The word size is typically the size of a CPU register or memory bus. x86 is a little weird with its register addressing, but if you look at an ARM Cortex-M you will see that its general-purpose CPU registers are 32 bits wide. There are instructions for working on smaller or larger units of data, but if you just do a generic MOV, LDR (load), or ADD instruction, you will act on 32 register bits. This is what it means for 32 bits to be the "natural" unit of data. So we say that an ARM Cortex-M is a 32-bit CPU, even though there are a few instructions that modify 64 bits (two registers) at once.
Some of the fuzziness in the definition comes from the fact that the sizes of the CPU registers, address space, and physical address bus can all be different. The original AMD64 CPUs had 64-bit registers, implemented a 48-bit address space, and brought out 40 address lines. x86-64 CPUs now have 256-bit SIMD instructions. "32-bit" and "64-bit" were also used as marketing terms, with the definitions stretched accordingly.
What it comes down to is that "word" is a very old term that is no longer quite as useful for describing CPUs. But memories also have word sizes, and here there is a concrete definition. The word size of a memory is the number of bits you can read or write at once -- that is, the number of data lines brought out from the memory IC.
(Note that a memory "word" is technically also a "byte" from the memory's point of view -- it's both the natural unit of data and the smallest addressable unit of data. CPU bytes are split out from the memory word by the memory bus or the CPU itself. Since computers are all about running software, we take the CPU's perspective when talking about byte size.)