At the machine level there is absolutely a difference. one means your data is right here, the other means the data is elsewhere. That poor data locality can cause cache misses and always consumes extra cycles.
Lets say it's 4 8bit ints. It can be sequentially and directly in memory like:
stack: [0x01, 0x02, 0x03, 0x03]
stack: [0x07002412, 0x0700241A, 0x07002424, 0x070036A0]
heap (or stack, really):
0x07002412: 0x01
0x0700241A: 0x02
0x07002424: 0x03
0x070036A0: 0x04
That is what "void*" is doing. You're using a pointer instead of a value.
In C# this is what generics allow you to avoid.
// This is like my first memory layout above.
struct Foo<T> { public T MyField; }
// This is like the second layout above.
struct Foo { public object MyField; }