HP 3PAR, StorSimple etc all have done it long time back but at a bigger scale. Apple is just scaling it down, and bringing it in front of the regular Joe (as regular as Apple customers tend to be) in a non-complex manner.
StorSimple can dominate the appliance it is made out of to provide its services. Fusion Drive cannot dominate the owner's machine.
How difficult is it to implement some thing like that in a pre known 2 tier configuration context? And what would be the effectiveness of such a solution?
In Windows filter drivers have been used effectively to implement file tombstoning for archive purposes and utilizing reparse points/extended attributes to store the remote URI (for recalling file contents from remote archive upon intercepting a read request). It's not too hard to write a mini filter driver in Windows and use it in conjunction with reparse points to implement tombstoning in filesystem.
It certainly seems unnatural.
Most of those appliance solutions (and probably Fusion Drive for that matter) implement their mojo at the block level and provide block device interfaces. This is to aid their seamlessness, it makes them a lot more straightforward to implement and it makes them a lot more flexible. I would have trouble believing it would be simpler to implement it at a mini-filter/VFS level.