The second part of your question is thought-provoking. There are a lot of works of fiction that have inspired me due to broad themes, but there are a lot fewer whose inherent "structure, language, [and] composition" have inspired. So I'll try and focus my answer on this latter idea.
Also, I'm going to list these from 1 to 2, but the first is an order of magnitude better then the second, so think of them on an inverse log scale.
1. David Foster Wallace's short story "The Awakening of My Interest in Annular Systems". DFW is a brilliant writer in general, but his writings on math and logic have always been so unique and powerful to me. This one is my favorite. There's this combination of melancholy, and mathematical precision that was one of the first times I started to think about the beauty of analysis, and in particular, abstracting n-dimensional geometric space.
2. Peter Watt's novel "Blindsight". Blindsight has a very different mathematical feel to it, compared to DFW's stories. It's almost the opposite. Very sloppy (stylistically, not in terms of accuracy), filled with jargon, and always slightly outside the limits of my comprehension, (actually a lot of times just plain outside the limits of my comprehension). But that layer of overwhelming, rapid-fire technical description ends up creating this very interesting, frenetic, cyberpunk/dystopian aesthetic, as the plot builds momentum.
Honorable Mentions/Other stuff:
- Watt's "Echopraxia" (sequel to Blindsight)
- Neal Stephenson's "Anathem"
- Arthur C. Clarke "Rendesvous with Rama"
- Greg Egan's "Permutation city" and "Diaspora"
Art/Architecture:
- Philip Beesley's Hylozoic Soil installations (my advisor in uni!)
- All of Junya Ishigami's installations/architecture.
- All of Rafael Moneo's buildings & writings.