The System Metaphor is from the first edition of *Extreme Programming Explained*. It is a metaphor that shapes decisions at mainly the more abstract levels of description.
The classic example is the C3 System Metaphor.
Vat and Cruise Ship is another example.
I speculate that system metaphor is less important in these days when everything is done with expansive frameworks like Rails or Phoenix. Smalltalk was itself a framework (think model-view-controller), but in a way that begs for something additional to provide more structure.