Immediate implants: Do you graft the space?

When immediate implants are placed there is almost always an unfilled gap and we are faced with the decision of graft or no graft. While there are many variables based on this presentation what would you do?