It is becoming increasingly apparent that without some form of explanation capability, the full potential of trained Artificial Neural Networks (ANNs) may not be realised. This survey gives an overview of techniques developed to redress this situation. Specifically the survey focuses on mechanisms, procedures, and algorithms designed to insert knowledge into ANNs (knowledge initialisation), extract rules from trained ANNs (rule extraction), and utilise ANNs to refine existing rule bases (rule refinement). The survey also introduces a new taxonomy for classifying the various techniques, discusses their modus operandi, and delineates criteria for evaluating their efficacy.