What does it mean to ask for an “explainable” algorithm?

One of the standard critiques of using algorithms for decision-making about people, and especially for consequential decisions about access to housing, credit, education, and so on, is that the algorithms don’t provide an “explanation” for their results or the results aren’t “interpretable.” This is a serious issue, but discussions of it are often frustrating.

Source: What does it mean to ask for an “explainable” algorithm?