Augustine of Hippo, On the Merits and Forgiveness of Sins, and on the Baptism of Infants II.46: Guilt May Be Taken Away But Concupiscence Remain

You must not be surprised at what I have said, that although the law of sin remains with its concupiscence, the guilt thereof is done away through the grace of the sacrament. For as wicked deeds, and words, and thoughts have already passed away, and cease to exist, so far as regards the mere movements of the mind and the body, and yet their guilt remains after they have passed away and no longer exist, unless it be done away by the remission of sins; so, contrariwise, in this law of concupiscence, which is not yet done away but still remains, its guilt is done away, and continues no longer, since in baptism there takes place a full forgiveness of sins. Indeed, if a man were to quit this present life immediately after his baptism, there would be nothing at all left to hold him liable, inasmuch as all which held him is released. As, on the one hand, therefore, there is nothing strange in the fact that the guilt of past sins of thought, and word, and deed remains before their remission; so, on the other hand, there ought to be nothing to create surprise, that the guilt of remaining concupiscence passes away after the remission of sin.