The Jayhawks (24-6, 13-4) trailed for much of the second half before Agbaji connected on two free throws and converted a steal into a dunk with 6:29 left to put Kansas in front for keeps.
Agbaji started 6 of 7 but finished just 8 of 19. TCU's second bid in three days to upset Kansas failed, though, as the Jayhawks used a 7-0 run to go up 69-61 before Dajuan Harris blocked a driving shot by Mike Miles with 2.8 seconds left and the Jayhawks leading by three.
Kansas, which led the league race for several weeks, is in position again to gain at least a share of its 20th Big 12 title in 26 years. It hosts Texas on Saturday while Baylor hosts Iowa State.
Christian Braun, Harris and David McCormack added 12, 11 and 10 points, respectively, while Jalen Wilson added nine points and a game-high 10 rebounds.
Going for its third straight win over a top 10 opponent, TCU (19-10, 8-9) made five straight shots to go up 47-41 and maintained a slim margin until Agbaji's go-ahead points. Emanuel Miller paced the Horned Frogs with 18 points. Damion Baugh added 14.
The Frogs forged a 33-30 advantage on the glass but committed 15 turnovers. They also committed 22 fouls to only 12 for the Jayhawks, who recorded an eight-point edge in free throws.
TCU trailed by 12 in the first half but responded with a late push to forge a 35-35 halftime tie behind eight points from Miller.
Agbaji countered with 14 first-half points for Kansas, which shot 50 percent but got outrebounded 19-12. A technical foul on the Jayhawks' Mitch Lightfoot for jawing after a dunk helped sway momentum with 5:33 left.
The Frogs converted that technical into a five-point possession that included a 3-pointer by JaKobe Coles. Eddie Lampkin scored TCU's first six points and none of his teammates scored until almost eight minutes had elapsed.