Abstract: As an online learning algorithm of approximate dynamic programming (ADP), direct heuristic dynamic programming (DHDP) has demonstrated its applicability to large state and control problems.