Skip to main content
SpringerLink
Log in

Adaptive Real-Time Dynamic Programming

  • Reference work entry
  • First Online:
Encyclopedia of Machine Learning and Data Mining

  • 750 Accesses

Synonyms

ARTDP

Definition

Adaptive Real-Time Dynamic Programming (ARTDP) is an algorithm that allows an agent to improve its behavior while interacting over time with an incompletely known dynamic environment. It can also be viewed as a heuristic search algorithm for finding shortest paths in incompletely known stochastic domains. ARTDP is based on Dynamic Programming (DP), but unlike conventional DP, which consists ofoff-line algorithms, ARTDP is an on-line algorithm because it uses agent behavior to guide its computation. ARTDP is adaptive because it does not need a complete and accurate model of the environment but learns a model from data collected during agent-environment interaction. When a good model is available, Real-Time Dynamic Programming (RTDP) is applicable, which is ARTDP without the model-learning component.

Motivation and Background

RTDP combines strengths of heuristic search and DP. Like heuristic search – and unlike conventional DP – it does not have to evaluate the...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 699.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 949.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Recommended Reading

  • Barto A, Bradtke S, Singh S (1995) Learning to act using real-time dynamic programming. Artif Intell 72(1–2):81–138

    Article  Google Scholar 

  • Bertsekas D, Tsitsiklis J (1989) Parallel and distributed computation: numerical methods. Prentice-Hall, Englewood Cliffs

    MATH  Google Scholar 

  • Bonet B, Geffner H (2003a) Labeled RTDP: improving the convergence of real-time dynamic programming. In: Proceedings of the 13th international conference on automated planning and scheduling (ICAPS-2003), Trento

    Google Scholar 

  • Bonet B, Geffner H (2003b) Faster heuristic search algorithms for planning with uncertainty and full feedback. In: Proceedings of the international joint conference on artificial intelligence (IJCAI-2003), Acapulco

    Google Scholar 

  • Feng Z, Hansen E, Zilberstein S (2003) Symbolic generalization for on-line planning. In: Proceedings of the 19th conference on uncertainty in artificial intelligence, Acapulco

    Google Scholar 

  • Hansen E, Zilberstein S (2001) LAO*: a heuristic search algorithm that finds solutions with loops. Artif Intell 129:35–62

    Article  MathSciNet  MATH  Google Scholar 

  • Jalali A, Ferguson M (1989) Computationally efficient control algorithms for Markov chains. In: Proceedings of the 28th conference on decision and control, Tampa, pp 1283–1288

    Google Scholar 

  • Korf R (1990) Real-time heuristic search. Artif Intell 42(2–3):189–211

    Article  MATH  Google Scholar 

  • Smith T, Simmons R (2006) Focused real-time dynamic programming for MDPs: squeezing more out of a heuristic. In: Proceedings of the national conference on artificial intelligence (AAAI). AAAI Press, Boston

    Google Scholar 

  • Sutton R (1990) Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In: Proceedings of the 7th international conference on machine learning. Morgan Kaufmann, San Mateo, pp 216–224

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Massachusetts, Amherst, MA, USA

    Andrew G. Barto

Authors
  1. Andrew G. Barto
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. The University of New South Wales, Sydney, NSW, Australia

    Claude Sammut

  2. Faculty of Information Technology, Monash University, Melbourne, VIC, Australia

    Geoffrey I. Webb

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media New York

About this entry

Cite this entry

Barto, A.G. (2017). Adaptive Real-Time Dynamic Programming. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning and Data Mining. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-7687-1_10

Download citation

Publish with us

Policies and ethics

Search

Navigation