An Approximate Dynamic Programming Algorithm for Monotone Value Functions

Daniel R. Jiang
Daniel R. Jiang
[email protected]
Department of Operations Research and Financial Engineering, Princeton University, Princeton, New Jersey 08540
Search for more papers by this author
,
Warren B. Powell
Warren B. Powell
[email protected]
Department of Operations Research and Financial Engineering, Princeton University, Princeton, New Jersey 08540
Search for more papers by this author

Daniel R. Jiang

[email protected]

Department of Operations Research and Financial Engineering, Princeton University, Princeton, New Jersey 08540

Search for more papers by this author

Warren B. Powell

[email protected]

Department of Operations Research and Financial Engineering, Princeton University, Princeton, New Jersey 08540

Search for more papers by this author

Published Online:4 Nov 2015https://doi.org/10.1287/opre.2015.1425

Supplemental Material

opre.2015.1425-sm-online-supplement.pdf (351 KB)

Cited by
- Two-echelon multi-depot location-routing problem with time windows and dynamic customer demands
  Transportation Research Part E: Logistics and Transportation Review, Vol. 212
- Frozen-State Value Iteration: Faster Reinforcement Learning by Freezing Slow States
  Yijia Wang,
  Daniel R. Jiang
  18 March 2026 | Management Science, Vol. 0, No. 0
- On-capacity vs. On-grid: Effect of subsidy on the adoption of high-quality energy storage facilities in the renewable energy industry
  Energy Policy, Vol. 208
- Solving Markov decision processes via state space decomposition and time aggregation
  European Journal of Operational Research, Vol. 324, No. 1
- Perturbed Decision-Focused Learning for Modeling Strategic Energy Storage
  IEEE Transactions on Smart Grid, Vol. 16, No. 3
- Multi-period integrated production and delivery scheduling to enhance online orders fulfillment considering due dates
  9 April 2025 | Flexible Services and Manufacturing Journal, Vol. 13
- Design of Denial-of-Service Attack Strategy With Energy Constraint: An Approximate Dynamic Programming Approach
- Joint optimization of opportunistic maintenance and speed control for continuous process manufacturing systems considering stochastic imperfect maintenance
  Computers & Industrial Engineering, Vol. 198
- Approximate dynamic programming for pickup and delivery problem with crowd-shipping
  Transportation Research Part B: Methodological, Vol. 187
- An Easier-to-Verify Sufficient Condition for Whittle Indexability and Application to AoI Minimization
- Real-time dispatch management of shared autonomous vehicles with on-demand and pre-booked requests
  Transportation Research Part A: Policy and Practice, Vol. 181
- Transferable Energy Storage Bidder
  IEEE Transactions on Power Systems, Vol. 39, No. 2
- Primal-Dual Value Function Approximation for Stochastic Dynamic Intermodal Transportation with Eco-Labels
  Arne Heinold,
  Frank Meisel,
  Marlin W. Ulmer
  14 October 2022 | Transportation Science, Vol. 57, No. 6
- Δ V-learning: An adaptive reinforcement learning algorithm for the optimal stopping problem
  Expert Systems with Applications, Vol. 231
- Real-time dispatch of integrated electricity and thermal system incorporating storages via a stochastic dynamic programming with imitation learning
  International Journal of Electrical Power & Energy Systems, Vol. 153
- Minimizing Age of Information for Mobile Edge Computing Systems: A Nested Index Approach
- Approximate Dynamic Programming: Value Iteration
  21 September 2023
- Approximate dynamic programming with policy-based exploration for microgrid dispatch under uncertainties
  International Journal of Electrical Power & Energy Systems, Vol. 142
- Economic dispatch for electricity merchant with energy storage and wind plant: State of charge based decision making considering market impact and uncertainties
  Journal of Energy Storage, Vol. 53
- Seasonal hydrogen storage decisions under constrained electricity distribution capacity
  Renewable Energy, Vol. 195
- Optimal Use and Replenishment of Two Substitutable Raw Materials in a Stochastic Capacitated Make-to-Order Production System
  Qi (George) Chen,
  Izak Duenyas,
  Stefanus Jasin
  24 February 2022 | Manufacturing & Service Operations Management, Vol. 24, No. 4
- A Monotone Approximate Dynamic Programming Approach for the Stochastic Scheduling, Allocation, and Inventory Replenishment Problem: Applications to Drone and Electric Vehicle Battery Swap Stations
  Amin Asadi,
  Sarah Nurre Pinkley
  7 January 2022 | Transportation Science, Vol. 56, No. 4
- A numerical study of Markov decision process algorithms for multi-component replacement problems
  European Journal of Operational Research, Vol. 299, No. 3
- Backward Approximate Dynamic Programming
  8 April 2022
- Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Numerical Applications
  19 January 2021 | Methodology and Computing in Applied Probability, Vol. 24, No. 1
- Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
  9 December 2021 | Nonlinearity, Vol. 35, No. 1
- Adaptive Learning of Drug Quality and Optimization of Patient Recruitment for Clinical Trials with Dropouts
  Zhili Tian,
  Weidong Han,
  Warren B. Powell
  24 March 2021 | Manufacturing & Service Operations Management, Vol. 24, No. 1
- Fitted Value Iteration in Continuous MDPs With State Dependent Action Sets
  IEEE Control Systems Letters, Vol. 6
- Multi-Agent Learning-Based Nearly Non-Iterative Stochastic Dynamic Transactive Energy Control of Networked Microgrids
  IEEE Transactions on Smart Grid, Vol. 13, No. 1
- Multi-Stage Real-Time Operation of a Multi-Energy Microgrid With Electrical and Thermal Energy Storage Assets: A Data-Driven MPC-ADP Approach
  IEEE Transactions on Smart Grid, Vol. 13, No. 1
- Solving high-dimensional optimal stopping problems using deep learning
  27 April 2021 | European Journal of Applied Mathematics, Vol. 32, No. 3
- Poisoning finite-horizon Markov decision processes at design time
  Computers & Operations Research, Vol. 129
- Analysis on the Training System of Archives Managers in Colleges and Universities Based on Dynamic Programming Algorithms
- Optimal PHY Configuration in Wireless Networks
  IEEE/ACM Transactions on Networking, Vol. 28, No. 6
- A Reinforcement Learning Approach to Optimize Discount and Reputation Tradeoffs in E-commerce Systems
  27 October 2020 | ACM Transactions on Internet Technology, Vol. 20, No. 4
- Experimental Validation of Approximate Dynamic Programming Based optimization and Convergence on Microgrid Applications
- Meso-parametric value function approximation for dynamic customer acceptances in delivery routing
  European Journal of Operational Research, Vol. 285, No. 1
- Approximate dynamic programming for planning a ride-hailing system using autonomous fleets of electric vehicles
  European Journal of Operational Research, Vol. 284, No. 3
- A Novel Fitted Rolling Horizon Control Approach for Real-Time Policy Making in Microgrid
  IEEE Transactions on Smart Grid, Vol. 11, No. 4
- Dynamic Programming Models for Maximizing Customer Lifetime Value: An Overview
  24 August 2019
- Two-Tiered Ambulance Dispatch and Redeployment considering Patient Severity Classification Errors
  Journal of Healthcare Engineering, Vol. 2019
- An Algorithmic Model Building Scheme Based on Dynamic Programming Algorithms
  Journal of Physics: Conference Series, Vol. 1345, No. 5
- Electric Power Industry: Operational and Public Policy Challenges and Opportunities
  1 November 2019 | Production and Operations Management, Vol. 28, No. 11
- Low-Rank Value Function Approximation for Co-Optimization of Battery Storage
  IEEE Transactions on Smart Grid, Vol. 9, No. 6
- Shape Constraints in Economics and Operations Research
  Statistical Science, Vol. 33, No. 4
- Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures
  Daniel R. Jiang,
  Warren B. Powell
  13 November 2017 | Mathematics of Operations Research, Vol. 43, No. 2
- Energy management for stationary electric energy storage systems: A systematic literature review
  European Journal of Operational Research, Vol. 264, No. 2
- Optimal Liquidation in a Level-I Limit Order Book for Large-Tick Stocks
  SIAM Journal on Financial Mathematics, Vol. 9, No. 3
- A divide and conquer algorithm for exploiting policy function monotonicity
  1 January 2018 | Quantitative Economics, Vol. 9, No. 2
- Linear programming formulation for non-stationary, finite-horizon Markov decision process models
  Operations Research Letters, Vol. 45, No. 6
- Computing monotone policies for Markov decision processes: a nearly-isotonic penalty approach
  IFAC-PapersOnLine, Vol. 50, No. 1
- Optimal Liquidation in a Level-I Limit Order Book for Large Tick Stocks
  SSRN Electronic Journal, Vol. 11
- A Divide and Conquer Algorithm for Exploiting Policy Function Monotonicity
  SSRN Electronic Journal, Vol. 98
- Near optimal control for microgrid energy systems considering battery lifetime characteristics
- Energy-efficient metro train rescheduling with uncertain time-variant passenger demands: An approximate dynamic programming approach
  Transportation Research Part B: Methodological, Vol. 91
- Energy storage system operation: Case studies in deterministic and stochastic environments
- A dynamic evolutionary strategy for time ahead energy storage management in microgrid
- An improvement selection methodology for key performance indicators
  20 April 2016 | Environment Systems and Decisions, Vol. 36, No. 2
- A perturbation approach to a class of discounted approximate value iteration algorithms with borel spaces
  Journal of Dynamics and Games, Vol. 3, No. <![CDATA[3]]>

Volume 63, Issue 6

November-December 2015

Pages ii-iv, 1245-1546

Article Information

Supplemental Material

Metrics

Information

Received:July 01, 2014
Accepted:August 01, 2015
Published Online:November 04, 2015

Cite as

Daniel R. Jiang, Warren B. Powell (2015) An Approximate Dynamic Programming Algorithm for Monotone Value Functions. Operations Research 63(6):1489-1511.

https://doi.org/10.1287/opre.2015.1425

Keywords

PDF download

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

Available Issues

An Approximate Dynamic Programming Algorithm for Monotone Value Functions

Supplemental Material

Volume 63, Issue 6

Article Information

Supplemental Material

Metrics

Information

Cite as

Keywords

Sign Up for INFORMS Publications Updates and News