Testing and Tuning Market Trading Systems - by Timothy Masters (Paperback)

Book Synopsis

Build, test, and tune financial, insurance or other market trading systems using C++ algorithms and statistics. You've had an idea and have done some preliminary experiments, and it looks promising. Where do you go from here? Well, this book discusses and dissects this case study approach.

Seemingly good backtest performance isn't enough to justify trading real money. You need to perform rigorous statistical tests of the system's validity. Then, if basic tests confirm the quality of your idea, you need to tune your system, not just for best performance, but also for robust behavior in the face of inevitable market changes. Next, you need to quantify its expected future behavior, assessing how bad its real-life performance might actually be, and whether you can live with that. Finally, you need to find its theoretical performance limits so you know if its actual trades conform to this theoretical expectation, enabling you to dump the system if it does not liveup to expectations.

This book does not contain any sure-fire, guaranteed-riches trading systems. Those are a dime a dozen... But if you have a trading system, this book will provide you with a set of tools that will help you evaluate the potential value of your system, tweak it to improve its profitability, and monitor its on-going performance to detect deterioration before it fails catastrophically. Any serious market trader would do well to employ the methods described in this book.

What You Will Learn

• See how the 'spaghetti-on-the-wall' approach to trading system development can be done legitimately
• Detect overfitting early in development
• Estimate the probability that your system's backtest results could have been due to just good luck
• Regularize a predictive model so it automatically selects an optimal subset of indicator candidates
• Rapidly find the global optimum for any type of parameterized trading system
• Assess the ruggedness of your trading system against market changes
• Enhance the stationarity and information content of your proprietary indicators
• Nest one layer of walkforward analysis inside another layer to account for selection bias in complex trading systems
• Compute a lower bound on your system's mean future performance
• Bound expected periodic returns to detect on-going system deterioration before it becomes severe
• Estimate the probability of catastrophic drawdown

Who This Book Is For

Experienced C++ programmers, developers, and software engineers. Prior experience with rigorous statistical procedures to evaluate and maximize the quality of systems is recommended as well.

From the Back Cover

"The algorithms in this book are essential tools for any serious trading system developer."

--David R. Aronson, Hood River Research Inc.

Build, test, and tune financial, insurance or other market trading systems using C++ algorithms and statistics. You've had an idea and have done some preliminary experiments, and it looks promising. Where do you go from here? Well, this book discusses and dissects this case study approach.

Seemingly good backtest performance isn't enough to justify trading real money. You need to perform rigorous statistical tests of the system's validity. Then, if basic tests confirm the quality of your idea, you need to tune your system, not just for best performance, but also for robust behavior in the face of inevitable market changes. Next, you need to quantify its expected future behavior, assessing how bad its real-life performance might actually be, and whether you can live with that. Finally, you need to find its theoretical performance limits so you know if its actual trades conform to this theoretical expectation, enabling you to dump the system if it does not live up to expectations.

This book does not contain any sure-fire, guaranteed-riches trading systems. Those are a dime a dozen... But if you have a trading system, this book will provide you with a set of tools that will help you evaluate the potential value of your system, tweak it to improve its profitability, and monitor its on-going performance to detect deterioration before it fails catastrophically. Any serious market trader would do well to employ the methods described in this book.

You will:

• See how the 'spaghetti-on-the-wall' approach to trading system development can be done legitimately
• Detect overfitting early in development
• Estimate the probability that your system's backtest results could have been due to just good luck
• Regularize a predictive model so itautomatically selects an optimal subset of indicator candidates
• Rapidly find the global optimum for any type of parameterized trading system
• Assess the ruggedness of your trading system against market changes
• Enhance the stationarity and information content of your proprietary indicators
• Nest one layer of walkforward analysis inside another layer to account for selection bias in complex trading systems
• Compute a lower bound on your system's mean future performance
• Bound expected periodic returns to detect on-going system deterioration before it becomes severe
• Estimate the probability of catastrophic drawdown

About the Author

Timothy Masters received a PhD in mathematical statistics with a specialization in numerical computing. Since then he has continuously worked as an independent consultant for government and industry. His early research involved automated feature detection in high-altitude photographs while he developed applications for flood and drought prediction, detection of hidden missile silos, and identification of threatening military vehicles. Later he worked with medical researchers in the development of computer algorithms for distinguishing between benign and malignant cells in needle biopsies. For the last twenty years he has focused primarily on methods for evaluating automated financial market trading systems. He has authored five books on practical applications of predictive modeling: Practical Neural Network Recipes in C++ (Academic Press, 1993); Signal and Image Processing with Neural Networks (Wiley, 1994); Advanced Algorithms for Neural Networks (Wiley, 1995); Neural, Novel, and Hybrid Algorithms for Time Series Prediction (Wiley, 1995); Data Mining Algorithms in C++ (Apress, 2018); Assessing and Improving Prediction and Classification (Apress, 2018); Deep Belief Nets in C++ and CUDA C: Volume 1 (Apress, 2018); and Deep Belief Nets in C++ and CUDA C: Volume 2 (Apress, 2018).