A good methodology is fundamental to create a profitable and successful system. These are our key rules when developing a system:
1. Testing system robustness
It is imperative to test the trading strategy in different markets and across large periods. The more the system is tested in different environments the more sure we can be that it is robust.
2. Testing parameter robustness
However, since the rules are only tested over past data, it is furthermore very important that we make sure these rules still work in the future, especially if the market does not behave 100% like in the past. Changing parameters of the system should not change its performance significantly. So, whenever we develop systems we test them over a large range of parameters. If the system works across various parameter values (and only then) we can recommend it, otherwise there is always the risk of over-optimizing and chances are high that good performance might be just caused by luck.
3. Monte Carlo simulation
Finally, we run a so-called Monte Carlo simulation in which many different simulation runs are performed. The trades are mixed so that each run has a different order of the trades. This gives us a probability range of the possible outcome of the system.