Trading psychology vs. statistical edge

[PipMeHappy]

[Kaitsu]

Interesting vid!

I was pleased to see he is also using an hourly chart, which is my personal favourite.

If I understood correctly, he is gauging statistical edge by comparing the results of his own discretionary use of a strategy with an automated version using the same strategy.

I tend to do the same but it is not always so simple as it might sound!

The main difficulty in trying to achieve any meaningful statistical comparison is that I am not always able or willing to actually trade every signal given by the strategy. This is especially so with hourly charts. Sometimes a strategy trade signal occurs overnight or on a day when I have other things to do, or prior to a significant event that I don't wish to trade through. One could choose to only compare those trades actually taken and ignore the other automated system trades but that then tends to distort the theoretical performance of the strategy as a whole.

Also, there is a tendency to start trying to outsmart one's own system which can lead to some bad habits such as anticipating a signal before it happens - only to find that the signal doesn't actually occur! I have that tendency! My strategy is based on hourly closes and there is always a temptation to pre-empt the close and try to enter at a better level during the candle - only to see the candle reverse direction before the end of the hour!  

On the other hand, there are legitimate ways to fine tune the results and produce a personal edge relative to the raw strategy. For example, statistically the following candle may usually retrace 10-30 pips from the previous candle close. This means one can often get a better entry level which can be then used in  a variety of ways such as a more distant stop or nearer target level for same R:R as the automated version, or a bigger profit, etc.

Another problem area is the size of trades. I tend to vary my position size depending on a subjective chart "feel" such as general volatity levels and may also close part or all of these positions entirely depending on how things feel e.g. if it seems that the market is struggling to reach those last few pips to my target then I will close it.

When there are a number of such variations, especially with variable position size, there is no meaningful statistical comparison with the automated results apart from the total pips earned at the end of a period such as a month-end.

There again, maybe I am looking at this the wrong way!   

[PipMeHappy]

@Kaitsu indeed a backtest should not be compared to the discretionary results as much as being used as a check on the validity of a discretionary idea. In short: one could make the discretionary strategy signal test one made purely of a series of binary rules, including elements of volatility and position size, making it as close to what we do as point-and-click discretionary traders.

I think this is very much where I want to head to next, i.e. looking at my discretionary trading to try defining the approach in terms that could be replicated in a signal test and that a computer would understand in binary terms.

I think Mathew Verdouw's work at Optuma has really opened my eyes to what I need to do next, which of course - as you may well know - is a lot of work!

https://www.optuma.com/videos/signal-testing-2/

[Caesar]

Nice one there Francesco, lot of good stuff and clear presentation as well.
Agree with Kaitsu, I've tried outsmarting myself and burned my fingers so I'm not likely to tinker with something that currently works, even if I could theoretically improve.

I do like how you're constantly looking to improve your knowledge base, you're already well ahead of me from what I can see.

Enjoy your day guys,
Caesar

[Kaitsu]

@Kaitsu indeed a backtest should not be compared to the discretionary results as much as being used as a check on the validity of a discretionary idea. In short: one could make the discretionary strategy signal test one made purely of a series of binary rules, including elements of volatility and position size, making it as close to what we do as point-and-click discretionary traders.

I think this is very much where I want to head to next, i.e. looking at my discretionary trading to try defining the approach in terms that could be replicated in a signal test and that a computer would understand in binary terms.

I think Mathew Verdouw's work at Optuma has really opened my eyes to what I need to do next, which of course - as you may well know - is a lot of work!

I watched the Optuma video and it certainly raises some interesting issues. I cannot claim to have fully appreciated the process going on there just from one viewing of this one video, but my first thought was that it only tells the performance of a particular strategy compared with a benchmark (in this case the SPX). You can adjust the parameters such as the stoploss but it does not appear to actually try to optimise the settings by running the test across a range of parameters from, say, X through to Y. So a strategy may well look OK but is it actually running at its best possible?

However, is this still not a form of backtesting? And does it not require that the strategy would have been run with precisely defined and totally unchanged parameters throughout the period of years included in the tests? That would be a tough order for most traders to meet!

I would find this very restrictive in my personal trading! But it would be understandable for a fund management needing to demonstrate to a client how a specific fund is performing. Personally, my "signals" are not that specifically defined. My strategy calls for a certain set-up at the end of a candle but that set-up can last for several candles before actually entering depending on a number of rather non-specific factors.

I have a problem with all these types of statistical analyses because the performance of any rigidly defined strategy is rigidly defined by the market itself and not the actual parameters of the strategy. And we all know how infinitely varied market movements can be especially when we factor in the increasing impact of algos and programmed trading. For this reason I have also always refrained from defining monthly targets and comparisons, because the same strategy can work very differently in different months and it is the market that determines this and not the strategy itself. Maybe over a longer time period these differences even themselves out but surely not always! For example, I can imagine a strategy designed for Crude Oil would have worked very differently in earlier years compared with the current and foreseeable market?

But some kind of benchmarking or comparative performance studies is certainly necessary where large sums are involved and/or when trading other people's money!

As you say, this is a big subject with many considerations and approaches and I guess for any trader the first task is to define what they actually need/want to know about their trading performance and their strategies and then to select what statistical studies help shed light on the subject for them.

But I am probably seriously understating the importance of quantitative studies, for which I sincerely apologise!    

[Kaitsu]

This is a topic worthy of deeper discussion and it would be great to hear views from a much wider audience in order to gain a broader appreciation, whether based on experience or just an opinion

[PipMeHappy]

This is a topic worthy of deeper discussion and it would be great to hear views from a much wider audience in order to gain a broader appreciation, whether based on experience or just an opinion

Indeed it is!! I am going to leave others to reply and step aside...Thanks for your very valid points, @Kaitsu!!

[Kaitsu]

Indeed it is!! I am going to leave others to reply and step aside...Thanks for your very valid points, @Kaitsu!!

TBH, I don't really know what I am talking about because I am not familiar with the subject! So I am only throwing in the thoughts that the video brought to mind and I am probably way off the path here!

But it has made me think more about what (little) statistical analysis I do. I don't think I do anywhere near enough trades to merit the kind of analysis in the video but, as I have said before, my journalling has migrated away from my actual trades and more on the trades that the strategy itself produces whether I trade it or not. And now I am thinking there is a lot more that I could do in systematically developing that work that would then reflect in the way I "copy" trade the strategy trades with my actual trades (or is that just gobbledegook! )

As we have said, retail trading is an isolated, even lonely, business, and one can tend to become lazy about looking for ways to develop further. I think you have given me some inspiration to investigate this area further, albeit not quite in the manner of Optuma!

[eddieb]

TBH, I don't really know what I am talking about because I am not familiar with the subject! So I am only throwing in the thoughts that the video brought to mind and I am probably way off the path here!

Same here. I've never backtested and didn't really understand big chunks of the Optuma video 😕

[BWS]

didn't really understand big chunks of the Optuma video

I think thats part of the intention.
Remember, they want to sell a product so its in their interest to tell you enough to tease you while at the same time holding back enough so you never really understand how they do it. If you fully understood, would you need to buy, or could you create your own programme or pay someone to do it for you cheaper than Optuma?
Also, if you don't fully understand it you may fear that you are missing out on something important, perhaps even feel that you are somehow inadequate compared to people using Optuma.
Now, I'm not saying their product couldn't enhance you trading, but is the amount of improvement sufficient to make it worthwhile for a retail trader with limited resources with which to trade? $995 for the basic software pack for private traders is a lot for many to pay out on a product they don't understand.

[PipMeHappy]

@Kaitsu @BWS

Of course it is expensive for the basic package, and TradingView Pro or Pro+ is cheaper, as is TradeStation (except it is not really set up for forex)...

But Mathew does (I think) a great job of explaining things here: https://www.optuma.com/quantitative-technical-analysis-strategies/

It is not about sticking to exactly the same trades as your test, it is about seeing whether you can save yourself a lot of pain and time by taking an idea and test it thousands of times within a few days rather than months or years...

I think this adds value to your trading and maybe can show you how you could optimise your discretionary strategy... And there would be no shame in automating a strategy if you thought you could get better results letting the machine do what you do manually but without having to manage every trade yourself. As David Bush (see Chat With Traders) said, when you automate trading you can start looking at managing a portfolio or the overall strategy rather than having to do all that plus clicking your way in and out of trades too, which takes away time from research into opportunities.

I am not a seasoned automated trader but I think that there is value in one's time and even if you did not want to manage other people's money it could work in your favour to take something that you had an intuition and some discretionary results behind (with positive P/L) and test it more thoroughly in a much wider set of possible scenario to see how robust it is. You could then decide that you would continue to trade it in a discretionary way but with confidence that it has a profitability/win rate that makes it worth your time.

I think that on that basis it is worth thinking about - and you can backtest in many different ways, you do not need Optuma for that, for sure!

PipMe

[PipMeHappy]

didn't really understand big chunks of the Optuma video

I think thats part of the intention.
Remember, they want to sell a product so its in their interest to tell you enough to tease you while at the same time holding back enough so you never really understand how they do it. If you fully understood, would you need to buy, or could you create your own programme or pay someone to do it for you cheaper than Optuma?
Also, if you don't fully understand it you may fear that you are missing out on something important, perhaps even feel that you are somehow inadequate compared to people using Optuma.
Now, I'm not saying their product couldn't enhance you trading, but is the amount of improvement sufficient to make it worthwhile for a retail trader with limited resources with which to trade? $995 for the basic software pack for private traders is a lot for many to pay out on a product they don't understand.

Thing is, Mathew and the Optuma team have a lot of investment professionals as clients and their products reflects that - it is not a colourful charting package like TradingView, it is rather less attractive but its strength is ibeing able to incorporate your own scripting or use their ready made for any strategy testing. You can code strategies with some providers too, of course, like Interactive Brokers, but you need to know how to script/code... The good thing about Optuma is
that Mathew is a passionate TA advocate who has put TA squarely at the centre of quant analysis through Optuma: as he says in the last video I posted, a lot of TA has been phased out by quants coming into the scene, and the quants' language is what clients like to see on money managers' brochures... You just cannot get clients if you talk about trend lines and Fibonacci, unless you test those TA concepts in a quantifiable way.

For us retail traders who do not benchmark our trading alpha or beta against the ever-present Standards & Poor 500 and who are much more agile with coming in and out of trades, it all seems like overkill... But even, for example, using advanced Excel functions to get down to understanding the modus operandi of how we traded in the last, say, twelve months, is a worthwhile endeavour of quant analysis: we are trying to summarise the essence of what we do through a study of our own trading performance through industry metrics such as volatility, risk of ruin, average win/loss ratio,  Sharpe and Sortino ratio, skewness and returns distribution, etc.

My goal is to get better at this side of things because otherwise I feel like I am swimming in the dark.

PipMe

[Kaitsu]

we are trying to summarise the essence of what we do through a study of our own trading performance through industry metrics such as volatility, risk of ruin, average win/loss ratio,  Sharpe and Sortino ratio, skewness and returns distribution, etc.

I can see the benefits, maybe even the necessity, of such systems for professionals and fund managers, but for me, as a retail trader, just trading for my own pleasure, these statistics would mean nearly nothing at all.

I watch my win/loss ratios, both in number of trades and in pip values, in a kind of "real time" because these mirror the stability of my strategy and my trading as well as possibly highlighting changes in the way the market is moving. These are just simple formulas in my Excel journal. But averaging these types of data over a period of even a year would be meaningless to me. As a retail trader, I trade when it suits and I select my position size according to my overall assessment of the chart - these are things I cannot build into such a system at all because they depend entirely on the "live" circumstances at the time, both personally and in the markets.

But I know these things have value for those trading professionally and who need to report to their bosses and to provide comparability with other traders/enterprises, as well as those looking to sell their trading skills to others.

Another dimension is one's reasons for trading. Obviously, profitability is the end result, but it is not the only motivation. Personally, I am not so interested in trying to improve my overall performance by a percentage point or two. For me, the challenge, the inspiration, and the passion in trading has always been in the intellectual dual between me and the market. I am just as happy with a 10 pip "victory" as I am with a 900 pip trade. Yesterday on the SP500 was a classic example of that. I saw that an early long was not going to perform and closed it in time, then I read the drop successively to last week's close, the 1H 200 SMA and last week's low - but I only made half of what I should have made. But I was entirely happy.


My point here is that applying a system like Optuma, and inevitably having to subsequently live with the results, I would be stifling my own creativity and flexibility and turning my trading into a money machine. My time frame is the present and the near future, the past is the past.

But that is just my personal profile and I only put it here to emphasise that each individual is different and has different aims and methods and needs. We each need to know what these are in order to know what systems we might need and what they might help us to achieve in line with our individual goals.

[Sharkie]

I liked the first video and understood it, but the Optuma video confused the hell out of me.
I follow that any improvement they can give us helps, but I can only assume that the retail traders who they expect to be interested enough to spend must be far more sophisticated than I am and with pockets to match.

[PipMeHappy]

@Kaitsu  I totally get where you are coming from... Indeed what one learns pretty early on is that there is no ONE WAY
  from trading the markets... Even the example of Julia Cordova - one among many successful private traders - is that doing mountains of quant/statistical analysis is not a prerequisite for getting good results in trading.

I think this discussion has been stimulating and I hope I did not come across as proselytising in any way.

@Sharkie , indeed such software is for deep pockets. Trading costs for professionals indeed come from charting services, for one, as well as things like Bloomberg Terminal, etc. But you can quite easily do certain things at a lower cost if you are not a money manager (i.e. carrying out a regulated function with clients' money), so what @Kaitsu  pointed out is very poignant.

I enjoyed this exchange of views as it touches the core question of what means to be a private trader in a data-driven era, and how much of this trend toward modelling is a new scientism (like when we thought that e-mails would reduce admin workload (ha!)) and how much adds value. At the end of the day, market opportunities cannot be around for everyone, so some models - regardless of smartness - will fail somewhere, for someone, at some point.