Not clear if it is learning #11
Description
Hello!
I was reviewing your code base, considered using it as part of a demo for a class I teach. The initial run didn't seem to be learning much. I went into the function AddIndicators and added 2 new indicators:
# Add a magic indicator that will tell you tomorrow's return
df['magic'] = df['Close'].pct_change().shift(-1)
df['magic8'] = df['Close'].pct_change(8).shift(-8)
So, the idea here is that I will tell the bot what the return will be in 1 hour and in 8 hours. With this information, a human trader could make a huge return. I've done this same test on 3 other code bases, and only 1 could actually learn this.
After implementing this change and re-running the bot, for thousands of episodes, it does not seem to have learned much. The average return and episodic return aren't zooming up like I would expect.
episode: 26340 worker: 21 net worth: 943.79 average: 1046.29 orders: 2
episode: 26341 worker: 14 net worth: 846.84 average: 1044.38 orders: 6
episode: 26342 worker: 26 net worth: 1019.90 average: 1045.11 orders: 34
episode: 26343 worker: 16 net worth: 1661.54 average: 1051.10 orders: 92
episode: 26344 worker: 20 net worth: 1020.38 average: 1051.17 orders: 49
episode: 26345 worker: 24 net worth: 989.14 average: 1052.00 orders: 3
episode: 26346 worker: 19 net worth: 990.24 average: 1052.65 orders: 4
This is very similar to the first few episodes, except generally the number of orders has declined.
This might be due to the convolution layer 'blurring out' or averaging away the ability of the bot to notice that one of its features is very helpful.
Expected behavior: Return should get much higher when the bot is provided with perfect information from the future.
Actual behavior: Doesn't seem to change anything.
Thank you!