Most Comprehensive Tactic Test in FM-Base History

  • Thread starter Thread starter Lucho616
  • Start date Start date
  • Replies Replies 500
  • Views Views 380K
core2FM said:
Please test spectre v2 as I want to see how your results compare against mine. This tactic has performed the best by a very long way in my own tests.
Click to expand...

It's tested, waiting for Lucio to update the public spreadsheet :) Meanwhile, it ended on 275 points, so it's defo a good tactic :) We must consider around 20 points deviation (0,1 points per match) in the test, so it could be around 295 or 255 also.
 
Thank you Mr Langvatn that you now agree that testing is not an exact science. I did not want to start a spamming war when earlier you said your test had a deviation of less than 0.1%.

I have tested tactics for a long time through all the FM versions until I stopped during FM 13. I have created super duper testing databases with as much randomness removed as possible, however just like in real life, there is always a random element to the game. This is intentionally programmed into the game to avoid situations where no giant killings would take place and the strongest CA teams would always win everything.

So yes tactic testing is not an exact science and neither is it a complete waste of time if done properly.

Mr Langvatn said:
It's tested, waiting for Lucio to update the public spreadsheet :) Meanwhile, it ended on 275 points, so it's defo a good tactic :) We must consider around 20 points deviation (0,1 points per match) in the test, so it could be around 295 or 255 also.
Click to expand...
 
Spreadsheet Updated! Sorry for the delay but I've been busy all the week!

Happy fming!
 
core2FM said:
Thank you Mr Langvatn that you now agree that testing is not an exact science. I did not want to start a spamming war when earlier you said your test had a deviation of less than 0.1%.

I have tested tactics for a long time through all the FM versions until I stopped during FM 13. I have created super duper testing databases with as much randomness removed as possible, however just like in real life, there is always a random element to the game. This is intentionally programmed into the game to avoid situations where no giant killings would take place and the strongest CA teams would always win everything.

So yes tactic testing is not an exact science and neither is it a complete waste of time if done properly.
Click to expand...

To be clear, I have never said the deviation was 0,1%. I always said the deviation was around 0,1 points per match, which makes around 20 points +/- in our test, due to the number of matches. But, yes, I agree with your post :) As I've said before, it is wrong to say that the nr 1 tactic in the test is the best tactic, as there will never be enough proof for it, unless it actually has 40 more points than nr 2. This is because nr 2 could have performed 20 points less than standard deviation and nr 1 could have performed 20 points more than standard deviation :)

What we can say is that there is a higher probability that nr 1 really is nr 1 than that nr 2 is the "true" nr 1. The only way to be 99% sure is to increase the number of matches to over 1000 (maybe even more), because as with all gambling, statistics beat luck over time.
 
Well said and agreed. I think our databases are quite similar to each other. Your tactic scored 149 pts in my first test which is the one published in my spreadsheet. I then tested it a further 3 times and the results from memory were 149, 148 and 156. This is a little better than 0.1 per match, however over 90 games the deviation from season to season can be +- 10 points, so as I said it's not an exact science.

Some people when testing their own tactics test for around 20-30 games and then multiply the results and believe this should correlate to the actual outcome. This is also inaccurate as their are peaks and troughs throughout a 90-180 game season(s). As you said the only way to conclusively declare one tactic as number one would be to increase the number of games to around 1000, which is way too time consuming.

The way I look at testing it to make sure the testing conditions are the same for all tactics, ie there are enough players for each role, natural footed players etc. Now if we provide the exact same conditions to each and every tactic and it is not unbalanced, then if a tactic performs consistently we can declare that it is a good tactic, however all effort must be made to ensure the conditions are same for each tested tactic. That is the challenge.

Mr Langvatn said:
To be clear, I have never said the deviation was 0,1%. I always said the deviation was around 0,1 points per match, which makes around 20 points +/- in our test, due to the number of matches. But, yes, I agree with your post :) As I've said before, it is wrong to say that the nr 1 tactic in the test is the best tactic, as there will never be enough proof for it, unless it actually has 40 more points than nr 2. This is because nr 2 could have performed 20 points less than standard deviation and nr 1 could have performed 20 points more than standard deviation :)

What we can say is that there is a higher probability that nr 1 really is nr 1 than that nr 2 is the "true" nr 1. The only way to be 99% sure is to increase the number of matches to over 1000 (maybe even more), because as with all gambling, statistics beat luck over time.
Click to expand...
 
Could you guys test Deep Sicilian Defense v5?

http://www.fm-base.co.uk/forum/shar...gvatn-fm16-tactic-center-135.html#post2983823

I was using Midsomer, the Crunge, Puskas, Kingdom, Wrath and few others but it seems that nothing can compare to this one... In other testing threads it scored a lot of points, not rarely even being best. I think it's worth giving a go as I found it absolute top for me.

Keep it up, great job.
 
gregu69 said:
Could you guys test Deep Sicilian Defense v5?

http://www.fm-base.co.uk/forum/shar...gvatn-fm16-tactic-center-135.html#post2983823

I was using Midsomer, the Crunge, Puskas, Kingdom, Wrath and few others but it seems that nothing can compare to this one... In other testing threads it scored a lot of points, not rarely even being best. I think it's worth giving a go as I found it absolute top for me.

Keep it up, great job.
Click to expand...

It's already been tested, but we prefer not to share results of my tactics, as this thread is about giving 100% of the focus to other and/or new tacticians here at FM-base, in an attempt to make the tactician-community grow.
 
Mr Langvatn said:
Just the CA 120 one. CA 145, 160 and 180 have other tops. 160/180 results haven't been released yet. Will be soon I think.
Click to expand...

Any news about the new results uploading?

Thanks guys

EDIT: My bad, just seeing you did it already ;)
 
Last edited:
What would interest me, is a custom database, that we could test (don't know how yet), which attributes influence the match engine the most and which the least. I believe tactics are important but so are attributes.

Anyone else on board with this?
 
You must log in or register to reply here.

FMBase

Beta

Resources

Back
Top