Let's talk about Tactic Testing for FM21

  • Thread starter Thread starter Liam
  • Start date Start date
  • Replies Replies 336
  • Views Views 86K
mornon said:
On the other side, this could lead to a testing queue monopolization by tactics from famous creators.
Hoping SI stops releasing hotfix every day, I think this first "bottleneck" will be left behind in few days of intense testing :)
Click to expand...

Exactly right. Once we can get a few days of consistent testing, we'll have loads of results.
 
Bonjour. What a great job. This afternoon I saw a tactic from StephenHK rated five stars, but u can't find it anymore. It was a 4231. Why ?
 
^so as above described. we discussed that problem with stephenHK tactics before, that multiple were uploaded. that got fixed.
It also has broken the next thing. a tested tactic on the current ME shouldn't be deleted, if another tactic would be uploaded in the same thread.
@Liam
 
Redstar.SoS said:
^so as above described. we discussed that problem with stephenHK tactics before, that multiple were uploaded. that got fixed.
It also has broken the next thing. a tested tactic on the current ME shouldn't be deleted, if another tactic would be uploaded in the same thread.
@Liam
Click to expand...

No, that isn't a bug - it was a part of fixing the original bug. His tactics will be tested again soon I'm sure!
I've created a forum for suggestions and bugs: https://fm-base.co.uk/forums/fm-base-beta.255/
 
Pianist said:
Sure:
1. There are four tactics (in subtop) with a difference of 57 points out of 80 matches, but they are all 5 star. I think the differences are too big to rate them the same amount of stars.
2. All current tactics are at leat 4 stars in subtop, but very bad in underdog. That seems strange to me. Can you give some more insights in what the test differences are between underdog and subtop? That may help in interpreting the results.

Excuses my bad english btw :)

*edit: i know it's still early days, so maybe it becomes more clear after many more tactics are tested... I'm a little bit impatient because i am struggling with Fulham.. ;)
But is Fulham subtop or underdog? That makes a big difference in choosing a tactic apparently..
Click to expand...

Good questions!

Firstly, there aren't inherent differences in the testing between underdog and subtop. Just imagine a league with the actual best 100 teams in the world, we then add two teams:
  • Subtop - i.e. top 40% of the league (translates to top 8 teams in a 20-team league)
  • Underdog - i.e. bottom 40% of the league (translates to worst 8 teams in a 20-team league)
We use the editor to estimate these numbers and add players respectively to the teams that match the supposed quality level for each team, making sure every role is covered and reducing the randomness factor to the minimum physically possible with the tools available today.

The rating system is based on an algorithm we've worked on that can be changed, this is how it currently should be working:

Take the three main values you see when opening the Tactics page
  • Number of Matches (NoM)
  • Total Points (TP)
  • Goal Difference (GD)
Assign weight values
  • TP - 7.5
  • GD - 2.5
Assuming the worst tactic GD, let's say -100 for simplicity, we create a MIN value
  • MIN = 250 (inverse of -100 * 2.5 (= GD weight value))
Assuming the best tactic GD, let's say 100 for simplicity, we can now compute

TOT = max(TP) * 7.5 + max(GD) * 2.5 + MIN

Finally, divide by x (e.g. 100) to get ratings over 100 and later use stars
  • Divide TOT by 100, then floor it (i.e. getting the largest integer that is smaller or equal to TOT)
Adjusting the numbers, we empirically define what's the minimum rating (out of 100) needed to get 5 stars
  • Assuming >75 is needed for 5 stars
  • We assign one point on the 0 -> 10 scale based on each rating band you surpass
  • One point = half star

This should explain the current results you see. By the way, I'm not surprised as the vast majority of tactics is built using subtop teams!

Basically, when using an underdog team in such a competitive environment without randomness, it is incredibly unlikely you produce such high ratings that we assign 5 stars to an underdog tactic!

The stars reflect the actual result the tactic brings in the test.

Take as an example the Leicester championship back in 2016. If we computed 5000 simulations of the Premier League in that year, only once Leicester would have won! That's 0.0002% - and it accounts for randomness, if we take it out of the equation as we strive to do, the odds are even smaller!

I hope this clarifies!
 
Liam said:
I don't think @Pianist knows how to respond
Click to expand...
Yes i do, i'm not 24/7 on this forum;)
Fra said:
Good questions!

Firstly, there aren't inherent differences in the testing between underdog and subtop. Just imagine a league with the actual best 100 teams in the world, we then add two teams:
  • Subtop - i.e. top 40% of the league (translates to top 8 teams in a 20-team league)
  • Underdog - i.e. bottom 40% of the league (translates to worst 8 teams in a 20-team league)
We use the editor to estimate these numbers and add players respectively to the teams that match the supposed quality level for each team, making sure every role is covered and reducing the randomness factor to the minimum physically possible with the tools available today.

The rating system is based on an algorithm we've worked on that can be changed, this is how it currently should be working:

Take the three main values you see when opening the Tactics page
  • Number of Matches (NoM)
  • Total Points (TP)
  • Goal Difference (GD)
Assign weight values
  • TP - 7.5
  • GD - 2.5
Assuming the worst tactic GD, let's say -100 for simplicity, we create a MIN value
  • MIN = 250 (inverse of -100 * 2.5 (= GD weight value))
Assuming the best tactic GD, let's say 100 for simplicity, we can now compute

TOT = max(TP) * 7.5 + max(GD) * 2.5 + MIN

Finally, divide by x (e.g. 100) to get ratings over 100 and later use stars
  • Divide TOT by 100, then floor it (i.e. getting the largest integer that is smaller or equal to TOT)
Adjusting the numbers, we empirically define what's the minimum rating (out of 100) needed to get 5 stars
  • Assuming >75 is needed for 5 stars
  • We assign one point on the 0 -> 10 scale based on each rating band you surpass
  • One point = half star

This should explain the current results you see. By the way, I'm not surprised as the vast majority of tactics is built using subtop teams!

Basically, when using an underdog team in such a competitive environment without randomness, it is incredibly unlikely you produce such high ratings that we assign 5 stars to an underdog tactic!

The stars reflect the actual result the tactic brings in the test.

Take as an example the Leicester championship back in 2016. If we computed 5000 simulations of the Premier League in that year, only once Leicester would have won! That's 0.0002% - and it accounts for randomness, if we take it out of the equation as we strive to do, the odds are even smaller!

I hope this clarifies!
Click to expand...

Yes that clarifies a lot, thanx! I thought the rating system was using different values for underdog or subtop tactic, so i didn't understand the difference in rating. But it's clear now.

Are you planning to add more categories, for instance elite teams, or relegation candidates ?
 
Pianist said:
Are you planning to add more categories, for instance elite teams, or relegation candidates ?
Click to expand...

In the long term plan, we are discussing edits of the testing database for future FM versions. At the moment, we have a lot to handle in doing this platform merge.
When we arrive at a stage when we have satisfying solutions proposed by our editors' team, we will discuss with the community which modifications make sense to implement!
In the end, we can make the most complex test save ever, but if it's not "consumable" by the community, it doesn't make sense :D
 
Luigi said:
In the long term plan, we are discussing edits of the testing database for future FM versions. At the moment, we have a lot to handle in doing this platform merge.
When we arrive at a stage when we have satisfying solutions proposed by our editors' team, we will discuss with the community which modifications make sense to implement!
In the end, we can make the most complex test save ever, but if it's not "consumable" by the community, it doesn't make sense :D
Click to expand...

I feel like we already have a database with all these categories sat there, waiting

?
 
i've made a tactic testing with my team 2 days ago used the former top 2 tactics Bobot&Aviator (7 test for each total of 14) and with both tactic i got league title 7 times and always above 90+ pts with my team (our league got 20 clubs and total of 40 matches) so testing seems very good ATM

Edit:Btw my team predicted to finish 4th
 
There will be an update this evening with some quality of life features. ?
 
Hi Liam, are all tactics uploaded tested? On average, how long does it normally take to test a tactic once uploaded? Thanks
 
CVP87 said:
Hi Liam, are all tactics uploaded tested? On average, how long does it normally take to test a tactic once uploaded? Thanks
Click to expand...

It's hard to say yet. We have a capacity of 24 tests per day right now.
 
CVP87 said:
ah ok cheers Liam, is there a way we can request it or is it down to you guys to make the decision?
Click to expand...

For the time being the order by which tactics get tested is purely random.
 
CVP87 said:
ah ok cheers Liam, is there a way we can request it or is it down to you guys to make the decision?
Click to expand...

I feel like you wanna know if all tactics will be eventually tested: the answer to that question is yes. :D
 
You must log in or register to reply here.

FMBase

Beta

Resources

Back
Top