TotalFootballFan9
Member
- Joined
- Dec 10, 2013
- Messages
- 4,113
- Reaction score
- 201
- Points
- 63
THIS TESTING LEAGUE CAN BE DOWNLOADED AT THE LINK BELOW
Click at this link --> http://www.fm-base.co.uk/forum/share-download-fm-17-tactics/365800-tffs-tactic-testing-league.html
Click at this link --> http://www.fm-base.co.uk/forum/share-download-fm-17-tactics/365800-tffs-tactic-testing-league.html
Hey guys,
I’m sure it’s obvious for everyone who play FM that you just can’t know how good your tactic is without giving it a comprehensive testing because I’m sure that any FM player has encountered a situation when on paper his tactic looked amazing but when he used it then the results were poor.
In this thread I want to share my experience with you how to create a testing system that allows you to spend the minimal possible of your time and efforts on testing and at the same time to get the most accurate representation of your tactic’s quality.
Soon, I’ll open another thread where I put a download link and a guide “how to use” for a testing system that was build on principles that will be explained in this thread, I don’t want to do that in this thread because I think it would made this thread look too messy and to complicated.
Three main principles of an efficient tactic testing system
1) Providing all tactics with “PROPER” players for them:
The first main principle of an efficient testing system is to provide all tactics in the testing with PROPER players for them and if this condition isn’t met in your testing system then any testing in such system is just waste your time because testing tactics without “proper” players for them it’s the same as testing a Video Card without proper drivers for it, everyone understand that testing Video card without proper drivers for it is a stupid thing to do because in this case the test result couldn’t have any value.
Players for tactics is like drivers for Video Cards and if you test a tactics without “proper” players for them it’s the same as you test Video Cards without proper drivers for them.
For example, let’s say you test a tactic that uses a very high D-Line that means it requires using fast defenders for this tactic, the defenders for this tactic might not be very strong or very tall but they must be fast but it happened that there are no fast defenders in your testing team and all defenders in your testing team are strong, tall but slow and due to that this tactic showed a very poor defensive record in the test and as result a poor total result in the test.
Another example, let’s say you test a tactic that uses Inside Forwards roles for the wide positions and it means that for these positions you need players that have good Passing and Vision attributes to deliver through ball passes for the striker and they should have good Finishing and Long Shots attributes to score goals but it happened that there are only Wingers in your testing team, they have good Crossing attribute but poor Passing/Finishing attributes and due to that this tactic showed a very poor attacking record in the test and as result a poor total result in the test.
One more example, let’s say you test a tactic that uses Advanced Forward role for the striker position and it means that it requires using a fast striker with good Dribbling attribute for this tactic but it happened that there are only slow and strong strikers in your testing team and due to that this tactic showed a very poor attacking record in the test and as result a poor total result in the test.
As you can see testing tactics without proper players for them is complete waste of time.
2) Providing all tactics with “EQUAL QUALITY” players:
The second main principle is to provide all tactic in testing with “EQUAL QUALITY” players.
As it can be seen from the previous chapter any tactic testing makes sense only if you provide all tactics with “proper” players but it’s also very important that these “proper” players were “equal quality” players and here’s why: let’s say we picked Messi(a fast striker) and Romelu Lukaku(a strong and tall striker) in our testing team and during our testing if some tactic uses Target Man role then we pick Romelu Lukaku and if some tactic uses Advanced Forward role then we pick Messi, all that seems logical and fair but of course, such testing wouldn’t be fair because Messi is much better player than Romelu Lukaku and all tactics with Advanced Forward would get huge advantage due to Messi presence.
So as you can see providing tactics with proper players isn’t enough, it’s also very important these “proper” were “equal quality” players.
Some people think that if you pick players with the same CA for the testing team or set the CA of all players in the testing team at the same level then it’ll make all players in the testing to be equal good in all Role but it’s a complete delusion because CA is just a total amount of points that a player has to spend on his attributes but his efficiency in Roles depends not only exclusively on his total CA amount but it also depends on the way his CA is spent/allocated.
In addition, many people don’t know that different attributes have different price for different positions, for example, if we set identical attributes for a striker and a defenders then their CAs will be different even that they have identical attributes because for a defender many attributes cost less CA that for a striker.
Also, many people don’t know that a player’s ability to play on many different positions costs additional CA.
It easily could be that Player “A” has lower CA than Player “B” but despite that Player “A” is a much better Winger than Player “B” because the CA of Player “A” is spent/allocated in more efficient way for a Winger than the CA of Player “B”, for example, the CA of Player “A” is spent/allocated into very important attributes for a Winger such as Crossing/Speed/Flair/Agility and so on and CA of Player “B” is spent/allocated into not very useful attributes for a Winger such as Strength, Positioning, Tackling, Jumping Reach and that’s why Player “A” is a better Winger than Player “B” even that the CA of Player “A” is lower than the CA of Player “B”.
So we can’t say that two players are equally good Wingers only because they have the same CA and we can say that two players are equally good Wingers only if they have the same attributes, the same ability to use their main/weak foot and the same PPMs.
3) Removing factors that add randomness to the test results
All factors that add randomness to the test results should be removed or minimized. I’m talking about such factors as:
- Injuries
- Transfers during the season
- Managers movements during the season
- and many other factors
Picking players for the Testing Team
I’m sure that anyone who read the previous chapters understand that testing tactics with any default team from the game is just a waste of time because there are simply no such teams in the game that have proper and equal quality players for all roles and tactical approaches and if you test tactics with any default team from the game then it inevitable happens that some tactics get advantage over other tactics in the test due to presence of more suitable players for them and that greatly affects the test results making them inaccurate.
Also, there are no equal quality players in the game, all players in the game have different quality and as it was explained in the previous chapter you can’t create a testing team that consist of equal quality players by picking players with the same CA into it or setting the CA of all payers in the testing team at the same level so the only way to get equal quality players is to create them using the official pre-game editor.
1) Creating a “universal” Attacker for the Testing Team
Let’s create for the testing team an “universal” attacker that will be equally good in all attacking tasks such as: dribbling, passing, crossing, moving without ball, scoring goals and so on; and in all defensive tasks such: marking, tackling, positioning and so on.
We need such “universal” attacker in order to test many different tactics without fear of some tactics get advantage in the test due to the attackers in the testing team suite them better than other tactics and all that effects the test results making them inaccurate.
In order to achieve that we need to set some Technical/Mental attributes of the “universal” attacker at the same level and some of his Technical/Mental/Physical attributes we might set at different level to tune his general quality.
For example, such attributes as Technique, Composure, First Touch, Decision and some other attributes have almost the same importance for all attacking roles so we can set them for our “universal” attacker at any level we want without fear of making him to be more efficient in some attacking roles than other attacking roles.
But we can’t set such attributes as Passing, Crossing, Finishing and some other attributes to different levels for our “universal” attacker because these attributes have significantly different weight for some roles, for example, if we set Crossing attribute at higher level than Passing for our “universal” attacker then it’ll make our “universal” attacker to be more efficient in Winger role than in Advanced Playmaker role but we need that our “universal” attacker to be equally efficient in all attacking roles.
Of course, some of our “universal” attacker should be right footed and some of our universal attacker should be left footed.
Here’s the attributes of the “universal” Fast Attacker that is used in the testing team:
Here’s the attributes of the “universal” Strong Attacker that is used in the testing team:
Here’s their playing positions:
2) Creating a “universal” Central Defender for the Testing Team
Our “universal” Central Defender will be used to test many different tactics and some of these tactics will be using a high D-Line and some will be using a low D-Line so the Physical attributes of our “universal” central defender should be well rounded, he shouldn’t be too slow in order not to put tactics that use a high D-Line into disadvantage and he shouldn’t be too short or too weak in order not to put tactics that use a low D-Line into disadvantage.
Here’s the attributes of the “universal” Central Defender that is used in the Testing Team:
Later when I have more free time I’ll explain what principles should be used for creating players for other positions in the Testing Team.
Picking AI teams for the Testing League
I’m sure that everyone agree with this slogan – “Time is money”; so everyone want to spend their time in the most efficient way when they test tactics, it means that the result of every match that is played during testing should give us the maximum possible information about the tactic’s quality that we test and in order to achieve that these conditions should be met:
1) Our testing team shouldn’t be stronger than any of AI teams in the testing league because when your team is stronger than the opposition side and you win a match against this opposition side then you can’t say for sure that you won that match due to your tactic’s quality because it might be that you won that match due to having better players than the opposition side but we want to know for sure we winning matches due to the quality of the tactic we test.
2) We can’t spend our time on playing against average/weak AI managers
3) We can’t spend our time on playing against AI teams that consist of average/weak players
4) We should play only vs the best AI mangers and the best players in the game.
So we pick the best 16 teams in the game into our testing league, these teams consist of the best players in the game and these teams are under control of the best 16 AI managers in the game, also we tweak the preferred formations of some these managers to increase variety of formations in our testing league and make the balance of formations to be somehow in line with the best league in the game such as English Premier Liga, Liga BBVA, Bundesliga, Seria A and some other top leagues.
Such Testing league is the most challenging test that you can get in the game and if a tactic can do good in it then this tactic can do good in any other league because anything else would be just a weaker version this league.
1) Barcelona
2) Real Madrid
3) Bayern
4) Juventus
5) Man City
6) Chelsea
7) Paris Saint-Germain
8) Man Utd
9) Arsenal
10) Milan
11) Liverpool
12) Atletico Madrid
13) Sevilla
14) Tottenham
15) Inter
16) Porto
“Underdog” and “Favorite” simulation
In the previous chapter it was said that if we want to test in the most efficient way then our testing team shouldn’t be stronger than any AI team in our testing league because when your team is stronger than the opposition side and you win a match against this opposition side then you can’t say for sure that you won that match due to your tactic’s quality because it might be that you won that match due to having better players than the opposition team but we want to know for sure that if we are winning matches then it happens due to the quality of the tactic that we test.
So if our team is the weakest team in the testing league then all AI teams use attacking approaches against our testing team and that means that we won’t find out how the tactic that we test performs when it’s used with a strong team when AI teams use defensive approaches against you.
The trick is that it isn’t necessary to test a tactic with the strongest team and the weakest team in the league in order to find out how the tactic performs when it’s used with a underdog or a favorite team, you can find out it testing with the same team and the same players but the only thing you need to do is change your team’s reputation before the start of the test because AI don’t look at your players’ quality when he chooses what approach to use the only what he looks at is your team’s reputation.
Every team in the game has Reputation, it varies from 0 – 10 000, it’s used for many things in the game and as I said AI uses it to determine what approaches to use against you, he compares his team’s reputation with your team’s reputation and if your team’s reputation is higher than his team’s reputation then AI thinks that you are the favorite and he’s the underdog so he’ll be using defensive approach against you.
For example, Barcelona’s reputation is about 9000 and if we edit its reputation to 3000 then all AI teams in Liga BBVA will start to consider Barcelona as a very weak underdog team and use very attacking approaches against it.
So we can test with the same team but make 2 different save:
1) in the 1st save our testing teams has Reputation that is much lower than any AI teams’ reputation which means that AI teams consider our testing team as a weak underdog team and they use attacking approaches against our testing team so when we test a tactic in this save then we find out how the tactic performs when it’s used with an weak underdog team.
2) in the 2nd save our testing teams has Reputation that is much higher than any AI teams’ reputation which means that AI teams consider our testing team as a strong favorite team and they use defensive approaches against our testing team so when we test a tactic in this save then we find out how the tactic performs when it’s used with an strong favorite team.
There are Testing Team A and Testing Team B in the testing league; Testing Team B is just a copy of Testing Team A; I added 2 identical testing teams into the testing league in order to test for double amount of matches per test.
Removing Injuries from the game
Yes, it’s possible completely removing Injuries from the game using the official pre-game editor
“Freezing” the Condition and Morale of all players in the testing league
I’m sure that it doesn’t require any excessive explaining that “freezing” the morale and conditions of all players in the testing league at the same level for all matches greatly reduces randomness of the test results.
It might be done by using in-game editors and I recommend using FM Fitness Coach by Panos99, it’s the best program for it
Making AI teams to use the same starting elven for all matches
The randomness of test result greatly reduces when all AI teams uses their best starting eleven for all matches during the season.
For example, this is the best starting eleven of Barcelona team
In order to make the AI manager to use this best starting elven for all matches during the season you should do these steps:
- remove all other players from the team
- edit Messi’s playing position and make him capable of playing only at AMR position
- edit Neymar’s playing position and make him capable of playing only at AML position
- edit Suarez’s playing position and make him capable of playing only at STC position
- edit the other players’ playing positions the same way as described above
- duplicate every player from the starting eleven that AI manager could use “duplicated” players for substitution during matches or if some player gets banned due to red card or excessive amount of yellow cards.
Preventing AI managers and you from being sacked due to poor results before the season ends
It inevitable happens every test that some of AI teams or even the testing team shows poor result at some point during it and the board decides to sack the manager.
All that brings great randomness into the test results because some of AI teams might randomly change its manager during the season and it means that these teams start to use different tactics and it’ll have impact on the test results.
In FM16 I already explained how to prevent you and AI managers from being sacked until the season ends so if anyone is interested then he can search this forum for my explanation how to do that and at some point later when I get more free time I might put it here.
Editing Age, PA, Professional and Ambition of all players in the testing league to make their attributes stay the same during the season
It greatly reduced randomness of the test result if you make all players in the testing league to have the same attributes during the season, it can be achieved by setting their Age, PA, Profession and Ambition at such level that prevents their CA from growing or declining.
Choosing how to play matches: Commentary Mode, Instant Result or On Holiday mode
Probably, many of you guys remember that for FM16 and FM15 I made few threads at this forum where I warned people that when you use Instant Result or On Holiday mode to play matches then your assistant manager might tweak your tactic or even use other tactics to play matches and I posted clear proves of that.
Those issues were reported to SI and for FM17 they fixed it so now you can use Instant Result or On Holiday mode to play matches and don’t be afraid that your tactic won’t be used to play matches and if you see that some person says opposite then don’t listen this person, he is simply not aware about things in FM17.
I must add that in FM17 I don’t notice that the assistant manager changes my tactic when I play matches with Instant Result or with On Holiday mode but sometimes I notice that when I play matches with Instant Result or On Holiday mode then the assistant manager might swap players during the matches, it doesn’t happens every match but it sometimes happens, for example, in my Demolisher tactic I use a right footed player at AMCL position and a left footed player at AMCR position and I notice that sometime the assistant manager swap the player at AMCR position with the player at ACML position because AI always prefers to use left footed players at the left side and right footed players at the right side.
Now back to the main question: how to play matches ?
I exclude using Commentary Mode for testing because it takes ages and requires a lot of efforts so there are left On Holiday Mode and Instant Result and I suggest using On Holiday mode for testing because in FM17 there’s no different between On Holiday and Instant Result when it comes to simulate matches but On Holiday Mode is faster and doesn’t require any efforts you can simple went On Holiday and do your business, on my PC it takes about 35 minutes to finish the season(160 matches) using On Holiday mode.
Only don’t forget to tick these 2 options when you go On Holiday
1) Use current match tactics
2) Use current team selection when possible
List of other modifications that should be made to remove or to reduce random factors in the testing league
- Set “Consistency” attribute at “20” for all players in the testing league.
- Set “Dirtiness” attribute at “1” for all players in the testing league (it greatly reduces number of straight red cards during matches).
- Set “Transfers Embargo” for all teams in the testing league.
- Set “International Retirement” for all players in the testing league.
- Set “Retiring After Spell At Current Club” for all players in the testing league (it prevents them from complaining about such stuff as lack of the first team football and other).
How long does it takes to test for one season
I’ve got a good PC - Processor i5 2500, SSD and 8Gb RAM and it takes about 35 minutes to test for one season and after that you get complete statistic for the tactic :
- Goals From Corners
- Conceded From Corners
- Sored From Free-Kicks
- and so on
Also, you get detailed information about distribution of Goals and Assists across the positions in the tactic that you test
How random is this testing system
I tested Strikerless V13 tactic for 10 times in this Testing League, here are the results:
There are Testing Team A and Testing Team B in the Testing League; Testing Team B is just a copy of Testing Team A and I added 2 identical Testing Teams into the testing league in order to test for 170 matches per one test. You should simply sum up the points and goals difference of the Testing Team A and Testing Team B.
Also, please notice that Testing Team A and Testing Team B play 5 times with each other during the season and if there no any draw result between them during the season then it adds + 15 points to the total score and if there are 5 draws between them during the season then it only adds +10 points to the total score; due to that every test result might be randomly increased by additional 10 or 15 points but of course, in average this amount will be between 10 and 15 points and that won’t have any significant impact on the result.
On my PC it took about 6 hours to test for 10 times, 35 minutes per test.
Average Results After 10 Test ( 1700 matches played )
Average Points = 315,5
Average Goal Difference = 145,6
Average Points = 315,5
Average Goal Difference = 145,6
Test 1 ( Pts = 315 ; G.D. = +157 )
View attachment 93441
Test 2 ( Pts = 314 ; G.D. = +128 )
View attachment 93440
Test 3 ( Pts = 310 ; G.D. = +158 )
View attachment 93439
Test 4 ( Pts = 312 ; G.D. = +158 )
View attachment 93438
Test 5 ( Pts = 346 ; G.D. = +188 ) - << THE BEST SCORE >>
View attachment 93437
Test 6 ( Pts = 310 ; G.D. = +128 )
View attachment 93435
Test 7 ( Pts = 311 ; G.D. = +147 )
View attachment 93434
Test 8 ( Pts = 323 ; G.D. = +149 )
View attachment 93433
Test 9 ( Pts = 287 ; G.D. = +102 ) - << THE WORST SCORE >>
View attachment 93432
Test 10 ( Pts = 327 ; G.D. = +145,6)
View attachment 93431
As you can see If we test only 4 times and saw only the result of Test 1, Test 2, Test 3 and Test 4 then we might got impression that this testing system is perfect because there’s almost no any difference in the test results after each test but further testing showed that the worst result might be 287 points and the best result might be 347 so in theory if you test only for 1 season then you might hit the worst and the best score and the difference between them is 60 points but as you can see the probability to do that is very low.
So to be safe you need to test at least for 2-3 seasons and if you want to get more accurate results then you need test for 10 seasons or more.
You simply can’t make a testing system that is less random than this system and if you see that some person say that his testing system is less random than this then he is lying or he hasn’t tested for enough seasons to see this randomness.
I don’t say that it isn’t possible to create system that will be less random that this but it can be improved not more than 10-15%.
Of course, there might be other approaches to testing, for example, you can save before some match and then repeating this match for many times taking records of the results but even such approaches is less random but it’s much more time consuming than this testing system so I don’t it’s worth efforts.
Also, it’s possible to create a testing system that won’t have any randomness in the results at all, for example, you can achieve that if the testing team is much stronger or much weaker than any AI team in the league so if the testing team is much stronger than any AI teams in the league then you’ll be winning 100% matches and there won’t be any randomness in the results or if the testing team is much weaker than any AI teams in the league then you’ll be losing 100% matches and there also won’t be any randomness in the results but the results of such testing system won’t have value due to obvious reasons. When all teams in the testing league are similar quality teams then there always will be randomness.
Last edited: