The Difference Between Tactic Testing Leagues

[Mr Hough83]

I want to provide Information regarding the tactic testing leagues/tables whatever you want to call it. At the moment of writing this thread there are three different tactic testing leagues.

TFF's 75 Siblings, Mr L and FM Base.

TFF's 75 Siblings uses a system based on a Team's Reputation - Underdog and Favourite. That's the only difference between the two sides.

• One has a Lower Reputation (Underdog)
• One has a Higher Reputation (Favourite)

This will cause the AI to act differently to you. For example, they will be more defensive against a higher reputation, while playing more attacking/natural game against lower reputation.

Both sides in terms of ability are the same and I can prove this in the screenshots below:

TFF's Hot Tub PLAYERS FAVOURITE TEAM	TFF's Hot Tub PLAYERS UNDERDOG TEAM

These are just two players for each Favourite and Underdog team. I've also included the players default profiles for comparison, from the standard FM Database as well.

Default Attributes in the FM Database

TFF's Hot Tub PLAYERS FAVOURITE TEAM	TFF's Hot Tub PLAYERS UNDERDOG TEAM

As you can see the TFF's 75 Siblings players are a bit overpowered especially compared to their default counterparts. Thus, you can see where potential problems can lie regarding tactic testing. For examples each individual player is able to create something out of nothing due to their flair, or being able to shoot on target from distance due to having such high attributes. Generally, in a player's FM save to, they're not going to have such players, while having such a low reputation. Adding further context to this, imagine multiple Messi's and Cristiano Ronaldo's playing for Norwich.

Mr L's testing approach is very similar to the theory we use at FM Base, however there are differences. For example, their sub-top team has an average CA165 and underdog have an average CA140. The teams within the league are also different. Outside of this, not much else is known.

Onto our approach at FM Base...

Our tactic tests are based on CA (Current Ability) we have two teams:

• Subtop - which have a CA of 145
• Underdog - which have a CA of 120

Below are screenshots I have provided for each team:

FM BASE PLAYERS CA145 TEAM	FM BASE PLAYERS 120 CA TEAM

We believe this gives a more realistic approach and should match better when you use these in your FM saves. If you'd like to have a further in-depth look into our testing method we have also provided its own thread which includes key highlights. We'd also love to have your feedback on our testing method to!

Conclusion
As you can see in the screenshots, teams on each test are very different. Thus, the cause of different results.
Which one is right? That's not for me to say I'm just showcasing the differences for each method.

Thank you for reading and most importantly your precious time.

 

[magicnut]

Lad it doesn't seem to me you put too much effort honestly to even write this "article".
" and i can prove this in the Screenshots here. " I think the vast majority of us know and theres no need to "prove" this as this is no secret. TFF's Hot Tub made that crystal clear. The whole point of this is to reflect how the AI adapts when you are the favorite or underdog. Under their test league , both teams with the same CA often perform dramatically different when they are deemed favorite.
"There are just to players for each Favourite & Underdog Team " what does that even mean?
"and you can see where the problems can lie regarding tactic Testing. " No I cannot see can you elaborate??

The players of fmarena are a bit overpowered testing against Barca , Juve , Liverpool etc with their Messis and whatnot , Neymars? Disagree.
Also I heard a notion that "almost every tactic wins the league as underdog" on fmarena , that is simply not true..Only about maybe the top 6 won the league.

 

[Liam]

Lad it doesn't seem to me you put too much effort honestly to even write this "article".
" and i can prove this in the Screenshots here. " I think the vast majority of us know and theres no need to "prove" this as this is no secret. TFF's Hot Tub made that crystal clear. The whole point of this is to reflect how the AI adapts when you are the favorite or underdog. Under their test league , both teams with the same CA often perform dramatically different when they are deemed favorite.
"There are just to players for each Favourite & Underdog Team " what does that even mean?
"and you can see where the problems can lie regarding tactic Testing. " No I cannot see can you elaborate??

The players of fmarena are a bit overpowered testing against Barca , Juve , Liverpool etc with their Messis and whatnot , Neymars? Disagree.
Also I heard a notion that "almost every tactic wins the league as underdog" on fmarena , that is simply not true..Only about maybe the top 6 won the league.

"The vast majority" is an overstatement. Join the Mr L discord and see what people say about TFF's Hot Tub. Tacticians such as Knap are not a fan of their testing method and it isn't just because they lose, it's because the reputation system is fundamentally flawed in comparison to the sub-top and underdog method created initially by Mr L.

 

[Dec]

The aim isn't to discredit TFF's Hot Tub at all, they have their viewpoint on how tactics should be tested and we have ours. What we want to do is show people how we test ours against TFF's Hot Tub in order to get feedback and improve our testing.

For full clarity, the issue with TFF's Hot Tub's testing in our view is:
* It doesn't provide a realistic, in-game test as it's unlikely that players for those reputations will be as good as TFF's Hot Tub's test.
* It means that tactics based on working for a lower league club won't be highlighted as strongly as those that work for a top club due to the player's being able to play in more demanding roles than what is expected
* We want to reward the most all-rounded tactics, with an ability to view based on who you are managing to match a user with a tactic. Whereas this doesn't seem to match the way TFF's Hot Tub display theirs.

 

[Dubhlaiin]

Ultimately, it depends on what game you'll play.
If you play with Liverpool, Juventus, Barcelona and whatnot, the TFF's 75 Siblings results might be more interesting to you because you'll know what tactic to use and when, because fundamentally your team will not get much worse or much better without major overhaul, so the variable will be your club's reputation relative to your opponent's reputation, which is the point of TFF's 75 Siblings testing.
On the other hand, if you want to play LLM and climb your way through the league system, or if you play with a top club from a smaller country, the underdog testing of MrL and fm-base will be more useful to you, since it represents, in a way, your newly promoted side in the higher league.

All in all, there's no wrong or right way to make a testing league, both ways have their uses, and no need to create drama, there's enough drama already in the FM scene.

 

[Dec]

I don't see it as drama @Dubhlaiin, we want to make sure our testing is as useful as possible to the majority. Both ours and FM Arenas have pros and cons, like all will do, however if we can open the discussion to find what the community thinks, then hopefully we can minimise the cons.

 

[Dubhlaiin]

Oh, not necessarily on this thread, but there were some people wondering why their favourite tactic didn't work as well as intended and saying that there must be something wrong with the set-up

 

[Mr Hough83]

Lad it doesn't seem to me you put too much effort honestly to even write this "article".
" and i can prove this in the Screenshots here. " I think the vast majority of us know and theres no need to "prove" this as this is no secret. TFF's Hot Tub made that crystal clear. The whole point of this is to reflect how the AI adapts when you are the favorite or underdog. Under their test league , both teams with the same CA often perform dramatically different when they are deemed favorite.
"There are just to players for each Favourite & Underdog Team " what does that even mean?
"and you can see where the problems can lie regarding tactic Testing. " No I cannot see can you elaborate??

The players of fmarena are a bit overpowered testing against Barca , Juve , Liverpool etc with their Messis and whatnot , Neymars? Disagree.
Also I heard a notion that "almost every tactic wins the league as underdog" on fmarena , that is simply not true..Only about maybe the top 6 won the league.

Let me reply to what your are saying

Regarding Honesty, I haven't been Dishonest or Honest, I'm just giving my opinion regarding Tactical Testing Leagues, Everyone's opinion is different and that's what great about debate

Instead of maybe TFF's Hot Tub saying Favourite and Underdog they could Use Higher Reputation and Lower Reputation as that's Essentially all they are testing, They are testing if the AI Change's regarding which Reputation they play and of course they do, But if the 'underdog' team is as strong as the 'favourite' team then thats going to be a flawed test, The AI is going to attack because they seem it as an Underdog Side in the Game yet the Underdog Side has such a strong side to counter them.

This Quote is taken from TFF's Hot Tub themselves - AI mangers don't look at your team CA at all when they play against you, the only thing they care about is the difference between your team reputation and their team reputation.

So let me give you an Example - Barcelona is Playing the Underdog side - AI Manager thinks "look at the Reputation of this team we can easy attack and Beat them" not looking at how good the underdog side is and you can see in the Screen Shots i have provided the actual Underdog side is just as good as the favourite side, This is Flawed, It just is. It wouldn't happen and you shouldn't basis tests on this. This could also be why the Underdog's Out Perform the Favourite teams on a regular basis.

 

[magicnut]

Let me reply to what your are saying

Regarding Honesty, I haven't been Dishonest or Honest, I'm just giving my opinion regarding Tactical Testing Leagues, Everyone's opinion is different and that's what great about debate

Instead of maybe TFF's Hot Tub saying Favourite and Underdog they could Use Higher Reputation and Lower Reputation as that's Essentially all they are testing, They are testing if the AI Change's regarding which Reputation they play and of course they do, But if the 'underdog' team is as strong as the 'favourite' team then thats going to be a flawed test, The AI is going to attack because they seem it as an Underdog Side in the Game yet the Underdog Side has such a strong side to counter them.

This Quote is taken from TFF's Hot Tub themselves - AI mangers don't look at your team CA at all when they play against you, the only thing they care about is the difference between your team reputation and their team reputation.

So let me give you an Example - Barcelona is Playing the Underdog side - AI Manager thinks "look at the Reputation of this team we can easy attack and Beat them" not looking at how good the underdog side is and you can see in the Screen Shots i have provided the actual Underdog side is just as good as the favourite side, This is Flawed, It just is. It wouldn't happen and you shouldn't basis tests on this. This could also be why the Underdog's Out Perform the Favourite teams on a regular basis.

Firstly "honesty" lad I didn't say you were dishonest. Perhaps you need to read what I say more carefully."Lad it doesn't seem to me you put too much effort honestly to even write this "article". " : so meaning I HONESTLY think that you didn't put the effort.That was the context of what I am saying.

Now: me standing behind the fmarena method of testing is because it shows you objectively how much your lets say odds of winning affect your actual chance to win.
Before the patch there was no real problem creating a tactic that works BOTH underdog and favorite. Now there is. As far as the language used I personally don't think there's much ambiguity in what FMArena labels it to be but they do explain the methology of their testing if one was to be confused .

Now a team could be on paper a favorite and in practice have a weak squad or vice versa , I personally don't think that flaws the nature of the test since the test just objectively shows the differences of how your "reputation" affects the adjustments of the ai Tactics and what it does to your tactics.
That being said this is my opinion and you have your own. If you like the FMBase method of testing and you believe in it then by all means go with it . All I am saying is I don't buy any of your arguments so far justifying it but thats my own personal judgement , thanks.

 

[Danwolf]

Firstly "honesty" lad I didn't say you were dishonest. Perhaps you need to read what I say more carefully."Lad it doesn't seem to me you put too much effort honestly to even write this "article". " : so meaning I HONESTLY think that you didn't put the effort.That was the context of what I am saying.

Now: me standing behind the fmarena method of testing is because it shows you objectively how much your lets say odds of winning affect your actual chance to win.
Before the patch there was no real problem creating a tactic that works BOTH underdog and favorite. Now there is. As far as the language used I personally don't think there's much ambiguity in what FMArena labels it to be but they do explain the methology of their testing if one was to be confused .

Now a team could be on paper a favorite and in practice have a weak squad or vice versa , I personally don't think that flaws the nature of the test since the test just objectively shows the differences of how your "reputation" affects the adjustments of the oi Tactics and what it does to your tactics.
That being said this is my opinion and you have your own. If you like the FMBase method of testing and you believe in it then by all means go with it . All I am saying is I don't buy any of your arguments so far justifying it but thats my own personal judgement , thanks.

But you have tried several times that TFF's Hot Tub is your preference on multiple different threads. In an ideal situation take the best of both worlds and you'll gain the experience of both.
Each testing method provides results that will be useful to one and maybe not the other. I was going to message you about this to, but I'll say it here, you do need to calm down in regards to posting in people's tactics threads stuff like "Needs to be tested on fmarena".

If you genuinely have feedback regarding the FM Base testing, then let your opinion on how we can improve be shared in the Tactic Testing Feedback thread.

One flaw I would for certainly say in the TFF's Hot Tub varitation of testing, it favours the counter/defensive approach tactically since teams will be pushing you more. I also share the same thought as Hough that the teams shouldn't be called Underdog / Topdog to, but instead Lower/Higher repuation. It would help paint a clearer picture for those looking but that's a different matter.

All Hough has done here, is tell people the difference in case they were unclear or unsure.

 

[Fenech]

It is important to establish the difference because no system is perfect, and you should never put all your eggs into one basket, and on that note I would say to only test in one way in never going to give you a full tactical picture, you have to look at and embrace the different types of tests. I am positive that TFF's Hot Tub testing paints a tactical picture and the FMBase testing paints a different picture still and each one will give you different insights into what happens in certain scenarios. And there are lots of different pictures and sometimes multiple tests before you can get a real idea of how good a tactic is and in what situation it will perform best of all. FM base is not attacking FM Arenas testing at all. They have looked at it and had the vision to see how it may be done differently and is looking to improve it even more and is reaching out to the community for discussion and debate regarding how testing can be improved even more. That is a good thing because the moment you think that your testing is definitive and can't be improved on, you become a closed book and close your mind off to anything different. The very fact that there are now different types of tests running is a massive help to the FM Community and In my opinion FMBase should be applauded for making a massive effort here with tactical testing. ? ?

 

[Danwolf]

I've stuck this thread at the top as it should provide some insight.

 

[Dec]

Firstly "honesty" lad I didn't say you were dishonest. Perhaps you need to read what I say more carefully."Lad it doesn't seem to me you put too much effort honestly to even write this "article". " : so meaning I HONESTLY think that you didn't put the effort.That was the context of what I am saying.

Now: me standing behind the fmarena method of testing is because it shows you objectively how much your lets say odds of winning affect your actual chance to win.
Before the patch there was no real problem creating a tactic that works BOTH underdog and favorite. Now there is. As far as the language used I personally don't think there's much ambiguity in what FMArena labels it to be but they do explain the methology of their testing if one was to be confused .

Now a team could be on paper a favorite and in practice have a weak squad or vice versa , I personally don't think that flaws the nature of the test since the test just objectively shows the differences of how your "reputation" affects the adjustments of the ai Tactics and what it does to your tactics.
That being said this is my opinion and you have your own. If you like the FMBase method of testing and you believe in it then by all means go with it . All I am saying is I don't buy any of your arguments so far justifying it but thats my own personal judgement , thanks.

So what if we said (and I’m not sure how possible this is yet) that we find an average reputation for teams in the CA bracket listed and test both the CA and reputation being modified. Would this be the fairest test?

 

[magicnut]

So what if we said (and I’m not sure how possible this is yet) that we find an average reputation for teams in the CA bracket listed and test both the CA and reputation being modified. Would this be the fairest test?

That would be interesting.

 

[Oritron]

I would simply put reputation number 1 or 10 000 for all teams. Would be interesting to watch AI picking tactic against every team. I guess there will be a lot of cautios approach therefore more draws, fewer goals. That are just my assumptions

 

[ByH]

가장 공정한 방법

 

[Danwolf]

가장 공정한 방법

Translated: The fairest way

 

[victorhugo4222]

Hey Guys, 1st off hope everyone had a fabulous xmas.

Not sure if this is the right place to post this but since it regards testing i thought i would post it here.

I was thinking since set pieces have played a big part in fm over the last few seasons would it be possible to set up some sort of testing table for set pieces? Not sure if it is even possible but was just an idea.

 

[Liam]

Hey Guys, 1st off hope everyone had a fabulous xmas.

Not sure if this is the right place to post this but since it regards testing i thought i would post it here.

I was thinking since set pieces have played a big part in fm over the last few seasons would it be possible to set up some sort of testing table for set pieces? Not sure if it is even possible but was just an idea.

It's something we've been looking at

 

[victorhugo4222]

It's something we've been looking at

As sad as this may sound while i was eating my xmas pudding this idea came to my mind haha

But would be great if it was possible.