That's not correct; club facilities obviously play a large part in how much of a player's potential he will reach, but it is not taken into consideration for scout reports. In a scout report, the potential ability of a player is what he could potentially reach under the best of circumstances; the best he can realistically still obtain at this point of his carreer.
The part about scouts getting it wrong is correct; that's why "judging player ability/potential" aren't just for show and why you should always scout a player multiple times before making decisions; the more often a scout watches a player play, the more correct his evaluation will be and the more information about hidden stats like injury proneness he will discover.