I'm assuming you're a really small club with pretty much no money? Unfortunately, it's just part of the game. As others have said, if your finances are bad you're always going to be at risk of that happening. Staying in your budget and not setting ridiculous bonuses tends to ensure your board leave you to handle all transfers, though.
What I'd suggest is you put an asking price on all of your top players, something that might scare off the clubs sniffing around. I tend to put an asking price of about £500,000 on my players if I'm in LG2 or below, which normally does the job. However, if an offer comes in for that kind of money when the player is worth £30,000 or something I take the money. You can completely rebuild a side and still have a few quid left in the bank; but then, I do always tend to start in the Blue Square Premier or below. If you're flying high at the top of League One and the board sell your main goalscorer, I can see how that might stick in your craw!