It's a remnant of the old days when supposedly the baseball writers were the experts because they covered the game. Few, if any, games were on TV. Nobody but them could possibly know these things, right? Then you end up with some voters who might have covered the game for a few years, even in a city without a team, becoming voters, and hey, if they didn't see Lou Whitaker play a lot and never saw him on the TV in highlights, how the hell can he be a Hall of Famer?
It's a good system, right?
In all seriousness, it's out dated. There's so much information available to potential voters that it's odd to leave it in their hands now. And do the smart baseball people even become writers anymore?