Remember when I said:
But then . . . I guess what I'm getting from your most recent post, which I did read, is that this whole thing is just not gonna be any fun. Maybe that's not fair since I didn't read the whole thread.
By the way, reviewing as many as possible is still a select group. Anything less than "all" is, by definition a select group. That's sort of the point of the nominating process. It narrows down the "all" to something manageable. And then, how does it not become a popularity contest? Can you imagine how many people would refuse to vote for a masterpiece I built on principle?
I have full-on enemies here. One guy said once that he was giving up the whole hobby because of me. Which doesn't even make sense because I wasn't even mean to him, but whatever. I know you can relate because of how many people have publicly expressed similar feelings for you.
So, it's gonna be flawed no matter what. Trying to create some system that will actually scientifically identify the best cab, with objectivity pure as the driven snow, is laughable. If you're gonna do it, make it manageable and make it fun. Those are by far the two most important factors (since the other factors are totally illusory anyway).
And if manageable and fun are what you're going for, the Academy Awards actually offer a fairly good template to go by.
Shmokes, there are some really great questions in there. I have my own opinion on how to keep it fun, keep it manageable, and rein in the user popularity test but the best way to flesh out a community solution is to execute this on a trial basis and then see where we are. Anything less than that and I fear we'll dissolve into bickering and end up throwing the baby out with the bathwater.
To address your concerns user to user I am for a popular vote for yearly judges: This may keep some of our users with "full-on enemies" out of the judges circle but the benefit to the project outweighs the consequence of lost experience just because so-n-so is a prickly character that gets in peoples' jell-o too much... or may not. Again, it's a popular vote, this thread has hundreds of views and like 30 votes so it's not going to be perfect.
Category winner selection - Users recommend projects for each category. Panel ensures nothing was left out that they think should be included and does a write-up on the X (3?) that they choose (2/3 of which need to be nominated). Those write-ups go up for a popular vote and the community decides the winner.
Maybe a clear front runner goes in but after the write-ups the underdog gets a more favorable look and comes out ahead? Maybe the write-up doesn't change the vote at all but at least provides some good analysis.
Or even simpler, the nominations is the voting, once it's closed the one with the most nom wins, but the panel still analyses their three pics for a "closer look" a this year's nominees.
When I say good analysis I don't know what that looks like yet. I'm hoping it's fun, I'm hoping it is pointed but a bit tongue in cheek, I'm hoping that people would enjoy reading it. That's why I'm putting mine up for a go so we can iron that out.
I am extremely concerned with the "gentleman's club" but want to have a standard form and consistent analysis of the projects as well. It's a tough line to walk but I'm on that line so hopefully I can get something that Jimmy likes and will read and something that Howard likes and will read even though they have almost didactic points of view.