[BudH]

My local ACBL club recently purchased a Dealer4 dealing machine and on at least two days a week, pre-dealt hands are being used where a novice section plays the same hands as the open section with hand records made available.

Today, I stopped at our club for a few minutes and was quickly approached by one of our better players who was very unhappy due to the hands having far more voids, singletons, and long suits than normal - and he is aware that normal shuffling tends to give you flatter hands, but insists the hands played so far are clearly beyond what he's seen when using hand records at ACBL tournaments.

Is there a good way to reduce the fears of the club players who are thinking the computer program generating the random hands has gone completely mad? (I'm thinking of taking all the PBN files in the first month and doing a statistical analysis - but I'd like other suggestions on how to convince suspicious players nothing "funny" is going on.)

And what are the best computer programs for generating hands that are as close to purely random as possible? (Note that being usable by game directors who have very little computer experience is a plus.)

Bud

[hrothgar]

The sample size necessary to validate whether a dealing program has acceptable randomness is enormous.

If you really care about this, inspect the underlying code and see how it was implemented.

If you decide to try to verify hand records, I'd recommend the following:

Ask the player who is complaining what the "worst" feature of the randomly dealt hands.

Are there too many voids...
Are there too many 7+ card suits...

Chose one specific feature

At this point in time, you can run hypothesis test <on a new sample> and see whether the actual is deviating too far from the expected...

[Cascade]

I think the people providing the software ought to have an obligation to make the code available for verification.

[helene_t]

hrothgar, on 2012-November-08, 17:12, said:

Ask the player who is complaining what the "worst" feature of the randomly dealt hands.

Are there too many voids...
Are there too many 7+ card suits...

Chose one specific feature

At this point in time, you can run hypothesis test and see whether the actual is deviating too far from the expected...

Well if he is complaining about too many voids, most likely there really were too many voids in his limited sample.

The alleged problem could be that
1) The software makes too many voids in general. What you need to do, after having heard his complaint, is to generate say 100000 new hands and see if the abnormality can be reproduced.
2) The software made too many voids on this specific occation. You would have to make a very conservative test, then, since it is post-hoc. A p-value of 0.01 is not interesting. If you get 0.00001 but it can't be reproduced then I suppose you could try to find a bug in the code. Obviously this will be a lot more difficult than a statistical analysis.

[gnasher]

BudH, on 2012-November-08, 16:21, said:

Is there a good way to reduce the fears of the club players who are thinking the computer program generating the random hands has gone completely mad?

No. Whatever you do or say, they'll never believe you.

[phil_20686]

gnasher, on 2012-November-09, 04:27, said:

No. Whatever you do or say, they'll never believe you.

So lie. Next week just produce an abnormally flat set and you can say that look it was just chance. You shouldn't have tolook through more than ten or twenty sets to find a set with no voids and no seven card suits probably.

[pigpenz]

Dealmaster pro works very nicely gives a break down of the hands with singletons voids etc at the end.
There has always been this perception that computer generated hands are somehow different than hand dealt hands
and what is it there is not a thourough shuffling of the hands that accounts for it.

[bidule5]

hrothgar, on 2012-November-08, 17:12, said:

The sample size necessary to validate whether a dealing program has acceptable randomness is enormous.

If you really care about this, inspect the underlying code and see how it was implemented.

If you decide to try to verify hand records, I'd recommend the following:

Ask the player who is complaining what the "worst" feature of the randomly dealt hands.

Are there too many voids...
Are there too many 7+ card suits...

Chose one specific feature

At this point in time, you can run hypothesis test <on a new sample> and see whether the actual is deviating too far from the expected...

Agree and disagree. Testing is not that hard.

The "too many voids test" is easy, as is the "too few singles test".

Average, about one of five boards have one or more void, one of five have no void of single.

If you frequently have more than 25% of the boards with one or more void (or more than 25% of the boards without any single or void)
your program is probably faulty.

Here are my stats for "super unbal boards" (P1V) and "super balanced boards" (PNS):

For 32 Boards :
P1V : prob. of exacty N boards with one or more void
PNS : prob. of exactly N boards without a single or void

N P1V TP1V PNS TPNS
0 0.16% 0.16% 0.06% 0.06%
1 1.11% 1.27% 0.50% 0.56%
2 3.86% 5.13% 2.04% 2.60%
3 8.65% 13.78% 5.31% 7.91%
4 14.05% 27.82% 10.03% 17.94%
5 17.63% 45.45% 14.63% 32.57%
6 17.77% 63.22% 17.16% 49.73%
7 14.79% 78.01% 16.61% 66.35%
8 10.35% 88.37% 13.53% 79.88%
9 6.19% 94.55% 9.40% 89.28%
10 3.19% 97.74% 5.64% 94.91%
11 1.43% 99.17% 2.94% 97.85%
12 0.56% 99.73% 1.34% 99.19%
13 0.19% 99.92% 0.54% 99.73%
14 0.06% 99.98% 0.19% 99.92%
15 0.02% 100.00% 0.06% 99.98%
16 0.00% 100.00% 0.02% 99.99%

PS: As someone else wrote, they wont believe it.

yvan

[hrothgar]

bidule5, on 2012-November-09, 15:34, said:

Agree and disagree. Testing is not that hard.

First of all, you aren't actually doing a hypothesis test. (I don't see any mention of confidence intervals or the like)

Second, you are only testing one possible feature. What makes this hard is that there's an extremely wide number of ways in which a Dealer program can be flawed.
If you really want to test this, you need to do something equivalent to testing the properties of the random number generator which is a pain in the butt (even using tools like dieharder)

[TimG]

Bud,

I'd suggest taking the first month of deals and counting things like voids, singletons, and 7-card suits. Compare this to the expected number of each (and provide a confidence interval). Pin the results on the club bulletin board. Repeat after 2 months and 3 months.

It won't convince everyone, but some players will find it interesting. And, some players will recognize that their perception doesn't quite match reality.

Tim

[bidule5]

hrothgar, on 2012-November-09, 16:04, said:

First of all, you aren't actually doing a hypothesis test. (I don't see any mention of confidence intervals or the like)

Second, you are only testing one possible feature. What makes this hard is that there's an extremely wide number of ways in which a Dealer program can be flawed.
If you really want to test this, you need to do something equivalent to testing the properties of the random number generator which is a pain in the butt (even using tools like dieharder)

Agree, but I was just trying to show how easy flawed dealer can be detected.

yvan

[BudH]

TimG, on 2012-November-09, 16:46, said:

Bud,

I'd suggest taking the first month of deals and counting things like voids, singletons, and 7-card suits. Compare this to the expected number of each (and provide a confidence interval). Pin the results on the club bulletin board. Repeat after 2 months and 3 months.

It won't convince everyone, but some players will find it interesting. And, some players will recognize that their perception doesn't quite match reality.

Tim

Tim, that is what I plan on doing at the end of this month........

[BudH]

Our club is presently using BridgeComposer to create the hands with the Dealer4 software itself as a backup method. Since the Big Deal program (recently revised by Hans Van Staveren two months ago), presumably has as good or better algorithms, even though it is a DOS type program that provides a very simple PBN file, I would tend to use that if I was generating the hands, but nobody else in our club knows about Big Deal.

Any other good dealing programs out there, especially if they are easy to use to make random hands?

[BudH]

How do I do a detailed analysis of a group of PBN files?

I would like to take a group of PBN files (for example, one month of them used at my local club), and do a detailed (not just balanced hand, 7+ card suits, number of singletons/doubletons) analysis of all the hands.

What is the best method (and software) to use to accomplish this?

And if needed, how do you most easily combine a few dozen PBN files into a single large PBN file?

[I note that Dealmaster Pro gives you a pretty good analysis, but it appears you can access that only if you just then generated the hands using Dealmaster Pro.]

[hrothgar]

BudH, on 2012-November-11, 15:09, said:

How do I do a detailed analysis of a group of PBN files?

I would like to take a group of PBN files (for example, one month of them used at my local club), and do a detailed (not just balanced hand, 7+ card suits, number of singletons/doubletons) analysis of all the hands.

What is the best method (and software) to use to accomplish this?

Necessary caveat: I think that you are on a fool's errand.

1. The sheer amount of time and effort necessary to do properly this is enormous
2. Doing this right requires a fair amount of programming skills to generate the sample sizes necessary
3. Regardless of the results that you come up with, the critics will still complain

With this said and done, here is how I would proceed:

Step 1: Map your bridge deals to a unique 96 bit number. See http://www.rpbridge.net/7z68.htm for an explanation
Step 2: Test the result setting of hands for randomness using "dieharder" or another equivalent test. http://www.phy.duke....l/dieharder.php

As an alternative, you could conduct a more standard hypothesis test using R or MATLAB. However, this requires formally specifying the hypothesis that you plan to test which can be rather difficult. (The reason that I originally suggested a small number of simple tests was to avoid trying to describe a more complex test case)

[BudH]

hrothgar, on 2012-November-11, 16:09, said:

Necessary caveat: I think that you are on a fool's errand.

1. The sheer amount of time and effort necessary to do properly this is enormous
2. Doing this right requires a fair amount of programming skills to generate the sample sizes necessary
3. Regardless of the results that you come up with, the critics will still complain

With this said and done, here is how I would proceed:

Step 1: Map your bridge deals to a unique 96 bit number. See http://www.rpbridge.net/7z68.htm for an explanation
Step 2: Test the result setting of hands for randomness using "dieharder" or another equivalent test. http://www.phy.duke....l/dieharder.php

As an alternative, you could conduct a more standard hypothesis test using R or MATLAB. However, this requires formally specifying the hypothesis that you plan to test which can be rather difficult. (The reason that I originally suggested a small number of simple tests was to avoid trying to describe a more complex test case)

OK, perhaps not quite THAT detailed! But I do appreciate the information.

[hrothgar]

BudH, on 2012-November-11, 19:09, said:

OK, perhaps not quite THAT detailed! But I do appreciate the information.

No worries.

On a more practical basis, I really do recommend engaging with the individual who is claiming that the hands are biased in some way.

Have this player specify what is "wrong" about with the past set of deals. Agree on three, or four, or five specific examples:

There are too many voids
There are too many seven + card suits
There are too many 5431 patterns

Document this all in advance.

At this point in time, you can use this specific set of complaints to design a set of hypothesis tests that you can perform on a new set of deals. These tests won't require all that large a sample size and you can probably run them all by hand.

[TimG]

You should be able to parse the PBN files with your favorite programming language, perl is probably suited well to this task. Then you can count whatever you want.