Knows when to hold ’em: Alberta scientists develop unbeatable poker program

Press “enter,” dealer — scientists have taught a computer how to play unbeatable poker.

EDMONTON — Press “enter,” dealer — scientists have taught a computer how to play unbeatable poker.

While the news may sadden the hearts of rec-room card sharps everywhere, the winners in this game are programmers trying to do everything from improve public security to help doctors treat patients with diabetes.

“We should be able to use these algorithms in any well-defined problem,” said Michael Bowling, the University of Alberta computer scientist who co-authored a paper in the journal Science that details how the program for two-handed, fixed-bet Texas Hold ‘Em can’t do worse than break even.

Scientists in the field of game theory long ago taught computers to play games such as checkers and chess. But poker has remained elusive because it’s a so-called “imperfect information” game. A player has to make decisions without knowing all the data such as what the other player is holding.

“This game has been, historically, an important challenge problem,” Bowling said. “Poker is one of the games that really motivated the whole founding of the field of game theory back in the ’20s.”

Bowling’s team made its breakthrough by refining a previously developed technique called counterfactual regret minimization that allows a computer to look back at previous hands and learn from its mistakes. Although that sounds similar to how humans improve, the computer used here became a one-player Las Vegas.

“It spent two months playing billions and billions of hands of poker against itself to find the perfect strategy,” said Bowling. “The strategy is 1,000 times larger than all the English-language Wikipedia.”

It’s unlikely to be of much use at anyone’s Saturday night game.

“You have to memorize a 10-terabyte table of probabilities.”

A terabyte is one byte followed by 12 zeros.

But the point was never to become an unbeatable online poker star. The same process that taught the computer when to hold ’em and when to fold ’em can be transferred to any problem with well-defined rules and outcomes, many options and imperfect information – terrorist security, for example.

“We run patrols, we do searches — we have these tools at our disposal, but how do we deploy them? We want to find a strategy that’s unbeatable.

“What we’ve done is shown that we can do these game theoretic analyses at a scale that hasn’t been done before – at a really enormous complexity. That means that we can start looking at problems in that security sphere.”

Game theory is already being used to help schedule air marshals on commercial flights in the United States.

Bowling’s team is also working with diabetes researchers to see if the computer poker work can help manage the disease.

Doctors and patients typically come up with a plan to adjust insulin intake to food consumption, exercise and other variables. But those variables can change. Nor do doctors have any guarantees how well the patient will follow the treatment plan.

“Building a policy that is robust to those uncertainties is not that different from building a poker policy that’s robust to not knowing what cards the opponent has,” Bowling said.

“If we could have a decision support system that could maybe help the patient tweak their formula on their own, or even assist the doctor to do it faster, then we could improve the effectiveness of these treatment policies.”

Despite its larger ambitions, there are lessons in Bowling’s paper for the casual player, although they will already be familiar to the experienced.

— Avoid simply calling bets. If you’re in, you’re probably best to raise.

— Don’t make the maximum allowed bet in the first round.

— Hang in there. The computer routinely played weaker hands than most human players.

“What the poker programs have always suggested is that the human players are too conservative at this game.”

Get local stories you won't find anywhere else right to your inbox.
Sign up here

Just Posted

(Image from Facebook)
Central Alberta Soroptimists celebrate International Women’s Day

Soroptimist International of Central Alberta will host its first virtual International Women’s… Continue reading

Samantha Sharpe, 25, was stabbed to death at Sunchild First Nation on Dec. 12, 2018. Chelsey Lagrelle was sentenced to 4.5 years in prison for manslaughter in a Red Deer courtroom on Tuesday. (Contributed photo)
Central Alberta woman sentenced to 4 1/2 years for stabbing friend to death in 2018

Chelsey Lagrelle earlier pleaded guilty to stabbing Samantha Sharpe during argument

Alberta's chief medical officer of health Dr. Deena Hinshaw says Albertans need to keep making safe choices to start bending the curve back down. (Photo by Chris Schwarz/Government of Alberta)
One new COVID-19 death in Red Deer, 257 additional cases province-wide

Red Deer sits at 459 active cases of the virus

Olds College logo
Olds College to host free, online agriculture celebration next month

Olds College will host a free live-streamed agriculture event next month. The… Continue reading

Alberta Health Services Logo
AHS upgrading online immunization booking tool

Alberta Health Services’ online booking tool for COVID-19 immunizations will be temporarily… Continue reading

Red Deer dogs Bunsen and Beaker helped save a missing pet recently. The two dogs have more than 80,000 followers on Twitter. (Contributed photo)
WATCH: Red Deer science dogs help save lost pet

Red Deer science-communicating dogs Bunsen and Beaker helped rescue a missing pet… Continue reading

opinion
Opinion: Crisis in long-term care must include data-driven change

More than 19,000 people in Canada have died from COVID-19 – more… Continue reading

The Dawe family home in the Michener Hill subdivision in Red Deer. This house was designed and built by Robert G. Dawe, a local engineer, in 1911 and has remained in the family ever since. (Contributed photo)
Michael Dawe: 65 years of Red Deer history

As a major milestone birthday looms, I thought that it might be… Continue reading

Dallas Stars' Mark Pysyk (13) and Tampa Bay Lightning's Ondrej Palat (18) compete for control of a loose puck in the first period of an NHL hockey game in Dallas, Tuesday, March 2, 2021. (AP Photo/Tony Gutierrez)
Vasilevskiy 3rd straight shutout as Lightning top Stars 2-0

Vasilevskiy 3rd straight shutout as Lightning top Stars 2-0

Montreal Canadiens goaltender Carey Price (31) grabs a loose puck as Canadiens defenceman Ben Chiarot (8) and Ottawa Senators left wing Brady Tkachuk (7) battle for the rebound during first-period NHL hockey action Tuesday, March 2, 2021 in Montreal. THE CANADIAN PRESS/Ryan Remiorz
Hab down Sens 3-1 to snap 5-game winless streak; Ducharme earns 1st NHL coaching win

Hab down Sens 3-1 to snap 5-game winless streak; Ducharme earns 1st NHL coaching win

FILE - In this Jan. 16, 2021, file photo, Creighton coach Greg McDermott watches the team during the second half of an NCAA college basketball game against Butler in Indianapolis. McDermott apologized publicly Tuesday, March 2, for using insensitive language in his postgame locker room talk with players and staff following a loss over the weekend. In a tweet, McDermott said he used a “terribly inappropriate analogy in making a point about staying together as a team despite the loss.” (AP Photo/Darron Cummings, File)
Creighton coach McDermott apologizes for ‘plantation’ remark

Creighton coach McDermott apologizes for ‘plantation’ remark

WHL’s B.C. Division clubs cleared to play in Kamloops and Kelowna bubble environments

WHL’s B.C. Division clubs cleared to play in Kamloops and Kelowna bubble environments

World Rugby recommends postponing 2021 women’s World Cup to next year

World Rugby recommends postponing 2021 women’s World Cup to next year

Sean Burke speaks at a news conference in Calgary, Alta., Tuesday, July 25, 2017. The Montreal Canadiens have made another change to their coaching staff, appointing Burke to take over as the director of goaltending. THE CANADIAN PRESS/Jeff McIntosh
Montreal Canadiens appoint Sean Burke as director of goaltending

Montreal Canadiens appoint Sean Burke as director of goaltending

Most Read