Knows when to hold ’em: Alberta scientists develop unbeatable poker program

Press “enter,” dealer — scientists have taught a computer how to play unbeatable poker.

EDMONTON — Press “enter,” dealer — scientists have taught a computer how to play unbeatable poker.

While the news may sadden the hearts of rec-room card sharps everywhere, the winners in this game are programmers trying to do everything from improve public security to help doctors treat patients with diabetes.

“We should be able to use these algorithms in any well-defined problem,” said Michael Bowling, the University of Alberta computer scientist who co-authored a paper in the journal Science that details how the program for two-handed, fixed-bet Texas Hold ‘Em can’t do worse than break even.

Scientists in the field of game theory long ago taught computers to play games such as checkers and chess. But poker has remained elusive because it’s a so-called “imperfect information” game. A player has to make decisions without knowing all the data such as what the other player is holding.

“This game has been, historically, an important challenge problem,” Bowling said. “Poker is one of the games that really motivated the whole founding of the field of game theory back in the ’20s.”

Bowling’s team made its breakthrough by refining a previously developed technique called counterfactual regret minimization that allows a computer to look back at previous hands and learn from its mistakes. Although that sounds similar to how humans improve, the computer used here became a one-player Las Vegas.

“It spent two months playing billions and billions of hands of poker against itself to find the perfect strategy,” said Bowling. “The strategy is 1,000 times larger than all the English-language Wikipedia.”

It’s unlikely to be of much use at anyone’s Saturday night game.

“You have to memorize a 10-terabyte table of probabilities.”

A terabyte is one byte followed by 12 zeros.

But the point was never to become an unbeatable online poker star. The same process that taught the computer when to hold ’em and when to fold ’em can be transferred to any problem with well-defined rules and outcomes, many options and imperfect information – terrorist security, for example.

“We run patrols, we do searches — we have these tools at our disposal, but how do we deploy them? We want to find a strategy that’s unbeatable.

“What we’ve done is shown that we can do these game theoretic analyses at a scale that hasn’t been done before – at a really enormous complexity. That means that we can start looking at problems in that security sphere.”

Game theory is already being used to help schedule air marshals on commercial flights in the United States.

Bowling’s team is also working with diabetes researchers to see if the computer poker work can help manage the disease.

Doctors and patients typically come up with a plan to adjust insulin intake to food consumption, exercise and other variables. But those variables can change. Nor do doctors have any guarantees how well the patient will follow the treatment plan.

“Building a policy that is robust to those uncertainties is not that different from building a poker policy that’s robust to not knowing what cards the opponent has,” Bowling said.

“If we could have a decision support system that could maybe help the patient tweak their formula on their own, or even assist the doctor to do it faster, then we could improve the effectiveness of these treatment policies.”

Despite its larger ambitions, there are lessons in Bowling’s paper for the casual player, although they will already be familiar to the experienced.

— Avoid simply calling bets. If you’re in, you’re probably best to raise.

— Don’t make the maximum allowed bet in the first round.

— Hang in there. The computer routinely played weaker hands than most human players.

“What the poker programs have always suggested is that the human players are too conservative at this game.”

Get local stories you won't find anywhere else right to your inbox.
Sign up here

Just Posted

If you're heading out to the West Country have a plan in case things go wrong, says Clearwater Regional Fire Rescue Services fire chief Steve Debienne.
(Photo from CRFRS Facebook)
West Country visitors should have an emergency plan: regional fire chief

Cellphones can’t be relied on in many back country areas

The Red Deer PCN Women's Fun Run will take on a different look this weekend with rising COVID-19 numbers.
Women’s Fun Run goes ahead this weekend

With new public health measures in place because of rising COVID-19 case… Continue reading

Alberta’s chief medical officer of health Dr. Deena Hinshaw asked Albertans to limit travel throughout the province as COVID-19 cases continue to rise. (Photo by Chris Schwarz/Government of Alberta)
Red Deer nears 900 active COVID-19 cases

Province reports additional 2,211 COVID-19 cases

David Eggen, the NDP’s advanced education critic, said the UCP government has been focused on cutting funding to post-secondary institutions across Alberta. (Contributed photo)
NDP worry new status for Red Deer College doesn’t mean more funding

This week the province announced that RDC will become a polytechnic institute

Stock photo
Report: Canada loves hockey, but sport has its issues

While hockey is strongly intertwined in Canada’s lexicon as a country, some… Continue reading

Alberta Health Services locked the Whistle Stop Cafe in Mirror on Wednesday morning after owner Christopher Scott refused to comply with health orders. (Photo by Paul Cowley/Advocate staff)
UPDATED: AHS shuts down Whistle Stop Cafe for defying health orders

Justice minister promises to get tough with those ignoring public health orders

Prime Minister Justin Trudeau listens to speakers appearing by video during a news conference in Ottawa on Tuesday May 4, 2021. THE CANADIAN PRESS/Adrian Wyld
Canada agrees to take part in WTO talks to waive patent protections on vaccines

Canada agrees to take part in WTO talks to waive patent protections on vaccines

Home renovation savings prove elusive as wood prices at record highs

Home renovation savings prove elusive as wood prices at record highs

Canadian Natural Resources Ltd. logo is shown at the company's annual meeting in Calgary on May 3, 2012. THE CANADIAN PRESS/Jeff McIntosh
Canadian Natural reports $1.38B Q1 profit, plans to use cash flow to reduce debt

Canadian Natural reports $1.38B Q1 profit, plans to use cash flow to reduce debt

Logs are piled up at West Fraser Timber in Quesnel, B.C., Tuesday, April 21, 2009. West Fraser Timber Co. Ltd. is reporting higher first-quarter sales and earnings after completing its $4-billion all-stock takeover of Norbord Inc. on Feb. 1. THE CANADIAN PRESS/Jonathan Hayward
West Fraser reports US$665M in earnings, $2.3 billion in sales as it acquires Norbord

West Fraser reports US$665M in earnings, $2.3 billion in sales as it acquires Norbord

An oil worker holds raw sand bitumen near Fort McMurray, Alta., on July 9, 2008. THE CANADIAN PRESS/Jeff McIntosh
Alberta eases security payment burden for oilsands companies

Alberta eases security payment burden for oilsands companies

A street sign along Bay Street in Toronto's financial district is shown on Tuesday, January 12, 2021. THE CANADIAN PRESS/Nathan Denette
S&P/TSX composite falls despite commodities, loonie hits highest level since 2017

S&P/TSX composite falls despite commodities, loonie hits highest level since 2017

Nicholas Marcus Thompson is shown in Toronto on Thursday April 29, 2021. THE CANADIAN PRESS/Frank Gunn
Action needed to end anti-Black racism in public service: advocates

OTTAWA — The federal government must address anti-Black racism in the public… Continue reading

Victoria Police help BC conservation officers carry a cougar which was tranquilized in the backyard of an apartment building in the community of James Bay in Victoria, B.C., Monday, October 5, 2015. THE CANADIAN PRESS/Chad Hipolito
Cougar believed to be responsible for B.C. attack killed: conservation service

AGASSIZ, B.C. — The British Columbia Conservation Officer Service says it believes… Continue reading

Most Read