Flipping the Script: Alberta Health Services’ online assessment tool becomes the core of the province’s COVID response

Alberta Health Services’ online assessment tool becomes the core of the province’s COVID response.

It was a Thursday afternoon, March 5 to be exact. Albertans faced a new reality: The first two positive cases of COVID-19.

“You know sometimes you’re in denial,” says Ammneh Azeim from her Edmonton home, “you think it’s not going to come here.”

A global pandemic is hard to fathom for anyone, but for Azeim this hit especially close to home. Her mother is a nurse at the University of Alberta Hospital. Her father is a retired physician. Azeim herself works for Alberta Health Services (AHS). A first in Canada, AHS provides integrated health services to the entire province of more than four million residents. One central hub where all Albertans go to access health information. Azeim is not a nurse or doctor like her parents, but instead a “tech-guru” heading up the AHS digital experience team.

“I have a bit of a habit,” she laughs. “I love social media.”

So naturally, when those first two cases were announced, Azeim grabbed her phone and started scrolling Twitter. She didn’t like what she was reading. Tweet after tweet showed Albertans were scared and frustrated. AHS’ health information call centre 8-1-1, known as Health Link, was overwhelmed. The nurses answering calls were in high demand, and panicked residents were on hold waiting hours to find out if the symptoms they had were COVID.

The social media tailspin left Azeim with one question: “What can we do?”

Azeim’s job is to design digital solutions for the AHS. Her team creates user friendly online solutions, with a focus on improving user experience. Their goal is to help Albertans navigate through the healthcare system. You could say they’re not saving lives, just making lives a little easier. At that moment though, Azeim had no idea just how big of a role she’d play in the province’s COVID-19 fight.

“I think as COVID hit, our whole team was busy posting information and I thought, we’ve got to do more.”

“You know that instinct,” says Azeim, “we are in a pandemic, I need to be helpful.”

Azeim started researching around the clock. On Sunday, she noticed technology being used in the UK: an online COVID assessment tool.

“I sent an email to my Health Link contact,” Azeim recalls, “I said ‘how about we look into digitizing some of your script?’.”

Little did they know this moment would shift the province’s whole pandemic strategy, and become the core of Alberta’s COVID response. The assessment tool with online booking and results would get attention from across North America, and spark a form of collaboration that would cast lasting doubt on the way things have always been done.

On Monday morning, Azeim approached her team with the idea of taking some of the questions the nurses ask patients over the phone and putting them into an online tool. It would mean Albertans could find out immediately, by clicking yes or no to a symptom list, if they need to speak with a Health Link nurse and/or need a COVID test.

Azeim’s lead Kass Rafih, Senior Provincial Director for Innovation and Digital Solutions immediately saw the potential and jumped on board.

“It was a simple but powerful idea,” Rafih recalls. “If you answered yes to this and no to that then maybe you need to self-isolate and call Health Link for an appointment. If you answered no to these questions well then you should be safe, just sleep well and get some rest. It’s the same advice our Health Link nurses would offer but delivered to folks in a way that would be easy to access.”

The team started creating a “humongous” flow chart with the UK algorithm. There were two challenges: they needed to convince the AHS medical team and health ministry that this tool could work, and there was a lack of time.

“You need to bring buy-in when you are doing something new, you have to bring in your study that it is something doable,” Azeim says.

The team rallied and quickly built a prototype. By Tuesday morning, Rafih had taken the idea to several key leaders including the Senior Medical Officer of Health and executive teams. It was given full support. The assessment tool would move forward.

“It moved really quick and it was incredible to see the level of support and trust,” Rafih says. “It’s been quite a ride. This was an opportunity to digitally deliver a front line service directly to Albertans, a chance to protect them and provide them with the information they needed to keep themselves and their families safe.”

The next step was meeting with AHS IT Personal Health Portal Services Manager Ben Wei. Wei’s team would be responsible for coding the tool.

Azeim knew they needed to get it up and running within the next few days. Panic across the province and around the world was rising. The commitment to get it live would be a big one, and so she approached Wei with some hesitation.

“So we’re doing this,” Azeim explained to Wei. “I don’t want to add too much to your plate.”

Wei’s team, whose job it is to focus on the AHS MyHealth Records platform, also works very closely with Health Link to provide support. Wei knew just how hard the nurses on the helpline were working, trying to keep up with demand.

“At one point the 8-1-1 phone line got overwhelmed,” said Wei. “We kept upgrading and increasing more lines but people just panicked. We had never seen this before.”

Wei and his team were already under pressure. Still, he knew this online assessment tool, even with all the work to get it going, could be the solution Albertans were searching for.

“I know it’s a challenge,” Wei told Azeim, “but let’s do it!”

Wei knew there were two main pieces when developing the coding. The tool must be easy to use and there must be technical support to ensure it runs smoothly.

“Technically we need to be ready,” Wei says. “We have 12 servers for our site. We have never tested this load in the past.”

Wei knew that rushing the technology could be disastrous.

“We’ve seen other organizations experience server capacity issues with the launch of new online services. We don’t want the same situation. We needed to make sure our technical piece was ready.”

Wei and his team then started performance tests to learn how many submissions the server could handle at once without crashing. The goal was 20,000 submissions in 90 seconds.

“I was crossing my fingers that this would take a few minutes and then after I clicked, it was about one minute and it was gone. I was thinking, ‘Is something wrong?’” Wei laughed.

“I went to the database and they were all there so it was very good,” he adds excitedly.

Wei knew instantly this tool would be a winner. Remember, the Health Link phone system was crashing; Albertans couldn’t even find someone to speak with. Now through Wei’s estimations, if 20,000 people were submitting symptoms online, on average 1-3% of them would need a follow-up with a nurse, reducing calls from 20,000 to a few hundred. He knew immediately this tool could change Alberta’s COVID fight.

The pressure was on. They would have just 36 hours to build and test the assessment tool. A request that would be considered highly unreasonable, even impossible in “pre-COVID” times.

So the team worked all day and all night until it was ready to launch. They consulted with the Medical Officer of Health and Health Link leaders to form the assessment questions.

“We’re not clinical experts,” Rafih admits. “We took that guidance but we were very much modelling the same guidance that was being used on the nurse helpline.”

“I still remember the day,” Azheim says with a nervous laugh; you can almost feel the pit that still sits in her stomach. “It was crazy, I’m telling you.”

It was Friday March 13. Alberta’s Health Minister, Tyler Shandro, would make the announcement at a news conference at 3:30 p.m. Rafih, Azeim and their teams along with AHS’ Senior Medical Officer of Health had to give the Minister’s office the go-ahead by 3 p.m. For the AHS digital team, this was the big time. The weight of a province sat on their shoulders. This tool could give Albertans a way to address their fears over COVID-19.

As the clock ticked down, the team, quietly behind the scenes, did demo after demo with all of the assessment centres, Health Link, the Emergency Command Centre and Public Health to ensure the tool worked correctly. At 11 a.m., staff from Public Health noticed a major glitch just hours before launch.

The digital team needed to reduce the scope of the tool by removing the referral section. Azeim felt like she was hit with a ton of bricks, but she knew she couldn’t stop, the team couldn’t stop, they had to get this tool up and running.

“In an hour we were working with the legal department to change the wording, working with communications, working with IT,” Azeim said.

There were bugs with the tool that needed to be fixed too. Weí warned them that his team may not have the time to do it before launch.

But Azeim pushed. “I said ‘OK guys, let’s change it,’ and they did.”

The IT team fixed the bugs while simultaneously testing the tool. It was performing.

It was now 1:30 p.m., two hours before launch, Rafih called Azeim asking if he could now give the Minister’s office the green light.

“He says ‘are you confident?’ and I say ‘yes, I’m pretty confident,’” she laughs. “And we launched it.”

Azeim and her team settled in to watch the Minister’s news conference. They should have been relieved but instead there was nervousness.

“You know when you’re so stressed out, you don’t necessarily enjoy it at that moment?”

“You’re just like ‘good thing it’s done’,” Azeim exhaled.

The entire team looked on with a sense of pride.

“It was a simple but powerful moment,” Rafih recalls. “When we launched our tool and the Minister tweeted it out and then we started actively tweeting and telling people to stop calling Health Link, that you can get your answers here.”

Rafih said some of the responses made the team laugh, like one that read, “Donald Trump should learn from you guys.” Others, though, were more meaningful.

“One of the first tweets that came back was ‘Wow you guys are brilliant! I spent two minutes doing this and I’m going to sleep well tonight!’,” Rafih says with a smile, “and I was just like, that’s so validating to know that you are able to actually help people.”

It wasn’t just the comments; it was the concrete results the team witnessed in just hours.

“The phone lines reduced a lot,” Wei says, “That is where we saw the reward.”

While the team briefly celebrated their success, they realized their job was far from done. In fact, it was just beginning.

“As soon as we launched it, we heard from several ministries and Saskatchewan was the first to reach out,” Rafih says. “We were on the phone with them the following morning and we gave them everything that we built. We just gave it to them.”

That would be the start of something very unorthodox for any government. Sharing code, sharing tools, no questions asked.

“There were no agreements signed,” Rafih says, “It was extraordinary times, there was a lot of fear and if you could do something to help your neighbour, why wouldn’t you? So we gave them everything. There wasn’t a large discussion around it.”

And Rafih says it wasn’t just the code AHS gave the province of Saskatchewan.

“We gave them our staff’s time to even help them refine their tool so they could launch it and they were launched the following day.”

It wasn’t just Saskatchewan. They had requests from Ontario, New York City, the Canadian Armed Forces, Ireland and the Asian Development Bank among others. They were soon on calls with other provinces and the federal government. The demand to share the code was so high, the AHS team couldn’t keep up. They decided to upload the code to GitHub, an online code repository where codes are shared freely with the public. Rafih says it was an unconventional decision, but it made sense.

“It was a very proud moment,” he says, “to be able to help people and share an Alberta built product, something that we built as a team.”

“One of the first tweets that came back was ‘Wow you guys are brilliant! I spent two minutes doing this and I’m going to sleep well tonight!’ and I was just like, that’s so validating to know that you are able to actually help people.” – Kass Rafih, Executive Director, Online and Creative Services, Alberta Health Services

It was a shining moment in a time of crisis. Public servants helping public servants and sharing knowledge that would affect millions of people not only across Canada but around the world. A form of collaboration that was getting noticed.

March 5 2020 – Alberta records first two positive cases of COVID-19

March 8 2020 – Ammneh Azeim, AHS Digital Experience Team Manager, discovers assessment tool being used in the UK

March 9 2020 – Azeim brings the idea to build an online assessment tool to her AHS team; the team builds a flowchart studying the UK algorithm

March 10 2020 – Ben Wei, AHS IT Personal Health Portal Services Manager, and his team are brought in to code the tool

March 12 2020 – The tool and AHS servers undergo rigorous performance tests

March 13 2020 – It is assessment tool launch day

8 a.m. the tool is demonstrated for assessment centre staff

11 a.m. Alberta Public Health demands a last minute overhaul of tool

Noon Wei’s team codes last minute changes

1:30 p.m. Azeim gives the final green-light to launch

3:30 p.m. Alberta Health Minister, Tyler Shandro, announces launch of new online assessment tool

As of Sept. 29, 2020, the online assessment tool has been accessed more than 6-million times and received more than 1.5 million referrals for testing.

“I think this shows us there is so much benefit to sharing,” says public policy expert and retired Deputy Minister of Executive Council of the Government of Alberta, Marcia Nelson.

Nelson says sharing information inter-provincially shouldn’t be an extraordinary move in a time of crisis but instead common practice.

“I was always of the mind that if there is a good idea, let’s just steal it, let’s use it. Find out the best practice and use it. It is very frustrating,” Nelson adds, “there is some hubris among governments. They don’t want to be beholden to other governments. Everyone wants to be masters of their own destiny. There is some pride involved. I also think some of it is muscle memory; if they aren’t in the habit of cooperating they’re not going to do it. I think the more you get in the practice of connecting and sharing and taking an open default, the more you’ll do that.”

Nelson, a public servant herself in Ontario and Alberta for more than 30 years, is familiar with the obstacles governments face when it comes to collaboration, but this she says should be a pivotal teaching moment.

“There are a lot of barriers. The professions are barriers, funding models are barriers, advocacy organizations, legislation and regulations, there are lots of barriers as to why these things don’t get done.”

“But I do think that what I’m really hopeful for coming out of all this is that we will be able to take lessons and see the efficiencies that have been driven out as a result.”

The success of this roll-out and direct advantages may be due in part to the Alberta Health Services structure as a whole. As the single healthcare platform for the entire province, there is one leadership team that makes decisions. One consistent message being relayed to all four million Albertans. Alberta was the first in the country to create a province-wide integrated health services model. Most others, including Ontario, work in a tiered approach, with local and regional health units all under a provincial body. .

While the pandemic had never been seen before, acting at a time of crisis wasn’t new to AHS. When Alberta was hit hard by floods in 2013, it was AHS who moved into action. AHS teams were helicoptering critical patients, building field hospitals and marshalling resources province-wide. It’s the positive that comes out of natural disasters, urgent care and strategic response that builds confidence within an organization.

“There was a real leap of faith. They had trust in us,” Rafih says, “that is a credit to our executive leadership.”

“If you tried to build a tool like this in traditional times it would have gone through so much review and so many approvals. It would have taken months, even up to a year to be able to launch something like this.”

“We were directly connected with our Senior Medical Officer of Health who was directly connected to Alberta’s Chief Medical Officer of Health. We were immediately given access to the highest levels in our organization to be able to make this happen.”

“It has a lot to do with the relationship we’ve been able to build over the years and the trust we’ve been able to develop,” he adds.

Nelson says despite the trust, the tool and the decision to launch it so quickly would have come with risk. “What would have happened if you get this tool up and someone hacks it and then everyone’s addresses logged into it are now available? There are reasons to be careful, but I just think sometimes we overthink things.”

In this case the risk of not having the tool, the pressure it put on frontline workers, outweighed the risks with launch.

That trust the leadership team had in developers only grew from there.

Days after the launch of the online assessment tool, it continued to evolve and develop. The digital team quickly realized Albertans didn’t just want a place to find out if their symptoms could indeed be COVID, but they wanted to know: If they did have symptoms, what should they do next?

The team wondered, what if Albertans could not only assess their symptoms online, but also book a test and get their results through the tool as well?

The team, still working around the clock since launch day, brought in AHS Web Presence and Collaboration Services Manager TJ Shin to the meetings. They looked at hiring a third party that already had the existing technology to book online to add to the site. That’s when Shin put up his hand and said, what if we build it ourselves?

“He messaged me and he messaged Kass,” Azeim recalls, “he says why would we go to a third party tool where we’re not probably going to have a great user experience and won’t have control and won’t be able to adapt to the changing needs of assessment centres. And we thought, absolutely.”

The trust in Shin and his team made things move faster. Within a month, the online booking component was up and running.

“It was the right decision,” Shin says. “Sustainability wise, I felt that in-house would make a lot of sense if we have the right people and the right infrastructure to support it.”

AHS now had the whole system behind them. Between the Health Link call centre, assessment centre scheduling and online assessment and booking, AHS could use the technology to support contact tracing and gather analytics.

In the six months after launch, the tool has now gone through more than a dozen iterations, with enhancements and adding new features. 

“We’re continuously able to respond to the need and practically work with the government and AHS as the situation continues to change,” Shin says.

Since then, AHS has added another tool. Albertans can now choose to get an SMS text message or robocall with the result of their COVID test. The goal is to have results returned within 24 to 72 hours. It’s taking a lot of pressure off the frontlines. That tool that was once just an idea and “leap of faith” is now considered the core of Alberta’s pandemic strategy.

“Over 97% of users are choosing the SMS option, about 1.7% are choosing robocalls and less than 1% are asking for a human to call them back,” Wei says. “When we talk about a human calling them back, that means Healthlink resources. It’s taken a lot of pressure off frontline staff.”

While other provinces have tried to follow Alberta’s lead, few others have moved so quickly and efficiently as them. Alberta still remains at the forefront of using technology to battle COVID.

“I think this should be a cautionary tale for governments,” Nelson says. “Sometimes you just have to rely on your own people.  You can’t outsource everything. It’s critical that governments ensure that they have current capabilities in critical management areas. Everything from IT to finance to clinical support and policy because you have to have those capabilities to effectively engage with those vendors, think tanks and other partners in the private sector.”

Nelson says all governments need to look at Alberta’s response to COVID and how what she calls this “one little moment” became a master class in healthcare implementation and emergency response.

“Sometimes in order to move at speed, you just have to trust, take some risks and rely on your own people. I think there is a lesson there for all of us.”

That lesson didn’t come easily for the AHS team. There were months of late nights and overtime, moments of frustration and celebration. In the end, though, they all agree their work was valued.

“I consider my team as critical thinkers,” Azeim says with confidence, “people who actually take complex problems and come up with simple solutions. That’s the number one criteria we hire for this team and that’s the number one criteria we take pride in. So for them, complexity is second nature.”

“We’re blessed,” she adds, “we’re still working day and night and lots of hours of overtime but it helps the team to constantly remember what impact we’re making and we’re super grateful.”

As COVID continues, so too does this digital team. Rafih says they’ve expanded and now have more structure and purpose.

“We now have a steering committee with tons of clinical representation. We take feedback regularly from Albertans. We take feedback from the government. We take feedback from our assessment centres who are doing the testing. We take feedback from pharmacies who are doing the testing. We’re extending it and growing it based on direct feedback from people,” says Rafih, “It’s core to our COVID response. This tool has been core to how AHS and the government of Alberta are responding to COVID. COVID is a terrible thing, it’s the worst thing that I’ve ever had to go through. But I’m also so proud to say that my team and I have been able to stand up and actually make a difference.”

The Government of Canada, The Wilson Foundation, The Lawson Foundation and Microsoft.