The Wrong Metric

In the year 2000, a rising executive with the Ritz-Carlton Hotel Company, a man whose whole career was teaching people how to deliver five-star service, felt a tingling in his fingers and toes. Within days, he was paralyzed. He spent the next three months in a hospital bed, intubated, fed through a tube. For 90 days and nights, by his own count, he was 100% dependent on the doctors, nurses, and therapists around him.

The man who wrote service standards for a living was now on the receiving end, and from that bed, he couldn't help but take notes. What he noticed would eventually change how one of the largest health systems in America does business. Because it turns out the thing patients were grading their hospitals on was not the thing the hospitals were measuring.

Today, the number on your dashboard that your customers do not care about. Also on the show, the emergency room, the most stopwatch-obsessed room in America, and why your flight home is officially more on time than it was in 1990, even though it takes longer. That's all coming up on Marginally Better

From the global headquarters of Johns and Taylor in beautiful New Jersey, It's Marginally Better. Here's your host, Joe Taylor Jr.

Every business has a number that it stares at. Average handle time in the call center, table turns in a restaurant. Pick the number, put it on a dashboard, watch it every Monday, and over time, a substitution sneaks in. The number stops being a way to see the customer and starts becoming the customer.

Nobody has lived this harder than the American emergency room, so that's where we'll start. For decades, the ER has been measured in minutes. The Centers for Medicare and Medicaid Services, that's Medicare's parent agency, publicly reports how long patients spend in each hospital's emergency department.

Anyone can look that up. The Joint Commission, the body that accredits most American hospitals, tracks a measure called ED One, the median time from the moment you arrive in the emergency department to the moment you leave for an inpatient bed. There are companion measures for how long you wait to see a provider and for the percentage of people who give up and leave without being seen at all.

And let's be fair about why. Those clocks exist because crowded emergency rooms are dangerous. When an ER backs up, treatment gets delayed, sick people walk out the door untreated, and patients get parked in hallways. The industry calls that boarding, and sometimes that happens for days. Time in an emergency room is not a trivial thing to watch.

So hospitals watched it. They hired consultants for it. They built patient flow war rooms around it. Door to provider, door to doc, arrival to departure, that's an entire vocabulary of things that folks are tracking. Now, that raises an uncomfortable question that almost nobody thought to ask. When a patient goes home and decides how they feel about your hospital, are minutes what they're counting?

So somebody did ask. In December 1996, a team led by Dr. David Thompson published a study in the Annals of Emergency Medicine. They called over 1,600 patients who'd been through a suburban community hospital's emergency department and asked about the visit. How long did you wait? How long did it feel like you waited?

Did anyone explain what was happening? Were people courteous? And overall, how satisfied were you? Then they lined up the answers next to the clock, and the actual waiting time, that's the real measurable minutes that those patients sat there, had no meaningful relationship to how satisfied they were. So what did predict satisfaction?

Whether the wait felt shorter than expected, whether somebody explained the delays and procedures, whether the staff came across as caring. The researchers' conclusion nearly 30 years ago now: managing perceptions, information, and expectations may do more for patient satisfaction than cutting the actual wait.

And that finding's not a fluke. In 2017, a team at Massachusetts General Hospital published a systematic review in the Journal of Patient Experience, 107 studies of what drives emergency department patient experience. The theme that came up most often in 78 of those 107 studies was communication between staff and patients.

Wait times came second at 56. Empathy and compassion showed up in 45. The stopwatch wasn't wrong exactly. It was just answering a different question than the one patients were asking. Rick Evans, the chief experience officer at NewYork-Presbyterian, put it flatly in a Becker's Hospital Review column just this past May.

"At its heart," he said, "improving the patient experience is all about communication." So how does a whole industry end up staring at the wrong number for decades? Turns out there's a name for this. In 1975, a British economist named Charles Goodhart was writing about monetary policy. That has nothing to do with hospitals.

And he observed that any statistical pattern tends to fall apart the moment you start using it as a target. An anthropologist named Marilyn Strathern later boiled that down to the version people quote today: "When a measure becomes a target, it ceases to be a good measure."

They call it Goodhart's Law, and once you know it, you'll see it everywhere. A 2021 editorial in the Journal of Graduate Medical Education walked through the medical versions. Residency programs coaching trainees on how to answer the survey that's supposed to catch overwork. Med students cramming for a licensing exam score because program directors screen on that score at the expense of the actual skills the score is supposed to represent.

The number gets better, but the thing the number stood for doesn't. Hospitals could hit the time targets and still send home patients who felt ignored and forgotten because nobody was measuring whether anyone explained anything. Which sets up the question at the heart of today's show. If the clock isn't the metric that matters, what happens when a major health system bets on the one that is?

When we come back, a luxury hotel executive loses the use of his entire body and then gets handed 21 hospitals and a mandate to fix how they feel.

That's next. It's Marginally Better

Quick word about something that our team built. Today's show is about the gap between what your dashboard says and what your customers actually feel. And if you've ever suspected that gap exists on your own website, check out numbers that look fine while the inbox fills with confused customers, that's exactly the kind of question Experience Helpdesk was made for.

It's our subscription helpdesk for user experience. You send us the question, "Why do people abandon this form? What should this page say? Which of these two designs works better?" And a UX specialist from our team sends back a real answer fast. No discovery calls, no six-figure engagement, just expert eyes when you need them, plus a library of guided playbooks that cover the most common customer experience fixes.

Come see it at experiencehelpdesk.com. That's experiencehelpdesk.com.

It's Marginally Better. I'm Joe Taylor Jr. Sven Gierlinger spent the first act of his career with the Ritz-Carlton Hotel Company, helping open hotels in Germany, Japan, Indonesia, and the United States. His job was the famous Ritz-Carlton service culture itself: training staff, installing standards, teaching teams how to anticipate what a guest needs before the guest even says it.

And years later, he would tell Becker's Hospital Review what that time taught him. What guests really want is, quote, "Much more than crystal chandeliers, marble floors, and fancy ocean views. It's the consistency of service delivered through highly engaged and empowered people that make the difference." In 2000, Gierlinger was a father of two young children, just starting off in a new Ritz-Carlton assignment when the tingling started.

Fingers first, then toes. Within days, he was paralyzed. The diagnosis was Guillain-Barré syndrome, a rare condition where your own immune system attacks the wiring of your nervous system. It's usually temporary, but the recovery is slow, and in the meantime, your body simply stops taking instructions He spent three months hospitalized.

He was intubated, that's a machine breathing for him, and then extubated, then fed through a tube, and then slowly worked his way back through liquids, soft foods, solids. He said later, "Suddenly, and for ninety days and nights, I was one hundred percent dependent on my doctors, nurses, and therapists." Think about who was lying in that bed.

A man professionally trained to notice every detail of a service experience now completely unable to leave one. And the details piled up. The food, by his own account, was atrocious. He lived on protein drinks he could barely stand while his wife, already raising their two kids alone, hauled meals in from home and from restaurants.

This at what was supposed to be the most caring institution a person should ever encounter. Years later, though, he called the whole ordeal a gift. Quote, "Believe it or not, I'm actually grateful for this experience," he told Beckers. Because it showed him that a patient is trying to heal in three ways at once: physically, mentally, and emotionally, and that service and empathy embedded into the clinical routine can make or break all three.

The hotel man had seen the product from the wrong side of the counter, and fourteen years after that incident, somebody handed him the keys. In twenty twelve, Northwell Health, the largest health system in New York State, that's more than twenty hospitals across the New York Metro area, created a brand-new job, chief experience officer, and the first person to hold it was Sven Gierlinger.

One of the first things he did was read what patients were actually writing on their surveys. Not the scores, the words. And on the subject of food, the words were brutal " Every meal brought to me was unappetizing. People need good food to get better." And the one that became a headline on Northwell's own website, "The food was atrocious, not fit to feed my dog."

The system was serving more than 10 million meals a year. Its food scores sat at the ninth percentile nationally, meaning roughly 90% of American hospitals scored better. And the CEO, Michael Dowling, was getting complaint letters about meatloaf. So for a survivor who had lived on hospital protein drinks, this was personal.

Gillinger recruited Bruno Tison, a Michelin-starred chef who had never set foot in a hospital kitchen, and rebuilt the food system. Took the fryers out, took the can openers out, brought in fresh ingredients and made-to-order menus. And because fresh ingredients turned out to cost less than processed ones once you stop throwing half of them away, the whole overhaul stayed inside the existing food budget.

Food scores went from the ninth percentile to the 84th. But the food was the easy problem. It was visible. It was concrete. You can taste the fix. The harder problem was hiding somewhere else in the survey. Since 2006, every hospital in America has been graded by its own patients through a standard survey nicknamed HCAHPS.

Three of its questions ask about the same thing the ER researchers kept finding: Did your doctor treat you with respect, listen carefully, and explain things in a way that you can understand? And in 2015, on that doctor communication measure, Northwell ranked in the 29th percentile compared with peer hospitals nationally.

So on the question of whether doctors talked to patients like human beings, about seven in ten American hospitals were doing better. Now, the standard corporate playbook here writes itself. Blast out a memo, tie scores to bonuses, put the number on a dashboard, and squeeze.

That's the good heart playbook, and we know how it ends. The number moves, but the experience often doesn't. Northwell did something a little slower. Working with its own medical school and with a group called the Academy of Communication in Healthcare, Gillingham's office built a course, Relationship-Centered Communication, a day of small groups, live demonstrations, and role play, teaching doctors a structure for the medical conversation.

Build the relationship, understand what's actually going on with this person, and manage the plan together. Communication with the patient instead of at them. Two details tell you why it worked. First, the course was voluntary. No mandate, no penalty for skipping it. But second, the faculty were forty-six of Northwell's own physicians and clinicians teaching on top of their day jobs with no extra pay.

When COVID hit in twenty twenty, they moved the whole thing to Zoom rather than pause it, and they even added telehealth scenarios because suddenly every bedside conversation was now happening through a screen. So here's what the scoreboard looked like after six years. Between the course's launch in June twenty seventeen and June twenty twenty-three, more than three thousand three hundred Northwell physicians and providers went through it.

In that same window, the system's doctor communication ranking climbed from the thirty-sixth percentile to the fifty-eighth, from the bottom third of American hospitals to just above the middle, against peers who were all trying to improve as well. The single biggest gain was on the question, did your doctor listen carefully?

That went up twenty-four percentile points. And the measure closest to a business's bottom line, would you recommend this hospital, rose from the forty-second percentile to the sixty-fourth. Before the program, four of Northwell's adult hospitals scored at or above the national median on doctor communication.

By late twenty twenty-three, it was eight. Northwell's own team, writing this up in the Journal of Patient Experience, cautions that the course may not be the only factor in those gains. The system overhauled access and care models and its medical records in those same years. That's a before and after story, not a controlled experiment.

The people who ran it say so out loud, which is why I trust the rest of their numbers. But there's a bigger reason to take communication seriously as an intervention, not as a nicety. In 2018, the British Medical Journal published results from seven North American hospitals that restructured how care teams talk with families during morning rounds.

They use plain language, families are invited to speak first, and there's a read back at the end to confirm that everyone understood. Harmful medical errors fell by about 38%. Families understood the plan, and the care itself became safer. One of Northwell's volunteer faculty members put the whole philosophy into a single line, quote, "Patients and families don't experience our caring and compassion until we give voice to it.

Our long hours, our sacrifice is not interpreted as compassion. What is are the words we use and the way we use them." End quote. And in this entire story, no one ever promised a shorter visit. The fix was never speed. A man who spent 90 days unable to move, with plenty of time to think about what actually reaches a person lying in that bed, bet that 20-plus hospitals on the i-

And in this whole story, nobody ever promised a shorter visit. The fix was never speed. A man who spent 90 days unable to move with plenty of time to think about what actually reaches a person lying in that bed bet 20-plus hospitals on the idea that the metric worth chasing was whether anybody explained anything, and the dashboard followed the conversation, not the other way around.

So the question for your business: somewhere in your operation, there's a stopwatch, a number that's easy to track, easy to benchmark, easy to report at the Monday meeting. What's your version of the conversation, the thing that your customers are actually grading you on while you're busy timing yourself?

I wanna end this week at a cruising altitude of 35,000 feet with the purest specimen of Goodhart's law in American life. Since the late 1980s, the US Department of Transportation has required airlines to report on-time performance, and it defines on time as arriving within 15 minutes of schedule.

Those rankings get published, airlines advertise them, careers ride on them. But there is a flaw built into the design. The government measures you against the schedule, and you write the schedule.

In 2018, a trio of economists published a study in the Review of Industrial Organization with a title that gives away the ending, "Do Airlines Pad Their Schedules?" They analyzed US flights from 1990 through 2016 and found that scheduled flight times have expanded steadily, with the biggest jumps after 2008.

Actual flight times grew, too, but the schedules grew faster than the flights. So on paper, delays went down while the same trips, wheels up to wheels down, take longer than they did in 1990. The Wall Street Journal captured it in a headline back in 2010, "Why a Six-Hour Flight Now Takes Seven." And the 15-minute line itself warps behavior in the air.

Earlier research by Forbes and Lederman found airlines systematically speed up the flights that are projected to land right around that 15-minute threshold. Fourteen minutes late still counts as on time, 16 minutes late doesn't. So the flights that get the extra push are the ones near that line, not necessarily the ones carrying the most potentially stranded passengers.

None of this requires anybody to lie. Every departure time, every arrival, every schedule is accurately reported. The airlines simply do what every organization does when a measure becomes a target, they optimize the measure. Your flight lands, the pilot announces you've arrived 20 minutes early, and everybody feels like they're winning because a victory over a schedule that was padded precisely so you could beat it.

The metric has never looked better. Your Tuesday has never been longer.

That's our show this week. If you take one thing away from this episode, I'd like it to be this. Your customers are not grading you on your dashboard. The ER patients were counting explanations, not minutes. Northwell's patients just wanted to know whether anybody was listening.

And the number that's easiest to measure has a way of replacing the experience it's supposed to represent, and nobody notices that swap until the customers stop coming back. So this week, ask the good heart question about your own business. What would your customers say you should be measuring instead?

And if the answer is, "I don't know what my customers are confused about," that's a fixable problem. It's exactly what our team built the Experience Helpdesk for. Real customer experience specialists answering your specific questions about where your customers get stuck. You can join us over at experiencehelpdesk.com.

Thanks for listening to Marginally Better. If something here was useful, send this episode to one person who runs a business. That's how the show grows, one episode at a time. And a review wherever you listen It helps one more business owner find us. For show notes and the research behind today's episode, head over to marginallybettershow.com or follow the link wherever you've been listening.

Marginally Better is a Calufrax Radio production. Our producer is Nicole Hubbard with research by Connie Evans. I'm Joe Taylor Jr.