Fixing the reporting pipeline that broke before every meeting

The situation

LexHive builds analytics and call-centre operations for US advertisers. The product was a set of Tableau dashboards on top of dbt and Tableau Prep jobs, and client analysts opened them in the morning.

A failed run meant the screen showed yesterday's numbers, or nothing. About one run in ten failed. The team had learned to check before trusting, which is a bad habit to teach a client.

Reliability before features

Cutting failures from about 10% to under 5% was unglamorous work. Read the job logs. Find the transforms that assumed a column would always exist. Put the checks in a place where they fail loudly at five in the morning instead of quietly at nine.

It was the first thing I fixed because everything else on the team sat on top of it. A dashboard nobody trusts is a dashboard nobody uses, and no amount of new charts fixes that.

The slow dashboards

The slowest dashboards took about a minute to open, long enough that people stopped opening them. I redesigned the pipelines behind them and trimmed the compute each refresh needed. Load time fell from roughly a minute to 15 seconds or less.

One conversion number

Marketing spent money with advertisers, and the call centre answered the phones. Each system kept its own version of a lead, so both sides could be right and still disagree.

I joined marketing lead data to call-centre outcomes on a single conversion number, and wrote the ETL that kept it current, so the weekly argument about performance started from the same figure.

A model that changed how staffing was planned

Headcount planning leaned on an average month. I trained a random forest on historical call volume to forecast staffing needs, and shipped it as a model file the pipeline ran on every refresh. The forecast arrived with the data instead of in a separate spreadsheet that someone had to remember to update.

I will be straight about the value. It helped, less than the pitch implied, and it did not replace the judgement of the person doing the planning. That is the right bar for a model like this.

What I would do differently

I would build the failure alerting before the dashboards it protects. For the first months I found out that a run had broken the same way the client did, which is not a monitoring strategy.

What this proves

I do not claim revenue growth at this company, because revenue moved for reasons that had little to do with me. What I can claim is reliability and speed, measured before and after, on a stack that other people depended on every morning.

If your reporting is late every day, the fix is usually closer to the models than to the charts.