Case study

Rota do Perfume

A Databricks lakehouse that turns 10 ERP and CRM CSV files into a weekly queue of 200 sales calls, right on the director's screen.

Fictional distributor. The data was generated for this study, in the shape it takes at real companies.

Code on GitHub Back to projects

This week's queue: 200contacts recommended by the model, for 36 sales reps
Actual conversion: 41.7%5 orders from the 12 calls made so far
Lift over random: 4.1x10.1% was the rate when calling at random
Expected revenue: R$ 559Kpurchase probability times average order value

The data every company has

The starting point was 10 ERP and CRM CSV files with over 300,000 rows. Dates in two formats in the same column. Company tax IDs (CNPJ) with stray spaces. Duplicates. The kind of mess that breaks reports and, worse, breaks decisions.

The answer was a layered pipeline (raw, bronze, silver, gold), defined entirely as code with Databricks Asset Bundles. Schemas, jobs, the dashboard and the Genie spaces are versioned and ship in the same deployment.

Three decisions I stand by

Cleaning must not change the business

The R$ 102 million in revenue reconciles exactly from bronze to gold. Data quality tests fail the pipeline when anything diverges. The rule is to fix the transformation, never the test.

Metadata is a contract

Every column an AI queries has an audited business comment. A job task fails if one is missing. That is what makes Genie answer "how much is the queue worth?" with the right number and the word "estimate".

Data becomes action

A purchase propensity model (scikit-learn and MLflow, 4x lift over a random queue) feeds the weekly queue. The team logs the outcome of every contact in the React Databricks App. In the first week, actual conversion was 41.7%, against the 41.5% the model predicted.

The app the director opens on Monday

Three tabs (the app itself is in Portuguese): this week (who to call), tracking (whether the model holds up) and ask (Genie, running with the user's own Unity Catalog permissions).

Rota do Perfume app on the weekly tab, with queue KPIs and the table of customers to call — **This week.** 200 contacts, R$ 559K in expected revenue, 42% predicted conversion vs. 10.1% calling at random. The queue shows the likelihood, the reason for the call, what to offer (in stock) and the outcome log: sold, thinking it over, not interested, no answer.

Tracking tab with 41.7% actual conversion vs. 41.5% predicted by the model and 10.1% calling at random — **Is the model holding up?** 41.7% actual, 41.5% predicted, 10.1% baseline. The sample is still small (confidence interval of 19.3% to 68.0%). The dashboard treats the match as a signal, not as settled proof.

Bar chart of queue size vs. calls made per sales rep, with an outcome table — **Are the reps with the biggest books working them?** The 12 reps with the most contacts in the queue, side by side with the calls already made. Grouping is by ID, not just by name, so reps who share a name don't collide.

The app's ask tab, with Genie listing the next five customers to call — **Ask.** Natural-language questions about the queue, the model and the customers. Queries run with the asker's own Unity Catalog permissions. The generated SQL stays visible, so the number can be checked before it goes into a meeting.

Genie in the workspace, not just in the app

The same metadata contract answers outside the app, in the Databricks Genie space. A business question becomes a number you can verify.

Genie in Databricks answering how many calls were logged and how many became orders — **How many calls have been logged so far?** 12 calls, 5 became orders, 41.67% conversion, with the breakdown by status (sold, no answer, thinking it over, not interested).

Genie in Databricks showing Débora Souza as the rep with the most calls (4) — **Which rep has made the most calls so far?** Débora Souza, with 4 logged calls. The chart compares that with Henrique Oliveira (3). The management question runs on the same data as the queue.

Deployment as code, app in production

The rotaperfume-direcao app is deployed with Databricks Asset Bundles. The serverless warehouse, the Genie space and the React front end ship in the same bundle.

Overview of the running rotaperfume-direcao Databricks App, with the serverless warehouse and the Genie space attached — **Overview.** App running, compute active, source in .bundle. Attached resources: a serverless SQL Warehouse and the Rota do Perfume - Direção Genie space.

Insights tab of the Databricks App with 100% uptime and zero incidents over the last seven days — **Insights.** 100% uptime over the period, zero incidents. App observability lives in the same workspace as the data.

What's next

The project is still evolving. The next steps are extracting the transformations into Python modules that can be tested outside the notebook, and moving infrastructure provisioning to Terraform, so the same environment can be reproduced in any workspace.

Code on GitHub Back to projects