dbt

dbt is the tool that made our data engineering practice scalable. Before dbt, transformation logic lived in scheduled SQL scripts, n8n workflows, or worse — in people's heads. dbt brought version control, testing, and documentation to SQL transformations, which means we can hand off data models to clients and they can actually maintain them.

dbt is the tool that made our data engineering practice scalable. Before dbt, transformation logic lived in scheduled SQL scripts, n8n workflows, or worse — in people's heads. dbt brought version control, testing, and documentation to SQL transformations, which means we can hand off data models to clients and they can actually maintain them.

Where It Excels

The SQL-first approach is dbt's genius. If you can write SELECT statements, you can use dbt. Models are just SQL files with some Jinja templating for dynamic references. This means any analyst who knows SQL can contribute to the transformation layer — you don't need a Python data engineer. For growth-stage companies, this dramatically expands who can maintain the data stack.

dbt Core is free and open-source, which means the transformation layer costs nothing. We run it via n8n on a schedule (every 6 hours for most clients) or trigger it after data loads complete. The testing framework (schema tests, data tests) catches data quality issues before they show up in dashboards — things like null values in required fields, duplicate primary keys, or referential integrity breaks.

Where It Falls Short

dbt has a real learning curve for people who aren't comfortable with command-line tools, Git, and YAML configuration. The "just SQL" marketing undersells the Jinja templating, ref() functions, and project structure that you need to learn. For small clients with simple transformation needs, BigQuery scheduled queries or views might be simpler than introducing dbt into the stack.

dbt Cloud pricing at $100/developer/month for the Team plan adds up for larger teams. And the IDE experience in dbt Cloud, while improving, still doesn't match working locally with VS Code. We've also found that dbt's documentation generation, while useful, produces docs that are more developer-readable than business-user-readable.

How JMK Uses It

dbt is our transformation layer in the BigQuery/Snowflake data stack. Standard setup: dbt Core running on a schedule via n8n, with models organized into staging (raw data cleanup), intermediate (business logic), and marts (final analytical tables). We use dbt tests to validate data quality and dbt docs to generate documentation for client handoff. For clients who want managed dbt, we set up dbt Cloud.

Who It's Right For

• Companies with a data warehouse (BigQuery, Snowflake) that need structured, tested transformation logic.
• Teams where analysts know SQL and want to contribute to the data transformation layer without learning Python.
• Organizations that value version-controlled, documented, tested data pipelines.
• Growth-stage data teams building their first proper analytics engineering practice.

Who Should Look Elsewhere

• Small businesses with simple analytics needs — BigQuery views or scheduled queries may be sufficient.
• Teams with no SQL expertise — dbt's value depends on SQL proficiency.
• Companies that need real-time transformations — dbt is batch-oriented by design.