dbt build: Simplify Scheduling Your Data Models
The dbt build command is a comprehensive command in dbt that streamlines the workflow for building, testing, and deploying data models.
It combines several individual commands (dbt run, dbt test, dbt snapshot, dbt seed) into a single operation, executing them in the correct order based on your project’s dependency graph (DAG).
What Does dbt build Do?
The dbt build command performs the following actions:
- Runs models: Executes SQL models defined in your project.
- Tests models: Runs tests on your models to ensure data integrity.
- Snapshots data: Captures historical data snapshots as defined by your snapshot configurations.
- Seeds data: Loads data from CSV files into your data warehouse for use in models.
These tasks are performed in DAG order for either selected resources or the entire project, ensuring that dependencies are correctly managed.
Why Use dbt build Over Individual Commands?
Using dbt build simplifies the process of preparing your data environment by combining multiple steps into one. This means you don’t have to manually orchestrate the order of run, test, snapshot, and seed commands — dbt build handles this automatically. It ensures that:
- Models are built before they are tested.
- Snapshots and seed data are updated before models and tests that depend on them are executed.
Additionally, dbt build helps maintain a clean and error-free data pipeline by enforcing test checks before progressing with dependent builds.
If a test fails, subsequent dependent models will not be built, which helps in identifying and isolating issues early in the development cycle.
How to Use dbt build
To use the dbt build command, simply type it into your command line while in your dbt project directory. Here’s an example of what the command might look like in action:
_28$ dbt build_28Running with dbt=0.21.0_28Found 1 model, 4 tests, 1 snapshot, 1 analysis, 300 macros, 0 operations, 1 seed file, 2 sources, 0 exposures_28_2818:49:43 | Concurrency: 1 threads (target='dev')_2818:49:43 |_2818:49:43 | 1 of 8 START seed file analytics.seed_data............................ [RUN]_2818:49:43 | 1 of 8 OK loaded seed file analytics.seed_data........................ [INSERT 100 in 0.05s]_2818:49:43 | 2 of 8 START view model analytics.customer_orders..................... [RUN]_2818:49:43 | 2 of 8 OK created view model analytics.customer_orders................ [CREATE VIEW in 0.10s]_2818:49:43 | 3 of 8 START test not_null_customer_orders_customer_id................ [RUN]_2818:49:44 | 3 of 8 PASS not_null_customer_orders_customer_id...................... [PASS in 0.06s]_2818:49:44 | 4 of 8 START test unique_customer_orders_order_id..................... [RUN]_2818:49:44 | 4 of 8 PASS unique_customer_orders_order_id........................... [PASS in 0.04s]_2818:49:44 | 5 of 8 START snapshot analytics.order_history_snapshot................ [RUN]_2818:49:44 | 5 of 8 OK snapshotted analytics.order_history_snapshot................ [INSERT 10 in 0.20s]_2818:49:44 | 6 of 8 START test relationships_customer_orders_refunds............... [RUN]_2818:49:44 | 6 of 8 PASS relationships_customer_orders_refunds..................... [PASS in 0.05s]_2818:49:44 | 7 of 8 START test unique_customer_orders_email........................ [RUN]_2818:49:44 | 7 of 8 PASS unique_customer_orders_email.............................. [PASS in 0.03s]_2818:49:44 | 8 of 8 START model analytics.daily_sales.............................. [RUN]_2818:49:44 | 8 of 8 OK created table model analytics.daily_sales................... [CREATE TABLE in 0.25s]_2818:49:44 |_2818:49:44 | Finished running 1 seed, 2 models, 4 tests, 1 snapshot in 1.01s._28_28Completed successfully_28_28Done. PASS=8 WARN=0 ERROR=0 SKIP=0 TOTAL=8
Artifacts and Outputs
When you run dbt build, it generates a unified set of artifacts:
- Manifest: A JSON file containing metadata about your project.
- Run Results: A JSON file that details the outcome of the models, tests, seeds, and snapshots executed.
These artifacts are essential for tracking the changes and outcomes of each dbt run, providing a clear audit trail for project deployments.
Selecting Resources
dbt build supports standard selection syntax (--select, --exclude, --selector), allowing you to specify exactly which models, tests, seeds, or snapshots to include in the build process. This feature is particularly useful in larger projects where you may want to focus on a subset of your environment.
Y42: Building sources and models together
If you’re looking to streamline the scheduling of your entire data pipeline, including sources, Y42 (opens in a new tab) provides a comprehensive solution that extends beyond the capabilities of standard dbt.
Lineage including both sources and models.
The y42 build command encompasses everything dbt does—running models, seeds, snapshots, and tests—and also includes the management of sources.
This integration ensures that all components of your data architecture are consistently updated and tested, reducing the complexity of maintaining separate processes.
For more details on how to leverage this functionality, you can refer to the Y42 documentation (opens in a new tab).
Manage Sources and dbt Models in one place
Build end-to-end pipelines using a single framework.
Get Started