dbt models

dbt model viewed in Data Catalog mode.

Overview

dbt, short for Data Build Tool, is an open-source tool used primarily for transforming data in your warehouse. It allows data pipeline builders to define, maintain and test datasets using modular SQL queries.

Enhanced functionalities

Y42 takes the power of dbt a step further by providing users with seamless integrations between the Data Catalog, Code Editor and Lineage modes. By leveraging Y42's integrated data development environment, dbt models are consistent, reliable and scalable.

Additionally, Y42 adopts a distinct asset execution strategy — Virtual Data Builds, which provides a streamlined approach to running dbt models. By integrating Git source control with its inner workings, Y42's build execution engine ensures the code and data remains in sync across time.

As opposed to dbt build execution, Y42 operations are inherently stateful. This means that each build job in Y42 is contextually aware of previous runs, allowing users to skip build steps for models that have already been materialized from a different Git branch.

Setting up dbt in Y42

dbt models are compatible and optimized for use in Y42 spaces, straight out of the box.

Project configuration

Each Y42 space contains a project configuration YAML file, which extends upon the dbt project schema that dbt users will be familiar with. To view it, navigate to the Y42 project's root directory in Code Editor mode.

Connection profile

All users in a Y42 space share the same dbt connection profile. In other words, all users materialize assets in the same Y42 space dataset (BigQuery) or schema (Snowflake), regardless of which branch they are working on.

Supported features

As of now, Y42 supports most common features of dbt, but there are a few that are not yet supported. Y42 is continually working to provide full parity with dbt in the months to come.

✨ = feature enhancement