Models
Packages

Packages

Packages represent libraries made available by other users for public or private usage.

Install and use a package

Locate the packages.yml file

In Code Editor mode, open the packages.yml file in the project's root directory. Create one if it does not yet exist. Note that it must be placed at the top-level of your Y42 project, alongside dbt_project.yml.

    • dbt_project.yml
    • packages.yml

    • Add the package

    To install packages in Y42, specify the package's name and version in the packages.yml file. dbt maintains a hub of open-source packages (opens in a new tab) that can be utilized in dbt models.

    For example, to install the dbt_expectations (opens in a new tab) package:

    packages.yml

    _10
    packages:
    _10
    - package: calogica/dbt_expectations
    _10
    version: [">=0.8.0", "<0.9.0"]

    You packages will be automatically installed once you commit your changes.

    Installing and refreshing package dependencies

    To display packages in the Asset List, Code, and Data Lineage, use CMD / CTRL + K and choose Install package dependencies (if you haven't installed any package dependencies yet) or Refresh package dependencies (if you've installed new packages and want to update the packages displayed in the UI).

    Use the package

    We can now reference any function from the dbt_expectation package in our models. For example, to check if a model's row count falls within a range of values, we can use the expect_table_row_count_to_be_between test from the package:

    models/orders.yml

    _13
    version: 2
    _13
    _13
    models:
    _13
    - name: orders
    _13
    tests:
    _13
    - dbt_expectations.expect_table_row_count_to_be_between:
    _13
    min_value: 100
    _13
    max_value: 1000
    _13
    columns:
    _13
    - name: order_id
    _13
    tests: # column-level tests
    _13
    - unique
    _13
    - not_null

    Git packages

    Packages hosted on Git can be installed by utilizing the Git syntax as shown below:

    packages.yml

    _10
    packages:
    _10
    - git: "https://github.com/dbt-labs/dbt-utils.git" # git URL
    _10
    revision: 0.9.2 # tag or branch name

    To add a Git package, add the Git URL and, and specify a revision. The revision can be:

    • a branch name (not recommended)
    • a tagged release
    • a specific commit (full 40-character hash)

    Example of a revision specifying a 40-character hash:

    packages.yml

    _10
    packages:
    _10
    - git: "https://github.com/goes-funky/dbt-data-reliability.git"
    _10
    revision: 20e348d7e683aa6e2ab5e1abafd65cca3517c075

    Avoid using branch names for revisions; they can change over time. Instead, use tags or commit hashes for consistent and predictable builds. You can retrieve the commit hash via the "Commits" menu in the repository, using the "Copy" option.

    Use the copy option to get the full 40-character-hash commit.

    Use the copy option to get the full 40-character-hash commit.

    Package specific configuration

    While many packages in Y42 function immediately upon installation, there are some that require additional steps for configuration. Certain packages, for example, require particular variables to be configured in the dbt_project.yml file for them to operate correctly.

    Example: Configuring the Salesforce package

    Consider the Salesforce package (opens in a new tab) as an example. It requires the salesforce_database and salesforce_schema variables to be configured within the dbt_project.yml file.

    By default, this package will run using your target database and the salesforce schema. If your Salesforce data is not located there, you need to add the following configuration to your root dbt_project.yml file:

    dbt_project.yml

    _10
    # ... Other Configurations ...
    _10
    _10
    vars:
    _10
    salesforce_database: 'your_database_name' # replace with your actual database name
    _10
    salesforce_schema: 'your_schema_name' # replace with your actual schema name
    _10
    _10
    # ... Other Configurations ...

    Always remember to check the documentation of the specific package you are installing. Each package may have its own unique configuration needs that are best detailed in their individual documentation, and configuring them correctly is crucial to ensure they function properly within Y42.

    FAQ

    Are all dbt packages cross-compatible with Y42?

    As is the case with macros, most packages will work out-of-the-box. However, packages that attempt to pattern-match table names within the data warehouse may not run as expected. This is because Y42 executes Virtual Builds, which issues a unique table/view name identifier instead of using the dbt model name.

    MacrosHooks