Schedule a build
With a robust pipeline and data quality tests in place, it's time to take advantage of Y42's scheduling. By setting up scheduled builds, we ensure our data is updated regularly. In this section, we'll set up a scheduler and alert.
Create and run a scheduled build
Set up a schedule
- Open the bottom drawer
- Navigate to
Build scheduler
- Click on
+ Add
- Provide a name for the scheduled build:
orders_every_monday
- Enter the following selector statement:
y42 build -s source:raw_space_fuel_shop.public_orders+
- Click on
Visualize DAG
and verify that you see the DAG preview
Scheduler DAG preview
- Set up the Schedule frequency unit:
Weekly
- Set Options to
Run every Monday
- Click on
Save
' - Commit and push these changes with an appropriate commit title (e.g.,
scheduler: add schedule to build orders every Monday
)
Trigger a manual run
- You should now see the schedule as a row in the
Build scheduler
tab - Click on
Trigger Run
to trigger an ad-hoc out-of-schedule run - You'll see a blue bar appearing in the
Last Runs
column - Once the run finishes, the blue bar will change color to reflect the run status: green for a successful run, yellow for a partially successful run, and red for an invalid one
View the Schedule as code
-
In the Asset Editor, select the
Code
view -
Click on the
orchestrations
directory -
Click on
orders_15m_schedule.yml
-
You should see Y42’s YML structure for orchestrations:
_10version: 2_10_10orchestrations:_10- name: orders_15m_schedule_10config:_10enabled: true_10command: y42 build -s y42 build -s source:raw_space_fuel_shop.public_orders+_10cron: 0 0/1 1/1 * *_10y42:_10apiVersion: v1
Create an alert
It's great to have automatic pipeline runs set up, but we also want to know when the pipeline fails. Therefore, we'll set up an alert that notifies us if something goes wrong.
- Open the bottom drawer
- Navigate to
Alerts
- Click on
Add an alert
- Provide a name for the alert:
invalid_orders_run
- Under
Trigger
, selectorders_15m_schedule
- Select
Partially invalid
andInvalid
as statuses - Under destinations, provide your user email
- Commit and push the changes with an appropriate commit title (e.g.,
alert: add failure alert for orders pipeline
)
Up next
That's all there is to setting up orchestrations! With everything configured and tested, there's just one thing left to do: time to merge our changes to the main
branch.