We have created scheduled queries in the previous post. The query execution results could be forwarded to a pub/sub topic. In this post we developed a function that reviews the execution results and takes action based on state.

I want to build the solution async and reusable. Therefore there are two functions.
The first function is responsible to review the execution results.
The second function is responsible to send emails.
I have built in that way, because I want to utilize email-sender function for project-wide usage. :)

Workflow

  1. BigQuery starts to process query based on schedule time.
  2. The query is succeeded or not, the execution result is forwarded to where specified pub/sub topic in creation process
  3. Pub/sub topic invokes the reviewer function.
  4. The reviewer function checks the body of event data against is it succeeded or not.
    If state is SUCCEEDED, then it does nothing.
    If not, then it publish new message to target topic which is the email-relay.

The reviewer actually does just checking the state is SUCCEEDED or not. Besides it publish new message to email-relay topic, if query is failed.

shoin-posts/gcp-bigquery-scheduled-q-monitoring at master ยท xsetra/shoin-posts
Contains devops scripts to re-use. Contribute to xsetra/shoin-posts development by creating an account on GitHub.
GitHubxsetra