dbt入門

$ mkdir dbt-pip && cd dbt-pip
$ python -m venv venv
$ source venv/bin/activate
$ pip install --upgrade pip

$ pip install dbt-bigquery

$ dbt --version

Core:
  - installed: 1.1.1
  - latest:    1.1.1 - Up to date!

Plugins:
  - bigquery: 1.1.0 - Up to date!

$ dbt init my_dbt_project

04:58:32  Running with dbt=1.1.1

Which database would you like to use?
[1] bigquery
(Don't see the one you want? https://docs.getdbt.com/docs/available-adapters)

Enter a number: 1

[1] oauth
[2] service_account

Desired authentication method option (enter a number): 2

keyfile (/path/to/bigquery/keyfile.json): /PATH/TO/YOUR CLIENT SECRET FILE

project (GCP project id): <PROJECT ID>

dataset (the name of your dbt dataset): sample_dbt

threads (1 or more): 1

job_execution_timeout_seconds [300]: 300

[1] US
[2] EU
Desired location option (enter a number): 1

04:59:32  Profile my_dbt_project written to /Users/xxx/.dbt/profiles.yml using target's profile_template.yml and your supplied values. Run 'dbt debug' to validate the connection.
04:59:32
Your new dbt project "my_dbt_project" was created!

For more information on how to configure the profiles.yml file,
please consult the dbt documentation here:

  https://docs.getdbt.com/docs/configure-your-profile

One more thing:

Need help? Don't hesitate to reach out to us via GitHub issues or on Slack:

  https://community.getdbt.com/

Happy modeling!

$ cat ~/.dbt/profiles.yml

my_dbt_project:
  outputs:
    dev:
      dataset: sample_dbt
      job_execution_timeout_seconds: 300
      job_retries: 1
      keyfile: <PATH TO CLIENT SECRET FILE>
      location: US
      method: service-account
      priority: interactive
      project: <PROJECT ID>
      threads: 1
      type: bigquery
  target: dev

$ dbt debug

Connection test: [OK connection ok]

/models/customers.sql

with customers as (
    select
        id as customer_id,
        first_name,
        last_name
    from sample_dbt.raw_customers
),
orders as (
    select
        id as order_id,
        user_id as customer_id,
        order_date,
        status
    from sample_dbt.raw_orders
),
customer_orders as (
    select
        customer_id,
        min(order_date) as first_order_date,
        max(order_date) as most_recent_order_date,
        count(order_id) as number_of_orders
    from orders
    group by 1

),
final as (
    select
        customers.customer_id,
        customers.first_name,
        customers.last_name,
        customer_orders.first_order_date,
        customer_orders.most_recent_order_date,
        coalesce(customer_orders.number_of_orders, 0) as number_of_orders
    from customers
    left join customer_orders using (customer_id)
)

select * from final

models:
  my_dbt_project:
    +materialized: table

$ dbt run

select
  order_id,
  {% for payment_method in ["bank_transfer", "credit_card", "coupon"] %}
  sum(case when payment_method = '{{payment_method}}' then amount end) as {{payment_method}}_amount,
  {% endfor %}
  sum(amount)
from payments
group by 1

{% macro dollar_to_yen(col_name, rate=100) %}
  {{col_name}} * rate
{% endmacro %}

select
  id,
  dollar_to_yen('price', 90)
from orders
group by 1