White Paper: Just-in-Time Database Access Best Practices for Enterprises

Data Masking with GitHub Actions Part 1 - Semantic Type and Global Masking Rule

Estimated: 30 mins

Bytebase is a database DevSecOps platform designed for developers, security, DBA, and platform engineering teams. While it offers an intuitive GUI for managing database schema changes and access control, some teams may want to integrate Bytebase into their existing DevOps platforms using the Bytebase API.

Bytebase provides database dynamic data masking in the Enterprise Plan, which can mask sensitive data in the SQL Editor query result based on the context on the fly. It helps organizations to protect sensitive data from being exposed to unauthorized users.

By using GitHub Actions with Bytebase API, you can implement policy-as-code to apply database masking policies via the GitOps workflow. This tutorial will guide you through the process.

This is Part 1 of our tutorial series on implementing automated database masking using GitHub Actions:

Overview

In this tutorial, you'll learn how to automate database masking policies using GitHub Actions and the Bytebase API. This integration allows you to:

  • Manage data masking rules as code
  • Automatically apply masking policies when PRs are merged

Here is a merged pull request as an example, for this tutorial, only "Semantic Type and Global Masking Rule" is covered.

The complete code for this tutorial is available at: database-security-github-actions-example

Prerequisites

Before you begin, make sure you have:

  • Docker installed
  • A GitHub account
  • An ngrok account
  • Bytebase Enterprise Plan subscription (you can request a free trial)

Setup Instructions

Step 1 - Start Bytebase in Docker and set the External URL generated by ngrok

ngrok is a reverse proxy tunnel, and in our case, we need it for a public network address in order to receive webhooks from VCS. ngrok we used here is for demonstration purposes. For production use, we recommend using Caddy.

ngrok-reverse-proxy

  1. Run Bytebase in Docker with the following command:

    docker run --rm --init \
  --name bytebase \
  --publish 8080:8080 --pull always \
  --volume ~/.bytebase/data:/var/opt/bytebase \
  bytebase/bytebase:3.3.0

  2. Bytebase is running successfully in Docker, and you can visit it via localhost:8080. Register an admin account and it will be granted the workspace admin role automatically.

  3. Login to ngrok Dashboard and complete the Getting Started steps to install and configure. If you want to use the same domain each time you launch ngrok, go to Cloud Edge > Domains, where you'll find the domain <<YOURS>>.ngrok-free.app linked to your account.

  4. Run the ngrok command ngrok http --domain=<<YOURS>>.ngrok-free.app 8080 to start ngrok with your specific domain, and you will see the output displayed below:

    terminal-ngrok

  5. Log in Bytebase and click the gear icon (Settings) on the top right. Click General under Workspace. Paste <<YOURS>>.ngrok-free.app as External URL under Network section and click Update.

    external-url

  6. Now you can access Bytebase via <<YOURS>>.ngrok-free.app.

Step 2 - Create Service Account

  1. Log in as the admin user, and go to Security & Policy > Users & Groups. Click + Add User, fill in with api-example, choose the DBA role that is sufficient for this tutorial and click Confirm. service-account-create

  2. Find the newly created service account and click on Copy Service Key. We will use this token to authenticate the API calls. service-account-key

Step 3 - Prepare Test Data

  1. Bytebase by default provides a project Sample Project with two database hr_test and hr_prod.
  2. Click IAM & Admin > Users & Groups on the left sidebar. Add users: dev@example.com, dev2@example.com and dev3@example.com with no roles.
  3. Add a group contractor@example.com with dev3@example.com as a member.
  4. Go to project Sample Project, click Manage > Members on the left sidebar.
  5. Click Grant Access and select users dev@example.com and dev2@example.com with Developer role and group contractor@example.com with Querier role.

Step 4 - Configure GitHub Actions

  1. Fork Database Security GitHub Actions Example.

  2. Click Settings and then click Secrets and variables > Actions. Add the following secrets:

    • BYTEBASE_URL: ngrok external URL
    • BYTEBASE_SERVICE_KEY: api-example@service.bytebase.com
    • BYTEBASE_SERVICE_SECRET: service key copied in the previous step

Step 5 - Understanding the GitHub Workflow

Let's dig into the GitHub Actions workflow code:

  1. Trigger: Workflow runs when PRs are merged to main.

  2. Authentication: The step Login Bytebase will log in Bytebase using the official bytebase-login action. The variables you configured in the GitHub Secrets and variables are mapped to the variables in the action.

  3. File Detection: The step Get changed files will monitor the changed files in the pull request. For this workflow, we only care about semantic type and global masking rule. So masking/semantic-type.json and masking/global-masking-rule.json are filtered out.

  4. PR Feedback: The step Comment on PR will comment on the merged pull to notify the result.

Semantic Type

Masking algorithm is associated with Semantic Type. You define semantic types and apply them to global masking rule or table columns. For example, you may define a semantic type birth_date with a masking algorithm to mask month and day.

In Bytebase Console

Go to Data Access > Semantic Types and click Add. You can create a new semantic type with a name and description, and customize the masking algorithm.

bb-add-semantic-type

bb-add-algorithm

In GitHub Workflow

Find the step Apply semantic type, which will apply the semantic type to the database via API. All the masking algorithms should be defined in one file in the root directory as masking/semantic-type.json.

response=$(curl -s -w "\n%{http_code}" --request PATCH "${BYTEBASE_API_URL}/settings/bb.workspace.semantic-types?allow_missing=true" \
   --header "Authorization: Bearer ${BYTEBASE_TOKEN}" \
   --header "Content-Type: application/json" \
   --data @"$CHANGED_FILE")

By changing file masking/semantic-type.json, creating a PR and merging, the semantic types will be applied. Go to Bytebase console, click Data Access > Semantic Types, you can see the applied semantic types.

Global Masking Rule

Global Masking Rule is configured by the admin.

In Bytebase Console

Go to Data Access > Global Masking and click Add. You can create a new global masking rule mapping condition to a semantic type.

bb-global-masking

In GitHub Workflow

Find the step Apply global masking rule, which will apply the global masking rule to the database via API. All the global masking rules should be defined in one file in the root directory as masking/global-masking-rule.json. The code it calls Bytebase API is as follows:

response=$(curl -s -w "\n%{http_code}" --request PATCH "${BYTEBASE_API_URL}/policies/masking_rule?allow_missing=true&update_mask=payload" \
   --header "Authorization: Bearer ${BYTEBASE_TOKEN}" \
   --header "Content-Type: application/json" \
   --data @"$CHANGED_FILE")

By changing file masking/global-masking-rule.json, creating a PR and merge, you can apply the global masking rules.

Next Step: Column Masking and Masking Exemption
Edit this page on GitHub

Subscribe to Newsletter

By subscribing, you agree with Bytebase's Terms of Service and Privacy Policy.