Master SQL Review with GUI, GitOps and API
When modifying data in the database, it's crucial to ensure that the changes are both safe and accurate. Bytebase offers a feature called SQL Review, which allows you to evaluate your SQL changes before they are applied to the database. SQL Review can be initiated through the Bytebase GUI, a GitOps workflow, or via API.
This tutorial will walk you through using SQL Review in Bytebase to improve your database schema change process. The Community Plan is sufficient for completing this tutorial.
Prerequisites
- Make sure you have Docker installed, and if you donβt have important existing Bytebase data locally, you can start over from scratch by
rm -rf ~/.bytebase/data
.
Preparation: Configure SQL Reviews in Bytebase
-
Copy and paste the commands to start one Bytebase via Docker.
-
Register an admin account and it will be granted the
workspace admin
role automatically. -
Click Security & Policy > SQL Review on the left side menu. You can find there is already one default SQL Review policy named
Default SQL Review Policy
. -
Click Edit, you'll be redirected to the SQL Review policy edit page. Here you may specify different rules for different database engines, and here our sample database is PostgreSQL, so we'll focus on the PostgreSQL rules.
-
The default policy is attached to Environment
Prod
, you may click Change attached resources to attach it to other Environments or Projects. Here we'll keep it as is. You may also click Change the template to change the rules. -
Click Change the template, keep the default template
Default SQL Review Template
and click Next. Add one ruleEnforce setting default value on columns
, set Error Level toError
and click Confirm and update.
Case 1: Trigger SQL Review in Bytebase GUI
-
Go to
Sample Project
, since the default SQL Review policy is attached to theProd
environment, we'll create a new table on it. Click Database > Databases on the left side menu, selecthr_prod
and click Edit Schema. -
Click Add Table while choosing
Prod hr_prod > public > Tables
on the left bar. Create a new tablet1
with one columnid
andname
, intentionly unclick Not Null forname
and click Preview issue. -
The SQL Review will run automatically before the issue preview, and you can see the violations against our defined rules. Click Continue anyway.
-
After redirecting to the issue page, click Create and Continue anyway, the issue is created with SQL review red marked as there is some
Error
. -
You may resolve this by changing the SQL as the following:
Here you need firstly click Settings on the left side menu, and then check Allow modify statement or otherwise the SQL is not editable.
Case 2: Trigger SQL Review in GitOps Workflow
Start ngrok
ngrok is a reverse proxy tunnel, and in our case, we need it for a public network address in order to receive webhooks from VCS. ngrok we used here is for demonstration purposes. For production use, we recommend using Caddy.
-
Run Bytebase in Docker with the following command:
-
Bytebase is running successfully in Docker, and you can visit it via
localhost:8080
. Register an admin account and it will be granted theworkspace admin
role automatically. -
Login to ngrok Dashboard and complete the Getting Started steps to install and configure. If you want to use the same domain each time you launch ngrok, go to Cloud Edge > Domains, where you'll find the domain
<<YOURS>>.ngrok-free.app
linked to your account. -
Run the ngrok command
ngrok http --domain=<<YOURS>>.ngrok-free.app 8080
to start ngrok with your specific domain, and you will see the output displayed below: -
Log in Bytebase and click the gear icon (Settings) on the top right. Click General under Workspace. Paste
<<YOURS>>.ngrok-free.app
as External URL under Network section and click Update. -
Now you can access Bytebase via
<<YOURS>>.ngrok-free.app
.
Configure GitOps
-
Use your domain from ngrok to visit Bytebase (if you use localhost, it won't work). Click Integrations > GitOps on the left side menu and then follow the instructions to set up a GitHub.com integration.
-
Go to your GitHub repository, click Settings > Developer Settings > Personal access tokens and generate a new token. Choose
All repository
as Repository access and configure the token permission according to Bytebase's instructions. -
Go back to Bytebase, paste the personal access token and click Confirm and add.
You may read the GitOps document for more details.
-
-
Create a repository
bb-gitops-814
in your GitHub account which will be used to submit SQL changes. -
Create another project
Sample Project GitOps
and click New DB on the Database > Databases. ChooseProd Sample Instance
, namegitops_prod
, environmentProd
, database owner namebbsample
and click Create. -
Within the project, click Integration > GitOps and then Add Repository. Choose the repository
bb-gitops-814
and click Add GitOps connector. Follow the 3 steps, keep the default settings and click Finish. Now the workflow is set. -
Go back to
bb-gitops-814
on GitHub, create a file202408141500_create_t2.sql
underbb-gitops-814/bytebase/
folder. Add the following SQL: -
Commit changes, create a new branch and click Propose changes. Then click Create pull request. On the new Pull Request page, you may see the SQL Review comment.
-
Edit the commit as follows and commit directly to the branch, and you may find the SQL Review comment is updated.
-
Merge the pull request in GitHub, and you may find the issue created and executed in Bytebase.
Case 3: Trigger SQL Review by Bytebase API
You may call Bytebase API in your internal portal or GitHub Actions to trigger SQL Review. We don't go into details here, but you may refer to the following examples:
-
Sample portal to call Bytebase API to trigger schema change including SQL Review.
-
π API in GitHub Actions Example
Sample github custom actions to call Bytebase API to coordinate the schema migration in Bytebase with the GitHub PR workflow.
Summary
Now you have learned how to trigger SQL Review in Bytebase GUI, GitHub, or by API. You may also refer to the SQL Review document for more details.