In the main config, we can specify which "backend" to use to store DevStream state.
A backend is where to actually store the state file. We support the following types of backend:
stateFile under the
options section is mandatory for local backend.
TL;DR: see the config example:
key under the
ptions section are all mandatory fields for s3 backend.
From release v0.6.0, using AWS S3 to store DevStream state is supported.
More on configuring state here.
In short, we can use the "backend" keyword to specify where to store the state: either locally or in an S3 bucket. If S3 is used, we need to specify the bucket, region, and the S3 key as well.
Config File Using S3 Backend¶
Using the S3 Backend¶
Before reading on, now is a good time to check if you have configured your AWS related environment variables correctly or not.
For macOS/Linux users, do:
For more information, see the official document here.
Then let's run
tiexin@mbp ~/work/devstream-io/devstream $ ./dtm apply 2022-05-30 17:07:59 ℹ [INFO] Apply started. 2022-05-30 17:07:59 ℹ [INFO] Using dir <.devstream> to store plugins. 2022-05-30 17:07:59 ℹ [INFO] Using s3 backend. Bucket: devstream-test-remote-state, region: ap-southeast-1, key: devstream.state. 2022-05-30 17:08:00 ℹ [INFO] Tool (repo-scaffolding/default) found in config but doesn't exist in the state, will be created. Continue? [y/n] Enter a value (Default is n): y 2022-05-30 17:08:08 ℹ [INFO] Start executing the plan. 2022-05-30 17:08:08 ℹ [INFO] Changes count: 1. 2022-05-30 17:08:08 ℹ [INFO] -------------------- [ Processing progress: 1/1. ] -------------------- 2022-05-30 17:08:08 ℹ [INFO] Processing: (repo-scaffolding/default) -> Create ... 2022-05-30 17:08:12 ℹ [INFO] The repo dtm-test-go has been created. 2022-05-30 17:08:29 ✔ [SUCCESS] Tool (repo-scaffolding/default) Create done. 2022-05-30 17:08:29 ℹ [INFO] -------------------- [ Processing done. ] -------------------- 2022-05-30 17:08:29 ✔ [SUCCESS] All plugins applied successfully. 2022-05-30 17:08:29 ✔ [SUCCESS] Apply finished.
As we can see from the output, the S3 backend is used, and it also shows the bucket and key you are using, and in which region this bucket lives.
apply, let's download the state file from S3 and check it out:
And if we open the downloaded file, we will see something similar to the following:
repo-scaffolding_default: name: repo-scaffolding instanceid: default dependson:  options: branch: main image_repo: ironcore864/dtm-test-go org: "" owner: IronCore864 repo: dtm-test-go resource: org: "" outputs: org: "" owner: IronCore864 repo: dtm-test-go repoURL: https://github.com/IronCore864/dtm-test-go.git owner: IronCore864 repoName: dtm-test-go
which is exactly the same as if we were using the local backend to store state.