The Fluree Schema Scenario Tool ( fsst ) is a tool for Test Driven Development with the blockchain based graph database FlureeDb.
The tool has many features that we aren't going to discuss in this post, but its main functionality lies in doing test driven development with the Fluree Database. The tool is a script that uses a docker image that embeds and uses the exact same script inside of the docker container. If you have docker installed, then you mostly don't even need to know that fsst uses docker as the outer fsst script will start docker and within docker that also conatins an instance of FlureeDB, and will then invoke a copy of itself inside of the docker image.
It is also possible to use the docker images in a CICD setup in for example in GitLab, and fsst also has features for deployment in such a context, but in this post we are going to look at the typical usage for local testing of database schemas and Fluree Smart functions.
If you aren't running fsst yet, or even have never done anything with FlureeDB yet, you can get fsst using pip
pip install fsst
Or if you are upgrading:
pip install fsst -U
The old behaviour and why it's not ideal
In past versions of fsst, the tool would look for the docker image locally, and only if the docker image wasn't found would it look for the version of the docker image online and download it.
To understand why this isn't always ideal, we need to look at the tag usage that currently get used in docker hub for the fsst+flureedb images.
The docker images on docker hub always come with two products with independent versioning. Sometimes both FlureeDB and fsst get a new version, dometimes only FlureeDB gets a new version and sometimes only fsst gets a new version while the latest version of flureedb stays the same. The fully specified name version of each actual docker image consists of a tag combining both version numbers. For example the latest fully explicit tag on docker hub at time of writing is v0.8.13-fluree-v2.0.4. The same image is currently also available using the tags:
- fluree-v2.0.4 -> v0.8.13-fluree-v2.0.4
- v0.8.13-stable -> v0.8.13-fluree-v2.0.4
- stable -> v0.8.13-fluree-v2.0.4
When FlureePBC releases a 2.0.5 version of FlureeDB, and we build and push a new docker image to docker hub, assuming there is no new version of fsst available yet, the stable tag and the v0.8.13-stable will be set to point to the newer image on github, while the tag fluree-v2.0.4 will keep pointing to the old current image.
- fluree-v2.0.4 -> v0.8.13-fluree-v2.0.4
- v0.8.13-stable -> v0.8.13-fluree-v2.0.5
- stable -> v0.8.13-fluree-v2.0.5
- fluree-v2.0.5 -> v0.8.13-fluree-v2.0.5
If instead there is no new version of FlureeDB, but a newer version of fsst is released and we build and push a new image, then almost the oposite happens, stable and fluree-v2.0.4 end up pointing to the new image while v0.8.13-stable will keep pointing to the old one.
- fluree-v2.0.4 -> v0.8.14-fluree-v2.0.4
- v0.8.13-stable -> v0.8.13-fluree-v2.0.4
- stable -> v0.8.14-fluree-v2.0.4
- v0.8.14-stable -> v0.8.14-fluree-v2.0.4
Now we can see how the old behaviour was a problem. What if we were using stable as our tag? In fact, stable is the default tag for fsst if no tag is specified implicitly. Well, as the old stable is likely available locally, nothing will get fetched from docker hub, and fsst will run with for example an old version of flureedb and not the recently patched and updated version.
The new behaviour
The latest version of fsst now changes this behaviour, but it goes a step further than checking just in one place. The maintenance of fsst has recently moved from this repo to this repo, but old images were not moved. This was done because the maintainer has changed from an individual to a team. So now if fsst starts, it first checks these two repos for the existance of the tag, if for some reasons the same tag (for example stable) exists in two repos it remembers the newest and will check if that specific version exists locally. If it does it will still use the local version, but if it doesn't, even if an other image with that specific tag does exist locally, it will download the newer image with that tag from the repo with the newest image for that tag.
Let's demonstrate.
$ fsst dockertest --runs 1 --hooks --tag v0.8.11-stable
COMMAND: fsst guesttest --target default --network test47879 --runs 1 --hooks --stages ALL
INFO: Newest image with tag v0.8.11-stable is from pibara
INFO: Looking for pibara/fsst:v0.8.11-stable
NOTICE: Not locally found
NOTICE: Fetching docker image from docker hub
- Fetched pibara/fsst:v0.8.11-stable from docker hub
INFO: Fetched docker image
# waiting for default-private-key.txt to appear
The tool found the image on the pibara/fsst repo, and as it was not available locally, it fetched it first. Lets run the same command again.
$ fsst dockertest --runs 1 --hooks --tag v0.8.11-stable
COMMAND: fsst guesttest --target default --network test47885 --runs 1 --hooks --stages ALL
INFO: Newest image with tag v0.8.11-stable is from pibara
INFO: Looking for pibara/fsst:v0.8.11-stable
INFO: No need to fetch from docker hub, newest image available loccally
INFO: Found docker image locally
# waiting for default-private-key.txt to appear
Now let's try a newer one:
$ fsst dockertest --runs 1 --hooks --tag v0.8.13-stable
COMMAND: fsst guesttest --target default --network test47898 --runs 1 --hooks --stages ALL
INFO: Newest image with tag v0.8.13-stable is from dlosatoes
INFO: Looking for dlosatoes/fsst:v0.8.13-stable
NOTICE: Not locally found
NOTICE: Fetching docker image from docker hub
- Fetched dlosatoes/fsst:v0.8.13-stable from docker hub
INFO: Fetched docker image
# waiting for default-private-key.txt to appear
We see the image is fetched from the new repo dlosatoes/fsst.
Independent dual-source releases ?: faster docker images releases
The new setup opens some other possibilities that will only be of use if FlureePBC chooses to use it. We changed fsst to be optimistic about this prospect, so next to the two valid repos, fsst also optimistically checks for an fsst repo under the fluree dockerhub account. The idea is that if FlureePBC ever chooses to use the fsst / fluree docker files to generate the fsst/flureedb images directly after generating and pushing a parent image, a new image would be available whenever either of the two projects releases a new version.
If that happens, stable will always immediately be up to date (currently it may be many weeks before stable gets updated after a new stable flureedb release). For now this isn't the case, but the code is in fsst so if flureePBC ever ends up integrating building the combined image into its releases, it should work instantly.
Not much functionality wa added in this fsst release, but we believe the convenience for the user of not having to worry about manually deleting or fetching images should prove a nice usability feature.
If you haven't checked out FlureeDB or ffst yet, please have a look at other posts from this @aioflureedb account. It's an amazing blockchain powered graph database that should be close to ideal for sidechain projects.