SPK Network - Meta Data Proposal - Thoughts and Feedback

@eddiespino tagged Testnet Sprinters in SPK Network's discord for feedback on SPK Network Meta Data Proposal by @disregardfiat. It took me a couple reads before I somehow grasped some of the intricacies of the post.

As the subject of @disregardfiat's post title suggests, it's about SPK Network drive's metadata ("data that provides information about other data").

1. Introduction

Similar to what we did in our previous Final Testnet Sprint post, we'll break down the post point-by-point, and provide feedback for each.

The subject of this feedback post is about formalizing metadata for SPK Network files. These metadata will be stored as customJson in Hive transactions.

The proposed metadata is composed of 6 items:

1. Auto-Renew (1 character)
2. Encryption and Sharing (string of characters that starts with #)
3. File Names (string - four slots)
4. File Types (string - four slots)
5. Thumbnails (string - four slots - full resource URL)
6. Security and Visibility (string - 6 bits, base64 numbers - 0 to 63)
   a. Licensing
   b. Labels

If you need or want something visual of what a sample customJson transaction looks like, here's one we made for testing:

{
  "block_num": 86770404,
  "expiration": "2024-07-01T13:53:40",
  "extensions": [],
  "operations": [
    [
      "custom_json",
      {
        "id": "spkccT_channel_open",
        "json": "{\"broca\":8000,\"broker\":\"dlux-io\",\"to\":\"eastmael\",\"contract\":\"0\"}",
        "required_auths": [
          "eastmael"
        ],
        "required_posting_auths": []
      }
    ]
  ],
  "ref_block_num": 739,
  "ref_block_prefix": 3049804882,
  "signatures": [
    "..."
  ],
  "transaction_id": "a54678a9a983d310ad59709efd2e7c426ddb8380",
  "transaction_num": 25
}

And here are some example strings provided in the post:

• 1,name1,,,,name2,,,,name3,,,
• 1,name1,mp3,,,name2,jpeg,,,name3,txt,,
• 1,cool,jpg,,6-
• 1,cool,jpg,,A-7
• 1#encKey@username,obscured,xxx,,1--15u

2. Details

The objective of storing this metadata is for better User eXperience (UX) for frontends who will be supporting or displaying SPK Network Files.

Borrowing the images from @disregardfiat's post, once these metadata are stored on the chain or together with the contract, displaying these files on frontends will enable visual representations of it like these (click on the images to enlarge):

Let's look into each metadata and see if we can share some thoughts.

2.1. Auto-Renew (1 character)

When a contract is up for expiration the smart contract will check if the Auto-Renew bit is set to true, then will use up to half of their available Broca to extend the contract for a month.
...
The first character of the metadata string will contain contract level flags like AutoRenew. Currently there is only the 1; leaving 5 bit flags unused.

2.1.1. Feedback

I think this is one of the critical info that a file contract's metadata should have. Files stored in Web2.0 Google drives don't have contracts attached to them, as such, if we want widespread adoption of Web3.0 files, we need to give users zero to minimal barrier in storing and keeping their files.

2.2. Encryption and Sharing (string of characters that starts with #)

Following the first character of the metadata string is optional contract level encryption keys.
...
This is one AES key encoded as following. 1#encrypted@toaccount;#encrypted@toaccount,...
This will allow authorized accounts to see they can decrypt the file, decrypt the message to them, and use the key to decrypt the files with out making changes to the file. This does mean that once a key is shared it can only be psuedo revoked, as the customJson will live with its data on the blockchain, but changing these keys in the future will allow you to add accounts to the scheme fairly easily.

2.2.1. Feedback

If you don't have any background with file encryption and what different encryption type or strategies are out there, here's link to what AES is. It stands for "Advanced Encryption Standard".

From what we understood or somehow grasp from its description, these AES encrypted string will allow the frontend to easily detect if a file can be decrypted or not. Front-end wise, I think this will allow frontends decide, that if a file is encrypted and if you're access a link or a page that has in the context owned by another author, for example https://www.dlux.io/me#files, then the front-end can decide whether to display a certain file or contract based on this encrypted string. Instead of displaying all contracts without any form of filtering, seeing files only relevant to you has a better UX.

2.3. File Names, File Types, and Thumbnails (string - four slots)

Following the contract level first string in the metadata, each file will have 4 slots on comma separated values. There is full unicode support here, and slot one in the file name. 1,name1,,,,name2,,,,name3,,, This will allow a user to find their files with relative ease.

So the contract-level first string basically refers to the first string in the array of strings stored as customJson and the string that follows it are representations o.

• 1st string - contract level
• 1st slot, 1st string - filename#1
• 1st slot, 2nd string - filetype#1
• 1st slot, 3rd string - thumbnail#1
• 1st slot, 4th string - security and visibility#1
• ...
• 4th slot, 1st string - filename#4
• 4th slot, 2nd string - filetype#4
• 4th slot, 3rd string - thumbnail#4
• 4th slot, 4th string - security and visibility#4

Here's a sample string that uses only the first slot.

• 1,cool,jpg,https://image.com/image-thumbnail.jpg,6-

So basically, this spring represents a file that:

1. Will auto-renews once it contract expires (auto-renew flag is set to '1')
2. Has 'cool' as its filename
3. Has a 'jpg' extension
4. Has a thumbnail whose URL is at https://image.com/image-thumbnail.jpg
5. Has a security and visibility represented with 6- - this will be further discussed below

2.3.1. Feedback

These all looks good. I'm just intrigued why there are four slots. Will one slot not suffice given that it's only one file? Perhaps @disregardfiat can elaborate more on this.

2.4. Security and Visibility (string - four slots)

The last slot in each file if for file level bit flags. All characters in this section are base64 numbers.

• 1 Is the file encrypted
• 2 Is the file a thumbnail
• 4 Is the file NSFW
• 8 Is the file executable
• 16 Unused
• 32 Unused
  ...
  The front ends can hide thumbnails from the file viewer by default, and use them to preview larger files, or obscure files if they are marked NSFW. Additional useful flag suggestions are welcome.

If you're unfamiliar with what Base64 is, here's a link to wikipedia that provides information about it. Base64 basically represents different characters using 6 bits of 0s and 1s. Here are examples:

• Binary 000000 represents the character A
• Binary 000001 represents the character B
• and so on and so forth...

If you're unfamiliar with binary numbers or bits, here's a sample explanation:

• First bit (0 or 1) - value of 1, if bit is 1
• Second bit (0 or 1) - value of 2, if bit is 1
• Third bit (0 or 1) - value of 4, if bit is 1
• Fourth bit (0 or 1) - value of 8, if bit is 1
• Fifth bit (0 or 1) - value of 16, if bit is 1
• Sixth bit (0 or 1) - value of 32, if bit is 1

An example string that uses this last slot: 1,cool,jpg,,6-. The string 6- uses the security and visibility slot. Converting the value 6 to 6-bit binary becomes 011000.

To get the value of 6, we need to have the second (which will have a value of 2 if the bit is 'on') and third bits to 1. Adding the decimal values 2 and 4 will give us 6. This just aims to give a brief intro into this subject. Do a net search or AI chat to get more details. :)

2.4.1. Licensing and Labels

I won't be able to tackle this topic in details anymore, but the licensing and label information are basically part of the security and visibility slot of the custom json string to be stored in the contract.

I'll just copy the example from @disregardfiat's original post:

1#encKey@username,obscured,xxx,,1--15u would be an encrypted file, with an obscured name, a hidden extension, no thumbnail, with an encryption flag, no license, and labeled as an Important Orange Idea

The part of the string to look into is 1--15u. This can be decoded using the table provided in the original post.

2.4.2. Feedback

I needed not go into details on what Base64 and binary numbers do, but for some readers to better grasp on how these metadata works, I tackled the topics to the best of my knowledge.

Moving to this part of the metadata, security and visibility, is important in the Web3.0 given that we are giving users control over data they share on the web. And given that these contracts and transactions will be made publicly available via blockchain ledgers, having some level of security and visibility is something that should not be neglected. Considering this in SPK Network's metada is a good call.

3. Summary

This took a span of more than a week to complete this feedback, but I think I was able to go through each metadata section to a certain level of understandability.

The aim of this post was basically to gather info and provide feedback on the different parts of the metadata. We were able to gather that this metadata will be stored together with the contract as a custom json.

The first part of the metadata represents the contract's auto-renewal. If this part is set to "1" the contract will be auto-renewed, and it will be otherwise if it's value is 0. We were able to provide feedback that this is a critical part of the metadata to gain better UX.

The second part is the encryption and sharing which will be encrypted using AES. We were also able to gather that this contributes to better UX by having front-ends that will support SPK Network files display only relevant files to the user.

The third to fifth parts of the metadata is about file names, file types, and thumbnails. There will be four slots for this. We were able to look at an example string that shows us a sample content or metadata that will represent a certain file. We have also requested @disregardfiat for additional explanation for the need to have four slots for a single file.

And lastly, the last part of the metadata is reserved for a files security and visibility, which will be a Base64 string that will represent different combinations. This last section also includes information regarding licensing and labels, but we were not able to dive deep into it and just referred to a sample string provided. Having a part of the metadata that concerns or involves security and visibility is a good decision.