@steemchiller that is an incredible engineering feat you just described. I think a lot of potential steem developers would be interested to learn more how you do it.
Can you still query a SQLite db which has been compressed with XZ ?
You are viewing a single comment's thread from:
No, that's not possible. I heard that there are compression plugins for SQLite available, but I've never tried them.
My parser automatically decompresses the files when needed without deleting the original archive files. After walking through 100.000 blocks it deletes the current decompressed file and continues with decompressing the next archive file and so on. Decompression for one file takes about 3-5 seconds, so it does not take much time when executing a full re-parse, and I don't need much additional disk space on my server.
Understood, you're decompressing only for cron jobs or full re-parse but not for regular user request. Thank you for the answer.