A few days ago I started to get more involved with the @splinterlands / @sps.dao node validator software, as some technical changes have been made to the software recently. The aim of the software is to validate all transactions for Splinterlands in connection with the $SPS and more. The software has been in development for some time but is now picking up speed as our Splinterlands DAO is investing in the development. Anyway, you probably already know this story.
We currently have a testing / developer group of 7 people, in addition to @clayboyn as contact person for the DAO and coordinator, there are also team members and developers in the Discord group - and we are making progress. Yesterday was the day when I said - now the software is up to date, I'll rent an external virtual server at my expense and install the software. No sooner said than done - I rented a VPS from a German hoster and started with the basic configuration of the server. After installing updates and the firewall, I started with the software - it was surprisingly easy to install the software with the current Testnet data. You can find the software here: https://github.com/TheSPSDAO/SPS-Validator
Overnight, my node actually ran great and signed a block more or less every 3 seconds and wrote it to our Hive blockchain as custom_json. You can see this under my Splinterlands Node Validator account @louis.sps e.g. at https://hivehub.dev/@louis.sps
When I went to bed, I didn't think about anything else at first, because we had agreed that we would leave the servers running overnight in testnet mode so that the others could continue working and testing later. So I just let my node run - when I looked at the node this morning, it had crashed - I didn't think anything of it at first - at around 4 o'clock in the morning I noticed, according to the logs, that the connection had been interrupted. I reported this and during the course of the day I did some further research into why this had happened. And now comes the stupid part of this article.
After further research in the team, it was found that when retrieving the current PostgreSQL database, the one in my Docker container was no longer accessible - What had happened? Automatic queries and encryption from outside happened - Unfortunately, the Docker container was set in such a way that it had a well... what can I say - very weak password as the default password - and simply “postgres” as the password. Well, it was clear that it wouldn't take long for automatic scrappers from the network to scan the ports and then gain access. I ran a command according to the instructions to see what exactly is going on with the database:
docker exec -e PGPASSWORD=postgres -it splinterlands-validator-pg-1 psql -U postgres -c "\l"
Something like the following should actually happen:
but instead I saw the following and bang, I could already guess what had happened:
You can see the line “readme_to_recover” and the line “validator” is missing. This means that the data in the database has been deleted / encrypted. Well, up to this point it's not a neck breaker, because on the one hand it's just a test system for development and help to get the software ready and on the other hand there are no important details in the database - and in the end no drama at all that everything runs in the Docker environment. You can read more details about this type of attack at https://medium.com/@border0/help-my-database-was-compromised-ec68ef15df65.
Well, I more or less wiped everything and started from scratch, blocking all incoming connections. I'll leave the box running again tonight and see if I wake up to a surprise tomorrow. But one lesson we should take away from this is the following - A) That we should not use standard passwords or connection encryption - regardless of whether we are working in a testnet / actively on it and the server / system is available to the outside world and B) that we include something like the Postgre data / user data in the .env - and that this is also in the documentation. Fortunately we noticed this today :)
Lessons learned :)
FINALLY
Works ;)
Vote for my Hive Witness
U can vote for my Witness using Hive Keychain here: https://vote.hive.uno/@louis.witness
Vote for my Hive Engine Witness
Vote for my Witness on Hive-Engine using Primersion Tool: https://primersion.com/he-witnesses Enter your Username and search for louis.witness
Glad you learned that lesson now and not later on. I've been holding a node license for a while, so I am happy to hear this is finally progressing.
It's a good lesson to learn and luckily you learned it on testnet :)
I also plan to set up a node! Looking forward to it.
Which specs does to machine have that you set up? How much is it per month?
The Server itself is under 20 Bucks a month - and have the following specs: 4vCPU - 8 GB RAM - 160 GB Disk lokal - running Ubuntu 24.04 and costs me roughly 2 cents an hour.
Thanks!