In early September, 2020, PlatON conducted the Macro-benchmark under the test conditions the same as EOS, a leading public blockchain with remarkable performance.
Below is the detailed test report of this Macro-benchmark on PlatON.
Test Indexes
The test indexes in this Macro-benchmark are Time to Finality (TTF), Transaction per Second (TPS) and resource index. The data collection interval was 10 secs and the average value of all indexes in each interval will be calculated at the end of each data collection interval.
Transaction to Finality (TTF)
Transaction confirmation time is the final confirmed time of transaction in the whole network. Generally, TTF consists of TI and TC, which are the intervals between blocks, the final verified and confirmed time of block, respectively. In this test, the TTF is calculated according to the combination of block production interval and the final irreversible time of blocks in the consensus mechanism. We will analyze each transaction from its starting to the final irreversible time delay.
Transaction per Second (TPS)
Transaction per Second is the number of successful transactions proceed in each second, the unit is: transaction/second. Here, a successful transaction refers to a transaction been packaged and proceed with receipt. We analyze the block data to set the number of transactions produced in the blocks packaged per unit time (usually, it’s 1 sec) as the final TPS.
Resource Utilization
Resource utilization refers to the system resource when the system is loading, including CPU, memory, disk storage and network traffic.
CPU : the CPU utilization when system is loading;
Memory : the memory utilization when system is loading;
Disk storage : the increased disk storage when the system is loading;
TPS : the amount of data flow in the network per unit time when the system and network work as usual. The unit is Byte/s.
Test Tool
In the test, we applied the test plug-in that is exclusively for performance tests. The transactions are conducted in the plug-in and will be broadcast to the consensus nodes in the P2P way. By doing so, the performance won’t be affected by the RPC interface.
Due to the difference between PlatON and EOS system, the test plug-in for PlatON used in this test and the one for EOS are different to some extent, saying the location of PlatON test accounts and token distribution. EOS test accounts and its token distribution will be initialized in its test plug-in.
Additionally, thanks to the interface support of PlatON, the test plug-in can count the TPS indexes precisely, while the TPS of EOS is counted according to the analysis of block data.
PlatON applied the standard performance assessment model it defined before in this test, there will be complete test later.
Testing Environment
We used 8000 test accounts in the Macro-benchmark for both PlatON and EOS under the same testing environment.
Server : AWS c5d.4xlarge server
CPU : Inter (R) Xeon (R) 8124M Hexadeca-core 3.0GHz
Memory: 32G
Disk Storage : 300GSSD
Test Load
We aimed at testing the performance of the whole blockchain system, here we selected several typical applications as the Macro-benchmark: native token transfer , smart contract transfer , and key-value storage contract .
Test Results
Native token transfer
The average TPS of PlatON transfer is 9604 transactions/sec, and the maximum value is 14755 transactions/sec, while the average TPS of EOS transfer is 3049 transactions/sec, and the maximum value is 5393 transactions/sec.
-
When the CPU utilization is not that high, there is no doubt that the performance of PlatON is better than the one of EOS. The parallel transaction mechanism of PlatON improves its performance greatly;
-
The memory and network resource consumption of PlatON is far lower than EOS. Seen from the consumption pics, the storage utilization of both PlatON and EOS are almost the same. However, when talking about the average utilization of transaction, the storage utilization of PlatON is far lower than EOS.
-
As for the transaction to finality (TTF), both PlatON and EOS are stable. Only if the ensuing 2 sub-block are confirmed through votes, each block of PlatON will be confirmed finally. According to the consensus mechanism of EOS, there will be 21 nodes to product block one by one in each consensus interval, and each node will product 12 blocks continuously, and there will be at least 2 rounds of consensus intervals before each block gets confirmed finally. Each interval will collect the indirect confirmations of at least 15 nodes, that is to say, each block has to wait for at least 15x12x2=360 blocks before it is confirmed finally. According to the average block production time of EOS, namely, 0.5 sec, the final confirmation time is 180 secs. Obviously, it is longer than PlatON.
Smart Contract Transfer
The performance of PlatON-EVM is higher than EOS slightly, while the one of PlatON-WASM is lower slightly. The memory network resource consumption of PlatON is far less than the one of EOS, while the storage consumption of them are almost the same.
As for the confirmation time, both PlatON and EOS are stable. The final confirmation of EOS block needs to wait for 360 blocks, which means, the transaction to finality of EOS is longer than PlatON.
Key-value Contract Transfer
In the performance test of key-value contract transfer, the average TPS that PlatON-EVM called keyvalue contract is 5237 transactions/sec, and the maximum value is 6304 transactions/sec. The average TPS that PlatON-WASM called keyvalue contract is 3287 transactions/sec, and the maximum value id 3797 transactions/sec.
The average TPS that EOS called keyvalue is 2451 transactions/sec, and the maxium value is 6268 transactions/sec.
-
It is obvious that the performance of PlatON is better than EOS, and the CPU is higher, indicating that PlatON’s multi-core CPU utilization is higher.
-
The memory and network resource consumption of PlatON are far lower than the ones of EOS, and the memory consumption of them are almost the same.
-
As for the TTF, both PlatON and EOS are stable. The final confirmation of EOS needs to wait for 360 blocks, which is longer than PlatON.
-
The consensus node memory kept increasing during the test, resulted in the memory system limit get killed by OS. Seen from the pic above, we can see the broken curve.
Conclusion
-
Thanks to the DAG’s parallel transaction mechanism, the native token transfer TPS of PlatON has the obvious advantages in performance.
-
There is no big different on performance when executing smart contract transactions, and PlatON’s WASM contract performance can be improved.
-
Due to the optimization on consensus mechanism, PlatON’s TTP is far lower than the one of EOS.
-
PlatON works better on utilizing CPU, and the consumption on memory and network is far lower than the ones of EOS. During the test, the consumption of then are almost the same, however, the consumption on average transaction of PlatON is lower than the one of EOS.
-
During the test, the memory utilization of EOS kept increasing, and get called by system kill when the memory utilization reached its limit.