3月30日上午9点53分关于PlatScan显示块高停止增长问题详细过程原因和解决方案,供大家审阅,多提建议:)

关于PlatScan显示块高停止增长的问题进展同步:

问题概述:

3月30日上午9点53分左右,PlatScan显示块高停止增长,测试网出块正常,已确认是PlatScan agent不工作导致,具体原因还在排查;

问题影响:

1、测试网底层出块正常,通过命令行发送各类型交易正常

2、PlatScan显示块高停止增长,ATON交易无法确认,相关显示信息异常;

1 个赞

我们会随时在社区和节点群内同步相关问题进展,给大家带来的影响,我们深表歉意!

2020年3月30日早上9点57分发现PlatScan块高停止更新,只显示到2880252块高。经过排查PlatON链出块正常,开始分析对应的PlatScan问题。

一、问题分析

1、查看PlatScan方面的日志,发现存在异常情况,异常信息如下:

Cause: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry ‘0xfa24468c3cd252da1ba8872665bfd8250db1017173ccb4fb8c42296bd2af51’ for key ‘PRIMARY’

; Duplicate entry ‘0xfa24468c3cd252da1ba8872665bfd8250db1017173ccb4fb8c42296bd2af51’ for key ‘PRIMARY’; nested exception is java.sql.SQLIntegrityConstraintViolationException: Duplicate entry ‘0xfa24468c3cd252da1ba8872665bfd8250db1017173ccb4fb8c42296bd2af51’ for key ‘PRIMARY’
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:242) ~[spring-jdbc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) ~[spring-jdbc-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]
at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:73) ~[mybatis-spring-2.0.0.jar!/:2.0.0]
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:446) ~[mybatis-spring-2.0.0.jar!/:2.0.0]
at com.sun.proxy.$Proxy63.update(Unknown Source) ~[na:na]
at org.mybatis.spring.SqlSessionTemplate.update(SqlSessionTemplate.java:294) ~[mybatis-spring-2.0.0.jar!/:2.0.0]
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:64) ~[mybatis-3.5.0.jar!/:3.5.0]
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:58) ~[mybatis-3.5.0.jar!/:3.5.0]
at com.sun.proxy.$Proxy64.create(Unknown Source) ~[na:na]
at sun.reflect.GeneratedMethodAccessor771.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_221]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_221]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.6.RELEASE.jar!/:5.1.6.RELEASE]

2、初步分析是节点信息在数据库中已经存在,后续插入的时候出现了主键冲突的情况。

3、继续分析数据库节点数据,发现并没有存在’0xfa24468c3cd252da1ba8872665bfd8250db1017173ccb4fb8c42296bd2af51’对应nodeId的节点信息。

4、开始查询停止区块的下一个区块(2880253)对应的交易回执,发现有多笔质押节点的交易信息。
{
blockHash: “0x3aed37ad35f255c2f1351b76303353faeea4423847e5b95e7ef675e87fd0b1a9”,
blockNumber: 2880253,
from: “0x726cf86f27db2f9428649f9525d25f2580f1bb79”,
gas: 86016,
gasPrice: 10000000000,
hash: “0xfb448fca1008c30d3c8277b00fb634d0c0eeb7edaf8c1298ce10e767d097958b”,
input: “0xf9018f838203e8818095946bd07c44a906c512012188ecdef768ae92759a03b842b840fa24468c3cd252da1ba8872665bfd8250db1017173ccb4fb8c42296bd2af5114bb4d575dad05ee2228e755a24665131c4b8ce0d8512c90c49bda82a3468624e38c8b4d794b65794261736549648483536f539998687474703a2f2f7777772e6d79636f6d70616e792e636f6d8483536f538b8ad3c21bcecceda10000006483820a01b843b8410dcc720827bba9dc24ea01dff48b2773bf5a37d35ed7445e9fe8e377a08a149b36cefe78ec61563567d862f6acb8a6aead995ad33fa6482d0a87b6be329da18b00b862b8606c63872a0495975998fbed9b93c7ddf0e987b30d8f837c92d675a665ec8f9b745a9e2c022b1713e761b7b06fba199106c2d2be3358a8ee4bc60e33de47628f959982cc2c7319cc50ce56bb2d39b0f8e94016b658be9dcf2ecfead790f1a5b18cb842b8404089009b51bd41cba82fbf67364c0b1ed81d4a56c47971bfd8c418f34401ed1d77c3b3784145fc8906353f43526c298d30062e1fe56f0802b29ec29a228ad82f”,
nonce: 1642,
r: “0xa9de24b57bf8234b48ac1b702ff008bca89c9b6f3506200213c5a5e5f46be88b”,
s: “0x55cc73306b1076871fc21fb9d81917ff329818eea717896463270eceacc8812a”,
to: “0x1000000000000000000000000000000000000002”,
transactionIndex: 35,
v: “0xed”,
value: 0
}, {
blockHash: “0x3aed37ad35f255c2f1351b76303353faeea4423847e5b95e7ef675e87fd0b1a9”,
blockNumber: 2880253,
from: “0x726cf86f27db2f9428649f9525d25f2580f1bb79”,
gas: 86016,
gasPrice: 10000000000,
hash: “0x182a63d028c321d4dc1da238046c3d8d178f855e5d36c3a020219c755536be77”,
input: “0xf9018f838203e8818095946bd07c44a906c512012188ecdef768ae92759a03b842b840fa24468c3cd252da1ba8872665bfd8250db1017173ccb4fb8c42296bd2af5114bb4d575dad05ee2228e755a24665131c4b8ce0d8512c90c49bda82a3468624e38c8b4d794b65794261736549648483536f539998687474703a2f2f7777772e6d79636f6d70616e792e636f6d8483536f538b8ad3c21bcecceda10000006483820a01b843b8410dcc720827bba9dc24ea01dff48b2773bf5a37d35ed7445e9fe8e377a08a149b36cefe78ec61563567d862f6acb8a6aead995ad33fa6482d0a87b6be329da18b00b862b8606c63872a0495975998fbed9b93c7ddf0e987b30d8f837c92d675a665ec8f9b745a9e2c022b1713e761b7b06fba199106c2d2be3358a8ee4bc60e33de47628f959982cc2c7319cc50ce56bb2d39b0f8e94016b658be9dcf2ecfead790f1a5b18cb842b8404089009b51bd41cba82fbf67364c0b1ed81d4a56c47971bfd8c418f34401ed1d77c3b3784145fc8906353f43526c298d30062e1fe56f0802b29ec29a228ad82f”,
nonce: 1644,
r: “0x294f0fcbf0aa824c4c6b7c55e6281b0e9be8abf38ca72f911865513b71a0b3c2”,
s: “0x27413517420fb53311d57b39f5376dda238f10a4c4bd485774e25e13478d8142”,
to: “0x1000000000000000000000000000000000000002”,
transactionIndex: 37,
v: “0xed”,
value: 0
},{
blockHash: “0x3aed37ad35f255c2f1351b76303353faeea4423847e5b95e7ef675e87fd0b1a9”,
blockNumber: 2880253,
from: “0x726cf86f27db2f9428649f9525d25f2580f1bb79”,
gas: 86016,
gasPrice: 10000000000,
hash: “0x88c4eb1277b7b7caa09c7faa891a47eb94e178feccc5399a86c0499550bd93ed”,
input: “0xf9018f838203e8818095946bd07c44a906c512012188ecdef768ae92759a03b842b840fa24468c3cd252da1ba8872665bfd8250db1017173ccb4fb8c42296bd2af5114bb4d575dad05ee2228e755a24665131c4b8ce0d8512c90c49bda82a3468624e38c8b4d794b65794261736549648483536f539998687474703a2f2f7777772e6d79636f6d70616e792e636f6d8483536f538b8ad3c21bcecceda10000006483820a01b843b8410dcc720827bba9dc24ea01dff48b2773bf5a37d35ed7445e9fe8e377a08a149b36cefe78ec61563567d862f6acb8a6aead995ad33fa6482d0a87b6be329da18b00b862b8606c63872a0495975998fbed9b93c7ddf0e987b30d8f837c92d675a665ec8f9b745a9e2c022b1713e761b7b06fba199106c2d2be3358a8ee4bc60e33de47628f959982cc2c7319cc50ce56bb2d39b0f8e94016b658be9dcf2ecfead790f1a5b18cb842b8404089009b51bd41cba82fbf67364c0b1ed81d4a56c47971bfd8c418f34401ed1d77c3b3784145fc8906353f43526c298d30062e1fe56f0802b29ec29a228ad82f”,
nonce: 1646,
r: “0x9f2745050423837cd6b3067aa9d6d567afc4b3dc03ce2eae7c981602f8c672ba”,
s: “0x794d2d1dccd7e4d0ab05e69f6243e432f1195ed87bcf208fa70d8957dc00b9c4”,
to: “0x1000000000000000000000000000000000000002”,
transactionIndex: 39,
v: “0xed”,
value: 0
}, {
blockHash: “0x3aed37ad35f255c2f1351b76303353faeea4423847e5b95e7ef675e87fd0b1a9”,
blockNumber: 2880253,
from: “0x726cf86f27db2f9428649f9525d25f2580f1bb79”,
gas: 86016,
gasPrice: 10000000000,
hash: “0x34bb08ce2484644b953bde326c0f8a93fdda11e875f0611e0614b4032e6519e1”,
input: “0xf9018f838203e8818095946bd07c44a906c512012188ecdef768ae92759a03b842b840fa24468c3cd252da1ba8872665bfd8250db1017173ccb4fb8c42296bd2af5114bb4d575dad05ee2228e755a24665131c4b8ce0d8512c90c49bda82a3468624e38c8b4d794b65794261736549648483536f539998687474703a2f2f7777772e6d79636f6d70616e792e636f6d8483536f538b8ad3c21bcecceda10000006483820a01b843b8410dcc720827bba9dc24ea01dff48b2773bf5a37d35ed7445e9fe8e377a08a149b36cefe78ec61563567d862f6acb8a6aead995ad33fa6482d0a87b6be329da18b00b862b8606c63872a0495975998fbed9b93c7ddf0e987b30d8f837c92d675a665ec8f9b745a9e2c022b1713e761b7b06fba199106c2d2be3358a8ee4bc60e33de47628f959982cc2c7319cc50ce56bb2d39b0f8e94016b658be9dcf2ecfead790f1a5b18cb842b8404089009b51bd41cba82fbf67364c0b1ed81d4a56c47971bfd8c418f34401ed1d77c3b3784145fc8906353f43526c298d30062e1fe56f0802b29ec29a228ad82f”,
nonce: 1648,
r: “0xcd970b194f6ae732ac61a366ebae9f7b8e7d62308d4b7bbc162484e88daebd6c”,
s: “0x5ede80c94fe051a7a2d70c734097ea5b741fef91d488b6c87d3af5be3c5d0ec0”,
to: “0x1000000000000000000000000000000000000002”,
transactionIndex: 41,
v: “0xee”,
value: 0
}

5、对质押信息解析input和对应交易回执,发现是同一个节点相同的创建请求并且所有质押交易都是成功的。

6、对代码进行分析,确认原有逻辑(解析区块->解析区块交易(按照交易打包顺序进行串行解析并执行逻辑->执行对应的交易逻辑->提交入库->统一进行commit)是由同一个事务进行管控的。当一个区块中出现多笔质押交易时候,重复的insert相同的nodeId数据会造成主键冲突的情况。

二、问题处理

由上分析,出现问题的主要原因在于同一个事务内多个相同主键的数据进行批量入库会导致主键冲突而失败。

最终解决方案如下:
将insert操作改为replace,保留质押时最后的数据。其中整个区块的处理逻辑是由一整个事务把控的,该事务保证了数据的顺序性和一致性。顺序性保证了最终更新的数据是最后一笔的数据也就是有效的数据。

2 个赞

原来是这个原因 :joy:
事先可能没有想到居然有节点在同一个区块发起多次Staking交易
不过如果不是超高压测试,正常情况下也不会有这种事情发生

是的,所以这也是我们发起超高压测试的目的,发现和解决问题,持续提升网络稳定性 :blush:

好的。1w个账户准备好了,请问什么时候开始第三次压测 :rofl: :rofl: :rofl: :crazy_face: :crazy_face: :crazy_face:

一个区块里可以同时出现一个地址的多笔交易吗?

兄弟,如果到时候让你KYC你咋办? :rofl:

这个问题难道不也是你的问题?

以论坛公告为准哈

哈哈哈,估计底层团队的小伙伴顶着黑眼圈瑟瑟发抖
不过测出问题是好事哈~希望攻击来得更猛烈些