I’m sure everyone is well aware of the large hashrate swings resulting in certain lanes dominating block production periodically or others getting drowned out entirely. This is causing block times to temporarily exceed the 2min target - often approaching 3min or more and sometimes resulting in chain reorgs many blocks deep.
I was thinking about proposing a modified DGW/MultiShield implementation, but I saw @blackwolfsa 's issue here about adding a consecutive block penalty to Tari’s current LWMA implementation. This seems like a much simpler/more elegant solution to me, and I’m of the belief that smaller architectural changes are typically better when it comes to consensus.
What are everyone’s thoughts on implementing this or something else like Digibyte’s MultiShield?
@blackwolfsa is there any reason that you decided not to go ahead with this proposal?
Its still in proposal stage. Changing the PoW is something that needs to be approached with caution and enough time.
But the lanes should not effect each other. But target difficulty changes is inherant to the design of pow and an a side effect of mining. Miners will swap to and from a chain if its more or less profitable to do so. That will just impact that chain, but should not impact the other chains. See my explanation on the other post about pow.
My proposal is about reorgs which we dont see atm.
Sorry, I didn’t mean to imply that hashrate on one lane is directly impacting block times on another - I am saying that the slow adjustment of the LWMA paired with the violent hashrate swings (in both directions) are resulting in overall block times exceeding the 2min target.
This is part of the problem, right now the ability to mine many blocks in quick succession is resulting in pools gaming the network to exploit small windows profitability on a particular lane, after which they usually switch off/abandon the network. You can see others observing this behavior in the hashrate concentration thread. This is causing a “hangover” effect with these hyper-fast blocks in the LWMA difficulty calculation window. Your solution somewhat remedies this by preventing these hyper-fast blocks in the first place.
I understand you initially proposed this to solve reorgs, but I believe it would indirectly solve some of this blocktime variance issue as well.
You also mentioned “tuning some of the parameters to make it adjust quicker” in your issue, which is the most appealing part of your proposal. If we could just shrink the window from 90 to 45, for example, that would be a much simpler fix compared to implementing something entirely new like DGW/MultiShield or an EMA. Maybe I misunderstood and you only meant tuning the target_time parameter and not block_window?
Also, we do see reorgs every day as far as I know. Just glancing at the network as I write this I can see a 3 block reorg from 27893-27895 with 4 blocks mined in rapid succession ~1min. It’s not a massive reorg, but it is an issue - and if bridge limits are removed like many in this community are asking for this could become significant unless confirmation requirements are raised (which has UX tradeoffs).
Bridge safety limits are an entirely different conversation, and I am of the opinion to keep it on the safe side. Most crypto hacks and attacks that steal money are targeted at a bridge.
But the LMWA jumps relatively quickly. But if you make the block window too small, it jumps too erratically. Having block times that are not exactly 2 mins are fine. A good example of this is Bitcoin, it only adjusts the difficulty every 2 weeks. Now I know they have much, much higher hash rates, so it’s difficult for it to swing.
Block target time is only there for an estimate for the pow and so you give the network time to sync new blocks. As long as the avg block time is roughly 2 mins, its fine.
I would love to be able to mine and hodl XTM again. But I cant because of the hashrate swings. Even though I dont sell the XTM I mine, I cant in good conscience mine it as a charity, which is what it is right now. A new difficulty targeting algo would definitely help build that loyal miner base I think we want.
I think its good to be honest about this, PoW swings are going to be a thing for smaller pow chains like Tari, nothing we do can stop this. We can try and optimise how its handled, but it will happen, its part of the economics of PoW.
A clever Rx Pool will be able to mitigate the algo changes very easily as the hardware required to mine RxT vs RxM is exactly the same. Sha3 vs C29 vs Rx{T/M} is different, in that it is ASIC/FPGA vs GPU vs CPU.
While I’d love to see the wild RxT and C29 swings gone, I’m not sure an incremental penalty, exponential or not, is the answer - wouldn’t that just make gaming the swings more predictable?
It’s easy to redirect hashpower, so now instead of booming for 4-5 hours, while the current algorithm slowly catches up, one could push for 2-3-4 quick blocks, and then redirect elsewhere, since no-one is going to find more blocks in that lane anytime soon. And, depending on how the penalty peters out, it could also be quite predictable when a given POW lane is ripe for the next push.
I could very well be missing something but couldn’t a simple penalty on consecutive blocks in a lane make for even wilder swings and maybe even fewer blocks found by the regular, non-switching miners?
the “penalty” only applies till a nother block is mined by any other alogrthim. All you are doing is making it harder to mine consecutive blocks. Its not going to increase the mining difficulty, any more than the current lmwa does.
Per our discussion from yesterday’s Tari Show.
I am going to push to code for this, but I added debug code to the node to print out the stats from the difficulty calculations.
But on the last 1000 blocks we have the following:
Avg block time: 124
RandomXMonero: 17,041 blocks
Sha3: 17,031 blocks
Cuckaroo: 16,257 blocks
RandomXTari: 16,186 blocks
These stats show that the difficulty adjustment is doing its job
I am digging more into the seeing if we can adjust quicker and the effects of that
I can’t post my previous simulations here either. I’m working on a workaround.
The sweet spot for the block window seems to be around 45 - perhaps even a little bit lower. But not <=30. If we’re only going to do one update in the near-term, the block window should certainly be decreased from the current 90.
Like you said in the show, this isn’t going to prevent the issue entirely, but it will mitigate it significantly, and updating LWMA is a much simpler change compared to multishield or implementing DGW for each separate lane. That is the primary appeal here - it’s minor enough to be a “quick” and non-contentious hardfork while still providing some necessary relief.
In terms of DGW its a simple moving avg, we use a Linear moving avg. I would argue the LWMA is better than a moving avg since the LWMA weights the most recent blocks more than a simple avg. This gives a better response in terms of Diffculty up or down.
Multishield uses a moving average with a geometric mean.
They have a Moving avg for each algo, and then they calculate the geometric mean between all.
Tari uses the LWMA and we skip the nth sqrt of the geometric mean.
So for example :
s_pow, c_pow, rm_pow, rt_pow being the pow for each algo
Multishield total Pow would be: 4^sqrt( (s_pow )(c_pow)(rm_pow)(rt_pow) )
Tari total Pow would be: (s_pow )(c_pow)(rm_pow)(rt_pow)
So we just skip the sqrt step as this is a floating operation which means the value is different on each cpu type, as well as not being a precice number. The downside is where the Multishield value would fit in a u128, we need a u512 to store this, so its a larger number.
Then another difference is the Multishield feeds in the current timestamp difference into the DAA, where we only feed in the actual mined blocks, so our algo has a 1 block delay. Making this real-time adds a whole range of complications which I dont think is worth it.
So where does this come to, There is big differance between Tari’s DAA and Multishield/DGW. Changing it wont make any differance
I agree with you. I want to put this entire issue to bed. I dislike that I feel like I’m repeating myself over and over and potentially coming across as confrontational/argumentative when I’m just trying to fix a very clear issue with the network. But I feel that I have to keep doing it because I see obvious potential for Tari to improve here and I’m sick of watching us bleed some of our most valuable community members/supporters because miners appear to be treated as an afterthought.
Calling DGW “a simple moving average” is a bit of an oversimplification and slightly inaccurate. DGW was specifically created to improve upon SMAs, and has a dynamically adjusted block window, time-based dampening, and real-time difficulty adjustments.
LWMA with a consecutive block penalty and reduced block window should perform similarly to DGW on the way up when handling these hashrate spikes, but will be much slower to adjust on the way down since difficulty isn’t retargeted on LWMA until after the most recent block is mined. We don’t have the “gravity well” signature to DGW that brings difficulty down immediately after a large miner leaves the network.
This real-time adjustment is the key feature both of these algorithms share and is likely what we would need to fix the blocktime variance entirely.
There are pros and cons to this as well, of course, with the largest con being complexity. You are absolutely correct there - they would both be massive changes going beyond just the core software. The pro is, block times would almost always be right on target. I’m not suggesting we attempt this right now, or ideally ever. My hope is that reducing the block window paired with this consecutive block penalty will be enough, but we’ve both acknowledged it may only mitigate the issue and not fix it entirely.
FWIW, Meowcoin has a similar PoW implementation to Tari with multiple lanes + merge-mining and I’ve just learned they use a block window of 45 for their LWMA.
Yeah, I am not trying to come across as confrontational/argumentative. Everything we do here has some pros and cons, and its up to us to find the best route. I hope to be able to publish some results soon of a few simulations I am running, and we can compare if its worth the tradeoffs.
We are going to fix it. It seems like the timeline has even been moved up based on feedback here and in the Tari Show.
Family has been staying with me for the holiday weekend so I’m running a bit behind, but I’d still like to post my simulations too. All I have of the originals are screenshots so I’m still in the process of redoing these in a more auditable format.
Hi y’all! I know some of you were attempting to upload attachments as part of the data you were sharing. This should work now-- for new attachments.
If you already uploaded an attachment before, it will think it already has it/the hash for it. Rather than trying to spelunk into the DB too much, I’ll ask you to just modify whatever file you were going to upload slightly so it has a different hash before trying to upload that one again.