It's not just programs that crash! Photo by me, a few weeks ago in Vancouver. At least the bus is sorry?
Cheetah
Fun fact: Did you know Cheetah is approaching $300/day in operational costs? Fortunately, this is currently not a limit (due to witness pay and SBD price being high) but it's something to keep in mind.
During the Steem witness forum hosted by @aggroed, I briefly talked about current issues facing Cheetah. I'll outline the two biggest ones here.
The first issue is a bug I discovered that is an artifact of the current blockchain state design. I discovered this bug as when Cheetah was voting extremely often, votes would fail despite my calculation putting the vote weight at much higher than the dust limit.
After digging into the problem to find out why, and some discussion with @vandeberg, we discovered that there's currently a minimum voting power consumed that is used to make a vote due to precision. This is a bit of a bug, as the limit is artificial -- regardless of your SP, there is a maximum number of votes you can issue per day. This is counter-intuitive to the idea that your influence is based upon your SP, not the number of accounts you own. A GitHub issue was opened here, and you can see some of the more technical details of the issue.
The second issue I am running into is the 20 second comment limit. There are 86,400 seconds per day, which means you can comment only 4,320 times per day maximum. Guess what? Cheetah has hit that limit on popular days!
But the bigger issue is that transactions are somewhat "bursty". For example, during prime time USA there are a lot more transactions than at midnight. This imbalance allows her to "catch up" during the slow times, but she becomes behind during prime time. This can cause problems with external services that watch for a Cheetah downvote -- for example, a voting bot may refuse to vote on a post that was downvoted by Cheetah -- however, the downvote will not arrive on the post quick enough for the service to see it, and the bad actor can circumvent the services rule.
There is some support for removing the 20 second comment limit: a GitHub issue has been opened here. I am completely in favour, as it also goes against the idea that your influence should be based upon your SP, not the number of accounts you own.
Now, if you're a developer the obvious solution that you will tell me to use is to simply use multiple accounts. I know. But this isn't the point: your actions on the blockchain should be limited by your SP (e.g. by the bandwidth assigned to you)! I am repeating this over and over because it is fundamental to Steem's design. In the same way splitting your SP across two different accounts doesn't give you any advantage to voting, neither should commenting or voting from two different accounts. The goal is to make a Sybil attack ineffective. At the moment however, these two issues have yet to be addressed. If the popularity of Steem continues to rise, I will indeed be making multiple Cheetah accounts to circumvent these artificial limits, sadly...
I will be making an announcement if the move to multiple Cheetah accounts happens, so that service operators know what to expect if they are looking for specific Cheetah actions.
Steemcleaners
For the most part recently I have been working in an administrative and development role for Steemcleaners -- we now have several full time members working hard in the daily operational aspects.
In terms of development, I have built several bots and tools over the past few months in conjunction with @patrice, which you have probably already heard of if you follow the Steemcleaners posts. We have new tools to mark spammers, tools to precisely remove rewards without over-voting, and more!
We have also recently hired a Web+DB Developer to redesign our website, steemcleaners.org. With the SBD price being beneficial to us, we have had the funding to order some cool stuff, so I am very much looking forward to this. We have in the pipeline a much more automated procedure with user roles: we will be able to incorporate more helpers into our ecosystem, and give them controlled access to certain operations. For example, helpers can have assigned roles to be on the lookout for identity theft, and others to look for plagiarism. After issuing their finds to our website, it can be validated by a senior member and some actions can be taken automatically (such as nullifying rewards on the post).
This will greatly increase the scalability of Steemcleaners, and we will be on-boarding a lot more helpers. If you're interested in becoming a helper in the future (remember, it pays in steem/SBD!) feel free to reach out to myself or @patrice. We hope to see more curation groups working directly with us.
SBD Peg discussion
There has been a lot of discussion on this lately, so I'll take a moment to chime in. A TL;DR of the SBD peg issue is: Although SBD has a price "floor", being supported by conversion of SBD->Steem, SBD currently does not have a price "ceiling". This has allowed this run up of SBD valuation to go unchecked by active forces to push it back to the intended valuation. There are a lot of good arguments for adding a price "ceiling" (for example, by allowing Steem->SBD conversions), but there are also good arguments for allowing the SBD price to be free market determined (for example, the huge payouts we are seeing is incentivizing people to join).
I haven't weighed in on a lot of the discussion on the SBD pegging issues. Honestly it is mostly because I am on the fence. I have an interest to wanting the SBD price high (as it is leading to excellent funding for Steemcleaners, for example), but I also have an interest in having a stable token maintained by a peg, mostly for adoption an merchant reasons. I keep coming back to the argument that if we wanted a token to speculate value on, it should be steem: if we have two speculative tokens it's just more confusing for users. If SBD is left as an unpegged asset like Steem, we should just move to only using one token for simplicity. So I think if there is a movement to include enforcement of the SBD peg (e.g. with steem->SBD conversions) I would be in favour.
Scaling Steem Resources
Full RPC nodes hit the 256 GB barrier recently, and apparently a few had issues. Although I'm a little concerned in the short term for the health of having public RPC nodes, I am not overly concerned in the long run simply due to how many optimizations can be made to the Steem code. I work with HPC (High Performance Computing) systems, and the size of the Steem blockchain and state is still incredibly tiny compared to what I am used to, especially considering how un-optimized it is. After my PhD I might find myself working on scalability in the Steem and blockchain world, as it's pretty clear to me that basically no one here has any idea what they're doing when it comes to performance or scalability. (While it might sound pompous of me to say that... well, it's my field of PhD-candidate-level expertise.)
Anyways, a few times I have seen the idea that witnesses should offer both a seed node and a full RPC node for public usage. While I think this is a great goal (to have a large amount of RPC nodes available to aid developers), I don't have one made public at the moment. I have actually been running my own RPC node since I started as a witness, but it is private: the main reason for this is because my own programs already absolutely hammer it -- especially Cheetah. Opening this one to the public would open DoS attack vectors on Cheetah (which I do not want), so if I were to offer an RPC node publically I would have to launch a second one.
I would be curious to see if developers are interested in having another RPC node available, or if they believe there are currently enough to service their needs. So please let me know in the comments if that is something that would interest you, and let me know what you think of the current state of RPC nodes!