Repository
https://github.com/utopian-io/utopian-bot
Components
- Utopian bot post sorting criteria for upvoting
This is a collection of ideas to improve the current Utopian bot sorting criteria for upvoting.
Note: some incorrect part is removed and revised based on @espoem and @amosbastian's comments. Thanks a lot! See the comments for that. In addition, suggestions 3, 4, and 5 are added.
If something is wrong, please see a big picture. For specific implementation, I'm not saying this is the best at all. It's just an example. I'm still a newbie in Utopian, so hope senior members :) will adopt it wisely. I'm already more than happy to see people think some change or discussions may be needed. Thanks!
Proposal Description
Background
Due to the voting value decrease, Utopian increased the incentives, as in We're Increasing the Utopian Incentives, which totally makes sense. (Unfortunately, voting value has been decreasing further. Now 1000 SP only gets about $0.02.)
One problem, however, is that there are some good posts that cannot be voted at all due to priorities. People understand that voting skip can happen, but it's not a good thing especially when it is a little bit "unfair." (Of course the definition of fairness is subjective, and that's why I used "unfair.") So I'd like to propose a solution that mitigate this no-voting problem and improve "fairness."
Current voting priority system
Currently, posts to be voted in the next round is determined this way.
sort_batch_contributionshttps://github.com/utopian-io/utopian-bot/blob/master/utopian_bot/upvote_bot.py#L110
- At each round, a fixed voting power target (around 18%, defined as
VP_TOTALin the code) is to be used. - Select the contribution posts and moderator comments until the expected voting power needed is up to
VP_TOTALas follows:- Moderator comments have the highest priorities (should be, since it's pay for their work, but see the suggestion 3).
- Contribution posts are sorted by the evaluation score.
- If there are multiple posts that have the same score, then the older has a higher priority. But this isn't usually a binding factor. I mean the last post to be upvoted and the first post to to try in the next round again usually do not have the same score.
Caveat: the bot is also trying to be "fairer" over the category, so the following may occur.
Posts with lower scores (48,48,37) are included instead of higher scores (71,70,...) to use VP fairer over the categories.
Well, some people may not like this, but actually I personally like this idea of "even" distribution. Every category is important.
But the problem is that fairly good postings (especially in a higher weighting category, e.g., dev) may suffer from a disadvantage. This happened and is happening. (Fortunately, it never happened to me so far, so to be honest, I didn't even know this problem exists, but I found this when I was trying to figure out another problem (no steem-ua voting for utopian posts, which is now resolved. But now one dev post of mine is also in danger. But this isn't the only reason why I'm writing this proposal :)
External contributor disadvantages
In particular, this may discourage external dev contributors. By "external," I meant a person who doesn't get paid as a team member. For instance, in the Busy (busy.org) case,
- internal: any busy team members (who is getting paid by Busy CEO)
- external: any other people except for busy team members.
Note that if no pay is involved, internal is who has a merge right. For instance, holger80 for beem library.
For the dev category, there is a rule that the merged PR should be within 14 days. But due to irregular and infrequent merges for PRs, it's unlikely for external contributors to make a big contribution within that window, which leads to a mediocre score.
Suggestion 1: put some weight on aging
I'm not saying that older should always get higher priority. But, at least there should be some boost when they were skipped in the previous round.
This is also one of main features of decentralization :), e.g., In the Bitcoin network, while a transaction with a higher fee gets a higher priority, to prevent an infinite waiting, older transactions accumulate some priority as time goes by.
Suggestion 2: last resort - small voting instead of nothing
Posts that are about to expire (i.e., will pass the 6.5 days in the next round) should at least receive some small courtesy votings rather than no voting.
Suggestion3: Give priorities to contributions until when reviews are expiring
I sincerely believe that moderator's review should have the highest priorities. But influx of the contributions tend to fluctuate. That is, there could be contributions that could have been voted if votings on reviews are postponed a bit. See details in Example section.
Suggestion 4: separate reserve or some bonus point for external contributors
Especially for dev, while POs can aggregate quite decent amount to publish the post, it's almost impossible for external contributors due to irregular and infrequent merges by POs. It'd be good if there's some reserve or bonus point for external contributors. Again, I'm not saying a huge bonus point. Yes everything is subjective and maybe a matter of weight. But let's say there is some courtesy small voting but it's still limited, then it might be good to prioritize external contributors.
Suggestion 5: leave a comment for posts with no voting or lower voting than usual due to VP
Currently, there is no comment when voting is skipped at all. It's also good to leave a comment when this situation occurs.
Examples
Suggestion 1: put some weight on aging
While keeping the current score for the voting weight itself, consider the age of the post in the post selection stage. For instance,
voting priority score := f(score, age) = (1 + alpha)^age * scorewhere
age is the number of rounds skipped: 0 if it's just in the que, 1 if it was skipped once, ... (Instead of number of rounds, just the age of the post can also be used.)
For instance, if alpha is 10%, then the original score with 60 becomes, 60, 66, 72.6, 79.86, ... . But this is only for the priority adjustment not for the actual voting weight.
Suggestion 2: last resort - small voting instead of nothing
Reserve some small VP, and spend it first for expiring posts with a voting weight proportional to their original scores. For instance, if the reserve is 20 and there are three expiring posts that need 20, 10, and 10, then each gets 10, 5, and 5, respectively.
Suggestion3: Give priorities to contributions until when reviews are expiring
- Until moderator's reviews are expiring (i.e., getting older than 6.5 days if it isn't voted in the upcoming round), contributions have higher priorities.
- But when reviews are expiring, they have the highest priorities.
- If expiring reviews cannot be voted in one round, that situation can be avoided in several ways: Further use VP, or vote earlier based on the forecast of expiring.
Suggestion 4: separate reserve or some bonus point for external contributors
This can be done with giving more score in the beginning or using some separate reserve for VP or adjustment of voting priorities.
Alternatively, increase 14 day window for merged PR, so external contributors can gather more decent commits with enough time, this might be much easier and realistic idea.
Suggestion 5: leave a comment for posts with no voting or lower voting than usual due to VP
Depending on the situation,
Benefits
- "Fairer" voting
I'm okay with this in general, but some may feel their posts are discriminated. But again, there is no perfect rule. But aging should better to be a factor. - Encouraging external contributors
I believe the open-source projects should be really open, and they can be greatly improved when external contributors join. Considering the reality that PO can't respond to PRs of external contributors that often, external contributions should still be well appreciated. - Prevent no voting
No voting isn't good. Currently, it doesn't even leave a comment, so contributors do not even know the reason. I believe that contributors do not contribute just for votings, so they will happily accept those adjusted small votings, and small votings are still better than noting :)