Seeking Alpha

David Jackson's  Instablog

David Jackson
Send Message
I'm founder and CEO of Seeking Alpha. I started Seeking Alpha after working for five years as a technology research analyst for Morgan Stanley in New York. Seeking Alpha is now the dominant crowdsourced equity research platform, with over seven thousand contributors, seven million monthly... More
My company:
Seeking Alpha
My blog:
A Founder's Notebook
  • Rethinking how Seeking Alpha calculates Top Commenters 33 comments
    Sep 18, 2011 9:18 AM
    We've been aware for a while that our Top Commenters leaderboard and badges are unsatisfactory. 

    Until now, we were calculating Top Commenters by a simple tally of the number of "likes" their comments received. The problem with that is that there are many groups on Seeking Alpha who repeatedly "like" their own members' comments, so the number of likes might not be an accurate recognition of comment quality.

    So we've decided to re-evaluate the algorithm we use to calculate Top Commenters. I suspect we'll still base the rankings on "likes", but we can be far more sophisticated, for example by taking account of the "authority" of the person who votes with a "like". (Think aboout Google's calculations of Page Rank based on links and the authority of the linking site.)

    While we rethink this, we're removing the Top Commenters badges and the link to the Top Commenters leaderboard in the footer of the site. But we're still storing all the data about "likes", so that when we launch the new algorithm it will take account of up to date data.

    If you have any thoughts about how we should calculate Top Commenters, we'd love to hear your ideas - please leave a comment below.
    Themes: comment ratings
Back To David Jackson's Instablog HomePage »

Instablogs are blogs which are instantly set up and networked within the Seeking Alpha community. Instablog posts are not selected, edited or screened by Seeking Alpha editors, in contrast to contributors' articles.

Comments (33)
Track new comments
  • I think providing a "like-value" weight based on the length of time someone has been commenting on SA and the number of comments or likes received may would be a good idea. It would make it harder for people who create 10 aliases and use them to give themselves thumbs up. But there should be an upper limit to how much weight a like can have as well. If someone has been around for 5 years and has made 15,000 comments, that person's like shouldn't count as 100 likes obviously. Just use a logarithmic function that asymptotes at 4 likes but requires a person to have at least 100 comments, 200 likes, and being a commentor for at least 6 months before that person more than a fraction of a like weighting.
    18 Sep 2011, 10:47 AM Reply Like
  • No system is ever perfect - its one reason why capitalism is often famously "the best available alternative of all flawed systems".


    The Japanese have a fascinating term for the pursuit of a form of aesthetic perfection "shibusa" (also shibui and shibumi), which ironically includes as one of its prerequisites "imperfection" (also simplicity, implicity, modesty, silence, naturalness, and everydayness). Translation into english for this concept is difficult, but the idea is that the pursuit has rules, but also that there is no real expectation of achievement of absolute perfection, just shades of grey (which also refers to how the shibusa palate often works by combining varied colors by "muddying" them with the addition of grey, marrying the whole to create harmony).


    Back to the issue of ratings, there is merit to weighting the worth of the accolade by the reader based upon their overall status in the community. One way to do this would be to mirror the current heirarchical ranking system, ie, to weight all those ranked less than the top 100 with a ratio of "1:1", then some additional weighting for those with more seniority or comments (either or some combination of both would work), summing up with some final weighting for the equivilant of the current Top 100, Top 50, Top 20, and Top 10. This pyramid would make things fairly simple, yet give some progressivity to give new participants a sense of the work yielding some acknowledgement.


    Finally, perhaps the site could include a "Top 10 Comments" of the month section that would highlight those comments each month which attracted special attention. (I suspect that the list would often feature a lot of ties, perhaps even 10 or more with a tie for first, but the point would still be the same).
    19 Sep 2011, 03:00 PM Reply Like
  • "Finally, perhaps the site could include a "Top 10 Comments" of the month section"


    I love this idea. What better way to highlight an interesting or trending topic with a great discussion than to give everyone an easy link to the comments (and the related articles) that were the highest rated? I can see how this would direct a reader like me not only to the comment of a thoughtful user, but also an interesting discussion.


    It would incentivize spending time on content quality while resisting the urge to use alienating ad hominem remarks. It would not ensure that great comments stuck in thinly read articles get read by a wider audience (since they would not tend to get enough eyeballs to get the highest ratings) but it would provide yet another way for readers of all sorts to find interesting content and maybe even a member they would like to follow. It would also give exposure to a new user who was saying good things. Fantastic idea, Triple.
    19 Sep 2011, 03:09 PM Reply Like
  • "This pyramid would make things fairly simple, yet give some progressivity to give new participants a sense of the work yielding some acknowledgement."


    Long ago and far away ...


    I had suggested a scheme with a similar objective. My feeling was that newcomers, especially those with no trading or investing background, might be intimidated and reluctant to author or comment.


    My thought was that more heavily weighting the efforts of newcomers to beneficially participate would encourage them and produce a net gain, long-term, to the quality of SA participation.


    And that should produce bottom-line results over time.


    The overweighting would reduce over time or over quantity or some combination.


    Anyway, I wanted to mention that your suggestion is both altruistic and "self enlightened" IMO and deserves great consideration.


    Disclosure: I may be subject to confirmation bias. ;-))


    25 Sep 2011, 07:55 PM Reply Like
  • Now, if you could remove the political views we could all save hours on the bunches of useless comments...... But, miracles are hard to come bye...................... , Thank you,David ............ gb
    20 Sep 2011, 10:33 AM Reply Like
  • What would be the criteria for an "Authority."? After all some of the best educated people around are still stupid. Does an educated idiot's comment count more than a self taught successful investor's comment? Does a commenter who has instablogs, a long history on SA and a solid record of comments over a wide variety of articles and topics have more credibility than a new user with an MBA?
    20 Sep 2011, 02:02 PM Reply Like
  • I've thought about this quite a lot, there are some obvious flaws in the way commenter ratings are determined.


    1) In baseball, its the ratio of hits to at bats (batting average), not total hits that counts the most. So a commenter who posts 2000 comments and has 2000 likes should rank higher than a commenter who posts 10,000 comments and has 2000 likes.


    2) Getting "likes" or "dislikes" (without the ability to register negative feedback, commentor ranks are pretty weak) from commentors with higher ratings should be worth more. In another sports comparison, this is similar to the "oponents strength of schedule" value used to do computer rankins of football teams.


    3) As someone else has pointed out, there are small groups (the PM bugs are one) who always "like" each other's comments. This artificially inflates rankings, and encourages long streams of similar comments. "Likes" of your comments by someone whose comments you frequently "like" should be devalued.


    I don't know how all of this could be implemented but is worth some thought.


    Perhaps something like this:


    When a person clicks the "like" button, the value of his "commentor strength ranking" multiplied by an inverse comment correlation factor between 0 and 1 is added to your raw ranking. A factor of 1 indicates that you never "liked" any of his comments, 0 would be if you had "liked" all of his comments. So in this manner, a "like" from a strong commentor who you rarely interact with would be worth much more than a "like" from a weak commentor who you frequently interact with.


    To determine a final ranking, your raw ranking score is divided by your total number of comments.


    And I think you should bring back the "thumbs down" and do the same thing with "thumbs down" clicks - they should subtract from your ranking.
    30 Sep 2011, 09:20 AM Reply Like
  • Since nearly all the comments I make are doubtlessly offensive to some one I'm surprised any of them survive the cut. LOL.
    30 Sep 2011, 09:39 AM Reply Like
  • Green, River: Greeting. That is an excellent suggestion regarding weighting comments. Regarding the thumbs down button while I rarely used it myself I thought it a beneficial feature until it's blatant abuse degraded it's usefulness. If a method could be devised to prevent thumb trolling it would be of use in the context and with the same weighting system you described. Thus a thumbs down from a strong commenter who rarely employs the thumbs down would be more heavily weighted than a thumbs down from a shill who employs it frequently and indiscriminately. What are your thoughts David?
    30 Sep 2011, 09:44 AM Reply Like
  • The problem with that is that it encourages something we used to see all the time. An article would be written ie, in 2009, you'd have an article that would say "Dow going to 2000". Then you get people who would rush to be the first 5 posters say "Yep, I agree - great article". Because the article would get a lot of attention and 90% of the people on SA back then were bears, those 5 people would each get 20 thumbs up. Anyone who posted within the first 5 posts who said "That isn't at all realistic" and explained why the Dow wasn't going to 2000 would get 20 thumbs down. This was obviously a problem. Meanwhile, there were many articles that didn't attract the masses of incorrect people at that time that were actually correct in their predictions, but people who commented in those articles would only receive a couple of thumbs or none at all.


    You can probably see that today. Open a popular article and look at the date of the comments. You will see that the earliest comments have the most likes and most of the later comments don't have any (unless they are posted by a couple of the top commentors who automatically get 10 likes every time they post something, ie, they have bots and several aliases that automatically like all their comments). So if you only look at the like ratio to posts, you will encourage more "Great article" posts and fewer real content posts. You will also discourage discussions that can take days to conclude, because they don't tend to get any likes.
    30 Sep 2011, 10:01 AM Reply Like
  • Here you go - this is a perfect example of this kind of behavior. Saying "excellent article" early on got you a lot of thumbs up (which have been since converted to likes).
    30 Sep 2011, 10:06 AM Reply Like
  • Sounds good to me.
    30 Sep 2011, 10:10 AM Reply Like
  • You're right to an extent, but under the system I propose, I think that would work out OK. Those 20 thumbs up / thumbs down will come from the casual traffic, likely people with fairly low "commentor rankings" and won't affect scores too much.


    But I'll add a piece to my algorithm to account for this. Take a look above.
    30 Sep 2011, 10:13 AM Reply Like
  • Thiazole has made a GREAT point regarding the fact that lots of comments are made of the "great post" or "nice article" variety; they get lots of thumbs up but DON'T add anything to the discussion.


    This is a common consideration for rating the strength of academic papers. About 80% of research papers are quite weak, and are never cited in later publications. The remaining 20% range from slightly stronger to earth-shattering, and this has a high correlation to the number of times the paper is cited in later research.


    So here's another facet to add to the algorithm:


    Consider the frequency that a commenters comments are REPLIED to, or how many of a commenters comments are replies. If a "+" from commenter who makes thousands of comments, none of which are ever replied to and none are in reply to another comment, should be less valuable than a "+" from a commentor takes part in lively debate.


    Call it a "reply ratio", factor it in when the commentors strength rating is considered. So, it would then be


    member's Strength Ranking = SUM [ SR * ICF * RR] / NC




    SR = strength ranking of the person giving the "+"
    ICF = inverse correlation factor
    RR = reply ratio
    NC = number of comments by the subject


    I haven't even considered whether or not this is a recursive algorithm; I would value of the "+" when it is given and add it to the raw sum, and ignore changes in the input factors which occur AFTER the "+" is given.
    30 Sep 2011, 10:27 AM Reply Like
  • Perhaps evaluating the content of posts should be weighted as well. Comments of the "Great article." variety won't have the like/dislike numbers counted at all because there is no valid content. Only comments that add value to the overall data/discussion thread by interjecting valid content will get any rating increase/decrease.
    30 Sep 2011, 11:02 AM Reply Like
  • Thiazole, I see it took you all of a few seconds to come up with an example of what you meant by the "great article" comment problem. I think your point is excellent.


    And GreenRiver, I had suggested something very similar to David in a PM a little while ago. In my scenario the more you "like" a given person, the less your + counted.


    But none of these ideas can solve a second problem that Thiazole and others have brought up. That is that you may get 10 likes in a widely read article for a perfectly acceptable but not exceptional comment. And yet you may make the mother of all comments (complete with research, humor, and a prediction that turns out to be true the very next day) and you may receive little or no positive feedback if the comment is late in the thread or is on an article with a narrow readership. So when we are thinking about what "top commenter" really means, most of the abstract things that we come up with -- intelligent, thoughtful, complete, enjoyable to read, containing actionable or relevant information, etc. -- are not really captured in a "most popular in the most heavily trafficked areas" quantification system.


    I can see why the like system is implemented, as it is easy to put into place, but even a casual consideration of the kinds of points mentioned in this thread reveal that it is unlikely to produce a list of the "best commenters". It is a little like running for office. Do you win the election by A) being the most intelligent and good humored person in the race, or by B) appealing to the most heavily populated districts in some shallow way? If you chose A, then your faith in democracy is admirable, but you should avoid a political career at all costs. The answer is of course B. And so it is with "top commenter". I can see no measurement strategy that is currently being discussed that wouldn't simply translate from "best commenter" to "most agreeable in the most populated areas of SA". If identifying and incentivizing real contribution to the site is the goal of a "top commenters" list, all of these strategies fall short.
    30 Sep 2011, 11:04 AM Reply Like
  • Perhaps we are missing the forest from staring too closely at the trees...


    SA should be a case where each component builds upon the basic concept and adds to the whole. Whatever methodology is applied to one component is less important than that the overall site is made better from it.


    Seeking perfection in this sort of system is a dead end, imo. There will always be some negative aspect or minor fault regardless which compromise is used. The more ornate the system, the more prone it will be to overtaxing system resources and becoming an end in and of itself.


    Segregating the various basic contributors (article author - stocktalker - commentor, etc) into walled compounds with seperate measuring sticks which may or may not be supportive to the membership base seems to me to be a risky path. Creating a rigid caste system could result, and that will please only the egotists among the brahmans.


    I would think that a more varied system of "rewards" and rankings would be more entertaining and motivating for the members of SA. Time has a value and quality (LOL, any options trader learns this), so perhaps splitting the rankings between "permanent results" (cumulative since the individual joined SA) and "monthly" (or weekly, yearly, quarterly, what have you) would help stir interest.


    I am less concerned with HOW the points are toted up, and more with whether or not they achieve the goals of the individual members and of SA (our common virtual environment).
    30 Sep 2011, 12:00 PM Reply Like
  • Great post TB!
    30 Sep 2011, 12:49 PM Reply Like
  • YES!


    I think your right that there should be some time-decay built into the ratings your comments receive.
    30 Sep 2011, 01:10 PM Reply Like
  • Please "like" this comment if you want comment ratings to include comments on instablogs as they did before this comment rating freeze.
    30 Sep 2011, 12:51 PM Reply Like
  • While SA is a wonderful form for its users, it is also a business for its creators.


    The various ratings systems encourage people to be more active, more page views, etc. Which equates to more ad revenue.


    So for that reason, some sort of commentor ranking system will improve the business model.
    30 Sep 2011, 01:11 PM Reply Like
  • There was a system. It was flawed, but it worked (and any different system will be comparably flawed, albeit in a more complex way that will probably be more expensive to calculate). As a business, they should be trying to figure out how to monetize the instablogs, not exclude them from rankings. When there are instablog discussion rooms that are beyond their 200th iteration, it is absurd to not think these are an important part of the user community.
    30 Sep 2011, 01:26 PM Reply Like
  • I agree with your point, Jon, but I'm not sure it's strictly true. Perhaps you've heard of the 80-20 rule? It applies in many aspects of business. 20% of your employees do 80% of the productive work. But it also applies in terms of customers. 20% of your customers take 80% of your customer service resources (the squeaky wheels as it were). I'm not so sure that measured in basic metrics like page views, the instablog folks are not just part of the 20% who are most demanding and consume most of the resources of the site. There are a vast numbers of quiet users who generate a great number of page views and never once send David a PM or bring their pet issues to the attention of an editor. In our minds we bring value to the site, giving our first person feedback and through our devotion providing a steady stream of instablog and comment content. But strictly from the numbers of the business, my suspicion is that the vast majority of silent readers provide the real clicks that the site relies on for its monetization. We're the vocal minority they must appease simply as a cost of doing business, I suspect.
    30 Sep 2011, 01:38 PM Reply Like
  • And along the same 80/20 rule, guess where the first early and, maybe, most valuable ideas and improvements originate from?


    I do believe that 80/20 is universal (almost) and that, in "our" case, this 20% who "squeak" also contribute a lot beyond just page hits.


    Of course, that does depend on SA actually responding to some things beyond what it takes to monetize the efforts short-term.


    Like getting new comments accurately flagged (open since January 2011). Adding such as "skip to next new comment" in articles and blogs that go multi-hundred comments, etc.


    30 Sep 2011, 02:38 PM Reply Like
  • DM,


    Perhaps we are indeed in the 20%. The number of followers that some ETF article authors have blows me away.


    However, a secret to building up any successful social networking site is getting people to stay on the site longer and be social. The blog chat rooms - wherein people post links to their favorite articles elsewhere on the site - would, to my way of thinking, be an avenue for building up more business.


    If the issue is that SA administration doesn't like that the same folks are stuck at the top of the top 10 commenters because they have been here longer, reset everyone to zero January 1 each year and see what happens.


    However, I still fail to see how changes in the comment policy will actually benefit the 80%. Folks who click on articles and seldom read the comments, let alone click "like" on someone's comment, seem unlikely to be affected by these changes, or even aware they are occurring.
    30 Sep 2011, 03:14 PM Reply Like
  • "However, I still fail to see how changes in the comment policy will actually benefit the 80%. Folks who click on articles and seldom read the comments, let alone click "like" on someone's comment, seem unlikely to be affected by these changes, or even aware they are occurring."


    Great point. It substantially weakens the thrust of my argument.


    It is unclear to me what SA's goals are for the hard core users much less how it plans to achieve them. I care less about it than I used to.
    30 Sep 2011, 03:38 PM Reply Like
  • Green, River: Greetings. David had a relatively lengthy discussion with the Renegade investors on the Quick Chat Insta Blog (QC) A few weeks ago regarding this. He assured us that this reevaluation of the ratings system and not using "Likes." on the Insta Blog format for ratings is not related to revenue. If I understand correctly it's an issue of fairness. I understand the argument as the Renegade investor group consists of many of the former top 10 commenters who are serious investors sharing investment ideas and information on a group of Insta Blogs anchored on the QC. This resulted in the group unintentionally driving up each others rankings and effectively closing off the top ten from significant changes from outside of the group. Most of us have been on SA almost from it's inception and arrived in the top ranks of the commenters prior to the creation of the QC by Options Girl a couple of years ago. However that does not change the fact that as a group we were dominating the top ranks as described above. We may be getting into territory where the pursuit of perfection is inhibiting the performance of the merely good.
    30 Sep 2011, 01:41 PM Reply Like
  • Caveat emptor.


    Users of this site derive value, or they wouldn't be here.


    The site wouldn't exist without ad revenue; page views, etc. drive that.


    If seeing ScottTrade content in the side bars is what it costs me to have access to the commentary and discussion here, I don't have a problem with that at all.


    If the publishers / owners of SA make a profit then everyone - the writers, bloggers, users, advertisers AND site owners are all happy. This is definitely NOT a zero sum game. We all benefit.
    30 Sep 2011, 02:53 PM Reply Like
  • All of that is true and I don't have any problem with those adds being on the page. Those who created and maintain this space obviously can't do so for free and we all need to eat. Much of the discussion here is valuable to me as an investor and I've been a better investor for it. I've made a fair amount of cash from discussions that the Renegade group has fostered and embellished with extensive research. That alone is what keeps me on this site. After all is said and done making money is the name of the game. If my peers find my comments informative and compelling enough to vote me into the top commenter ranks that is icing on the cake but not of paramount importance. Should this site loose it's productive potential I will find another site that is productive.
    30 Sep 2011, 03:38 PM Reply Like
  • David, buena suete..... I can't comment with all these theories and you know what I emailed still applies.


    Truthfully, it is a hole lot easier to make money than figure your issues out.... lol
    30 Sep 2011, 01:53 PM Reply Like
  • Here's a crazy idea. Don't remove anything from the current system. But make each author add a survey question that you must answer before posting that is related to the article. For example, an author writes an article saying that Apple will crush expectations in the coming earnings report. A relevant survey would be "Will Apple's earning be within 5% of expectations, more than 5% below expectations, or more than 5% above expectations?". You make your pick before you are allowed to comment, then when earnings comes out, everyone who picked correctly get +50 likes and everyone who picked incorrectly gets -10 likes. This will force people to put their money where their mouths are and make them accountable for being wrong (instead of a simple popularity contest). You still get the popularity votes, but you can't be destroyed for being right (ie, the person who said the Dow wasn't going to 2000 back in 2009 who didn't get any popularity votes for going against the grain).
    30 Sep 2011, 04:57 PM Reply Like
  • I really like this idea.
    1 Oct 2011, 12:50 PM Reply Like
  • Thiazole,


    You said "This will force people to put their money where their mouths are...." Interesting observation. Their money.. Money always adds vitality to any discussion that's why SA is the leading site I would guess. We are all interested in making more money.


    SA as I understand it gets its money from site advertisers who in turn get their money from site visitors who buy their products. The attraction to the visitors is the articles and the commenters many of whom could also be the product buyers and not just passers by.


    The article writers can earn 1 cent for each visitor that comes their way and the commenters get a 'like' tick which possibly is not that rewarding simply because the most joy for many of the commenters probably comes from being able to add some thing interesting to the thread regardless if it is approved by others or not. Though having a genuine commendation that is not seeking to solicit a reciprocal appraisal can be a real buzz.


    David Jackson seems keen to reward the commenters so possibly that means they must add a good amount of revenue for SA and so he might like to share those profits and possibly promote more interesting comments but he has a problem fixing who and how to reward with money. This must be awkward because the general wealth of most commenters here appear well above average so 1 cent for each like tick ( he can't pay more than he pays the article writers) probably is going to have less effect than just the tick alone. Therefore I think the best thing to do is get rid of the ticks altogether as they are maybe a lazy way of showing approval which is best done by making a comment which also has the benefit of adding more comments and identifying the agreer. Nice new word, short and easily understood though not as interesting as agreeee. Having the tickers add a comment allows others to see their musings which can also create more comment and I would guess more comment is probably good for David's business.


    There is an alternative method where the agreers themselves pay the commenters and the article writers. If I like what you write why can't I pay you some thing. Give you a real tip. And better still why can't I pay you in real silver or gold. Lets say I am so impressed with Thiazoles comment I want to show my appreciation so I give him 1 gram of silver which is roughly today about 1US dollar. Maybe I am so impressed I give him 1 gram of gold. And everyone can see who gave him the PM. So a list could be compiled of the highest metal earners which could also draw more people to read those same articles and comments. This means we would all be putting our money where our mouths are which is what Thiazoles suggested. 2 grams of silver to Thiazole for that suggestion.


    Now the only problem is how could the mechanics of that delivery system be designed. Well isn't that a coincidence. I just happen to own the world's only self funding micro payment system which uses AUD, gold and silver. Now isn't that just about the most amazing coincidence ever.


    David, you get 5% same as me.
    2 Oct 2011, 10:15 AM Reply Like
Full index of posts »
Latest Followers


More »

Latest Comments

Posts by Themes
Instablogs are Seeking Alpha's free blogging platform customized for finance, with instant set up and exposure to millions of readers interested in the financial markets. Publish your own instablog in minutes.